I probably just haven't watched enough trashy isekai anime to appreciate Subaru actually having to put in some effort (sometimes), the closest I've come to watching anything with a parallel world (Re: zero aside) was NGNL, and I dropped that after 1 episode. So for me one or two episodes of him being retarded and failing would have sufficed, anything more that and he's just being needlessly cringey and blockheaded.
Oh, I suppose I've also watched a bit of Zero no Tsukaima, but that was for the "plot", not the plot.