I can agree with some cases, sure - but tone of voice can convey the meaning of something like "You...." Unless a reader has voices turned off, inflection is a universal thing - anyone can tell when someone is angry, sad, happy, etc - pair that with the actual wording, and you got the intended meaning across, without adding words that could change the context and personality.
Though of course that can only really be applied to things with voices. non-voiced is a different matter - though even then things like "You..." wouldn't be that difficult to understand, if the reader understands the scenes intended mood. Besides, "rude verb form" is easy with things like "tch" - or if a writer is willing to put in a bit of extra work, elaborate with a description "so-and-so clicks there tongue in irritation" or "so-and-so growls in response"
As for swears in Maitetsu specifically, they are so infrequent it's not really an issue - beyond not fitting the initial meaning. A good example is Hibiki leaving in irritation from the room, and the MC says "Shit, looks like I screwed up this time" - but really, it would have fit his tone of voice (and words) a lot better with something like "Oops, guess she's mad huh?" The MC didn't even use an explicitive verb or sound in that sentence, so having a swear thrown in changes the tone of the sentence, and alters the readers perception of his personality to a tiny extent.
But like I said, it's so rare in this VN, I feel somewhat silly going into it, heh.