Not having it on the screen all the time but having it available immediately is pretty important, a backlock/rewind, a number of save slots greater than or equal to 2x+10, where x is the number of choices in the VN, a skip unread only/skip all toggle (especially if there are multiple routes) and a skip button on the interface so you don't have to sit around like a jackass with your finger on ctrl, a quicksave is pretty nice but I personally rarely use those, toggling individual voices, separate audio sliders for BGM/voice/effects, text speed options that include making the text appear instantly. Oh, also hotkeys, such as making space, enter, and down arrow move to the next line, and the up arrow bringing up the aforementioned backlog.
Some of these belong in the options menu but that's UI too, technically.