Again I am forced to find myself agreeing with Ari, here. About the "ugliest" heroine I can even think of is Hanako in Katawa Shoujo, and she's not really ugly, just fire-scarred, making her tragic and even more attractive.
So yeah. Personality pretty much dictates it.