That's a pretty good point actually. I took a course on modern Japanese literature at my college and although it was taught in English, I learned a lot about the history of Japan beginning from the Meiji restoration to the current day. I think that for history though, there aren't really textbooks per se. Instead, you might get interested in say, the Sengoku period from playing Sengoku Rance and decide you want to learn more. Then I think reading the Wikipedia article usually gives a decent overview, and then you could try reading some of the references listed for more in-depth information.
Culture's definitely important, but I'm not so sure you can "learn" a culture. I feel that you gradually absorb Japanese culture over time, but even then, you have your own perspective that will be different from everyone else. Even among Japan, I'm sure people from Osaka and people from Tokyo will have had different experiences that lead them to emphasize different parts of Japanese culture.
Oh, and I do have a recommendation for a book, although it's not strictly about culture or history: http://www.amazon.com/Oxford-Japanese-Short-Stories-Books/dp/0199583196. This is a collection of Japanese short stories translated into English and arranged into chronological order.