It depends on when you want to go, how long you are planning on staying, what your interests are etc. When I went to Japan it was great to just visit places like Tokyo, Kyoto, Hiroshima etc as they are quite different to the cities that I'm used to back home. Remember to by a JR railpass if you are going to travel around a lot and note that you have to buy this before you go to Japan. Here is a trip report with some photos that I wrote after my trip there
http://gardkarlsen.com/japan_tokyo.htm . Maybe that can give you an idea of what to expect.