If I used dirt in my walls instead of insulation, would I have a cheaper energy bill?

That would be the concept behind rammed earth homes, cobb homes and earth sheltered homes, it's called thermal mass. If the walls were thick enough, you would get the summer heat in winter and the winter cool in summer but standard wood frame walls are not very thick. The best you could hope for would be to get the days heat at night and the night's cool during the day. Many hot climates have concrete homes for this reason. However, you would probably find your wood framed walls collapsing from the weight and the wood rotting in the soil.

The ants and other bugs would move in

Not enough in walls. Underground temps are constant because of the volume of ground, not the material.

No. Insulation works due the micro-cellular structure of air pockets. Dirt would be worthless in comparison.

Would it hold the heating and cooling in better? Why or why not?

My dad works in Sheetrock has his whole life, i highly recommend not using dirt, unless you want hot summers and cold winters, found this link that says it can lead it unconformable inside temperatures

http://earthbagbuilding.com/faqs/insulation.htm