Some foods you can't help, they change a bit. But about others, you have to put in water or wait until the ice melts, and then heat it, if you quickly heat it, it will taste bad.
I agree with others in that what you're referring to ("tasting like a freezer") is usually what's called *freezer burn.*
Freezer burn causes an off flavor and odor to develop in frozen foods (especially meats) that have been exposed to air while frozen. Areas of freezer burn have actually dried out and become harder, and will look kind of whitish. Hard to get rid of that flavor once it's there btw.
(Freezer burn won't happen in an hour or few days though; it takes longer, but will also develop faster the more air that's allowed to get to the food.)
The solution is to wrap foods as air-tight as possible and also to burp out as much air before freezing as possible (or just don't leave those foods in the freezer very long).
The very best air-tight wrapping comes from using a vacuum sealer (like a Foodsaver, for home use) along with special vacuum-sealing bags which are extremely non-porous. The Foodsaver machine (and other brands) will both suck/vacuum all the air out of the bag of food and seal the bag *completely.* For example, meats that have been correctly vacuum sealed will taste and look perfectly fresh after 2 or more years in the freezer (and of course, no freezer burn at all).
As for other ways to remove and keep out as much air as possible, the idea is to use something like plastic bags (the thicker the plastic the better; even Ziplock bags, etc, are *somewhat* porous over time though), or wrap with foil (though that can be hard to remove from some foods later), or wrap with butcher paper, or put into non-porous containers made from glass or thick plastic which have fairly air-tight lids, etc.
Often more than one of those is used to lengthen the time before freezer burn might happen (even double-wrapping in ziptop plastic bags).
Wrap it well. If your food is just tossed in there, with the package half open, that frosty air gets in there and dries out the food. Ever see dried leathery spots on the meat? Thats freezer burn from air exposure. Basically if you wrap your food well, you won't get freezer taste or freezer burn
Seal everything well before you put it in the freezer.
Now it depends on what your freezer tastes like. If it is a foul taste you are getting I suggest that you clean out the freezer and toss items that may no longer be at peak freshness. Items that are freezer burned, items that you look at them and wonder what they are, items that you got when either of the Bush's were in office probably should be tossed out. Once you have found the things that smell "off" arrange the freezer in a way that you can find things easily.
Use older products first.
Use one product before buying another just because it is on sale and "such an awesome deal"
(I really should follow my own suggestions!)
Anything that is in the freezer that is going back in should be well wrapped.
Do not free anything in just the supermarket wrapping, always re-wrap to get out as much air as possible. If it is meat you can wrap the label so you know the weight and cut.
Last...put a label and date on everything.