There's no big difference between these two. One is describing the day, and another is to using adjective to describe the feeling. Basically, both are used commonly. :)
First of all that's not how you ask the question. You should say What is the difference between "It's a cloudy day" and "It's cloudy".
Actually they both mean pretty much the same thing, both two phrases describe the weather of the day.
2011-11-02 06:34:44 補充:
However, as "it's a cloudy day" specifically describe the weather of the day, "it's cloudy" can describe what the weather's like for just now (as for this moment only).