I prefer heat myself. People say "oh you can always put MORE clothes on." No, no you can't. There is a limit to the amount of clothes a person can put on their body.
I think when people say "you can always put on more clothes", they're referring to the act of replacing your shorts with longer pants and such. Anyone who's spent a decent amount of time in cold climate knows it's all about layering. You don't need more than three layers, but the combination makes all the difference:
1st layer - Thermal underwear (breathable) (optional)
2nd layer - Fleece jacket
3rd layer - Windbreaker
That should keep you warm down to crazy levels of cold while also preventing you from overheating if the weather clears up. I've been using that combination in -30ºC and +10ºC weather, and my body temperature was perfect in both cases.
Preaching to the choir here, probably, but the way this combination works is the same way penguins prevent hypothermia while swimming around in ice cold water. The fleece jacket itself is not really warm, but it excels at trapping air around it because of the fabric. The windbreaker acts as a sealant that prevents the cold wind from swiping away this layer of air around the fleece jacket. The thermal underwear (only really needed if you reach arctic temperatures, or if you're not moving much) creates a second layer of air closer to your skin. These layers of air are kept warm by your body heat and act as insulation.
If you're cold in the winter, you're doing it wrong.
If you're hot in the summer, you're sh*t out of luck.
Could take a cold shower, but the effect is temporary (and often counterproductive).