Actually, the coldest I've ever been (comfort-wise) was in the Florida keys and the hottest I've ever been was in Minnesota in early April.
The cold in GA, AL, AR, and southern MO is just incredibly bone chilling because of the humidity. It doesn't get the notoriety it deserves. Here it's a dry cold and it's really not too bad unless the wind is blowing. We're also set up for it. I'd be inside too if it's the kind of cold I remember from a winter trip to AR. There's more to "cold" than latitude. Just so we don't feel like total morons for living here, we Minnesotans make a macho thing out of the cold.
But generally, Minnesotans don't even have a faint understanding of how to deal with heat properly. They don't understand how fans work or what windows really do. It's quite amusing, but it can be scary. A few years ago a lifelong Minnesotan (and highly educated) friend of ours collapsed with heat exhaustion. She didn't know how to handle a 100 degree day.