I'd say wear all the layers you want, but only if you're willing the haul them along with you once they come off.
I tend to get reaaaally warm when I run, so even if it's below freezing, I'll still run in shorts and - at most - a long-sleeved shirt. Granted, it takes a mile or two to get warmed-up, but it's worth being free of that sticky, heavy feeling of sweating through too much clothing.
As for headgear, I'll usually don a hat with earflaps. Apparently a good deal of heat from your body escapes through your head, so it's important to keep that part covered-up. But my ears stick out so much that schmooshing them for too long makes them ache, so I roll up the ear flaps (usually mid-way through a run) and let my satellite dishes enjoy the chilly breeze