Dress in thin layers and protect yourself from the wind. If you ride slow, or conditions will make you ride slow, wear more layers and peel off as needed. The point is to avoid sweating. Protect yourself from cold wind and rain with a breathable windbreaker. Your legs require fewer layers, because they are doing all the work, but keep your knees protected.
Hands are easier to layer than feet.
A thin glove, followed with a fleece glove, covered by a waterproof lobster mitten will keep the coolest hand toasty for the longest time. Mittens help retain heat but make shifting gears and braking difficult. For feet, caged pedals allow a greater variety of footwear but clip-in pedal systems have fewer pressure points at the toes. Cyclists with caged pedals can wear anything from hiking boots to rubber boots. Wear eye protection.