The materials you wear on your upper body can make the difference between cold comfort. Avoid wearing cotton as an inside layer next to your skin. It soaks up perspiration and holds onto it, so you'll freeze on long downhills or in a stiff wind. You're better off with a fabric that wicks the moisture away from your body and dries as you move through the air. Wool is a good one, although it tends to be itchy.
Synthetic materials designed for winter outdoor sports are even better.
They're available in many catalogs or at bike and outdoors shops. The goal is to keep the wind off your fingers, ears and toes. Cover your ears with a winter sports band. They're available in any outdoors store, and fit well under most helmets. Choose layered gloves.