Clothes that wick moisture away from the body will keep you dryer and therefore warmer. I try to stay away from cotton. When cotton gets wet, it stays wet. I dress in layers, so I can regulate temperature throughout the ride. As it warms up, I can remove a layer, and when it starts to cool down again when the sun goes down, I can start adding clothes again.
Zippers are another essential for temperature regulation.
Turtlenecks with zip fronts and jackets with pit zips can allow a rider to be comfortable through a varied range of temperatures. Silk is a natural fiber with great insulating properties. It makes a great base layer. I often use silk for glove and sock liners. There are lots of synthetics available that wick moisture away from the skin and dry quickly.