There is no precise answer to this question, as it depends on both the temperature and your personal preference. If it is very cold outside, you may want to keep your beanie on at all times to maximize warmth. However, if you start to feel too hot or uncomfortable, lowering your beanie (or taking it off entirely) can help regulate your body temperature. Additionally, some people find that wearing a beanie can give them headaches after awhile, so lowering it or taking it off when this happens can be helpful. Ultimately, it is up to you to decide when to lower or remove your beanie based on how you are feeling.