Sometimes, your baby's overdressed, over-excited, or your hands are too cold when you touch them. Teething can also cause babies to feel warm without a fever. You can always check with a thermometer if you want to make sure. With time, you will know whether their hot forehead means they have a fever.