It is completely fine to serve your baby's formula cold that is cold or at room temperature, although not all babies are going to want to consume it this way. Getting your little one used to drinking cold formula may take a few tries, but with time and consistency, most babies will adjust to formula feeding this way.