When you let the Balloon warm up again, it starts inflating itself because of warmer air molecules. The warmer air molecules rise and enter into the Balloon, making it expand. Hot air molecules are less dense in weight and tend to travel upwards. And they require more space since they like to scatter in larger areas.