Because air mattresses are filled with air, you’ll often find that the mattress will slowly get cooler during the night as the temperature drops, unlike a regular mattress. But you can use a thermal mattress topper made for an air mattress so that you don’t feel the cold coming from your air mattress.