Latex foam is widely considered the best material to use for pillows.
Natural latex offers support without heating you up like memory foam will.
Most of the time, people who use memory foam pillows don't even realize they're getting hot at night.
Even still, this can lead to restless sleep. Sleeping cooler allows you to sleep deeper.
Latex foam is usually your best option, but if you have a latex allergy, you will want to use memory foam.