In general, yes, it’s safe to sleep with headphones in. Just as you might sleep with earplugs in a hotel room, sleeping with headphones in is not that different.
Of course, the difference is that with headphones you might be listening to music or sounds through the night. The main thing to be concerned with here is that you don’t have the sound high enough to cause damage to your ear drums.
Aside from noise-related issues, there is the possibility that wearing headphones deep in your ears can cause irritation and even promote the buildup of wax. To ensure this doesn’t happen, don’t push them in all of the way.
Of course, this only applies to in-ear headphones. If you are going to use over the ear headphones, the main issue will be staying in the same position so as not to press them against your ears in the middle of the night.
The other main type of sleep headphones, the bands, should be a much safer option. This band at worst will cause you to sweat a little during the night, but shouldn’t pose any real danger to your ears.