Do you let muffins cool in the pan?
I like to let the muffins cool in the pan on a cooling rack for about 5 minutes before removing the muffins and letting them continue to cool completely on the cooling rack.