If your cupcakes are filled or decorated with anything containing dairy such as a custard filling or a cream cheese frosting, you should store them in the fridge for food safety. But if they are unfrosted, you may keep them wrapped at room temperature.