Chocolate Oreo cupcakes should be fine stored at room temperature for several hours when serving, but refrigerate the leftovers. There is cream in the frosting, and it spoils easily. Keep your Oreo cookie cupcakes covered to prevent them from drying out, and enjoy them within 2 to 3 days.