Yes, you need to refrigerate a dump cake after it has been baked. It can be at room temperature for up to two hours before it should be stored in the fridge or freezer.