Always refrigerate any cake with a frosting that contains eggs or egg whites, or one that has whipped-cream frosting or any type of filling — be it whipped cream, custard, fruit or mousse. You won't hurt a cake by refrigerating it, but the cold does dry it out.