You can keep this fresh peach cake at room temperature the day it is made and the following day. But beyond that, leftover cake should be refrigerated as moist cake with fruit can go bad, especially in warmer weather.