Fresh bananas have the tendency to oxidize and turn brown. The layer of fresh banana inside the cake will be covered fully with whipped cream, preventing them from oxidizing. If you plan to top the banana pudding cake off with more fresh bananas, only do so up to 30 minutes before serving!