The bottom layer of this banana pudding trifle is going to get soft, no matter what. To add some crunch, I recommend adding fresh wafers or fresh crumbled wafer to the top of the dessert just before serving.