You do not need to refrigerate banana bread. It actually may dry it out if you store it in the refrigerator. The best way to store banana bread is to place it on a plate, cover it with plastic wrap and keep it on the counter for 3-4 days.