Baking soda helps the batter rise by producing carbon dioxide gas once exposed to heat. It also improves the overall taste. Overripe bananas are naturally acidic. Baking soda helps neutralize this acidity, which can enhance the flavor profile of the bread and balance the sweetness of the bananas.