This happens when the batter gets overmixed. For this reason, I recommend folding the wet batter into the dry ingredients until just combined. This helps keep the loaf light and airy.