Over mixing the batter can leave the bread dry and crumbly. When adding the dry ingredients to the wet ingredients, stir until just combined. Also, overbaking can dry out the bread and leave it crumbly. All ovens are different, so check the bread at the 55-minute mark to avoid overbaking.