There are many factors that may contribute to breast asymmetry, but the exact cause remains unknown. Hormonal changes, injuries on the chest area, or breastfeeding are believed to be contributors to asymmetrical breasts. There is also a condition called juvenile hypertrophy that causes one of the breasts to become severely out of proportion to the smaller breast.