Hormonal Changes: Women develop their breasts during puberty as estrogen increases. Pregnancy and milk production can also cause the breasts to enlarge due to hormonal changes. Drug Use: Certain drugs can lead to enlarged breast tissue in both men and women.