Stress can cause male breast tissue to enlarge periodically. The adrenal gland may increase secretion of estrogen precursors in response to stress.