There are other factors that can turn your breasts sag, aging is one such factor. When you breastfeed, the flow of milk can stretch your breast skin and tissues. Many moms worry at this stage which is not abnormal to experience. But other factors can change your breast appearance more than breastfeeding.