When you lose weight, you lose it from everywhere, including the breasts. The fat cells begin to shrink, but since there is not a lot of muscle tissue there, the stretched supportive ligaments remain loose. This results in sagginess.