Not wearing a bra can cause major muscular discomfort in areas like the back, neck and shoulders, esp. if you have bigger breasts. Breasts do tend to sag due to a lot of reasons like age, weight loss or gain and health issues. Not wearing a bra can further worsen the condition.