It is unfortunate that more men who suffer from enlarged breasts don’t have surgery to correct the problem. After the first week or two at most, patients can return to light exercise, such as walking. Strenuous exercise needs to wait for a month or so. But you’ll quickly be able to return to work and regular activities.