It is generally recommended to avoid strenuous activities, such as heavy lifting or exercises like push-ups, if you have varicocele. These activities can increase pressure in the scrotum and exacerbate symptoms.