Factors such as age, overall health, diet, and lifestyle choices (like smoking or drug use) can significantly affect sperm count and quality.