A few signs, including low sex drive, erectile dysfunction, testicle pain, or swelling, may signal low sperm count.