Women are not always the source of a couple's infertility—in approximately one-third of cases, men may have difficulty with conception. In another third of cases, men are a contributing factor, along with female infertility and other problems. Men should be evaluated by a physician if there is a family history of infertility, if they have undergone cancer treatment, if they have small testicles, a swollen scrotum, or a low sperm count, or if they have any other testicular, prostate, or sexual problems.