Yes, infertility can occur even if your sperm count is "normal." Low sperm motility, abnormal sperm, anti-sperm antibodies, poor sperm viability and obstructions in the sperm ducts can all contribute to male factor infertility.