Yes, stress can affect your reproductive health. But it cannot be the only reason to cause fertility issues in women. Infertility is not a psychological problem. There are many health issues and reproductive disorders that can affect fertility.