The hair growing on the nipples could be because of hormonal imbalance. Such imbalance can be a cause of infertility.
You should undergo investigations - hormonal, and ultrasound to see the ovaries - to find the cause of your infertility.