I had parents who loved me, however my father suffered mental illness. I was never sexually molested as a child, but on the other hand I was exposed to pornography at an early age. I never suffered physical abandonment, but I was a terrible athlete. Your experiences may have been better or worse than mine in each of these things. Only God knows the heart.
Patience implies hope.
To be patient is to be patient for something. It is not abandonment to sin or immaturity. We are not being patient when we give up. We are patient when we acknowledge that one day we will be pure and righteous, one day we will be fully comfortable with our manhood or womanhood. Patience properly manifested will be God-centered.