No, James didn't say it was always pleasant and he didn't say it comes natural. Christian growth isn't as automatic or natural as physical growth. Physical growth usually comes very natural as the body matures from an infant to adulthood. However, mental maturity is not a natural process. Mental development is full of frustrations and failures, which will result in maturity or immaturity.
Maturity is a personal choice.
If we choose to mature our difficulties will become stepping-stones to happiness. If we make a choice to remain immature, our difficulties will bring destruction and we will not discover happiness. James is urging us not to give up in our struggle for maturity. Christian maturity is something we must work at constantly. There is a difference between age and maturity.