Michael, I once read that the population of girls who play softball or basketball has exploded in the United States. I think maybe it's because the media is emphasizing professional athletes male and female a lot recently.
Taste here means experience.
So, this expression actually means someone experience something for the first time. I got my first taste of theater in kindergarten when I played a court jester. If you are just playing catch, that would not be an organized sport. Where I grew up, we had little league, which were sort of an organized baseball league for children. I think organized sport for children should probably emphasize teamwork and fun rather than winning.