It will require time and some give and take on all sides to develop a working relationship between the adult skating community and the community of clubs, judges and officials. Physical talent may be required to be a high level competitor, but anyone with a strong desire to improve can learn at least some of the jumps and master all of the basic skills. The final problem is not having the time to skate. For most people, this means a minimum of twice a week, though three times with lessons wouldn't be bad. Once a basic skill is mastered, then it can be retained even if you only skate occasionally.
Progress always comes in fits and starts.
You'll generally know when you've picked up a new trick or mastered a skill, but it's difficult to assess your overall progress. If you're still skating, enjoying yourself and being challenged, count that as progress. But the sight of the skaters on TV gets something going in the pit of your stomach so you get some skates and take a few lessons. You find yourself practicing spread-eagles while waiting for the bus. You start planning your work and family life around your skating sessions.