The vast majority of students are forced to pay the fee, whether they attend athletic events or not. That includes both graduates and undergraduates, except for doctoral candidates and students with very few credit hours. The fee supports intercollegiate athletics exclusively. The Recreation Center is a separate facility with a separate fee. It is a great facility.
Every student can use it, and many do.
There is not enough space for everyone. For UCSU fees, students can appeal to the appellate court to have their fees waived if they do not use the services. The largest complaint about the athletic fee is that it offers no choice. The University of Texas has a very challenging budget. They meet once every two years.