No, you do not have to be a frequent flyer to have an airline credit card. In fact, even if you only fly with an airline a few times a year you might still benefit from holding an airline credit card thanks to the perks they offer.
That said, if your airline credit card has an annual fee, you’ll want to make sure you are flying often enough to receive more value from the card than the fee you pay.