It was designed early in the history of information retrieval when the alternative was a laborious search of multiple print indexes in a well-stocked library. Not too many years ago, the budget and staff of NLM were threatened by a round of cuts. Indexing was slowed down to a near standstill, and updating was delayed radically. I don't remember why the database survived, but it's obviously recovered fully.
Commercial databases survive on their own profits.
Government databases survive on the good will of the government. It's not always true that you get what you pay for, but paying is the best way to ensure a continuing supply. As a US taxpayer, I'm not sure that I agree with that policy either. One intent of NLM in providing this information for free is to aid the medical consumer.