I think the real reason is mostly obvious to anyone who is looking: Its the rise of the bureaucracy. Its the same thing that's strangling education, and basically all other public resources.

The automation, and now AI, we've adopted over the years by-and-large does not serve to increase the productivity or efficiency of care-givers. Care-givers are not seeing more patients-per-hour today than they were 40 years ago (though, they might be working more hours). It might, rarely, increase the quality of care (e.g. ensuring they adhere to best-practices, centrally documenting patient information for better continuity of care, AI-based radiological reading); but while I've listed a few examples there, it is not a common situation where a patient is better off having a computer in the loop; and this speaks nothing to the cost of implementing these technologies.

Automation and now AI almost exclusively exists to increase the productivity and efficiency of the bureaucracy that sits on top of care-givers. If you have someone making calls to schedule patients, the rate at which you can schedule patients is limited by that one person; but with a digitized scheduling system, you can schedule an infinite bandwidth of patients (to your very limited and resource-constrained staff of caregivers). Forcing a caregiver to follow some checklist of best practices might help the 0.N% of patients where a step is missed; but it will definitely help 100% of the bureaucracy meet some kind of compliance framework mandated by the government or malpractice insurance company. Having this checklist will also definitely hurt the 1-0.N% of patients which would have been fine without it, because adopting the checklist is non-free, and adhering to it questions the caregiver's professionalism and agency in providing care. These are two small examples among millions.

When we talk about increasing the efficiency of the bureaucracy, what we're really stating is: Automation is a tool that enables the bureaucracy to exist in the first place. Multi-state billion dollar interconnected centrally owned healthcare provider networkers simply did not exist 70 years ago; today its how most Americans receive what care they do. The argument follows: This is the free market at work, automation has enabled organizations like these to become more efficient than the alternative; but:

1. Healthcare is among the furthest things from a laissez-faire free market in the United States; the extreme regulation (from both the government and from health insurance providers, which lest you forget was mandated by law that all americans carry, by democracts, with the passage of the ACA, and despite that being rolled back is still a requirement in some states). Bureaucracy is not the free-market end-state of a system which is trying to optimize itself for higher efficiency (lower costs + better outcomes); it was induced upon our system by corporations and a corporate-captured government seeking their share of the pie; it was forced upon independent medical providers who saw their administrative costs soar.

2. Competition itself is an economic mechanism which simply does not function as well in the medical sector than in other sectors, for so many reasons but the most obvious one: If you're dying, you aren't going to reject care. You oftentimes cannot judge the quality of the care you're receiving until you're a statistic. And, medical care is, even in a highly efficient system, going to be expensive and difficult to scale resources to provide, so provider selection isn't great. Thus, the market can't select-out overly-bureaucratic organizations; they've become "too big to fail", and the quality of the care they provide actually isn't material.

And, like, to be clear: I'm not discounting what you're saying. There are dozens of factors at play. Let's be real, the bureaucracy has enabled us to treat a wider range of illnesses, because the wide net it casts can better-support niche care offices. But, characterizing this as generally non-nefarious is also dangerous! One trend we've seen in these gigacorporation medical care providers is a bias of resources toward "expensive care" and away from general practice / family care. The reason is obvious: One patient with a rare disease that costs $100,000 to care for represents a more profitable allocation of resources than a thousand patients getting annual checkups. Fewer patients get their annual checkups -> Cancers get missed early -> They become $100,000 patients too. The medical companies love this! But: Zero people ANYWHERE in this system want this. Insurance doesn't want this. Government doesn't want this. Doctors don't want it. Administration doesn't want it. No one wants the system to work like this. The companies love it; the system loves it; the people don't. Its Moloch; the system craves this state, even if no one in it actually wants it.

Here's the point of all this: I think you can have a medical system that is centrally ran. You can let the bureaucracy go crazy, and I think you'll actually get really good outcomes in a system like this because you can appoint authoritarians to the top of the bureaucracy to slay moloch when he rears his ugly head. I think you can also go in the opposite direction, kill regulation, kill the insurance-state, just a few light touch sensible legislations mostly positioned toward ensuring care providers are educated appropriately and are accountable, and you'll get a great system too. Not as good as the other state, but better than the one we have right now, which is effectively the result of ping-ponging back and forth between two political ruling classes who each believe their side of the coin is the only side of the coin, so they'd rather keep flipping it than just let it lay.