> it's hard to ignore that the technological world primarily operates on languages such as C, Java, JavaScript, Python, C#, and a handful of others.

I think you are focusing on the trees and so not seeing the size and the shape of the forest.

Most organisations use C and languages implemented in C, on OSes implemented in C, because they do the job, the people are cheap and readily available, and the dominant OS is free and costs nothing to deploy.

Which can be reduced to:

Most people use the tools most people use.

That's not a very useful observation, but it poses an interesting question:

Why?

That's easier.

Here is the shape of the outside of the answer:

They use them not because they are good -- they aren't very good, measured objectively -- but because they are ubiquitous and cheap.

Other tools are better, and just as free, but then the people cost more, and the associated tooling costs more. (Frameworks, supporting libraries, deployment costs, whatever. E.g. it's very cheap to deploy Javascript because all you need is a reasonably modern browser, and those are free and almost all OSes have them.)

Those are the externalities, in a manner of speaking.

But the other side of the answer is the inside: the area, not the shape.

The mainstream, conventional, software industry is huge, and hugely lucrative.

Writing just-barely-good-enough apps, minimum viable products, gets you out there and making money. Then you can put a share of your revenues into incrementally improving it.

Every now and then you can push out a big new version, with disruptive changes. You can charge for getting the new major releases, but more to the point, once an old version is officially obsolete, you can charge for continued fixes to the now-obsolete versions.

It makes money. It's an ecosystem, a food chain or more accurately a web. Some members are predators, some are prey, but they all work together and if you just eliminate either predators or prey, the system collapses.

In other words:

Most people use the tools most people use, because most people use them, because you can make big money from volume of cheap junk.

But there is another model of making and selling stuff: make small volumes of really good products, using highly skilled workers, and sell those high-quality products in very small volumes but for very high prices, to discerning customers who know they're buying something built to last and who may not come back to you for new versions every couple of years, but that's fine if you made a couple of decades' revenue from them on the original sale.

Because cars are a long standing metaphor in computers and computing:

If you have a mass market for cars, then you get cheap cars, and everyone's cars are much the same because they are built down to a price and mass produced.

These car consumers can be upsold some extra buttons and minor features, and a lot of those combined may double the price of the car.

(This is how different Linux distros survive. Some have more buttons. Some have A/C. Some are fully automatic, others have fully manual controls. Some have power steering, some don't.)

But such a market also supports hand-made sports cars (and, inevitably, superficially similar cheaper sports cars from the mass-producers). It also supports vast tanklike cars that weigh as much as 10 normal cars, but produce 10x the engine power of those cars so they still perform.

Very different products, but they cost an order of magnitude more than the mass-produced cars... and do the same job. Some owners of mass-produced cars aspire to own fancy sports cars, and some aspire to own luxury behemoths. Most never will.

People who don't care much for cars and just see them as a tool for getting from A to B do not see why anyone would pay for fancier cars. That's OK, too.

Some people live in other countries and see more clearly, because for them trains and bicycles are a perfectly viable way of getting from A to B, and are both cleaner, healthier, use less resources and create less waste.

Tools like Lisp are the artisanal hand-made cars compared to the mass market. People who've never used anything but cheap mass-produced tin boxes can't even imagine that there are things that are so much better, let alone that in the long run, you might be better off using them.

As Terry Pratchett put it:

«
“The reason that the rich were so rich, Vimes reasoned, was because they managed to spend less money.

Take boots, for example. He earned $38 a month plus allowances. A really good pair of leather boots cost $50. But an affordable pair of boots, which were sort of OK for a season or two and then leaked like hell when the cardboard gave out, cost about $10. Those were the kind of boots Vimes always bought, and wore until the soles were so thin that he could tell where he was in Ankh-Morpork on a foggy night by the feel of the cobbles.

But the thing was that good boots lasted for years and years. A man who could afford $50 had a pair of boots that'd still be keeping his feet dry in ten years' time, while the poor man who could only afford cheap boots would have spent a $199 on boots in the same time and would still have wet feet.

This was the Captain Samuel Vimes 'Boots' theory of socioeconomic unfairness.”
»

Some of us live in countries with really good public transport, and know that it's possible to replace the entire category of personal automobiles with something better for everyone...

But try telling that to an American. They won't even try to understand; they will instead earnestly explain why they need cars, and their country is better because everyone has cars.

More wild generalisation:

In the late 20th century, there was another model of software construction, an alternative to the "Worse is better" model. The WIB model is that one type of software fits all, and build one (or a very few) minimum viable operating systems, from minimal viable programming languages, and make them cheap or give them away for free.

The artisanal software model was more common in Europe and Japan: pick the best language for the job, and build tiny bespoke OSes for each category of device. Have multiple whole incompatible families of desktop OSes, and families of totally different unrelated server OSes, and families of different OSes for handhelds and games consoles and school computers for teaching kids, and so on.

Unify them, minimally, with some standard formats: network protocols, disk and file formats, maybe some quite similar programming languages in lots of weird little nonstandard dialects.

iTron, RISC OS, SiBO/EPOC/EPOC32/Symbian, QDOS/Minerva/SMSQe, Con-Tiki, Symbos, GEOS, Novell Netware.

Keep the complexity are the market level, in which multiple radically different products compete for sales in their market segments. Lots of overlap, lots of duplication... but also competition, evolution, rivalry, advancement.

The software remains small and relatively simple. This aids development, but mainly, it keeps the resource requirements low, so the devices are cheaper. Human brainpower is cheap: spend it on clever software to enable cheap hardware.

The approach I am calling WIB is of vast general-purpose OSes which can do anything, so you only need a handful of them... but you need massive hardware to run it, so devices powered by WIB software are expensive, and very complicated, and the software is vastly complicated, so you need armies of programmers to maintain it, meaning frequent updates, so you need lots of storage and fast connections.

And there is no overview, because it's much too big to fit into a single human head, so improvement is incremental, not radical.

The end result is a handful of vast monoliths, full of holes and leaks, but a vast economic machine that generates continuous income and lots of jobs.

When you live in one of these monoliths, the fact that there are happy accomplished people working in weird tools making weird little products for tiny markets seems incomprehensible. Why would they?

Most people use the standard tools, meaning the ones most people use. So obviously they are good enough: look at these trillion-dollar corporations that use them!

So obviously, there isn't really any point to the weirdoes.