An observation I've made over decades is that people stop optimising when things get "good enough". That threshold is typically 200ms-2s, depending on the context. After that, developers or infrastructure people just stop bothering to fix issues, even if things are 1,000x slower than they "should be" or "could be".

Call this the performance perceptibility threshold, or PPT, for want of a better term.

There's a bunch of related problems and effects, but they all seem to come back to PPT one way or another.

For example, languages like PHP, Ruby, and Python are all notoriously "slow", many times slower than the equivalent program written in C#, Java, or whatever. When they were first used to write websites with minimal logic, basically 90% HTML template with a few parameters pulled from a database, this was okay, because the click-to-render time was dominated by slow internet and slow databases of the era. There was, a decade ago, an acceptable trade-off between developer-friendliness and performance. But inevitably, feature-creep set it, and now enormous websites are entirely written in PHP, with 99% of the content dynamically generated. With rising internet speeds and dramatic performance improvements in databases, PHP "suddenly" became a huge performance pain point.

In that scenario, the root cause of the issue is that the attitude that "PHP/Python/Ruby" is acceptable because lightweight code using them falls under the PPT is a false economy. Eventually people will want a lot more out of them, they'll want heavyweight applications, and then having locked into the language is now a mistake that cannot be unwound.

The most absurd example of this is probably Python -- designed for quick and dirty lightweight scripting -- used for big data and machine learning, some of the most performance intensive work currently done on computers.

Similarly, I see astonishingly wasteful network architectures, especially in the cloud. Wind the clock back just 10 years, and network latencies were vastly lower than mechanical drive random seek times. Practically "any" topology would work. Everything split into subnets. Routers everywhere. Firewalls between everything. Load balancers on top of load balancers. Applications broken up into tier after tier. The proxy talking to the app layer talking to a broker talking to a service talking to a database talking to remote storage. Nobody cared, because sum fell under the PPT. I've seen apps with 1/2 second response times to a trivial query, but that's still "acceptable". Multiply that by the 5 or so roundtrips for TCP+TLS for every layer, because security must be end-to-end these days, and its not uncommon to see apps starting to approach the 2 second mark.

These days, typical servers have anywhere between 20 to 400 Gbps NICs with latencies measured in tens of microseconds, yet apps are responding 10,000x slower even when doing no processing. Why? Because everyone involved has their own little problem to solve, and nobody cares about the big picture as long as the threshold isn't exceeded. HTTPS was "easy" for a bunch of web-devs moving into full-stack programming. Binary RPC is "hard" and they didn't bother, because for simple apps it makes "no difference" as both fall under the PPT.

Answer me this: How many HTTPS client programming libraries (not web browsers!) actually do TCP fast open and TLS 1.3 0-RTT handshakes? How many do that by default? Name a load balancer product that turns those features on by default. Name a reverse proxy that does that by default.

Nobody(1) turns on jumbo frames. Nobody does RDMA, or SR-IOV, or cut-through switching, or ECN, or whatever. Everybody has firewalls for no reason. I say no reason, because if all you're doing is doing some ACLs, your switches can almost certainly do that at wire-rate with zero latency overheads.

It always comes back to the PPT. As long as a design, network, architecture, system, language, or product is under, people stop caring. They stop caring even if 1000x better performance is just a checkbox away. Even if it is something they have already paid for. Even if it's free.

1) I'm generalising, clearly. AWS, Azure, and GCP actually do most of that, but then they rate limit anyway, negating the benefits for all but the largest VM sizes.