> There's no inherent reason that this should require code changes though - that's a C++ limitation.

In some simple cases static "constant inference" could help, but there is an inherent reason why this is not the case in general. Specialization requires deciding some property, and in any case -- even in the left-right example -- that property can be undecidable (or not decidable by the compiler). It is simply impossible for a compiler to always know whether some value at some site is constant. For example, instead of random_kind() I could use a_complicated_computation_that_yields_a_kind(x). The cost-free abstraction philosophy says that the programmer must be able to tell if an optimization is made, and this is simply not possible (it could be done with dependent types, but that certainly wouldn't lower the intrusive). For a JIT, that's not a problem. It would still specialize.

But it's always the question of control: do you prefer to work hard to guarantee that some information is known statically, or use a tool that figures it out automatically, sometimes worse than you, but usually better.

> so complexity in the performance characteristics of some construct is not accidental.

I would say that this kind of performance is. It's unrelated to the algorithm chosen by the programmer but to some compiler knob.

> I've come to think that hiding something without having a good model for it is one of the most dangerous things in software: the harder a program papers over the details, the harder it is to access them when one needs to - but inevitably one does eventually need to. Sooner or later you need to understand why two very similar pieces of code run at radically different speed, and finding that out with a JIT is far harder than putting extra information in your function signatures.

I don't know if I agree with your premise, but I strongly disagree with your conclusion, because a JIT doesn't take away this freedom. In fact, JITs (like in OpenJDK) have excellent tools that show you the decisions they've made[1], and you can control those decisions with compiler directives[2] (those are sometimes recommendations, but there's no problem making them requirements and extending their scope). So a JIT doesn't take away any of that control, it just doesn't push this complexity into the language. Besides, what you put in the signaure are only certain compiler knobs. Even in Rust/C++ we rely on compiler optimizations that we have no direct control over in the language.

> Java with Graal/Truffle making the various kinds of optimisation and staging more visible and developer-controlled.

I'm not sure what you're referring to. Graal and Truffle can be used on effectively a wide variety of languages without requiring them to change. But another example from Java demonstrates the different philosophy: value types. Choosing the "inline" memory layout for value types has some semantic implications, but it is also a technical detail; unfortunatly it's one that the compiler isn't good at guessing, so there will be an explicit annotation on the type's declaration. But some of the reason adding value types is taking so long is precisely because they don't want that detail to affect use sites, and are aiming for an outcome that would leave the vast majority of use-sites unchanged (possibly not even requiring recompilation, but I don't follow that project too closely and it's changing all the time so I'm not sure what the current prototype does).

[1]: https://github.com/AdoptOpenJDK/jitwatch/wiki, https://www.graalvm.org/docs/reference-manual/tools/#ideal-g...

[2]: https://docs.oracle.com/en/java/javase/11/vm/compiler-contro...