This Will Eventually Replace C++
By: Oleksandr Kaleniuk
Tags: Cpp, Future, Software Engineering, Software Development, Cloud Computing

I think, not only C++ in particular, but optimizing compilers in general overstayed their welcome. They come from the 1950s and although every particular compiler advanced greatly over the years, the core of the technology remained mostly unchallenged.

NASA, Public domain, via Wikimedia Commons

But what exactly would we want to challenge? What is a compiler?

I’d like to propose a mental model. It might feel alien at first but hear me out.

An optimizing compiler is a constrained minimization problem solver.

The target function, the thing we want to minimize is the run time of the resulting machine code.

The constraints are our source code. We used to see it as text, as some language construct. But essentially it’s just a set of constraints. We don’t care about how exactly it is executed, how exactly the machine code will look, we only want that, if there is 2+2 in the code, there will be 4 in the output.

And the mathematical space, where we’re looking for the minimum, is the space of all the possible valid machine code.

Of course, you can’t just compile a program by running some gradient descent algorithm in this space. The constraints are too many, the space is too large, none of the classical minimization algorithms are realistic in this space. So compilers use a special strategy to keep within constraints. They translate code from one representation to another and only “travel” the optimization space sparingly within the current representation.

This “travel” means that in every given representation, for instance, one of GCC intermediate languages, or LLVM intermediate representation, or even the machine code at the back end, the code is altered to run faster while maintaining some level of semantic equivalency.

E. g. Intel Atom Z has a very small fetch frame. It’s only 8 bytes. This means that given the average size of Intel instruction, it can’t normally take more than one instruction a fetch. GCC knows that. It rearranges instructions so the longer ones are followed with the shortest ones and now more pairs are fetched together. The program is semantically the same but it runs faster.

Sometimes you have to sacrifice some of the equivalency for performance. Arithmetic optimizations in floating-point numbers are such an example. Floating-point numbers are not strictly associative meaning that (a+b)+c ≠ a+(b+c) . They are “almost associative” but due to the way they manage the computational error, they aren’t. So if you want to rearrange your instructions to make code faster, you have to give up some of the equivalency strictness.

Also, with compilers, all the optimizations they do are programmed by their programmers. It’s essentially the refined wisdom of the crowd. People think of smart ways to make code faster, and these ways end up in compilers’ code.

Sometimes, they surpass any expectations. For instance, clang can rearrange array traversal into a binary search by itself!

But in the end, all that any specific optimizing compiler does is programmed by people. There is no intelligence in compilers.