> First, you've actually said "i have no idea if the compiler would have fixed this, because i didn't run it"

No I did not. Please don't distort what I wrote (which appears to be the only way you have of 'making' your point) What I did write was "I didn't measure with and without", meaning I didn't do the cross check of what the total improvement would have been without the normal compiler optimizations, because both the original and the final were run with standard -Os optimizations, and because I have a fairly robust intuition in terms of the bounds of the effect.

For shits and giggles, I now did the cross check. The difference between -O0 and -Ofast was in the noise. Sometimes the -O0 was faster, sometimes the -Ofast was faster, slightly skewing towards -Ofast and with times varying by around 10%.

So let's be generous and assume that it wasn't noise, that it was 10% in the direction of the -Ofast option. Heck, let's double it to 20% just to be on the safe side. That's 20% vs. 1000x. Compared to the total, the contribution of the compiler optimizations vs. the manual optimizations is 2:10000. Or 99.98% vs 0.02%. Compare Proebsting's Law.

The rest of your post is full of "had you" and "might have" and "could have". And you berate others for not delivering data. Hmm... And then you go off and cite research that did something that seems vaguely keyword-related to the things I described, but is actually nothing at all like it.

Again, my example is

CSV -> ( Object Model : ORM (CoreData) : SQL Database (SQLite) )

Transformed to

CSV -> ( Optimized binary data model ) -> fwrite()

And now tell me what concrete, existing optimizing compiler (production preferred, research might be OKish) is going to do that transformation and get me my 1000:1 performance improvement. No "there was a research compiler that did some data layout transformations somewhere", but concretely, a compiler that will take that first solution and automatically generate the second solution with the 1000:1 perf. improvement.

Please also take into consideration that the CoreData ORM and SQLite DB are not available in source, but only via shared library / API.

The original is described here: https://www.objc.io/issues/4-core-data/importing-large-data-...

My modified version is described in the talk: https://www.youtube.com/watch?v=kHG_zw75SjE

Code: https://github.com/mpw/issue-4-importing-and-fetching

Sample data sets: https://vbb-gtfs.jannisr.de/2017-05-10

> Had you written it in a language ...

If my grandmother had wheels, she'd be a bus. Starting an argument (or a logical deduction) with a false premise is not exactly useful. Sorry, if your idea of the power of optimizing compilers is "you need to first rewrite the code", then I can just rewrite the code to be fast and leave out the middle-man. Your argument here is ridiculous beyond the pale.

> 15 years ago I saw a compiler...

And I saw a UFO. This is your strong "actual data"? Yes, research compilers can sometimes do funky stuff in research settings. Same for JITs. I very much remember that special issue of the IBM system's journal on Java performance. It featured a bunch of reports from the researchers touting performance numbers, and then one real world report from the San Francisco project that reported their performance "challenges". See Jitterdämmerung: http://blog.metaobject.com/2015/10/jitterdammerung.html

> ...that did automatic splitting and distribution of COM objects to split up a game to run half on a server and half on a client.

And how is this relevant to the example at hand, except in "proving" (which it doesn't, by the way) that compilers can do crazy transformations (which so isn't the point)? Was this a production compiler? Was it general purpose? Is it still around today?

The point is not what compilers can do. It is how useful that is in practice. And yes, there exists code outside of Google data centers.

> ...qbasic and then say "optimizing compilers suck at reorganizing my data layout"

Nobody said that, except you. You need to stop making up straw-man arguments that you can then demolish, but rather deal with actual facts.