We're [low-level programmers] not dead, we're pining for the fjords. Low level programming is a dark art, and there are naturally plenty of people celebrating how fantastic it is that they don't _have_ to know this stuff. Don't celebrate your ignorance; you may be 2-4 orders of magnitude off after your performance is sucked away by all those layers of abstraction.

My take on it is that "low-level stuff" isn't going away any time soon, and that there will always be a equilibrium reached between those who understand it and are fluent with it (and, sometimes, are going to be able to write code that is 5-100x faster) and those who don't.

It's orthogonal to an understanding of algorithms. No amount of bit-bashing can fix really poor choices of algorithms (N^2 vs NlogN, say, on a big input). That being said, there are a lot of tasks where everyone is going to land on the same linear or logN basic algorithm and the main difference is going to be the algorithm that misses cache frequently and is stuffed with branch mispredicts and pipeline stalls vs. some algorithm that avoids all these problems and runs 10x faster. Sometimes you've just got to go do something to every bit of data and there's no classic algorithmic trick.

For example, I helped a former academic colleague write the 'worlds fastest floating point minimum' routine using SSE and lots of unrolling/software pipelining and I think he got about 8-10x (he had a library for most of the common operations like vector add, mul, etc but it didn't do 'min'). No amount of rampant algorithmic cleverness would avoid the need to look at each data element once when trying to calculate the min of a vector and if you're doing WORSE than linear, you've got real problems.

The major point that I've discovered from doing this stuff for years (on considerably more complex cases than the example above) is that programming efficiently for a modern architecture is qualitatively different than designing algorithms for the abstract '1 operation counts as 1 operation' machine in your average algorithms textbook. Oddly, quite a bit of improvement in my scalar, non-parallel programming came about after having used CUDA fairly intensively - if you're writing code for a Core 2 Duo or beyond, you're already parallel programming even if you're designing code that's single-threaded. Understanding how to rethink your algorithm to have data-parallelism (not to mention using SSE) is just plain conceptually different and more akin to parallel programming than scalar programming.

Knowing when to do this is important; I like bashing out a quick Python script as much as the next guy, or perhaps some pretty random C++ STL code that's probably an order of magnitude from where it should be (not because the STL is bad but because, say, I've been bone-lazy with design). So all those 'I have 10 layers of abstraction above this level" nincompoops shouldn't get too smug; we (low-level guys) can go there too - just because we know how to optimize C/asm loops to within an inch of their lives doesn't mean that we're going to compulsively do it with every last one.

Also worthy of note - the right thing to do changes frequently; some of the resources (especially on branch prediction) listed are already out of date. Don't bring P4 knowledge to a Sandy Bridge fight. Some bit twiddling hacks are great, others (especially ones that assume multiply is ultra-expensive) are obsolete on recent x86. The concepts still keep their validity a lot more than, say, all that newfangled crud that you youngsters fill your heads with (LAMP stacks, etc.) :-)