Variable-width is the new hotness but it doesn’t work well on all tasks. If your task is fixed-width, or the algorithm changes based on task width, you kind of need to know the hardware width. And shuffling, perjuring, swizzling, and bit-shifting don’t translate very well to these approaches. Nor is there really an easy way to operate these in a fixed-width mode.

https://gist.github.com/zingaburga/805669eb891c820bd220418ee...

It’s not all bad but the people who just say “just treat it like an n-element array and let the hardware handle it bro!” are hand waving an enormous amount of algorithms that may not actually be translatable to that.

And broadly speaking, having a couple code-paths and choosing at runtime is not that bad. HPC is used to working “close to the metal” and tuning their code to run optimally. You can’t make that complexity go away - things will always run better on X hardware but Y hardware runs that code path way worse. In this domain all you do with clever auto-programming magic is obscure the problem - you are now writing for the optimizer instead of writing for the hardware, but different hardware still runs differently and you need multiple code paths to hit that optimally. So now you have two problems - keeping the hardware running optimally and keeping the optimizer from messing you up.

Since you are now dispatching large operations that will not complete atomically in one cycle (they simply won’t, you can’t do 2M elements on a 128-bit vector) it doesn’t seem likely that we will stay strictly in-order on these, and the next step is you start extracting parallelism from the stream and that’s the problem that immediately emerges from that. Now you have an optimizer running a SIMT program (opmasks are basically SIMT) extracted from the instruction stream, and you need to not just write code that runs fast - you can’t - you need to write code that the optimizer turns into code that runs fast, and that seems like absolute hell on an architecture where you can’t even know the number of registers-per-thread in advance. Maybe I am reading this all wrong but that’s where you’d really end up going with a “run this op on an N element array” model. If you run one vector-instruction at a time you are in cache/register hell constantly spilling to memory and back, so you want to optimize that to keep things in-register/in-cache, and that means extracting parallelism from the stream to run more stuff while it’s still somewhere reasonably hot.

This is the exact garden-path that GPU drivers went down and it was a huge mistake with OpenGL/DX9/DX11 that had to be walked back with the DX12/Vulkan APIs to get back closer to the metal, because the optimizer stage became completely inscrutable and managing it to keep it from doing something stupid became impossible.

There is a level of irreducible complexity in HPC and exposing the hardware is the best way to avoid it. Leaky abstractions make things worse.

Writing for a phone is not HPC but if you care about energy efficiency you still need to make sure you’re running reasonably optimally and not wasting cycles, especially with a super wide vector unit. That means you’re leaning heavily on the “sensible” code path having decent efficiency, which means you’re leaning on the hardware to behave sanely. So it’s the same thing.

Again, not saying it’s a completely bad idea, but you can already see the abstraction starting to leak with the various types of operations that don’t really work on a variable-width hardware concept, and that’s worrying. There is a lot more opportunity for this to go bad than I think people are at first glance.