So the expanded matrix is just too slow unfortunately.

The parallel mode makes things much harder to read, but the overhead should be much much better.