I went a bit crazy and pushed most speed-sensitive code into Cython with direct BLAS calls.

Some work is still needed to make sure the widespread changes integrate
properly with everything else.