A bunch of performance improvements. Targeted at #981