Some more attempst at efficiency.

We're seeing some modest wins, but the overall strategy may be flawed.
We really just need a way to further segregate the queues, because right now
we're probably getting gigantic factors that we realistically won't be able to
use.

Probably worth testing with a cut-off for now.
Bugs fixed, performance looking OK.

It would still be nice to try some more optimizations.