Branch and cache miss speedups

Just some low hanging fruit.

Signed-off-by: Jens Axboe <axboe@fb.com>