Don't eat extra cycles when idling.

Rescheduling between threads takes about 3000 cycles, but that's not true
for both sides of e.g. a delay.

Fixes #6409 (assuming 60fps is what a PSP gets.)