Fix the stupid scheduling for better performance (should be much faster).

Also address some other issues that came up in code review.