ENH Try all tasks in the inner loop

This inner execution loop is simple, but hard to get exactly right.