Exit 1 on scheduler abort.
Fixed the state-dump tests and one restart test.

On this branch we exit from main loop immediately on shutdown instead of
going through the main loop once more (or partially) leading to minor
state differences at shutdown (e.g. ready vs waiting on shutdown -
the former triggers immediately on restart; the latter only after
prereqs are satisfied).