Cli tests

Opening this up for further discussion. Closes #450.
I also think we should wait for job events here (was removed in 98129329f3871fd1629439c23b0732ae3026be3a), just to confirm that the scaling is functioning correctly.