Cli tests

Opening this up for further discussion. Closes #450.
This should test the stdout/stderr splitting as well.