Update tests to match new executor logging

Also, fix flakey tests and reduce long timeout by
resetting executor_runner.Config on stop