Fix run-in-machine testcase + add it to the tests