test: Make the runner easier to update

Signed-off-by: Lewis Marshall <lewis@lmars.net>
Merge pull request #491 from flynn/test-runner-updates

test: Make the runner easier to update