Fix unit tests so they run on both Windows and Unix
Refactor shared logic in tests

Extract the part that tests error/stderr as this is going to be a common pattern for ensuing tests