Factor out building the test paths

Make sure the organised directory exists.