Add option to pass in third directory

Assume we will use the integration test marker that was
already committed to mark tests as requiring the third
directory