Fix tmpdir integration tests

This will fix the tests that broke in #6456.

Ping @unclejack @crosbymichael