core: don't use Tempdir for tests, use Dir.mktmpdir

Tempdir didn't work on Windows, just spun forever.