Fixes a few issues linked to ClasspathTests

Discovered while looking at the failures in scala/scala#3939.