Make classpath text file if it doesn't exist

Fixes regressions in 88062b4b06