Remove more filesystem manipulation from instantiation of Config

This avoids unintentionally creating directories when running tests.
Swap method names, to show which one is destructive