Remove usage of ConfigDirCreator in ConfigResolver for now

This behaviour is not really useful yet here as it is not fully
implemented yet. And it might cause confusion now.