Fix issues when a `config.rb` file is present somewhere in the directories

In this case the `config.yml` or `config.json` files might not get parsed, resulting in missing configuration values, and/or errors