Create config dir and move newrelic.yml to it.