Making it so that all config can be provided via task config

Ok, thanks for the use case description, that's rather different from our approach so I didn't think of that. Seems to work just nicely. We have been keeping a sample version of the main config file in the repo and kept credentials in a git-ignored file ourselves so the need for such approach hasn't come up.