Don't overwrite existing config files

Addresses #1094. Also, there was no :written_file notification for
Capfile even though there was for all other config files.