Don't delete outdated config files, just back them up