Properly save a non-existing configuration file