Make configuration file a temporary file