Actually save the configuration changes