Mark the config as noreplace