Clean up config file after successful discard