Make the Delete button on config dialogs actually work