Selectively re-enable the add/remove button if the button is in a known state