Adds an alert when deleting a pair