Add a dismiss button to the alert