http://developer.android.com/guide/topics/ui/dialogs.html
Please review the last line:
So you should generally call dismiss() when the user presses the positive button in your dialog in order to remove the dialog from view.
It's better to call dismiss() in positivebutton, and call cancel in Negative button.

Also there is some example to call cancel in Negative button
http://www.sourcecodester.com/android/5575/alert-dialog.html