Properly handle the provider setup defer cancel.

Fix issues related to "Cancel login does not work".

[Closes #4869]
[Closes #4973]