CB-2680: account for dismissed dialogs in Notification