Repository: thunderbird/thunderbird-android (language: kotlin)

Issue:
Duplicate notifications for new messages
### Checklist

- [X] I have used the search function to see if someone else has already submitted the same bug report.
- [X] I will describe the problem with as much detail as possible.

### App version

6.000

### Where did you get the app from?

Google Play

### Android version

11

### Device model

Moto g10

### Steps to reproduce

Part of the trigger seems to be only closing the notifications about a few of the received mails and the  receiving more mails. More I didn't find out in the last months.

### Expected behavior

If n mails arrive I normally get notifications with all their title lines 

### Actual behavior

Rarely I get n notifications with the title lone of one of my mails instead

### Logs

_No response_


Discussion hints:
![Screenshot_20220519-111601](https://user-images.githubusercontent.com/5041537/169515974-17934db9-97d2-4509-8b59-b181aa1ade81.png)

I actually got 8 mails, bit only one was about wxMaxima 
This also happens to me sometimes, a bunch of duplicate notifications for the same email. I am using K-9 Mail 6.0 on GrapheneOS (Android 12) on a Pixel 4a. 

This person has also noticed the issue: https://www.reddit.com/r/fdroid/comments/vob87m/why_does_k9mail_keep_sending_me_multiple/
Same problem. Driving me crazy!
On Android 11 the contents of the "collapsed notification" is generated by the OS. It usually takes the titles of the individual notifications to generate the summary. 

Does everything look okay when you expand the notification to see the individual notifications? If that's the case, this might be a bug in your Android version.
Unfortunately this time it seems not to be a bug in this specific Android11 feature:
![Screenshot_20220709-114305](https://user-images.githubusercontent.com/5041537/178101951-0a372527-ef71-4e59-9de7-02b366bcbd89.png)

Issue is still present in version 6.20
I have the impression that a trigger might be that a reply arrives to an unread message that is dropped out of the mails kept in K9 by the arrival of the new message.
When this happens again, please try the following:

1. Force stop the app. This should also remove all notifications.
2. Do something that will start the app again. That will recreate the notifications from K-9 Mail's database. However, don't just open the app if this will open a message list view that contains the messages in the notifications (because that will lead to the notifications being removed). You could add a new "K-9 Unread" widget to the home screen. That process will launch the app, triggering the notifications to be recreated. But it won't display a message list that will lead to the notifications being removed.
3. Check if there are still duplicate notifications and report back.
We are lucky: Normally this triggers about once per week. Today I found two more puzzle pieces: I got 6 repeated notifications and one notification about a different mail.
And killing K9 and restarting it using the widget caused only the repeated notifications to reappear.
![Screenshot_20220710-080401](https://user-images.githubusercontent.com/5041537/178133664-633c0e43-0893-4811-b792-a4f3f2652134.png)

That result is raising more questions than it is able to provide answers.

So you started out with 7 notifications, 6 of them for the same message? And you ended up with 3 notifications, all of them for the same message, the one that was repeated 6 times before the app restart?

What happened to the other message? Was it deleted, moved, or marked as read using another client?

Before I made the screenshot the 6 repeated Mails were still displayed in the notification drawer. I believed that the other mail that was no more shown might have been marked as read by tapping on the K9 widget. My kmail wasn't in a working order when I made this screenshot...
Ok. So it sounds like all duplicate notifications are being restored. The database table that stores information about notifications doesn't allow more than one entry per internal message ID. That suggests that for K-9 Mail all of these entries are distinct messages.

When this happens again, please do the following:
1. Tap an individual notification to open the message. That should open the message view and remove the notification for that message.
2. Don't go back to the message list screen (because that would remove all notifications for messages in that folder). Leave the app using the home button.

Does that remove all duplicate notifications or just the one you tapped?

Repeat the process with the other notifications. Does tapping one of them remove more than one notification?

It removes only the duplicate I tried to remove. As if I get 10 duplicates that means I got 10 messages I guess that already made sense.

...but going back to the message list screen doesn't remove the duplicate notifications for me. As it often doesn't do so for non-duplicate messages that might possibly not mean anything, though...
@gunterkoenigsmann, do you pull/push multiple folders that contain the same message?

I had a similar problem with gmail, where k9mail gave duplicate notifications for all received messages.
I disabled pull/push for the Gmail/All-mail folder, so that I am pushing/polling just my inbox folder.
And now I get just one notification.
Maybe the title is misleading now: I actually get one notification per message. But sometimes (very rarely) many of the notifications get the notification text from the same message.

Pull request: Fix removing multiple notifications at once
Fixes #6077

Test command: ./gradlew :app:core:testDebugUnitTest --tests "com.fsck.k9.notification.NotificationDataStoreTest"

Fix patch:
diff --git a/app/core/src/main/java/com/fsck/k9/notification/NotificationDataStore.kt b/app/core/src/main/java/com/fsck/k9/notification/NotificationDataStore.kt
index db99f40340e..de9f8c94bea 100644
--- a/app/core/src/main/java/com/fsck/k9/notification/NotificationDataStore.kt
+++ b/app/core/src/main/java/com/fsck/k9/notification/NotificationDataStore.kt
@@ -108,32 +108,43 @@ internal class NotificationDataStore {
         account: Account,
         selector: (List<MessageReference>) -> List<MessageReference>
     ): RemoveNotificationsResult? {
-        val notificationData = getNotificationData(account)
+        var notificationData = getNotificationData(account)
         if (notificationData.isEmpty()) return null
 
         val removeMessageReferences = selector.invoke(notificationData.messageReferences)
+        if (removeMessageReferences.isEmpty()) return null
 
         val operations = mutableListOf<NotificationStoreOperation>()
         val newNotificationHolders = mutableListOf<NotificationHolder>()
         val cancelNotificationIds = mutableListOf<Int>()
 
-        for (messageReference in removeMessageReferences) {
-            val notificationHolder = notificationData.activeNotifications.firstOrNull {
-                it.content.messageReference == messageReference
+        val activeMessageReferences = notificationData.activeNotifications.map { it.content.messageReference }.toSet()
+        val (removeActiveMessageReferences, removeInactiveMessageReferences) = removeMessageReferences
+            .partition { it in activeMessageReferences }
+
+        if (removeInactiveMessageReferences.isNotEmpty()) {
+            val inactiveMessageReferences = notificationData.inactiveNotifications
+                .map { it.content.messageReference }.toSet()
+
+            for (messageReference in removeInactiveMessageReferences) {
+                if (messageReference in inactiveMessageReferences) {
+                    operations.add(NotificationStoreOperation.Remove(messageReference))
+                }
             }
 
-            if (notificationHolder == null) {
-                val inactiveNotificationHolder = notificationData.inactiveNotifications.firstOrNull {
-                    it.content.messageReference == messageReference
-                } ?: continue
+            val removeMessageReferenceSet = removeInactiveMessageReferences.toSet()
+            notificationData = notificationData.copy(
+                inactiveNotifications = notificationData.inactiveNotifications
+                    .filter { it.content.messageReference !in removeMessageReferenceSet }
+            )
+        }
 
-                operations.add(NotificationStoreOperation.Remove(messageReference))
+        for (messageReference in removeActiveMessageReferences) {
+            val notificationHolder = notificationData.activeNotifications.first {
+                it.content.messageReference == messageReference
+            }
 
-                val newNotificationData = notificationData.copy(
-                    inactiveNotifications = notificationData.inactiveNotifications - inactiveNotificationHolder
-                )
-                notificationDataMap[account.uuid] = newNotificationData
-            } else if (notificationData.inactiveNotifications.isNotEmpty()) {
+            if (notificationData.inactiveNotifications.isNotEmpty()) {
                 val newNotificationHolder = notificationData.inactiveNotifications.first()
                     .toNotificationHolder(notificationHolder.notificationId)
 
@@ -148,29 +159,29 @@ internal class NotificationDataStore {
                     )
                 )
 
-                val newNotificationData = notificationData.copy(
+                notificationData = notificationData.copy(
                     activeNotifications = notificationData.activeNotifications - notificationHolder +
                         newNotificationHolder,
                     inactiveNotifications = notificationData.inactiveNotifications.drop(1)
                 )
-                notificationDataMap[account.uuid] = newNotificationData
             } else {
                 cancelNotificationIds.add(notificationHolder.notificationId)
 
                 operations.add(NotificationStoreOperation.Remove(messageReference))
 
-                val newNotificationData = notificationData.copy(
+                notificationData = notificationData.copy(
                     activeNotifications = notificationData.activeNotifications - notificationHolder
                 )
-                notificationDataMap[account.uuid] = newNotificationData
             }
         }
 
+        notificationDataMap[account.uuid] = notificationData
+
         return if (operations.isEmpty()) {
             null
         } else {
             RemoveNotificationsResult(
-                notificationData = getNotificationData(account),
+                notificationData = notificationData,
                 notificationStoreOperations = operations,
                 notificationHolders = newNotificationHolders,
                 cancelNotificationIds = cancelNotificationIds


Test patch:
diff --git a/app/core/src/test/java/com/fsck/k9/notification/NotificationDataStoreTest.kt b/app/core/src/test/java/com/fsck/k9/notification/NotificationDataStoreTest.kt
index 57ffc8aba4f..c3000f77111 100644
--- a/app/core/src/test/java/com/fsck/k9/notification/NotificationDataStoreTest.kt
+++ b/app/core/src/test/java/com/fsck/k9/notification/NotificationDataStoreTest.kt
@@ -92,6 +92,38 @@ class NotificationDataStoreTest : RobolectricTest() {
         }
     }
 
+    @Test
+    fun `remove multiple notifications`() {
+        repeat(MAX_NUMBER_OF_NEW_MESSAGE_NOTIFICATIONS + 1) { index ->
+            notificationDataStore.addNotification(account, createNotificationContent(index.toString()), TIMESTAMP)
+        }
+
+        val result = notificationDataStore.removeNotifications(account) { it.dropLast(1) }
+
+        assertNotNull(result) { removeResult ->
+            assertThat(removeResult.notificationData.newMessagesCount).isEqualTo(1)
+            assertThat(removeResult.cancelNotificationIds).hasSize(MAX_NUMBER_OF_NEW_MESSAGE_NOTIFICATIONS)
+        }
+    }
+
+    @Test
+    fun `remove all notifications`() {
+        repeat(MAX_NUMBER_OF_NEW_MESSAGE_NOTIFICATIONS + 1) { index ->
+            notificationDataStore.addNotification(account, createNotificationContent(index.toString()), TIMESTAMP)
+        }
+
+        val result = notificationDataStore.removeNotifications(account) { it }
+
+        assertNotNull(result) { removeResult ->
+            assertThat(removeResult.notificationData.newMessagesCount).isEqualTo(0)
+            assertThat(removeResult.notificationHolders).hasSize(0)
+            assertThat(removeResult.notificationStoreOperations).hasSize(MAX_NUMBER_OF_NEW_MESSAGE_NOTIFICATIONS + 1)
+            for (notificationStoreOperation in removeResult.notificationStoreOperations) {
+                assertThat(notificationStoreOperation).isInstanceOf(NotificationStoreOperation.Remove::class.java)
+            }
+        }
+    }
+
     @Test
     fun testRemoveDoesNotLeakNotificationIds() {
         for (i in 1..MAX_NUMBER_OF_NEW_MESSAGE_NOTIFICATIONS + 1) {