Repository: zulip/zulip

Issue:
Push bouncer: Deduplicate device registrations
In the migration to using uuid instead of simple id as the user identifier in the push bouncer (d800ac33a0322b0d3ef327bcebf318a27448bbee) we preserved legacy `id`-based registrations to avoid breaking notifications for all the devices already registered. However this means the possibility of duplicate registrations for one device - if a device was registered with `id` originally and then re-registered at some point after this migration, creating a new, `uuid`-based registration. 

The bouncer will send duplicate notifications to such a device.

The `RemotePushDeviceToken` data model doesn't have a way of linking  `id`s to `uuid`s of a user, so we can't just write a database migration to detect and delete duplicates.  The proposed solution is the following:
1. Currently, when a Zulip server sends a notification to the bouncer (https://github.com/zulip/zulip/blob/3f199d570d5b238e28ee6b0e7c7a0ca72156a790/zerver/lib/push_notifications.py#L487) it sends both the `user_id` and `user_uuid` to the bouncer. Thus we can tweak the bouncer to detect in that moment if both registrations exist for the same `token`  - and if so, delete the old `user_id` registration. This will over time clean up existing duplicates.
2. When the bouncer receives a new `uuid` registration request to `remotes/push/register` it should check if there isn't already an existing `id` registration for the `token` and if there is, delete it. This will prevent future duplications.

https://chat.zulip.org/#narrow/stream/48-mobile/topic/iOS.20duplicate.20notifications for more background


Discussion:
Hello @zulip/server-production members, this issue was labeled with the "area: production" label, so you may want to check it out!

<!-- areaLabelAddition -->


Resolving patch:
diff --git a/zerver/lib/push_notifications.py b/zerver/lib/push_notifications.py
--- a/zerver/lib/push_notifications.py
+++ b/zerver/lib/push_notifications.py
@@ -547,6 +547,9 @@ def add_push_device_token(
         post_data = {
             "server_uuid": settings.ZULIP_ORG_ID,
             "user_uuid": str(user_profile.uuid),
+            # user_id is sent so that the bouncer can delete any pre-existing registrations
+            # for this user+device to avoid duplication upon adding the uuid registration.
+            "user_id": str(user_profile.id),
             "token": token_str,
             "token_kind": kind,
         }
diff --git a/zilencer/views.py b/zilencer/views.py
--- a/zilencer/views.py
+++ b/zilencer/views.py
@@ -1,5 +1,6 @@
 import datetime
 import logging
+from collections import Counter
 from typing import Any, Dict, List, Optional, Type, TypeVar
 from uuid import UUID
 
@@ -157,23 +158,26 @@ def register_remote_push_device(
     if user_id is None and user_uuid is None:
         raise JsonableError(_("Missing user_id or user_uuid"))
     if user_id is not None and user_uuid is not None:
-        # We don't want "hybrid" registrations with both.
-        # Our RemotePushDeviceToken should be either in the new uuid format
-        # or the legacy id one.
-        raise JsonableError(_("Specify only one of user_id or user_uuid"))
-
+        kwargs: Dict[str, object] = {"user_uuid": user_uuid, "user_id": None}
+        # Delete pre-existing user_id registration for this user+device to avoid
+        # duplication. Further down, uuid registration will be created.
+        RemotePushDeviceToken.objects.filter(
+            server=server, token=token, kind=token_kind, user_id=user_id
+        ).delete()
+    else:
+        # One of these is None, so these kwargs will lead to a proper registration
+        # of either user_id or user_uuid type
+        kwargs = {"user_id": user_id, "user_uuid": user_uuid}
     try:
         with transaction.atomic():
             RemotePushDeviceToken.objects.create(
-                # Exactly one of these two user identity fields will be None.
-                user_id=user_id,
-                user_uuid=user_uuid,
                 server=server,
                 kind=token_kind,
                 token=token,
                 ios_app_id=ios_app_id,
                 # last_updated is to be renamed to date_created.
                 last_updated=timezone.now(),
+                **kwargs,
             )
     except IntegrityError:
         pass
@@ -216,13 +220,73 @@ def unregister_all_remote_push_devices(
     return json_success(request)
 
 
+def delete_duplicate_registrations(
+    registrations: List[RemotePushDeviceToken], server_id: int, user_id: int, user_uuid: str
+) -> List[RemotePushDeviceToken]:
+    """
+    When migrating to support registration by UUID, we introduced a bug where duplicate
+    registrations for the same device+user could be created - one by user_id and one by
+    user_uuid. Given no good way of detecting these duplicates at database level, we need to
+    take advantage of the fact that when a remote server sends a push notification request
+    to us, it sends both user_id and user_uuid of the user.
+    See https://github.com/zulip/zulip/issues/24969 for reference.
+
+    This function, knowing the user_id and user_uuid of the user, can detect duplicates
+    and delete the legacy user_id registration if appropriate.
+
+    Return the list of registrations with the user_id-based duplicates removed.
+    """
+
+    # All registrations passed here should be of the same kind (apple vs android).
+    assert len({registration.kind for registration in registrations}) == 1
+    kind = registrations[0].kind
+
+    tokens_counter = Counter(device.token for device in registrations)
+
+    tokens_to_deduplicate = []
+    for key in tokens_counter:
+        if tokens_counter[key] <= 1:
+            continue
+        if tokens_counter[key] > 2:
+            raise AssertionError(
+                f"More than two registrations for token {key} for user id:{user_id} uuid:{user_uuid}, shouldn't be possible"
+            )
+        assert tokens_counter[key] == 2
+        tokens_to_deduplicate.append(key)
+
+    if not tokens_to_deduplicate:
+        return registrations
+
+    logger.info(
+        "Deduplicating push registrations for server id:%s user id:%s uuid:%s and tokens:%s",
+        server_id,
+        user_id,
+        user_uuid,
+        sorted(tokens_to_deduplicate),
+    )
+    RemotePushDeviceToken.objects.filter(
+        token__in=tokens_to_deduplicate, kind=kind, server_id=server_id, user_id=user_id
+    ).delete()
+
+    deduplicated_registrations_to_return = []
+    for registration in registrations:
+        if registration.token in tokens_to_deduplicate and registration.user_id is not None:
+            # user_id registrations are the ones we deleted
+            continue
+        deduplicated_registrations_to_return.append(registration)
+
+    return deduplicated_registrations_to_return
+
+
 @has_request_variables
 def remote_server_notify_push(
     request: HttpRequest,
     server: RemoteZulipServer,
     payload: Dict[str, Any] = REQ(argument_type="body"),
 ) -> HttpResponse:
-    user_identity = UserPushIdentityCompat(payload.get("user_id"), payload.get("user_uuid"))
+    user_id = payload.get("user_id")
+    user_uuid = payload.get("user_uuid")
+    user_identity = UserPushIdentityCompat(user_id, user_uuid)
 
     gcm_payload = payload["gcm_payload"]
     apns_payload = payload["apns_payload"]
@@ -235,6 +299,10 @@ def remote_server_notify_push(
             server=server,
         )
     )
+    if android_devices and user_id is not None and user_uuid is not None:
+        android_devices = delete_duplicate_registrations(
+            android_devices, server.id, user_id, user_uuid
+        )
 
     apple_devices = list(
         RemotePushDeviceToken.objects.filter(
@@ -243,6 +311,8 @@ def remote_server_notify_push(
             server=server,
         )
     )
+    if apple_devices and user_id is not None and user_uuid is not None:
+        apple_devices = delete_duplicate_registrations(apple_devices, server.id, user_id, user_uuid)
 
     logger.info(
         "Sending mobile push notifications for remote user %s:%s: %s via FCM devices, %s via APNs devices",


Test patch:
diff --git a/zerver/tests/test_push_notifications.py b/zerver/tests/test_push_notifications.py
--- a/zerver/tests/test_push_notifications.py
+++ b/zerver/tests/test_push_notifications.py
@@ -192,12 +192,6 @@ def test_register_remote_push_user_params(self) -> None:
             self.server_uuid, endpoint, {"token": token, "token_kind": token_kind}
         )
         self.assert_json_error(result, "Missing user_id or user_uuid")
-        result = self.uuid_post(
-            self.server_uuid,
-            endpoint,
-            {"user_id": user_id, "user_uuid": "xxx", "token": token, "token_kind": token_kind},
-        )
-        self.assert_json_error(result, "Specify only one of user_id or user_uuid")
         result = self.uuid_post(
             self.server_uuid, endpoint, {"user_id": user_id, "token": token, "token_kind": 17}
         )
@@ -280,6 +274,40 @@ def test_register_remote_push_user_params(self) -> None:
             401,
         )
 
+    def test_register_device_deduplication(self) -> None:
+        hamlet = self.example_user("hamlet")
+        token = "111222"
+        user_id = hamlet.id
+        user_uuid = str(hamlet.uuid)
+        token_kind = PushDeviceToken.GCM
+
+        endpoint = "/api/v1/remotes/push/register"
+
+        # First we create a legacy user_id registration.
+        result = self.uuid_post(
+            self.server_uuid,
+            endpoint,
+            {"user_id": user_id, "token_kind": token_kind, "token": token},
+        )
+        self.assert_json_success(result)
+
+        registrations = list(RemotePushDeviceToken.objects.filter(token=token))
+        self.assert_length(registrations, 1)
+        self.assertEqual(registrations[0].user_id, user_id)
+        self.assertEqual(registrations[0].user_uuid, None)
+
+        # Register same user+device with uuid now. The old registration should be deleted
+        # to avoid duplication.
+        result = self.uuid_post(
+            self.server_uuid,
+            endpoint,
+            {"user_id": user_id, "user_uuid": user_uuid, "token_kind": token_kind, "token": token},
+        )
+        registrations = list(RemotePushDeviceToken.objects.filter(token=token))
+        self.assert_length(registrations, 1)
+        self.assertEqual(registrations[0].user_id, None)
+        self.assertEqual(str(registrations[0].user_uuid), user_uuid)
+
     def test_remote_push_user_endpoints(self) -> None:
         endpoints = [
             ("/api/v1/remotes/push/register", "register"),
@@ -308,6 +336,7 @@ def test_send_notification_endpoint(self) -> None:
         server = RemoteZulipServer.objects.get(uuid=self.server_uuid)
         token = "aaaa"
         android_tokens = []
+        uuid_android_tokens = []
         for i in ["aa", "bb"]:
             android_tokens.append(
                 RemotePushDeviceToken.objects.create(
@@ -317,6 +346,19 @@ def test_send_notification_endpoint(self) -> None:
                     server=server,
                 )
             )
+
+            # Create a duplicate, newer uuid-based registration for the same user to verify
+            # the bouncer will handle that correctly, without triggering a duplicate notification,
+            # and will delete the old, legacy registration.
+            uuid_android_tokens.append(
+                RemotePushDeviceToken.objects.create(
+                    kind=RemotePushDeviceToken.GCM,
+                    token=hex_to_b64(token + i),
+                    user_uuid=str(hamlet.uuid),
+                    server=server,
+                )
+            )
+
         apple_token = RemotePushDeviceToken.objects.create(
             kind=RemotePushDeviceToken.APNS,
             token=hex_to_b64(token),
@@ -326,6 +368,7 @@ def test_send_notification_endpoint(self) -> None:
         many_ids = ",".join(str(i) for i in range(1, 250))
         payload = {
             "user_id": hamlet.id,
+            "user_uuid": str(hamlet.uuid),
             "gcm_payload": {"event": "remove", "zulip_message_ids": many_ids},
             "apns_payload": {
                 "badge": 0,
@@ -355,12 +398,14 @@ def test_send_notification_endpoint(self) -> None:
             logger.output,
             [
                 "INFO:zilencer.views:"
-                f"Sending mobile push notifications for remote user 6cde5f7a-1f7e-4978-9716-49f69ebfc9fe:<id:{hamlet.id}>: "
-                "2 via FCM devices, 1 via APNs devices"
+                f"Deduplicating push registrations for server id:{server.id} user id:{hamlet.id} uuid:{str(hamlet.uuid)} and tokens:{sorted([t.token for t in android_tokens[:]])}",
+                "INFO:zilencer.views:"
+                f"Sending mobile push notifications for remote user 6cde5f7a-1f7e-4978-9716-49f69ebfc9fe:<id:{hamlet.id}><uuid:{str(hamlet.uuid)}>: "
+                "2 via FCM devices, 1 via APNs devices",
             ],
         )
 
-        user_identity = UserPushIdentityCompat(user_id=hamlet.id)
+        user_identity = UserPushIdentityCompat(user_id=hamlet.id, user_uuid=str(hamlet.uuid))
         apple_push.assert_called_once_with(
             user_identity,
             [apple_token],
@@ -377,7 +422,7 @@ def test_send_notification_endpoint(self) -> None:
         )
         android_push.assert_called_once_with(
             user_identity,
-            list(reversed(android_tokens)),
+            list(reversed(uuid_android_tokens)),
             {"event": "remove", "zulip_message_ids": ",".join(str(i) for i in range(50, 250))},
             {},
             remote=server,