Repository: zulip/zulip

Issue:
Presence dot should be in offline mode when not sharing availability
We have an Account & privacy setting to turn off "Display my availability to other users when online". When a user selects this setting, they are indicating that they don't want others to know that they are online.

We should therefore *immediately* update their presence indicator dot to show them as offline. Users will still be able to see "Active now" in the tooltip for the first couple of minutes, but this is much less salient than the presence dot. Based on a brief investigation, other chat apps take this approach as well.

This issue might address the user impact of #18846.

We should probably treat it as a blocker for #21178, which will make the hiding availability feature much more prominent.



Discussion:
Hello @zulip/server-sidebars members, this issue was labeled with the "area: right-sidebar" label, so you may want to check it out!

<!-- areaLabelAddition -->

@zulipbot claim
Hello @raghavluthra20, it looks like you've currently claimed 1 issue in this repository. We encourage new contributors to focus their efforts on at most 1 issue at a time, so please complete your work on your other claimed issues before trying to claim this issue again.

We look forward to your valuable contributions!

@alya Can you assign this to me?
Hello @raghavluthra20, you claimed this issue to work on it, but this issue and any referenced pull requests haven't been updated for 10 days. Are you still working on this issue?

If so, please update this issue by leaving a comment on this issue to let me know that you're still working on it. Otherwise, I'll automatically remove you from this issue in 4 days.

If you've decided to work on something else, simply comment `@zulipbot abandon` so that someone else can claim it and continue from where you left off.

Thank you for your valuable contributions to Zulip!

<!-- inactiveWarning -->

@zulipbot abandon
@alya I need some clarification on this issue. I am unclear about the actual change that needs to be implemented because the presence dot gets changed to offline immediately the user unchecks the "Display my availability to other users when online" in the privacy settings. Are you suggesting a change in the text within the tooltip? If Yes what would be more ideal?
Hey @nafiudanlawal, the problem is that it doesn't update immediately for _other users_ (takes around 3-5 minutes). I clarified this in the related [CZO discussion thread](https://chat.zulip.org/#narrow/stream/2-general/topic/.22unavailable.22.20status).
@alya WIP [PR - pls read notes there](https://github.com/zulip/zulip/pull/22387).


Resolving patch:
diff --git a/zerver/actions/presence.py b/zerver/actions/presence.py
--- a/zerver/actions/presence.py
+++ b/zerver/actions/presence.py
@@ -3,6 +3,7 @@
 from typing import Optional
 
 from django.conf import settings
+from django.db import transaction
 
 from zerver.actions.user_activity import update_user_activity_interval
 from zerver.decorator import statsd_increment
@@ -13,7 +14,9 @@
 from zerver.tornado.django_api import send_event
 
 
-def send_presence_changed(user_profile: UserProfile, presence: UserPresence) -> None:
+def send_presence_changed(
+    user_profile: UserProfile, presence: UserPresence, *, force_send_update: bool = False
+) -> None:
     # Most presence data is sent to clients in the main presence
     # endpoint in response to the user's own presence; this results
     # data that is 1-2 minutes stale for who is online.  The flaw with
@@ -24,7 +27,10 @@ def send_presence_changed(user_profile: UserProfile, presence: UserPresence) ->
     # See https://zulip.readthedocs.io/en/latest/subsystems/presence.html for
     # internals documentation on presence.
     user_ids = active_user_ids(user_profile.realm_id)
-    if len(user_ids) > settings.USER_LIMIT_FOR_SENDING_PRESENCE_UPDATE_EVENTS:
+    if (
+        len(user_ids) > settings.USER_LIMIT_FOR_SENDING_PRESENCE_UPDATE_EVENTS
+        and not force_send_update
+    ):
         # These immediate presence generate quadratic work for Tornado
         # (linear number of users in each event and the frequency of
         # users coming online grows linearly with userbase too).  In
@@ -64,7 +70,12 @@ def consolidate_client(client: Client) -> Client:
 
 @statsd_increment("user_presence")
 def do_update_user_presence(
-    user_profile: UserProfile, client: Client, log_time: datetime.datetime, status: int
+    user_profile: UserProfile,
+    client: Client,
+    log_time: datetime.datetime,
+    status: int,
+    *,
+    force_send_update: bool = False,
 ) -> None:
     client = consolidate_client(client)
 
@@ -103,8 +114,18 @@ def do_update_user_presence(
             update_fields.append("status")
         presence.save(update_fields=update_fields)
 
-    if not user_profile.realm.presence_disabled and (created or became_online):
-        send_presence_changed(user_profile, presence)
+    if force_send_update or (
+        not user_profile.realm.presence_disabled and (created or became_online)
+    ):
+        # We do a the transaction.on_commit here, rather than inside
+        # send_presence_changed, to help keep presence transactions
+        # brief; the active_user_ids call there is more expensive than
+        # this whole function.
+        transaction.on_commit(
+            lambda: send_presence_changed(
+                user_profile, presence, force_send_update=force_send_update
+            )
+        )
 
 
 def update_user_presence(
diff --git a/zerver/actions/user_settings.py b/zerver/actions/user_settings.py
--- a/zerver/actions/user_settings.py
+++ b/zerver/actions/user_settings.py
@@ -2,11 +2,13 @@
 from typing import List, Optional, Union
 
 import orjson
+from django.conf import settings
 from django.db import transaction
 from django.db.models import F
 from django.utils.timezone import now as timezone_now
 
 from confirmation.models import Confirmation, create_confirmation_link
+from zerver.actions.presence import do_update_user_presence
 from zerver.lib.avatar import avatar_url
 from zerver.lib.cache import (
     cache_delete,
@@ -26,9 +28,11 @@
     RealmAuditLog,
     ScheduledEmail,
     ScheduledMessageNotificationEmail,
+    UserPresence,
     UserProfile,
     active_user_ids,
     bot_owner_user_ids,
+    get_client,
     get_user_profile_by_id,
 )
 from zerver.tornado.django_api import send_event
@@ -454,3 +458,43 @@ def do_change_user_setting(
         # not deleted every previously synced draft - to do that use the DELETE
         # endpoint.
         Draft.objects.filter(user_profile=user_profile).delete()
+
+    if setting_name == "presence_enabled":
+        # The presence_enabled setting's primary function is to stop
+        # doing presence updates for the user altogether.
+        #
+        # When a user toggles the presence_enabled setting, we
+        # immediately trigger a presence update, so all users see the
+        # user's current presence state as consistent with the new
+        # setting; not doing so can make it look like the settings
+        # change didn't have any effect.
+        if setting_value:
+            status = UserPresence.ACTIVE
+            presence_time = timezone_now()
+        else:
+            # HACK: Remove existing presence data for the current user
+            # when disabling presence. This hack will go away when we
+            # replace our presence data structure with a simpler model
+            # that doesn't separate individual clients.
+            UserPresence.objects.filter(user_profile_id=user_profile.id).delete()
+
+            # We create a single presence entry for the user, old
+            # enough to be guaranteed to be treated as offline by
+            # correct clients, such that the user will, for as long as
+            # presence remains disabled, appear to have been last
+            # online a few minutes before they disabled presence.
+            #
+            # We add a small additional offset as a fudge factor in
+            # case of clock skew.
+            status = UserPresence.IDLE
+            presence_time = timezone_now() - datetime.timedelta(
+                seconds=settings.OFFLINE_THRESHOLD_SECS + 120
+            )
+
+        do_update_user_presence(
+            user_profile,
+            get_client("website"),
+            presence_time,
+            status,
+            force_send_update=True,
+        )


Test patch:
diff --git a/zerver/tests/test_event_system.py b/zerver/tests/test_event_system.py
--- a/zerver/tests/test_event_system.py
+++ b/zerver/tests/test_event_system.py
@@ -5,6 +5,7 @@
 import orjson
 from django.conf import settings
 from django.http import HttpRequest, HttpResponse
+from django.test import override_settings
 from django.utils.timezone import now as timezone_now
 
 from version import API_FEATURE_LEVEL, ZULIP_MERGE_BASE, ZULIP_VERSION
@@ -1200,8 +1201,12 @@ def test_get_raw_user_data_on_system_bot_realm(self) -> None:
 
 
 class TestUserPresenceUpdatesDisabled(ZulipTestCase):
+    # For this test, we verify do_update_user_presence doesn't send
+    # events for organizations with more than
+    # USER_LIMIT_FOR_SENDING_PRESENCE_UPDATE_EVENTS users, unless
+    # force_send_update is passed.
+    @override_settings(USER_LIMIT_FOR_SENDING_PRESENCE_UPDATE_EVENTS=3)
     def test_presence_events_disabled_on_larger_realm(self) -> None:
-        # First check that normally the mocked function gets called.
         events: List[Mapping[str, Any]] = []
         with self.tornado_redirected_to_list(events, expected_num_events=1):
             do_update_user_presence(
@@ -1209,15 +1214,14 @@ def test_presence_events_disabled_on_larger_realm(self) -> None:
                 get_client("website"),
                 timezone_now(),
                 UserPresence.ACTIVE,
+                force_send_update=True,
             )
 
-        # Now check that if the realm has more than the USER_LIMIT_FOR_SENDING_PRESENCE_UPDATE_EVENTS
-        # amount of active users, send_event doesn't get called.
         with self.tornado_redirected_to_list(events, expected_num_events=0):
-            with self.settings(USER_LIMIT_FOR_SENDING_PRESENCE_UPDATE_EVENTS=1):
-                do_update_user_presence(
-                    self.example_user("hamlet"),
-                    get_client("website"),
-                    timezone_now(),
-                    UserPresence.ACTIVE,
-                )
+            do_update_user_presence(
+                self.example_user("hamlet"),
+                get_client("website"),
+                timezone_now(),
+                UserPresence.ACTIVE,
+                force_send_update=False,
+            )
diff --git a/zerver/tests/test_events.py b/zerver/tests/test_events.py
--- a/zerver/tests/test_events.py
+++ b/zerver/tests/test_events.py
@@ -1733,7 +1733,11 @@ def test_change_is_guest(self) -> None:
 
     def test_change_notification_settings(self) -> None:
         for notification_setting, v in self.user_profile.notification_setting_types.items():
-            if notification_setting in ["notification_sound", "desktop_icon_count_display"]:
+            if notification_setting in [
+                "notification_sound",
+                "desktop_icon_count_display",
+                "presence_enabled",
+            ]:
                 # These settings are tested in their own tests.
                 continue
 
@@ -1769,6 +1773,26 @@ def test_change_notification_settings(self) -> None:
                 check_user_settings_update("events[0]", events[0])
                 check_update_global_notifications("events[1]", events[1], setting_value)
 
+    def test_change_presence_enabled(self) -> None:
+        presence_enabled_setting = "presence_enabled"
+
+        for val in [True, False]:
+            events = self.verify_action(
+                lambda: do_change_user_setting(
+                    self.user_profile, presence_enabled_setting, val, acting_user=self.user_profile
+                ),
+                num_events=3,
+            )
+            check_user_settings_update("events[0]", events[0])
+            check_update_global_notifications("events[1]", events[1], val)
+            check_presence(
+                "events[2]",
+                events[2],
+                has_email=True,
+                presence_key="website",
+                status="active" if val else "idle",
+            )
+
     def test_change_notification_sound(self) -> None:
         notification_setting = "notification_sound"