Repository: zulip/zulip

Issue:
sync_ldap_user_data error
It seems that after switching to version 5.1, the `LDAP_DEACTIVATE_NON_MATCHING_USERS` setting stopped working correctly for me.

```
root@zserver:/home/zulip/deployments/current# ./scripts/get-django-setting LDAP_DEACTIVATE_NON_MATCHING_USERS
True
```
Sync command 
`root@zserver:/home/zulip/deployments/current# su zulip -c './manage.py sync_ldap_user_data'`
ends with an error. I replaced the real username and domain for privacy reasons.
```
2022-04-06 17:11:01.106 DEBG [django_auth_ldap] search_s('OU=MyOU,DC=MyDC', 2, '(sAMAccountName=%(user)s)') returned 0 objects: 
2022-04-06 17:11:01.317 ERR  [zulip.sync_ldap_user_data] LDAP sync failed
Traceback (most recent call last):
  File "/home/zulip/deployments/2022-04-03-15-35-19/zproject/backends.py", line 1099, in sync_user_from_ldap
    ldap_username = backend.django_to_ldap_username(user_profile.delivery_email)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zproject/backends.py", line 668, in django_to_ldap_username
    raise ZulipLDAPExceptionNoMatchingLDAPUser(
zproject.backends.ZulipLDAPExceptionNoMatchingLDAPUser: No LDAP user matching django_to_ldap_username result: myusername. Input username: MyUsername@MyDomain.my

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zulip/deployments/2022-04-03-15-35-19/zerver/management/commands/sync_ldap_user_data.py", line 30, in sync_ldap_user_data
    sync_user_from_ldap(u, logger)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zproject/backends.py", line 1106, in sync_user_from_ldap
    do_deactivate_user(user_profile, acting_user=None)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zerver/lib/actions.py", line 1463, in do_deactivate_user
    delete_user_sessions(user_profile)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zerver/lib/sessions.py", line 47, in delete_user_sessions
    delete_session(session)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zerver/lib/sessions.py", line 41, in delete_session
    session_engine.SessionStore(session.session_key).delete()
  File "/home/zulip/deployments/2022-04-03-15-35-19/zerver/lib/safe_session_cached_db.py", line 24, in delete
    assert not get_connection().in_atomic_block
AssertionError
Traceback (most recent call last):
  File "/home/zulip/deployments/2022-04-03-15-35-19/zproject/backends.py", line 1099, in sync_user_from_ldap
    ldap_username = backend.django_to_ldap_username(user_profile.delivery_email)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zproject/backends.py", line 668, in django_to_ldap_username
    raise ZulipLDAPExceptionNoMatchingLDAPUser(
zproject.backends.ZulipLDAPExceptionNoMatchingLDAPUser: No LDAP user matching django_to_ldap_username result: myusername. Input username: MyUsername@MyDomain.my

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./manage.py", line 157, in <module>
    execute_from_command_line(sys.argv)
  File "./manage.py", line 122, in execute_from_command_line
    utility.execute()
  File "/srv/zulip-venv-cache/726c7d1d35d9cd61bb4b70e98167a1db7b0c524d/zulip-py3-venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/zulip-venv-cache/726c7d1d35d9cd61bb4b70e98167a1db7b0c524d/zulip-py3-venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/zulip-venv-cache/726c7d1d35d9cd61bb4b70e98167a1db7b0c524d/zulip-py3-venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zerver/management/commands/sync_ldap_user_data.py", line 80, in handle
    sync_ldap_user_data(user_profiles, not options["force"])
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zerver/management/commands/sync_ldap_user_data.py", line 30, in sync_ldap_user_data
    sync_user_from_ldap(u, logger)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zproject/backends.py", line 1106, in sync_user_from_ldap
    do_deactivate_user(user_profile, acting_user=None)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zerver/lib/actions.py", line 1463, in do_deactivate_user
    delete_user_sessions(user_profile)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zerver/lib/sessions.py", line 47, in delete_user_sessions
    delete_session(session)
  File "/home/zulip/deployments/2022-04-03-15-35-19/zerver/lib/sessions.py", line 41, in delete_session
    session_engine.SessionStore(session.session_key).delete()
  File "/home/zulip/deployments/2022-04-03-15-35-19/zerver/lib/safe_session_cached_db.py", line 24, in delete
    assert not get_connection().in_atomic_block
AssertionError

```
As a result, the user remains active.


Discussion:
Thanks for the report! I [posted this on chat.zulip.org](https://chat.zulip.org/#narrow/stream/31-production-help/topic/sync_ldap_user_data.20error.20.2321709/near/1361750) for more interactive discussion.
We're hitting this as well on our production instance.
Happened on 4.11 and now on 5.1 as well.

sync_ldap_user_data never completes, same trace back in ``safe_session_cached_db.py", line 24``

Happens regardless of ``LDAP_DEACTIVATE_NON_MATCHING_USERS`` true or false, both config settings trigger the same traceback.

Any ideas on a fix/workaround?

Zulip chat has a workaround, documenting here so it's discoverable (Zulip chats are not indexed by search engines seemingly)

Carrying this patch in our local branch for now and testing.


```quote
For a short-term fix, @Aggr, you can remove the @transaction.atomic line in zerver/management/commands/sync_ldap_user_data.py and re-run the import.
```
Hello @zulip/server-authentication members, this issue was labeled with the "area: authentication" label, so you may want to check it out!

<!-- areaLabelAddition -->


Resolving patch:
diff --git a/zerver/actions/users.py b/zerver/actions/users.py
--- a/zerver/actions/users.py
+++ b/zerver/actions/users.py
@@ -163,21 +163,30 @@ def do_deactivate_user(
         if settings.BILLING_ENABLED:
             update_license_ledger_if_needed(user_profile.realm, event_time)
 
-    delete_user_sessions(user_profile)
-    event = dict(
-        type="realm_user",
-        op="remove",
-        person=dict(user_id=user_profile.id, full_name=user_profile.full_name),
-    )
-    send_event(user_profile.realm, event, active_user_ids(user_profile.realm_id))
+        transaction.on_commit(lambda: delete_user_sessions(user_profile))
 
-    if user_profile.is_bot:
-        event = dict(
-            type="realm_bot",
+        event_remove_user = dict(
+            type="realm_user",
             op="remove",
-            bot=dict(user_id=user_profile.id, full_name=user_profile.full_name),
+            person=dict(user_id=user_profile.id, full_name=user_profile.full_name),
         )
-        send_event(user_profile.realm, event, bot_owner_user_ids(user_profile))
+        transaction.on_commit(
+            lambda: send_event(
+                user_profile.realm, event_remove_user, active_user_ids(user_profile.realm_id)
+            )
+        )
+
+        if user_profile.is_bot:
+            event_remove_bot = dict(
+                type="realm_bot",
+                op="remove",
+                bot=dict(user_id=user_profile.id, full_name=user_profile.full_name),
+            )
+            transaction.on_commit(
+                lambda: send_event(
+                    user_profile.realm, event_remove_bot, bot_owner_user_ids(user_profile)
+                )
+            )
 
 
 @transaction.atomic(durable=True)


Test patch:
diff --git a/zerver/tests/test_decorators.py b/zerver/tests/test_decorators.py
--- a/zerver/tests/test_decorators.py
+++ b/zerver/tests/test_decorators.py
@@ -1266,7 +1266,8 @@ def test_send_deactivated_user(self) -> None:
         """
         user_profile = self.example_user("hamlet")
         self.login_user(user_profile)
-        do_deactivate_user(user_profile, acting_user=None)
+        with self.captureOnCommitCallbacks(execute=True):
+            do_deactivate_user(user_profile, acting_user=None)
 
         result = self.client_post(
             "/json/messages",
diff --git a/zerver/tests/test_users.py b/zerver/tests/test_users.py
--- a/zerver/tests/test_users.py
+++ b/zerver/tests/test_users.py
@@ -1625,7 +1625,8 @@ def test_clear_sessions(self) -> None:
         self.assert_json_success(result)
         self.assertEqual(Session.objects.filter(pk=session_key).count(), 1)
 
-        do_deactivate_user(user, acting_user=None)
+        with self.captureOnCommitCallbacks(execute=True):
+            do_deactivate_user(user, acting_user=None)
         self.assertEqual(Session.objects.filter(pk=session_key).count(), 0)
 
         result = self.client_get("/json/users")