Repository: zulip/zulip
Issue: `delete_realm` deletes users before flushing, causing `delete_user_profile_caches` to do nothing

`manage.py delete_realm` calls `realm.delete()`, which deletes all of the UserProfile objects.  As a post_delete handler, we call `flush_realm`:
https://github.com/zulip/zulip/blob/6a3e98d14bab04fc06ce6b9c9ccfbd69b6030a83/zerver/models.py#L852-L859

But by then, there are no UserProfile objects, which means that `delete_user_profile_caches` doesn't find get users to delete the caches of:
https://github.com/zulip/zulip/blob/fae92f2e3f6e59aac4d5a2edb4657db00194bc11/zerver/lib/cache.py#L667-L668

We should make `manage.py delete_realm` call first loop through users in the realm and delete those using `do_delete_user` or something.

Changed files:
zerver/models.py, zerver/tests/test_realm.py

diff --git a/zerver/models.py b/zerver/models.py
index a078d5cd51..93d22c97ac 100644
--- a/zerver/models.py
+++ b/zerver/models.py
@@ -29,7 +29,7 @@ from django.core.validators import MinLengthValidator, RegexValidator, URLValida
 from django.db import models, transaction
 from django.db.models import CASCADE, Manager, Q, Sum
 from django.db.models.query import QuerySet
-from django.db.models.signals import post_delete, post_save
+from django.db.models.signals import post_delete, post_save, pre_delete
 from django.utils.functional import Promise
 from django.utils.timezone import now as timezone_now
 from django.utils.translation import gettext as _
@@ -849,14 +849,27 @@ class Realm(models.Model):
         return self.is_zephyr_mirror_realm
 
 
-def realm_post_delete_handler(*, instance: Realm, **kwargs: object) -> None:
+post_save.connect(flush_realm, sender=Realm)
+
+
+# We register realm cache flushing in a duplicate way to be run both
+# pre_delete and post_delete on purpose:
+# 1. pre_delete is needed because flush_realm wants to flush the UserProfile caches,
+#    and UserProfile objects are deleted via on_delete=CASCADE before the post_delete handler
+#    is called, which results in the `flush_realm` logic not having access to the details
+#    for the deleted users if called at that time.
+# 2. post_delete is run as a precaution to reduce the risk of races where items might be
+#    added to the cache after the pre_delete handler but before the save.
+#    Note that it does not eliminate this risk, not least because it only flushes
+#    the realm cache, and not the user caches, for the reasons explained above.
+def realm_pre_and_post_delete_handler(*, instance: Realm, **kwargs: object) -> None:
     # This would be better as a functools.partial, but for some reason
     # Django doesn't call it even when it's registered as a post_delete handler.
     flush_realm(instance=instance, from_deletion=True)
 
 
-post_save.connect(flush_realm, sender=Realm)
-post_delete.connect(realm_post_delete_handler, sender=Realm)
+pre_delete.connect(realm_pre_and_post_delete_handler, sender=Realm)
+post_delete.connect(realm_pre_and_post_delete_handler, sender=Realm)
 
 
 def get_realm(string_id: str) -> Realm:
diff --git a/zerver/tests/test_realm.py b/zerver/tests/test_realm.py
index b6c384564e..4bda7f1927 100644
--- a/zerver/tests/test_realm.py
+++ b/zerver/tests/test_realm.py
@@ -186,6 +186,14 @@ class RealmTest(ZulipTestCase):
         user = get_user_profile_by_id(hamlet_id)
         self.assertTrue(user.realm.deactivated)
 
+    def test_do_change_realm_delete_clears_user_realm_cache(self) -> None:
+        hamlet_id = self.example_user("hamlet").id
+        get_user_profile_by_id(hamlet_id)
+        realm = get_realm("zulip")
+        realm.delete()
+        with self.assertRaises(UserProfile.DoesNotExist):
+            get_user_profile_by_id(hamlet_id)
+
     def test_do_change_realm_subdomain_clears_user_realm_cache(self) -> None:
         """The main complicated thing about changing realm subdomains is
         updating the cache, and we start by populating the cache for