Repository: matrix-org/synapse

Issue:
Auto-join rooms rate limited
<!--

**THIS IS NOT A SUPPORT CHANNEL!**
**IF YOU HAVE SUPPORT QUESTIONS ABOUT RUNNING OR CONFIGURING YOUR OWN HOME SERVER**,
please ask in **#synapse:matrix.org** (using a matrix.org account if necessary)

If you want to report a security issue, please see https://matrix.org/security-disclosure-policy/

This is a bug report template. By following the instructions below and
filling out the sections with your information, you will help the us to get all
the necessary data to fix your issue.

You can also preview your report before submitting it. You may remove sections
that aren't relevant to your particular case.

Text between <!-- and --​> marks will be invisible in the report.

-->

### Description

Adding a new user to rooms defined in `auto_join_rooms` is incorrectly rate-limited.

### Steps to reproduce

- configure `homeserver.yaml` with `N>3` rooms listed in `auto_join_rooms`
- register a new user
- user is joined to first three rooms correctly, however all subsequent joins are incorrectly rate-limited:

```
synapse.handlers.register - 453 - ERROR - POST-119262 - Failed to join new user to '#<alias>:<domain>': LimitExceededError('429: Too Many Requests')
```

<!--
Describe how what happens differs from what you expected.

If you can identify any relevant log snippets from _homeserver.log_, please include
those (please be careful to remove any personal or private data). Please surround them with
``` (three backticks, on a line on their own), so that they are formatted legibly.
-->

### Version information



<!-- IMPORTANT: please answer the following questions, to help us narrow down the problem -->

<!-- Was this issue identified on matrix.org or another homeserver? -->

<!--
 What version of Synapse is running?

You can find the Synapse version with this command:

$ curl http://localhost:8008/_synapse/admin/v1/server_version

(You may need to replace `localhost:8008` if Synapse is not configured to
listen on that port.)
-->
- **Version**:
```javascript
{
    "python_version": "3.8.6",
    "server_version": "1.23.0"
}
```

- **Install method**: Docker `matrixdotorg/synapse:v1.23.0`
<!-- examples: package manager/git clone/pip  -->

- **Platform**: Host is Ubuntu 20.04.1 LTS
<!--
Tell us about the environment in which your homeserver is operating
distro, hardware, if it's running in a vm/container, etc.
-->

### Additional info

For this particular user, also setting their display name from our LDAP failed:
```
rest_auth_provider - 76 - INFO - POST-119262 - Registration based on REST data was successful for @<user>:<domain>
rest_auth_provider - 81 - INFO - POST-119262 - Handling profile data
rest_auth_provider - 92 - INFO - POST-119262 - Setting display name to '<displayname>' based on profile data
rest_auth_provider - 99 - INFO - POST-119262 - Handling 3PIDs
rest_auth_provider - 106 - INFO - POST-119262 - Looking for 3PID email:<user>@<domain> in user profile
twisted - 254 - WARNING - POST-119262 - /usr/local/lib/python3.8/site-packages/twisted/internet/defer.py:1418: builtins.RuntimeWarning: coroutine 'ProfileWorkerStore.set_profile_displayname' was never awaited
rest_auth_provider - 119 - INFO - POST-119262 - 3PID is present, skipping
twisted - 254 - WARNING -  - /usr/local/lib/python3.8/site-packages/twisted/internet/defer.py:1475: builtins.RuntimeWarning: coroutine 'RegistrationWorkerStore.get_user_id_by_threepid' was never awaited
```
I assume this is the same issue as the above messages occur less than one second after *Failed to join new user*, but otherwise let me know if you would like me to open a separate report.

### Related issues

I can see a few issues opened recently that seem similar: #8735,  #8791 and #8736.


Discussion:
Ah, this would be another case of the rate limit on joins not being overridden in the correct place. However, I notice that that particular codepath already has `ratelimit` set to `False`, so it looks like there's a bug somewhere:

https://github.com/matrix-org/synapse/blob/6fde6aa9c02d35e0a908437ea49b275df9b58427/synapse/handlers/register.py#L453-L462

For the display name issue, it looks like you're using https://github.com/ma1uta/matrix-synapse-rest-password-provider? I'm aware of that library needing some updates, although keep in mind that is a community repo. It looks like your particular issue is tracked at https://github.com/ma1uta/matrix-synapse-rest-password-provider/issues/6.

Resolving patch:
diff --git a/synapse/handlers/room.py b/synapse/handlers/room.py
--- a/synapse/handlers/room.py
+++ b/synapse/handlers/room.py
@@ -440,6 +440,7 @@ async def clone_existing_room(
             invite_list=[],
             initial_state=initial_state,
             creation_content=creation_content,
+            ratelimit=False,
         )
 
         # Transfer membership events
@@ -735,6 +736,7 @@ async def create_room(
             room_alias=room_alias,
             power_level_content_override=power_level_content_override,
             creator_join_profile=creator_join_profile,
+            ratelimit=ratelimit,
         )
 
         if "name" in config:
@@ -838,6 +840,7 @@ async def _send_events_for_new_room(
         room_alias: Optional[RoomAlias] = None,
         power_level_content_override: Optional[JsonDict] = None,
         creator_join_profile: Optional[JsonDict] = None,
+        ratelimit: bool = True,
     ) -> int:
         """Sends the initial events into a new room.
 
@@ -884,7 +887,7 @@ async def send(etype: str, content: JsonDict, **kwargs) -> int:
             creator.user,
             room_id,
             "join",
-            ratelimit=False,
+            ratelimit=ratelimit,
             content=creator_join_profile,
         )
 
diff --git a/synapse/handlers/room_member.py b/synapse/handlers/room_member.py
--- a/synapse/handlers/room_member.py
+++ b/synapse/handlers/room_member.py
@@ -203,7 +203,7 @@ async def _local_membership_update(
 
             # Only rate-limit if the user actually joined the room, otherwise we'll end
             # up blocking profile updates.
-            if newly_joined:
+            if newly_joined and ratelimit:
                 time_now_s = self.clock.time()
                 (
                     allowed,
@@ -488,17 +488,20 @@ async def update_membership_locked(
                     raise AuthError(403, "Guest access not allowed")
 
             if not is_host_in_room:
-                time_now_s = self.clock.time()
-                (
-                    allowed,
-                    time_allowed,
-                ) = self._join_rate_limiter_remote.can_requester_do_action(requester,)
-
-                if not allowed:
-                    raise LimitExceededError(
-                        retry_after_ms=int(1000 * (time_allowed - time_now_s))
+                if ratelimit:
+                    time_now_s = self.clock.time()
+                    (
+                        allowed,
+                        time_allowed,
+                    ) = self._join_rate_limiter_remote.can_requester_do_action(
+                        requester,
                     )
 
+                    if not allowed:
+                        raise LimitExceededError(
+                            retry_after_ms=int(1000 * (time_allowed - time_now_s))
+                        )
+
                 inviter = await self._get_inviter(target.to_string(), room_id)
                 if inviter and not self.hs.is_mine(inviter):
                     remote_room_hosts.append(inviter.domain)


Test patch:
diff --git a/tests/rest/client/v1/test_rooms.py b/tests/rest/client/v1/test_rooms.py
--- a/tests/rest/client/v1/test_rooms.py
+++ b/tests/rest/client/v1/test_rooms.py
@@ -26,6 +26,7 @@
 import synapse.rest.admin
 from synapse.api.constants import EventContentFields, EventTypes, Membership
 from synapse.handlers.pagination import PurgeStatus
+from synapse.rest import admin
 from synapse.rest.client.v1 import directory, login, profile, room
 from synapse.rest.client.v2_alpha import account
 from synapse.types import JsonDict, RoomAlias, UserID
@@ -625,6 +626,7 @@ class RoomJoinRatelimitTestCase(RoomBase):
     user_id = "@sid1:red"
 
     servlets = [
+        admin.register_servlets,
         profile.register_servlets,
         room.register_servlets,
     ]
@@ -703,6 +705,20 @@ def test_join_local_ratelimit_idempotent(self):
                 request, channel = self.make_request("POST", path % room_id, {})
                 self.assertEquals(channel.code, 200)
 
+    @unittest.override_config(
+        {
+            "rc_joins": {"local": {"per_second": 0.5, "burst_count": 3}},
+            "auto_join_rooms": ["#room:red", "#room2:red", "#room3:red", "#room4:red"],
+            "autocreate_auto_join_rooms": True,
+        },
+    )
+    def test_autojoin_rooms(self):
+        user_id = self.register_user("testuser", "password")
+
+        # Check that the new user successfully joined the four rooms
+        rooms = self.get_success(self.hs.get_datastore().get_rooms_for_user(user_id))
+        self.assertEqual(len(rooms), 4)
+
 
 class RoomMessagesTestCase(RoomBase):
     """ Tests /rooms/$room_id/messages/$user_id/$msg_id REST events. """