internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: matrix-org/synapse

Issue:
synapse does not escape forward slashes in key IDs when making a federation `key/v2/server/...` request
https://github.com/matrix-org/synapse/blob/d8cc86eff484b6f570f55a5badb337080c6e4dcd/synapse/crypto/keyring.py#L857-L874

urllib.parse.quote doesn't escape slashes by default: because it expects to given an entire url path

ouchie


Discussion:

Resolving patch:
diff --git a/synapse/crypto/keyring.py b/synapse/crypto/keyring.py
--- a/synapse/crypto/keyring.py
+++ b/synapse/crypto/keyring.py
@@ -857,7 +857,7 @@ async def get_server_verify_key_v2_direct(
                 response = await self.client.get_json(
                     destination=server_name,
                     path="/_matrix/key/v2/server/"
-                    + urllib.parse.quote(requested_key_id),
+                    + urllib.parse.quote(requested_key_id, safe=""),
                     ignore_backoff=True,
                     # we only give the remote server 10s to respond. It should be an
                     # easy request to handle, so if it doesn't reply within 10s, it's


Test patch:
diff --git a/tests/crypto/test_keyring.py b/tests/crypto/test_keyring.py
--- a/tests/crypto/test_keyring.py
+++ b/tests/crypto/test_keyring.py
@@ -469,6 +469,18 @@ async def get_json(destination, path, **kwargs):
         keys = self.get_success(fetcher.get_keys(SERVER_NAME, ["key1"], 0))
         self.assertEqual(keys, {})
 
+    def test_keyid_containing_forward_slash(self) -> None:
+        """We should url-encode any url unsafe chars in key ids.
+
+        Detects https://github.com/matrix-org/synapse/issues/14488.
+        """
+        fetcher = ServerKeyFetcher(self.hs)
+        self.get_success(fetcher.get_keys("example.com", ["key/potato"], 0))
+
+        self.http_client.get_json.assert_called_once()
+        args, kwargs = self.http_client.get_json.call_args
+        self.assertEqual(kwargs["path"], "/_matrix/key/v2/server/key%2Fpotato")
+
 
 class PerspectivesKeyFetcherTestCase(unittest.HomeserverTestCase):
     def make_homeserver(self, reactor, clock):
Repository: matrix-org/synapse

Issue:
Support Matrix v1.5
See https://matrix.org/blog/2022/11/17/matrix-v-1-5-release

## Client-Server API

* [x] Add `m.reference` relations, as per [MSC3267](https://github.com/matrix-org/matrix-spec-proposals/pull/3267).
* [x] ~~Add missing documentation for `m.key.verification.request` msgtype for in-room verification.~~

The other sections have no changes.



Discussion:
I think this just needs adding the numbers to the versions endpoint since Synapse already supports `m.reference` relations? Although IIRC the bundled format for `m.reference` got dropped, so we might want to update that at the same time?
> Although IIRC the bundled format for `m.reference` got dropped, so we might want to update that at the same time?

It didn't get dropped, see [here](https://spec.matrix.org/v1.5/client-server-api/#server-side-aggregation-of-mreference):

> The aggregation format of `m.reference` relations consists of a single `chunk` property, which lists all the events which `m.reference` the event (the parent). Currently, only a single `event_id` field is present on the events in the `chunk`.

The thing we do "wrong" is generate pagination tokens and return those.
#14508 ended up dropping the unspecced fields.

Resolving patch:
diff --git a/synapse/rest/client/versions.py b/synapse/rest/client/versions.py
--- a/synapse/rest/client/versions.py
+++ b/synapse/rest/client/versions.py
@@ -77,6 +77,7 @@ def on_GET(self, request: Request) -> Tuple[int, JsonDict]:
                     "v1.2",
                     "v1.3",
                     "v1.4",
+                    "v1.5",
                 ],
                 # as per MSC1497:
                 "unstable_features": {


Test patch:
Repository: matrix-org/synapse

Issue:
Stabilize set_displayname() in the module API
**Description:**
A stable method for setting a user's display name should be added to the Module API.
<!-- Describe here the feature you are requesting. -->



Discussion:

Resolving patch:
diff --git a/synapse/module_api/__init__.py b/synapse/module_api/__init__.py
--- a/synapse/module_api/__init__.py
+++ b/synapse/module_api/__init__.py
@@ -1585,6 +1585,33 @@ async def create_room(
 
         return room_id_and_alias["room_id"], room_id_and_alias.get("room_alias", None)
 
+    async def set_displayname(
+        self,
+        user_id: UserID,
+        new_displayname: str,
+        deactivation: bool = False,
+    ) -> None:
+        """Sets a user's display name.
+
+        Added in Synapse v1.76.0.
+
+        Args:
+            user_id:
+                The user whose display name is to be changed.
+            new_displayname:
+                The new display name to give the user.
+            deactivation:
+                Whether this change was made while deactivating the user.
+        """
+        requester = create_requester(user_id)
+        await self._hs.get_profile_handler().set_displayname(
+            target_user=user_id,
+            requester=requester,
+            new_displayname=new_displayname,
+            by_admin=True,
+            deactivation=deactivation,
+        )
+
 
 class PublicRoomListManager:
     """Contains methods for adding to, removing from and querying whether a room


Test patch:
diff --git a/tests/module_api/test_api.py b/tests/module_api/test_api.py
--- a/tests/module_api/test_api.py
+++ b/tests/module_api/test_api.py
@@ -110,6 +110,24 @@ def test_can_set_admin(self):
         self.assertEqual(found_user.user_id.to_string(), user_id)
         self.assertIdentical(found_user.is_admin, True)
 
+    def test_can_set_displayname(self):
+        localpart = "alice_wants_a_new_displayname"
+        user_id = self.register_user(
+            localpart, "1234", displayname="Alice", admin=False
+        )
+        found_userinfo = self.get_success(self.module_api.get_userinfo_by_id(user_id))
+
+        self.get_success(
+            self.module_api.set_displayname(
+                found_userinfo.user_id, "Bob", deactivation=False
+            )
+        )
+        found_profile = self.get_success(
+            self.module_api.get_profile_for_user(localpart)
+        )
+
+        self.assertEqual(found_profile.display_name, "Bob")
+
     def test_get_userinfo_by_id(self):
         user_id = self.register_user("alice", "1234")
         found_user = self.get_success(self.module_api.get_userinfo_by_id(user_id))
Repository: matrix-org/synapse

Issue:
synapse_port_db misbehaves unintuitively if you pass the wrong path to the sqlite.db
and tries to migrate an empty DB


Discussion:
If anyone wants to take this on:

- A good first step would be to use `os.path.isfile` just before [here](https://github.com/matrix-org/synapse/blob/4db7862e0fce1a7b1a84458d572ba006b032c737/synapse/_scripts/synapse_port_db.py#L1302-L1310) to check that `args.sqlite_database` is a file.
- A more robust approach would be to follow the [advice here](https://stackoverflow.com/questions/12932607/how-to-check-if-a-sqlite3-database-exists-in-python) and try to open the database in read-only mode (`r`) rather than read-write-create (`rwc`). (AFAICS the script only ever reads from the sqlite db and writes to the postgres DB.

Resolving patch:
diff --git a/synapse/_scripts/synapse_port_db.py b/synapse/_scripts/synapse_port_db.py
--- a/synapse/_scripts/synapse_port_db.py
+++ b/synapse/_scripts/synapse_port_db.py
@@ -1307,7 +1307,7 @@ def main() -> None:
     sqlite_config = {
         "name": "sqlite3",
         "args": {
-            "database": args.sqlite_database,
+            "database": "file:{}?mode=rw".format(args.sqlite_database),
             "cp_min": 1,
             "cp_max": 1,
             "check_same_thread": False,


Test patch:
Repository: matrix-org/synapse

Issue:
synapse returns undocumented `device` property from `GET /_matrix/client/v3/pushrules/`
Synapse's response to a `/pushrules` query looks something like:

```json5
[
  "global": {
    "content": { /*... */ },
    "override": { /*... */ },
    "room": { /*... */ },
    "sender": { /*... */ },
    "underride": { /*... */ }
  },
  "device": {}
]
```

This `device` property is not [specced](https://spec.matrix.org/v1.2/client-server-api/#get_matrixclientv3pushrules) and should be removed.





Discussion:

Resolving patch:
diff --git a/synapse/push/clientformat.py b/synapse/push/clientformat.py
--- a/synapse/push/clientformat.py
+++ b/synapse/push/clientformat.py
@@ -26,10 +26,7 @@ def format_push_rules_for_user(
     """Converts a list of rawrules and a enabled map into nested dictionaries
     to match the Matrix client-server format for push rules"""
 
-    rules: Dict[str, Dict[str, List[Dict[str, Any]]]] = {
-        "global": {},
-        "device": {},
-    }
+    rules: Dict[str, Dict[str, List[Dict[str, Any]]]] = {"global": {}}
 
     rules["global"] = _add_empty_priority_class_arrays(rules["global"])
 


Test patch:
Repository: matrix-org/synapse

Issue:
Add `Synapse-Trace-Id` to `access-control-allow-headers` header
### Description

The `Synapse-Trace-Id` response header added in https://github.com/matrix-org/synapse/issues/9480 needs to be [whitelisted](https://github.com/matrix-org/synapse/blob/v1.75.0/synapse/http/server.py#L903) as well so web clients using CORS can read it.

### Steps to reproduce

one a jaeger enabled synapse, make a request that is traced by jaeger.

### Homeserver

matrix.org

### Synapse Version

1.76.0rc1

### Installation Method

I don't know

### Database

don't know

### Workers

I don't know

### Platform

don't know

### Configuration

_No response_

### Relevant log output

```shell
don't have any
```


### Anything else that would be useful to know?

_No response_


Discussion:

Resolving patch:
diff --git a/synapse/http/server.py b/synapse/http/server.py
--- a/synapse/http/server.py
+++ b/synapse/http/server.py
@@ -892,6 +892,10 @@ def set_cors_headers(request: SynapseRequest) -> None:
             b"Access-Control-Allow-Headers",
             b"X-Requested-With, Content-Type, Authorization, Date",
         )
+        request.setHeader(
+            b"Access-Control-Expose-Headers",
+            b"Synapse-Trace-Id",
+        )
 
 
 def set_corp_headers(request: Request) -> None:


Test patch:
diff --git a/tests/test_server.py b/tests/test_server.py
--- a/tests/test_server.py
+++ b/tests/test_server.py
@@ -266,6 +266,10 @@ def _check_cors_standard_headers(self, channel: FakeChannel) -> None:
             [b"X-Requested-With, Content-Type, Authorization, Date"],
             "has correct CORS Headers header",
         )
+        self.assertEqual(
+            channel.headers.getRawHeaders(b"Access-Control-Expose-Headers"),
+            [b"Synapse-Trace-Id"],
+        )
 
     def _check_cors_msc3886_headers(self, channel: FakeChannel) -> None:
         # Ensure the correct CORS headers have been added
Repository: matrix-org/synapse

Issue:
Apply limiter patch from Gitter import
While importing historical Gitter archives into Gitter's homeserver we added a patch to improve performance:

```diff
diff --git a/synapse/handlers/message.py b/synapse/handlers/message.py
--- a/synapse/handlers/message.py
+++ b/synapse/handlers/message.py
@@ -502,7 +502,7 @@

         # We arbitrarily limit concurrent event creation for a room to 5.
         # This is to stop us from diverging history *too* much.
-        self.limiter = Linearizer(max_count=5, name="room_event_creation_limit")
+        self.limiter = Linearizer(max_count=1, name="room_event_creation_limit")

         self._bulk_push_rule_evaluator = hs.get_bulk_push_rule_evaluator()
```

----

Digging some context out of internal issues:

> The idea is to reduce the concurrency of event sends into rooms from 5 -> 1 in order to eliminate state resolution from occurring when sending bursts of events to a local room: [matrix-org/synapse@`54c012c`/synapse/handlers/message.py#L502-L504](https://github.com/matrix-org/synapse/blob/54c012c5a8722725cf104fa6205f253b5b9b0192/synapse/handlers/message.py#L502-L504)

For cross-linking, there was an old issue about doing this in #8612.

----

Unfortunately this changes was applied at the same time as a few other changes (disabling push and increasing the number of workers), so it is unclear how much impact this change had.


Discussion:

Resolving patch:
diff --git a/synapse/handlers/message.py b/synapse/handlers/message.py
--- a/synapse/handlers/message.py
+++ b/synapse/handlers/message.py
@@ -499,9 +499,9 @@ def __init__(self, hs: "HomeServer"):
 
         self.request_ratelimiter = hs.get_request_ratelimiter()
 
-        # We arbitrarily limit concurrent event creation for a room to 5.
-        # This is to stop us from diverging history *too* much.
-        self.limiter = Linearizer(max_count=5, name="room_event_creation_limit")
+        # We limit concurrent event creation for a room to 1. This prevents state resolution
+        # from occurring when sending bursts of events to a local room
+        self.limiter = Linearizer(max_count=1, name="room_event_creation_limit")
 
         self._bulk_push_rule_evaluator = hs.get_bulk_push_rule_evaluator()
 


Test patch:
Repository: matrix-org/synapse

Issue:
Remove type parameter from /thumbnail endpoint
This is unspecced and per matrix-org/matrix-spec#1414 we should use the `Accept` header instead.


Discussion:
There's not a single request to matrix.org using this parameter. I think we can just rip it out.

Resolving patch:
diff --git a/synapse/rest/media/v1/thumbnail_resource.py b/synapse/rest/media/v1/thumbnail_resource.py
--- a/synapse/rest/media/v1/thumbnail_resource.py
+++ b/synapse/rest/media/v1/thumbnail_resource.py
@@ -69,7 +69,8 @@ async def _async_render_GET(self, request: SynapseRequest) -> None:
         width = parse_integer(request, "width", required=True)
         height = parse_integer(request, "height", required=True)
         method = parse_string(request, "method", "scale")
-        m_type = parse_string(request, "type", "image/png")
+        # TODO Parse the Accept header to get an prioritised list of thumbnail types.
+        m_type = "image/png"
 
         if server_name == self.server_name:
             if self.dynamic_thumbnails:


Test patch:
Repository: matrix-org/synapse

Issue:
Could not unpickle M_UNRECOGNIZED: MatrixError: [405] Unrecognized request (.../org.matrix.msc2697.v2/dehydrated_device/claim)
### Description

Since upgraded to Synapse v1.78.0 I am getting the below issue: 

```
POST /_matrix/client/unstable/org.matrix.msc2697.v2/dehydrated_device/claim 405
fetch	@	fetch.ts?76d0:62
requestOtherUrl	@	fetch.ts?76d0:264
request	@	fetch.ts?76d0:205
authedRequest	@	fetch.ts?76d0:156
rehydrateDevice	@	client.ts?5faf:1571
```


```
could not unpickle M_UNRECOGNIZED: MatrixError: [405] Unrecognized request (/_matrix/client/unstable/org.matrix.msc2697.v2/dehydrated_device/claim)
    at parseErrorResponse (webpack-internal:///120:83:12)
    at MatrixHttpApi.requestOtherUrl (webpack-internal:///116:248:
```

This might be a regression again, see ...

https://github.com/matrix-org/synapse/issues/14334
https://github.com/matrix-org/matrix-js-sdk/issues/2809

### Steps to reproduce

See above.

### Homeserver

.

### Synapse Version

.

### Installation Method

pip (from PyPI)

### Database

Both PostgreSQL & SQLITE

### Workers

I don't know

### Platform

.

### Configuration

.

### Relevant log output

```shell
.
```


### Anything else that would be useful to know?

.


Discussion:
I think this is due to #14605.

Resolving patch:
diff --git a/synapse/rest/client/devices.py b/synapse/rest/client/devices.py
--- a/synapse/rest/client/devices.py
+++ b/synapse/rest/client/devices.py
@@ -255,7 +255,7 @@ class DehydratedDeviceServlet(RestServlet):
 
     """
 
-    PATTERNS = client_patterns("/org.matrix.msc2697.v2/dehydrated_device", releases=())
+    PATTERNS = client_patterns("/org.matrix.msc2697.v2/dehydrated_device$", releases=())
 
     def __init__(self, hs: "HomeServer"):
         super().__init__()


Test patch:
Repository: matrix-org/synapse

Issue:
Failed to find memberships for ["@user:...."] in partial state room !... in the auth events of ["$..."].
https://sentry.tools.element.io/organizations/element/issues/63004/?project=2&query=is%3Aunresolved

When pushing events in partial state rooms down incremental /sync, we try to find the `m.room.member` state event for their sender by digging through their auth events, so that we can present the membership to the client. Events usually have a membership event in their auth events, with the exception of the `m.room.create` event and a user's first join into the room.

When implementing https://github.com/matrix-org/synapse/issues/13477, we forgot to consider the exceptions.


Discussion:
What's the impact of this---does this block incremental syncs?
Nope, it's just an error-level log message.

Resolving patch:
diff --git a/synapse/handlers/sync.py b/synapse/handlers/sync.py
--- a/synapse/handlers/sync.py
+++ b/synapse/handlers/sync.py
@@ -1226,6 +1226,10 @@ async def _find_missing_partial_state_memberships(
                 continue
 
             event_with_membership_auth = events_with_membership_auth[member]
+            is_create = (
+                event_with_membership_auth.is_state()
+                and event_with_membership_auth.type == EventTypes.Create
+            )
             is_join = (
                 event_with_membership_auth.is_state()
                 and event_with_membership_auth.type == EventTypes.Member
@@ -1233,9 +1237,10 @@ async def _find_missing_partial_state_memberships(
                 and event_with_membership_auth.content.get("membership")
                 == Membership.JOIN
             )
-            if not is_join:
+            if not is_create and not is_join:
                 # The event must include the desired membership as an auth event, unless
-                # it's the first join event for a given user.
+                # it's the `m.room.create` event for a room or the first join event for
+                # a given user.
                 missing_members.add(member)
             auth_event_ids.update(event_with_membership_auth.auth_event_ids())
 


Test patch:
Repository: matrix-org/synapse

Issue:
Allow endpoint:  _matrix/client/<>/register/available on workers 
This endpoint is not working with worker configuration. , it has to be routed through the main process . can we enable it please, on worker configuration as well 



Discussion:
Are you sure about this? (Which Synapse version are you running?)

The [worker documentation](https://matrix-org.github.io/synapse/latest/workers.html) lists
`^/_matrix/client/(r0|v3|unstable)/register$`  as something that can be handled by a generic worker.
It is only available on main process `UsernameAvailabilityRestServlet`:
https://github.com/matrix-org/synapse/blob/ce54477f6fa0264ef00b15bc3e0c2503d85ab061/synapse/rest/client/register.py#L956-L959
@DMRobertson  yes  i am sure , v1.79.0rc1 , i tried with this version

Resolving patch:
diff --git a/docker/configure_workers_and_start.py b/docker/configure_workers_and_start.py
--- a/docker/configure_workers_and_start.py
+++ b/docker/configure_workers_and_start.py
@@ -163,6 +163,7 @@
             "^/_matrix/client/versions$",
             "^/_matrix/client/(api/v1|r0|v3|unstable)/voip/turnServer$",
             "^/_matrix/client/(r0|v3|unstable)/register$",
+            "^/_matrix/client/(r0|v3|unstable)/register/available$",
             "^/_matrix/client/(r0|v3|unstable)/auth/.*/fallback/web$",
             "^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/messages$",
             "^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/event",
diff --git a/synapse/rest/client/register.py b/synapse/rest/client/register.py
--- a/synapse/rest/client/register.py
+++ b/synapse/rest/client/register.py
@@ -956,7 +956,7 @@ def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
     if hs.config.worker.worker_app is None:
         EmailRegisterRequestTokenRestServlet(hs).register(http_server)
         MsisdnRegisterRequestTokenRestServlet(hs).register(http_server)
-        UsernameAvailabilityRestServlet(hs).register(http_server)
         RegistrationSubmitTokenServlet(hs).register(http_server)
+    UsernameAvailabilityRestServlet(hs).register(http_server)
     RegistrationTokenValidityRestServlet(hs).register(http_server)
     RegisterRestServlet(hs).register(http_server)


Test patch:
Repository: matrix-org/synapse

Issue:
synapse_port_db misbehaves unintuitively if you pass the wrong path to the sqlite.db
and tries to migrate an empty DB
synapse_port_db misbehaves unintuitively if you pass the wrong path to the sqlite.db
and tries to migrate an empty DB


Discussion:
If anyone wants to take this on:

- A good first step would be to use `os.path.isfile` just before [here](https://github.com/matrix-org/synapse/blob/4db7862e0fce1a7b1a84458d572ba006b032c737/synapse/_scripts/synapse_port_db.py#L1302-L1310) to check that `args.sqlite_database` is a file.
- A more robust approach would be to follow the [advice here](https://stackoverflow.com/questions/12932607/how-to-check-if-a-sqlite3-database-exists-in-python) and try to open the database in read-only mode (`r`) rather than read-write-create (`rwc`). (AFAICS the script only ever reads from the sqlite db and writes to the postgres DB.
#14692 was backed out in #15301.
If anyone wants to take this on:

- A good first step would be to use `os.path.isfile` just before [here](https://github.com/matrix-org/synapse/blob/4db7862e0fce1a7b1a84458d572ba006b032c737/synapse/_scripts/synapse_port_db.py#L1302-L1310) to check that `args.sqlite_database` is a file.
- A more robust approach would be to follow the [advice here](https://stackoverflow.com/questions/12932607/how-to-check-if-a-sqlite3-database-exists-in-python) and try to open the database in read-only mode (`r`) rather than read-write-create (`rwc`). (AFAICS the script only ever reads from the sqlite db and writes to the postgres DB.
#14692 was backed out in #15301.

Resolving patch:
diff --git a/synapse/_scripts/synapse_port_db.py b/synapse/_scripts/synapse_port_db.py
--- a/synapse/_scripts/synapse_port_db.py
+++ b/synapse/_scripts/synapse_port_db.py
@@ -18,6 +18,7 @@
 import argparse
 import curses
 import logging
+import os
 import sys
 import time
 import traceback
@@ -1326,6 +1327,13 @@ def main() -> None:
         filename="port-synapse.log" if args.curses else None,
     )
 
+    if not os.path.isfile(args.sqlite_database):
+        sys.stderr.write(
+            "The sqlite database you specified does not exist, please check that you have the"
+            "correct path."
+        )
+        sys.exit(1)
+
     sqlite_config = {
         "name": "sqlite3",
         "args": {


Test patch:
Repository: matrix-org/synapse

Issue:
`POST /keys/upload/` responses include an internal `_INT_STREAM_POS` dict
Looks like we inadvertently include this replication machinery related dict in the response, possibly after https://github.com/matrix-org/synapse/pull/15024? Not harmful, but should be removed.


Discussion:
> possibly after #15024?

My money's on https://github.com/matrix-org/synapse/commit/9187fd940e2b2bbfd4df7204053cc26b2707aad4#diff-7f96e43d79073f987d96491251fe0af2754414c87be73eeec7bcd705c20603cfR346-R358 from https://github.com/matrix-org/synapse/pull/14820


Resolving patch:
diff --git a/synapse/replication/http/_base.py b/synapse/replication/http/_base.py
--- a/synapse/replication/http/_base.py
+++ b/synapse/replication/http/_base.py
@@ -345,7 +345,7 @@ async def send_request(*, instance_name: str = "master", **kwargs: Any) -> Any:
                 _outgoing_request_counter.labels(cls.NAME, 200).inc()
 
                 # Wait on any streams that the remote may have written to.
-                for stream_name, position in result.get(
+                for stream_name, position in result.pop(
                     _STREAM_POSITION_KEY, {}
                 ).items():
                     await replication.wait_for_stream_position(


Test patch: