internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: matrix-org/synapse

Issue:
SSO rest API for redirect is loaded if no SSO is configured
### Description

Rest api is loaded and can be called if no SSO is configured.

API returns an error "Homeserver not configured for SSO." But I can not see why the server needs to be run this api.

### Steps to reproduce

- setup a new homeserver
- request /_matrix/client/r0/login/sso/redirect?redirectUrl=www.example.org

### Homeserver

another

### Synapse Version

1.78.0

### Installation Method

Docker (matrixdotorg/synapse)

### Database

PostgreSQL

### Workers

Single process

### Platform

K8s

### Configuration

only password login

### Relevant log output

```shell
None
```


### Anything else that would be useful to know?

Probably there could be a check here for SSO capabilities:

https://github.com/matrix-org/synapse/blob/98fd558382226b347d78e5f6e6782c6e74c25e69/synapse/rest/client/login.py#L675

Similar to:
https://github.com/matrix-org/synapse/blob/98fd558382226b347d78e5f6e6782c6e74c25e69/synapse/rest/client/login.py#L148




Discussion:
I'm in two minds about this one. Firstly, I think you have a point that its unnecessary, however I think it is useful for debugging purposes to see that the URL is correct but SSO has been disabled. 
IMO this kind of debugging or validating a configuration is not the right one for productive systems. 
There is a feature flag. This flag is for disabling the feature and not only parts of it.
Loading the api also needs resources. Even if it is probably insignificant.

Resolving patch:
diff --git a/synapse/rest/client/login.py b/synapse/rest/client/login.py
--- a/synapse/rest/client/login.py
+++ b/synapse/rest/client/login.py
@@ -672,7 +672,12 @@ def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
     LoginRestServlet(hs).register(http_server)
     if hs.config.registration.refreshable_access_token_lifetime is not None:
         RefreshTokenServlet(hs).register(http_server)
-    SsoRedirectServlet(hs).register(http_server)
+    if (
+        hs.config.cas.cas_enabled
+        or hs.config.saml2.saml2_enabled
+        or hs.config.oidc.oidc_enabled
+    ):
+        SsoRedirectServlet(hs).register(http_server)
     if hs.config.cas.cas_enabled:
         CasTicketServlet(hs).register(http_server)
 


Test patch:
Repository: matrix-org/synapse

Issue:
Complement Dockerfiles should use fully-qualified image names
I have switched to podman for rootless operation, given recent security auditing stuff, and I notice that podman prompts you to choose where to get images from:

```
[2/3] STEP 1/1: FROM redis:6-bullseye AS redis_base
? Please select an image: 
  ▸ docker.io/library/redis:6-bullseye
    quay.io/redis:6-bullseye
```

It seems like we ought to just use the fully-qualified image names anyway, as far as I know that is considered good practice given that people can have different registries configured as default?



Discussion:

Resolving patch:
diff --git a/scripts-dev/build_debian_packages.py b/scripts-dev/build_debian_packages.py
--- a/scripts-dev/build_debian_packages.py
+++ b/scripts-dev/build_debian_packages.py
@@ -20,6 +20,8 @@
 from types import FrameType
 from typing import Collection, Optional, Sequence, Set
 
+# These are expanded inside the dockerfile to be a fully qualified image name.
+# e.g. docker.io/library/debian:bullseye
 DISTS = (
     "debian:buster",  # oldstable: EOL 2022-08
     "debian:bullseye",


Test patch:
Repository: matrix-org/synapse

Issue:
Build packages for Debian Trixie
Please can we publish packages to the apt repository for [Debian Trixie (13)](https://wiki.debian.org/DebianTrixie) which is the current testing release at the time of writing. It became the current testing release on 2023-06-10.

I run debian testing on the server I run synapse on and the change from bookworm to trixie has meant that I now get errors on `apt update`:

```
E: The repository 'https://packages.matrix.org/debian trixie Release' does not have a Release file.
```


Discussion:
Yup, we should add it to the list in https://github.com/matrix-org/pkg-repo-configs
See #15362 for what needs to be done here.

Resolving patch:
diff --git a/scripts-dev/build_debian_packages.py b/scripts-dev/build_debian_packages.py
--- a/scripts-dev/build_debian_packages.py
+++ b/scripts-dev/build_debian_packages.py
@@ -34,6 +34,7 @@
     "ubuntu:jammy",  # 22.04 LTS (EOL 2027-04) (our EOL forced by Python 3.10 is 2026-10-04)
     "ubuntu:kinetic",  # 22.10 (EOL 2023-07-20) (our EOL forced by Python 3.10 is 2026-10-04)
     "ubuntu:lunar",  # 23.04 (EOL 2024-01) (our EOL forced by Python 3.11 is 2027-10-24)
+    "debian:trixie",  # (EOL not specified yet)
 )
 
 DESC = """\


Test patch:
Repository: matrix-org/synapse

Issue:
`/_matrix/client/r0/profile` returns 404 if only displayname is not set
### Description

If a user has been created without a displayname, requests are sent to `/_matrix/client/r0/profile` return a 404 response.

### Steps to reproduce

- Create a user without displayname but an avatar
- Try to get the profile for the user

Expected: got a response with empty displayname
What happens: 404


### Homeserver

matrix.jamesdemo.ems-support-self-serve.element.io

### Synapse Version

1.84.0

### Installation Method

I don't know

### Database

I don't know

### Workers

I don't know

### Platform

EMS

### Configuration

_No response_

### Relevant log output

```shell
-
```


### Anything else that would be useful to know?

Link to source https://github.com/matrix-org/synapse/blob/99b7b801c31b9428f9503ad6f83f11804fef048a/synapse/handlers/profile.py#L71


Discussion:
This almost matches the spec as I read it https://spec.matrix.org/v1.7/client-server-api/#get_matrixclientv3profileuserid

`404 There is no profile information for this user or this user does not exist.`

Not having a name or avatar set would be "no profile information" - Synapse only checking displayname is wrong though

I did some quick testing and there *are* users with a null displayname and a configured avatar, so yeah this seems wrong.

Resolving patch:
diff --git a/synapse/handlers/profile.py b/synapse/handlers/profile.py
--- a/synapse/handlers/profile.py
+++ b/synapse/handlers/profile.py
@@ -68,7 +68,7 @@ async def get_profile(self, user_id: str, ignore_backoff: bool = True) -> JsonDi
 
         if self.hs.is_mine(target_user):
             profileinfo = await self.store.get_profileinfo(target_user)
-            if profileinfo.display_name is None:
+            if profileinfo.display_name is None and profileinfo.avatar_url is None:
                 raise SynapseError(404, "Profile was not found", Codes.NOT_FOUND)
 
             return {


Test patch:
diff --git a/tests/handlers/test_profile.py b/tests/handlers/test_profile.py
--- a/tests/handlers/test_profile.py
+++ b/tests/handlers/test_profile.py
@@ -179,6 +179,16 @@ def test_get_my_avatar(self) -> None:
 
         self.assertEqual("http://my.server/me.png", avatar_url)
 
+    def test_get_profile_empty_displayname(self) -> None:
+        self.get_success(self.store.set_profile_displayname(self.frank, None))
+        self.get_success(
+            self.store.set_profile_avatar_url(self.frank, "http://my.server/me.png")
+        )
+
+        profile = self.get_success(self.handler.get_profile(self.frank.to_string()))
+
+        self.assertEqual("http://my.server/me.png", profile["avatar_url"])
+
     def test_set_my_avatar(self) -> None:
         self.get_success(
             self.handler.set_avatar_url(
Repository: matrix-org/synapse

Issue:
Remove backwards compat hack from `EventContext` serialzation code 
https://github.com/matrix-org/synapse/pull/15233 changed the format of `EventContext` and as such required two small hacks to make the serialization backwards compatible:

https://github.com/matrix-org/synapse/blob/553f2f53e7930d721d2070ffa45af6397f5ecb60/synapse/events/snapshot.py#L206-L212

https://github.com/matrix-org/synapse/blob/553f2f53e7930d721d2070ffa45af6397f5ecb60/synapse/events/snapshot.py#L189-L191

We should remove these after a few (let's say 4?) releases. The second one (lines 189-191) can just be removed, and the first one just requires a change:
```
state_group_deltas = _decode_state_group_delta(input["state_group_deltas"])
```
and the removal of the `if` statement. 


Discussion:

Resolving patch:
diff --git a/synapse/events/snapshot.py b/synapse/events/snapshot.py
--- a/synapse/events/snapshot.py
+++ b/synapse/events/snapshot.py
@@ -186,9 +186,6 @@ async def serialize(self, event: EventBase, store: "DataStore") -> JsonDict:
             ),
             "app_service_id": self.app_service.id if self.app_service else None,
             "partial_state": self.partial_state,
-            # add dummy delta_ids and prev_group for backwards compatibility
-            "delta_ids": None,
-            "prev_group": None,
         }
 
     @staticmethod
@@ -203,13 +200,6 @@ def deserialize(storage: "StorageControllers", input: JsonDict) -> "EventContext
         Returns:
             The event context.
         """
-        # workaround for backwards/forwards compatibility: if the input doesn't have a value
-        # for "state_group_deltas" just assign an empty dict
-        state_group_deltas = input.get("state_group_deltas", None)
-        if state_group_deltas:
-            state_group_deltas = _decode_state_group_delta(state_group_deltas)
-        else:
-            state_group_deltas = {}
 
         context = EventContext(
             # We use the state_group and prev_state_id stuff to pull the
@@ -217,7 +207,7 @@ def deserialize(storage: "StorageControllers", input: JsonDict) -> "EventContext
             storage=storage,
             state_group=input["state_group"],
             state_group_before_event=input["state_group_before_event"],
-            state_group_deltas=state_group_deltas,
+            state_group_deltas=_decode_state_group_delta(input["state_group_deltas"]),
             state_delta_due_to_event=_decode_state_dict(
                 input["state_delta_due_to_event"]
             ),


Test patch:
Repository: matrix-org/synapse

Issue:
Ensure we don't time out background index creation
We added a global timeout in #15853, which might break such (expected) long running queries.


Discussion:
As far as I know, this should be overridable at the transaction level with `SET LOCAL statement_timeout`. Shouldn't be a difficult fix but just needs someone to research that this actually works and add the exact incantation.

Resolving patch:
diff --git a/synapse/storage/background_updates.py b/synapse/storage/background_updates.py
--- a/synapse/storage/background_updates.py
+++ b/synapse/storage/background_updates.py
@@ -238,6 +238,7 @@ class BackgroundUpdater:
     def __init__(self, hs: "HomeServer", database: "DatabasePool"):
         self._clock = hs.get_clock()
         self.db_pool = database
+        self.hs = hs
 
         self._database_name = database.name()
 
@@ -758,6 +759,11 @@ def create_index_psql(conn: Connection) -> None:
                 logger.debug("[SQL] %s", sql)
                 c.execute(sql)
 
+                # override the global statement timeout to avoid accidentally squashing
+                # a long-running index creation process
+                timeout_sql = "SET SESSION statement_timeout = 0"
+                c.execute(timeout_sql)
+
                 sql = (
                     "CREATE %(unique)s INDEX CONCURRENTLY %(name)s"
                     " ON %(table)s"
@@ -778,6 +784,12 @@ def create_index_psql(conn: Connection) -> None:
                     logger.debug("[SQL] %s", sql)
                     c.execute(sql)
             finally:
+                # mypy ignore - `statement_timeout` is defined on PostgresEngine
+                # reset the global timeout to the default
+                default_timeout = self.db_pool.engine.statement_timeout  # type: ignore[attr-defined]
+                undo_timeout_sql = f"SET statement_timeout = {default_timeout}"
+                conn.cursor().execute(undo_timeout_sql)
+
                 conn.set_session(autocommit=False)  # type: ignore
 
         def create_index_sqlite(conn: Connection) -> None:


Test patch:
Repository: matrix-org/synapse

Issue:
Clean-up event_txn_id table
1. In a future version we need to stop reading & writing to the `event_txn_id` table & bump the schema version.
2. In a future future version, bump the minimum schema version to the new version in step 1 & drop the `event_txn_id` table.
3. In step 2, we can also stop writing the access token ID into the internal metadata for events.

See https://github.com/matrix-org/synapse/pull/15629#discussion_r1279686691
Gracefully drop MSC2716 related tables (incrementally import history)
Gracefully drop MSC2716 related tables (incrementally import history) in multiple steps/phases.

### Step 1

In one Synapse version, stop writing to any MSC2716 related tables. The normal [steps to gracefully drop a table](https://github.com/matrix-org/synapse/blob/fcc3ca37e1b404981d9a0d6f2708e14407775b97/docs/development/database_schema.md) usually include a step to first switch over reads but keep writing but since this is a experimental feature, we don't need to worry about the feature working we can just stop reading and writing in the same step.

This is being tackled in https://github.com/matrix-org/synapse/pull/15748


### Step 2

In the Synapse release after the first step ships, we can bump `SCHEMA_COMPAT_VERSION` and drop the tables altogether.

Relevant tables to drop:

```sql
-- Drop tables related to MSC2716 since the implementation is being removed
DROP TABLE insertion_events;
DROP TABLE insertion_event_edges;
DROP TABLE insertion_event_extremities;
DROP TABLE batch_events;
```


Discussion:

Resolving patch:
diff --git a/synapse/handlers/message.py b/synapse/handlers/message.py
--- a/synapse/handlers/message.py
+++ b/synapse/handlers/message.py
@@ -693,13 +693,9 @@ async def create_event(
         if require_consent and not is_exempt:
             await self.assert_accepted_privacy_policy(requester)
 
-        # Save the access token ID, the device ID and the transaction ID in the event
-        # internal metadata. This is useful to determine if we should echo the
-        # transaction_id in events.
+        # Save the the device ID and the transaction ID in the event internal metadata.
+        # This is useful to determine if we should echo the transaction_id in events.
         # See `synapse.events.utils.EventClientSerializer.serialize_event`
-        if requester.access_token_id is not None:
-            builder.internal_metadata.token_id = requester.access_token_id
-
         if requester.device_id is not None:
             builder.internal_metadata.device_id = requester.device_id
 
diff --git a/synapse/storage/schema/__init__.py b/synapse/storage/schema/__init__.py
--- a/synapse/storage/schema/__init__.py
+++ b/synapse/storage/schema/__init__.py
@@ -125,8 +125,8 @@
 
 
 SCHEMA_COMPAT_VERSION = (
-    # The `event_txn_id_device_id` must be written to for new events.
-    80
+    # The event_txn_id table and tables from MSC2716 no longer exist.
+    82
 )
 """Limit on how far the synapse codebase can be rolled back without breaking db compat
 


Test patch:
Repository: matrix-org/synapse

Issue:
Support MSC4028 (Push all encrypted events except for muted rooms)
### Description

The purpose of [MSC4028](https://github.com/matrix-org/matrix-spec-proposals/pull/4028) is to be able to support on mobile clients the "mentions-and-keywords-only" room notification setting in encrypted rooms. This MSC adds a new "server-default" **override** push rule (`.m.rule.encrypted_event`) in order to force the server to push all encrypted events to the registered client **except for muted room**. 

I request here to add this new "server-default" push rule with its unstable id `org.matrix.msc4028.encrypted_event`. Contrary to the stable push rule, this unstable one should not be enabled -> its `enabled` attribute must be `false`. Indeed we should not turn it on until the clients implementations are ready.

I listed [here](https://github.com/matrix-org/matrix-spec-proposals/blob/giomfo/push_encrypted_events/proposals/4028-push-all-encrypted-events-except-for-muted-rooms.md#server-side) the potential issues of this MSC server side. We talked about them in an internal discussion, the conclusion was:

> So calculating what to push and the unread counts (its the same process) is a big performance bottleneck currently when sending in large rooms. A bunch of optimisation has taken place to make that better, but its still quite expensive
I'm not so concerned about /sync, as we've mostly already done the work at that point

I would let the backend team use this issue or create new ones to handle any other dev related to this MSC.


Discussion:

Resolving patch:
diff --git a/synapse/config/experimental.py b/synapse/config/experimental.py
--- a/synapse/config/experimental.py
+++ b/synapse/config/experimental.py
@@ -415,3 +415,7 @@ def read_config(self, config: JsonDict, **kwargs: Any) -> None:
         LimitExceededError.include_retry_after_header = experimental.get(
             "msc4041_enabled", False
         )
+
+        self.msc4028_push_encrypted_events = experimental.get(
+            "msc4028_push_encrypted_events", False
+        )
diff --git a/synapse/storage/databases/main/push_rule.py b/synapse/storage/databases/main/push_rule.py
--- a/synapse/storage/databases/main/push_rule.py
+++ b/synapse/storage/databases/main/push_rule.py
@@ -88,6 +88,7 @@ def _load_rules(
         msc1767_enabled=experimental_config.msc1767_enabled,
         msc3664_enabled=experimental_config.msc3664_enabled,
         msc3381_polls_enabled=experimental_config.msc3381_polls_enabled,
+        msc4028_push_encrypted_events=experimental_config.msc4028_push_encrypted_events,
     )
 
     return filtered_rules


Test patch:
Repository: matrix-org/synapse

Issue:
Clean-up event_txn_id table
1. In a future version we need to stop reading & writing to the `event_txn_id` table & bump the schema version.
2. In a future future version, bump the minimum schema version to the new version in step 1 & drop the `event_txn_id` table.
3. In step 2, we can also stop writing the access token ID into the internal metadata for events.

See https://github.com/matrix-org/synapse/pull/15629#discussion_r1279686691
Gracefully drop MSC2716 related tables (incrementally import history)
Gracefully drop MSC2716 related tables (incrementally import history) in multiple steps/phases.

### Step 1

In one Synapse version, stop writing to any MSC2716 related tables. The normal [steps to gracefully drop a table](https://github.com/matrix-org/synapse/blob/fcc3ca37e1b404981d9a0d6f2708e14407775b97/docs/development/database_schema.md) usually include a step to first switch over reads but keep writing but since this is a experimental feature, we don't need to worry about the feature working we can just stop reading and writing in the same step.

This is being tackled in https://github.com/matrix-org/synapse/pull/15748


### Step 2

In the Synapse release after the first step ships, we can bump `SCHEMA_COMPAT_VERSION` and drop the tables altogether.

Relevant tables to drop:

```sql
-- Drop tables related to MSC2716 since the implementation is being removed
DROP TABLE insertion_events;
DROP TABLE insertion_event_edges;
DROP TABLE insertion_event_extremities;
DROP TABLE batch_events;
```


Discussion:
The first part of this landed in v1.92.0rc1.
See #16465
See #16465

Resolving patch:
diff --git a/synapse/handlers/message.py b/synapse/handlers/message.py
--- a/synapse/handlers/message.py
+++ b/synapse/handlers/message.py
@@ -693,13 +693,9 @@ async def create_event(
         if require_consent and not is_exempt:
             await self.assert_accepted_privacy_policy(requester)
 
-        # Save the access token ID, the device ID and the transaction ID in the event
-        # internal metadata. This is useful to determine if we should echo the
-        # transaction_id in events.
+        # Save the the device ID and the transaction ID in the event internal metadata.
+        # This is useful to determine if we should echo the transaction_id in events.
         # See `synapse.events.utils.EventClientSerializer.serialize_event`
-        if requester.access_token_id is not None:
-            builder.internal_metadata.token_id = requester.access_token_id
-
         if requester.device_id is not None:
             builder.internal_metadata.device_id = requester.device_id
 
diff --git a/synapse/storage/schema/__init__.py b/synapse/storage/schema/__init__.py
--- a/synapse/storage/schema/__init__.py
+++ b/synapse/storage/schema/__init__.py
@@ -129,8 +129,8 @@
 
 
 SCHEMA_COMPAT_VERSION = (
-    # The `event_txn_id_device_id` must be written to for new events.
-    80
+    # The event_txn_id table and tables from MSC2716 no longer exist.
+    83
 )
 """Limit on how far the synapse codebase can be rolled back without breaking db compat
 


Test patch:
Repository: matrix-org/synapse

Issue:
Declare support for Matrix v1.9
See [the changelog](https://spec.matrix.org/v1.9/changelog/#v19) (note that I stripped out spec clarifications).

This also requires us to finish support for Matrix v1.8: see #16176.

# Client-Server API

## Backwards Compatible Changes

* [x] Add the `m.rule.suppress_edits` default push rule, as per [MSC3958](https://github.com/matrix-org/matrix-spec-proposals/pull/3958).
  * #16113



Discussion:

Resolving patch:
diff --git a/synapse/rest/client/versions.py b/synapse/rest/client/versions.py
--- a/synapse/rest/client/versions.py
+++ b/synapse/rest/client/versions.py
@@ -80,6 +80,9 @@ def on_GET(self, request: Request) -> Tuple[int, JsonDict]:
                     "v1.4",
                     "v1.5",
                     "v1.6",
+                    "v1.7",
+                    "v1.8",
+                    "v1.9",
                 ],
                 # as per MSC1497:
                 "unstable_features": {


Test patch: