Repository: matrix-org/synapse

Issue:
Stabilize support for MSC2946: spaces summary
Once [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946) lands we should (at some point) switch to a stable endpoint and make some updates. There's few bits to this that we need to do:

## Phase 1

* [x] Support to the stable endpoint.
* [x] ~~Require `limit` to be > 0.~~ (This has always been correct.)
* [x] Ensure the endpoint is ratelimited.

## Phase 2

* [x] Remove the fallback logic of `get_room_hierarchy` to request the `/spaces` endpoint over federation.
* [x] Remove the `/spaces` client and federation endpoints (and corresponding code in the handlers).
* [x] Update the API responses to remove `allowed_spaces` (this was replaced with `allowed_room_ids`).
* [x] Clean-up code around max-children, which is not used by the `/hierarchy` endpoint.
* [x] Remove `room_id` from the serialized `children_state`, it is redundant and removed from the API response.

## Phase 3

* [x] Remove unstable `/hierarchy` client and federation endpoints.
* [x] Remove the typo `join_rules` field from the response.


Discussion:
I think this work needs to be split into a few pieces.

Phase 1:

* Support the stable endpoint.
* Remove some of the legacy code (the `/spaces` endpoint? and surrounding bits?)
* Enforce the `limit` / ratelimiting.

Phase 2:

* Remove the unstable `/hierarchy` endpoint.
> Remove some of the legacy code (the `/spaces` endpoint? and surrounding bits?)

It looks like these endpoints still get some usage on matrix.org, so might be best to just leave them for now and remove all the unstable code at once.
Client bugs for using the stable endpoint:

* vector-im/element-ios#5144
* vector-im/element-android#4462
* vector-im/element-web#19738

The only other client I know of which implements spaces is Cinny, which uses the matrix-js-sdk.

See also vector-im/element-meta#84.
For phase 2 it is useful to know which version of clients first started using (unstable or stable) `/hierarchy` instead of `/spaces`.

* Element Android v1.2.1 (vector-im/element-android#3926)
* matrix-ios-sdk v0.20.11 (matrix-org/matrix-ios-sdk#1259)
  * Element iOS: v1.6.9 (https://github.com/vector-im/element-ios/blob/v1.6.9/Podfile#L16)
* matrix-js-sdk v12.4.0 (matrix-org/matrix-js-sdk#1818):
	* Element Web: v1.8.2 (https://github.com/vector-im/element-web/blob/v1.8.2/package.json#L64)
	* Cinny: v1.2.1 (https://github.com/ajbura/cinny/blob/v1.2.1/package.json#L27)

Note that I don't think Element iOS ever shipped a version using `/spaces`? I think before v1.6.9 it didn't even have spaces support.
Using our opt-in analytics to determine "market share" of each Element platform (where values at or near zero aren't considered) for the Feb 17th - 22nd period, the following conclusions can be drawn. Note that scraping user agents is likely to be more reliable, however I think given the numbers below it's safe enough to move forward.

TLDR: All platforms are reporting versions which exceed the minimum outlined by previous comments.

* iOS (requires 1.6.9+ to be safe):
  * 1.7.0 is at 21.61%
  * 1.8.1 is at 77.04%
* Android (requires 1.2.1+ to be safe):
  * 1.3.15 is at 7.00%
  * 1.3.18 is at 12.65%
  * 1.4.0 is at 79.89%
* Web (requires 1.8.2+ to be safe):
  * No data (reporting error)
* Desktop (requires 1.8.2+ to be safe):
  * No data (reporting error)
  
While I don't have data on Element Web/Desktop, 1.8.2 was August 2021 and it's now February - if people haven't updated, I don't hold hope that they will by now.
Thanks for pulling those numbers @turt2live! I'll dust off my branch that removes `/spaces` support and go from there. 👍 

I think we need to leave the unstable `/hierarchy` endpoint for now since the stable one isn't yet used in clients to my knowledge.
> I think we need to leave the unstable /hierarchy endpoint for now since the stable one isn't yet used in clients to my knowledge.

can we try to at least agree a timescale when it might be possible to change this? one month? two months? six months?
For phase 3 it is useful to know which version of clients first started using the stable `/hierarchy` endpoint:

* Element Android v1.4.6 (vector-im/element-android#5443)
* matrix-ios-sdk v0.23.2 (matrix-org/matrix-ios-sdk#1415)
  * Element iOS: v1.8.11 (https://github.com/vector-im/element-ios/blob/v1.8.11/Podfile#L16)
* matrix-js-sdk v15.4.0 (matrix-org/matrix-js-sdk#2022):
	* Element Web: v1.9.9 (https://github.com/vector-im/element-web/blob/v1.9.9/package.json#L63
	* Cinny: v1.2.1 (https://github.com/ajbura/cinny/blob/v1.8.0/package.json#L31)

All known clients using the unstable `/hierarchy` endpoint have now been updated and had a release.
From taking a look at the latest client numbers it seems that ~93% of Android and ~97% of iOS users are on versions of Element which support the stable endpoint.
I put up #12851 to close this issue out! 🎉 
> Remove `room_id` from the serialized `children_state`, it is redundant and removed from the API response.

This was never done for some reason...even though I checked it off.

Resolving patch:
diff --git a/synapse/handlers/room_summary.py b/synapse/handlers/room_summary.py
--- a/synapse/handlers/room_summary.py
+++ b/synapse/handlers/room_summary.py
@@ -452,7 +452,6 @@ async def _summarize_local_room(
                 "type": e.type,
                 "state_key": e.state_key,
                 "content": e.content,
-                "room_id": e.room_id,
                 "sender": e.sender,
                 "origin_server_ts": e.origin_server_ts,
             }


Test patch: