Repository: matrix-org/synapse

Issue:
Failed to send_join via any server due to "Object of type Decimal is not JSON serializable"
<!--

**THIS IS NOT A SUPPORT CHANNEL!**
**IF YOU HAVE SUPPORT QUESTIONS ABOUT RUNNING OR CONFIGURING YOUR OWN HOME SERVER**,
please ask in **#synapse:matrix.org** (using a matrix.org account if necessary)

If you want to report a security issue, please see https://matrix.org/security-disclosure-policy/

This is a bug report template. By following the instructions below and
filling out the sections with your information, you will help the us to get all
the necessary data to fix your issue.

You can also preview your report before submitting it. You may remove sections
that aren't relevant to your particular case.

Text between <!-- and --​> marks will be invisible in the report.

-->

### Description
 When trying to join the channel #nextcloud:matrix.org an error is shown that reads 'Failed to send_join via matrix.org'
The issue persists on every join attempt.
<!-- Describe here the problem that you are experiencing -->

### Steps to reproduce

Join the channel #nextcloud:matrix.org
Review Logs and get the following error.
```
2021-10-20 09:17:35,911 - synapse.federation.federation_client - 616 - WARNING - POST-61221 - Failed to send_join via matrix.org

Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/twisted/internet/defer.py", line 1661, in _inlineCallbacks
result = current_context.run(gen.send, result)
StopIteration: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/twisted/internet/defer.py", line 1661, in _inlineCallbacks
result = current_context.run(gen.send, result)

ation: {'ed25519:auto': FetchKeyResult(verify_key=<nacl.signing.VerifyKey object at 0x151c2e943f70>, valid_until_ts=1576767829750), 'ed25519:a_RXGa': FetchKeyResult(verify_key=<nacl.signing.VerifyKey object at 0x151be8e8f790>, valid_until_ts=1634773315345)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/synapse/federation/federation_client.py", line 576, in _try_destination_list
return await callback(destination)
File "/usr/local/lib/python3.8/site-packages/synapse/federation/federation_client.py", line 825, in send_request
await concurrently_execute(
File "/usr/local/lib/python3.8/site-packages/twisted/internet/defer.py", line 1661, in _inlineCallbacks
result = current_context.run(gen.send, result)
File "/usr/local/lib/python3.8/site-packages/synapse/util/async_helpers.py", line 192, in _concurrently_execute_inner
await maybe_awaitable(func(value))
File "/usr/local/lib/python3.8/site-packages/synapse/federation/federation_client.py", line 815, in _execute
valid_pdu = await self._check_sigs_and_hash_and_fetch_one(
File "/usr/local/lib/python3.8/site-packages/synapse/federation/federation_client.py", line 458, in _check_sigs_and_hash_and_fetch_one
res = await self._check_sigs_and_hash(room_version, pdu)
File "/usr/local/lib/python3.8/site-packages/synapse/federation/federation_base.py", line 65, in _check_sigs_and_hash
if not check_event_content_hash(pdu):
File "/usr/local/lib/python3.8/site-packages/synapse/crypto/event_signing.py", line 42, in check_event_content_hash
name, expected_hash = compute_content_hash(event.get_pdu_json(), hash_algorithm)
File "/usr/local/lib/python3.8/site-packages/synapse/crypto/event_signing.py", line 96, in compute_content_hash
event_json_bytes = encode_canonical_json(event_dict)
File "/usr/local/lib/python3.8/site-packages/canonicaljson.py", line 73, in encode_canonical_json
s = _canonical_encoder.encode(json_object)
File "/usr/local/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/local/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/usr/local/lib/python3.8/site-packages/canonicaljson.py", line 30, in _default
raise TypeError(
TypeError: Object of type Decimal is not JSON serializable
2021-10-20 09:17:35,911 - synapse.http.server - 88 - INFO - POST-61221 - <XForwardedForRequest at 0x151c15df5370 method='POST' uri='/_matrix/client/r0/join/!TEwfEWdDwdaFazXmwD%3Amatrix.org?server_name=matrix.org' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to send_join via any server
```
<!--
Describe how what happens differs from what you expected.

If you can identify any relevant log snippets from _homeserver.log_, please include
those (please be careful to remove any personal or private data). Please surround them with
``` (three backticks, on a line on their own), so that they are formatted legibly.
-->

### Version information

<!-- IMPORTANT: please answer the following questions, to help us narrow down the problem -->

<!-- Was this issue identified on matrix.org or another homeserver? -->
- **Homeserver**: matrix.jenovarain.com

<!--
 What version of Synapse is running?

You can find the Synapse version with this command:

$ curl http://localhost:8008/_synapse/admin/v1/server_version

(You may need to replace `localhost:8008` if Synapse is not configured to
listen on that port.)
-->
- **Version**: {"server_version":"1.45.0","python_version":"3.8.12"}

- **Install method**: Docker
<!-- examples: package manager/git clone/pip  -->

- **Platform**: Docker
<!--
Tell us about the environment in which your homeserver is operating
distro, hardware, if it's running in a vm/container, etc.
-->



Discussion:
Duplicate of a comment in #10021: https://github.com/matrix-org/synapse/issues/10021#issuecomment-880424718 . #10021's become a dumping ground for various send_join failures with different causes, so let's keep this issue for this particular cause.

The remote homeserver is returning a `/send_join` response containing floats with decimal points, which makes `ijson` return a `Decimal` to Synapse.

@DMRobertson suggests using `use_float=True`, which would solve the immediate error. Whether the event hash matches afterwards is another matter.

Note that the problematic room in the post above is `#NextCloud:matrix.org` (`!TEwfEWdDwdaFazXmwD:matrix.org`), not `#nextcloud:matrix.org` (`!BTFFlyEeuhcFipzRnZ:matrix.org`). The problematic event is `$15668961892578633CPOCY:matrix.org`:
```
{"origin":"matrix.org","origin_server_ts":1634756526450,"pdus":[{"auth_events":[["$14889644723000232lIUEU:matrix.org",{"sha256":"ct0/Rr2g95h8cUZoPFXgSBXbyHOSYSwnyDHK+hwOfD4"}],["$156604474753394BKyyU:matrix.org",{"sha256":"iHuRPwfobe45nb1NDHZgpd3xUurLYDacTWzQELlxg+8"}],["$14889644723000230Leyrk:matrix.org",{"sha256":"uvY+tfZUSsN1r4r3pR30ywK/8DOvsrxRjGcmQPN95CA"}]],"content":{"ban":50,"events":{"im.vector.modular.widgets":50,"m.room.avatar":100,"m.room.canonical_alias":100,"m.room.history_visibility":100,"m.room.name":100,"m.room.power_levels":100,"m.room.topic":100},"events_default":0,"invite":0,"kick":50,"notifications":{"room":100.0},"redact":50,"state_default":100,"users":{"@Creak:matrix.org":50,"@brad:koehn.com":50,"@illukas:matrix.org":50,"@natrius:matrix.org":50,"@satriani:matrix.org":100},"users_default":0},"depth":2581,"event_id":"$15668961892578633CPOCY:matrix.org","hashes":{"sha256":"BI82yYRWZ6H0p146e+HICkPztlKfObAuBJW8xb54Yo8"},"origin":"matrix.org","origin_server_ts":1566896189852,"prev_events":[["$15668924972564026SXoZV:matrix.org",{"sha256":"H6YJjoffHc/Q85GdNzsIOJ7/Bm2LmAf5/p3R8mPUctA"}]],"prev_state":[],"room_id":"!TEwfEWdDwdaFazXmwD:matrix.org","sender":"@satriani:matrix.org","state_key":"","type":"m.room.power_levels","signatures":{"matrix.org":{"ed25519:auto":"viI1Pp4qxVs1S4jDCqutGZuWdd4UWUjzplWgH+zBYOJnNkrAdW6tzA8sDMqqByDc5QK1O0M0tCJYrcFmE/xjBg"}},"unsigned":{"replaces_state":"$156604474753394BKyyU:matrix.org","age":67860336598}}]}
```
`content.notifications.room` is `100.0` instead of `100`

As noted in #10021, joining `#ghidra:matrix.org` / `!RistbroappQPsmzqnr:matrix.org` is broken, and it's for the same reason. Event `$1574893667299573PRkGI:matrix.org`:
```
{"origin":"matrix.org","origin_server_ts":1634755280753,"pdus":[{"auth_events":[["$1553000572438nTzgN:matrix.org",{"sha256":"NGWtwRETqCrBH1fMpyE+8wTaYE022TJ6kxCWGxzvxZY"}],["$1553617609357765HBZlL:matrix.org",{"sha256":"tsXB6/GkpfASvz+cqGJ2WUqh+IjFRg5Vd6ihMva5bS0"}],["$156771980867333fZwzd:matrix.org",{"sha256":"1R54GWyy24wtLdKgtZrCvQmdHreFZO8iKQ9hbBPU1oQ"}]],"content":{"ban":50,"events":{"im.vector.modular.widgets":50,"m.room.avatar":49,"m.room.canonical_alias":50,"m.room.history_visibility":100,"m.room.name":45,"m.room.power_levels":100,"m.room.topic":44},"events_default":0,"invite":0,"kick":47,"notifications":{"room":46.0},"redact":40,"state_default":49,"users":{"@_discord_bot:t2bot.io":40,"@kasifdekel:matrix.org":100,"@steps-:matrix.org":100,"@telegram:t2bot.io":40,"@telegram_240575894:t2bot.io":50,"@telegram_604709192:t2bot.io":50},"users_default":0},"depth":6771,"event_id":"$1574893667299573PRkGI:matrix.org","hashes":{"sha256":"k5n0pljeGmYFhgoKLf9CUxcHE9Ckl65qyA9gNGlLsvw"},"origin":"matrix.org","origin_server_ts":1574893667507,"prev_events":[["$157489351328421Jeqab:t2bot.io",{"sha256":"vbe12RzeyxM1n+p5o9Ay6f6zKpc086EUEpZBO4aDv8k"}]],"prev_state":[],"room_id":"!RistbroappQPsmzqnr:matrix.org","sender":"@steps-:matrix.org","state_key":"","type":"m.room.power_levels","signatures":{"matrix.org":{"ed25519:auto":"dvxSidd5vBdQSSA7YG+P8UmBuxteImK2rCeDFEJig7iJuG56ezyBFZRq1x7sKy4xk8N20hOWtBLO/vrOt+YYAw"}},"unsigned":{"replaces_state":"$156771980867333fZwzd:matrix.org","age":59861613246}}]}
```
`content.notifications.room` is `46.0` instead of `46`

Both of these affected rooms are version 1
The hashes for both the above events match, as long as the trailing `.0`s are included
Is there any workaround for this? The Nextcloud room is the main reason I use Matrix and I'm trying to switch to a Matrix account on a different server, but I'm blocked because that account can't join the room that is most important to me.

What will the fix for this issue involve? Is it on the Nextcloud room folks to fix it? Is there any solution I could work on?
@localjo A fix for this particular failure is on the todo list for the Synapse team. It's likely to be fixed in 1.47.0rc1, in two weeks, though this is a very rough estimate and I can't make any promises with regards to timelines.

In the meantime, if you're handy with Python and are the admin of the homeserver, you could try modifying the code to pass `use_float=True` wherever Synapse uses the `ijson` package and reporting back on whether it works.

Resolving patch:
diff --git a/synapse/federation/transport/client.py b/synapse/federation/transport/client.py
--- a/synapse/federation/transport/client.py
+++ b/synapse/federation/transport/client.py
@@ -1310,14 +1310,17 @@ def __init__(self, room_version: RoomVersion, v1_api: bool):
         self._coro_state = ijson.items_coro(
             _event_list_parser(room_version, self._response.state),
             prefix + "state.item",
+            use_float=True,
         )
         self._coro_auth = ijson.items_coro(
             _event_list_parser(room_version, self._response.auth_events),
             prefix + "auth_chain.item",
+            use_float=True,
         )
         self._coro_event = ijson.kvitems_coro(
             _event_parser(self._response.event_dict),
             prefix + "org.matrix.msc3083.v2.event",
+            use_float=True,
         )
 
     def write(self, data: bytes) -> int:


Test patch: