Repository: matrix-org/synapse

Issue #15500: Initial sync caching can be awkward for stateless clients that want to get fresh information
Created: 2023-04-27T12:40:53Z
I recently ran into a small catch with Synapse's implementation of sync: it caches sync responses by some token along the lines of (user_id, device_id, since_token) (and a few other fields which are not relevant for this case really).
This was well-intentioned: it is basically needed to make initial syncs taking longer than 30 seconds to compute possible — the client keeps requesting an initial sync until the response is available without timing out.
However, this can be problematic for client developers who actually want the freshest state. For example, a stateless CLI tool may want to fetch the latest state events and apply some changes based on what it sees.
Currently it is difficult to do so because it is hard to request a fresh/uncached sync response from Synapse.
This can introduce subtle bugs into CLI tools where they may have been executed less than 2 minutes ago, since they won't see any changes made in the meantime (and in my case it led to a critically embarrassing bug which I'm glad I noticed; I also notice someone else ran into this today and it was confusing to them!).
The best approach I've found is to write some unpredictable account data (for me: I write a freshly generated UUID into net.librepush.sync_buster) and keep syncing until you get an echo for it — at that point you know you have a fresh sync response. (It may also be possible to use to-device events if you're not using the admin API to log in to the user in question.)
Just bear in mind that the more often you run this tool in a 2 minute window, the higher the number of stale syncs you will see in a chain until you get the sync-busting echo.

Duplicate issue #15502: /sync slow to notify about events when timeout=0
Created: 2023-04-28T01:04:37Z
Description

If you query /_matrix/client/v3/sync?since=SYNCTOKEN&timeout=TIMEOUT (where SYNCTOKEN is some sync token and TIMEOUT is some timeout) when there isn't any new state, the response's next_batch will be identical to SYNCTOKEN.
So, when the client is polling, it will make repeated requests to this same URL until there is new state. (Which is normal/expected behaviour as far as I know)
Expected behaviour is:

If an event is sent to a room after a sync request has been sent, but before the timeout has elapsed

THEN, the sync request is responded with the new events to fairly soon after the event has sent


If an event is sent to a room before a sync request has been sent

THEN, as soon as the next sync request is sent, a sync response will be immediate and will contain the new events





Whilst this seems to be the most of the time, there's a scenario where it doesn't happen

timeout=0
At least one empty sync response has been received for the current sync token (i.e. the sync request referred to in the expected behaviour above comes after at least one empty sync response)

In this scenario, it takes about 2 minutes for the new event to show up in the sync response
Here's a script that recreates the bug (with included example output) https://gist.github.com/swedgwood/3d17193e465871518d499b584c7c574c
There's also a similar sounding issue which was marked as resolved by the author #14713 which they fixed by adjusting the cache time.
I'm not familiar with synapse internals, but I'd hazard to guess that something about timeout=0 changes the way caching works - so when the same sync token is used twice, the previous result is cached for the default cache time of about 2 minutes, which coincides with how long it takes for the event to show up in the sync response.
Steps to reproduce

Set up a new account and join one new room
In one session, keep syncing, with timeout=0 until the sync response is empty, and so the response's next_batch provides the same sync token that was used in the request.
Send a message to the room
Sync again, and notice that the response is still empty, despite the new event
Keep trying to sync over the next two minutes, and notice that the result keeps being empty, until after the 2 minutes is up, when the sync response finally has the new event

(These steps to reproduce rely on sending a message to a room, but if my caching theory is correct, then I think this should occur with any kind of state change that would cause the sync response to be non-empty)
Also see this script for reproducing this error https://gist.github.com/swedgwood/3d17193e465871518d499b584c7c574c
Homeserver
personal homeserver
Synapse Version
v1.82.0
Installation Method
Docker (matrixdotorg/synapse)
Database
single postgresql + previously ported from sqlite
Workers
I don't know
Platform
Running in a container on podman on Fedora CoreOS, on a VPS.
Configuration
All defaults
Relevant log output
Not sure what logs would be relevant.
Anything else that would be useful to know?
No response

Closing comment (2023-05-01T12:42:36Z): I think this is a duplicate of #15500, but I'm not really sure.