Repository: matrix-org/synapse

Issue:
Upgrade from 1.76.0 to 1.95.1 results in  synapse.storage.background_updates: [background_updates-0] Error doing update: 'int' object is not iterable
### Description

After standard upgrade with pip service starts and apparently working, but log is full of errors about 
`Error doing update: 'int' object is not iterable`
[matrix.log](https://github.com/matrix-org/synapse/files/13223443/matrix.log)

### Homeserver

self hosted

### Synapse Version

{"server_version":"1.95.1"}

### Installation Method

PIP

### Database

PostgreSQL single server

### Workers

Single process

### Platform

Oracle linux 8

### Configuration

_No response_

### Relevant log output

```shell
Nov 01 05:06:58  matrix-synapse[3419535]: 2023-11-01 05:06:58,673 - synapse.storage.background_updates - 412 - ERROR - background_updates-0 - Error doing update: 'int' object is not iterable
Nov 01 05:06:58  matrix-synapse[3419535]: Traceback (most recent call last):
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/background_updates.py", line 409, in run_background_updates
Nov 01 05:06:58  matrix-synapse[3419535]:     result = await self.do_next_background_update(sleep)
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/background_updates.py", line 540, in do_next_background_update
Nov 01 05:06:58  matrix-synapse[3419535]:     await self._do_background_update(desired_duration_ms)
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/background_updates.py", line 583, in _do_background_update
Nov 01 05:06:58  matrix-synapse[3419535]:     items_updated = await update_handler(progress, batch_size)
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/databases/main/pusher.py", line 635, in _set_device_id_for_pushers
Nov 01 05:06:58  matrix-synapse[3419535]:     nb_processed = await self.db_pool.runInteraction(
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/database.py", line 934, in runInteraction
Nov 01 05:06:58  matrix-synapse[3419535]:     return await delay_cancellation(_runInteraction())
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib/python3.8/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks
Nov 01 05:06:58  matrix-synapse[3419535]:     result = context.run(
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib/python3.8/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator
Nov 01 05:06:58  matrix-synapse[3419535]:     return g.throw(self.type, self.value, self.tb)
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/database.py", line 900, in _runInteraction
Nov 01 05:06:58  matrix-synapse[3419535]:     result = await self.runWithConnection(
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/database.py", line 1029, in runWithConnection
Nov 01 05:06:58  matrix-synapse[3419535]:     return await make_deferred_yieldable(
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib/python3.8/site-packages/twisted/python/threadpool.py", line 244, in inContext
Nov 01 05:06:58  matrix-synapse[3419535]:     result = inContext.theWork()  # type: ignore[attr-defined]
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib/python3.8/site-packages/twisted/python/threadpool.py", line 260, in <lambda>
Nov 01 05:06:58  matrix-synapse[3419535]:     inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib/python3.8/site-packages/twisted/python/context.py", line 117, in callWithContext
Nov 01 05:06:58  matrix-synapse[3419535]:     return self.currentContext().callWithContext(ctx, func, *args, **kw)
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib/python3.8/site-packages/twisted/python/context.py", line 82, in callWithContext
Nov 01 05:06:58  matrix-synapse[3419535]:     return func(*args, **kw)
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib/python3.8/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
Nov 01 05:06:58  matrix-synapse[3419535]:     result = func(conn, *args, **kw)
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/database.py", line 1022, in inner_func
Nov 01 05:06:58  matrix-synapse[3419535]:     return func(db_conn, *args, **kwargs)
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/database.py", line 762, in new_transaction
Nov 01 05:06:58  matrix-synapse[3419535]:     r = func(cursor, *args, **kwargs)
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/databases/main/pusher.py", line 614, in set_device_id_for_pushers_txn
Nov 01 05:06:58  matrix-synapse[3419535]:     self.db_pool.simple_update_many_txn(
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/database.py", line 2078, in simple_update_many_txn
Nov 01 05:06:58  matrix-synapse[3419535]:     args = [tuple(x) + tuple(y) for x, y in zip(value_values, key_values)]
Nov 01 05:06:58  matrix-synapse[3419535]:   File "/home/matrix/synapse/env/lib64/python3.8/site-packages/synapse/storage/database.py", line 2078, in <listcomp>
Nov 01 05:06:58  matrix-synapse[3419535]:     args = [tuple(x) + tuple(y) for x, y in zip(value_values, key_values)]
Nov 01 05:06:58  matrix-synapse[3419535]: TypeError: 'int' object is not iterable
```


### Anything else that would be useful to know?

_No response_


Discussion:
Hi @Artiom-M, thanks, this indeed looks like a bug. It's in a background update though so shouldn't affect day-to-day operation beyond spamming the logs with errors. Usually I'd suggest downgrading again, but since v1.95.1 is a security release you might want to stay on v1.95.1.

The problem is:

https://github.com/matrix-org/synapse/blob/a11511954a58975d2e5400257a0cecfd27413447/synapse/storage/databases/main/pusher.py#L617-L618

That second line should be a list of tuples, i.e. `[(row[0],) ...]`

Broke in #16431.

cc @clokep.

Resolving patch:
diff --git a/synapse/storage/databases/main/pusher.py b/synapse/storage/databases/main/pusher.py
--- a/synapse/storage/databases/main/pusher.py
+++ b/synapse/storage/databases/main/pusher.py
@@ -601,7 +601,7 @@ def set_device_id_for_pushers_txn(txn: LoggingTransaction) -> int:
                 (last_pusher_id, batch_size),
             )
 
-            rows = txn.fetchall()
+            rows = cast(List[Tuple[int, Optional[str], Optional[str]]], txn.fetchall())
             if len(rows) == 0:
                 return 0
 
@@ -617,7 +617,7 @@ def set_device_id_for_pushers_txn(txn: LoggingTransaction) -> int:
                 txn=txn,
                 table="pushers",
                 key_names=("id",),
-                key_values=[row[0] for row in rows],
+                key_values=[(row[0],) for row in rows],
                 value_names=("device_id", "access_token"),
                 # If there was already a device_id on the pusher, we only want to clear
                 # the access_token column, so we keep the existing device_id. Otherwise,


Test patch: