Repository: matrix-org/synapse

Issue:
 Background process 'background_updates' threw an exception, after updating to version 1.81.0rc1
### Description

Just updated to `1.81.0rc1` and got below `exceptions`


### Steps to reproduce

- just updated to ` version 1.81.0rc1`
- after restart observing exceptions



### Homeserver

my

### Synapse Version

1.81.0rc1

### Installation Method

pip (from PyPI)

### Database

PostgreSQL 12, single, no portdb, not from backup

### Workers

Single process

### Platform

VM, python3.10, venv

### Configuration

_No response_

### Relevant log output

```shell
2023-04-04 16:09:47,102 - synapse.storage.background_updates - 431 - INFO - background_updates-0 - Starting update batch on background update 'set_device_id_for_pushers'
2023-04-04 16:09:47,117 - synapse.storage.background_updates - 302 - ERROR - background_updates-0 - Error doing update
Traceback (most recent call last):
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/storage/background_updates.py", line 294, in run_background_updates
    result = await self.do_next_background_update(sleep)
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/storage/background_updates.py", line 424, in do_next_background_update
    await self._do_background_update(desired_duration_ms)
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/storage/background_updates.py", line 467, in _do_background_update
    items_updated = await update_handler(progress, batch_size)
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/storage/databases/main/pusher.py", line 570, in _set_device_id_for_pushers
    nb_processed = await self.db_pool.runInteraction(
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/storage/database.py", line 902, in runInteraction
    return await delay_cancellation(_runInteraction())
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/twisted/internet/defer.py", line 1656, in _inlineCallbacks
    result = current_context.run(
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/twisted/python/failure.py", line 489, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/storage/database.py", line 868, in _runInteraction
    result = await self.runWithConnection(
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/storage/database.py", line 997, in runWithConnection
    return await make_deferred_yieldable(
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/twisted/python/threadpool.py", line 244, in inContext
    result = inContext.theWork()  # type: ignore[attr-defined]
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/twisted/python/threadpool.py", line 260, in <lambda>
    inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/twisted/python/context.py", line 117, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/twisted/python/context.py", line 82, in callWithContext
    return func(*args, **kw)
  File "/home/deployer/projects/synapseenv/env3.10/lib/python3.10/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
    result = func(conn, *args, **kw)
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/storage/database.py", line 990, in inner_func
    return func(db_conn, *args, **kwargs)
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/storage/database.py", line 730, in new_transaction
    r = func(cursor, *args, **kwargs)
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/storage/databases/main/pusher.py", line 565, in set_device_id_for_pushers_txn
    txn, "set_device_id_for_pushers", {"pusher_id": rows[-1]["id"]}
KeyError: 'id'

....


KeyError: 'id'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/metrics/background_process_metrics.py", line 242, in run
    return await func(*args, **kwargs)
  File "/home/<redacted>/synapseenv/env3.10/lib/python3.10/site-packages/synapse/storage/background_updates.py", line 299, in run_background_updates
    raise RuntimeError(
RuntimeError: 5 back-to-back background update failures; aborting.
```


### Anything else that would be useful to know?

_No response_


Discussion:
I think this was introduced by: https://github.com/matrix-org/synapse/pull/15280, as it changed the name of the row:

https://github.com/matrix-org/synapse/commit/5b70f240cf70b390db7e74ab614ace108fc08d70#diff-62af9acfbf12e3a6c8b8b348ba374f4ce834f7c4e3a1b3d5d6b9242b1714c702R522

Resolving patch:
diff --git a/synapse/storage/databases/main/pusher.py b/synapse/storage/databases/main/pusher.py
--- a/synapse/storage/databases/main/pusher.py
+++ b/synapse/storage/databases/main/pusher.py
@@ -562,7 +562,7 @@ def set_device_id_for_pushers_txn(txn: LoggingTransaction) -> int:
             )
 
             self.db_pool.updates._background_update_progress_txn(
-                txn, "set_device_id_for_pushers", {"pusher_id": rows[-1]["id"]}
+                txn, "set_device_id_for_pushers", {"pusher_id": rows[-1]["pusher_id"]}
             )
 
             return len(rows)


Test patch: