Repository: matrix-org/synapse

Issue:
synapse_port_db fails with "argument of type 'int' is not iterable"
### Description

When running `synapse_port_db` to convert an existing SQLite installation to PostgreSQL, the script fails after a short time with the following output:

<details>
<summary>
synapse_port_db output
</summary>

```
2022-02-05 12:55:44,665 - synapse.config.key - 147 - WARNING - This server is configured to use 'matrix.org' as its trusted key server via the
'trusted_key_servers' config option. 'matrix.org' is a good choice for a key
server since it is long-lived, stable and trusted. However, some admins may
wish to use another server for this purpose.

To suppress this warning and continue using 'matrix.org', admins should set
'suppress_key_server_warning' to 'true' in homeserver.yaml.
--------------------------------------------------------------------------------
2022-02-05 12:55:44,834 - synapse.storage.prepare_database - 116 - INFO - ('main', 'state'): Checking existing schema version
2022-02-05 12:55:44,881 - synapse.storage.prepare_database - 120 - INFO - ('main', 'state'): Existing schema is 67 (+0 deltas)
2022-02-05 12:55:44,881 - synapse.storage.databases.main - 355 - INFO - Checking database for consistency with configuration...
2022-02-05 12:55:44,882 - synapse.storage.prepare_database - 412 - INFO - Applying schema deltas for v67
2022-02-05 12:55:44,884 - synapse.storage.prepare_database - 544 - INFO - Schema now up to date
2022-02-05 12:55:44,954 - synapse.storage.util.id_generators - 88 - INFO - Initialising stream generator for pushers(id): 1
2022-02-05 12:55:44,956 - synapse.storage.util.id_generators - 88 - INFO - Initialising stream generator for deleted_pushers(stream_id): 1
2022-02-05 12:55:44,962 - synapse.storage.util.id_generators - 88 - INFO - Initialising stream generator for events(stream_ordering): 2772100
2022-02-05 12:55:44,964 - synapse.storage.util.id_generators - 88 - INFO - Initialising stream generator for events(stream_ordering): -66907
2022-02-05 12:55:44,967 - synapse.storage.util.id_generators - 88 - INFO - Initialising stream generator for ex_outlier_stream(event_stream_ordering): -66908
2022-02-05 12:55:44,971 - synapse.metrics - 678 - INFO - Collecting gc 1
2022-02-05 12:55:45,093 - synapse.storage.prepare_database - 116 - INFO - ('main', 'state'): Checking existing schema version
2022-02-05 12:55:45,104 - synapse.storage.prepare_database - 120 - INFO - ('main', 'state'): Existing schema is 67 (+0 deltas)
2022-02-05 12:55:45,104 - synapse.storage.databases.main - 355 - INFO - Checking database for consistency with configuration...
2022-02-05 12:55:45,110 - synapse.storage.prepare_database - 412 - INFO - Applying schema deltas for v67
2022-02-05 12:55:45,112 - synapse.storage.prepare_database - 544 - INFO - Schema now up to date
2022-02-05 12:55:45,126 - synapse.storage.util.id_generators - 88 - INFO - Initialising stream generator for pushers(id): 1
2022-02-05 12:55:45,127 - synapse.storage.util.id_generators - 88 - INFO - Initialising stream generator for deleted_pushers(stream_id): 1
2022-02-05 12:55:45,359 - synapse_port_db - 664 - INFO - Found 152 tables
2022-02-05 13:04:41,177 - synapse_port_db - 332 - INFO - Table user_directory: 0/33573 (rows 1-0) already ported
2022-02-05 13:04:41,179 - synapse_port_db - 332 - INFO - Table pushers: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,185 - synapse_port_db - 332 - INFO - Table stream_positions: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,185 - synapse_port_db - 332 - INFO - Table local_group_updates: 0/2 (rows 1-0) already ported
2022-02-05 13:04:41,186 - synapse_port_db - 332 - INFO - Table room_account_data: 0/135 (rows 1-0) already ported
2022-02-05 13:04:41,188 - synapse_port_db - 332 - INFO - Table event_push_summary_stream_ordering: 0/1 (rows 1-0) already ported
2022-02-05 13:04:41,189 - synapse_port_db - 332 - INFO - Table monthly_active_users: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,190 - synapse_port_db - 332 - INFO - Table room_depth: 0/154 (rows 1-0) already ported
2022-02-05 13:04:41,191 - synapse_port_db - 332 - INFO - Table users_in_public_rooms: 0/41067 (rows 1-0) already ported
2022-02-05 13:04:41,194 - synapse_port_db - 332 - INFO - Table group_rooms: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,194 - synapse_port_db - 332 - INFO - Table remote_media_cache: 11588/12419 (rows 1-11588) already ported
2022-02-05 13:04:41,195 - synapse_port_db - 332 - INFO - Table event_to_state_groups: 444000/963245 (rows 1-446838) already ported
2022-02-05 13:04:41,197 - synapse_port_db - 332 - INFO - Table applied_module_schemas: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,197 - synapse_port_db - 332 - INFO - Table presence_stream: 4168/10840 (rows 1-22312624) already ported
2022-02-05 13:04:41,198 - synapse_port_db - 332 - INFO - Table group_attestations_renewals: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,204 - synapse_port_db - 332 - INFO - Table state_events: 245376/276591 (rows 1-264677) already ported
2022-02-05 13:04:41,204 - synapse_port_db - 332 - INFO - Table ratelimit_override: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,205 - synapse_port_db - 332 - INFO - Table batch_events: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,205 - synapse_port_db - 332 - INFO - Table room_stats_state: 0/154 (rows 1-0) already ported
2022-02-05 13:04:41,206 - synapse_port_db - 332 - INFO - Table device_federation_inbox: 0/1525 (rows 1-0) already ported
2022-02-05 13:04:41,207 - synapse_port_db - 332 - INFO - Table state_groups: 101281/126611 (rows 1-102652) already ported
2022-02-05 13:04:41,210 - synapse_port_db - 332 - INFO - Table room_memberships: 241453/271194 (rows 1-260618) already ported
2022-02-05 13:04:41,211 - synapse_port_db - 332 - INFO - Table receipts_graph: 0/28770 (rows 1-0) already ported
2022-02-05 13:04:41,213 - synapse_port_db - 332 - INFO - Table device_federation_outbox: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,214 - synapse_port_db - 332 - INFO - Table local_media_repository_url_cache: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,214 - synapse_port_db - 332 - INFO - Table room_tags: 0/2 (rows 1-0) already ported
2022-02-05 13:04:41,216 - synapse_port_db - 332 - INFO - Table group_users: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,217 - synapse_port_db - 332 - INFO - Table event_auth_chain_to_calculate: 0/40 (rows 1-0) already ported
2022-02-05 13:04:41,217 - synapse_port_db - 332 - INFO - Table application_services_state: 0/1 (rows 1-0) already ported
2022-02-05 13:04:41,218 - synapse_port_db - 332 - INFO - Table redactions: 26502/27881 (rows 1-26505) already ported
2022-02-05 13:04:41,219 - synapse_port_db - 332 - INFO - Table event_push_actions: 0/227 (rows 1-0) already ported
2022-02-05 13:04:41,220 - synapse_port_db - 332 - INFO - Table presence: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,221 - synapse_port_db - 332 - INFO - Table insertion_event_edges: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,222 - synapse_port_db - 332 - INFO - Table insertion_events: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,224 - synapse_port_db - 332 - INFO - Table current_state_events: 0/69575 (rows 1-0) already ported
2022-02-05 13:04:41,225 - synapse_port_db - 332 - INFO - Table group_summary_rooms: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,226 - synapse_port_db - 332 - INFO - Table e2e_room_keys: 0/1336 (rows 1-0) already ported
2022-02-05 13:04:41,227 - synapse_port_db - 332 - INFO - Table remote_profile_cache: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,228 - synapse_port_db - 332 - INFO - Table device_lists_outbound_last_success: 0/2962 (rows 1-0) already ported
2022-02-05 13:04:41,230 - synapse_port_db - 332 - INFO - Table receipts_linearized: 0/28771 (rows 1-0) already ported
2022-02-05 13:04:41,232 - synapse_port_db - 332 - INFO - Table events: 387000/1086444 (rows -64760-437060) already ported
2022-02-05 13:04:41,233 - synapse_port_db - 332 - INFO - Table remote_media_cache_thumbnails: 46488/54681 (rows 1-46488) already ported
2022-02-05 13:04:41,235 - synapse_port_db - 332 - INFO - Table blocked_rooms: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,236 - synapse_port_db - 332 - INFO - Table event_relations: 0/51357 (rows 1-0) already ported
2022-02-05 13:04:41,237 - synapse_port_db - 332 - INFO - Table user_filters: 0/5 (rows 1-0) already ported
2022-02-05 13:04:41,237 - synapse_port_db - 332 - INFO - Table threepid_validation_session: 0/1 (rows 1-0) already ported
2022-02-05 13:04:41,239 - synapse_port_db - 332 - INFO - Table dehydrated_devices: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,240 - synapse_port_db - 332 - INFO - Table e2e_one_time_keys_json: 0/330 (rows 1-0) already ported
2022-02-05 13:04:41,242 - synapse_port_db - 332 - INFO - Table users: 16/16 (rows 1-16) already ported
2022-02-05 13:04:41,243 - synapse_port_db - 332 - INFO - Table stream_ordering_to_exterm: 118642/277197 (rows 1-1705616) already ported
2022-02-05 13:04:41,244 - synapse_port_db - 332 - INFO - Table push_rules: 0/13 (rows 1-0) already ported
2022-02-05 13:04:41,246 - synapse_port_db - 332 - INFO - Table device_lists_remote_resync: 0/31 (rows 1-0) already ported
2022-02-05 13:04:41,247 - synapse_port_db - 332 - INFO - Table event_push_summary: 0/1 (rows 1-0) already ported
2022-02-05 13:04:41,249 - synapse_port_db - 332 - INFO - Table user_daily_visits: 0/984 (rows 1-0) already ported
2022-02-05 13:04:41,249 - synapse_port_db - 332 - INFO - Table event_edges: 430000/1209485 (rows 1-450860) already ported
2022-02-05 13:04:41,249 - synapse_port_db - 332 - INFO - Table device_inbox: 0/4339 (rows 1-0) already ported
2022-02-05 13:04:41,250 - synapse_port_db - 332 - INFO - Table device_lists_remote_cache: 0/73405 (rows 1-0) already ported
2022-02-05 13:04:41,250 - synapse_port_db - 332 - INFO - Table rooms: 132/159 (rows 1-144) already ported
2022-02-05 13:04:41,251 - synapse_port_db - 332 - INFO - Table erased_users: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,251 - synapse_port_db - 332 - INFO - Table local_current_membership: 0/184 (rows 1-0) already ported
2022-02-05 13:04:41,252 - synapse_port_db - 332 - INFO - Table insertion_event_extremities: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,252 - synapse_port_db - 332 - INFO - Table room_aliases: 0/6 (rows 1-0) already ported
2022-02-05 13:04:41,253 - synapse_port_db - 332 - INFO - Table destinations: 0/4483 (rows 1-0) already ported
2022-02-05 13:04:41,253 - synapse_port_db - 332 - INFO - Table room_tags_revisions: 0/3 (rows 1-0) already ported
2022-02-05 13:04:41,254 - synapse_port_db - 332 - INFO - Table user_signature_stream: 0/7 (rows 1-0) already ported
2022-02-05 13:04:41,255 - synapse_port_db - 332 - INFO - Table users_who_share_private_rooms: 0/176 (rows 1-0) already ported
2022-02-05 13:04:41,255 - synapse_port_db - 332 - INFO - Table e2e_fallback_keys_json: 0/5 (rows 1-0) already ported
2022-02-05 13:04:41,256 - synapse_port_db - 332 - INFO - Table event_reports: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,256 - synapse_port_db - 332 - INFO - Table public_room_list_stream: 3/3 (rows 1-3) already ported
2022-02-05 13:04:41,256 - synapse_port_db - 332 - INFO - Table group_summary_roles: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,257 - synapse_port_db - 332 - INFO - Table room_alias_servers: 0/7 (rows 1-0) already ported
2022-02-05 13:04:41,257 - synapse_port_db - 332 - INFO - Table device_lists_remote_extremeties: 0/13308 (rows 1-0) already ported
2022-02-05 13:04:41,258 - synapse_port_db - 332 - INFO - Table ignored_users: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,258 - synapse_port_db - 332 - INFO - Table group_invites: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,258 - synapse_port_db - 332 - INFO - Table state_group_edges: 96665/120820 (rows 1-98013) already ported
2022-02-05 13:04:41,258 - synapse_port_db - 332 - INFO - Table e2e_cross_signing_signatures: 0/18 (rows 1-0) already ported
2022-02-05 13:04:41,259 - synapse_port_db - 332 - INFO - Table group_summary_room_categories: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,259 - synapse_port_db - 332 - INFO - Table threepid_validation_token: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,260 - synapse_port_db - 332 - INFO - Table e2e_room_keys_versions: 0/1 (rows 1-0) already ported
2022-02-05 13:04:41,260 - synapse_port_db - 332 - INFO - Table application_services_txns: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,261 - synapse_port_db - 332 - INFO - Table sessions: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,261 - synapse_port_db - 332 - INFO - Table local_media_repository: 210/216 (rows 1-210) already ported
2022-02-05 13:04:41,262 - synapse_port_db - 332 - INFO - Table background_updates: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,262 - synapse_port_db - 332 - INFO - Table event_backward_extremities: 0/2464 (rows 1-0) already ported
2022-02-05 13:04:41,263 - synapse_port_db - 332 - INFO - Table ui_auth_sessions: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,263 - synapse_port_db - 332 - INFO - Table e2e_device_keys_json: 0/7 (rows 1-0) already ported
2022-02-05 13:04:41,264 - synapse_port_db - 332 - INFO - Table refresh_tokens: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,264 - synapse_port_db - 332 - INFO - Table push_rules_enable: 0/13 (rows 1-0) already ported
2022-02-05 13:04:41,264 - synapse_port_db - 332 - INFO - Table room_retention: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,265 - synapse_port_db - 332 - INFO - Table device_lists_stream: 0/146908 (rows 1-0) already ported
2022-02-05 13:04:41,266 - synapse_port_db - 332 - INFO - Table threepid_guest_access_tokens: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,266 - synapse_port_db - 332 - INFO - Table server_keys_json: 0/8770 (rows 1-0) already ported
2022-02-05 13:04:41,266 - synapse_port_db - 332 - INFO - Table appservice_room_list: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,268 - synapse_port_db - 332 - INFO - Table user_stats_current: 0/16 (rows 1-0) already ported
2022-02-05 13:04:41,268 - synapse_port_db - 332 - INFO - Table group_roles: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,269 - synapse_port_db - 332 - INFO - Table account_validity: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,269 - synapse_port_db - 332 - INFO - Table local_group_membership: 0/1 (rows 1-0) already ported
2022-02-05 13:04:41,269 - synapse_port_db - 332 - INFO - Table event_search: 386000/796913 (rows 1-387322) already ported
2022-02-05 13:04:41,270 - synapse_port_db - 332 - INFO - Table pusher_throttle: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,270 - synapse_port_db - 332 - INFO - Table device_lists_outbound_pokes: 0/3573 (rows 1-0) already ported
2022-02-05 13:04:41,271 - synapse_port_db - 332 - INFO - Table profiles: 0/16 (rows 1-0) already ported
2022-02-05 13:04:41,271 - synapse_port_db - 332 - INFO - Table federation_stream_position: 0/2 (rows 1-0) already ported
2022-02-05 13:04:41,272 - synapse_port_db - 332 - INFO - Table user_directory_search: 0/33573 (rows 1-0) already ported
2022-02-05 13:04:41,272 - synapse_port_db - 332 - INFO - Table appservice_stream_position: 0/1 (rows 1-0) already ported
2022-02-05 13:04:41,273 - synapse_port_db - 332 - INFO - Table event_forward_extremities: 0/167 (rows 1-0) already ported
2022-02-05 13:04:41,273 - synapse_port_db - 332 - INFO - Table event_auth_chains: 0/276551 (rows 1-0) already ported
2022-02-05 13:04:41,274 - synapse_port_db - 332 - INFO - Table room_stats_earliest_token: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,274 - synapse_port_db - 332 - INFO - Table registration_tokens: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,274 - synapse_port_db - 332 - INFO - Table server_signature_keys: 0/7450 (rows 1-0) already ported
2022-02-05 13:04:41,275 - synapse_port_db - 332 - INFO - Table groups: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,275 - synapse_port_db - 332 - INFO - Table state_groups_state: 428000/19773199 (rows 1-428079) already ported
2022-02-05 13:04:41,276 - synapse_port_db - 332 - INFO - Table account_data: 0/15 (rows 1-0) already ported
2022-02-05 13:04:41,276 - synapse_port_db - 332 - INFO - Table rejections: 112/239 (rows 1-112) already ported
2022-02-05 13:04:41,276 - synapse_port_db - 332 - INFO - Table devices: 0/31 (rows 1-0) already ported
2022-02-05 13:04:41,277 - synapse_port_db - 332 - INFO - Table event_auth_chain_links: 0/561112 (rows 1-0) already ported
2022-02-05 13:04:41,277 - synapse_port_db - 332 - INFO - Table group_attestations_remote: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,278 - synapse_port_db - 332 - INFO - Table push_rules_stream: 37/37 (rows 1-37) already ported
2022-02-05 13:04:41,278 - synapse_port_db - 332 - INFO - Table received_transactions: 403000/3552865 (rows 1-25873373) already ported
2022-02-05 13:04:41,278 - synapse_port_db - 332 - INFO - Table event_json: 393000/1130759 (rows 1-413785) already ported
2022-02-05 13:04:41,279 - synapse_port_db - 332 - INFO - Table event_reference_hashes: 432000/1130520 (rows 1-452785) already ported
2022-02-05 13:04:41,279 - synapse_port_db - 332 - INFO - Table room_stats_current: 0/154 (rows 1-0) already ported
2022-02-05 13:04:41,280 - synapse_port_db - 332 - INFO - Table current_state_delta_stream: 0/218076 (rows 1-0) already ported
2022-02-05 13:04:41,280 - synapse_port_db - 332 - INFO - Table federation_inbound_events_staging: 0/1 (rows 1-0) already ported
2022-02-05 13:04:41,281 - synapse_port_db - 332 - INFO - Table schema_compat_version: 0/1 (rows 1-0) already ported
2022-02-05 13:04:41,282 - synapse_port_db - 332 - INFO - Table device_auth_providers: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,282 - synapse_port_db - 332 - INFO - Table local_media_repository_thumbnails: 778/778 (rows 1-778) already ported
2022-02-05 13:04:41,282 - synapse_port_db - 332 - INFO - Table e2e_cross_signing_keys: 0/85905 (rows 1-0) already ported
2022-02-05 13:04:41,283 - synapse_port_db - 332 - INFO - Table event_labels: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,283 - synapse_port_db - 332 - INFO - Table user_threepid_id_server: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,284 - synapse_port_db - 332 - INFO - Table stats_incremental_position: 0/1 (rows 1-0) already ported
2022-02-05 13:04:41,284 - synapse_port_db - 332 - INFO - Table worker_locks: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,284 - synapse_port_db - 332 - INFO - Table user_directory_stream_pos: 0/1 (rows 1-0) already ported
2022-02-05 13:04:41,285 - synapse_port_db - 332 - INFO - Table user_threepids: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,292 - synapse_port_db - 332 - INFO - Table event_expiry: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,294 - synapse_port_db - 332 - INFO - Table user_ips: 0/41 (rows 1-0) already ported
2022-02-05 13:04:41,294 - synapse_port_db - 332 - INFO - Table ex_outlier_stream: 11184/12472 (rows 1-11184) already ported
2022-02-05 13:04:41,295 - synapse_port_db - 332 - INFO - Table event_auth: 444000/938618 (rows 1-506665) already ported
2022-02-05 13:04:41,295 - synapse_port_db - 332 - INFO - Table group_summary_users: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,296 - synapse_port_db - 332 - INFO - Table user_external_ids: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,296 - synapse_port_db - 332 - INFO - Table deleted_pushers: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,297 - synapse_port_db - 332 - INFO - Table open_id_tokens: 0/26 (rows 1-0) already ported
2022-02-05 13:04:41,297 - synapse_port_db - 332 - INFO - Table users_pending_deactivation: 0/0 (rows 1-0) already ported
2022-02-05 13:04:41,298 - synapse_port_db - 332 - INFO - Table event_push_actions_staging: 0/10 (rows 1-0) already ported
2022-02-05 13:04:41,298 - synapse_port_db - 332 - INFO - Table group_room_categories: 0/0 (rows 1-0) already ported
2022-02-05 13:04:47,838 - synapse.metrics - 678 - INFO - Collecting gc 1
2022-02-05 13:04:52,809 - synapse_port_db - 722 - ERROR -
Traceback (most recent call last):
  File "/usr/bin/synapse_port_db", line 701, in run
    await make_deferred_yieldable(
twisted.internet.defer.FirstError: FirstError[#97, [Failure instance: Traceback: <class 'TypeError'>: argument of type 'int' is not iterable
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:701:errback
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:764:_startRunCallbacks
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:858:_runCallbacks
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:1751:gotResult
--- <exception caught here> ---
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:1657:_inlineCallbacks
/usr/lib/python3.10/site-packages/twisted/python/failure.py:500:throwExceptionIntoGenerator
/usr/bin/synapse_port_db:347:handle_table
/usr/bin/synapse_port_db:502:handle_search_table
/usr/lib/python3.10/site-packages/synapse/storage/database.py:741:runInteraction
/usr/lib/python3.10/site-packages/synapse/storage/database.py:846:runWithConnection
/usr/lib/python3.10/site-packages/twisted/python/threadpool.py:238:inContext
/usr/lib/python3.10/site-packages/twisted/python/threadpool.py:254:<lambda>
/usr/lib/python3.10/site-packages/twisted/python/context.py:118:callWithContext
/usr/lib/python3.10/site-packages/twisted/python/context.py:83:callWithContext
/usr/lib/python3.10/site-packages/twisted/enterprise/adbapi.py:293:_runWithConnection
/usr/lib/python3.10/site-packages/twisted/python/deprecate.py:298:deprecatedFunction
/usr/lib/python3.10/site-packages/twisted/python/compat.py:404:reraise
/usr/lib/python3.10/site-packages/twisted/enterprise/adbapi.py:284:_runWithConnection
/usr/lib/python3.10/site-packages/synapse/storage/database.py:841:inner_func
/usr/lib/python3.10/site-packages/synapse/storage/database.py:609:new_transaction
/usr/bin/synapse_port_db:471:insert
]]
Traceback (most recent call last):
  File "/usr/bin/synapse_port_db", line 701, in run
    await make_deferred_yieldable(
twisted.internet.defer.FirstError: FirstError[#97, [Failure instance: Traceback: <class 'TypeError'>: argument of type 'int' is not iterable
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:701:errback
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:764:_startRunCallbacks
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:858:_runCallbacks
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:1751:gotResult
--- <exception caught here> ---
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:1657:_inlineCallbacks
/usr/lib/python3.10/site-packages/twisted/python/failure.py:500:throwExceptionIntoGenerator
/usr/bin/synapse_port_db:347:handle_table
/usr/bin/synapse_port_db:502:handle_search_table
/usr/lib/python3.10/site-packages/synapse/storage/database.py:741:runInteraction
/usr/lib/python3.10/site-packages/synapse/storage/database.py:846:runWithConnection
/usr/lib/python3.10/site-packages/twisted/python/threadpool.py:238:inContext
/usr/lib/python3.10/site-packages/twisted/python/threadpool.py:254:<lambda>
/usr/lib/python3.10/site-packages/twisted/python/context.py:118:callWithContext
/usr/lib/python3.10/site-packages/twisted/python/context.py:83:callWithContext
/usr/lib/python3.10/site-packages/twisted/enterprise/adbapi.py:293:_runWithConnection
/usr/lib/python3.10/site-packages/twisted/python/deprecate.py:298:deprecatedFunction
/usr/lib/python3.10/site-packages/twisted/python/compat.py:404:reraise
/usr/lib/python3.10/site-packages/twisted/enterprise/adbapi.py:284:_runWithConnection
/usr/lib/python3.10/site-packages/synapse/storage/database.py:841:inner_func
/usr/lib/python3.10/site-packages/synapse/storage/database.py:609:new_transaction
/usr/bin/synapse_port_db:471:insert
]]
FirstError[#97, [Failure instance: Traceback: <class 'TypeError'>: argument of type 'int' is not iterable
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:701:errback
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:764:_startRunCallbacks
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:858:_runCallbacks
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:1751:gotResult
--- <exception caught here> ---
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:1657:_inlineCallbacks
/usr/lib/python3.10/site-packages/twisted/python/failure.py:500:throwExceptionIntoGenerator
/usr/bin/synapse_port_db:347:handle_table
/usr/bin/synapse_port_db:502:handle_search_table
/usr/lib/python3.10/site-packages/synapse/storage/database.py:741:runInteraction
/usr/lib/python3.10/site-packages/synapse/storage/database.py:846:runWithConnection
/usr/lib/python3.10/site-packages/twisted/python/threadpool.py:238:inContext
/usr/lib/python3.10/site-packages/twisted/python/threadpool.py:254:<lambda>
/usr/lib/python3.10/site-packages/twisted/python/context.py:118:callWithContext
/usr/lib/python3.10/site-packages/twisted/python/context.py:83:callWithContext
/usr/lib/python3.10/site-packages/twisted/enterprise/adbapi.py:293:_runWithConnection
/usr/lib/python3.10/site-packages/twisted/python/deprecate.py:298:deprecatedFunction
/usr/lib/python3.10/site-packages/twisted/python/compat.py:404:reraise
/usr/lib/python3.10/site-packages/twisted/enterprise/adbapi.py:284:_runWithConnection
/usr/lib/python3.10/site-packages/synapse/storage/database.py:841:inner_func
/usr/lib/python3.10/site-packages/synapse/storage/database.py:609:new_transaction
/usr/bin/synapse_port_db:471:insert
]]Preparing sqlite3...
Preparing psycopg2...
Creating port tables...
Setting up sequence generators...
Fetching tables...
Checking on port progress...
Copying to postgres...
user_directory: 100% (33573/33573)
users_who_share_private_rooms: 100% (176/176)
user_directory_search: 100% (33573/33573)
user_directory_stream_pos: 100% (1/1)
event_push_summary_stream_ordering: 100% (1/1)
local_group_updates: 100% (2/2)
room_depth: 100% (154/154)
room_account_data: 100% (135/135)
room_stats_state: 100% (154/154)
users_in_public_rooms: 2% (1000/41067)
event_to_state_groups: 46% (445000/963245)
remote_media_cache: 100% (12419/12419)
room_tags: 100% (2/2)
application_services_state: 100% (1/1)
event_auth_chain_to_calculate: 100% (40/40)
state_events: 89% (246376/276591)
device_federation_inbox: 65% (1000/1525)
presence_stream: 47% (5168/10840)
state_groups: 80% (102281/126611)
event_push_actions: 100% (227/227)
room_memberships: 89% (242453/271194)
receipts_graph: 3% (1000/28770)
redactions: 98% (27502/27881)
current_state_events: 1% (1000/69575)
device_lists_outbound_last_success: 33% (1000/2962)
e2e_room_keys: 74% (1000/1336)
user_filters: 100% (5/5)
receipts_linearized: 3% (1000/28771)
threepid_validation_session: 100% (1/1)
push_rules: 100% (13/13)
device_lists_remote_resync: 100% (31/31)
event_push_summary: 100% (1/1)
e2e_one_time_keys_json: 100% (330/330)
event_relations: 1% (1000/51357)
remote_media_cache_thumbnails: 86% (47488/54681)
stream_ordering_to_exterm: 43% (119642/277197)
user_daily_visits: 100% (984/984)
rooms: 100% (159/159)
event_edges: 35% (431000/1209485)
room_aliases: 100% (6/6)
local_current_membership: 100% (184/184)
room_tags_revisions: 100% (3/3)
user_signature_stream: 100% (7/7)
e2e_fallback_keys_json: 100% (5/5)
room_alias_servers: 100% (7/7)
device_inbox: 23% (1000/4339)
e2e_cross_signing_signatures: 100% (18/18)
e2e_room_keys_versions: 100% (1/1)
events: 35% (389000/1086444)
local_media_repository: 100% (216/216)
push_rules_enable: 100% (13/13)
e2e_device_keys_json: 100% (7/7)
device_lists_remote_cache: 1% (1000/73405)
device_lists_remote_extremeties: 7% (1000/13308)
destinations: 22% (1000/4483)
state_group_edges: 80% (97665/120820)
local_group_membership: 100% (1/1)
user_stats_current: 100% (16/16)
event_backward_extremities: 40% (1000/2464)
device_lists_stream: 0% (1000/146908)
server_keys_json: 11% (1000/8770)
```
</details>

The main error seems to be this:

```
Traceback (most recent call last):
  File "/usr/bin/synapse_port_db", line 701, in run
    await make_deferred_yieldable(
twisted.internet.defer.FirstError: FirstError[#97, [Failure instance: Traceback: <class 'TypeError'>: argument of type 'int' is not iterable
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:701:errback
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:764:_startRunCallbacks
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:858:_runCallbacks
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:1751:gotResult
--- <exception caught here> ---
/usr/lib/python3.10/site-packages/twisted/internet/defer.py:1657:_inlineCallbacks
/usr/lib/python3.10/site-packages/twisted/python/failure.py:500:throwExceptionIntoGenerator
/usr/bin/synapse_port_db:347:handle_table
/usr/bin/synapse_port_db:502:handle_search_table
/usr/lib/python3.10/site-packages/synapse/storage/database.py:741:runInteraction
/usr/lib/python3.10/site-packages/synapse/storage/database.py:846:runWithConnection
/usr/lib/python3.10/site-packages/twisted/python/threadpool.py:238:inContext
/usr/lib/python3.10/site-packages/twisted/python/threadpool.py:254:<lambda>
/usr/lib/python3.10/site-packages/twisted/python/context.py:118:callWithContext
/usr/lib/python3.10/site-packages/twisted/python/context.py:83:callWithContext
/usr/lib/python3.10/site-packages/twisted/enterprise/adbapi.py:293:_runWithConnection
/usr/lib/python3.10/site-packages/twisted/python/deprecate.py:298:deprecatedFunction
/usr/lib/python3.10/site-packages/twisted/python/compat.py:404:reraise
/usr/lib/python3.10/site-packages/twisted/enterprise/adbapi.py:284:_runWithConnection
/usr/lib/python3.10/site-packages/synapse/storage/database.py:841:inner_func
/usr/lib/python3.10/site-packages/synapse/storage/database.py:609:new_transaction
/usr/bin/synapse_port_db:471:insert
]]
```

### Steps to reproduce

- `systemctl stop synapse.service`
- `cp --reflink /var/lib/synapse/homeserver.db{,.snapshot}`
- `systemctl start synapse.service`
- `synapse_port_db --sqlite-database /var/lib/synapse/homeserver.db.snapshot --postgres-config /etc/synapse/homeserver_postgres.yaml`

### Version information

- **Homeserver**: xiretza.xyz
- **Version**: 1.50.2
- **Install method**: `pacman -S matrix-synapse`
- **Platform**: Arch Linux


Discussion:

Resolving patch:
diff --git a/synapse/storage/databases/main/events.py b/synapse/storage/databases/main/events.py
--- a/synapse/storage/databases/main/events.py
+++ b/synapse/storage/databases/main/events.py
@@ -1473,10 +1473,10 @@ def _store_rejected_events_txn(self, txn, events_and_contexts):
 
     def _update_metadata_tables_txn(
         self,
-        txn,
+        txn: LoggingTransaction,
         *,
-        events_and_contexts,
-        all_events_and_contexts,
+        events_and_contexts: List[Tuple[EventBase, EventContext]],
+        all_events_and_contexts: List[Tuple[EventBase, EventContext]],
         inhibit_local_membership_updates: bool = False,
     ):
         """Update all the miscellaneous tables for new events
@@ -1953,20 +1953,20 @@ def _handle_redaction(self, txn, redacted_event_id):
             txn, table="event_relations", keyvalues={"event_id": redacted_event_id}
         )
 
-    def _store_room_topic_txn(self, txn, event):
-        if hasattr(event, "content") and "topic" in event.content:
+    def _store_room_topic_txn(self, txn: LoggingTransaction, event: EventBase):
+        if isinstance(event.content.get("topic"), str):
             self.store_event_search_txn(
                 txn, event, "content.topic", event.content["topic"]
             )
 
-    def _store_room_name_txn(self, txn, event):
-        if hasattr(event, "content") and "name" in event.content:
+    def _store_room_name_txn(self, txn: LoggingTransaction, event: EventBase):
+        if isinstance(event.content.get("name"), str):
             self.store_event_search_txn(
                 txn, event, "content.name", event.content["name"]
             )
 
-    def _store_room_message_txn(self, txn, event):
-        if hasattr(event, "content") and "body" in event.content:
+    def _store_room_message_txn(self, txn: LoggingTransaction, event: EventBase):
+        if isinstance(event.content.get("body"), str):
             self.store_event_search_txn(
                 txn, event, "content.body", event.content["body"]
             )
diff --git a/synapse/storage/databases/main/search.py b/synapse/storage/databases/main/search.py
--- a/synapse/storage/databases/main/search.py
+++ b/synapse/storage/databases/main/search.py
@@ -115,6 +115,7 @@ class SearchBackgroundUpdateStore(SearchWorkerStore):
     EVENT_SEARCH_ORDER_UPDATE_NAME = "event_search_order"
     EVENT_SEARCH_USE_GIST_POSTGRES_NAME = "event_search_postgres_gist"
     EVENT_SEARCH_USE_GIN_POSTGRES_NAME = "event_search_postgres_gin"
+    EVENT_SEARCH_DELETE_NON_STRINGS = "event_search_sqlite_delete_non_strings"
 
     def __init__(
         self,
@@ -147,6 +148,10 @@ def __init__(
             self.EVENT_SEARCH_USE_GIN_POSTGRES_NAME, self._background_reindex_gin_search
         )
 
+        self.db_pool.updates.register_background_update_handler(
+            self.EVENT_SEARCH_DELETE_NON_STRINGS, self._background_delete_non_strings
+        )
+
     async def _background_reindex_search(self, progress, batch_size):
         # we work through the events table from highest stream id to lowest
         target_min_stream_id = progress["target_min_stream_id_inclusive"]
@@ -372,6 +377,27 @@ def reindex_search_txn(txn):
 
         return num_rows
 
+    async def _background_delete_non_strings(
+        self, progress: JsonDict, batch_size: int
+    ) -> int:
+        """Deletes rows with non-string `value`s from `event_search` if using sqlite.
+
+        Prior to Synapse 1.44.0, malformed events received over federation could cause integers
+        to be inserted into the `event_search` table when using sqlite.
+        """
+
+        def delete_non_strings_txn(txn: LoggingTransaction) -> None:
+            txn.execute("DELETE FROM event_search WHERE typeof(value) != 'text'")
+
+        await self.db_pool.runInteraction(
+            self.EVENT_SEARCH_DELETE_NON_STRINGS, delete_non_strings_txn
+        )
+
+        await self.db_pool.updates._end_background_update(
+            self.EVENT_SEARCH_DELETE_NON_STRINGS
+        )
+        return 1
+
 
 class SearchStore(SearchBackgroundUpdateStore):
     def __init__(


Test patch:
diff --git a/tests/storage/test_room_search.py b/tests/storage/test_room_search.py
--- a/tests/storage/test_room_search.py
+++ b/tests/storage/test_room_search.py
@@ -13,13 +13,16 @@
 # limitations under the License.
 
 import synapse.rest.admin
+from synapse.api.constants import EventTypes
+from synapse.api.errors import StoreError
 from synapse.rest.client import login, room
 from synapse.storage.engines import PostgresEngine
 
-from tests.unittest import HomeserverTestCase
+from tests.unittest import HomeserverTestCase, skip_unless
+from tests.utils import USE_POSTGRES_FOR_TESTS
 
 
-class NullByteInsertionTest(HomeserverTestCase):
+class EventSearchInsertionTest(HomeserverTestCase):
     servlets = [
         synapse.rest.admin.register_servlets_for_client_rest_resource,
         login.register_servlets,
@@ -72,3 +75,113 @@ def test_null_byte(self):
         )
         if isinstance(store.database_engine, PostgresEngine):
             self.assertIn("alice", result.get("highlights"))
+
+    def test_non_string(self):
+        """Test that non-string `value`s are not inserted into `event_search`.
+
+        This is particularly important when using sqlite, since a sqlite column can hold
+        both strings and integers. When using Postgres, integers are automatically
+        converted to strings.
+
+        Regression test for #11918.
+        """
+        store = self.hs.get_datastores().main
+
+        # Register a user and create a room
+        user_id = self.register_user("alice", "password")
+        access_token = self.login("alice", "password")
+        room_id = self.helper.create_room_as("alice", tok=access_token)
+        room_version = self.get_success(store.get_room_version(room_id))
+
+        # Construct a message with a numeric body to be received over federation
+        # The message can't be sent using the client API, since Synapse's event
+        # validation will reject it.
+        prev_event_ids = self.get_success(store.get_prev_events_for_room(room_id))
+        prev_event = self.get_success(store.get_event(prev_event_ids[0]))
+        prev_state_map = self.get_success(
+            self.hs.get_storage().state.get_state_ids_for_event(prev_event_ids[0])
+        )
+
+        event_dict = {
+            "type": EventTypes.Message,
+            "content": {"msgtype": "m.text", "body": 2},
+            "room_id": room_id,
+            "sender": user_id,
+            "depth": prev_event.depth + 1,
+            "prev_events": prev_event_ids,
+            "origin_server_ts": self.clock.time_msec(),
+        }
+        builder = self.hs.get_event_builder_factory().for_room_version(
+            room_version, event_dict
+        )
+        event = self.get_success(
+            builder.build(
+                prev_event_ids=prev_event_ids,
+                auth_event_ids=self.hs.get_event_auth_handler().compute_auth_events(
+                    builder,
+                    prev_state_map,
+                    for_verification=False,
+                ),
+                depth=event_dict["depth"],
+            )
+        )
+
+        # Receive the event
+        self.get_success(
+            self.hs.get_federation_event_handler().on_receive_pdu(
+                self.hs.hostname, event
+            )
+        )
+
+        # The event should not have an entry in the `event_search` table
+        f = self.get_failure(
+            store.db_pool.simple_select_one_onecol(
+                "event_search",
+                {"room_id": room_id, "event_id": event.event_id},
+                "event_id",
+            ),
+            StoreError,
+        )
+        self.assertEqual(f.value.code, 404)
+
+    @skip_unless(not USE_POSTGRES_FOR_TESTS, "requires sqlite")
+    def test_sqlite_non_string_deletion_background_update(self):
+        """Test the background update to delete bad rows from `event_search`."""
+        store = self.hs.get_datastores().main
+
+        # Populate `event_search` with dummy data
+        self.get_success(
+            store.db_pool.simple_insert_many(
+                "event_search",
+                keys=["event_id", "room_id", "key", "value"],
+                values=[
+                    ("event1", "room_id", "content.body", "hi"),
+                    ("event2", "room_id", "content.body", "2"),
+                    ("event3", "room_id", "content.body", 3),
+                ],
+                desc="populate_event_search",
+            )
+        )
+
+        # Run the background update
+        store.db_pool.updates._all_done = False
+        self.get_success(
+            store.db_pool.simple_insert(
+                "background_updates",
+                {
+                    "update_name": "event_search_sqlite_delete_non_strings",
+                    "progress_json": "{}",
+                },
+            )
+        )
+        self.wait_for_background_updates()
+
+        # The non-string `value`s ought to be gone now.
+        values = self.get_success(
+            store.db_pool.simple_select_onecol(
+                "event_search",
+                {"room_id": "room_id"},
+                "value",
+            ),
+        )
+        self.assertCountEqual(values, ["hi", "2"])