Repository: matrix-org/synapse

Issue:
`TestMediaFilenames` test fails in Complement when using workers
```python
synapse_media_repository1 | 2022-05-19 15:21:54,990 - synapse.http.server - 183 - ERROR - POST-4 - Failed handle request via 'UploadResource': <SynapseRequest at 0x7f8b1c5cdd30 method='POST' uri='/_matrix/media/r0/upload?filename=%F0%9F%90%94' clientproto='HTTP/1.0' site='18015'>
synapse_media_repository1 | Traceback (most recent call last):
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 366, in _async_render_wrapper
synapse_media_repository1 |     callback_return = await self._async_render(request)
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 396, in _async_render
synapse_media_repository1 |     callback_return = await raw_callback_return
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/rest/media/v1/upload_resource.py", line 96, in _async_render_POST
synapse_media_repository1 |     content_uri = await self.media_repo.create_content(
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/rest/media/v1/media_repository.py", line 191, in create_content
synapse_media_repository1 |     await self._generate_thumbnails(None, media_id, media_id, media_type)
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/rest/media/v1/media_repository.py", line 832, in _generate_thumbnails
synapse_media_repository1 |     await self.store.store_local_thumbnail(
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/media_repository.py", line 437, in store_local_thumbnail
synapse_media_repository1 |     await self.db_pool.simple_upsert(
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 1119, in simple_upsert
synapse_media_repository1 |     return await self.runInteraction(
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 832, in runInteraction
synapse_media_repository1 |     return await delay_cancellation(_runInteraction())
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1656, in _inlineCallbacks
synapse_media_repository1 |     result = current_context.run(
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/twisted/python/failure.py", line 514, in throwExceptionIntoGenerator
synapse_media_repository1 |     return g.throw(self.type, self.value, self.tb)
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 804, in _runInteraction
synapse_media_repository1 |     result = await self.runWithConnection(
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 927, in runWithConnection
synapse_media_repository1 |     return await make_deferred_yieldable(
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/twisted/python/threadpool.py", line 244, in inContext
synapse_media_repository1 |     result = inContext.theWork()  # type: ignore[attr-defined]
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/twisted/python/threadpool.py", line 260, in <lambda>
synapse_media_repository1 |     inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/twisted/python/context.py", line 117, in callWithContext
synapse_media_repository1 |     return self.currentContext().callWithContext(ctx, func, *args, **kw)
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/twisted/python/context.py", line 82, in callWithContext
synapse_media_repository1 |     return func(*args, **kw)
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
synapse_media_repository1 |     result = func(conn, *args, **kw)
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 920, in inner_func
synapse_media_repository1 |     return func(db_conn, *args, **kwargs)
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 668, in new_transaction
synapse_media_repository1 |     r = func(cursor, *args, **kwargs)
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 1168, in simple_upsert_txn
synapse_media_repository1 |     return self.simple_upsert_txn_native_upsert(
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 1294, in simple_upsert_txn_native_upsert
synapse_media_repository1 |     txn.execute(sql, list(allvalues.values()))
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 350, in execute
synapse_media_repository1 |     self._do_execute(self.txn.execute, sql, *args)
synapse_media_repository1 |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 392, in _do_execute
synapse_media_repository1 |     return func(sql, *args, **kwargs)
synapse_media_repository1 | psycopg2.errors.InvalidColumnReference: there is no unique or exclusion constraint matching the ON CONFLICT specification
```

At first glance, it looks like the upsert might be trying to take place before the background index is created.
We probably need to register the table as unsafe to upsert whilst the index is pending, like we've done for others.


Discussion:
Seems to be the same for `TestLocalPngThumbnail` and probably the remote one too.

Resolving patch:
diff --git a/synapse/storage/database.py b/synapse/storage/database.py
--- a/synapse/storage/database.py
+++ b/synapse/storage/database.py
@@ -90,6 +90,8 @@
     "device_lists_remote_extremeties": "device_lists_remote_extremeties_unique_idx",
     "device_lists_remote_cache": "device_lists_remote_cache_unique_idx",
     "event_search": "event_search_event_id_idx",
+    "local_media_repository_thumbnails": "local_media_repository_thumbnails_method_idx",
+    "remote_media_cache_thumbnails": "remote_media_repository_thumbnails_method_idx",
 }
 
 


Test patch: