Repository: matrix-org/synapse

Issue:
Traceback when fetching remote media
<!--

**THIS IS NOT A SUPPORT CHANNEL!**
**IF YOU HAVE SUPPORT QUESTIONS ABOUT RUNNING OR CONFIGURING YOUR OWN HOME SERVER**,
please ask in **#synapse:matrix.org** (using a matrix.org account if necessary)

If you want to report a security issue, please see https://matrix.org/security-disclosure-policy/

This is a bug report template. By following the instructions below and
filling out the sections with your information, you will help the us to get all
the necessary data to fix your issue.

You can also preview your report before submitting it. You may remove sections
that aren't relevant to your particular case.

Text between <!-- and --​> marks will be invisible in the report.

-->

### Description

When I was watching the homeserver.log file I noticed the following lines going by.
```text
2021-07-05 22:56:01,678 - synapse.http.server - 93 - ERROR - GET-114720 - Failed handle request via 'ThumbnailResource': <XForwardedForRequest at 0x7f74e7313400 method='GET' uri='/_matrix/media/r0/thumbnail/matrix.org/dOhWUOiqfTEPQASnYCLKLINu?width=800&height=600&method=scale' clientproto='HTTP/1.1' site='8008'>
Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/http/server.py", line 258, in _async_render_wrapper
    callback_return = await self._async_render(request)
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/http/server.py", line 286, in _async_render
    callback_return = await raw_callback_return
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/rest/media/v1/thumbnail_resource.py", line 83, in _async_render_GET
    await self._respond_remote_thumbnail(
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/rest/media/v1/thumbnail_resource.py", line 265, in _respond_remote_thumbnail
    await self._select_and_respond_with_thumbnail(
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/rest/media/v1/thumbnail_resource.py", line 306, in _select_and_respond_with_thumbnail
    file_info = self._select_thumbnail(
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/rest/media/v1/thumbnail_resource.py", line 481, in _select_thumbnail
    thumbnail_info = min(info_list)[-1]
TypeError: '<' not supported between instances of 'dict' and 'dict'
```
I couldn't find an open or closed issue regarding this error so I thought I'd open an issue for it.

### Steps to reproduce

I can reproduce it with curl
```
curl "https://matrix.moritzdietz.com/_matrix/media/r0/thumbnail/matrix.org/dOhWUOiqfTEPQASnYCLKLINu?width=800&height=600&method=scale"
```
which results in the error above.
I can fetch the media just fine via matrix.org:
```bash
curl -L "https://matrix-federation.matrix.org/_matrix/media/r0/thumbnail/matrix.org/dOhWUOiqfTEPQASnYCLKLINu?width=800&height=600&method=scale" -o somemedia.png
```

<!--
Describe how what happens differs from what you expected.

If you can identify any relevant log snippets from _homeserver.log_, please include
those (please be careful to remove any personal or private data). Please surround them with
``` (three backticks, on a line on their own), so that they are formatted legibly.
-->

### Version information

<!-- IMPORTANT: please answer the following questions, to help us narrow down the problem -->

<!-- Was this issue identified on matrix.org or another homeserver? -->
- **Homeserver**: `matrix.moritzdietz.com`

If not matrix.org:

<!--
 What version of Synapse is running?

You can find the Synapse version with this command:

$ curl http://localhost:8008/_synapse/admin/v1/server_version

(You may need to replace `localhost:8008` if Synapse is not configured to
listen on that port.)
-->
- **Version**: `1.37.1`

- **Install method**: `package manager`
<!-- examples: package manager/git clone/pip  -->

- **Platform**: `Ubuntu 20.04 Server`
<!--
Tell us about the environment in which your homeserver is operating
distro, hardware, if it's running in a vm/container, etc.
-->
I have `dynamic_thumbnails` set to false (as it's the default).


Discussion:
What is your `thumbnail_sizes` config? I think you might hit this if it has duplicate values in it?
I checked my yaml file and in neither the one shipped with my package manager nor the drop in yaml file in `.conf.d/` has an activated yaml hash for that configuration.
Meaning: everything is commented out which controls the setting you mentioned.

My reasoning was that there must be some default and I can leave it like it is unless I really need to change or adapt it.
The issue is here: https://github.com/matrix-org/synapse/blob/4b965c862dc66c0da5d3240add70e9b5f0aa720b/synapse/rest/media/v1/thumbnail_resource.py#L481, `info_list` is a list of tuples, where the last entry is a dict. I think what is happening is we're getting entries in there where all the numbers match, and so it tries and compares by the last entry in the tuple, which then fails. The fix would be to either a) deduplicate the list before sorting or b) using the `key` param to `min` to make the comparisons ignore the dict part.

Resolving patch:
diff --git a/synapse/rest/media/v1/thumbnail_resource.py b/synapse/rest/media/v1/thumbnail_resource.py
--- a/synapse/rest/media/v1/thumbnail_resource.py
+++ b/synapse/rest/media/v1/thumbnail_resource.py
@@ -15,7 +15,7 @@
 
 
 import logging
-from typing import TYPE_CHECKING, Any, Dict, List, Optional
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple
 
 from twisted.web.server import Request
 
@@ -414,9 +414,9 @@ def _select_thumbnail(
 
         if desired_method == "crop":
             # Thumbnails that match equal or larger sizes of desired width/height.
-            crop_info_list = []
+            crop_info_list: List[Tuple[int, int, int, bool, int, Dict[str, Any]]] = []
             # Other thumbnails.
-            crop_info_list2 = []
+            crop_info_list2: List[Tuple[int, int, int, bool, int, Dict[str, Any]]] = []
             for info in thumbnail_infos:
                 # Skip thumbnails generated with different methods.
                 if info["thumbnail_method"] != "crop":
@@ -451,15 +451,19 @@ def _select_thumbnail(
                             info,
                         )
                     )
+            # Pick the most appropriate thumbnail. Some values of `desired_width` and
+            # `desired_height` may result in a tie, in which case we avoid comparing on
+            # the thumbnail info dictionary and pick the thumbnail that appears earlier
+            # in the list of candidates.
             if crop_info_list:
-                thumbnail_info = min(crop_info_list)[-1]
+                thumbnail_info = min(crop_info_list, key=lambda t: t[:-1])[-1]
             elif crop_info_list2:
-                thumbnail_info = min(crop_info_list2)[-1]
+                thumbnail_info = min(crop_info_list2, key=lambda t: t[:-1])[-1]
         elif desired_method == "scale":
             # Thumbnails that match equal or larger sizes of desired width/height.
-            info_list = []
+            info_list: List[Tuple[int, bool, int, Dict[str, Any]]] = []
             # Other thumbnails.
-            info_list2 = []
+            info_list2: List[Tuple[int, bool, int, Dict[str, Any]]] = []
 
             for info in thumbnail_infos:
                 # Skip thumbnails generated with different methods.
@@ -477,10 +481,14 @@ def _select_thumbnail(
                     info_list2.append(
                         (size_quality, type_quality, length_quality, info)
                     )
+            # Pick the most appropriate thumbnail. Some values of `desired_width` and
+            # `desired_height` may result in a tie, in which case we avoid comparing on
+            # the thumbnail info dictionary and pick the thumbnail that appears earlier
+            # in the list of candidates.
             if info_list:
-                thumbnail_info = min(info_list)[-1]
+                thumbnail_info = min(info_list, key=lambda t: t[:-1])[-1]
             elif info_list2:
-                thumbnail_info = min(info_list2)[-1]
+                thumbnail_info = min(info_list2, key=lambda t: t[:-1])[-1]
 
         if thumbnail_info:
             return FileInfo(


Test patch:
diff --git a/tests/rest/media/v1/test_media_storage.py b/tests/rest/media/v1/test_media_storage.py
--- a/tests/rest/media/v1/test_media_storage.py
+++ b/tests/rest/media/v1/test_media_storage.py
@@ -21,7 +21,7 @@
 from urllib import parse
 
 import attr
-from parameterized import parameterized_class
+from parameterized import parameterized, parameterized_class
 from PIL import Image as Image
 
 from twisted.internet import defer
@@ -473,6 +473,43 @@ def _test_thumbnail(self, method, expected_body, expected_found):
                 },
             )
 
+    @parameterized.expand([("crop", 16), ("crop", 64), ("scale", 16), ("scale", 64)])
+    def test_same_quality(self, method, desired_size):
+        """Test that choosing between thumbnails with the same quality rating succeeds.
+
+        We are not particular about which thumbnail is chosen."""
+        self.assertIsNotNone(
+            self.thumbnail_resource._select_thumbnail(
+                desired_width=desired_size,
+                desired_height=desired_size,
+                desired_method=method,
+                desired_type=self.test_image.content_type,
+                # Provide two identical thumbnails which are guaranteed to have the same
+                # quality rating.
+                thumbnail_infos=[
+                    {
+                        "thumbnail_width": 32,
+                        "thumbnail_height": 32,
+                        "thumbnail_method": method,
+                        "thumbnail_type": self.test_image.content_type,
+                        "thumbnail_length": 256,
+                        "filesystem_id": f"thumbnail1{self.test_image.extension}",
+                    },
+                    {
+                        "thumbnail_width": 32,
+                        "thumbnail_height": 32,
+                        "thumbnail_method": method,
+                        "thumbnail_type": self.test_image.content_type,
+                        "thumbnail_length": 256,
+                        "filesystem_id": f"thumbnail2{self.test_image.extension}",
+                    },
+                ],
+                file_id=f"image{self.test_image.extension}",
+                url_cache=None,
+                server_name=None,
+            )
+        )
+
     def test_x_robots_tag_header(self):
         """
         Tests that the `X-Robots-Tag` header is present, which informs web crawlers