Repository: searxng/searxng

Issue:
Images in instant answer are not loaded if "image proxy" is enables

SearXNG Version: 1.0.0-1669-10e6881c

Simple theme is used

**How did you install SearXNG?**

SearxNG was installed by "installation scripts", without Morty or Filtron.

**What happened?**

If  "image proxy" is enabled, images searches work correctly. However, images in "Instant Answers" under the  "General" category are absent and just a broken image placeholder is shown.

For instance, if I search for "Enrico Fermi" in the General category, with the Image Proxy enabled Wikipedia's image is broken.

**How To Reproduce**
This bug is reproducible. Just enable "Image Proxy" an settings GUI and perform the aforementioned search

**Expected behavior**

An image should be displayed in place of the broken image placeholder

**Screenshots & Logs**

Expected result:
![expected](https://user-images.githubusercontent.com/25054477/152677025-85823d66-728c-4f6b-850a-9722c67bc474.png)

Actual result:
![actual](https://user-images.githubusercontent.com/25054477/152677023-0821723f-bae7-44db-a9a8-c46d47e75ebb.png)


Inspection of the page source gives the following:

`<img src="/image_proxy?url=https%3A%2F%2Fcommons.wikimedia.org%2Fwiki%2FSpecial%3AFilePath%2FEnrico%2520Fermi%25201943-49.jpg%3Fwidth%3D500%26height%3D400&amp;h=dff389296b41618681172e6ca9f6a4e50b6ad32554484ff828889e64f4a419ab" title="Enrico Fermi" alt="Enrico Fermi">`

Somewhat "image_proxy" is misbehaving on the home page.




Discussion:
I can confirm that the image is not loaded:

```
INFO    werkzeug                      : 127.0.0.1 - - [06/Feb/2022 12:24:41] "GET /static/themes/simple/js/searxng.min.js HTTP/1.1" 304 -
DEBUG   searx.network.image_proxy     : HTTP Request: GET https://commons.wikimedia.org/wiki/Special:FilePath/Enrico%20Fermi%201943-49.jpg?width=500&height=400 "HTTP/1.1 302 Found" (text/html; charset=utf-8)
DEBUG   searx.webapp                  : image-proxy: wrong response code: 302
INFO    werkzeug                      : 127.0.0.1 - - [06/Feb/2022 12:24:41] "GET /image_proxy?url=https%3A%2F%2Fcommons.wikimedia.org%2Fwiki%2FSpecial%3AFilePath%2FEnrico%2520Fermi%25201943-49.jpg%3Fwidth%3D500%26height%3D400&h=c5affd8c6746b4d64eb63447da9f4138674fcab7dbc0a60576267cbea2232578 HTTP/1.1" 400 -
```

But the URL (now without image_proxy) ...

![grafik](https://user-images.githubusercontent.com/554536/152678861-641a58ee-e9bb-451d-b79d-9748df5de509.png)

is valid: https://commons.wikimedia.org/wiki/Special:FilePath/Enrico%20Fermi%201943-49.jpg?width=500&height=400

~~I can't reproduce this.~~

If wikidata is enabled the image is broken.

See https://searx.tiekoetter.com/search?q=%21wikidata+enrico+fermi&language=en-US&time_range=&safesearch=0&theme=simple
> ~I can't reproduce this.~
> 
> If wikidata is enabled the image is broken.
> 
> See https://searx.tiekoetter.com/search?q=%21wikidata+enrico+fermi&language=en-US&time_range=&safesearch=0&theme=simple

Indeed, this it it. Disabling wikidata, everything works as expected on my instance.
Thank you for sharing this finding.
@return42 Does the image proxy have a size limit?
It seems the img URL from Wikidata's PNG is redirected (twice)

https://commons.wikimedia.org/wiki/Special:FilePath/Enrico%20Fermi%201943-49.jpg?width=500&height=400

![grafik](https://user-images.githubusercontent.com/554536/152684337-7d4e5752-6eda-49a6-942c-45c5b8437b90.png)

while the URL of the JPG from wikipedia is not redirected

https://upload.wikimedia.org/wikipedia/commons/thumb/d/d4/Enrico_Fermi_1943-49.jpg/258px-Enrico_Fermi_1943-49.jpg

![grafik](https://user-images.githubusercontent.com/554536/152684434-a5950ec8-651a-43c5-ae8f-8bd7bf8760e0.png)


I am currently working on a solution. I think I can do a PR soon.
> I am currently working on a solution. I think I can do a PR soon.

Thanks .. I also working on a solution ;-)  ..  I just stuck at the meaning of priority . and why it must be lower:

https://github.com/searxng/searxng/blob/10e6881ca861cfce70e99ededf2c864a9274b5dd/searx/engines/wikidata.py#L223-L225

----
EDIT: I think image proxy should support redirects
> EDIT: I think image proxy should support redirects

Maybe but then image proxy would do 2 redirects for every wikidata image. I don't think this would be good for performance.
> Maybe but then image proxy would do 2 redirects for every wikidata image. I don't think this would be good for performance.

This arguments counts .. I just think about rewrite the image URL in the wikidata engine ... needs some more analysis .. I just digging :)

Resolving patch:
diff --git a/searx/engines/wikidata.py b/searx/engines/wikidata.py
--- a/searx/engines/wikidata.py
+++ b/searx/engines/wikidata.py
@@ -1,10 +1,11 @@
 # SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""Wikidata
 """
- Wikidata
-"""
-
+# pylint: disable=missing-class-docstring
 
-from urllib.parse import urlencode
+from hashlib import md5
+from urllib.parse import urlencode, unquote
 from json import loads
 
 from dateutil.parser import isoparse
@@ -185,7 +186,51 @@ def response(resp):
     return results
 
 
+_IMG_SRC_DEFAULT_URL_PREFIX = "https://commons.wikimedia.org/wiki/Special:FilePath/"
+_IMG_SRC_NEW_URL_PREFIX = "https://upload.wikimedia.org/wikipedia/commons/thumb/"
+
+
+def get_thumbnail(img_src):
+    """Get Thumbnail image from wikimedia commons
+
+    Images from commons.wikimedia.org are (HTTP) redirected to
+    upload.wikimedia.org.  The redirected URL can be calculated by this
+    function.
+
+    - https://stackoverflow.com/a/33691240
+
+    """
+    logger.debug('get_thumbnail(): %s', img_src)
+    if not img_src is None and _IMG_SRC_DEFAULT_URL_PREFIX in img_src.split()[0]:
+        img_src_name = unquote(img_src.replace(_IMG_SRC_DEFAULT_URL_PREFIX, "").split("?", 1)[0].replace("%20", "_"))
+        img_src_name_first = img_src_name
+        img_src_name_second = img_src_name
+
+        if ".svg" in img_src_name.split()[0]:
+            img_src_name_second = img_src_name + ".png"
+
+        img_src_size = img_src.replace(_IMG_SRC_DEFAULT_URL_PREFIX, "").split("?", 1)[1]
+        img_src_size = img_src_size[img_src_size.index("=") + 1 : img_src_size.index("&")]
+        img_src_name_md5 = md5(img_src_name.encode("utf-8")).hexdigest()
+        img_src = (
+            _IMG_SRC_NEW_URL_PREFIX
+            + img_src_name_md5[0]
+            + "/"
+            + img_src_name_md5[0:2]
+            + "/"
+            + img_src_name_first
+            + "/"
+            + img_src_size
+            + "px-"
+            + img_src_name_second
+        )
+        logger.debug('get_thumbnail() redirected: %s', img_src)
+
+    return img_src
+
+
 def get_results(attribute_result, attributes, language):
+    # pylint: disable=too-many-branches
     results = []
     infobox_title = attribute_result.get('itemLabel')
     infobox_id = attribute_result['item']
@@ -194,7 +239,7 @@ def get_results(attribute_result, attributes, language):
     infobox_attributes = []
     infobox_content = attribute_result.get('itemDescription', [])
     img_src = None
-    img_src_priority = 100
+    img_src_priority = 0
 
     for attribute in attributes:
         value = attribute.get_str(attribute_result, language)
@@ -220,8 +265,8 @@ def get_results(attribute_result, attributes, language):
                 # this attribute is an image.
                 # replace the current image only the priority is lower
                 # (the infobox contain only one image).
-                if attribute.priority < img_src_priority:
-                    img_src = value
+                if attribute.priority > img_src_priority:
+                    img_src = get_thumbnail(value)
                     img_src_priority = attribute.priority
             elif attribute_type == WDGeoAttribute:
                 # geocoordinate link
@@ -278,6 +323,7 @@ def get_query(query, language):
 
 
 def get_attributes(language):
+    # pylint: disable=too-many-statements
     attributes = []
 
     def add_value(name):
@@ -418,7 +464,7 @@ def add_date(name):
 
 
 class WDAttribute:
-
+    # pylint: disable=no-self-use
     __slots__ = ('name',)
 
     def __init__(self, name):
@@ -439,7 +485,7 @@ def get_wikibase_label(self):
     def get_group_by(self):
         return ""
 
-    def get_str(self, result, language):
+    def get_str(self, result, language):  # pylint: disable=unused-argument
         return result.get(self.name + 's')
 
     def __repr__(self):
@@ -580,6 +626,7 @@ def __init__(self, name, url_id=None, priority=100):
 
 
 class WDDateAttribute(WDAttribute):
+    # pylint: disable=no-self-use
     def get_select(self):
         return '?{name} ?{name}timePrecision ?{name}timeZone ?{name}timeCalendar'.replace('{name}', self.name)
 
@@ -600,7 +647,7 @@ def get_where(self):
     def get_group_by(self):
         return self.get_select()
 
-    def format_8(self, value, locale):
+    def format_8(self, value, locale):  # pylint: disable=unused-argument
         # precision: less than a year
         return value
 
@@ -673,7 +720,7 @@ def get_str(self, result, language):
                     else:
                         value = t[0]
                 return format_method(value, language)
-            except Exception:
+            except Exception:  # pylint: disable=broad-except
                 return value
         return value
 
@@ -687,7 +734,7 @@ def debug_explain_wikidata_query(query, method='GET'):
     return http_response.content
 
 
-def init(engine_settings=None):
+def init(engine_settings=None):  # pylint: disable=unused-argument
     # WIKIDATA_PROPERTIES : add unit symbols
     WIKIDATA_PROPERTIES.update(WIKIDATA_UNITS)
 


Test patch: