Repository: searx/searx

Issue:
Get image button from the Google Images engine doesn't give the raw URL to the image
When searching for images using the "Google Images" engine, the "Get image" button doesn't give the raw URL (that usually ends with `.jpg` or `.png`) to the image of every result.
In fact the button gives the same exact URL as the button "View source".

On other engines (like Bing images) it gives the raw URL to the image.

Here is a screenshot explaining the issue:
![getimagegoogleimages](https://user-images.githubusercontent.com/4016501/88662178-e19f1000-d0d9-11ea-8893-a7fc9798b82c.png)

Here is a URL for testing the bug: https://darmarit.org/searx/?q=%21goi+ok&time_range=&language=en-US&category_images=on

@return42 Any thoughts about that?



Discussion:
I have to look at that, I remember that I was a bit confused about the different result URLs at some time. May be I made a mistake .. unfortunately I have no time this week to clarify / hope next week I can have a eye on it.
+1, issue appears only with google images, while other engines, such as qwant, duckduckgo and bing... work properly
I found out how to get real image url from search page.
Enter query at google images (Searx already do it)
![Screenshot from 2020-08-01 17-57-18](https://user-images.githubusercontent.com/66268477/89104328-441d4680-d421-11ea-8062-bfce36b81de1.png)
Inspect element by clicking right mouse button on image (Searx already search image element in request) and find image parents property `data-tbnid` or `data-id` (Searx don't do this, it retrieves `data:url`):
![Screenshot from 2020-08-01 17-57-34](https://user-images.githubusercontent.com/66268477/89104411-f81ed180-d421-11ea-8df0-6f3b05fffe48.png)
Then search this id in `<script nonce>` tag (in chrome devtools it is done by going to Source tab then `Ctrl+F` and pasting id). A line below will contain image URL (for me it was fourth occurrence):
![Screenshot from 2020-08-01 17-59-35](https://user-images.githubusercontent.com/66268477/89104574-f1dd2500-d422-11ea-828e-dbca7c9e3a78.png)
As you can see it is not small thumbnail image:
![Screenshot from 2020-08-01 18-01-32](https://user-images.githubusercontent.com/66268477/89104584-04575e80-d423-11ea-85cd-489ce8f8e0bc.png)
All we need is to implement obtaining that `data-id` and searching this in source code. But this can slow down image search. Or google images searx engine can just go through all these URLs at `<script nonce>` and return it to user. There are all required info for image (full URL, webpage where it is and title) :thinking: ...
@gangural do you mind creating a PR for that?
@unixfox ok, I will do it if I succeed.
Almost done :smiley: ! Have to fix some bug that was caused by my code, will do it tomorrow

Resolving patch:
diff --git a/searx/engines/google_images.py b/searx/engines/google_images.py
--- a/searx/engines/google_images.py
+++ b/searx/engines/google_images.py
@@ -24,6 +24,7 @@
 
 """
 
+import urllib
 from lxml import html
 from flask_babel import gettext
 from searx import logger
@@ -77,6 +78,19 @@ def scrap_out_thumbs(dom):
     return ret_val
 
 
+def scrap_img_by_id(script, data_id):
+    """Get full image URL by data-id in parent element
+    """
+    img_url = ''
+    _script = script.split('\n')
+    for i, line in enumerate(_script):
+        if 'gstatic.com/images' in line and data_id in line:
+            url_line = _script[i + 1]
+            img_url = url_line.split('"')[1]
+            img_url = urllib.parse.unquote(img_url.replace(r'\u00', r'%'))
+    return img_url
+
+
 def request(query, params):
     """Google-Video search request"""
 
@@ -133,6 +147,7 @@ def response(resp):
     # convert the text to dom
     dom = html.fromstring(resp.text)
     img_bas64_map = scrap_out_thumbs(dom)
+    img_src_script = eval_xpath(dom, '//script[contains(., "AF_initDataCallback({key: ")]')[1].text
 
     # parse results
     #
@@ -142,8 +157,7 @@ def response(resp):
     #     <div jsmodel="tTXmib"> / <div jsaction="..." data-id="..."
     #     The data-id matches to a item in a json-data structure in::
     #         <script nonce="I+vqelcy/01CKiBJi5Z1Ow">AF_initDataCallback({key: 'ds:1', ... data:function(){return [ ...
-    #     In this structure the ling to the origin PNG, JPG or whatever is given
-    #     (we do not blow out the link there, you could still implement that)
+    #     In this structure the link to the origin PNG, JPG or whatever is given
     # first link per image-div contains a <img> with the data-iid for bas64 encoded image data::
     #      <img class="rg_i Q4LuWd" data-iid="0"
     # second link per image-div is the target link::
@@ -186,12 +200,17 @@ def response(resp):
                 pub_descr = extract_text(pub_nodes[0])
                 pub_source = extract_text(pub_nodes[1])
 
+            img_src_id = eval_xpath(img_node, '../../../@data-id')[0]
+            src_url = scrap_img_by_id(img_src_script, img_src_id)
+            if not src_url:
+                src_url = thumbnail_src
+
             results.append({
                 'url': url,
                 'title': img_alt,
                 'content': pub_descr,
                 'source': pub_source,
-                'img_src': url,
+                'img_src': src_url,
                 # 'img_format': img_format,
                 'thumbnail_src': thumbnail_src,
                 'template': 'images.html'


Test patch: