Repository: searx/searx

Issue:
Requests does not follow on an HTTP 301
We had already an issue when kickass redirected from https to http I seem to remember. Or it was on another engine. And the request wasn't followed so we had an empty response with status_code 301 in the engine.

Anyway, that is still the case in kickass if we try to perform a search when the query contains capitalized letters.

So why isn't 301 followed ? Is there a way to force it ?



Discussion:
Seems strange since it was the case for the google engine : #193
There is an allow_redirects parameter for the Requests

Yes, I know, and it is set at True (it is set in poolrequest). But in some cases, it doesn't follow.

@Cqoicebordel could you show a reproducable example?

Works : http://searx.me/?q=%21ka+22+jump+street&category_general=on
Doesn't work : http://searx.me/?q=%21ka+22+Jump+street&category_general=on

The only difference is the capitalized J.
Kickass tolower() it to put it in URL. And so, does a redirection in the second case, returning a 301 that is not followed.

seems, it's fixed, thanks @pointhi =)

Uh ? Where ?

ah, right, it isn't..


Resolving patch:
diff --git a/searx/engines/kickass.py b/searx/engines/kickass.py
--- a/searx/engines/kickass.py
+++ b/searx/engines/kickass.py
@@ -20,7 +20,7 @@
 paging = True
 
 # search-url
-url = 'https://kickass.so/'
+url = 'https://kickass.to/'
 search_url = url + 'search/{search_term}/{pageno}/'
 
 # specific xpath variables
@@ -45,6 +45,11 @@ def request(query, params):
 def response(resp):
     results = []
 
+    # check if redirect comparing to the True value,
+    # because resp can be a Mock object, and any attribut name returns something.
+    if resp.is_redirect is True:
+        return results
+
     dom = html.fromstring(resp.text)
 
     search_res = dom.xpath('//table[@class="data"]//tr')


Test patch:
diff --git a/searx/tests/engines/test_kickass.py b/searx/tests/engines/test_kickass.py
--- a/searx/tests/engines/test_kickass.py
+++ b/searx/tests/engines/test_kickass.py
@@ -14,7 +14,7 @@ def test_request(self):
         params = kickass.request(query, dicto)
         self.assertIn('url', params)
         self.assertIn(query, params['url'])
-        self.assertIn('kickass.so', params['url'])
+        self.assertIn('kickass.to', params['url'])
         self.assertIn('verify', params)
         self.assertFalse(params['verify'])
 
@@ -98,7 +98,7 @@ def test_response(self):
         self.assertEqual(type(results), list)
         self.assertEqual(len(results), 1)
         self.assertEqual(results[0]['title'], 'This should be the title')
-        self.assertEqual(results[0]['url'], 'https://kickass.so/url.html')
+        self.assertEqual(results[0]['url'], 'https://kickass.to/url.html')
         self.assertEqual(results[0]['content'], 'Posted by riri in Other &gt; Unsorted')
         self.assertEqual(results[0]['seed'], 10)
         self.assertEqual(results[0]['leech'], 1)
@@ -381,7 +381,7 @@ def test_response(self):
         self.assertEqual(type(results), list)
         self.assertEqual(len(results), 5)
         self.assertEqual(results[0]['title'], 'This should be the title')
-        self.assertEqual(results[0]['url'], 'https://kickass.so/url.html')
+        self.assertEqual(results[0]['url'], 'https://kickass.to/url.html')
         self.assertEqual(results[0]['content'], 'Posted by riri in Other &gt; Unsorted')
         self.assertEqual(results[0]['seed'], 10)
         self.assertEqual(results[0]['leech'], 1)