Repository: searx/searx

Issue:
Wikipedia crash on specific search query
Instance: searx.be
Browser: Chrome Version 84.0.4147.89

Hello,

Not sure whether this is a bug, problem with the instance itself , or my browser, but when I search for certain query e.g. `port dover`, I get an error message:
```
Engines cannot retrieve results:
wikipedia (unexpected crash 'NoneType' object has no attribute 'replace')
```

![Image 6](https://user-images.githubusercontent.com/8424198/88169288-fc8b0380-cc45-11ea-98ea-c1b897b40273.png)



This error doesn't happen if I change the language to Deutsch, Espanol, or other language. Seems like it only happens with English language.  


![Image 2](https://user-images.githubusercontent.com/8424198/88170044-33154e00-cc47-11ea-8560-2711f8323bd6.png)


![Image 3](https://user-images.githubusercontent.com/8424198/88170063-3d374c80-cc47-11ea-9ad6-5bd75e1eaa18.png)



This error doesn't happen with other search query.

![Image 4](https://user-images.githubusercontent.com/8424198/88170309-ad45d280-cc47-11ea-85bc-e7ce87ba76a3.png)


I also test this on other searx instances and the results are mixed. 

No error:
https://searx.ninja
https://search.snopyta.org/
Both are still on 0.16.0

Same error:
https://searx.fmac.xyz
https://searx.xyz/




Discussion:
> Not sure whether this is a bug, problem with the instance itself , or my browser ..

Its a problem of the instances.
Oh ok thanks
> Oh ok thanks

No problem, feel welcome ..  if you are looking for well maintained instances .. you can use [searx.be from @unifox ](https://github.com/asciimoo/searx/issues/1977#issuecomment-636300719) or even mine [darmarit.org/searx](https://github.com/asciimoo/searx/issues/1977#issuecomment-636293224).
Seems like your instance also gives the same error. But no error when I change to other language.

![Image 5](https://user-images.githubusercontent.com/8424198/88175713-73c59500-cc50-11ea-8936-954c34d94103.png)

> Seems like your instance also gives the same error. 

Wow, haven't seen this before (since I use de) / thanks for the hint!

I reopen the issue / if I have time, I will check twice what is going wrong here.


I broke this with #1686. The obvious solution would be to simply validate that `summary` is not `None`, which would cause searx to return the infobox with no error but also with no paragraph. That's because the [extract_first_paragraph](https://github.com/asciimoo/searx/blob/master/searx/engines/wikipedia.py#L52) function in the wikipedia engine is a mess. 
I'm going to check out the Wikipedia API again and try a bunch of tests, because I think in the last years they have fixed a lot of the issues that our code was working around.
Also search query "WSGI" crash wikipedia with same error: `Engines cannot retrieve results: wikipedia (unexpected crash 'NoneType' object has no attribute 'replace')`. I am using SearX on Ubuntu 18.04 on localhost installed with uWSGI. Using English as default language. Tried different themes, same result.
Made some research and figured out that sometimes wikipedia api don't return full info. For example, for query "wsgi" it returns this (`resp.text`):
`{"batchcomplete":"","query":{"normalized":[{"from":"wsgi","to":"Wsgi"}],"redirects":[{"from":"Wsgi","to":"Web Server Gateway Interface"}],"pages":{"1441500":{"pageid":1441500,"ns":0,"title":"Web Server Gateway Interface","extract":""}}}}`
Please note that `extract` is empty. I think Searx Wikipedia engine should follow redirects given from Wikipedia API.
Made an API request manually with Python, response is correct. Perhaps something is wrong with SearX html parser.
Edit: I discovered that for some reason function `search_one_http_request` (line 84, file `searx/search.py`) receives that empty response from API

Resolving patch:
diff --git a/searx/engines/wikipedia.py b/searx/engines/wikipedia.py
--- a/searx/engines/wikipedia.py
+++ b/searx/engines/wikipedia.py
@@ -49,29 +49,6 @@ def request(query, params):
     return params
 
 
-# get first meaningful paragraph
-# this should filter out disambiguation pages and notes above first paragraph
-# "magic numbers" were obtained by fine tuning
-def extract_first_paragraph(content, title, image):
-    first_paragraph = None
-
-    failed_attempts = 0
-    for paragraph in content.split('\n'):
-
-        starts_with_title = paragraph.lower().find(title.lower(), 0, len(title) + 35)
-        length = len(paragraph)
-
-        if length >= 200 or (starts_with_title >= 0 and (image or length >= 150)):
-            first_paragraph = paragraph
-            break
-
-        failed_attempts += 1
-        if failed_attempts > 3:
-            return None
-
-    return first_paragraph
-
-
 # get response from search-request
 def response(resp):
     results = []
@@ -97,10 +74,7 @@ def response(resp):
     if image:
         image = image.get('source')
 
-    extract = page.get('extract')
-
-    summary = extract_first_paragraph(extract, title, image)
-    summary = summary.replace('() ', '')
+    summary = page.get('extract', '').split('\n')[0].replace('()', '')
 
     # link to wikipedia article
     wikipedia_link = base_url.format(language=url_lang(resp.search_params['language'])) \


Test patch: