Repository: searxng/searxng

Issue:
Bug: bing sometimes missing first result
**Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG**
Repository: https://github.com/searxng/searxng
Branch: master
Version: 2022.12.30-647a0aa9


**How did you install SearXNG?**
script, also occurs in some public instances
**What happened?**
Bing is sometimes missing the first result.

**How To Reproduce**
`!bi pontiac aztek`

No Wikipedia article returned, 9 results are listed.

`!bi pontiac aztek wiki`

Multiple Wikipedia articles are returned, none of which are the English article for the Pontiac Aztek. (The Simple English and Scots articles appear.) 9 results are listed.

**Expected behavior**
The expected first result is the Wikipedia article for Pontiac Aztek. Bing.com returns 10 results on the first page, so SearXNG should return 10 results on its first page.

**Screenshots & Logs**
![SearXNG search](https://user-images.githubusercontent.com/1284317/210190306-e0f1bc25-9686-4ac4-8b31-8277c92b6d5b.png)
![Bing.com search](https://user-images.githubusercontent.com/1284317/210190309-83427d11-0893-4677-9ffd-93231b3e11a0.png)

**Additional context**
It seems consistent for specific search terms, but it doesn't always happen. My suspicion is that when the first result is fancy, such as a Wikipedia infobox, SearXNG doesn't see it.


Discussion:
Hello. I did some digging on this and I think I found out the reason.

The searx's bing engine seek the search results from bing that match the following xpath: `'//li[@class="b_algo"]'` as we can see in the following line of the `searx/engines/bing.py`:

https://github.com/searxng/searxng/blob/647a0aa98bd1d2000eb1ca5270e017d85d5c3463/searx/engines/bing.py#L87

Now, if we check the bing search results for the "pontiac aztek", we'll see that the Wikipedia result's element has a different class:

<img width="1532" alt="Screenshot 2023-01-03 at 08 14 13" src="https://user-images.githubusercontent.com/22837764/210314185-73e1a7ad-a6ee-4cac-945b-20d0de887cb9.png">

than the other normal results' element:

<img width="1532" alt="Screenshot 2023-01-03 at 08 14 50" src="https://user-images.githubusercontent.com/22837764/210314218-3731f090-05d7-483e-9308-07d3b88e4624.png">

I can modify the xpath for bing engine and then make a PR from it. Just need to make the xpath to be "every `<li>` element which has the class name containing `b_algo` (thus, the xpath will become `'//li[contains(@class, "b_algo")]'` instead of  `'//li[@class="b_algo"]'`) but I'm not sure if this is the correct step to take. Maybe the choice of using xpath `'//li[@class="b_algo"]'` is by design so that searxng will exclude the Wikipedia widget result. Can others give their input also on this? Thank you. 
> I can modify the xpath for bing engine and then make a PR from it. 

@ahmad-alkadri thanks for taking the time and digging into this issue .. would be great to receive your PR :+1:
> > I can modify the xpath for bing engine and then make a PR from it.
> 
> @ahmad-alkadri thanks for taking the time and digging into this issue .. would be great to receive your PR +1

@return42 Currently working on this; the xpath modification seemingly works well though now I need to find out how to remove this little part here 😅 

![screenshot-127 0 0 1_8888-2023 01 03-20_54_14](https://user-images.githubusercontent.com/22837764/210431786-a3805a50-be06-4dfc-8385-2e3f7d9ba43d.png)

Gonna keep trying things and once ready I'll push and make the PR.

Resolving patch:
diff --git a/searx/engines/bing.py b/searx/engines/bing.py
--- a/searx/engines/bing.py
+++ b/searx/engines/bing.py
@@ -4,6 +4,7 @@
 
 - https://github.com/searx/searx/issues/2019#issuecomment-648227442
 """
+# pylint: disable=too-many-branches
 
 import re
 from urllib.parse import urlencode, urlparse, parse_qs
@@ -74,7 +75,6 @@ def request(query, params):
 
 
 def response(resp):
-
     results = []
     result_len = 0
 
@@ -84,12 +84,20 @@ def response(resp):
 
     url_to_resolve = []
     url_to_resolve_index = []
-    for i, result in enumerate(eval_xpath_list(dom, '//li[@class="b_algo"]')):
+    for i, result in enumerate(eval_xpath_list(dom, '//li[contains(@class, "b_algo")]')):
 
         link = eval_xpath(result, './/h2/a')[0]
         url = link.attrib.get('href')
         title = extract_text(link)
-        content = extract_text(eval_xpath(result, './/p'))
+
+        # Make sure that the element is free of <a href> links and <span class='algoSlug_icon'>
+        content = eval_xpath(result, '(.//p)[1]')
+        for p in content:
+            for e in p.xpath('.//a'):
+                e.getparent().remove(e)
+            for e in p.xpath('.//span[@class="algoSlug_icon"]'):
+                e.getparent().remove(e)
+        content = extract_text(content)
 
         # get the real URL either using the URL shown to user or following the Bing URL
         if url.startswith('https://www.bing.com/ck/a?'):


Test patch: