Repository: searxng/searxng

Issue:
Escaped HTML creeping into search responses
<!-- PLEASE FILL THESE FIELDS, IT REALLY HELPS THE MAINTAINERS OF SearXNG -->

**Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG**
<!-- If you are running on master branch using git execute this command
in order to fetch the latest commit ID:
```
``` 
If you are using searxng-docker then look at the bottom of the SearXNG page
and check for the version after "Powered by SearXNG"

Please also stipulate if you are using a forked version of SearXNG and
include a link to the fork source code.
-->
2023.12.11+8a4104b99

**How did you install SearXNG?**
<!-- Did you install SearXNG using the official wiki or using searxng-docker
or manually by executing the searx/webapp.py file? -->
The issue shows itself on any public searxng instances with current searxng, for example https://search.hbubli.cc/

**What happened?**
<!-- A clear and concise description of what the bug is. -->
Parts of escaped HTML creep in search responses.

**How To Reproduce**
<!-- How can we reproduce this issue? (as minimally and as precisely as possible) -->
Perform search for `class attribute` query.

**Expected behavior**
<!-- A clear and concise description of what you expected to happen. -->
I'd prefer not to see HTML in responses in any form :)

**Screenshots & Logs**
<!-- If applicable, add screenshots, logs to help explain your problem. -->
![Untitled](https://github.com/searxng/searxng/assets/1695323/00ad0164-0368-4979-ae14-d9b5785a086f)

**Additional context**
<!-- Add any other context about the problem here. -->



Discussion:
Disabling `hightlight_content` https://github.com/searxng/searxng/blob/8a4104b9922cbe4c33864b5c0de02e88d3c9665d/searx/webutils.py#L276 makes the problem go away, sans missing search word hightlighting.

TODO comment in that function probably explains it all... :)

Resolving patch:
diff --git a/searx/webutils.py b/searx/webutils.py
--- a/searx/webutils.py
+++ b/searx/webutils.py
@@ -290,10 +290,8 @@ def highlight_content(content, query):
         if len(qs) > 0:
             queries.extend(re.findall(regex_highlight_cjk(qs), content, flags=re.I | re.U))
     if len(queries) > 0:
-        for q in set(queries):
-            content = re.sub(
-                regex_highlight_cjk(q), f'<span class="highlight">{q}</span>'.replace('\\', r'\\'), content
-            )
+        regex = re.compile("|".join(map(regex_highlight_cjk, queries)))
+        return regex.sub(lambda match: f'<span class="highlight">{match.group(0)}</span>'.replace('\\', r'\\'), content)
     return content
 
 


Test patch:
diff --git a/tests/unit/test_webutils.py b/tests/unit/test_webutils.py
--- a/tests/unit/test_webutils.py
+++ b/tests/unit/test_webutils.py
@@ -57,6 +57,11 @@ def test_highlight_content(self):
                     ]
                 ),
             ),
+            (
+                'a class',
+                'a string with class.',
+                '<span class="highlight">a</span> string with <span class="highlight">class</span>.',
+            ),
         )
         for query, content, expected in data:
             self.assertEqual(webutils.highlight_content(content, query), expected)