Repository: searxng/searxng

Issue:
Backport of searx/plugins/hostname_replace.py
See 
* https://github.com/searx/searx/blob/master/searx/plugins/hostname_replace.py
* https://github.com/searx/searx/pull/2724

A quick win: use `false` to remove the host name.
```
hostname_replace:
  www.libhunt.com: false
```


Discussion:
I'm not happy with https://github.com/searx/searx/pull/2724 there is also https://github.com/searx/searx/pull/2638 but  I would suggest to implement something complete new like I suggested in https://github.com/searx/searx/pull/2724#issuecomment-813393855 
About the syntax: I prefer to have rewrite rules per URL component (host name, query, path) : it avoids the issue describe just after.

> A list of expressions can be configured in settings.yml at engine level

What is the use case ? About the give example (rewrite the result of the youtube engine), it won't work if an another engine returns a youtube links.


> About the syntax: I prefer to have rewrite rules per URL component (host name, query, path) 

Good point!

> it won't work if an another engine returns a youtube links.

Hm .. can't remember, now where you point it out I can't see any proper use case. May be I have not thought to the end while I have written this comment 🤦🏻 / lets forget rewrite on engine level.
Possible syntax to combine both:

```yaml
rewrite_result:
    hostname:
      - '^([a-z]{2,3})?.m.wikipedia.org$': '\1.wikipedia.org'
      - '(.*\.)?youtube\.com$': 'invidiou.site'
      - '(.*\.)?youtu\.be$': 'invidiou.site'
      - '(.*\.)?youtube-noocookie\.com$': 'invidiou.site'
      - '(.*\.)?libhunt.com': false  # remove the result
    url:  # dangerous settings
      - '^https://twitter\.com/([^\\?]*)?\?lang=.*$': 'https://nitter.net/\1'
```

I assume we use attribute names from [urllib.parse.urlparse](https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlparse) .. right? Should we have the other attributes also configurable? .. is what I would suggest.

The other fields:
- `scheme` : ? not sure, this is related https everywhere (BTW: https://www.eff.org/deeplinks/2021/04/https-everywhere-now-uses-duckduckgos-smarter-encryption )
- `path`: most probably it require the hostname --> `url` (or a different syntax)
- `query`: related to the tracker_url_remover plugin.
- `fragment`: ??

I think we should offer to rewrite all attributes  / I think about private engines and windows file shares and all the stuff I do not have in mind right now ...  

Should be simple to implement, when we iterate over the names from urllib.parse.urlsplit, replace regexp and then join the elements using urllib.parse.urlunsplit 
May sounds OT but while we are talking about _rewrite_ why not adding a _blocking_ mechanism, better called a "_ignore_ feature". Lets say, when match condition is _True_ and the replacement is `null` the site is ignored (not shown in the results).  The following example will filter out all the _Youtube_ links from the result list.


```yaml
rewrite_result:
    hostname:
      - '(.*\.)?youtube\.(.*\.)$': null
```
Related-to: https://github.com/searx/searx/issues/2001#issuecomment-664812970

Resolving patch:
diff --git a/searx/plugins/__init__.py b/searx/plugins/__init__.py
--- a/searx/plugins/__init__.py
+++ b/searx/plugins/__init__.py
@@ -31,6 +31,7 @@
                            hash_plugin,
                            infinite_scroll,
                            self_info,
+                           hostname_replace,
                            search_on_category_select,
                            tracker_url_remover,
                            vim_hotkeys)
@@ -182,6 +183,7 @@ def sha_sum(filename):
 plugins.register(hash_plugin)
 plugins.register(infinite_scroll)
 plugins.register(self_info)
+plugins.register(hostname_replace)
 plugins.register(search_on_category_select)
 plugins.register(tracker_url_remover)
 plugins.register(vim_hotkeys)
diff --git a/searx/plugins/hostname_replace.py b/searx/plugins/hostname_replace.py
new file mode 100644
--- /dev/null
+++ b/searx/plugins/hostname_replace.py
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+import re
+from urllib.parse import urlunparse
+from searx import settings
+from searx.plugins import logger
+from flask_babel import gettext
+
+name = gettext('Hostname replace')
+description = gettext('Rewrite result hostnames or remove results based on the hostname')
+default_on = False
+preference_section = 'general'
+
+plugin_id = 'hostname_replace'
+
+replacements = {re.compile(p): r for (p, r) in settings[plugin_id].items()} if plugin_id in settings else {}
+
+logger = logger.getChild(plugin_id)
+parsed = 'parsed_url'
+
+
+def on_result(request, search, result):
+    if parsed not in result:
+        return True
+    for (pattern, replacement) in replacements.items():
+        if pattern.search(result[parsed].netloc):
+            if not replacement:
+                return False
+            result[parsed] = result[parsed]._replace(netloc=pattern.sub(replacement, result[parsed].netloc))
+            result['url'] = urlunparse(result[parsed])
+
+    return True


Test patch: