Repository: searxng/searxng

Issue:
Add unit conversion like google.
<!-- PLEASE FILL THESE FIELDS, IT REALLY HELPS THE MAINTAINERS OF SearXNG -->

**Is your feature request related to a problem? Please describe.**
no

**Describe the solution you'd like**
Unit conversion like google when performing searched like "10 miles in km"

**Describe alternatives you've considered**
use google.com

**Additional context**
Currency conversion already works, however distance, weight, speed would be nice to have it implemented.



Discussion:
Looks like ddg_definitions does handle unit conversion, but SearXNG engine crashes when it tries to parse the result.

Search query: `!ddg_definitions km to miles`
Error produced: 

```
Error
  * Error: httpx.ConnectTimeout
  * Percentage: 0
  * Parameters: `(None, None, 'api.duckduckgo.com')`
  * File name: `searx/search/processors/online.py:118`
  * Function: `_send_http_request`
  * Code: `response = req(params['url'], **request_args)`

Error
  * Error: AttributeError
  * Percentage: 0
  * Parameters: `()`
  * File name: `searx/engines/duckduckgo_definitions.py:105`
  * Function: `response`
  * Code: `results.append({'answer': html_to_text(answer), 'url': search_res.get('AbstractURL', '')})`
  ```
before we start implementing too much, it might be easier to use wolframalpha

- [!wa 10 miles in km](https://darmarit.org/searx/search?q=%21wa+10+miles+in+km)
- [!!wa 10 miles in km](https://darmarit.org/searx/search?q=%21%21wa+10+miles+in+km)

@thunder-coding can you please open an issue / thanks!


The only two ways we could really achieve this without having to do all the logic ourselves seem to be using
- using the [pint](https://github.com/hgrecco/pint) library
- [GNU units](https://www.gnu.org/software/units/manual/units.html) via a subprocess

I personally think that it'd be okay to depend on the python units library, using GNU units would be more flexible and supports more currencies but running it in a subprocess is probably not really a good solution?

What are your opinions on that @dalf, @unixfox and @return42?
I don't have a clear answer right now.
Here some words, perhaps not in the right direction:

> without having to do all the logic ourselves seem

A naive (?) solution:
* There is already a [JSON file with all the unit from wikidata](https://github.com/searxng/searxng/blob/master/searx/data/wikidata_units.json). The [existing SPARQL query](https://github.com/searxng/searxng/blob/e2af3e49702f6fb40e1614f826544dc3b03bca2f/searxng_extra/update/update_wikidata_units.py#L35-L44). 
* Updated query to get the conversion to SI unit: https://w.wiki/9VFr
* From that we can implement something to convert unit:
    * "source unit" to SI
    * SI to "destination unit"

---

More or less related to this 9 years old PR: https://github.com/searx/searx/pull/384
Especially this: https://docs.sympy.org/latest/modules/physics/units/index.html

---

[EDIT] There is https://github.com/Qalculate/libqalculate but there is no Python binding.
<details>

<summary>untested binding written by ChatGPT4</summary>

based on http://qalculate.github.io/reference/index.html

qalculate_wrapper.pyx:

```python
# distutils: language = c++
# distutils: sources = qalculate/Calculator.cpp

cimport cython

# External C++ class declaration
cdef extern from "qalculate/Calculator.h" namespace "libqalculate":
    cdef cppclass Calculator:
        Calculator() except +
        void loadExchangeRates()
        void loadGlobalDefinitions()
        void loadLocalDefinitions()
        string calculateAndPrint(string str, int msecs, const EvaluationOptions &eo, const PrintOptions &po, string *parsed_expression)

# Cython wrapper for the Calculator class
cdef class PyCalculator:
    cdef Calculator* c_calculator

    def __cinit__(self):
        self.c_calculator = new Calculator()
        self.c_calculator.loadExchangeRates()
        self.c_calculator.loadGlobalDefinitions()
        self.c_calculator.loadLocalDefinitions()

    def __dealloc__(self):
        del self.c_calculator

    def calculate_and_print(self, expression, int msecs=10000):
        cdef string result
        cdef string c_expression = expression.encode('UTF-8')
        result = self.c_calculator.calculateAndPrint(c_expression, msecs, default_user_evaluation_options, default_print_options, NULL)
        return result.decode('UTF-8')
```

setup.py
```python
from setuptools import setup
from Cython.Build import cythonize

setup(
    ext_modules=cythonize("qalculate_wrapper.pyx",
                          language_level="3"),
    compiler_directives={'language_level': "3"}
)
```

</details>



---

[EDIT2] [Spot](https://gitlab.e.foundation/e/infra/spot) uses [mathjs](https://mathjs.org/index.html)
> GNU units via a subprocess

Terrible idea, increases attack surface for instances for RCE vulnerabilities.

Resolving patch:
diff --git a/searx/engines/duckduckgo_definitions.py b/searx/engines/duckduckgo_definitions.py
--- a/searx/engines/duckduckgo_definitions.py
+++ b/searx/engines/duckduckgo_definitions.py
@@ -238,7 +238,10 @@ def unit_to_str(unit):
     for prefix in WIKIDATA_PREFIX:
         if unit.startswith(prefix):
             wikidata_entity = unit[len(prefix) :]
-            return WIKIDATA_UNITS.get(wikidata_entity, unit)
+            real_unit = WIKIDATA_UNITS.get(wikidata_entity)
+            if real_unit is None:
+                return unit
+            return real_unit['symbol']
     return unit
 
 
diff --git a/searx/plugins/unit_converter.py b/searx/plugins/unit_converter.py
new file mode 100644
--- /dev/null
+++ b/searx/plugins/unit_converter.py
@@ -0,0 +1,76 @@
+# SPDX-License-Identifier: AGPL-3.0-or-later
+"""Calculate mathematical expressions using ack#eval
+"""
+
+from flask_babel import gettext
+
+from searx.data import WIKIDATA_UNITS
+
+name = "Unit converter plugin"
+description = gettext("Convert between units")
+default_on = True
+
+CONVERT_KEYWORDS = ["in", "to", "as"]
+
+
+def _convert(from_value, source_si_factor, target_si_factor):
+    return from_value * source_si_factor / target_si_factor
+
+
+def _parse_text_and_convert(search, splitted_query):
+    if len(splitted_query) != 2 or splitted_query[0].strip() == "" or splitted_query[1].strip() == "":
+        return
+
+    from_value = ""
+    from_unit_key = ""
+
+    # only parse digits as value that belong together
+    read_alpha = False
+    for c in splitted_query[0]:
+        if not read_alpha and (c in ("-", ".") or str.isdigit(c)):
+            from_value += c
+            read_alpha = True
+        elif c != " ":
+            from_unit_key += c
+
+    to_unit_key = splitted_query[1].strip()
+
+    from_unit = None
+    to_unit = None
+
+    for unit in WIKIDATA_UNITS.values():
+        if unit['symbol'] == from_unit_key:
+            from_unit = unit
+
+        if unit['symbol'] == to_unit_key:
+            to_unit = unit
+
+        if from_unit and to_unit:
+            break
+
+    if from_unit is None or to_unit is None or to_unit.get('si_name') != from_unit.get('si_name'):
+        return
+
+    result = _convert(float(from_value), from_unit['to_si_factor'], to_unit['to_si_factor'])
+    search.result_container.answers['conversion'] = {'answer': f"{result:g} {to_unit['symbol']}"}
+
+
+def post_search(_request, search):
+    # only convert between units on the first page
+    if search.search_query.pageno > 1:
+        return True
+
+    query = search.search_query.query
+    query_parts = query.split(" ")
+
+    if len(query_parts) < 3:
+        return True
+
+    for query_part in query_parts:
+        for keyword in CONVERT_KEYWORDS:
+            if query_part == keyword:
+                keyword_split = query.split(keyword, 1)
+                _parse_text_and_convert(search, keyword_split)
+                return True
+
+    return True
diff --git a/searxng_extra/update/update_wikidata_units.py b/searxng_extra/update/update_wikidata_units.py
--- a/searxng_extra/update/update_wikidata_units.py
+++ b/searxng_extra/update/update_wikidata_units.py
@@ -29,31 +29,46 @@
 # * https://www.wikidata.org/wiki/Help:Ranking
 # * https://www.mediawiki.org/wiki/Wikibase/Indexing/RDF_Dump_Format ("Statement representation" section)
 # * https://w.wiki/32BT
+# * https://en.wikibooks.org/wiki/SPARQL/WIKIDATA_Precision,_Units_and_Coordinates#Quantities
 #   see the result for https://www.wikidata.org/wiki/Q11582
 #   there are multiple symbols the same rank
 SARQL_REQUEST = """
-SELECT DISTINCT ?item ?symbol
+SELECT DISTINCT ?item ?symbol ?tosi ?tosiUnit
 WHERE
 {
   ?item wdt:P31/wdt:P279 wd:Q47574 .
   ?item p:P5061 ?symbolP .
   ?symbolP ps:P5061 ?symbol ;
            wikibase:rank ?rank .
+  OPTIONAL {
+    ?item p:P2370 ?tosistmt .
+    ?tosistmt psv:P2370 ?tosinode .
+    ?tosinode wikibase:quantityAmount ?tosi .
+    ?tosinode wikibase:quantityUnit ?tosiUnit .
+  }
   FILTER(LANG(?symbol) = "en").
 }
 ORDER BY ?item DESC(?rank) ?symbol
 """
 
+_wikidata_url = "https://www.wikidata.org/entity/"
+
 
 def get_data():
     results = collections.OrderedDict()
     response = wikidata.send_wikidata_query(SARQL_REQUEST)
     for unit in response['results']['bindings']:
-        name = unit['item']['value'].replace('http://www.wikidata.org/entity/', '')
-        unit = unit['symbol']['value']
+        name = unit['item']['value'].replace(_wikidata_url, '')
+        symbol = unit['symbol']['value']
+        si_name = unit.get('tosiUnit', {}).get('value', '').replace(_wikidata_url, '')
+        to_si_factor = unit.get('tosi', {}).get('value', '')
         if name not in results:
             # ignore duplicate: always use the first one
-            results[name] = unit
+            results[name] = {
+                'symbol': symbol,
+                'si_name': si_name if si_name else None,
+                'to_si_factor': float(to_si_factor) if to_si_factor else None,
+            }
     return results
 
 


Test patch: