Repository: ckan/ckan

Issue:
datastore_search_sql returns numeric as string
**CKAN version**
2.8.4

**Describe the bug**
Datastore api function datastore_search_sql returns fields with type "numeric" as string.

**Steps to reproduce**
Create a resource and upload the following CSV file, making sure it was uploaded to the datastore:

```csv
foo,bar
1,2
3,4
```

Then access it via api using datastore_search_sql, for example like this:

    GET https://beta.ckan.org/api/3/action/datastore_search_sql?sql=SELECT * from "66ca1e80-e697-40ce-9c50-2579b59e48f2"

Which returns:

```json
{
    "help": "https://beta.ckan.org/api/3/action/help_show?name=datastore_search_sql",
    "success": true,
    "result": {
        "sql": "SELECT * from \"66ca1e80-e697-40ce-9c50-2579b59e48f2\"",
        "records": [
            {
                "_id": 1,
                "_full_text": "'1':1 '2':2",
                "foo": "1",
                "bar": "2"
            },
            {
                "_id": 2,
                "_full_text": "'3':1 '4':2",
                "foo": "3",
                "bar": "4"
            }
        ],
        "fields": [
            {
                "id": "_id",
                "type": "int4"
            },
            {
                "id": "_full_text",
                "type": "tsvector"
            },
            {
                "id": "foo",
                "type": "numeric"
            },
            {
                "id": "bar",
                "type": "numeric"
            }
        ]
    }
}
```

In the response json, result.records.foo is a string.

**Expected behavior**
result.records.foo should be a number in the json response

**Additional details**
If the data is instead queried with

    GET https://beta.ckan.org/api/3/action/datastore_search?resource_id=66ca1e80-e697-40ce-9c50-2579b59e48f2

The response is:

```json
{
    "help": "https://beta.ckan.org/api/3/action/help_show?name=datastore_search",
    "success": true,
    "result": {
        "include_total": true,
        "limit": 100,
        "records_format": "objects",
        "resource_id": "66ca1e80-e697-40ce-9c50-2579b59e48f2",
        "total_estimation_threshold": null,
        "records": [
            {
                "_id": 1,
                "foo": 1,
                "bar": 2
            },
            {
                "_id": 2,
                "foo": 3,
                "bar": 4
            }
        ],
        "fields": [
            {
                "id": "_id",
                "type": "int"
            },
            {
                "id": "foo",
                "type": "numeric"
            },
            {
                "id": "bar",
                "type": "numeric"
            }
        ],
        "_links": {
            "start": "/api/3/action/datastore_search?resource_id=66ca1e80-e697-40ce-9c50-2579b59e48f2",
            "next": "/api/3/action/datastore_search?resource_id=66ca1e80-e697-40ce-9c50-2579b59e48f2&offset=100"
        },
        "total": 2,
        "total_was_estimated": false
    }
}
```

So this seems like an inconsistency to me.


Discussion:

Resolving patch:
diff --git a/ckanext/datastore/backend/postgres.py b/ckanext/datastore/backend/postgres.py
--- a/ckanext/datastore/backend/postgres.py
+++ b/ckanext/datastore/backend/postgres.py
@@ -18,6 +18,7 @@
 import datetime
 import hashlib
 import json
+import decimal
 from collections import OrderedDict
 
 from urllib.parse import (
@@ -832,6 +833,8 @@ def convert(data: Any, type_name: str) -> Any:
         return data.isoformat()
     if isinstance(data, (int, float)):
         return data
+    if isinstance(data, (int, float, decimal.Decimal)):
+        return data
     return str(data)
 
 


Test patch:
diff --git a/ckanext/datastore/tests/test_search.py b/ckanext/datastore/tests/test_search.py
--- a/ckanext/datastore/tests/test_search.py
+++ b/ckanext/datastore/tests/test_search.py
@@ -3,6 +3,7 @@
 import json
 import pytest
 import sqlalchemy.orm as orm
+import decimal
 
 import ckan.lib.create_test_data as ctd
 import ckan.logic as logic
@@ -1848,6 +1849,28 @@ def test_search_limit(self):
         assert [res[u"the year"] for res in result["records"]] == [2014, 2013]
         assert result[u"records_truncated"]
 
+    @pytest.mark.ckan_config("ckan.plugins", "datastore")
+    @pytest.mark.usefixtures("clean_datastore", "with_plugins")
+    def test_search_numeric_data_through_sql(self):
+        resource = factories.Resource()
+        data = {
+            "resource_id": resource["id"],
+            "force": True,
+            "fields": [
+                {"id": "foo", "type": "numeric"},
+                {"id": "bar", "type": "numeric"}
+            ],
+            "records": [
+                {"foo": 1, "bar": 2},
+                {"foo": 3, "bar": 4}
+            ]
+        }
+        result = helpers.call_action("datastore_create", **data)
+        sql = 'SELECT * FROM "{0}"'.format(resource["id"])
+        result = helpers.call_action("datastore_search_sql", sql=sql)
+        record_new = result["records"]
+        assert (isinstance(record_new[0]["foo"], decimal.Decimal))
+
 
 class TestDatastoreSearchRecordsFormat(object):
     @pytest.mark.ckan_config("ckan.plugins", "datastore")