Repository: ckan/ckan

Issue:
Error when posting empty array with type json using datastore_create
### CKAN Version if known (or site URL)
2.8.0

### Please describe the expected behaviour
Posting data (see below) using datastore_create with specified field type `json` and value `[]` should return status code 200 along with the details of the resource created.

### Please describe the actual behaviour
Status code 409 is returned with the following error:
`{
    "help": "http://localhost:5000/api/3/action/help_show?name=datastore_create",
    "success": false,
    "error": {
        "message": "The data was invalid (for example: a numeric value is out of range or was inserted into a text field).",
        "__type": "Validation Error"
    }
}`

Checking the log file gives the following information:

`db       | 2019-05-08 09:50:19.098 UTC [43] ERROR:  malformed record literal: "{}" at character 103
db       | 2019-05-08 09:50:19.098 UTC [43] DETAIL:  Missing left parenthesis.
db       | 2019-05-08 09:50:19.098 UTC [43] STATEMENT:  INSERT INTO "d9bd906e-39d7-4191-8514-622e33f0afa5" ("conditionId", "symptoms")
db       |                  VALUES (1, '{}');
ckan     | 2019-05-08 09:50:19,100 INFO  [ckan.views.api] Validation error (Action API): "{'message': u'The data was invalid (for example: a numeric value is out of range or was inserted into a text field).', u'__type': u'Validation Error'}"`

### What steps can be taken to reproduce the issue? 
1. Create a dataset called 'my-dataset'.
2. Post the following data in a request to datastore_create:
`{
  "resource": {
    "package_id": "my-dataset",
    "name": "conditions"
  },
  "fields": [
    {
      "id": "conditionId",
      "type": "int"
    },
    {
      "id": "symptoms",
      "type": "json"
    }
  ],
  "primary_key": "conditionId",
  "records": [
  {
    "conditionId": 1,
    "symptoms": []
  }
  ]
}`

### Comments
I found the following issue where a similar behaviour appeared, but for datastore_upsert:
https://github.com/ckan/ckan/pull/1776
Looking at the code in ckanext/datastore/backend/postgres.py, I believe that the bug was only fixed for upsert/update and not for insert, but I could be wrong.


Discussion:
@Icontech would you mind submitting a PR to copy the upsert fix to the create code in ckanext/datastore/backend/postgres.py?
@wardi Sure, let me take a look at it. I'm not a member of the CKAN project though, so should I fork the project and then branch from the 2.8.0 release? My first time doing a PR so sorry for any stupid questions. :)
Nevermind, I found the contributing guidelines on the wiki, so I think I have all the info that I need to get started fixing the bug.

Resolving patch:
diff --git a/ckanext/datastore/backend/postgres.py b/ckanext/datastore/backend/postgres.py
--- a/ckanext/datastore/backend/postgres.py
+++ b/ckanext/datastore/backend/postgres.py
@@ -1042,7 +1042,7 @@ def upsert_data(context, data_dict):
             row = []
             for field in fields:
                 value = record.get(field['id'])
-                if value and field['type'].lower() == 'nested':
+                if value is not None and field['type'].lower() == 'nested':
                     # a tuple with an empty second value
                     value = (json.dumps(value), '')
                 row.append(value)


Test patch:
diff --git a/ckanext/datastore/tests/test_create.py b/ckanext/datastore/tests/test_create.py
--- a/ckanext/datastore/tests/test_create.py
+++ b/ckanext/datastore/tests/test_create.py
@@ -21,6 +21,32 @@
 
 
 class TestDatastoreCreateNewTests(DatastoreFunctionalTestBase):
+    def test_create_works_with_empty_array_in_json_field(self):
+        package = factories.Dataset()
+        data = {
+            'resource': {
+                'package_id': package['id']
+            },
+            'fields': [{'id': 'movie', 'type': 'text'},
+                       {'id': 'directors', 'type': 'json'}],
+            'records': [{'movie': 'sideways', 'directors': []}]
+        }
+        result = helpers.call_action('datastore_create', **data)
+        assert result['resource_id'] is not None
+
+    def test_create_works_with_empty_object_in_json_field(self):
+        package = factories.Dataset()
+        data = {
+            'resource': {
+                'package_id': package['id']
+            },
+            'fields': [{'id': 'movie', 'type': 'text'},
+                       {'id': 'director', 'type': 'json'}],
+            'records': [{'movie': 'sideways', 'director': {}}]
+        }
+        result = helpers.call_action('datastore_create', **data)
+        assert result['resource_id'] is not None     
+
     def test_create_creates_index_on_primary_key(self):
         package = factories.Dataset()
         data = {