Repository: ckan/ckan

Issue:
Datastore explodes with duplicate columns
When provided with duplicate columns from datapushers (to datastore_create) datastore explodes!

```
<class 'sqlalchemy.exc.ProgrammingError'>: (ProgrammingError) column "CITY OF BRISTOL" 
specified more than once 'CREATE TABLE "652faf02-02db-4b49-911a-8305e7a6d7c1" ("_id" serial 
primary key, "_full_text" tsvector, "{1FB4E294-102B-475C-876A-000005E78B1E}" text, "177000" 
numeric, "2012-05-29 00:00" timestamp, "BS4 5AW" text, "S" text, "N" text, "F" text, "11" text, "WATER 
LANE" text, "BRISLINGTON" text, "BRISTOL" text, "CITY OF BRISTOL" text, "CITY OF BRISTOL" 
text, "A" text); ' {}
```

The solution for duplicate columns is probably non-obvious, however it would be nicer for datastore to log the error cleanly rather than raising a 500.

Relevant part of stack trace:

```
Module ckanext.datastore.logic.action:140 in datastore_create
>>  result = db.create(context, data_dict)
Module ckanext.datastore.db:1064 in create
>>  create_table(context, data_dict)
Module ckanext.datastore.db:342 in create_table
>>  context['connection'].execute(sql_string.replace('%', '%%'))
Module sqlalchemy.engine.base:719 in execute
>>  return self._execute_text(object, multiparams, params)
Module sqlalchemy.engine.base:868 in _execute_text
>>  statement, parameters
Module sqlalchemy.engine.base:954 in _execute_context
>>  context)
Module sqlalchemy.engine.base:1116 in _handle_dbapi_exception
>>  exc_info
Module sqlalchemy.util.compat:189 in raise_from_cause
>>  reraise(type(exception), exception, tb=exc_tb)
Module sqlalchemy.engine.base:947 in _execute_context
>>  context)
Module sqlalchemy.engine.default:435 in do_execute
>>  cursor.execute(statement, parameters)
```



Discussion:

Resolving patch:
diff --git a/ckanext/datastore/db.py b/ckanext/datastore/db.py
--- a/ckanext/datastore/db.py
+++ b/ckanext/datastore/db.py
@@ -316,6 +316,13 @@ def create_table(context, data_dict):
                 })
             field['type'] = _guess_type(records[0][field['id']])
 
+    # Check for duplicate fields
+    unique_fields = set([f['id'] for f in supplied_fields])
+    if not len(unique_fields) == len(supplied_fields):
+        raise ValidationError({
+            'field': ['Duplicate column names are not supported']
+        })
+
     if records:
         # check record for sanity
         if not isinstance(records[0], dict):


Test patch:
diff --git a/ckanext/datastore/tests/test_create.py b/ckanext/datastore/tests/test_create.py
--- a/ckanext/datastore/tests/test_create.py
+++ b/ckanext/datastore/tests/test_create.py
@@ -1,7 +1,7 @@
 import json
 import nose
 import sys
-from nose.tools import assert_equal
+from nose.tools import assert_equal, raises
 
 import pylons
 from pylons import config
@@ -138,6 +138,21 @@ def test_create_doesnt_add_more_indexes_when_updating_data(self):
         current_index_names = self._get_index_names(resource['id'])
         assert_equal(previous_index_names, current_index_names)
 
+    @raises(p.toolkit.ValidationError)
+    def test_create_duplicate_fields(self):
+        package = factories.Dataset()
+        data = {
+            'resource': {
+                'book': 'crime',
+                'author': ['tolstoy', 'dostoevsky'],
+                'package_id': package['id']
+            },
+            'fields': [{'id': 'book', 'type': 'text'},
+                       {'id': 'book', 'type': 'text'}],
+        }
+        result = helpers.call_action('datastore_create', **data)
+
+
     def _has_index_on_field(self, resource_id, field):
         sql = u"""
             SELECT