Repository: ckan/ckan

Issue:
Deleting one Dataset deletes all datasets of an org; Patch available
Dear CKAN developers!

### CKAN Version if known (or site URL)
2.7.2, SOLR 6.6.2 and 7. SOLR in a one core mulitcore setup.
OS Debian stretch.
CKAN installed from source.
SOLR installed from TGZ.

### Please describe the expected behaviour
Deleting a single dataset should delete only this dataset.

### Please describe the actual behaviour
Deleting a single dataset deletes all dataset of the organisation

### What steps can be taken to reproduce the issue? 
Mark a dataset and click delete.

The data in the PG DB is not affected. In the PG DB only the desired datasets were deleted. But the complete data for the organization in SOLR vanishes.

I traced this problem down to the query-string going out to solr.

This query kills all the datasets in organization "inqbus"
`2017-12-06 21:25:12.561 INFO  (qtp128526626-13) [   x:ckan2] o.a.s.u.p.LogUpdateProcessorFactory [ckan2]  webapp=/solr path=/update params={commit=true}{deleteByQuery=+entity_type:package (+id:"eddb8f26-dd90-45f4-8404-8690e5fc75a4" OR +name:"eddb8f26-dd90-45f4-8404-8690e5fc75a4") +site_id:"inqbus" (-1586071352148754432),commit=} 0 28`

If I feed in the following Query as a select into the SOLR console it returns all the datasets for this organization.
+entity_type:package (+id:"eddb8f26-dd90-45f4-8404-8690e5fc75a4" OR +name:"eddb8f26-dd90-45f4-8404-8690e5fc75a4") +site_id:"inqbus"
Therefore it is correct that all datasets were deleted. I am surely no SOLR guru but the parenthesis may cause the trouble.

I patched ckan/src/ckan/ckan/lib/search/index.py to be more explicit on the boolean nature of the query.
```
    def delete_package(self, pkg_dict):
        conn = make_connection()
#        query = "+%s:%s (+id:\"%s\" OR +name:\"%s\") +site_id:\"%s\"" % (TYPE_FIELD, PACKAGE_TYPE,
#                                                       pkg_dict.get('id'), pkg_dict.get('id'),
#                                                       config.get('ckan.site_id'))
        query = "%s:%s AND (id:\"%s\" OR name:\"%s\") AND site_id:\"%s\"" % (TYPE_FIELD, PACKAGE_TYPE,
                                                       pkg_dict.get('id'), pkg_dict.get('id'),
                                                       config.get('ckan.site_id'))

```
Yielding the following delete query:

`2017-12-06 21:28:06.769 INFO  (qtp128526626-14) [   x:ckan2] o.a.s.u.p.LogUpdateProcessorFactory [ckan2]  webapp=/solr path=/update params={commit=true}{deleteByQuery=entity_type:package AND (id:"4001437b-7d33-46c9-9653-939257fe418d" OR name:"4001437b-7d33-46c9-9653-939257fe418d") AND site_id:"inqbus" (-1586071534809645056),commit=} 0 38`

This query deletes only the desired dataset.

Being a beginner with CKAN maybe I got some things messed up in my installation causing this trouble. But I double checked with SOLR 6.6 and SOLR 7 and got the same result.

I volunteer to make CKAN better. If you like to have a pull request let me know.

Cheers,
Volker



Discussion:

Resolving patch:
diff --git a/ckan/lib/search/index.py b/ckan/lib/search/index.py
--- a/ckan/lib/search/index.py
+++ b/ckan/lib/search/index.py
@@ -314,12 +314,10 @@ def commit(self):
             log.exception(e)
             raise SearchIndexError(e)
 
-
     def delete_package(self, pkg_dict):
         conn = make_connection()
-        query = "+%s:%s (+id:\"%s\" OR +name:\"%s\") +site_id:\"%s\"" % (TYPE_FIELD, PACKAGE_TYPE,
-                                                       pkg_dict.get('id'), pkg_dict.get('id'),
-                                                       config.get('ckan.site_id'))
+        query = "+%s:%s AND +(id:\"%s\" OR name:\"%s\") AND +site_id:\"%s\"" % \
+                (TYPE_FIELD, PACKAGE_TYPE, pkg_dict.get('id'), pkg_dict.get('id'), config.get('ckan.site_id'))
         try:
             commit = asbool(config.get('ckan.search.solr_commit', 'true'))
             conn.delete(q=query, commit=commit)


Test patch:
diff --git a/ckan/tests/lib/search/test_index.py b/ckan/tests/lib/search/test_index.py
--- a/ckan/tests/lib/search/test_index.py
+++ b/ckan/tests/lib/search/test_index.py
@@ -81,6 +81,28 @@ def test_clear_index(self):
         response = self.solr_client.search(q='name:monkey', fq=self.fq)
         assert_equal(len(response), 0)
 
+    def test_delete_package(self):
+        self.package_index.index_package(self.base_package_dict)
+
+        pkg_dict = self.base_package_dict.copy()
+        pkg_dict.update({
+            'id': 'test-index-2',
+            'name': 'monkey2'
+        })
+        self.package_index.index_package(pkg_dict)
+
+        response = self.solr_client.search(q='title:Monkey', fq=self.fq)
+        assert_equal(len(response), 2)
+        response_ids = sorted([x['id'] for x in response.docs])
+        assert_equal(response_ids, ['test-index', 'test-index-2'])
+
+        self.package_index.delete_package(pkg_dict)
+
+        response = self.solr_client.search(q='title:Monkey', fq=self.fq)
+        assert_equal(len(response), 1)
+        response_ids = sorted([x['id'] for x in response.docs])
+        assert_equal(response_ids, ['test-index'])
+
     def test_index_illegal_xml_chars(self):
 
         pkg_dict = self.base_package_dict.copy()