Repository: Kinto/kinto

Issue:
IntegrityError if you try to create two records with the same last_modified value in a batch
```
demo:
  collections:
    demo:
      records:
        abc:
          data:
            last_modified: 123
        efg:
          data:
            last_modified: 123
```

```
$ kinto-wizard load -s https://kinto.dev.mozaws.net/v1 -a admin:admin  demo.yaml --force -b demo -c demo
kinto_http.exceptions.KintoException: POST /v1/batch - 503 503 - 
{'message': 'Service temporary unavailable due to overloading or maintenance, please retry later.',
 'code': 503, 'errno': 201, 'error': 'Service Unavailable'}
```


Discussion:
Nice catch :) 
Hi Guys, new here. I was trying to reproduce the issue on my local but no luck:
Please check below, I might be missing something....

`kinto-wizard load -s http://localhost:8888/v1 -a admin:admin demo.yaml --force -b demo -c demo`

```
demo:
  collections:
     demo:
        records:
           abcd:
              data:
               last_modified: 123

           efgh:
              data:
               last_modified: 123
```
```
$ kinto-wizard load -s http://localhost:8888/v1 -a admin:admin demo.yaml --force -b demo -c demo
Batch #0: PUT /buckets/demo - 200 
Batch #1: PUT /buckets/demo/collections/demo - 200 
Batch #2: PUT /buckets/demo/collections/demo/records/abcd - 201 
Batch #3: PUT /buckets/demo/collections/demo/records/efgh - 201 
```

```
$ http GET http://localhost:8888/v1/buckets/demo/collections/demo/records -a admin:admin
HTTP/1.1 200 OK
Access-Control-Expose-Headers: Cache-Control, Total-Records, Retry-After, Backoff, Alert, ETag, Pragma, Next-Page, Expires, Content-Length, Last-Modified
Cache-Control: no-cache, no-store
Content-Length: 78
Content-Type: application/json
Date: Thu, 19 Apr 2018 20:00:13 GMT
Etag: "1524167996115"
Last-Modified: Thu, 19 Apr 2018 19:59:56 GMT
Server: waitress
Total-Records: 2
X-Content-Type-Options: nosniff

{
    "data": [
        {
            "id": "abcd",
            "last_modified": 123
        },
        {
            "id": "efgh",
            "last_modified": 123
        }
    ]
}

```


Natim might be able to give you more information, but I looked at it quickly to help get you started and it seems that it's caused by turning on the history plugin in config/kinto.ini
Excellent, thanks for taking the time to investigate this!
It may only happen with the PostgreSQL backend actually... 
Hmm, when I'm just using the default_bucket and flush plugins, with a PostgreSQL backend, I get the proper 409 constraint violation on the second bucket. When I turn history plugin on I get a 503.
> When I turn history plugin on I get a 503.

Nice one!


Resolving patch:
diff --git a/kinto/core/views/batch.py b/kinto/core/views/batch.py
--- a/kinto/core/views/batch.py
+++ b/kinto/core/views/batch.py
@@ -144,6 +144,13 @@ def post_batch(request):
             resp, subrequest = request.follow_subrequest(subrequest,
                                                          use_tweens=False)
         except httpexceptions.HTTPException as e:
+            # Since some request in the batch failed, we need to stop the parent request
+            # through Pyramid's transaction manager. 5XX errors are already caught by
+            # pyramid_tm's commit_veto
+            # https://github.com/Kinto/kinto/issues/624
+            if e.status_code == 409:
+                request.tm.abort()
+
             if e.content_type == 'application/json':
                 resp = e
             else:


Test patch:
diff --git a/tests/core/test_views_transaction.py b/tests/core/test_views_transaction.py
--- a/tests/core/test_views_transaction.py
+++ b/tests/core/test_views_transaction.py
@@ -56,7 +56,7 @@ def test_modifications_are_rolled_back_on_error(self):
         resp = self.app.get('/mushrooms', headers=self.headers)
         self.assertEqual(len(resp.json['data']), 0)
 
-    def test_modifications_are_not_rolled_back_on_4XX_error(self):
+    def test_modifications_are_not_rolled_back_on_401_error(self):
         request_create = {
             'method': 'POST',
             'path': '/mushrooms',
@@ -71,6 +71,24 @@ def test_modifications_are_not_rolled_back_on_4XX_error(self):
         resp = self.app.get('/mushrooms', headers=self.headers)
         self.assertEqual(len(resp.json['data']), 1)
 
+    def test_modifications_are_rolled_back_on_409_error(self):
+        bucket_create = {
+            'method': 'POST',
+            'path': '/mushrooms',
+            'body': {'data': {'name': 'Vesse de loup', 'last_modified': 123}}
+        }
+        bucket_ccreate = {
+            'method': 'POST',
+            'path': '/mushrooms',
+            'body': {'data': {'name': 'Vesse de loup', 'last_modified': 123}}
+        }
+        body = {'requests': [bucket_create, bucket_ccreate]}
+        resp = self.app.post_json('/batch', body, headers=self.headers)
+        response = resp.json['responses'][1]
+        self.assertEqual(response['status'], 409)
+        resp = self.app.get('/mushrooms', headers=self.headers)
+        self.assertEqual(len(resp.json['data']), 0)
+
     def test_modifications_are_rolled_back_on_error_accross_backends(self):
         self.run_failing_post()