Repository: Kinto/kinto

Issue:
If-Match: * does not match the spec
Per https://github.com/Kinto/kinto/pull/1064#issuecomment-279351031, the behavior of `If-Match: *` is the opposite of `If-None-Match: *`. `If-None-Match: *` should succeed if no resource exists at the given URL and fail if any resource exists at the given URL, but `If-Match: *` should fail if no resource exists at the given URL and succeed if any resource exists at the given URL. Currently `If-Match: *` does nothing (it always succeeds).
If-Match: * does not match the spec
Per https://github.com/Kinto/kinto/pull/1064#issuecomment-279351031, the behavior of `If-Match: *` is the opposite of `If-None-Match: *`. `If-None-Match: *` should succeed if no resource exists at the given URL and fail if any resource exists at the given URL, but `If-Match: *` should fail if no resource exists at the given URL and succeed if any resource exists at the given URL. Currently `If-Match: *` does nothing (it always succeeds).


Discussion:
Ugh, I guess this got opened by someone else in the meantime..
You are so lagging 🗡 
What about the behavior of `If-Match` on a numeric value? Should we ignore it when a resource doesn't exists? What does matching on a non-existing resource means?

I'm not sure how much this would affect the current API, but my feeling is that it should fail on non existing even if the value of `If-Match` is an ETag.
I agree with you: I think `If-Match: "abcd"` should fail if the record is not there.

While I'm here, I should also note that I think the current behavior of the codebase is slightly wrong. If I read the code correctly, it tries to compare the If-Match header against the current record's ETag, and fail if the record is newer. I think this is different from what the spec says, which is that the request should fail unless the record's ETag is exactly the same (neither older nor newer) than the If-Match header, but I don't think it's super important as long as the clients behave properly (because time, and thus our ETags, are monotonic). We assign more meaning to ETags than the standard really requires -- ETags are just opaque cookies representing the version of a document and don't have to behave monotonically or anything.

> it tries to compare the If-Match header against the current record's ETag, and fail if the record is newer.

Yes that's because before we used `If-Modified-Since` and we kept the same behaviour.
Ugh, I guess this got opened by someone else in the meantime..
You are so lagging 🗡 
What about the behavior of `If-Match` on a numeric value? Should we ignore it when a resource doesn't exists? What does matching on a non-existing resource means?

I'm not sure how much this would affect the current API, but my feeling is that it should fail on non existing even if the value of `If-Match` is an ETag.
I agree with you: I think `If-Match: "abcd"` should fail if the record is not there.

While I'm here, I should also note that I think the current behavior of the codebase is slightly wrong. If I read the code correctly, it tries to compare the If-Match header against the current record's ETag, and fail if the record is newer. I think this is different from what the spec says, which is that the request should fail unless the record's ETag is exactly the same (neither older nor newer) than the If-Match header, but I don't think it's super important as long as the clients behave properly (because time, and thus our ETags, are monotonic). We assign more meaning to ETags than the standard really requires -- ETags are just opaque cookies representing the version of a document and don't have to behave monotonically or anything.

> it tries to compare the If-Match header against the current record's ETag, and fail if the record is newer.

Yes that's because before we used `If-Modified-Since` and we kept the same behaviour.

Resolving patch:
diff --git a/kinto/__init__.py b/kinto/__init__.py
--- a/kinto/__init__.py
+++ b/kinto/__init__.py
@@ -13,7 +13,7 @@
 __version__ = pkg_resources.get_distribution(__package__).version
 
 # Implemented HTTP API Version
-HTTP_API_VERSION = '1.14'
+HTTP_API_VERSION = '1.15'
 
 # Main kinto logger
 logger = logging.getLogger(__name__)
diff --git a/kinto/core/resource/__init__.py b/kinto/core/resource/__init__.py
--- a/kinto/core/resource/__init__.py
+++ b/kinto/core/resource/__init__.py
@@ -260,7 +260,8 @@ def collection_get(self):
         self._add_timestamp_header(self.request.response)
         self._add_cache_header(self.request.response)
         self._raise_304_if_not_modified()
-        self._raise_412_if_modified()
+        # Collections are considered resources that always exist
+        self._raise_412_if_modified(record={})
 
         headers = self.request.response.headers
 
@@ -355,7 +356,8 @@ def collection_delete(self):
         :raises: :exc:`~pyramid:pyramid.httpexceptions.HTTPBadRequest`
             if filters are invalid.
         """
-        self._raise_412_if_modified()
+        # Collections are considered resources that always exist
+        self._raise_412_if_modified(record={})
 
         filters = self._extract_filters()
         limit = self._extract_limit()
@@ -452,8 +454,7 @@ def put(self):
             if len(tombstones) > 0:
                 existing = tombstones[0]
         finally:
-            if existing:
-                self._raise_412_if_modified(existing)
+            self._raise_412_if_modified(record=existing)
 
         # If `data` is not provided, use existing record (or empty if creation)
         post_record = self.request.validated['body'].get('data', existing) or {}
@@ -814,7 +815,7 @@ def _raise_304_if_not_modified(self, record=None):
         else:
             current_timestamp = self.model.timestamp()
 
-        if current_timestamp <= if_none_match:
+        if current_timestamp == if_none_match:
             response = HTTPNotModified()
             self._add_timestamp_header(response, timestamp=current_timestamp)
             raise response
@@ -829,18 +830,27 @@ def _raise_412_if_modified(self, record=None):
         if_match = self.request.validated['header'].get('If-Match')
         if_none_match = self.request.validated['header'].get('If-None-Match')
 
+        # Check if record exists and it's not a tombstone
+        record_exists = record is not None and not record.get(self.model.deleted_field)
+
+        # If no precondition headers, just ignore
         if not if_match and not if_none_match:
             return
 
-        if record and if_none_match == '*':
-            if record.get(self.model.deleted_field, False):
-                # Tombstones should not prevent creation.
-                return
+        # If-None-Match: * should always raise if a record exists
+        if if_none_match == '*' and record_exists:
             modified_since = -1  # Always raise.
+
+        # If-Match should always raise if a record doesn't exist
+        elif if_match and not record_exists:
+            modified_since = -1
+
+        # If-Match with ETag value on existing records should compare ETag
         elif if_match and if_match != '*':
             modified_since = if_match
+
+        # If none of the above applies, don't raise
         else:
-            # In case _raise_304_if_not_modified() did not raise.
             return
 
         if record:
@@ -848,7 +858,7 @@ def _raise_412_if_modified(self, record=None):
         else:
             current_timestamp = self.model.timestamp()
 
-        if current_timestamp > modified_since:
+        if current_timestamp != modified_since:
             error_msg = 'Resource was modified meanwhile'
             details = {'existing': record} if record else {}
             response = http_error(HTTPPreconditionFailed(),


Test patch:
diff --git a/tests/core/resource/test_model.py b/tests/core/resource/test_model.py
--- a/tests/core/resource/test_model.py
+++ b/tests/core/resource/test_model.py
@@ -62,6 +62,7 @@ def test_delete_supports_collection_filters(self):
 class IsolatedModelsTest(BaseTest):
     def setUp(self):
         super().setUp()
+        self.resource.request.validated = {'header': {}, 'querystring': {}}
         self.stored = self.model.create_record({}, parent_id='bob')
         self.resource.record_id = self.stored['id']
 
@@ -81,7 +82,7 @@ def test_list_is_filtered_by_user(self):
         self.assertEquals(len(records), 0)
 
     def test_update_record_of_another_user_will_create_it(self):
-        self.resource.request.validated = {'body': {'data': {'some': 'record'}}}
+        self.resource.request.validated['body'] = {'data': {'some': 'record'}}
         self.resource.put()
         self.model.get_record(record_id=self.stored['id'],
                               parent_id='basicauth:alice')  # not raising
diff --git a/tests/core/resource/test_preconditions.py b/tests/core/resource/test_preconditions.py
--- a/tests/core/resource/test_preconditions.py
+++ b/tests/core/resource/test_preconditions.py
@@ -54,11 +54,18 @@ def test_single_record_last_modified_is_returned(self):
 class ModifiedMeanwhileTest(BaseTest):
     def setUp(self):
         super().setUp()
+
+        # Create a record using model (will have an incremented last_modified)
         self.stored = self.model.create_record({})
-        self.resource.collection_get()
+        # Update the record we just created (will set last_modified with the server ETag)
+        self.resource.record_id = self.stored['id']
+        self.resource.put()
+        # Update our record with last_modified provided by the server
+        self.stored['last_modified'] = int(self.last_response.headers['ETag'][1:-1])
+
         self.validated = self.resource.request.validated
-        current = self.last_response.headers['ETag'][1:-1]
-        previous = int(current) - 10
+        self.current = int(self.last_response.headers['ETag'][1:-1])
+        previous = self.current - 10
         self.validated['header']['If-Match'] = previous
 
     def test_preconditions_errors_are_json_formatted(self):
@@ -76,9 +83,13 @@ def test_collection_returns_412_if_changed_meanwhile(self):
         self.assertRaises(httpexceptions.HTTPPreconditionFailed,
                           self.resource.collection_get)
 
-    def test_collection_returns_200_if_if_match_is_superior(self):
-        current = self.last_response.headers['ETag'][1:-1]
-        self.validated['header']['If-Match'] = int(current) + 10
+    def test_collection_returns_412_if_if_match_is_superior(self):
+        self.validated['header']['If-Match'] = self.current + 10
+        self.assertRaises(httpexceptions.HTTPPreconditionFailed,
+                          self.resource.collection_get)
+
+    def test_collection_returns_200_if_if_match_is_equal(self):
+        self.validated['header']['If-Match'] = self.current
         self.resource.collection_get()
 
     def test_412_errors_on_collection_do_not_provide_existing_record(self):
@@ -97,21 +108,20 @@ def test_412_on_collection_has_last_modified_timestamp(self):
         self.assertIsNotNone(error.headers.get('Last-Modified'))
 
     def test_412_errors_on_record_provide_existing_data(self):
-        self.resource.record_id = self.stored['id']
         try:
             self.resource.put()
         except httpexceptions.HTTPPreconditionFailed as e:
             error = e
-        self.assertEqual(error.json['details']['existing'],
-                         self.stored)
+        print(error.json['details']['existing'])
+        print(self.stored)
+        self.assertDictEqual(error.json['details']['existing'],
+                             self.stored)
 
     def test_single_record_returns_412_if_changed_meanwhile(self):
-        self.resource.record_id = self.stored['id']
         self.assertRaises(httpexceptions.HTTPPreconditionFailed,
                           self.resource.get)
 
     def test_412_on_single_record_has_last_modified_timestamp(self):
-        self.resource.record_id = self.stored['id']
         try:
             self.resource.get()
         except httpexceptions.HTTPPreconditionFailed as e:
@@ -125,7 +135,6 @@ def test_create_returns_412_if_changed_meanwhile(self):
                           self.resource.collection_post)
 
     def test_put_returns_412_if_changed_meanwhile(self):
-        self.resource.record_id = self.stored['id']
         self.model.delete_record(self.stored)
         self.assertRaises(httpexceptions.HTTPPreconditionFailed,
                           self.resource.put)
@@ -133,7 +142,6 @@ def test_put_returns_412_if_changed_meanwhile(self):
     def test_put_returns_412_if_changed_and_none_match_present(self):
         self.validated['body'] = {'data': {'field': 'new'}}
         self.validated['header']['If-None-Match'] = 42
-        self.resource.record_id = self.stored['id']
         self.assertRaises(httpexceptions.HTTPPreconditionFailed,
                           self.resource.put)
 
@@ -147,21 +155,18 @@ def test_put_returns_last_modified_if_changed_meanwhile(self):
         self.assertNotIn('1970', error.headers['Last-Modified'])
 
     def test_put_returns_412_if_deleted_meanwhile(self):
-        self.resource.record_id = self.stored['id']
         self.assertRaises(httpexceptions.HTTPPreconditionFailed,
                           self.resource.put)
 
     def test_put_if_none_match_star_fails_if_record_exists(self):
         self.validated['header'].pop('If-Match')
         self.validated['header']['If-None-Match'] = '*'
-        self.resource.record_id = self.stored['id']
         self.assertRaises(httpexceptions.HTTPPreconditionFailed,
                           self.resource.put)
 
     def test_get_if_none_match_star_fails_if_record_exists(self):
         self.validated['header'].pop('If-Match')
         self.validated['header']['If-None-Match'] = '*'
-        self.resource.record_id = self.stored['id']
         self.assertRaises(httpexceptions.HTTPPreconditionFailed,
                           self.resource.get)
 
@@ -177,7 +182,6 @@ def test_put_if_none_match_star_succeeds_if_tombstone_exists(self):
         self.validated['header'].pop('If-Match')
         self.validated['header']['If-None-Match'] = '*'
         self.validated['body'] = {'data': {'field': 'new'}}
-        self.resource.record_id = self.stored['id']
         self.resource.put()  # not raising.
 
     def test_post_if_none_match_star_fails_if_record_exists(self):
@@ -200,22 +204,84 @@ def test_post_if_none_match_star_succeeds_if_record_does_not_exist(self):
                 'field': 'new'}}
         self.resource.collection_post()  # not raising.
 
-    def test_post_if_match_star_succeeds_if_record_does_exist(self):
+    def test_get_if_none_match_star_fails_on_collections(self):
+        self.validated['header'].pop('If-Match')
+        self.validated['header']['If-None-Match'] = '*'
+        self.assertRaises(httpexceptions.HTTPPreconditionFailed,
+                          self.resource.collection_get)
+
+    def test_delete_if_none_match_star_fails_on_collections(self):
+        self.validated['header'].pop('If-Match')
+        self.validated['header']['If-None-Match'] = '*'
+        self.assertRaises(httpexceptions.HTTPPreconditionFailed,
+                          self.resource.collection_delete)
+
+    def test_get_if_match_star_succeeds_if_record_exists(self):
         self.validated['header']['If-Match'] = '*'
-        self.validated['body'] = {
+        self.resource.record_id = self.stored['id']
+        self.resource.get()
+
+    def test_post_if_match_star_succeeds_if_record_exists(self):
+        self.validated['header']['If-Match'] = '*'
+        self.resource.request.json = {
             'data': {
-                'id': self.resource.model.id_generator(),
+                'id': self.stored['id'],
                 'field': 'new'}}
-        self.resource.collection_post()  # not raising.
+        self.validated['body'] = self.resource.request.json
+        self.resource.collection_post()
+
+    def test_put_if_match_star_succeeds_if_record_exists(self):
+        self.validated['header']['If-Match'] = '*'
+        self.resource.put()
+
+    def test_patch_if_match_star_succeeds_if_record_exists(self):
+        self.validated['header']['If-Match'] = '*'
+        self.resource.request.json = {
+            'data': {
+                'id': self.stored['id'],
+                'field': 'new'}}
+        self.validated['body'] = self.resource.request.json
+        self.resource.patch()
+
+    def test_delete_if_match_star_succeeds_if_record_exists(self):
+        self.validated['header']['If-Match'] = '*'
+        self.resource.delete()
+
+    def test_put_if_match_star_fails_if_record_does_not_exist(self):
+        self.validated['header']['If-Match'] = '*'
+        self.resource.request.json = {'data': {'field': 'new'}}
+        self.validated['body'] = self.resource.request.json
+        self.resource.record_id = self.resource.model.id_generator()
+        self.assertRaises(httpexceptions.HTTPPreconditionFailed,
+                          self.resource.put)
+
+    def test_put_if_match_current_fails_if_record_does_not_exist(self):
+        self.validated['header']['If-Match'] = self.current  # wouldn't raise on existing
+        self.resource.request.json = {'data': {'field': 'new'}}
+        self.validated['body'] = self.resource.request.json
+        self.resource.record_id = self.resource.model.id_generator()
+        self.assertRaises(httpexceptions.HTTPPreconditionFailed,
+                          self.resource.put)
+
+    def test_put_if_fails_if_if_match_is_superior(self):
+        self.validated['header']['If-Match'] += self.current + 10
+        self.assertRaises(httpexceptions.HTTPPreconditionFailed,
+                          self.resource.put)
+
+    def test_get_if_match_star_suceed_on_collections(self):
+        self.validated['header']['If-Match'] = '*'
+        self.resource.collection_get()
+
+    def test_delete_if_match_star_suceed_on_collections(self):
+        self.validated['header']['If-Match'] = '*'
+        self.resource.collection_delete()
 
     def test_patch_returns_412_if_changed_meanwhile(self):
-        self.resource.record_id = self.stored['id']
         self.assertRaises(httpexceptions.HTTPPreconditionFailed,
                           self.resource.patch)
 
     def test_patch_returns_last_modified_if_changed_meanwhile(self):
         self.resource.timestamp = 0
-        self.resource.record_id = self.stored['id']
         try:
             self.resource.patch()
         except httpexceptions.HTTPPreconditionFailed as e:
@@ -223,7 +289,6 @@ def test_patch_returns_last_modified_if_changed_meanwhile(self):
         self.assertNotIn('1970', error.headers['Last-Modified'])
 
     def test_delete_returns_412_if_changed_meanwhile(self):
-        self.resource.record_id = self.stored['id']
         self.assertRaises(httpexceptions.HTTPPreconditionFailed,
                           self.resource.delete)