internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: Kinto/kinto

Issue:
What should `event.payload["timestamp"]` represent?
Refs #945 and #1731. The documentation describes the timestamp field as representing "the time of the event". However, as far as I can tell, the `timestamp` field in an event represents a collection timestamp calculated relatively early in the request, before the action that triggered event happened. It doesn't represent the time that the event happens, nor the current last_modified / collection_timestamp of anything reflected in the event.

I think I'd like to change the `timestamp` key in the payload to correspond to the updated collection timestamp of the `(parent_id, collection_id)` for which the event was fired.


Discussion:

Resolving patch:
diff --git a/kinto/core/resource/__init__.py b/kinto/core/resource/__init__.py
--- a/kinto/core/resource/__init__.py
+++ b/kinto/core/resource/__init__.py
@@ -737,8 +737,13 @@ def postprocess(self, result, action=ACTIONS.READ, old=None):
         }
 
         parent_id = self.get_parent_id(self.request)
+        # Use self.model.timestamp() instead of self.timestamp because
+        # self.timestamp is @reify'd relatively early in the request,
+        # so doesn't correspond to any time that is relevant to the
+        # event. See #1769.
+        timestamp = self.model.timestamp()
         self.request.notify_resource_event(parent_id=parent_id,
-                                           timestamp=self.timestamp,
+                                           timestamp=timestamp,
                                            data=result,
                                            action=action,
                                            old=old)


Test patch:
diff --git a/tests/core/resource/test_events.py b/tests/core/resource/test_events.py
--- a/tests/core/resource/test_events.py
+++ b/tests/core/resource/test_events.py
@@ -118,6 +118,18 @@ def test_put_sends_create_action(self):
         self.assertEqual(self.events[0].payload['action'],
                          ACTIONS.CREATE.value)
 
+    def test_put_event_has_sensible_timestamp(self):
+        body = {**self.body}
+        body['data']['id'] = record_id = str(uuid.uuid4())
+        record_url = self.get_item_url(record_id)
+        resp = self.app.put_json(record_url, body,
+                                 headers=self.headers, status=201)
+        self.assertEqual(len(self.events), 1)
+        self.assertEqual(self.events[0].payload['action'],
+                         ACTIONS.CREATE.value)
+        self.assertEqual(self.events[0].payload['timestamp'],
+                         resp.json['data']['last_modified'])
+
     def test_not_triggered_on_failed_put(self):
         body = {**self.body}
         body['data']['id'] = record_id = str(uuid.uuid4())
Repository: Kinto/kinto

Issue:
Remove colander deprecations
```
/home/mathieu/Code/Mozilla/kinto/.venv/lib/python3.6/site-packages/cornice/validators/_colander.py:110: DeprecationWarning: Setting schema to a class is deprecated.  Set schema to an instance instead.
  schema = _ensure_instantiated(schema)
```


Discussion:
Hey! Can I work on this issue? Also can you please elaborate on the issue? Thanks :)

Yes, you can work on it!

Basically, when you run the tests, you will see some deprecations like the one I pasted above.

The idea is to fix the code to get of rid of them :)

Resolving patch:
diff --git a/kinto/core/views/batch.py b/kinto/core/views/batch.py
--- a/kinto/core/views/batch.py
+++ b/kinto/core/views/batch.py
@@ -111,7 +111,7 @@ class ErrorResponseSchema(colander.MappingSchema):
                 description='Batch operations')
 
 
-@batch.post(schema=BatchRequest,
+@batch.post(schema=BatchRequest(),
             validators=(colander_validator,),
             content_type=CONTENT_TYPES,
             permission=NO_PERMISSION_REQUIRED,


Test patch:
Repository: Kinto/kinto

Issue:
Crash in Postgres backend when querystring contains NUL character (0x00)
![screenshot from 2018-07-02 10-40-31](https://user-images.githubusercontent.com/546692/42153765-64ff0752-7de4-11e8-81f6-2531fd8719cd.png)


:laughing: 

![screenshot_2018-07-02 valueerror a string literal cannot contain nul 0x00 characters](https://user-images.githubusercontent.com/546692/42153904-b238b82e-7de4-11e8-8dbd-5e9bbef7efed.png)



Discussion:
For those who want to have a look at this, this could be avoided by adding some safety check here for example:

https://github.com/Kinto/kinto/blob/21bac9b787639da76b38f535aa34e1f0391e48e9/kinto/core/utils.py#L106-L117



Hey, if this is still open I would like to work on this - where would be the best place to put a test for this?
And to add to this, what would the expected result be? 
> Hey, if this is still open I would like to work on this

Great!

>  where would be the best place to put a test for this?

I guess that somewhere around here:
https://github.com/Kinto/kinto/blob/f9e3c6ea2c2a44e855c16cf2969dd73200ac94ac/tests/core/test_views_errors.py#L114

> And to add to this, what would the expected result be?

If the querystring contains a bad character, the view should return 400

Resolving patch:
diff --git a/kinto/core/resource/__init__.py b/kinto/core/resource/__init__.py
--- a/kinto/core/resource/__init__.py
+++ b/kinto/core/resource/__init__.py
@@ -1022,6 +1022,9 @@ def _extract_filters(self):
                 if has_invalid_value:
                     raise_invalid(self.request, **error_details)
 
+            if '\x00' in field or '\x00' in str(value):
+                raise_invalid(self.request, **error_details)
+
             if field == self.model.modified_field and value == '':
                 raise_invalid(self.request, **error_details)
 


Test patch:
diff --git a/tests/core/test_views_errors.py b/tests/core/test_views_errors.py
--- a/tests/core/test_views_errors.py
+++ b/tests/core/test_views_errors.py
@@ -116,6 +116,18 @@ def test_400_with_invalid_url_path(self):
         self.assertFormattedError(response, 400, ERRORS.INVALID_PARAMETERS, "Bad Request",
                                   "Invalid URL path.")
 
+    def test_400_when_query_field_contains_nul_character(self):
+        response = self.app.get(self.sample_url + '?field\x00="2"',
+                                headers=self.headers, status=400)
+        self.assertFormattedError(response, 400, ERRORS.INVALID_PARAMETERS,
+                                  "Invalid parameters", "Invalid value for")
+
+    def test_400_when_query_value_contains_nul_character(self):
+        response = self.app.get(self.sample_url + '?field="\x00"',
+                                headers=self.headers, status=400)
+        self.assertFormattedError(response, 400, ERRORS.INVALID_PARAMETERS,
+                                  "Invalid parameters", "Invalid value for")
+
     def test_info_link_in_error_responses_can_be_configured(self):
         with mock.patch('tests.core.testapp.views.Mushroom._extract_filters',
                         side_effect=ValueError):
Repository: Kinto/kinto

Issue:
Invalid account_create_principals key
The `account:create` check_permission code is looking at `account_account:create_principals` settings key rather than `account_create_principals`


Discussion:

Resolving patch:
diff --git a/kinto/core/authorization.py b/kinto/core/authorization.py
--- a/kinto/core/authorization.py
+++ b/kinto/core/authorization.py
@@ -176,7 +176,11 @@ def check_permission(self, principals, bound_perms):
         if not bound_perms:
             bound_perms = [(self.resource_name, self.required_permission)]
         for (_, permission) in bound_perms:
-            setting = "{}_{}_principals".format(self.resource_name, permission)
+            # With Kinto inheritance tree, we can have: `permission = "record:create"`
+            if self.resource_name and permission.startswith(self.resource_name):
+                setting = "{}_principals".format(permission.replace(":", "_"))
+            else:
+                setting = "{}_{}_principals".format(self.resource_name, permission)
             allowed_principals = aslist(self._settings.get(setting, ""))
             if allowed_principals:
                 if bool(set(allowed_principals) & set(principals)):


Test patch:
diff --git a/tests/core/test_authorization.py b/tests/core/test_authorization.py
--- a/tests/core/test_authorization.py
+++ b/tests/core/test_authorization.py
@@ -157,6 +157,15 @@ def test_fetch_shared_records_sets_shared_ids_if_empty(self):
 
         self.assertEqual(context.shared_ids, [])
 
+    def test_permits_takes_route_factory_allowed_principals_into_account_for_object_creation(self):
+        request = DummyRequest()
+        context = RouteFactory(request)
+        context._check_permission.return_value = False
+        context.resource_name = "book"
+        context.required_permission = "book:create"
+        context._settings = {"book_create_principals": "fxa:user"}
+        self.assertTrue(context.check_permission(["fxa:user"], None))
+
 
 class AuthorizationPolicyTest(unittest.TestCase):
     def setUp(self):
Repository: Kinto/kinto

Issue:
Per collection caching using dot separator
Per collection caching can be configured in settings using `_` as separator:

```ini
kinto.bid_cid_record_cache_expires_seconds = 60
```
Since `_` is valid character in both bucket and collection ids, this could be confusing. I propose that we change it to `.`


Discussion:

Resolving patch:
diff --git a/kinto/views/records.py b/kinto/views/records.py
--- a/kinto/views/records.py
+++ b/kinto/views/records.py
@@ -104,12 +104,16 @@ def _handle_cache_expires(self, response):
 
         cache_expires = self._collection.get('cache_expires')
         if cache_expires is None:
-            by_bucket = '{}_record_cache_expires_seconds'.format(self.bucket_id)
-            by_collection = '{}_{}_record_cache_expires_seconds'.format(
+            by_collection = '{}.{}.record_cache_expires_seconds'.format(
                 self.bucket_id, self.collection_id)
+            by_bucket = '{}.record_cache_expires_seconds'.format(self.bucket_id)
+            by_collection_legacy = '{}_{}_record_cache_expires_seconds'.format(
+                self.bucket_id, self.collection_id)
+            by_bucket_legacy = '{}_record_cache_expires_seconds'.format(self.bucket_id)
             settings = self.request.registry.settings
-            cache_expires = settings.get(by_collection,
-                                         settings.get(by_bucket))
-
+            for s in (by_collection, by_bucket, by_collection_legacy, by_bucket_legacy):
+                cache_expires = settings.get(s)
+                if cache_expires is not None:
+                    break
         if cache_expires is not None:
             response.cache_expires(seconds=int(cache_expires))


Test patch:
diff --git a/tests/test_views_collections_cache.py b/tests/test_views_collections_cache.py
--- a/tests/test_views_collections_cache.py
+++ b/tests/test_views_collections_cache.py
@@ -40,8 +40,8 @@ class SpecificSettingsTest(BaseWebTest, unittest.TestCase):
     @classmethod
     def get_app_settings(cls, extras=None):
         settings = super().get_app_settings(extras)
-        settings['kinto.blog_record_cache_expires_seconds'] = '30'
-        settings['kinto.browser_top500_record_cache_expires_seconds'] = '60'
+        settings['kinto.blog.record_cache_expires_seconds'] = '30'
+        settings['kinto.browser.top500.record_cache_expires_seconds'] = '60'
         return settings
 
     def setUp(self):
Repository: Kinto/kinto

Issue:
Migrate to JSONschema Draft v7
Currently we use Draft4. The library that we use supports Draft7

https://json-schema.org/specification.html

The idea would be:
* to use `Draft7Validator` instead of `Draft4Validator` (nothing more...)
* list main changes and add links in CHANGELOG (see spec release notes)
* check compatibility with kinto-admin



Discussion:
I would like to take on this issue if it's still open?
Yes sure!

Resolving patch:
diff --git a/kinto/schema_validation.py b/kinto/schema_validation.py
--- a/kinto/schema_validation.py
+++ b/kinto/schema_validation.py
@@ -1,5 +1,5 @@
 import colander
-from jsonschema import Draft4Validator, ValidationError, SchemaError, RefResolutionError, validate
+from jsonschema import Draft7Validator, ValidationError, SchemaError, RefResolutionError, validate
 from pyramid.settings import asbool
 
 from kinto.core import utils
@@ -27,7 +27,7 @@ def deserialize(self, cstruct=colander.null):
 
 def check_schema(data):
     try:
-        Draft4Validator.check_schema(data)
+        Draft7Validator.check_schema(data)
     except SchemaError as e:
         message = e.path.pop() + e.message
         raise ValidationError(message)


Test patch:
Repository: Kinto/kinto

Issue:
Updated Error on Version File Missing
4XX errors are client errors. In that case, if the file is not on the server, the client is not guilty
Raise error instead of returning 404 on __version__ if version file is missing
https://github.com/Kinto/kinto/blame/master/kinto/core/views/version.py#L49

suggested by @peterbe 

4XX errors are client errors. In that case, if the file is not on the server, the client is not guilty ;)


Discussion:

Django has a great "solution" to this, the perfectly named [`django.exceptions.ImproperlyConfigured` exception](https://docs.djangoproject.com/en/2.1/ref/exceptions/#improperlyconfigured). 


Pyramid has the same `ConfigurationError` :)
wouldn't something like `raise VersionNotFound` do the trick?
> wouldn't something like raise VersionNotFound do the trick?

Whatever, we just need to raise an exception with a clear message that the server administrators would receive when a client tries to access this endpoint if the file does not exist...
I'll do this in an hour or so, no worries!
Is this issue still open?
Sorry, I've been busy.
I made a pull request here: https://github.com/Kinto/kinto/pull/1860

If I messed up, feel free to take it

Resolving patch:
diff --git a/kinto/core/views/version.py b/kinto/core/views/version.py
--- a/kinto/core/views/version.py
+++ b/kinto/core/views/version.py
@@ -2,7 +2,6 @@
 import os
 
 import colander
-from pyramid import httpexceptions
 from pyramid.security import NO_PERMISSION_REQUIRED
 from kinto.core import Service
 
@@ -46,4 +45,4 @@ def version_view(request):
                 version_view.__json__ = json.load(f)
                 return version_view.__json__  # First one wins.
 
-    raise httpexceptions.HTTPNotFound()
+    raise FileNotFoundError("Version file missing from {}".format(files.join(",")))


Test patch:
diff --git a/tests/core/test_views_version.py b/tests/core/test_views_version.py
--- a/tests/core/test_views_version.py
+++ b/tests/core/test_views_version.py
@@ -25,8 +25,8 @@ def test_return_the_version_file_in_current_folder_if_present(self):
                 response = self.app.get("/__version__")
                 assert response.json == content
 
-    def test_return_a_404_if_version_file_if_not_present(self):
-        self.app.get("/__version__", status=404)
+    def test_return_a_500_if_version_file_if_not_present(self):
+        self.app.get("/__version__", status=500)
 
     def test_return_the_version_file_specified_in_setting_if_present(self):
         custom_path = tempfile.mktemp()
Repository: Kinto/kinto

Issue:
Remove deprecation: The 'warn' method is deprecated, use 'warning' instead
```
/home/mathieu/Code/Mozilla/kinto/kinto/core/authorization.py:107: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
    extra=dict(userid=principals[0], uri=object_id, perm=permission),
  /home/mathieu/Code/Mozilla/kinto/kinto/core/authorization.py:107: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
    extra=dict(userid=principals[0], uri=object_id, perm=permission),

```


Discussion:
Hello, can I begin working on this issue? Is there anything else I should know before I begin?
Please, go on!
Not much more to add to the instructions given in the message :)

Resolving patch:
diff --git a/kinto/core/authorization.py b/kinto/core/authorization.py
--- a/kinto/core/authorization.py
+++ b/kinto/core/authorization.py
@@ -99,7 +99,7 @@ def permits(self, context, principals, permission):
                 allowed = context.check_permission(principals, bound_perms)
 
         if not allowed:
-            logger.warn(
+            logger.warning(
                 "Permission %r on %r not granted to %r.",
                 permission,
                 object_id,
diff --git a/kinto/core/errors.py b/kinto/core/errors.py
--- a/kinto/core/errors.py
+++ b/kinto/core/errors.py
@@ -204,7 +204,7 @@ def request_GET(request):
     except UnicodeDecodeError:
         querystring = request.environ.get("QUERY_STRING", "")
         logger = logging.getLogger(__name__)
-        logger.warn("Error decoding QUERY_STRING: %s" % request.environ)
+        logger.warning("Error decoding QUERY_STRING: %s" % request.environ)
         raise http_error(
             httpexceptions.HTTPBadRequest(),
             errno=ERRORS.INVALID_PARAMETERS,


Test patch:
diff --git a/tests/core/test_authorization.py b/tests/core/test_authorization.py
--- a/tests/core/test_authorization.py
+++ b/tests/core/test_authorization.py
@@ -191,7 +191,7 @@ def test_permits_logs_authz_failures(self):
         userid = "portier:yourself"
         object_id = "/articles/43/comments/2"
         perm = "read"
-        mocked.warn.assert_called_with(
+        mocked.warning.assert_called_with(
             "Permission %r on %r not granted to %r.",
             perm,
             object_id,
Repository: Kinto/kinto

Issue:
Fallback to jsonschema DraftV4 in case Draftv7 is not available
Mitigate jsonschema alfa version mandatory.

This is quite a blocker to be honest. I upgraded a bunch of packages related to Kinto and I guess we could mitigate the issue by fallbacking to Draftv4.

Refs #1808 


Discussion:

Resolving patch:
diff --git a/kinto/schema_validation.py b/kinto/schema_validation.py
--- a/kinto/schema_validation.py
+++ b/kinto/schema_validation.py
@@ -1,5 +1,11 @@
 import colander
-from jsonschema import Draft7Validator, ValidationError, SchemaError, RefResolutionError, validate
+from jsonschema import ValidationError, SchemaError, RefResolutionError, validate
+
+try:  # pragma: no cover
+    from jsonschema import Draft7Validator as DraftValidator
+except ImportError:
+    from jsonschema import Draft4Validator as DraftValidator
+
 from pyramid.settings import asbool
 
 from kinto.core import utils
@@ -27,7 +33,7 @@ def deserialize(self, cstruct=colander.null):
 
 def check_schema(data):
     try:
-        Draft7Validator.check_schema(data)
+        DraftValidator.check_schema(data)
     except SchemaError as e:
         message = e.path.pop() + e.message
         raise ValidationError(message)
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -24,7 +24,7 @@ def read_file(filename):
     "cornice",
     "cornice_swagger >= 0.5.1",
     "dockerflow",
-    "jsonschema >= 3.0.0a1",
+    "jsonschema",
     "jsonpatch",
     "logging-color-formatter >= 1.0.1",  # Message interpolations.
     "python-dateutil",


Test patch:
Repository: Kinto/kinto

Issue:
Remove deprecation warning: Resource `mapping` is deprecated, use `schema`
```
tests/core/resource/test_viewset.py::ViewSetTest::test_a_default_schema_is_added_when_method_doesnt_match
  /home/mathieu/Code/Mozilla/kinto/kinto/core/resource/viewset.py:160: DeprecationWarning: Resource `mapping` is deprecated, use `schema`
    warnings.warn(message, DeprecationWarning)
```



Discussion:

Resolving patch:
diff --git a/kinto/core/resource/viewset.py b/kinto/core/resource/viewset.py
--- a/kinto/core/resource/viewset.py
+++ b/kinto/core/resource/viewset.py
@@ -1,5 +1,4 @@
 import functools
-import warnings
 
 import colander
 from cornice.validators import colander_validator
@@ -156,10 +155,6 @@ def get_record_schema(self, resource_cls, method):
             resource_schema = SimpleSchema
         else:
             resource_schema = resource_cls.schema
-            if hasattr(resource_cls, "mapping"):
-                message = "Resource `mapping` is deprecated, use `schema`"
-                warnings.warn(message, DeprecationWarning)
-                resource_schema = resource_cls.mapping.__class__
 
         record_schema = RecordSchema().bind(data=resource_schema())
 


Test patch:
diff --git a/tests/core/resource/test_viewset.py b/tests/core/resource/test_viewset.py
--- a/tests/core/resource/test_viewset.py
+++ b/tests/core/resource/test_viewset.py
@@ -379,16 +379,6 @@ def test_get_service_arguments_has_default_factory(self):
         args = viewset.get_service_arguments()
         self.assertEqual(args["factory"], authorization.RouteFactory)
 
-    def test_mapping_is_deprecated(self):
-        viewset = ShareableViewSet()
-        viewset.responses = mock.MagicMock()
-        resource = mock.MagicMock()
-        resource.mapping = mock.MagicMock()
-        with mock.patch("kinto.core.resource.viewset.warnings") as mocked:
-            viewset.collection_arguments(resource, "GET")
-            msg = "Resource `mapping` is deprecated, use `schema`"
-            mocked.warn.assert_called_with(msg, DeprecationWarning)
-
 
 class RegisterTest(unittest.TestCase):
     def setUp(self):
Repository: Kinto/kinto

Issue:
Kinto not able to start - Regression due to jsonschema Draft7Validator
#1808 introduced a regression, because the Draft7Validator is not currently released by the jsonschema package (they introduced it in master but it's not released yet)

Solution is either to depend on the master branch of jsonschema (not recommended I guess) or revert this change back until a new version is issued.


Discussion:

Resolving patch:
diff --git a/kinto/schema_validation.py b/kinto/schema_validation.py
--- a/kinto/schema_validation.py
+++ b/kinto/schema_validation.py
@@ -1,5 +1,11 @@
 import colander
-from jsonschema import Draft7Validator, ValidationError, SchemaError, RefResolutionError, validate
+from jsonschema import ValidationError, SchemaError, RefResolutionError, validate
+
+try:  # pragma: no cover
+    from jsonschema import Draft7Validator as DraftValidator
+except ImportError:
+    from jsonschema import Draft4Validator as DraftValidator
+
 from pyramid.settings import asbool
 
 from kinto.core import utils
@@ -27,7 +33,7 @@ def deserialize(self, cstruct=colander.null):
 
 def check_schema(data):
     try:
-        Draft7Validator.check_schema(data)
+        DraftValidator.check_schema(data)
     except SchemaError as e:
         message = e.path.pop() + e.message
         raise ValidationError(message)
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -61,7 +61,7 @@ def read_file(filename):
 
 setup(
     name="kinto",
-    version="11.2.0",
+    version="11.2.1",
     description="Kinto Web Service - Store, Sync, Share, and Self-Host.",
     long_description="{}\n\n{}\n\n{}".format(README, CHANGELOG, CONTRIBUTORS),
     license="Apache License (2.0)",


Test patch: