Repository: marshmallow-code/marshmallow
Pull request #2051 (2022-09-15 15:49:38)

pre-commit autoupdate
Looks like pre-commit libs are a bit outdated (maybe showing we should be using pre-commit-ci ?).

I had to fix the config due to complicated issues I tried to explain in https://github.com/PyCQA/flake8/issues/1687.

Also, this updates flake8-bugbear which raises 2 new issues.

For one of them I think we can disable locally with a comment saying we know what we're doing.

The other error is about the use of `lru_cache`:

    src/marshmallow/schema.py:1063:6: B019 Use of `functools.lru_cache` or `functools.cache` on methods can lead to memory leaks. The cache may retain instance references, preventing garbage collection.

I'm not sure it is that bad since we set a maxsize. pylint doesn't seem to care in this case (https://github.com/PyCQA/pylint/pull/6181). We could ignore locally. Or rework the method but is it worth it?

Edit: remove `extend-select` thanks to @asottile suggestion in https://github.com/PyCQA/flake8/issues/1687#issuecomment-1248290052.

Edit: perhaps define a function rather than disable the warning about the lambda

lafrech: Pushing another approach for the variable in loop, using a function. Inspired by https://stackoverflow.com/a/3431699/4653485.
lafrech: About the lru_cache issue, I was about to add a noqa but I opened an issue in bugbear first: https://github.com/PyCQA/flake8-bugbear/issues/310.

Since the cache is global, it is at worst 8 instances so it shouldn't be an issue.

Patch:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 239f9a0e..95628ab8 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -1,21 +1,21 @@
 repos:
 - repo: https://github.com/asottile/pyupgrade
-  rev: v2.31.1
+  rev: v2.37.3
   hooks:
   - id: pyupgrade
     args: [--py37-plus]
 - repo: https://github.com/python/black
-  rev: 22.3.0
+  rev: 22.8.0
   hooks:
   - id: black
     language_version: python3
 - repo: https://github.com/pycqa/flake8
-  rev: 4.0.1
+  rev: 5.0.4
   hooks:
   - id: flake8
-    additional_dependencies: [flake8-bugbear==22.1.11]
+    additional_dependencies: [flake8-bugbear==22.9.11]
 - repo: https://github.com/pre-commit/mirrors-mypy
-  rev: v0.960
+  rev: v0.971
   hooks:
   - id: mypy
     additional_dependencies: [types-simplejson, types-pytz, packaging]
diff --git a/AUTHORS.rst b/AUTHORS.rst
index a0a928a9..f4aec167 100644
--- a/AUTHORS.rst
+++ b/AUTHORS.rst
@@ -170,4 +170,3 @@ Contributors (chronological)
 - Isira Seneviratne `@Isira-Seneviratne <https://github.com/Isira-Seneviratne>`_
 - Karthikeyan Singaravelan `@tirkarthi  <https://github.com/tirkarthi>`_
 - Marco Satti `@marcosatti  <https://github.com/marcosatti>`_
-- Ivo Reumkens `@vanHoi <https://github.com/vanHoi>`_
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 51cb1d4b..ec2a084b 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -1,23 +1,6 @@
 Changelog
 ---------
 
-3.20.0 (unreleased)
-*******************
-
-Other changes:
-
-- Officially support Python 3.11 (:pr:`2067`).
-
-3.19.0 (2022-11-11)
-*******************
-
-Features:
-
-- Add ``timestamp`` and ``timestamp_ms`` formats to `fields.DateTime`
-  (:issue:`612`).
-  Thanks :user:`vgavro` for the suggestion and thanks :user:`vanHoi` for
-  the PR.
-
 3.18.0 (2022-09-15)
 *******************
 
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index 18d4b92d..fd895e45 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -15,7 +15,7 @@ resources:
 jobs:
 - template: job--python-tox.yml@sloria
   parameters:
-    toxenvs: [lint, mypy-test, py37, py311]
+    toxenvs: [lint, mypy-test, py37, py310]
     os: linux
 - template: job--pypi-release.yml@sloria
   parameters:
diff --git a/docs/conf.py b/docs/conf.py
index 88318819..00669532 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -81,14 +81,14 @@ html_theme_options = {
 }
 
 html_sidebars = {
-    "index": ["about.html", "searchbox.html", "donate.html", "useful-links.html"],
+    "index": ["about.html", "donate.html", "useful-links.html", "searchbox.html"],
     "**": [
         "about.html",
-        "searchbox.html",
         "donate.html",
         "useful-links.html",
         "localtoc.html",
         "relations.html",
+        "searchbox.html",
     ],
 }
 
diff --git a/setup.cfg b/setup.cfg
index 330163ae..72024443 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -2,10 +2,9 @@
 license_files = LICENSE
 
 [flake8]
-extend-ignore = E203, E266, E501, E731, B903
 max-line-length = 90
 max-complexity = 18
-select = B,C,E,F,W,T4,B9
+extend-ignore = E203, E266, E501, E731, B903
 
 [tool:pytest]
 norecursedirs = .git .ropeproject .tox docs env venv tests/mypy_test_cases
diff --git a/setup.py b/setup.py
index 3ad3f385..31c877b0 100644
--- a/setup.py
+++ b/setup.py
@@ -4,8 +4,8 @@ from setuptools import setup, find_packages
 EXTRAS_REQUIRE = {
     "tests": ["pytest", "pytz", "simplejson"],
     "lint": [
-        "mypy==0.991",
-        "flake8==6.0.0",
+        "mypy==0.982",
+        "flake8==5.0.4",
         "flake8-bugbear==22.10.25",
         "pre-commit~=2.4",
     ],
@@ -83,7 +83,6 @@ setup(
         "Programming Language :: Python :: 3.8",
         "Programming Language :: Python :: 3.9",
         "Programming Language :: Python :: 3.10",
-        "Programming Language :: Python :: 3.11",
     ],
     test_suite="tests",
     project_urls={
diff --git a/src/marshmallow/__init__.py b/src/marshmallow/__init__.py
index 3ec24fe4..6ff75896 100644
--- a/src/marshmallow/__init__.py
+++ b/src/marshmallow/__init__.py
@@ -16,7 +16,7 @@ from marshmallow.utils import EXCLUDE, INCLUDE, RAISE, missing, pprint
 
 from . import fields
 
-__version__ = "3.19.0"
+__version__ = "3.18.0"
 __parsed_version__ = Version(__version__)
 __version_info__: tuple[int, int, int] | tuple[
     int, int, int, str, int
diff --git a/src/marshmallow/decorators.py b/src/marshmallow/decorators.py
index 54005daa..c71038d7 100644
--- a/src/marshmallow/decorators.py
+++ b/src/marshmallow/decorators.py
@@ -61,7 +61,7 @@ Example: ::
 from __future__ import annotations
 
 import functools
-from typing import Any, Callable, Dict, Tuple, cast
+from typing import Any, Callable, Dict, Optional, Tuple, Union, cast
 
 PRE_DUMP = "pre_dump"
 POST_DUMP = "post_dump"
@@ -72,7 +72,9 @@ VALIDATES_SCHEMA = "validates_schema"
 
 
 class MarshmallowHook:
-    __marshmallow_hook__ = None  # type: Dict[Tuple[str, bool] | str, Any] | None
+    __marshmallow_hook__ = (
+        None
+    )  # type: Optional[Dict[Union[Tuple[str, bool], str], Any]]
 
 
 def validates(field_name: str) -> Callable[..., Any]:
diff --git a/src/marshmallow/fields.py b/src/marshmallow/fields.py
index 386aa7c3..f153b5f9 100644
--- a/src/marshmallow/fields.py
+++ b/src/marshmallow/fields.py
@@ -157,9 +157,9 @@ class Field(FieldABC):
         default: typing.Any = missing_,
         data_key: str | None = None,
         attribute: str | None = None,
-        validate: (
-            None
-            | typing.Callable[[typing.Any], typing.Any]
+        validate: None
+        | (
+            typing.Callable[[typing.Any], typing.Any]
             | typing.Iterable[typing.Callable[[typing.Any], typing.Any]]
         ) = None,
         required: bool = False,
@@ -1218,14 +1218,11 @@ class DateTime(Field):
     Example: ``'2014-12-22T03:12:58.019077+00:00'``
 
     :param format: Either ``"rfc"`` (for RFC822), ``"iso"`` (for ISO8601),
-        ``"timestamp"``, ``"timestamp_ms"`` (for a POSIX timestamp) or a date format string.
-        If `None`, defaults to "iso".
+        or a date format string. If `None`, defaults to "iso".
     :param kwargs: The same keyword arguments that :class:`Field` receives.
 
     .. versionchanged:: 3.0.0rc9
         Does not modify timezone information on (de)serialization.
-    .. versionchanged:: 3.19
-        Add timestamp as a format.
     """
 
     SERIALIZATION_FUNCS = {
@@ -1233,17 +1230,13 @@ class DateTime(Field):
         "iso8601": utils.isoformat,
         "rfc": utils.rfcformat,
         "rfc822": utils.rfcformat,
-        "timestamp": utils.timestamp,
-        "timestamp_ms": utils.timestamp_ms,
-    }  # type: typing.Dict[str, typing.Callable[[typing.Any], str | float]]
+    }  # type: typing.Dict[str, typing.Callable[[typing.Any], str]]
 
     DESERIALIZATION_FUNCS = {
         "iso": utils.from_iso_datetime,
         "iso8601": utils.from_iso_datetime,
         "rfc": utils.from_rfc,
         "rfc822": utils.from_rfc,
-        "timestamp": utils.from_timestamp,
-        "timestamp_ms": utils.from_timestamp_ms,
     }  # type: typing.Dict[str, typing.Callable[[str], typing.Any]]
 
     DEFAULT_FORMAT = "iso"
@@ -1259,7 +1252,7 @@ class DateTime(Field):
         "format": '"{input}" cannot be formatted as a {obj_type}.',
     }
 
-    def __init__(self, format: str | None = None, **kwargs) -> None:
+    def __init__(self, format: str | None = None, **kwargs):
         super().__init__(**kwargs)
         # Allow this to be None. It may be set later in the ``_serialize``
         # or ``_deserialize`` methods. This allows a Schema to dynamically set the
@@ -1274,7 +1267,7 @@ class DateTime(Field):
             or self.DEFAULT_FORMAT
         )
 
-    def _serialize(self, value, attr, obj, **kwargs) -> str | float | None:
+    def _serialize(self, value, attr, obj, **kwargs):
         if value is None:
             return None
         data_format = self.format or self.DEFAULT_FORMAT
@@ -1284,7 +1277,7 @@ class DateTime(Field):
         else:
             return value.strftime(data_format)
 
-    def _deserialize(self, value, attr, data, **kwargs) -> dt.datetime:
+    def _deserialize(self, value, attr, data, **kwargs):
         if not value:  # Falsy values, e.g. '', None, [] are not valid
             raise self.make_error("invalid", input=value, obj_type=self.OBJ_TYPE)
         data_format = self.format or self.DEFAULT_FORMAT
@@ -1305,7 +1298,7 @@ class DateTime(Field):
                 ) from error
 
     @staticmethod
-    def _make_object_from_format(value, data_format) -> dt.datetime:
+    def _make_object_from_format(value, data_format):
         return dt.datetime.strptime(value, data_format)
 
 
@@ -1330,11 +1323,11 @@ class NaiveDateTime(DateTime):
         *,
         timezone: dt.timezone | None = None,
         **kwargs,
-    ) -> None:
+    ):
         super().__init__(format=format, **kwargs)
         self.timezone = timezone
 
-    def _deserialize(self, value, attr, data, **kwargs) -> dt.datetime:
+    def _deserialize(self, value, attr, data, **kwargs):
         ret = super()._deserialize(value, attr, data, **kwargs)
         if is_aware(ret):
             if self.timezone is None:
@@ -1367,11 +1360,11 @@ class AwareDateTime(DateTime):
         *,
         default_timezone: dt.tzinfo | None = None,
         **kwargs,
-    ) -> None:
+    ):
         super().__init__(format=format, **kwargs)
         self.default_timezone = default_timezone
 
-    def _deserialize(self, value, attr, data, **kwargs) -> dt.datetime:
+    def _deserialize(self, value, attr, data, **kwargs):
         ret = super()._deserialize(value, attr, data, **kwargs)
         if not is_aware(ret):
             if self.default_timezone is None:
@@ -2024,14 +2017,14 @@ class Function(Field):
 
     def __init__(
         self,
-        serialize: (
-            None
-            | typing.Callable[[typing.Any], typing.Any]
+        serialize: None
+        | (
+            typing.Callable[[typing.Any], typing.Any]
             | typing.Callable[[typing.Any, dict], typing.Any]
         ) = None,
-        deserialize: (
-            None
-            | typing.Callable[[typing.Any], typing.Any]
+        deserialize: None
+        | (
+            typing.Callable[[typing.Any], typing.Any]
             | typing.Callable[[typing.Any, dict], typing.Any]
         ) = None,
         **kwargs,
diff --git a/src/marshmallow/schema.py b/src/marshmallow/schema.py
index 92152696..f6ff3530 100644
--- a/src/marshmallow/schema.py
+++ b/src/marshmallow/schema.py
@@ -658,9 +658,17 @@ class Schema(base.SchemaABC, metaclass=SchemaMeta):
                     d_kwargs["partial"] = sub_partial
                 else:
                     d_kwargs["partial"] = partial
-                getter = lambda val: field_obj.deserialize(
-                    val, field_name, data, **d_kwargs
-                )
+
+                def getter(
+                    val, field_obj=field_obj, field_name=field_name, d_kwargs=d_kwargs
+                ):
+                    return field_obj.deserialize(
+                        val,
+                        field_name,
+                        data,
+                        **d_kwargs,
+                    )
+
                 value = self._call_and_store(
                     getter_func=getter,
                     data=raw_value,
@@ -1055,7 +1063,7 @@ class Schema(base.SchemaABC, metaclass=SchemaMeta):
             raise error
         self.on_bind_field(field_name, field_obj)
 
-    @lru_cache(maxsize=8)
+    @lru_cache(maxsize=8)  # noqa (https://github.com/PyCQA/flake8-bugbear/issues/310)
     def _has_processors(self, tag) -> bool:
         return bool(self._hooks[(tag, True)] or self._hooks[(tag, False)])
 
diff --git a/src/marshmallow/utils.py b/src/marshmallow/utils.py
index 7a16ff6d..fd651335 100644
--- a/src/marshmallow/utils.py
+++ b/src/marshmallow/utils.py
@@ -190,34 +190,6 @@ def from_iso_date(value):
     return dt.date(**kw)
 
 
-def from_timestamp(value: typing.Any) -> dt.datetime:
-    value = float(value)
-    if value < 0:
-        raise ValueError("Not a valid POSIX timestamp")
-
-    # Load a timestamp with utc as timezone to prevent using system timezone.
-    # Then set timezone to None, to let the Field handle adding timezone info.
-    return dt.datetime.fromtimestamp(value, tz=dt.timezone.utc).replace(tzinfo=None)
-
-
-def from_timestamp_ms(value: typing.Any) -> dt.datetime:
-    value = float(value)
-    return from_timestamp(value / 1000)
-
-
-def timestamp(
-    value: dt.datetime,
-) -> float:
-    if not is_aware(value):
-        # When a date is naive, use UTC as zone info to prevent using system timezone.
-        value = value.replace(tzinfo=dt.timezone.utc)
-    return value.timestamp()
-
-
-def timestamp_ms(value: dt.datetime) -> float:
-    return timestamp(value) * 1000
-
-
 def isoformat(datetime: dt.datetime) -> str:
     """Return the ISO8601-formatted representation of a datetime object.
 
diff --git a/tox.ini b/tox.ini
index 217bcfdb..af884d9f 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,5 +1,5 @@
 [tox]
-envlist = lint,mypy-test,py37,py38,py39,py310,py311,docs
+envlist = lint,mypy-test,py37,py38,py39,py310,docs
 
 [testenv]
 extras = tests


Test patch:
diff --git a/tests/test_deserialization.py b/tests/test_deserialization.py
index 32aa0991..c80a2581 100644
--- a/tests/test_deserialization.py
+++ b/tests/test_deserialization.py
@@ -526,50 +526,6 @@ class TestFieldDeserialization:
         else:
             assert field.deserialize(value) == expected
 
-    @pytest.mark.parametrize(
-        ("fmt", "value", "expected"),
-        [
-            ("timestamp", 1384043025, dt.datetime(2013, 11, 10, 0, 23, 45)),
-            ("timestamp", "1384043025", dt.datetime(2013, 11, 10, 0, 23, 45)),
-            ("timestamp", 1384043025, dt.datetime(2013, 11, 10, 0, 23, 45)),
-            ("timestamp", 1384043025.12, dt.datetime(2013, 11, 10, 0, 23, 45, 120000)),
-            (
-                "timestamp",
-                1384043025.123456,
-                dt.datetime(2013, 11, 10, 0, 23, 45, 123456),
-            ),
-            ("timestamp", 1, dt.datetime(1970, 1, 1, 0, 0, 1)),
-            ("timestamp_ms", 1384043025000, dt.datetime(2013, 11, 10, 0, 23, 45)),
-            ("timestamp_ms", 1000, dt.datetime(1970, 1, 1, 0, 0, 1)),
-        ],
-    )
-    def test_timestamp_field_deserialization(self, fmt, value, expected):
-        field = fields.DateTime(format=fmt)
-        assert field.deserialize(value) == expected
-
-        # By default, a datetime from a timestamp is never aware.
-        field = fields.NaiveDateTime(format=fmt)
-        assert field.deserialize(value) == expected
-
-        field = fields.AwareDateTime(format=fmt)
-        with pytest.raises(ValidationError, match="Not a valid aware datetime."):
-            field.deserialize(value)
-
-        # But it can be added by providing a default.
-        field = fields.AwareDateTime(format=fmt, default_timezone=central)
-        expected_aware = expected.replace(tzinfo=central)
-        assert field.deserialize(value) == expected_aware
-
-    @pytest.mark.parametrize("fmt", ["timestamp", "timestamp_ms"])
-    @pytest.mark.parametrize(
-        "in_value",
-        ["", "!@#", 0, -1, dt.datetime(2013, 11, 10, 1, 23, 45)],
-    )
-    def test_invalid_timestamp_field_deserialization(self, fmt, in_value):
-        field = fields.DateTime(format="timestamp")
-        with pytest.raises(ValidationError, match="Not a valid datetime."):
-            field.deserialize(in_value)
-
     @pytest.mark.parametrize(
         ("fmt", "timezone", "value", "expected"),
         [
diff --git a/tests/test_serialization.py b/tests/test_serialization.py
index 56f0773e..51671ddf 100644
--- a/tests/test_serialization.py
+++ b/tests/test_serialization.py
@@ -579,38 +579,6 @@ class TestFieldSerialization:
         field = fields.DateTime(format=fmt)
         assert field.serialize("d", {"d": value}) == expected
 
-    @pytest.mark.parametrize(
-        ("fmt", "value", "expected"),
-        [
-            ("timestamp", dt.datetime(1970, 1, 1), 0),
-            ("timestamp", dt.datetime(2013, 11, 10, 0, 23, 45), 1384043025),
-            (
-                "timestamp",
-                dt.datetime(2013, 11, 10, 0, 23, 45, tzinfo=dt.timezone.utc),
-                1384043025,
-            ),
-            (
-                "timestamp",
-                central.localize(dt.datetime(2013, 11, 10, 0, 23, 45), is_dst=False),
-                1384064625,
-            ),
-            ("timestamp_ms", dt.datetime(2013, 11, 10, 0, 23, 45), 1384043025000),
-            (
-                "timestamp_ms",
-                dt.datetime(2013, 11, 10, 0, 23, 45, tzinfo=dt.timezone.utc),
-                1384043025000,
-            ),
-            (
-                "timestamp_ms",
-                central.localize(dt.datetime(2013, 11, 10, 0, 23, 45), is_dst=False),
-                1384064625000,
-            ),
-        ],
-    )
-    def test_datetime_field_timestamp(self, fmt, value, expected):
-        field = fields.DateTime(format=fmt)
-        assert field.serialize("d", {"d": value}) == expected
-
     @pytest.mark.parametrize("fmt", ["iso", "iso8601", None])
     @pytest.mark.parametrize(
         ("value", "expected"),