Repository: marshmallow-code/marshmallow
Pull request #2102 (2023-02-15 08:23:56)

fix: unhandled OSError and OverflowErrors from utils.from_timestamp
When running [schemathesis](https://github.com/schemathesis/schemathesis) against my smorest app I found some unexpected 500 errors.  They where caused by  `utils.from_timestamp` which doesn't  handle the [OverflowError or OSError that can be raised by  datetime.fromtimestamp](https://docs.python.org/3/library/datetime.html#datetime.datetime.fromtimestamp)

What's has been done:

- OverflowError and OSError are caught and re-throwed as ValueError so that they can be handle properly.
- Tests for `utiles.from_timestamp` were added to avoid regression

TheBigRoomXXL: One thing to note is that a large timestamp will sometime raise ValueError 'year XXXXX is out of range' instead of OSError, that depends on the machine or OS used. 
lafrech: Thanks.

It would be nice to test both exceptions, if this is feasible (not sure how to reproduce "localtime() or gmtime() failure".

Also, while it is nice to keep the unit test close to the util function, what we really want to assert is that the field generates a `ValidationError` rather than a crash. Maybe this ought to be tested too.

I had never heard about schemathesis. I shall try this out.
lafrech: Good!

> I could not find information on how to reproduce. I don't think it's possible without running the CI on multiple differents systems.

What you could do is mock `fromtimestamp` to force `OSError`, if only to have a non-reg test in case we'd be tempted someday to remove the `OSError` catch.
TheBigRoomXXL: I don't have much experience with mocking in Python. I couldn't mock` datetime.datetime.fromtimestamp` directly because python built-in types are immutable so i mocked the entire `datetime` class instead.


TheBigRoomXXL: OK! It should be the last change to the merge request. Thank you for your time and all the feedbacks! 
TheBigRoomXXL: Rebase done. Don't worry for the delay. 

Thank you.

Patch:
diff --git a/AUTHORS.rst b/AUTHORS.rst
index a4a64eff..dc6dd929 100644
--- a/AUTHORS.rst
+++ b/AUTHORS.rst
@@ -172,3 +172,4 @@ Contributors (chronological)
 - Marco Satti `@marcosatti  <https://github.com/marcosatti>`_
 - Ivo Reumkens `@vanHoi <https://github.com/vanHoi>`_
 - Aditya Tewary `@aditkumar72 <https://github.com/aditkumar72>`_
+- Sebastien Lovergne `@TheBigRoomXXL <https://github.com/TheBigRoomXXL>`_
diff --git a/src/marshmallow/utils.py b/src/marshmallow/utils.py
index 7a16ff6d..757f4a8d 100644
--- a/src/marshmallow/utils.py
+++ b/src/marshmallow/utils.py
@@ -197,7 +197,12 @@ def from_timestamp(value: typing.Any) -> dt.datetime:
 
     # Load a timestamp with utc as timezone to prevent using system timezone.
     # Then set timezone to None, to let the Field handle adding timezone info.
-    return dt.datetime.fromtimestamp(value, tz=dt.timezone.utc).replace(tzinfo=None)
+    try:
+        return dt.datetime.fromtimestamp(value, tz=dt.timezone.utc).replace(tzinfo=None)
+    except OverflowError as exc:
+        raise ValueError("Timestamp is too large") from exc
+    except OSError as exc:
+        raise ValueError("Error converting value to datetime") from exc
 
 
 def from_timestamp_ms(value: typing.Any) -> dt.datetime:


Test patch:
diff --git a/tests/test_deserialization.py b/tests/test_deserialization.py
index 213c4b41..6a2c7a87 100644
--- a/tests/test_deserialization.py
+++ b/tests/test_deserialization.py
@@ -4,6 +4,8 @@ import ipaddress
 import decimal
 import math
 
+from unittest.mock import patch
+
 import pytest
 
 from marshmallow import EXCLUDE, INCLUDE, RAISE, fields, Schema, validate
@@ -21,6 +23,20 @@ from tests.base import (
 )
 
 
+class MockDateTimeOverflowError(dt.datetime):
+    """Used to simulate the possible OverflowError of datetime.fromtimestamp"""
+
+    def fromtimestamp(self, *args, **kwargs):
+        raise OverflowError()
+
+
+class MockDateTimeOSError(dt.datetime):
+    """Used to simulate the possible OSError of datetime.fromtimestamp"""
+
+    def fromtimestamp(self, *args, **kwargs):
+        raise OSError()
+
+
 class TestDeserializingNone:
     @pytest.mark.parametrize("FieldClass", ALL_FIELDS)
     def test_fields_allow_none_deserialize_to_none(self, FieldClass):
@@ -565,10 +581,21 @@ class TestFieldDeserialization:
         ["", "!@#", 0, -1, dt.datetime(2013, 11, 10, 1, 23, 45)],
     )
     def test_invalid_timestamp_field_deserialization(self, fmt, in_value):
-        field = fields.DateTime(format="timestamp")
+        field = fields.DateTime(format=fmt)
         with pytest.raises(ValidationError, match="Not a valid datetime."):
             field.deserialize(in_value)
 
+    #  Regression test for https://github.com/marshmallow-code/marshmallow/pull/2102
+    @pytest.mark.parametrize("fmt", ["timestamp", "timestamp_ms"])
+    @pytest.mark.parametrize(
+        "mock_fromtimestamp", [MockDateTimeOSError, MockDateTimeOverflowError]
+    )
+    def test_oversized_timestamp_field_deserialization(self, fmt, mock_fromtimestamp):
+        with patch("datetime.datetime", mock_fromtimestamp):
+            field = fields.DateTime(format=fmt)
+            with pytest.raises(ValidationError, match="Not a valid datetime."):
+                field.deserialize(99999999999999999)
+
     @pytest.mark.parametrize(
         ("fmt", "timezone", "value", "expected"),
         [
diff --git a/tests/test_utils.py b/tests/test_utils.py
index 0a134d1f..e440ce26 100644
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -228,6 +228,31 @@ def test_from_iso_date():
     assert_date_equal(result, d)
 
 
+@pytest.mark.parametrize(
+    ("value", "expected"),
+    [
+        (1676386740, dt.datetime(2023, 2, 14, 14, 59, 00)),
+        (1676386740.58, dt.datetime(2023, 2, 14, 14, 59, 00, 580000)),
+    ],
+)
+def test_from_timestamp(value, expected):
+    result = utils.from_timestamp(value)
+    assert type(result) == dt.datetime
+    assert result == expected
+
+
+def test_from_timestamp_with_negative_value():
+    value = -10
+    with pytest.raises(ValueError, match=r"Not a valid POSIX timestamp"):
+        utils.from_timestamp(value)
+
+
+def test_from_timestamp_with_overflow_value():
+    value = 9223372036854775
+    with pytest.raises(ValueError):
+        utils.from_timestamp(value)
+
+
 def test_get_func_args():
     def f1(foo, bar):
         pass