Repository: HypothesisWorks/hypothesis

Issue:
`st.datetimes(allow_imaginary=False)` can raise internal error
`strategies.datetime` may raise such an error when passed `allow_imaginary=False` and tzinfo is None.  

I have the following:

```
dates = datetimes(
    min_value=datetime.datetime(2000, 1,1),
    max_value=datetime.datetime(9000, 12,31),
    allow_imaginary=False,
)
```

The error looks like (pytest style):

```
value = datetime.datetime(2000, 1, 1, 0, 0)

    def datetime_does_not_exist(value):
        # This function tests whether the given datetime can be round-tripped to and
        # from UTC.  It is an exact inverse of (and very similar to) the dateutil method
        # https://dateutil.readthedocs.io/en/stable/tz.html#dateutil.tz.datetime_exists
        try:
            # Does the naive portion of the datetime change when round-tripped to
            # UTC?  If so, or if this overflows, we say that it does not exist.
            roundtrip = value.astimezone(dt.timezone.utc).astimezone(value.tzinfo)
        except OverflowError:
            # Overflows at datetime.min or datetime.max boundary condition.
            # Rejecting these is acceptable, because timezones are close to
            # meaningless before ~1900 and subject to a lot of change by
            # 9999, so it should be a very small fraction of possible values.
            return True
>       assert value.tzinfo is roundtrip.tzinfo, "so only the naive portions are compared"
E       AssertionError: so only the naive portions are compared
E       assert None is datetime.timezone(datetime.timedelta(days=-1, seconds=68400), 'CST')
E        +  where None = datetime.datetime(2000, 1, 1, 0, 0).tzinfo
E        +  and   datetime.timezone(datetime.timedelta(days=-1, seconds=68400), 'CST') = datetime.datetime(2000, 1, 1, 0, 0, tzinfo=datetime.timezone(datetime.timedelta(days=-1, seconds=68400), 'CST')).tzinfo

.venv/lib/python3.8/site-packages/hypothesis/strategies/_internal/datetime.py:75: AssertionError
```
Notice that `value.tzinfo` is None, so `astimezone` uses the local timezone.



Discussion:
To work around this issue I'm using:

```python
dates = st.datetimes(
    min_value=datetime.datetime(2000, 1, 1),
    max_value=datetime.datetime(9000, 12, 31),
    timezones=st.just(datetime.timezone.utc),
    allow_imaginary=False,
).map(lambda d: d.replace(tzinfo=None, fold=0))
```
Thanks for reporting this!  I think our fix is probably to disallow `allow_imaginary=False` for naive (`tzinfo=None`) datetimes, since imaginary times are only defined with respect to a particular timezone.

--------

Unfortunately your workaround still permits the generation of imaginary times - while UTC has no imaginary times, replacing UTC with your local timezone may create one.  Instead, use:

```python
dates = st.datetimes(
    min_value=datetime.datetime(2000, 1, 1),
    max_value=datetime.datetime(9000, 12, 31),
    timezones=st.just(datetime.timezone.utc),
).map(lambda d: d.astimezone(tz=None))
```
@Zac-HD 

Thanks.  

But, if I understood the [docs](https://docs.python.org/3/library/datetime.html?highlight=astimezone#datetime.datetime.astimezone) correctly, `d.astimezone(tz=None)`  does use the local timezone and returns a non-naive datetime.  

Since, what I need are naive datetimes (but still regarded as datetimes in UTC), I think `d.replace(tzinfo=None)` is the way to go.  Am I missing something?
There's no such thing as an imaginary datetime with `tzinfo=None`, so you can simply use 

```python
dates = st.datetimes(
    min_value=datetime.datetime(2000, 1, 1), max_value=datetime.datetime(9000, 12, 31)
)
```

More generally, if your datetime objects represent a time in UTC I would **strongly** recommend using `tzinfo=UTC` - as you've seen above, `tzinfo=None` has some pretty unpredicatble semantics around comparisons and conversion to local time.  The [docs for `datetime.utcnow()` carry the same warning](https://docs.python.org/3/library/datetime.html#datetime.datetime.utcnow).
@Zac-HD 

Thank a lot.  I'm still wrapping my head around the imaginary datetime stuff.  Unfortunately, I cannot use `tzinfo=UTC` because I'm interfacing with the Odoo framework which requires all datetime objects be naive although regarded as in UTC.

Best regards,
Manuel.
Well... I'd try to use `tzinfo=UTC` in all your code, and then replace with `None` *just* before calling into their framework.  And open a ticket asking for support of non-naive datetimes, because naive-only is really just asking for bizzare transient bugs depending on the server timezone, daylight savings, and so on.

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/strategies/_internal/datetime.py b/hypothesis-python/src/hypothesis/strategies/_internal/datetime.py
--- a/hypothesis-python/src/hypothesis/strategies/_internal/datetime.py
+++ b/hypothesis-python/src/hypothesis/strategies/_internal/datetime.py
@@ -59,9 +59,15 @@ def replace_tzinfo(value, timezone):
 
 
 def datetime_does_not_exist(value):
-    # This function tests whether the given datetime can be round-tripped to and
-    # from UTC.  It is an exact inverse of (and very similar to) the dateutil method
-    # https://dateutil.readthedocs.io/en/stable/tz.html#dateutil.tz.datetime_exists
+    """This function tests whether the given datetime can be round-tripped to and
+    from UTC.  It is an exact inverse of (and very similar to) the dateutil method
+    https://dateutil.readthedocs.io/en/stable/tz.html#dateutil.tz.datetime_exists
+    """
+    # Naive datetimes cannot be imaginary, but we need this special case because
+    # chaining .astimezone() ends with *the system local timezone*, not None.
+    # See bug report in https://github.com/HypothesisWorks/hypothesis/issues/2662
+    if value.tzinfo is None:
+        return False
     try:
         # Does the naive portion of the datetime change when round-tripped to
         # UTC?  If so, or if this overflows, we say that it does not exist.


Test patch:
diff --git a/hypothesis-python/tests/cover/test_datetimes.py b/hypothesis-python/tests/cover/test_datetimes.py
--- a/hypothesis-python/tests/cover/test_datetimes.py
+++ b/hypothesis-python/tests/cover/test_datetimes.py
@@ -147,3 +147,8 @@ def test_can_generate_datetime_with_fold_1():
 @pytest.mark.skipif(PYPY, reason="see comment")
 def test_can_generate_time_with_fold_1():
     find_any(times(), lambda d: d.fold)
+
+
+@given(datetimes(allow_imaginary=False))
+def test_allow_imaginary_is_not_an_error_for_naive_datetimes(d):
+    pass