Repository: joke2k/faker

Issue:
OSError: [Errno 22] Invalid argument on Windows
In some cases random dates generated by faker fail on Windows.
There was another issue on this, but it was closed and a stack trace was asked. It was issue #460 
I have the stack trace included.
Unfortunately I don't have the exact line where it fails, because there are several in my code, and the stack trace doesn't say which statement is the direct cause, but here are all of the fake dates in my code:

```
dateofbirth = factory.Faker('date_between', start_date="-70y", end_date="-20y")
created = factory.Faker('date_time_this_decade', before_now=True, after_now=False)
dateLastModified = factory.Faker('date_time_this_year', before_now=True, after_now=False)
password_changed = factory.Faker('date_time_this_year', before_now=True, after_now=False)
status_date = factory.Faker('date_time_this_year', before_now=True, after_now=False)
```


### Steps to reproduce
This occurs randomly, propably when the randomly chosen date is out of specific bounds.
Please refer to #460 for further details.

### Expected behavior
A random date should be assigned

### Actual behavior
OSError: [Errno 22] Invalid argument

### stack trace

```
  File "C:\projects\clubassistant\features\current\ca5\tests\businesslogic\reminderlogic.py", line 20, in test_amount_too_high_ok
    member = MemberFactory()
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\factory\base.py", line 46, in __call__
    return cls.create(**kwargs)
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\factory\base.py", line 563, in create
    return cls._generate(enums.CREATE_STRATEGY, kwargs)
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\factory\base.py", line 500, in _generate
    return step.build()
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\factory\builder.py", line 272, in build
    step.resolve(pre)
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\factory\builder.py", line 221, in resolve
    self.attributes[field_name] = getattr(self.stub, field_name)
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\factory\builder.py", line 375, in __getattr__
    extra=context,
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\factory\faker.py", line 56, in evaluate
    return self.generate(extra or {})
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\factory\faker.py", line 53, in generate
    return subfaker.format(self.provider, **kwargs)
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\faker\generator.py", line 76, in format
    return self.get_formatter(formatter)(*args, **kwargs)
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\faker\providers\date_time\__init__.py", line 1580, in date_between
    return self.date_between_dates(date_start=start_date, date_end=end_date)
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\faker\providers\date_time\__init__.py", line 1680, in date_between_dates
    return self.date_time_between_dates(date_start, date_end).date()
  File "C:\projects\clubassistant\features\current\ca5\.venv\lib\site-packages\faker\providers\date_time\__init__.py", line 1665, in date_time_between_dates
    pick = datetime.fromtimestamp(timestamp, tzlocal())
OSError: [Errno 22] Invalid argument
```




Discussion:
Anything < `-(datetime.now().year - 1970)` will cause this error. Proof:
Try `fake.date_between(start_date="-48y", end_date="today")`
Now try `fake.date_between(start_date="-49y", end_date="today")`

The second one will invariably fail.

I haven't looked into it, but maybe this could be re-implemented by subtracting a `timedelta` from the start day. Thoughts?
Hi @Mattwmaster58 ,

What version of Python and what version/architecture of Windows are you using?
I've created a branch that tests `.date_between(start_date="-49y", end_date="today")` at https://github.com/joke2k/faker/pull/881.

The build is passing on appveyor https://ci.appveyor.com/project/joke2k/faker/builds/20961012, so we'll need more detail about your system in order to find a way to reproduce this bug.
Windows 10, Python 3.7.1

On Wed, Dec 12, 2018 at 10:22 AM Flavio Curella <notifications@github.com>
wrote:

> I've created a branch that tests .date_between(start_date="-49y",
> end_date="today") at #881 <https://github.com/joke2k/faker/pull/881>.
>
> The build is passing on appveyor
> https://ci.appveyor.com/project/joke2k/faker/builds/20961012, so we'll
> need more detail about your system in order to find a way to reproduce this
> bug.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/joke2k/faker/issues/829#issuecomment-446670587>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AZHfLezeygnc2xjB4i88mH84o-4S2E-Dks5u4TtkgaJpZM4XHzE7>
> .
>


-- 
*Either I will find a way, or I will make one.** - Phillip Sydney*

@Mattwmaster58 I'm assuming 64 bit architecture?
@Mattwmaster58 AppVeyor can test under Windows Server 2012 R2 or Windows Server 2016 (https://www.appveyor.com/docs/windows-images-software/#operating-system), but not on Windows 10.

The first step is to add Windows 10 to our CI infrastructure somehow, so that we can actually reproduce the bug now and for the future.
It looks the error is caused by calling `datetime.fromtimestamp` with
anything < 0. On Windows 10 x64/Python 3.7.1 it throws an OS Error, but on
my linux box it works fine. Is there a way to completely factor out the use
of `fromtimestamp`?
This almost works, but timezones strike again (The expected result is 7 hours off, my timezone is UTC-7:00)
(code from inside `date_time_between_dates`)
```
pick = datetime_end - self.generator.random.randint(0, datetime_to_timestamp(datetime_end) - datetime_to_timestamp(datetime_start))
```

I don't really see another straighforward approach when `datetime.fromtimestamp` doesn't accept negative values on some systems.
As of Nov 11, 2020, I'm still running into the same issue with Windows 10 2003 with Python 3.9.0 and faker 4.14.2

How do you suggest I fix this in the meantime?
@Loucool111 It's been a while since I've worked with the lib, but maybe play around with my suggestion above and get something that is timezone agnostic.  Once you get that down submit a PR.
This is a wild guess, but having ran into this issue while debugging an unrelated project [0], I am wondering whether the culprit may be some characters that are not permitted as Windows filenames (such as say colons). Here is the (potentially) relevant StackOverflow answer: https://stackoverflow.com/questions/43196750/python-oserror-errno-22-invalid-argument

Cheers!

[0] https://github.com/RoboCupJuniorTC/rcj-soccer-sim/pull/40

Resolving patch:
diff --git a/faker/providers/date_time/__init__.py b/faker/providers/date_time/__init__.py
--- a/faker/providers/date_time/__init__.py
+++ b/faker/providers/date_time/__init__.py
@@ -19,10 +19,10 @@ def datetime_to_timestamp(dt):
 
 def timestamp_to_datetime(timestamp, tzinfo):
     if tzinfo is None:
-        pick = datetime.fromtimestamp(timestamp, tzlocal())
+        pick = convert_timestamp_to_datetime(timestamp, tzlocal())
         pick = pick.astimezone(tzutc()).replace(tzinfo=None)
     else:
-        pick = datetime.fromtimestamp(timestamp, tzinfo)
+        pick = convert_timestamp_to_datetime(timestamp, tzinfo)
 
     return pick
 
@@ -1697,8 +1697,11 @@ def date_time_between_dates(
         )
         try:
             if tzinfo is None:
-                pick = datetime.fromtimestamp(timestamp, tzlocal())
-                pick = pick.astimezone(tzutc()).replace(tzinfo=None)
+                pick = convert_timestamp_to_datetime(timestamp, tzlocal())
+                try:
+                    pick = pick.astimezone(tzutc()).replace(tzinfo=None)
+                except OSError:
+                    pass
             else:
                 pick = datetime.fromtimestamp(timestamp, tzinfo)
         except OverflowError:
@@ -2045,3 +2048,11 @@ def date_of_birth(self, tzinfo=None, minimum_age=0, maximum_age=115):
         dob = self.date_time_ad(tzinfo=tzinfo, start_datetime=start_date, end_datetime=end_date).date()
 
         return dob if dob != start_date else dob + timedelta(days=1)
+
+
+def convert_timestamp_to_datetime(timestamp, tzinfo):
+    import datetime as dt
+    if timestamp >= 0:
+        return dt.datetime.fromtimestamp(timestamp, tzinfo)
+    else:
+        return dt.datetime(1970, 1, 1, tzinfo=tzinfo) + dt.timedelta(seconds=int(timestamp))


Test patch:
diff --git a/tests/providers/test_date_time.py b/tests/providers/test_date_time.py
--- a/tests/providers/test_date_time.py
+++ b/tests/providers/test_date_time.py
@@ -229,6 +229,10 @@ def test_date_between_dates(self):
         assert date_start <= random_date
         assert date_end >= random_date
 
+    def test_date_time_between_long_past_dates(self):
+        random_date = self.fake.date_between("-100y", "-50y")
+        assert random_date
+
     def _datetime_to_time(self, value):
         return int(time.mktime(value.timetuple()))