Repository: joke2k/faker

Issue:
is faking time series data in the scope of this project ?
I wanted some fake time series data for a project and couldn't find anything suitable for my needs.
Is something like [this](http://www.xaprb.com/blog/2014/01/24/methods-generate-realistic-time-series-data/) in the scope of this project ?



Discussion:
Yes is in the scope absolutely, do you have some ideas how to implement that?

Interested on this also -- it's been 3 years...
@dodysw It's been 3 years and we haven't received any PR for it yet. Could you prepare one?
I could invest some hours for this. 

Is there a expectation for this PR? e.g. what's the input, what's the output?

Currently I have  a simple implementation that generates x number of random datetime(s) between two datetimes that follows a normal distribution. These list of datetimes then can be iterated as one of attribute or input to the the actual time series data generator.

The post above mentioned [this blog](https://www.xaprb.com/blog/2014/01/24/methods-generate-realistic-time-series-data/) which seems overall a great source of ideas, but I don't have the time to implement all of them in one PR.

Don't know yet if I can help, but I'm sure interested.

One PR is certainly not enough. Referring to '[The Cathedral and the Bazaar](https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar)', lesson 7: **Release early. Release often. And listen to your customers**
The other lessons are great for inspiration also ...
What should the output be? just a list of `datetime`s?

I would prefer an iterable list instead of a 'simple' list. The applications I have in mind often need to augment (or combine) the series with other data. That way, it's easy to add more data to the items in the series.

E.g. a dateseries that represents number of (simulated) visitors of a museum/airport/... at a given moment. 
I'm thinking of a method with this signature:
```
def time_serie(start, end, precision, distribution):
```

`distribution` will be the function generating the values. `precision` is basically 'how often you want a datapoint'.

For example:
```
>>> import random
>>> from datetime import datetime, timedelta

>>> distribution = lambda: random.uniform(0, 5)
>>> time_serie(datetime.now(), datetime.now()+timedelta(days=7), precision=timedelta(days=1), distribution=distribution)
```

The output could be an iterable of tuples, something like:
```
(
    (datetime.datetime(2017, 8, 8, 17, 9, 0, 569562), 0.767017023571912),
    (datetime.datetime(2017, 8, 9, 17, 9, 0, 569562), 0.550568452469854),
    (datetime.datetime(2017, 8, 10, 17, 9, 0, 569562), 1.8060492750124613),
    (datetime.datetime(2017, 8, 11, 17, 9, 0, 569562), 1.0195647347289332),
    (datetime.datetime(2017, 8, 12, 17, 9, 0, 569562), 4.063748959116069),
    (datetime.datetime(2017, 8, 13, 17, 9, 0, 569562), 1.2355522029691586),
    (datetime.datetime(2017, 8, 14, 17, 9, 0, 569562), 3.9570822924224434),
)
```
That seems very workable. My idea was to have a 'class' with an initializer where you could set (most) of the parameters and then have a method to generate the series. That was closer aligned to the idea description in the blog. But I don't object to your suggestion at all. First thing is to have something workable, then we can fine-tune.

I've got a draft working, let me know if there's anything wrong that jumps to your eye

https://gist.github.com/fcurella/8316c3a27c322eb0e02c3d4714de15c6

Resolving patch:
diff --git a/faker/providers/date_time/__init__.py b/faker/providers/date_time/__init__.py
--- a/faker/providers/date_time/__init__.py
+++ b/faker/providers/date_time/__init__.py
@@ -26,6 +26,20 @@ def datetime_to_timestamp(dt):
     return timegm(dt.timetuple())
 
 
+def timestamp_to_datetime(timestamp, tzinfo):
+    if tzinfo is None:
+        pick = datetime.fromtimestamp(timestamp, tzlocal())
+        pick = pick.astimezone(tzutc()).replace(tzinfo=None)
+    else:
+        pick = datetime.fromtimestamp(timestamp, tzinfo)
+
+    return pick
+
+
+class ParseError(ValueError):
+    pass
+
+
 timedelta_pattern = r''
 for name, sym in [('years', 'y'), ('weeks', 'w'), ('days', 'd'), ('hours', 'h'), ('minutes', 'm'), ('seconds', 's')]:
     timedelta_pattern += r'((?P<{0}>(?:\+|-)\d+?){1})?'.format(name, sym)
@@ -316,6 +330,37 @@ def time_object(cls):
         """
         return cls.date_time().time()
 
+    @classmethod
+    def _parse_date_string(cls, value):
+        parts = cls.regex.match(value)
+        if not parts:
+            raise ParseError("Can't parse date string `{}`.".format(value))
+        parts = parts.groupdict()
+        time_params = {}
+        for (name, param) in parts.items():
+            if param:
+                time_params[name] = int(param)
+
+        if 'years' in time_params:
+            if 'days' not in time_params:
+                time_params['days'] = 0
+            time_params['days'] += 365.24 * time_params.pop('years')
+
+        if not time_params:
+            raise ParseError("Can't parse date string `{}`.".format(value))
+        return time_params
+
+    @classmethod
+    def _parse_timedelta(cls, value):
+        if isinstance(value, timedelta):
+            return value.total_seconds()
+        if is_string(value):
+            time_params = cls._parse_date_string(value)
+            return timedelta(**time_params).total_seconds()
+        if isinstance(value, (int, float)):
+            return value
+        raise ParseError("Invalid format for timedelta '{0}'".format(value))
+
     @classmethod
     def _parse_date_time(cls, text, tzinfo=None):
         if isinstance(text, (datetime, date, real_datetime, real_date)):
@@ -326,24 +371,11 @@ def _parse_date_time(cls, text, tzinfo=None):
         if is_string(text):
             if text == 'now':
                 return datetime_to_timestamp(datetime.now(tzinfo))
-            parts = cls.regex.match(text)
-            if not parts:
-                return
-            parts = parts.groupdict()
-            time_params = {}
-            for (name, param) in parts.items():
-                if param:
-                    time_params[name] = int(param)
-
-            if 'years' in time_params:
-                if 'days' not in time_params:
-                    time_params['days'] = 0
-                time_params['days'] += 365.24 * time_params.pop('years')
-
+            time_params = cls._parse_date_string(text)
             return datetime_to_timestamp(now + timedelta(**time_params))
         if isinstance(text, int):
             return datetime_to_timestamp(now + timedelta(text))
-        raise ValueError("Invalid format for date '{0}'".format(text))
+        raise ParseError("Invalid format for date '{0}'".format(text))
 
     @classmethod
     def date_time_between(cls, start_date='-30y', end_date='now', tzinfo=None):
@@ -552,6 +584,35 @@ def date_time_this_month(cls, before_now=True, after_now=False, tzinfo=None):
         else:
             return now
 
+    @classmethod
+    def time_series(cls, start_date='-30d', end_date='now', precision=None, distrib=None, tzinfo=None):
+        """
+        Returns a generator yielding tuples of ``(<datetime>, <value>)``.
+
+        The data points will start at ``start_date``, and be at every time interval specified by
+        ``precision``.
+        """
+        start_date = cls._parse_date_time(start_date, tzinfo=tzinfo)
+        end_date = cls._parse_date_time(end_date, tzinfo=tzinfo)
+
+        if end_date < start_date:
+            raise ValueError("`end_date` must be greater than `start_date`.")
+
+        if precision is None:
+            precision = (end_date - start_date) / 30
+        precision = cls._parse_timedelta(precision)
+
+        if distrib is None:
+            distrib = lambda: random.uniform(0, precision)  # noqa
+
+        if not callable(distrib):
+            raise ValueError("`distrib` must be a callable. Got {} instead.".format(distrib))
+
+        datapoint = start_date
+        while datapoint < end_date:
+            datapoint += precision
+            yield (timestamp_to_datetime(datapoint, tzinfo), distrib())
+
     @classmethod
     def am_pm(cls):
         return cls.date('%p')


Test patch:
diff --git a/tests/providers/date_time.py b/tests/providers/date_time.py
--- a/tests/providers/date_time.py
+++ b/tests/providers/date_time.py
@@ -231,6 +231,51 @@ def test_date_time_this_period_with_tzinfo(self):
             datetime.now(utc).replace(second=0, microsecond=0)
         )
 
+    def test_parse_timedelta(self):
+        from faker.providers.date_time import Provider
+
+        td = timedelta(days=7)
+        seconds = Provider._parse_timedelta(td)
+        self.assertEqual(seconds, 604800.0)
+
+        seconds = Provider._parse_timedelta('+1w')
+        self.assertEqual(seconds, 604800.0)
+
+        seconds = Provider._parse_timedelta('+1y')
+        self.assertEqual(seconds, 31556736.0)
+
+        with self.assertRaises(ValueError):
+            Provider._parse_timedelta('foobar')
+
+    def test_time_series(self):
+        from faker.providers.date_time import Provider
+
+        series = [i for i in Provider.time_series()]
+        self.assertTrue(len(series), 30)
+        self.assertTrue(series[1][0] - series[0][0], timedelta(days=1))
+
+        uniform = lambda: random.uniform(0, 5)  # noqa
+        series = [i for i in Provider.time_series('now', '+1w', '+1d', uniform)]
+        self.assertTrue(len(series), 7)
+        self.assertTrue(series[1][0] - series[0][0], timedelta(days=1))
+
+        end = datetime.now() + timedelta(days=7)
+        series = [i for i in Provider.time_series('now', end, '+1d', uniform)]
+        self.assertTrue(len(series), 7)
+        self.assertTrue(series[1][0] - series[0][0], timedelta(days=1))
+
+        self.assertTrue(series[-1][0] <= end)
+
+        with self.assertRaises(ValueError):
+            [i for i in Provider.time_series('+1w', 'now', '+1d', uniform)]
+
+        with self.assertRaises(ValueError):
+            [i for i in Provider.time_series('now', '+1w', '+1d', 'uniform')]
+
+        series = [i for i in Provider.time_series('now', end, '+1d', uniform, tzinfo=utc)]
+        self.assertTrue(len(series), 7)
+        self.assertTrue(series[1][0] - series[0][0], timedelta(days=1))
+
 
 class TestPlPL(unittest.TestCase):