Repository: celery/celery

Issue:
celery.schedules.crontab.__reduce__ does not return kwargs.
## Checklist

- [x] I have included the output of ``celery -A proj report`` in the issue.
      (if you are not able to do this, then at least specify the Celery
       version affected).
- [X] I have verified that the issue exists against the `master` branch of Celery.

```
software -> celery:4.0.2 (latentcall) kombu:4.0.2 py:3.5.2
            billiard:3.5.0.2 py-amqp:2.1.4
platform -> system:Linux arch:64bit, ELF imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:amqp results:disabled

timezone: 'UTC'
beat_schedule: {
    'test task': {   'args': ('hello',),
                     'schedule': <crontab: 35 18 * * * (m/h/d/dM/MY)>,
                     'task': 'test.test'},
    'test task2': {   'args': ('world',),
                      'schedule': <crontab: 35 15 * * * (m/h/d/dM/MY)>,
                      'task': 'test.test'}}
accept_content: ['json']
task_serializer: 'json'
result_serializer: 'json'
```

## Steps to reproduce

Set a nowfun.  The nowfun will not be restored.  When starting up celery beat, I never get a working nowfun back.

## Expected behavior

relevant kwargs should be returned instead of `None` here: https://github.com/celery/celery/blob/master/celery/schedules.py#L532.  An appropriate `__setstate__` may also be necessary to cope with this (as `app` doesn't get stored in `__dict__` as `app`, but rather `_app`).

## Actual behavior

Only the constructor elements are returned from `__reduce__`,  dropping all kwargs passed to the original `crontab.__init__` when pickling.


Discussion:

Resolving patch:
diff --git a/celery/schedules.py b/celery/schedules.py
--- a/celery/schedules.py
+++ b/celery/schedules.py
@@ -101,6 +101,11 @@ def to_local(self, dt):
             return timezone.to_local_fallback(dt)
         return dt
 
+    def __eq__(self, other):
+        if isinstance(other, BaseSchedule):
+            return other.nowfun == self.nowfun
+        return NotImplemented
+
 
 @python_2_unicode_compatible
 class schedule(BaseSchedule):
@@ -398,6 +403,7 @@ def __init__(self, minute='*', hour='*', day_of_week='*',
         self._orig_day_of_week = cronfield(day_of_week)
         self._orig_day_of_month = cronfield(day_of_month)
         self._orig_month_of_year = cronfield(month_of_year)
+        self._orig_kwargs = kwargs
         self.hour = self._expand_cronspec(hour, 24)
         self.minute = self._expand_cronspec(minute, 60)
         self.day_of_week = self._expand_cronspec(day_of_week, 7)
@@ -529,7 +535,12 @@ def __reduce__(self):
                                  self._orig_hour,
                                  self._orig_day_of_week,
                                  self._orig_day_of_month,
-                                 self._orig_month_of_year), None)
+                                 self._orig_month_of_year), self._orig_kwargs)
+
+    def __setstate__(self, state):
+        # Calling super's init because the kwargs aren't necessarily passed in
+        # the same form as they are stored by the superclass
+        super(crontab, self).__init__(**state)
 
     def remaining_delta(self, last_run_at, tz=None, ffwd=ffwd):
         # pylint: disable=redefined-outer-name
@@ -624,7 +635,8 @@ def __eq__(self, other):
                 other.day_of_month == self.day_of_month and
                 other.day_of_week == self.day_of_week and
                 other.hour == self.hour and
-                other.minute == self.minute
+                other.minute == self.minute and
+                super(crontab, self).__eq__(other)
             )
         return NotImplemented
 


Test patch:
diff --git a/t/unit/app/test_schedules.py b/t/unit/app/test_schedules.py
--- a/t/unit/app/test_schedules.py
+++ b/t/unit/app/test_schedules.py
@@ -91,13 +91,28 @@ def test_pickle(self):
         assert s1 == s2
 
 
+# This is needed for test_crontab_parser because datetime.utcnow doesn't pickle
+# in python 2
+def utcnow():
+    return datetime.utcnow()
+
+
 class test_crontab_parser:
 
     def crontab(self, *args, **kwargs):
         return crontab(*args, **dict(kwargs, app=self.app))
 
     def test_crontab_reduce(self):
-        assert loads(dumps(self.crontab('*')))
+        c = self.crontab('*')
+        assert c == loads(dumps(c))
+        c = self.crontab(
+            minute='1',
+            hour='2',
+            day_of_week='3',
+            day_of_month='4',
+            month_of_year='5',
+            nowfun=utcnow)
+        assert c == loads(dumps(c))
 
     def test_range_steps_not_enough(self):
         with pytest.raises(crontab_parser.ParseException):