Repository: celery/celery

Issue:
Timezone calculations in app.now() are incorrect
From `celery/app/base.py`

```

    def now(self):
        """Return the current time and date as a datetime."""
        from datetime import datetime
        return datetime.utcnow().replace(tzinfo=self.timezone)
```

This is just semantically wrong. This takes the UTC timezone-aware time and replaces the timezone, _without actually ensuring it's the same point in time_.

So 9 AM UTC becomes 9 AM JTC, even though 9 AM UTC is 6 PM JTC

The "proper" way of doing this is `self.timezone.normalize(datetime.utcnow())`

This bug breaks things like `crontab` for celery beat, for example, since the schedules get shifted around. And it's the wrong time too


Discussion:
This may be caused by the changes in this [Pull Request](https://github.com/celery/celery/pull/3867/files).
Perhaps also related: https://github.com/celery/celery/issues/4145

@rtpg Based on your suggestion, I think this might be the correct implementation:
```
    def now(self):
        """Return the current time and date as a datetime."""
        from celery.utils.time import to_utc
        from datetime import datetime

        now_in_utc = to_utc(datetime.utcnow())
        return self.timezone.normalize(now_in_utc)

```
I need to test it some more though.
one detail (I'm not sure how it applies here, but it feels relevant) is whether `self.timezone` differs from the system timezone. 

For example if self.timezone is UTC+1 but the system timezone is UTC+9

If those differs, then `datetime.utcnow()` will return the current timestamp (probably) by subtracting 9 hours from the `now`, but then `self.timezone.normalize(now_in_utc)` will shift it forward by only 1 hour? 

Though..... maybe the system timezone is "always right".  This might be ramblings based off of imprecise understanding of this
@rtpg I think it is actually the other way round, the system clock counts in UTC (Universal Coordinated Time) and when you set the system timezone, it changes the representation only. Thus UTC is an absolute measure of time, whereas timezones are relative offsets to UTC.

Resolving patch:
diff --git a/celery/app/base.py b/celery/app/base.py
--- a/celery/app/base.py
+++ b/celery/app/base.py
@@ -2,6 +2,7 @@
 """Actual App instance implementation."""
 from __future__ import absolute_import, unicode_literals
 
+from datetime import datetime
 import os
 import threading
 import warnings
@@ -36,7 +37,8 @@
 from celery.utils.collections import AttributeDictMixin
 from celery.utils.dispatch import Signal
 from celery.utils.functional import first, maybe_list, head_from_fun
-from celery.utils.time import timezone, get_exponential_backoff_interval
+from celery.utils.time import timezone, \
+    get_exponential_backoff_interval, to_utc
 from celery.utils.imports import gen_task_name, instantiate, symbol_by_name
 from celery.utils.log import get_logger
 from celery.utils.objects import FallbackContext, mro_lookup
@@ -880,8 +882,8 @@ def prepare_config(self, c):
 
     def now(self):
         """Return the current time and date as a datetime."""
-        from datetime import datetime
-        return datetime.utcnow().replace(tzinfo=self.timezone)
+        now_in_utc = to_utc(datetime.utcnow())
+        return now_in_utc.astimezone(self.timezone)
 
     def select_queues(self, queues=None):
         """Select subset of queues.


Test patch:
diff --git a/t/unit/app/test_app.py b/t/unit/app/test_app.py
--- a/t/unit/app/test_app.py
+++ b/t/unit/app/test_app.py
@@ -79,10 +79,10 @@ def test_now(self):
         tz_utc = timezone.get_timezone('UTC')
         tz_us_eastern = timezone.get_timezone(timezone_setting_value)
 
-        now = datetime.utcnow().replace(tzinfo=tz_utc)
+        now = to_utc(datetime.utcnow())
         app_now = self.app.now()
 
-        assert app_now.tzinfo == tz_utc
+        assert app_now.tzinfo is tz_utc
         assert app_now - now <= timedelta(seconds=1)
 
         # Check that timezone conversion is applied from configuration
@@ -92,7 +92,8 @@ def test_now(self):
         del self.app.timezone
 
         app_now = self.app.now()
-        assert app_now.tzinfo == tz_us_eastern
+
+        assert app_now.tzinfo.zone == tz_us_eastern.zone
 
         diff = to_utc(datetime.utcnow()) - localize(app_now, tz_utc)
         assert diff <= timedelta(seconds=1)
@@ -102,7 +103,7 @@ def test_now(self):
         del self.app.timezone
         app_now = self.app.now()
         assert self.app.timezone == tz_us_eastern
-        assert app_now.tzinfo == tz_us_eastern
+        assert app_now.tzinfo.zone == tz_us_eastern.zone
 
     @patch('celery.app.base.set_default_app')
     def test_set_default(self, set_default_app):