Repository: celery/celery

Issue:
task priority disappear after self.retry()
<!--
Please fill this template entirely and do not erase parts of it.
We reserve the right to close without a response
bug reports which are incomplete.
-->
# Checklist
<!--
To check an item on the list replace [ ] with [x].
-->

- [x] I have read the relevant section in the
  [contribution guide](http://docs.celeryproject.org/en/latest/contributing.html#other-bugs)
  on reporting bugs.
- [x] I have checked the [issues list](https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22I5597ssue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22)
  for similar or identical bug reports.


## Mandatory Debugging Information

- [x] I have included the output of ``celery -A proj report`` in the issue.
    (if you are not able to do this, then at least specify the Celery
     version affected).


## Optional Debugging Information
<!--
Try some of the below if you think they are relevant.
It will help us figure out the scope of the bug and how many users it affects.
-->
- [x] I have tried reproducing the issue on more than one Python version
  and/or implementation.
- [x] I have tried reproducing the issue on more than one version of the message
  broker and/or result backend.
- [x] I have tried reproducing the issue after downgrading
  and/or upgrading Celery and its dependencies.

## Related Issues and Possible Duplicates
<!--
Please make sure to search and mention any related issues
or possible duplicates to this issue as requested by the checklist above.

This may or may not include issues in other repositories that the Celery project
maintains or other repositories that are dependencies of Celery.

If you don't know how to mention issues, please refer to Github's documentation
on the subject: https://help.github.com/en/articles/autolinked-references-and-urls#issues-and-pull-requests
-->

#### Related Issues

- #5597


## Environment & Settings
<!-- Include the contents of celery --version below -->
**Celery version**: 4.3.0 (rhubarb)
<!-- Include the output of celery -A proj report below -->
<details>
<summary><b><code>celery report</code> Output:</b></summary>
<p>

```
software -> celery:4.3.0 (rhubarb) kombu:4.4.0 py:3.6.7
            billiard:3.6.0.0 py-amqp:2.4.1
platform -> system:Linux arch:64bit
            kernel version:4.18.0-25-generic imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:amqp results:disabled

task_queues: [<unbound Queue test -> <unbound Exchange test(direct)> -> test>]
```

</p>
</details>

# Steps to Reproduce

## Required Dependencies
<!-- Please fill the required dependencies to reproduce this issue -->
* **Minimal Python Version**: 3.6.7
* **Minimal Celery Version**: 4.3.0
* **Minimal Kombu Version**: 4.4.0

## Minimally Reproducible Test Case
<!--
Please provide a reproducible test case.
Refer to the Reporting Bugs section in our contribution guide.

We prefer submitting test cases in the form of a PR to our integration test suite.
If you can provide one, please mention the PR number below.
If not, please attach the most minimal code example required to reproduce the issue below.
If the test case is too large, please include a link to a gist or a repository below.
-->

<details>
<p>

```python
import celery
from kombu import Exchange
from kombu import Queue


class Config:
    task_queues = [
        Queue('test', Exchange('test'), routing_key='test', queue_arguments={'x-max-priority': 3})
    ] # yapf: disable


app = celery.Celery('test')
app.config_from_object(Config)


@app.task(bind=True)
def task(self):
    print(self.request.delivery_info['priority'])
    self.retry(countdown=1)


if __name__ == '__main__':
    task.s().apply_async(priority=1, queue='test')
```

</p>
</details>

# Expected Behavior
Expect task to have a priority 1 after self.retry()

Expected output:
```[python]
[tasks]
  . test.task

[2019-07-24 22:31:50,810: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2019-07-24 22:31:50,823: INFO/MainProcess] mingle: searching for neighbors
[2019-07-24 22:31:51,855: INFO/MainProcess] mingle: all alone
[2019-07-24 22:31:51,869: INFO/MainProcess] celery@ArtSobes-Home-Ubuntu-18 ready.
[2019-07-24 22:31:51,870: INFO/MainProcess] Received task: test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2]  
[2019-07-24 22:31:51,973: WARNING/ForkPoolWorker-16] 1
[2019-07-24 22:31:51,995: INFO/MainProcess] Received task: test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2]  ETA:[2019-07-24 19:31:52.974357+00:00] 
[2019-07-24 22:31:51,995: INFO/ForkPoolWorker-16] Task test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2] retry: Retry in 1s
[2019-07-24 22:31:54,828: WARNING/ForkPoolWorker-2] 1
[2019-07-24 22:31:54,850: INFO/MainProcess] Received task: test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2]  ETA:[2019-07-24 19:31:55.829945+00:00] 
[2019-07-24 22:31:54,850: INFO/ForkPoolWorker-2] Task test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2] retry: Retry in 1s
[2019-07-24 22:31:56,831: WARNING/ForkPoolWorker-4] 1
[2019-07-24 22:31:56,853: INFO/MainProcess] Received task: test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2]  ETA:[2019-07-24 19:31:57.832523+00:00] 
[2019-07-24 22:31:56,853: INFO/ForkPoolWorker-4] Task test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2] retry: Retry in 1s
[2019-07-24 22:31:58,833: WARNING/ForkPoolWorker-6] 1
```

# Actual Behavior
On the first call task priority is 1, but after retry it is None

```[python]
[tasks]
  . test.task

[2019-07-24 22:30:51,901: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2019-07-24 22:30:51,913: INFO/MainProcess] mingle: searching for neighbors
[2019-07-24 22:30:52,940: INFO/MainProcess] mingle: all alone
[2019-07-24 22:30:52,956: INFO/MainProcess] celery@ArtSobes-Home-Ubuntu-18 ready.
[2019-07-24 22:30:52,957: INFO/MainProcess] Received task: test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e]  
[2019-07-24 22:30:53,060: WARNING/ForkPoolWorker-16] 1
[2019-07-24 22:30:53,083: INFO/MainProcess] Received task: test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e]  ETA:[2019-07-24 19:30:54.062046+00:00] 
[2019-07-24 22:30:53,083: INFO/ForkPoolWorker-16] Task test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e] retry: Retry in 1s
[2019-07-24 22:30:54,958: WARNING/ForkPoolWorker-2] None
[2019-07-24 22:30:54,979: INFO/MainProcess] Received task: test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e]  ETA:[2019-07-24 19:30:55.959481+00:00] 
[2019-07-24 22:30:54,979: INFO/ForkPoolWorker-2] Task test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e] retry: Retry in 1s
[2019-07-24 22:30:56,960: WARNING/ForkPoolWorker-4] None
[2019-07-24 22:30:56,982: INFO/MainProcess] Received task: test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e]  ETA:[2019-07-24 19:30:57.962134+00:00] 
[2019-07-24 22:30:56,983: INFO/ForkPoolWorker-4] Task test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e] retry: Retry in 1s
[2019-07-24 22:30:58,963: WARNING/ForkPoolWorker-6] None
```



Discussion:
Hi, it looks like the problem could be potentially fixed here:

https://github.com/celery/celery/blob/8e34a67bdb95009df759d45c7c0d725c9c46e0f4/celery/app/task.py#L113

One might copy the priority here as well.
can you come with a PR?

Resolving patch:
diff --git a/celery/app/task.py b/celery/app/task.py
--- a/celery/app/task.py
+++ b/celery/app/task.py
@@ -595,10 +595,13 @@ def signature_from_request(self, request=None, args=None, kwargs=None,
         args = request.args if args is None else args
         kwargs = request.kwargs if kwargs is None else kwargs
         options = request.as_execution_options()
+        delivery_info = request.delivery_info or {}
+        priority = delivery_info.get('priority')
+        if priority is not None:
+            options['priority'] = priority
         if queue:
             options['queue'] = queue
         else:
-            delivery_info = request.delivery_info or {}
             exchange = delivery_info.get('exchange')
             routing_key = delivery_info.get('routing_key')
             if exchange == '' and routing_key:
diff --git a/t/integration/tasks.py b/t/integration/tasks.py
--- a/t/integration/tasks.py
+++ b/t/integration/tasks.py
@@ -145,6 +145,15 @@ def retry_once(self, *args, expires=60.0, max_retries=1, countdown=0.1):
                      max_retries=max_retries)
 
 
+@shared_task(bind=True, expires=60.0, max_retries=1)
+def retry_once_priority(self, *args, expires=60.0, max_retries=1, countdown=0.1):
+    """Task that fails and is retried. Returns the priority."""
+    if self.request.retries:
+        return self.request.delivery_info['priority']
+    raise self.retry(countdown=countdown,
+                     max_retries=max_retries)
+
+
 @shared_task
 def redis_echo(message):
     """Task that appends the message to a redis list."""


Test patch:
diff --git a/t/integration/test_tasks.py b/t/integration/test_tasks.py
--- a/t/integration/test_tasks.py
+++ b/t/integration/test_tasks.py
@@ -5,7 +5,7 @@
 from celery import group
 
 from .conftest import get_active_redis_channels
-from .tasks import add, add_ignore_result, print_unicode, retry_once, sleeping
+from .tasks import add, add_ignore_result, print_unicode, retry_once, retry_once_priority, sleeping
 
 
 class test_tasks:
@@ -21,6 +21,11 @@ def test_task_retried(self):
         res = retry_once.delay()
         assert res.get(timeout=10) == 1  # retried once
 
+    @pytest.mark.flaky(reruns=5, reruns_delay=2)
+    def test_task_retried_priority(self):
+        res = retry_once_priority.apply_async(priority=7)
+        assert res.get(timeout=10) == 7  # retried once with priority 7
+
     @pytest.mark.flaky(reruns=5, reruns_delay=2)
     def test_unicode_task(self, manager):
         manager.join(
diff --git a/t/unit/tasks/test_tasks.py b/t/unit/tasks/test_tasks.py
--- a/t/unit/tasks/test_tasks.py
+++ b/t/unit/tasks/test_tasks.py
@@ -212,6 +212,22 @@ def test_retry(self):
         self.retry_task.apply([0xFF, 0xFFFF], {'max_retries': 10})
         assert self.retry_task.iterations == 11
 
+    def test_retry_priority(self):
+        priority = 7
+        
+        # Technically, task.priority doesn't need to be set here
+        # since push_request() doesn't populate the delivery_info
+        # with it. However, setting task.priority here also doesn't
+        # cause any problems.
+        self.retry_task.priority = priority
+
+        self.retry_task.push_request()
+        self.retry_task.request.delivery_info = {
+            'priority': priority
+        }
+        sig = self.retry_task.signature_from_request()
+        assert sig.options['priority'] == priority
+
     def test_retry_no_args(self):
         self.retry_task_noargs.max_retries = 3
         self.retry_task_noargs.iterations = 0