Repository: celery/celery

Issue:
Bad behavior when a chord chains with a group
## Checklist

✓ I have included the output of ``celery -A proj report`` in the issue.
✓ I have verified that the issue exists against the `master` branch of Celery.

Output of `celery -A tasks report`:

```
software -> celery:4.0.2 (latentcall) kombu:4.0.2 py:3.6.0
            billiard:3.5.0.2 py-amqp:2.1.4
platform -> system:Darwin arch:64bit imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:amqp results:redis:///

result_backend: 'redis:///'
```

## Steps to reproduce

Create a `tasks.py` file with the following:

```python
from celery import Celery, group
app = Celery('tasks', backend='redis://')

@app.task
def echo(string):
    print(string)

def get_sig():
    before = group(echo.si('before {}'.format(i)) for i in range(3))
    connect = echo.si('connect')
    after = group(echo.si('after {}'.format(i)) for i in range(2))
    return before | connect | after
```

Then at the python prompt run:

```pycon
Python 3.6.0 (default, Jan  5 2017, 23:49:18)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tasks; tasks.get_sig().delay()
```

Then run a celery worker with `celery -A tasks worker`, and see what output it gives.

## Expected behavior

I expected to see output that indicated the three `before` tasks ran, then the `connect` task, then the two `after` tasks.

## Actual behavior

This is the output that I saw from the celery worker:

```
celery@Ryans-MacBook-Air.local v4.0.2 (latentcall)

Darwin-16.3.0-x86_64-i386-64bit 2017-01-16 23:03:45

[config]
.> app:         tasks:0x1061ec5f8
.> transport:   amqp://guest:**@localhost:5672//
.> results:     redis://
.> concurrency: 4 (prefork)
.> task events: OFF (enable -E to monitor tasks in this worker)

[queues]
.> celery           exchange=celery(direct) key=celery


[2017-01-16 23:03:47,275: WARNING/PoolWorker-4] before 1
[2017-01-16 23:03:47,276: WARNING/PoolWorker-1] before 2
[2017-01-16 23:03:47,276: WARNING/PoolWorker-2] before 0
[2017-01-16 23:03:47,370: WARNING/PoolWorker-3] after 0
[2017-01-16 23:03:47,378: WARNING/PoolWorker-2] after 0
[2017-01-16 23:03:47,380: WARNING/PoolWorker-1] after 1
[2017-01-16 23:03:47,388: WARNING/PoolWorker-1] after 0
[2017-01-16 23:03:47,388: WARNING/PoolWorker-2] after 1
[2017-01-16 23:03:47,393: WARNING/PoolWorker-1] after 1
[2017-01-16 23:03:47,397: WARNING/PoolWorker-2] connect
[2017-01-16 23:03:47,403: WARNING/PoolWorker-3] after 1
[2017-01-16 23:03:47,403: WARNING/PoolWorker-1] after 0
```

## Impact

This is working with Celery 3.1.25, and is blocking my production project from upgrading currently. If there's a workaround, I'd be happy to use it. It seems like it's running the whole `after` group for every task in the `before` group, and then once again after connect runs.



Discussion:

Resolving patch:
diff --git a/celery/canvas.py b/celery/canvas.py
--- a/celery/canvas.py
+++ b/celery/canvas.py
@@ -1282,6 +1282,9 @@ def run(self, header, body, partial_args, app=None, interval=None,
             group_id=group_id, chord=body, root_id=root_id).results
         bodyres = body.freeze(task_id, root_id=root_id)
 
+        # Chains should not be passed to the header tasks. See #3771
+        options.pop('chain', None)
+
         parent = app.backend.apply_chord(
             header, partial_args, group_id, body,
             interval=interval, countdown=countdown,


Test patch: