Repository: celery/celery

Issue:
maximum recursion depth exceeded for a canvas in Celery 4.4.0 (cliffs)
<!--

-->
# Checklist
<!--
-->
- [x] I have verified that the issue exists against the `master` branch of Celery.
- [ ] This has already been asked to the [discussion group](https://groups.google.com/forum/#!forum/celery-users) first. (No option to post for me)
- [x] I have read the relevant section in the
  [contribution guide](http://docs.celeryproject.org/en/latest/contributing.html#other-bugs)
  on reporting bugs.
- [x] I have checked the [issues list](https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22)
  for similar or identical bug reports.
- [x] I have checked the [pull requests list](https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22)
  for existing proposed fixes.
- [x] I have checked the [commit log](https://github.com/celery/celery/commits/master)
  to find out if the bug was already fixed in the master branch.
- [x] I have included all related issues and possible duplicate issues
  in this issue (If there are none, check this box anyway).

## Mandatory Debugging Information

- [x] I have included the output of ``celery -A proj report`` in the issue.
    (if you are not able to do this, then at least specify the Celery
     version affected). - Celery 4.4.0 (cliffs)
- [x] I have verified that the issue exists against the `master` branch of Celery.
- [ ] I have included the contents of ``pip freeze`` in the issue.
- [ ] I have included all the versions of all the external dependencies required
  to reproduce this bug.

## Optional Debugging Information
<!--
-->
- [ ] I have tried reproducing the issue on more than one Python version
  and/or implementation.
- [ ] I have tried reproducing the issue on more than one message broker and/or
  result backend.
- [ ] I have tried reproducing the issue on more than one version of the message
  broker and/or result backend.
- [ ] I have tried reproducing the issue on more than one operating system.
- [ ] I have tried reproducing the issue on more than one workers pool.
- [ ] I have tried reproducing the issue with autoscaling, retries,
  ETA/Countdown & rate limits disabled.
- [x] I have tried reproducing the issue after downgrading
  and/or upgrading Celery and its dependencies. Works in Celery 4.1

## Related Issues and Possible Duplicates
<!--
-->

#### Related Issues

- None

#### Possible Duplicates

- None

## Environment & Settings
<!-- Include the contents of celery --version below -->
**Celery version 4.4.0 (cliffs)**:
<!-- Include the output of celery -A proj report below -->
<details>
<summary><b><code>celery report</code> Output:</b></summary>
<p>

```
```

</p>
</details>

# Steps to Reproduce
```
sig = signature('any_taskname', queue='any_q')
chain( [ chain( sig ) ] ).apply_async()
```
## Required Dependencies
<!-- Please fill the required dependencies to reproduce this issue -->
* **Minimal Python Version**: 2.7
* **Minimal Celery Version**: 4.4
* **Minimal Kombu Version**: 4.6.7
* **Minimal Broker Version**: N/A 
* **Minimal Result Backend Version**: N/A 
* **Minimal OS and/or Kernel Version**: N/A
* **Minimal Broker Client Version**: N/A
* **Minimal Result Backend Client Version**: N/A

### Python Packages
<details>
<summary><b><code>pip freeze</code> Output:</b></summary>
<p>

```
```

</p>
</details>

### Other Dependencies
<!--
-->
<details>
<p>
N/A
</p>
</details>

## Minimally Reproducible Test Case
<!--

-->

<details>
<p>

```
sig = signature('any_taskname', queue='any_q')
chain( [ chain( sig ) ] ).apply_async()
```

</p>
</details>

# Expected Behavior
It should publish task 'any_taskname' to queue 'any_q'

# Actual Behavior
Max recursion depth exceeded
```
Traceback (most recent call last):
  File "test.py", line 30, in <module>
    chain([chain(s2)]).apply_async() # issue
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 642, in apply_async
    dict(self.options, **options) if options else self.options))
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 660, in run
    task_id, group_id, chord,
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 721, in prepare_steps
    task = task.clone(args, kwargs)
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 620, in clone
    for sig in signature.kwargs['tasks']
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 1513, in maybe_signature
    d = d.clone()
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 620, in clone
    for sig in signature.kwargs['tasks']
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 1513, in maybe_signature
    d = d.clone()
...
..
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 620, in clone
    for sig in signature.kwargs['tasks']
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 1513, in maybe_signature
    d = d.clone()
keeps repeating
..
..
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 617, in clone
    signature = Signature.clone(self, *args, **kwargs)
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 272, in clone
    app=self._app)
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 153, in from_dict
    return target_cls.from_dict(d, app=app)
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 599, in from_dict
    return _upgrade(d, _chain(tasks, app=app, **d['options']))
  File "/bb/bin/dl/celery/4.4/celery/canvas.py", line 602, in __init__
    tasks = (regen(tasks[0]) if len(tasks) == 1 and is_list(tasks[0])
  File "/bb/bin/dl/celery/4.4/kombu/utils/functional.py", line 256, in is_list
    return isinstance(l, iters) and not isinstance(l, scalars or ())
  File "/opt/bb/lib/python2.7/abc.py", line 132, in __instancecheck__
    if subclass is not None and subclass in cls._abc_cache:
  File "/opt/bb/lib/python2.7/_weakrefset.py", line 72, in __contains__
    wr = ref(item)
RuntimeError: maximum recursion depth exceeded
```


Discussion:
I just ran the test case and indeed I see the same error.
It seems like the error occurs when you apply the task. If you don't everything proceeds correctly.
The problem is in line 881.
https://github.com/celery/celery/blob/01dd66ceb5b9167074c2f291b165055e7377641b/celery/canvas.py#L876-L882

reduce is called on a pair of arguments, of which there are currently none in this test case.

Resolving patch:
diff --git a/celery/canvas.py b/celery/canvas.py
--- a/celery/canvas.py
+++ b/celery/canvas.py
@@ -408,8 +408,12 @@ def __or__(self, other):
             other = maybe_unroll_group(other)
             if isinstance(self, _chain):
                 # chain | group() -> chain
+                tasks = self.unchain_tasks()
+                if not tasks:
+                    # If the chain is empty, return the group
+                    return other
                 return _chain(seq_concat_item(
-                    self.unchain_tasks(), other), app=self._app)
+                    tasks, other), app=self._app)
             # task | group() -> chain
             return _chain(self, other, app=self.app)
 
@@ -622,7 +626,7 @@ def clone(self, *args, **kwargs):
         return signature
 
     def unchain_tasks(self):
-        # Clone chain's tasks assigning sugnatures from link_error
+        # Clone chain's tasks assigning signatures from link_error
         # to each task
         tasks = [t.clone() for t in self.tasks]
         for sig in self.options.get('link_error', []):
@@ -878,7 +882,9 @@ def __new__(cls, *tasks, **kwargs):
         if not kwargs and tasks:
             if len(tasks) != 1 or is_list(tasks[0]):
                 tasks = tasks[0] if len(tasks) == 1 else tasks
-                return reduce(operator.or_, tasks)
+                # if is_list(tasks) and len(tasks) == 1:
+                #     return super(chain, cls).__new__(cls, tasks, **kwargs)
+                return reduce(operator.or_, tasks, chain())
         return super(chain, cls).__new__(cls, *tasks, **kwargs)
 
 


Test patch:
diff --git a/celery/contrib/testing/app.py b/celery/contrib/testing/app.py
--- a/celery/contrib/testing/app.py
+++ b/celery/contrib/testing/app.py
@@ -34,6 +34,7 @@ def __getattr__(self, name):
         # in Python 3.8 and above.
         if name == '_is_coroutine':
             return None
+        print(name)
         raise RuntimeError('Test depends on current_app')
 
 
diff --git a/t/integration/test_canvas.py b/t/integration/test_canvas.py
--- a/t/integration/test_canvas.py
+++ b/t/integration/test_canvas.py
@@ -123,6 +123,10 @@ def test_group_results_in_chain(self, manager):
         res = c()
         assert res.get(timeout=TIMEOUT) == [4, 5]
 
+    def test_chain_of_chain_with_a_single_task(self, manager):
+        sig = signature('any_taskname', queue='any_q')
+        chain([chain(sig)]).apply_async()
+
     def test_chain_on_error(self, manager):
         from .tasks import ExpectedException
 
diff --git a/t/unit/tasks/test_canvas.py b/t/unit/tasks/test_canvas.py
--- a/t/unit/tasks/test_canvas.py
+++ b/t/unit/tasks/test_canvas.py
@@ -269,6 +269,10 @@ def test_chunks(self):
 
 class test_chain(CanvasCase):
 
+    def test_chain_of_chain_with_a_single_task(self):
+        s = self.add.s(1, 1)
+        assert chain([chain(s)]).tasks == list(chain(s).tasks)
+
     def test_clone_preserves_state(self):
         x = chain(self.add.s(i, i) for i in range(10))
         assert x.clone().tasks == x.tasks