Repository: numba/numba

Issue:
Jitting linked list takes an unexpectedly long time
## Reporting a bug

- [x] I am using the latest released version of Numba (most recent is visible in
 the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
- [x] I have included below a minimal working reproducer (if you are unsure how
 to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).

```python
from numba import jit

@jit
def make_list(n):
    if n <= 0:
        return None
    
    return (n, make_list(n - 1))

print(make_list(100))
```

This code makes a linked list with 100 nodes and takes over 40 seconds to run with `@jit` or 0.02 seconds without.

My actual goal is to implement a kdtree, but type conversion between a jitclass and an optional child member of that same class does not work yet, so I tried tuples instead and ran into this issue.


Discussion:
Thanks for the report. I presume the time reported includes compilation time? Also, please could you let us know which Numba version you are using, thanks.
> I presume the time reported includes compilation time?

Yes. The first call to `make_list` takes a while and consumes several gigabytes of RAM, but subsequent runs are faster (0.1 to 0.2 milliseconds).

> Also, please could you let us know which Numba version you are using, thanks.

I get 0.45.0 when running `print(numba.__version__)`.

In addition, I just noticed that the process finishes immediately when sending a SIGINT, so maybe the jit simply crashes due to running out of memory and falls back to some other method.
CC @sklam, probably of interest to you.

Resolving patch:
diff --git a/numba/typeinfer.py b/numba/typeinfer.py
--- a/numba/typeinfer.py
+++ b/numba/typeinfer.py
@@ -1320,6 +1320,8 @@ def resolve_call(self, fnty, pos_args, kw_args):
 
             sig = typing.signature(return_type, *args)
             sig.pysig = pysig
+            # Keep track of unique return_type
+            frame.add_return_type(return_type)
             return sig
         else:
             # Normal non-recursive call
diff --git a/numba/typing/context.py b/numba/typing/context.py
--- a/numba/typing/context.py
+++ b/numba/typing/context.py
@@ -110,10 +110,23 @@ def __init__(self, typeinfer, func_id, args):
         self.typeinfer = typeinfer
         self.func_id = func_id
         self.args = args
+        self._inferred_retty = set()
 
     def __repr__(self):
         return "CallFrame({}, {})".format(self.func_id, self.args)
 
+    def add_return_type(self, return_type):
+        """Add *return_type* to the list of inferred return-types.
+        If there are too many, raise `TypingError`.
+        """
+        # The maximum limit is picked arbitrarily.
+        # Don't think that this needs to be user configurable.
+        RETTY_LIMIT = 16
+        self._inferred_retty.add(return_type)
+        if len(self._inferred_retty) >= RETTY_LIMIT:
+            m = "Return type of recursive function does not converge"
+            raise errors.TypingError(m)
+
 
 class BaseContext(object):
     """A typing context for storing function typing constrain template.


Test patch:
diff --git a/numba/tests/recursion_usecases.py b/numba/tests/recursion_usecases.py
--- a/numba/tests/recursion_usecases.py
+++ b/numba/tests/recursion_usecases.py
@@ -215,3 +215,14 @@ def bar(x):
             return x * bar(out)
 
     return bar
+
+
+def make_growing_tuple_case(jit=lambda x: x):
+    # From issue #4387
+    @jit
+    def make_list(n):
+        if n <= 0:
+            return None
+
+        return (n, make_list(n - 1))
+    return make_list
diff --git a/numba/tests/test_recursion.py b/numba/tests/test_recursion.py
--- a/numba/tests/test_recursion.py
+++ b/numba/tests/test_recursion.py
@@ -55,6 +55,15 @@ def test_optional_return(self):
         for arg in (0, 5, 10, 15):
             self.assertEqual(pfunc(arg), cfunc(arg))
 
+    def test_growing_return_tuple(self):
+        cfunc = self.mod.make_growing_tuple_case(jit(nopython=True))
+        with self.assertRaises(TypingError) as raises:
+            cfunc(100)
+        self.assertIn(
+            "Return type of recursive function does not converge",
+            str(raises.exception),
+        )
+
 
 class TestMutualRecursion(TestCase):