Repository: numba/numba

Issue:
prange segfault, invalid loop invariant detection
<!--

Thanks for opening an issue! To help the Numba team handle your information
efficiently, please first ensure that there is no other issue present that
already describes the issue you have
(search at https://github.com/numba/numba/issues?&q=is%3Aissue).

For more general "how do I do X?" type questions, please speak to us in real
time on https://gitter.im/numba/numba or post to the Numba mailing list
https://groups.google.com/a/continuum.io/forum/#!forum/numba-users.

-->

## Reporting a bug

<!--

Before submitting a bug report please ensure that you can check off these boxes:

-->

- [x] I am using the latest released version of Numba (most recent is visible in
 the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
- [x] I have included below a minimal working reproducer (if you are unsure how
 to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).

The following code fails with `interrupted by signal 11: SIGSEGV` but runs fine if `Parallel=False` (Ubuntu 18.04, Python 3.6)

```python

import random

from numba import njit, prange


@njit
def pick_card(picked: list) -> int:
    a = random.randint(0, 51)
    if a in picked:
        return pick_card(picked)
    else:
        picked.append(a)
    return a


@njit(parallel=True)
def calc(total):
    cards = []
    for suit in range(4):
        for card in range(13):
            cards.append(card)

    for i in prange(total):
        hand = []
        picked = [100]
        for _ in range(13):
            a = pick_card(picked)
            hand.append(cards[a])
    return


calc(100000)
print("Finished!")```



Discussion:
Thanks for the report, I can reproduce. Augmenting the code slightly to add in [Numba's `gdb` support](http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#debugging-jit-compiled-code-with-gdb) and reducing the example to a minimal reproducer:
```python
import random

from numba import njit, prange, gdb_init

@njit
def pick_card(picked: list) -> int:
    a = random.randint(0, 51)
    if a in picked:
        return pick_card(picked)
    else:
        picked.append(a)
    return a


@njit(parallel=True)
def calc(total):
    gdb_init('-ex', 'set confirm off', '-ex', 'bt -1', '-ex', 'q')
    for i in prange(total):
        picked = [100]
        a = pick_card(picked)
    return

calc(100000)
print("Finished!")
```
gives:
```
<snip>
Thread 3 "python" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fbf90dce700 (LWP 11638)]
0x00007fbf9eb19014 in __main__::pick_card$243(list$28int64$29) ()
#87338 0x00007fbfaa1c2ead in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
Detaching from program: <env>/bin/python3.7, process 11630
```
the most interesting thing being the 87000+ stack frames, which is a good indicator something has gone wrong, this will likely be the cause of the segfault...

Switching on parallel diagnostics:
```
$ NUMBA_PARALLEL_DIAGNOSTICS=4 python issue3699.py 
 
================================================================================
======== Parallel Accelerator Optimizing: Internal parallel functions  =========
================================================================================
No source available
Performing sequential lowering of loops...
--------------------------------------------------------------------------------
----------------------------- Before Optimisation ------------------------------
--------------------------------------------------------------------------------
------------------------------ After Optimisation ------------------------------
Parallel structure is already optimal.
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
 
---------------------------Loop invariant code motion---------------------------

Instruction hoisting:
No instruction hoisting found
--------------------------------------------------------------------------------
 
================================================================================
===== Parallel Accelerator Optimizing:  Function calc, issue3699.py (15)  ======
================================================================================


Parallel loop listing for  Function calc, issue3699.py (15) 
-------------------------------------------------------------------------|loop #ID
@njit(parallel=True)                                                     | 
def calc(total):                                                         | 
    #gdb_init()#'-ex', 'set confirm off', '-ex', 'bt -1', '-ex', 'q')    | 
    for i in prange(total):----------------------------------------------| #0
        picked = [100]                                                   | 
        a = pick_card(picked)                                            | 
    return                                                               | 
--------------------------------- Fusing loops ---------------------------------
Attempting fusion of parallel loops (combines loops with similar properties)...
----------------------------- Before Optimisation ------------------------------
--------------------------------------------------------------------------------
------------------------------ After Optimisation ------------------------------
Parallel structure is already optimal.
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
 
---------------------------Loop invariant code motion---------------------------

Instruction hoisting:
loop #0:
  Has the following hoisted:
    $const12.2 = const(int, 100)
    $picked.1 = build_list(items=[Var($const12.2, issue3699.py (19))])
    $12.4 = global(pick_card: CPUDispatcher(<function pick_card at 0x7f0eb2926268>))
  Failed to hoist the following:
    not pure: $a.2 = call $12.4($picked.1, func=$12.4, args=[Var($picked.1, issue3699.py (19))], kws=(), vararg=None)
--------------------------------------------------------------------------------
```
it seems like `build_list` for `picked` is being erroneously hoisted out of the `prange` loop as it's been analysed as loop invariant, when clearly it shouldn't be. As a result all the threads race in writing to a single `picked` list and that list quickly fills with `[0, 52)` and as a result the recursive branch is always taken (and `list.append` is not threadsafe, https://github.com/numba/numba/issues/2408, which probably doesn't help either). This is a bug!

Ping @DrTodd13, I think the analysis code needs to assume anything mutable and used as an argument in a call is not safe as a candidate for hoisting?
@stuartarchibald good job diagnosing the issue! I think we haven't considered `build_list` in analysis/optimization codes in general, and we need to find possible implications elsewhere as well.
@stuartarchibald What's the best "is mutable" check?
```
        if isinstance(rhs, ir.Expr) and rhs.op == 'call':
            # If a mutable object is passed to a function, then it may be changed and
            # therefore can't be hoisted.
            # For each argument to the function...
            for argvar in rhs.args:
                # Get the argument's type.
                if isinstance(argvar, ir.Var):
                    argvar = argvar.name
                avtype = typemap[argvar]
                # If that type doesn't have a mutable attribute or it does and it's set to
                # not mutable then this usage is safe for hoisting.
                if hasattr(avtype, 'mutable') and avtype.mutable:
                    # Here we have a mutable variable passed to a function so add this variable
                    # to the def lists.
                    add_to_def_once_sets(argvar, def_once, def_more)
```
This seems to fix this problem.  Any suggestions @stuartarchibald for style before I create a PR?

Resolving patch:
diff --git a/numba/npyufunc/parfor.py b/numba/npyufunc/parfor.py
--- a/numba/npyufunc/parfor.py
+++ b/numba/npyufunc/parfor.py
@@ -493,7 +493,7 @@ def add_to_def_once_sets(a_def, def_once, def_more):
     else:
         def_once.add(a_def)
 
-def compute_def_once_block(block, def_once, def_more, getattr_taken):
+def compute_def_once_block(block, def_once, def_more, getattr_taken, typemap):
     '''Effect changes to the set of variables defined once or more than once
        for a single block.
     '''
@@ -513,8 +513,23 @@ def compute_def_once_block(block, def_once, def_more, getattr_taken):
             # Calling a method on an object is like a def of that object.
             base_obj = getattr_taken[rhs.func.name]
             add_to_def_once_sets(base_obj, def_once, def_more)
-
-def compute_def_once_internal(loop_body, def_once, def_more, getattr_taken):
+        if isinstance(rhs, ir.Expr) and rhs.op == 'call':
+            # If a mutable object is passed to a function, then it may be changed and
+            # therefore can't be hoisted.
+            # For each argument to the function...
+            for argvar in rhs.args:
+                # Get the argument's type.
+                if isinstance(argvar, ir.Var):
+                    argvar = argvar.name
+                avtype = typemap[argvar]
+                # If that type doesn't have a mutable attribute or it does and it's set to
+                # not mutable then this usage is safe for hoisting.
+                if getattr(avtype, 'mutable', False):
+                    # Here we have a mutable variable passed to a function so add this variable
+                    # to the def lists.
+                    add_to_def_once_sets(argvar, def_once, def_more)
+
+def compute_def_once_internal(loop_body, def_once, def_more, getattr_taken, typemap):
     '''Compute the set of variables defined exactly once in the given set of blocks
        and use the given sets for storing which variables are defined once, more than
        once and which have had a getattr call on them.
@@ -523,22 +538,22 @@ def compute_def_once_internal(loop_body, def_once, def_more, getattr_taken):
     for label, block in loop_body.items():
         # Scan this block and effect changes to def_once, def_more, and getattr_taken
         # based on the instructions in that block.
-        compute_def_once_block(block, def_once, def_more, getattr_taken)
+        compute_def_once_block(block, def_once, def_more, getattr_taken, typemap)
         # Have to recursively process parfors manually here.
         for inst in block.body:
             if isinstance(inst, parfor.Parfor):
                 # Recursively compute for the parfor's init block.
-                compute_def_once_block(inst.init_block, def_once, def_more, getattr_taken)
+                compute_def_once_block(inst.init_block, def_once, def_more, getattr_taken, typemap)
                 # Recursively compute for the parfor's loop body.
-                compute_def_once_internal(inst.loop_body, def_once, def_more, getattr_taken)
+                compute_def_once_internal(inst.loop_body, def_once, def_more, getattr_taken, typemap)
 
-def compute_def_once(loop_body):
+def compute_def_once(loop_body, typemap):
     '''Compute the set of variables defined exactly once in the given set of blocks.
     '''
     def_once = set()   # set to hold variables defined exactly once
     def_more = set()   # set to hold variables defined more than once
     getattr_taken = {}
-    compute_def_once_internal(loop_body, def_once, def_more, getattr_taken)
+    compute_def_once_internal(loop_body, def_once, def_more, getattr_taken, typemap)
     return def_once
 
 def find_vars(var, varset):
@@ -587,7 +602,7 @@ def hoist(parfor_params, loop_body, typemap, wrapped_blocks):
     not_hoisted = []
 
     # Compute the set of variable defined exactly once in the loop body.
-    def_once = compute_def_once(loop_body)
+    def_once = compute_def_once(loop_body, typemap)
     (call_table, reverse_call_table) = get_call_table(wrapped_blocks)
 
     setitems = set()
@@ -645,6 +660,22 @@ def redarraytype_to_sig(redarraytyp):
     assert isinstance(redarraytyp, types.npytypes.Array)
     return types.npytypes.Array(redarraytyp.dtype, max(1, redarraytyp.ndim - 1), redarraytyp.layout)
 
+def legalize_names_with_typemap(names, typemap):
+    """ We use ir_utils.legalize_names to replace internal IR variable names
+        containing illegal characters (e.g. period) with a legal character
+        (underscore) so as to create legal variable names.
+        The original variable names are in the typemap so we also
+        need to add the legalized name to the typemap as well.
+    """
+    outdict = legalize_names(names)
+    # For each pair in the dict of legalized names...
+    for x, y in outdict.items():
+        # If the name had some legalization change to it...
+        if x != y:
+            # Set the type of the new name the same as the type of the old name.
+            typemap[y] = typemap[x]
+    return outdict
+
 def _create_gufunc_for_parfor_body(
         lowerer,
         parfor,
@@ -737,7 +768,7 @@ def _create_gufunc_for_parfor_body(
 
     # Some Var are not legal parameter names so create a dict of potentially illegal
     # param name to guaranteed legal name.
-    param_dict = legalize_names(parfor_params + parfor_redvars)
+    param_dict = legalize_names_with_typemap(parfor_params + parfor_redvars, typemap)
     if config.DEBUG_ARRAY_OPT == 1:
         print(
             "param_dict = ",
@@ -748,7 +779,7 @@ def _create_gufunc_for_parfor_body(
 
     # Some loop_indices are not legal parameter names so create a dict of potentially illegal
     # loop index to guaranteed legal name.
-    ind_dict = legalize_names(loop_indices)
+    ind_dict = legalize_names_with_typemap(loop_indices, typemap)
     # Compute a new list of legal loop index names.
     legal_loop_indices = [ind_dict[v] for v in loop_indices]
     if config.DEBUG_ARRAY_OPT == 1:


Test patch:
diff --git a/numba/tests/test_parfors.py b/numba/tests/test_parfors.py
--- a/numba/tests/test_parfors.py
+++ b/numba/tests/test_parfors.py
@@ -2039,6 +2039,34 @@ def test_impl(n):
             return B
         self.prange_tester(test_impl, 1.0)
 
+    @skip_unsupported
+    def test_mutable_list_param(self):
+        """ issue3699: test that mutable variable to call in loop
+            is not hoisted.  The call in test_impl forces a manual
+            check here rather than using prange_tester.
+        """
+        @njit
+        def list_check(X):
+            """ If the variable X is hoisted in the test_impl prange
+                then subsequent list_check calls would return increasing
+                values.
+            """
+            ret = X[-1]
+            a = X[-1] + 1
+            X.append(a)
+            return ret
+        def test_impl(n):
+            for i in prange(n):
+                X = [100]
+                a = list_check(X)
+            return a
+        python_res = test_impl(10)
+        njit_res = njit(test_impl)(10)
+        pa_func = njit(test_impl, parallel=True)
+        pa_res = pa_func(10)
+        self.assertEqual(python_res, njit_res)
+        self.assertEqual(python_res, pa_res)
+
 
 @skip_parfors_unsupported
 @x86_only