Repository: numba/numba
Pull request: Don't do the parfor diagnostics pass for the parfor gufunc.
Resolves #8967 


Problem statement:
RuntimeError with NUMBA_PARALLEL_DIAGNOSTICS = 4
Self contained code sample to reproduce the problem:
```
import os
os.environ['NUMBA_PARALLEL_DIAGNOSTICS']='4'
import numba as nb
import numpy as np

@nb.njit(parallel=True)
def g():
    n = 10000
    b = np.zeros((n), dtype=np.float64)
    for i in nb.prange(n):
        b[i] = 1
    return b

g()
```

Returns following error:
```
LoweringError: Failed in nopython mode pipeline (step: native parfor lowering)
Failed in full_parfor_gufunc mode pipeline (step: dump parfor diagnostics)
self.setup has not been called

File "../../../../../../../../var/folders/bh/g86lgxvj67x8_qdx30vwmw500000gn/T/ipykernel_33020/3850650943.py", line 20:


During: lowering "id=0[LoopNest(index_variable = parfor_index.1, range = (0, n, 1))]{47: }Var(parfor_index.1, 3850650943.py:20)" at /var/folders/bh/g86lgxvj67x8_qdx30vwmw500000gn/T/ipykernel_33020/3850650943.py (20)
```
I would expect the the code to run without errors and returning some parallel diagnostic output.

Run with Numba 0.57.0 on MacBook Air M1

Hints from issue discussion:
Thanks for the report, this reproduces as described. I think this is a regression against Numba 0.56 in which this works as expected.
I can confirm, the reproducer passes with 0.56.4.
I used `git bisect` with the above reproducer and I ended up at: https://github.com/numba/numba/commit/21fc31ee972563431fd9b811c4ac516c10d75a72

```
 💣 zsh» git bisect good
21fc31ee972563431fd9b811c4ac516c10d75a72 is the first bad commit
commit 21fc31ee972563431fd9b811c4ac516c10d75a72
Author: Todd A. Anderson <drtodd13@comcast.net>
Date:   Wed Feb 15 16:30:34 2023 -0800

    Use a very minimal pipeline for parfor gufunc compilation.

 numba/core/compiler.py           | 12 ++++++++++++
 numba/parfors/parfor_lowering.py | 10 +++++++++-
 2 files changed, 21 insertions(+), 1 deletion(-)
```
@DrTodd13 can you take a look perhaps?

Commit message: Don't do the parfor diagnostics pass for the parfor gufunc.  We only need to do it for the main function.

Patch to review:
diff --git a/numba/core/compiler.py b/numba/core/compiler.py
index 71b658d0519..0943f49fb38 100644
--- a/numba/core/compiler.py
+++ b/numba/core/compiler.py
@@ -601,6 +601,25 @@ def define_nopython_lowering_pipeline(state, name='nopython_lowering'):
         pm.finalize()
         return pm
 
+    @staticmethod
+    def define_parfor_gufunc_nopython_lowering_pipeline(state, name='parfor_gufunc_nopython_lowering'):
+        pm = PassManager(name)
+        # legalise
+        pm.add_pass(NoPythonSupportedFeatureValidation,
+                    "ensure features that are in use are in a valid form")
+        pm.add_pass(IRLegalization,
+                    "ensure IR is legal prior to lowering")
+        # Annotate only once legalized
+        pm.add_pass(AnnotateTypes, "annotate types")
+        # lower
+        if state.flags.auto_parallel.enabled:
+            pm.add_pass(NativeParforLowering, "native parfor lowering")
+        else:
+            pm.add_pass(NativeLowering, "native lowering")
+        pm.add_pass(NoPythonBackend, "nopython mode backend")
+        pm.finalize()
+        return pm
+
     @staticmethod
     def define_typed_pipeline(state, name="typed"):
         """Returns the typed part of the nopython pipeline"""
diff --git a/numba/parfors/parfor_lowering.py b/numba/parfors/parfor_lowering.py
index 2af8ca66381..98aaa4b8362 100644
--- a/numba/parfors/parfor_lowering.py
+++ b/numba/parfors/parfor_lowering.py
@@ -1539,7 +1539,7 @@ def define_pipelines(self):
             pm = PassManager("full_parfor_gufunc")
             parfor_gufunc_passes = dpb.define_parfor_gufunc_pipeline(self.state)
             pm.passes.extend(parfor_gufunc_passes.passes)
-            lowering_passes = dpb.define_nopython_lowering_pipeline(self.state)
+            lowering_passes = dpb.define_parfor_gufunc_nopython_lowering_pipeline(self.state)
             pm.passes.extend(lowering_passes.passes)
 
             pm.finalize()


Code review comments:
diff_hunk: @@ -601,6 +601,25 @@ def define_nopython_lowering_pipeline(state, name='nopython_lowering'):
         pm.finalize()
         return pm
 
+    @staticmethod
+    def define_parfor_gufunc_nopython_lowering_pipeline(state, name='parfor_gufunc_nopython_lowering'):
path: numba/core/compiler.py
text: @user1:
```suggestion
    def define_parfor_gufunc_nopython_lowering_pipeline(
        state, name='parfor_gufunc_nopython_lowering'):
```

☝️ To fix the flake8 error 👇 

numba/core/compiler.py:605:81: E501 line too long (103 > 80 characters)