Repository: numba/numba

Issue:
with.. as construct breaks bytecode analysis and leads to an uncontrolled failure
numba.core.errors.UnsupportedError: Failed in nopython mode pipeline (step: analyzing bytecode)
'try' block not supported until python3.7 or later



Discussion:
@Sequential-circuits thank you for submitting this. Please could you say which version of Python you are using? It seems like your Python may be to old to compile the code in question.
I've updated to 3.8 and now getting this
Traceback (most recent call last):
  File "/venv/lib/python3.8/site-packages/numba/core/ir.py", line 267, in get
    return self._con[name]
KeyError: '$2114setup_with.1'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/venv/lib/python3.8/site-packages/numba/core/ir.py", line 1099, in get_exact
    return self.localvars.get(name)
  File "/venv/lib/python3.8/site-packages/numba/core/ir.py", line 269, in get
    raise NotDefinedError(name)
numba.core.errors.NotDefinedError: The compiler failed to analyze the bytecode. Variable '$2114setup_with.1' is not defined.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/venv/lib/python3.8/site-packages/numba/core/ir.py", line 267, in get
    return self._con[name]
KeyError: '$2114setup_with.1'

@Sequential-circuits thank you for updating this issue. I have never seen such an error before. In order to debug it, please submit a self-contained reproducer. I.e. a short, runnable snippet of code including imports that exhibits this failure when run. Thank you.
@esc This sort of thing will do it, it's the `with... as` construct.
```python
from numba import njit
import numpy as np

@njit
def foo():
    with open('aaa') as f:
        pass

foo()
```

@Sequential-circuits Numba cannot compile everything, it's really good at loops, math/numerical  code and NumPy. If you are new to Numba, perhaps take a look at the [5 minute guide](http://numba.pydata.org/numba-doc/latest/user/5minguide.html) first and/or click one of the `Try Now` buttons here http://numba.pydata.org/ to open up an interactive Numba demo in your browser, [this one](https://mybinder.org/v2/gh/numba/numba-examples/master?filepath=notebooks%2Fbasics.ipynb) demonstrates the basics. This might help you decide if Numba is a good fit for your use case.
The code is long and my motivation to remove it to its bare minimum very small. Feel free to drop me an email to alcosta@tekntrash.com and I will send it complete 


Yeah, but being unable to compile something does not mean throwing an error. At most, it should just ignore the code. My 5 cents :)
> Yeah, but being unable to compile something does not mean throwing an error. At most, it should just ignore the code. My 5 cents :)

Whilst there's an understandable convenience aspect to this, feedback from Numba users suggests it manifests as a reasonably challenging experience. A user may write code that compiles fine and runs fast, then they may inadvertently edit it to something that cannot be compiled and it just executes slowly instead. Numba actually has a [version of this mode](https://numba.readthedocs.io/en/stable/user/5minguide.html#what-is-nopython-mode), it has caused so many problems due to erratic performance (and various other internal issues) that it is rarely/never recommended and will eventually be replaced by an explicit opt-in for the behaviour.

Also, I'm also fairly sure that it's a common failure mode for a compiler that cannot compile something to raise an exception. In the case above it's an ICE (Internal Compiler Error) that should not have occurred. Marking this as a bug, immediate fix is to catch the condition and provide a more suitable error message. Thanks for reporting it.
In all honesty, all I wanted from this software was the ability to run the code on the GPU instead of the CPU. I thought it'd be something like CUDA.

Or, at worst, to read barcodes on the GPU
> In all honesty, all I wanted from this software was the ability to run the code on the GPU instead of the CPU. I thought it'd be something like CUDA.
> 
> Or, at worst, to read barcodes on the GPU

You can potentially do that :) Numba has a [CUDA target](https://numba.readthedocs.io/en/stable/cuda/index.html), I would suggest watching a couple of tutorials on it and then direction questions about how to do specific things with it to [Numba's discourse forum](https://numba.discourse.group/), chances are someone will jump in to help.

Resolving patch:
diff --git a/numba/core/byteflow.py b/numba/core/byteflow.py
--- a/numba/core/byteflow.py
+++ b/numba/core/byteflow.py
@@ -130,6 +130,9 @@ def run(self):
                         state.advance_pc()
                         # Must the new PC be a new block?
                         if self._is_implicit_new_block(state):
+                            # check if this is a with...as, abort if so
+                            self._guard_with_as(state)
+                            # else split
                             state.split_new_block()
                             break
                 _logger.debug("end state. edges=%s", state.outgoing_edges)
@@ -245,6 +248,19 @@ def _is_implicit_new_block(self, state):
         else:
             return False
 
+    def _guard_with_as(self, state):
+        """Checks if the next instruction after a SETUP_WITH is something other
+        than a POP_TOP, if it is something else it'll be some sort of store
+        which is not supported (this corresponds to `with CTXMGR as VAR(S)`)."""
+        current_inst = state.get_inst()
+        if current_inst.opname == "SETUP_WITH":
+            next_op = self._bytecode[current_inst.next].opname
+            if next_op != "POP_TOP":
+                msg = ("The 'with (context manager) as "
+                       "(variable):' construct is not "
+                       "supported.")
+                raise UnsupportedError(msg)
+
 
 class TraceRunner(object):
     """Trace runner contains the states for the trace and the opcode dispatch.
diff --git a/numba/core/controlflow.py b/numba/core/controlflow.py
--- a/numba/core/controlflow.py
+++ b/numba/core/controlflow.py
@@ -847,6 +847,7 @@ def jump(self, target, pops=0):
     def _iter_inst(self):
         for inst in self.bytecode:
             if self._use_new_block(inst):
+                self._guard_with_as(inst)
                 self._start_new_block(inst)
             self._curblock.body.append(inst.offset)
             yield inst
@@ -867,6 +868,18 @@ def _start_new_block(self, inst):
         self.blocks[inst.offset] = self._curblock
         self.blockseq.append(inst.offset)
 
+    def _guard_with_as(self, current_inst):
+        """Checks if the next instruction after a SETUP_WITH is something other
+        than a POP_TOP, if it is something else it'll be some sort of store
+        which is not supported (this corresponds to `with CTXMGR as VAR(S)`)."""
+        if current_inst.opname == "SETUP_WITH":
+            next_op = self.bytecode[current_inst.next].opname
+            if next_op != "POP_TOP":
+                msg = ("The 'with (context manager) as "
+                       "(variable):' construct is not "
+                       "supported.")
+                raise UnsupportedError(msg)
+
     def op_SETUP_LOOP(self, inst):
         end = inst.get_jump_target()
         self._blockstack.append(end)


Test patch:
diff --git a/numba/tests/test_withlifting.py b/numba/tests/test_withlifting.py
--- a/numba/tests/test_withlifting.py
+++ b/numba/tests/test_withlifting.py
@@ -1022,6 +1022,20 @@ def test_invalid(self):
             str(raises.exception),
             )
 
+    def test_with_as_fails_gracefully(self):
+        @njit
+        def foo():
+            with open('') as f:
+                pass
+
+        with self.assertRaises(errors.UnsupportedError) as raises:
+            foo()
+
+        excstr = str(raises.exception)
+        msg = ("The 'with (context manager) as (variable):' construct is not "
+               "supported.")
+        self.assertIn(msg, excstr)
+
 
 if __name__ == '__main__':
     unittest.main()