Repository: numba/numba

Issue:
Globals error in objmode context-manager for generated functions
<!--

Thanks for opening an issue! To help the Numba team handle your information
efficiently, please first ensure that there is no other issue present that
already describes the issue you have
(search at https://github.com/numba/numba/issues?&q=is%3Aissue).

For more general "how do I do X?" type questions, please speak to us in real
time on https://gitter.im/numba/numba or post to the Numba mailing list
https://groups.google.com/a/continuum.io/forum/#!forum/numba-users.

-->

## Reporting a bug

<!--

Before submitting a bug report please ensure that you can check off these boxes:

-->

- [x] I am using the latest released version of Numba (most recent is visible in
 the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
- [x] I have included below a minimal working reproducer (if you are unsure how
 to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).

<!--

Please include details of the bug here, including, if applicable, what you
expected to happen!

-->
For some reason, `objmode` cannot handle the global variables properly if the function is generated with text and `exec`. Here is an example:

```python
import numpy as np
from numba import objmode, njit

def gen_g():
    func_text = "def func():\n"
    func_text += "    with objmode():\n"
    func_text += "        np.array([1,2,3])\n"
    loc_vars = {}
    exec(func_text, {'objmode': objmode, 'np': np}, loc_vars)
    func = loc_vars['func']
    return njit(func)

g = gen_g()
g()
```
```
Traceback (most recent call last):
  File "tmp_tests/obj_gen_test.py", line 14, in <module>
    g()
NameError: global name 'np' is not defined
```


Discussion:
That's because the `func()` doesn't see that `global` dictionary.  You need to fill `loc_vars` with `{'np': np, 'objmode': objmode}`  or just `loc_vars.update(globals())`.
Still the same. Does it work on your end?
I just realized my previous comment is incorrect.  I didn't see that you already have `{'objmode': objmode, 'np': np}` in the 2 arg of `exec`.

Btw, this works:  `exec(func_text, globals(), loc_vars)`

I think the objmode is reporting the wrong constant is missing.
@sklam using `globals()` works, but I get the warning below in HPAT. I can't reproduce it in a small example though. Any idea? Looks like the compile flags may be wrong for the lifted function.

```
<string>:2: NumbaWarning: Function "csv_reader_py" failed type inference: Unknown attribute 'read_csv' of type Module(<module 'pandas' from '/Users/etotoni/pse-hpc/python/pandas/pandas/__init__.py'>)

File "<string>", line 4:
<source missing, REPL in use?>

[1] During: typing of get attribute at <string> (4)

File "<string>", line 4:
<source missing, REPL in use?>

<string>:2: NumbaWarning: Function "csv_reader_py" was compiled in object mode without forceobj=True.
```
The warning is issued to let user know that they may be "accidentally" using objectmode.  I'll check and make sure `forceobj=True` is set in `objmode()`.
The original error is not `objmode()` specific.  The following gives the same error.

```python
import numpy as np
from numba import objmode, njit, jit
from pprint import pprint

def gen_g():
    func_text = "def func():\n"
    func_text += "    np.array([1,2,3])\n"
    loc_vars = {}
    what = {'np': np}
    exec(func_text, what, loc_vars)
    func = loc_vars['func']
    return jit(forceobj=True)(func)

g = gen_g()
g()
```
Found the reason.  The compiled function is taking the module's `__globals__` as the global dictionary for the compiled object-mode function.  That is true for most function creation.  But for `exec`, the code's module is `<dynamic>`.

Resolving patch:
diff --git a/numba/funcdesc.py b/numba/funcdesc.py
--- a/numba/funcdesc.py
+++ b/numba/funcdesc.py
@@ -33,14 +33,16 @@ class FunctionDescriptor(object):
     """
     __slots__ = ('native', 'modname', 'qualname', 'doc', 'typemap',
                  'calltypes', 'args', 'kws', 'restype', 'argtypes',
-                 'mangled_name', 'unique_name', 'env_name',
+                 'mangled_name', 'unique_name', 'env_name', 'global_dict',
                  'inline', 'noalias')
 
     def __init__(self, native, modname, qualname, unique_name, doc,
                  typemap, restype, calltypes, args, kws, mangler=None,
-                 argtypes=None, inline=False, noalias=False, env_name=None):
+                 argtypes=None, inline=False, noalias=False, env_name=None,
+                 global_dict=None):
         self.native = native
         self.modname = modname
+        self.global_dict = global_dict
         self.qualname = qualname
         self.unique_name = unique_name
         self.doc = doc
@@ -71,6 +73,14 @@ def __init__(self, native, modname, qualname, unique_name, doc,
         self.inline = inline
         self.noalias = noalias
 
+    def lookup_globals(self):
+        """
+        Return the global dictionary of the function.
+        It may not match the Module's globals if the function is created
+        dynamically (i.e. exec)
+        """
+        return self.global_dict or self.lookup_module().__dict__
+
     def lookup_module(self):
         """
         Return the module in which this function is supposed to exist.
@@ -133,23 +143,29 @@ def _get_function_info(cls, func_ir):
         doc = func.__doc__ or ''
         args = tuple(func_ir.arg_names)
         kws = ()        # TODO
+        global_dict = None
 
         if modname is None:
             # Dynamically generated function.
             modname = _dynamic_modname
+            # Retain a reference to the dictionary of the function.
+            # This disables caching, serialization and pickling.
+            global_dict = func_ir.func_id.func.__globals__
 
         unique_name = func_ir.func_id.unique_name
 
-        return qualname, unique_name, modname, doc, args, kws
+        return qualname, unique_name, modname, doc, args, kws, global_dict
 
     @classmethod
     def _from_python_function(cls, func_ir, typemap, restype, calltypes,
                               native, mangler=None, inline=False, noalias=False):
-        (qualname, unique_name, modname, doc, args, kws,
+        (qualname, unique_name, modname, doc, args, kws, global_dict,
          )= cls._get_function_info(func_ir)
+
         self = cls(native, modname, qualname, unique_name, doc,
                    typemap, restype, calltypes,
-                   args, kws, mangler=mangler, inline=inline, noalias=noalias)
+                   args, kws, mangler=mangler, inline=inline, noalias=noalias,
+                   global_dict=global_dict)
         return self
 
 
diff --git a/numba/lowering.py b/numba/lowering.py
--- a/numba/lowering.py
+++ b/numba/lowering.py
@@ -28,12 +28,11 @@ class Environment(_dynfunc.Environment):
 
     @classmethod
     def from_fndesc(cls, fndesc):
-        mod = fndesc.lookup_module()
         try:
             # Avoid creating new Env
             return cls._memo[fndesc.env_name]
         except KeyError:
-            inst = cls(mod.__dict__)
+            inst = cls(fndesc.lookup_globals())
             inst.env_name = fndesc.env_name
             cls._memo[fndesc.env_name] = inst
             return inst


Test patch:
diff --git a/numba/tests/test_object_mode.py b/numba/tests/test_object_mode.py
--- a/numba/tests/test_object_mode.py
+++ b/numba/tests/test_object_mode.py
@@ -158,6 +158,19 @@ def test(a1, a2):
         self.assertEqual(test(*args), compiled(*args))
         self._ensure_objmode(compiled)
 
+    def test_dynamic_func_objmode(self):
+        """
+        Test issue https://github.com/numba/numba/issues/3355
+        """
+        func_text = "def func():\n"
+        func_text += "    np.array([1,2,3])\n"
+        loc_vars = {}
+        custom_globals = {'np': np}
+        exec(func_text, custom_globals, loc_vars)
+        func = loc_vars['func']
+        jitted = jit(forceobj=True)(func)
+        jitted()
+
 
 if __name__ == '__main__':
     unittest.main()