Repository: numba/numba

Issue:
ir_utils causes a DeprecationWarning in numpy 1.20
These lines:

https://github.com/numba/numba/blob/c99f3dea5f78f8268b361f06fa7800d1b656bc20/numba/core/ir_utils.py#L1525-L1527

result in the following behavior in numpy 1.20 due to numpy/numpy#14882
```python
In [8]: @numba.njit
   ...: def foo():
   ...:     return bool(1)
   ...:

In [9]: foo()
c:\users\wiese\repos\ga\numba\numba\core\ir_utils.py:1525: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. Use `bool` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.bool_` here.
  if (hasattr(numpy, value)
c:\users\wiese\repos\ga\numba\numba\core\ir_utils.py:1526: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. Use `bool` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.bool_` here.
  and def_val == getattr(numpy, value)):
c:\users\wiese\repos\ga\numba\numba\core\ir_utils.py:1525: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. Use `bool` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.bool_` here.
  if (hasattr(numpy, value)
c:\users\wiese\repos\ga\numba\numba\core\ir_utils.py:1526: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. Use `bool` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.bool_` here.
  and def_val == getattr(numpy, value)):
Out[9]: True
```

This doesn't make any sense to me - why is `numba` even looking at numpy when I use a builtin function?


Discussion:
Thanks for the report - I can see that those lines get hit even when using the builtin.

> This doesn't make any sense to me - why is `numba` even looking at numpy when I use a builtin function?

At this point Numba is still trying to determine what the function and module names are - it happens to try looking at numpy first, and because `np.bool` simply aliases the builtin `bool`, it ends up thinking this is a numpy function. However for other builtins, I wouldn't expect the branch

https://github.com/numba/numba/blob/c99f3dea5f78f8268b361f06fa7800d1b656bc20/numba/core/ir_utils.py#L1525-L1527

to get taken.
It's probably being run when scanning for closures, reductions and inlinable array calls.
Note that it's the `hasattr(numpy, value)` call that emits the error, so whether the branch is taken is irrelevant.

Can you given me an example of code that _does_ take that branch, and explain why it needs to?
This would trigger the branch:
```python
from numba import njit
from numpy import array

@njit
def foo(n):
   return array([x for x in range(n)])

foo(4)
```
the code in question is trying to work out where `array` comes from. It's seeing if NumPy even has the attribute and if it does then is checking that it's definitely NumPy's `array` and not something else's array. The purpose in this case is to permit the array-call inliner to reinterpet that array construction as a call to `np.empty` and then directly assign the values from the `range(n)` into the array without constructing a list.
Does that mean that the following doesn't perform that optimization?
```python
from numba import njit
from numpy import array as not_array

@njit
def foo(n):
   return not_array([x for x in range(n)])

foo(4)
```
That'll still get optimised, `expr` is the global `<built-in function array>` which has `__name__`  as `'array'` which is picked up here:
https://github.com/numba/numba/blob/c99f3dea5f78f8268b361f06fa7800d1b656bc20/numba/core/ir_utils.py#L1507-L1512
Been thinking about this a bit, I'm not sure it's avoidable. I think for Numba perhaps it just needs to make sure that internally none of these deprecated aliases are used? If user code is the cause of the deprecation warnings then it's sign the code may need to change? I also think that `callee_def.value.__module__ == 'numpy'` could be used as a guard to determine whether it's something from NumPy to save tripping this branch for definitions aliasing names in the NumPy attrs?
> I think for Numba perhaps it just needs to make sure that internally none of these deprecated aliases are used?

Done in #6176

> If user code is the cause of the deprecation warnings then it's sign the code may need to change?

The user code at the top of this issue does not reference the deprecated aliases at all. `numba`s attempt to understand them is causing the problem.


> I also think that `callee_def.value.__module__ == 'numpy'` could be used as a guard to determine whether it's something from NumPy to save tripping this branch for definitions aliasing names in the NumPy attrs?

That seems quite reasonable to me
> > I also think that `callee_def.value.__module__ == 'numpy'` could be used as a guard to determine whether it's something from NumPy to save tripping this branch for definitions aliasing names in the NumPy attrs?
> 
> That seems quite reasonable to me

If this is done, I think this applies:
> If user code is the cause of the deprecation warnings then it's sign the code may need to change? 

Numba would only be asking NumPy about things that are likely actually in NumPy and then if one of these things is deprecated and NumPy warns, that's legitimate?
Yes, sounds good to me.

Resolving patch:
diff --git a/numba/core/ir_utils.py b/numba/core/ir_utils.py
--- a/numba/core/ir_utils.py
+++ b/numba/core/ir_utils.py
@@ -1503,7 +1503,7 @@ def find_callname(func_ir, expr, typemap=None, definition_finder=get_definition)
         if isinstance(callee_def, (ir.Global, ir.FreeVar)):
             # require(callee_def.value == numpy)
             # these checks support modules like numpy, numpy.random as well as
-            # calls like len() and intrinsitcs like assertEquiv
+            # calls like len() and intrinsics like assertEquiv
             keys = ['name', '_name', '__name__']
             value = None
             for key in keys:
@@ -1521,15 +1521,25 @@ def find_callname(func_ir, expr, typemap=None, definition_finder=get_definition)
                 def_val = def_val._defn
             if hasattr(def_val, '__module__'):
                 mod_name = def_val.__module__
+                # The reason for first checking if the function is in NumPy's
+                # top level name space by module is that some functions are
+                # deprecated in NumPy but the functions' names are aliased with
+                # other common names. This prevents deprecation warnings on
+                # e.g. getattr(numpy, 'bool') were a bool the target.
+                # For context see #6175, impacts NumPy>=1.20.
+                mod_not_none = mod_name is not None
+                numpy_toplevel = (mod_not_none and
+                                  (mod_name == 'numpy'
+                                   or mod_name.startswith('numpy.')))
                 # it might be a numpy function imported directly
-                if (hasattr(numpy, value)
+                if (numpy_toplevel and hasattr(numpy, value)
                         and def_val == getattr(numpy, value)):
                     attrs += ['numpy']
                 # it might be a np.random function imported directly
                 elif (hasattr(numpy.random, value)
                         and def_val == getattr(numpy.random, value)):
                     attrs += ['random', 'numpy']
-                elif mod_name is not None:
+                elif mod_not_none:
                     attrs.append(mod_name)
             else:
                 class_name = def_val.__class__.__name__


Test patch: