Repository: numpy/numpy

Issue:
reload(numpy);numpy.max([]) raises TypeError
Using `maintenance/1.11.x` on Windows (msvc builds), the following code fails:

``` Python
Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:18:55) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import reload
>>> import numpy
>>> reload(numpy)
<module 'numpy' from 'X:\\Python35\\lib\\site-packages\\numpy\\__init__.py'>
>>> numpy.max([])
Traceback (most recent call last):
  File "X:\Python35\lib\site-packages\numpy\core\fromnumeric.py", line 2290, in amax
    amax = a.max
AttributeError: 'list' object has no attribute 'max'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "X:\Python35\lib\site-packages\numpy\core\fromnumeric.py", line 2293, in amax
    out=out, **kwargs)
  File "X:\Python35\lib\site-packages\numpy\core\_methods.py", line 26, in _amax
    return umr_maximum(a, axis, None, out, keepdims)
TypeError: an integer is required (got type type)
```

I traced this to #7736, backport of #4619 



Discussion:
The _NoValue sentinel changes on reload, but not its values captured in 
function default arguments. The problem is referring to it via `import 
numpy as np; np._NoValue` instead of `from numpy import _NoValue` in 
fromnumeric.py, which is not reload-safe.

@pv Good catch.

I wonder if it would make more sense for  `_NoValue` to be locally defined in the modules? It is a private class in any case.

That said, I thought C modules could not be reloaded?

Curiously, there is no error with 2.7 or 3.5 in Fedora 24.

Yes, I think the reload-safe solution is to use module-specific private 
sentinels. Otherwise, different modules end up with different singletons 
on reload. Or maybe just put the sentinel in `numpy.core` etc. private 
place, which people are unlikely to reload.

I can confirm that using `from numpy import _NoValue; _NoValue` instead of `import numpy as np; np._NoValue` fixes this issue.

Looks like we need a global, reload safe, `_NoValue`, as calls from other modules pass through `fromnumeric` which checks the values. The quick work around is probably to define it somewhere more reload safe than `numpy/__init__.py`.

Possible alternative approach: add something like this to `numpy/__init__.py`:

``` python
if "_already_loaded" in globals():
    raise RuntimeError("Reloading numpy isn't supported")
_already_loaded = True
```

Probably the explicit import works also, looks like the local binding hangs onto the instance in the original import of Numpy before any reloads.

This all makes me wonder if the various warnings defined in `__init__.py` are also safe if used in `catch_warnings`. We don't do that, just wondering.

@njsmith Yeah, reloading numpy alone doesn't look to achieve much except mess up the local stuff defined in the `__init__.py`.

I note that `isinstance` also fails for reloaded classes. I'm inclined to follow @njsmith and just disallow reloading numpy, it isn't clear to me that reloading does anything useful. Thoughts?


Resolving patch:
diff --git a/numpy/__init__.py b/numpy/__init__.py
--- a/numpy/__init__.py
+++ b/numpy/__init__.py
@@ -107,8 +107,21 @@
 from __future__ import division, absolute_import, print_function
 
 import sys
+import warnings
 
+# Disallow reloading numpy. Doing that does nothing to change previously
+# loaded modules, which would need to be reloaded separately, but it does
+# change the identity of the warnings and sentinal classes defined below
+# with dire consequences when checking for identity.
+if '_is_loaded' in globals():
+    raise RuntimeError('Reloading numpy is not supported')
+_is_loaded = True
 
+
+# Define some global warnings and the _NoValue sentinal. Defining them here
+# means that their identity will change if numpy is reloaded, hence if that is
+# to be allowed they should be moved into their own, non-reloadable module.
+# Note that these should be defined (or imported) before the other imports.
 class ModuleDeprecationWarning(DeprecationWarning):
     """Module deprecation warning.
 
@@ -135,9 +148,8 @@ class VisibleDeprecationWarning(UserWarning):
 class _NoValue:
     """Special keyword value.
 
-    This class may be used as the default value assigned to a
-    deprecated keyword in order to check if it has been given a user
-    defined value.
+    This class may be used as the default value assigned to a deprecated
+    keyword in order to check if it has been given a user defined value.
     """
     pass
 
@@ -155,11 +167,8 @@ class _NoValue:
 except NameError:
     __NUMPY_SETUP__ = False
 
-
 if __NUMPY_SETUP__:
-    import sys as _sys
-    _sys.stderr.write('Running from numpy source directory.\n')
-    del _sys
+    sys.stderr.write('Running from numpy source directory.\n')
 else:
     try:
         from numpy.__config__ import show as show_config
@@ -206,7 +215,7 @@ def pkgload(*packages, **options):
     from .compat import long
 
     # Make these accessible from numpy name-space
-    #  but not imported in from numpy import *
+    # but not imported in from numpy import *
     if sys.version_info[0] >= 3:
         from builtins import bool, int, float, complex, object, str
         unicode = str
@@ -222,8 +231,8 @@ def pkgload(*packages, **options):
     __all__.extend(lib.__all__)
     __all__.extend(['linalg', 'fft', 'random', 'ctypeslib', 'ma'])
 
+
     # Filter annoying Cython warnings that serve no good purpose.
-    import warnings
     warnings.filterwarnings("ignore", message="numpy.dtype size changed")
     warnings.filterwarnings("ignore", message="numpy.ufunc size changed")
     warnings.filterwarnings("ignore", message="numpy.ndarray size changed")


Test patch:
diff --git a/numpy/tests/test_reloading.py b/numpy/tests/test_reloading.py
new file mode 100644
--- /dev/null
+++ b/numpy/tests/test_reloading.py
@@ -0,0 +1,26 @@
+from __future__ import division, absolute_import, print_function
+
+import sys
+
+import numpy as np
+from numpy.testing import assert_raises, assert_, run_module_suite
+
+if sys.version_info[:2] >= (3, 4):
+    from importlib import reload
+else:
+    from imp import reload
+
+def test_reloading_exception():
+    # gh-7844. Also check that relevant globals retain their identity.
+    _NoValue = np._NoValue
+    VisibleDeprecationWarning = np.VisibleDeprecationWarning
+    ModuleDeprecationWarning = np.ModuleDeprecationWarning
+
+    assert_raises(RuntimeError, reload, np)
+    assert_(_NoValue is np._NoValue)
+    assert_(ModuleDeprecationWarning is np.ModuleDeprecationWarning)
+    assert_(VisibleDeprecationWarning is np.VisibleDeprecationWarning)
+
+
+if __name__ == "__main__":
+    run_module_suite()