Repository: numpy/numpy

Issue:
import_array hides true cause of import errors
When compiling for Python 3, it would be useful to utilize exception chaining to explain why "numpy.core.multiarray failed to import". I think you can use [PyException_SetCause](https://docs.python.org/3/c-api/exceptions.html#c.PyException_SetCause) to do this?



Discussion:
Sorry, can you elaborate?

If there's some odd error causing numpy not to load  ([in this case, it was a linking error of some sort](http://www.chiefdelphi.com/forums/showthread.php?t=145631)) when I call [`import_array()`](http://docs.scipy.org/doc/numpy-1.10.1/user/c-info.how-to-extend.html) from the C API, then you end up with an error that says the following:

```
ImportError: numpy.core.multiarray failed to import
```

While this error is _technically_ correct, it's fairly useless. And in fact, it's hiding the true error, which one _could_ find by running the following code prior to calling `import_array()`:

```
if (PyImport_ImportModule("numpy.core.multiarray") == NULL)
    PyErr_Print();
```

Which printed out this:

```
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/numpy/__init__.py", line 180, in <module>
    from . import add_newdocs
  File "/usr/local/lib/python3.4/dist-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/local/lib/python3.4/dist-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/usr/local/lib/python3.4/dist-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/usr/local/lib/python3.4/dist-packages/numpy/core/__init__.py", line 14, in <module>
    from . import multiarray
ImportError: /usr/local/lib/python3.4/dist-packages/numpy/core/multiarray.cpython-34m.so: undefined symbol: PyType_GenericNew
```

As you can see, this is much more helpful for figuring out what went wrong. Now, in python3 they've introduced this idea of doing something like:

```
try:
    foo()
except Something as e:
    raise SomethingElse() from e
```

Which then displays a stack trace saying something like "SomethingElse was raised, caused by original exception Something". I'm not 100% sure, but it seems like that functionality is accomplished in the C API by using PyException_SetCause. It would be great if `import_array()` could utilize that so that import errors were more clear.

Thanks!

From a quick skim of the source, it looks like the solution would just be to delete this line: https://github.com/numpy/numpy/blob/46bb5350148ea271dc336ccda33b3a8857d57ad0/numpy/core/code_generators/generate_numpy_api.py#L53
so that `_import_array` lets the underlying exception propagate instead of overwriting it.

Everything that calls `_import_array` (e.g. `import_array`) calls `PyErr_Print()` on exceptions, so you'd have seen the real exception (even on python 2) if that line wasn't there.

@virtuald: Want to submit a pull request? It sounds like you might be in a situation where you can actually test this, which is difficult otherwise -- I don't know how to write a test case that makes sure that importing `numpy.core.multiarray` fails :-)

It appears that this line was added in 2010, and prior to that it was the behavior you described (see https://github.com/numpy/numpy/commit/10e5c4c86607ca694dcf03ccaf0e7570a511f1cb). It looks like the prior fix was to ensure that an error was always printed out. Perhaps a better fix would be to check if there were an error set, and if not then call PyErr_SetString (not sure what API is best... but I'm sure it exists somewhere)?

I haven't tried particularly hard to reproduce the other user's error, but I can put together a PR and break an import or two. 

`PyImport_ImportModule` should always leave an exception set if it fails; my guess is that the commit you link to set out to clean up a bunch of similar issues (notice how it adds a bunch of genuinely meaningful exceptions for other error conditions), and then it got a little bit overenthusiastic when handling failures in `PyImport_ImportModule`.

any update on this one?

Resolving patch:
diff --git a/numpy/core/code_generators/generate_numpy_api.py b/numpy/core/code_generators/generate_numpy_api.py
--- a/numpy/core/code_generators/generate_numpy_api.py
+++ b/numpy/core/code_generators/generate_numpy_api.py
@@ -50,7 +50,6 @@
   PyObject *c_api = NULL;
 
   if (numpy == NULL) {
-      PyErr_SetString(PyExc_ImportError, "numpy.core._multiarray_umath failed to import");
       return -1;
   }
   c_api = PyObject_GetAttrString(numpy, "_ARRAY_API");


Test patch: