Repository: pandas-dev/pandas

Issue:
Shouldn't the try: __import__ in __init__ re-raise original exception?
#### Problem description

It is confusing when `import pandas` raises a message like:

    >>> import pandas
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/raf/anaconda3/lib/python3.7/site-packages/pandas/__init__.py", line 19, in <module>
        "Missing required dependencies {0}".format(missing_dependencies))
    ImportError: Missing required dependencies ['pytz']

Where actually the true error is not in `pytz`, but in a missing library required by `pytz`:

    >>> import pytz
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/raf/anaconda3/lib/python3.7/site-packages/pytz/__init__.py", line 20, in <module>
        from pytz.tzinfo import unpickler, BaseTzInfo
      File "/home/raf/anaconda3/lib/python3.7/site-packages/pytz/tzinfo.py", line 4, in <module>
        from bisect import bisect_right
    ImportError: cannot import name 'bisect_right' from 'bisect' (/home/raf/tmp/bisect.py)


As you can see, I happen to have a file called `bisect.py` in the same folder where I'm trying to `import pandas`, but the error message raised by `pandas` doesn't reflect that. It was a silly error after all, but It gave me a headache to sort that out.

Tracking back the source of it, in `pandas/__init__.py`, note that the original error `e` is not used after `except ImportError as e`. I think it should.

    for dependency in hard_dependencies:
        try:
            __import__(dependency)
        except ImportError as e:
            missing_dependencies.append(dependency)
    
    if missing_dependencies:
        raise ImportError(
            "Missing required dependencies {0}".format(missing_dependencies))

#### Expected Output

`raise ImportError(...)` should mention the error `e` raised by `__import__(dependency)`


---
Question also asked in SO:
https://stackoverflow.com/questions/53445555/shouldnt-the-try-import-in-init-re-raise-original-exception



Discussion:
I'm -1 here as I think this is somewhat nuanced. At the end of the day pytz wasn't fully installed, hence the original error so it's not as if pandas is misleading in that regards. 
I actually wouldn't be too opposed to clarifying this.  Would it be so bad to print out all relevant error messages related to each dependency that failed?
A +1 for this. I do think Pandas is being misleading in this case - I've just spend a good ten minutes trying to figure out why Pandas thinks Numpy isn't installed when `pip freeze` says it is, before finding out it's masking an underlying ImportError in Numpy (which I think is caused by Conda shenanigans).

I don't see any particular advantage to what Pandas is doing - why not just allow the underlying ImportError to be raised, whether it's the requirement itself missing or some dependency of that, like just about every other package? Turning an ImportError into a potentially less informative ImportError doesn't seem to be very useful.

If you really, really want to have the little message that can give a single message saying that all three of numpy, pytz, and dateutil missing, I'd be tempted to go with:
```
for dependency in hard_dependencies:
    try:
        __import__(dependency)
    except ImportError as e:
        expected_msg = 'cannot import name {}'.format(dependency)
        if expected_msg not in str(e):
            # Dependency installed but raised a separate ImportError
            raise
        missing_dependencies.append(dependency)
```
We're python 3 only now. All we need to do is

```

except ImportError as e:
    raise ImportError(...) from e
```

and the full traceback is printed. Are you interested in working on it @DanielFEvans?
I'm not sure of an easy way to test this. I suspect the current behavior is untested.
Is there any need to raise a separate import error - i.e. what would Pandas add that isn't already there? If there's a default ImportError that already says that it couldn't find `numpy`, I'm not sure there's much more to say.

(This is different from some other cases further down in `__init__.py`, where the try-catch gives some hints suggesting that compiled code is missing).

I'd be happy to give this one a go - seems like a fairly simple one to start with!


Ah, sorry, I misunderstood the issue. The idea is to raise a single error message with all of the import errors, so https://github.com/pandas-dev/pandas/issues/23868#issuecomment-499051118 won't quite work.

I think the simplest thing is to include the original exception in the `missing_dependencies.append(dependency)`. So something like `missing_dependencies.append((dependency, e))`

Resolving patch:
diff --git a/pandas/__init__.py b/pandas/__init__.py
--- a/pandas/__init__.py
+++ b/pandas/__init__.py
@@ -10,11 +10,13 @@
     try:
         __import__(dependency)
     except ImportError as e:
-        missing_dependencies.append(dependency)
+        missing_dependencies.append((dependency, e))
 
 if missing_dependencies:
-    raise ImportError(
-        "Missing required dependencies {0}".format(missing_dependencies))
+    msg = "Unable to import required dependencies:"
+    for dependency, e in missing_dependencies:
+        msg += "\n{0}: {1}".format(dependency, str(e))
+    raise ImportError(msg)
 del hard_dependencies, dependency, missing_dependencies
 
 # numpy compat


Test patch:
diff --git a/pandas/tests/test_base.py b/pandas/tests/test_base.py
--- a/pandas/tests/test_base.py
+++ b/pandas/tests/test_base.py
@@ -1,7 +1,9 @@
 from datetime import datetime, timedelta
+from importlib import reload
 from io import StringIO
 import re
 import sys
+from unittest.mock import patch
 
 import numpy as np
 import pytest
@@ -1341,3 +1343,28 @@ def test_to_numpy_dtype(as_series):
     expected = np.array(['2000-01-01T05', '2001-01-01T05'],
                         dtype='M8[ns]')
     tm.assert_numpy_array_equal(result, expected)
+
+
+@patch("builtins.__import__")
+def test_missing_required_dependency(mock_import):
+    def mock_import_fail(name, *args, **kwargs):
+        if name == "numpy":
+            raise ImportError("cannot import name numpy")
+        elif name == "pytz":
+            raise ImportError("cannot import name some_dependency")
+        elif name == "dateutil":
+            raise ImportError("cannot import name some_other_dependency")
+        else:
+            return __import__(name, *args, **kwargs)
+
+    mock_import.side_effect = mock_import_fail
+
+    expected_msg = (
+        "Unable to import required dependencies:"
+        "\nnumpy: cannot import name numpy"
+        "\npytz: cannot import name some_dependency"
+        "\ndateutil: cannot import name some_other_dependency"
+    )
+
+    with pytest.raises(ImportError, match=expected_msg):
+        reload(pd)