Repository: pex-tool/pex

Issue:
virtualenvwrapper breaks Pex (Python 3.5.2 on Ubuntu 16.04)
```console
$ virtualenv env
Running virtualenv with interpreter /usr/bin/python2
New python executable in /tmp/env/bin/python2
Also creating executable in /tmp/env/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
$ . env/bin/activate
$ pip install pex
Collecting pex
  Using cached pex-1.1.18-py2.py3-none-any.whl
Collecting setuptools<31.0,>=5.7 (from pex)
  Using cached setuptools-30.4.0-py2.py3-none-any.whl
Installing collected packages: setuptools, pex
  Found existing installation: setuptools 32.3.1
    Uninstalling setuptools-32.3.1:
      Successfully uninstalled setuptools-32.3.1
Successfully installed pex-1.1.18 setuptools-30.4.0
$ pex -o require3.5 --python-shebang=/usr/bin/env\ python3.5
$ deactivate
$ python3.5 --version
Python 3.5.2
$ ./require3.5 
Traceback (most recent call last):
  File ".bootstrap/_pex/pex.py", line 324, in execute
  File "/usr/lib/python3.5/contextlib.py", line 59, in __enter__
    return next(self.gen)
  File ".bootstrap/_pex/pex.py", line 250, in patch_sys
  File ".bootstrap/_pex/pex.py", line 210, in minimum_sys
  File ".bootstrap/_pex/pex.py", line 146, in minimum_sys_modules
TypeError: '_NamespacePath' object does not support indexing
```

But it works with Python 3.4:

```console
$ python3.4 ./require3.5 
Python 3.4.5 (default, Jul 15 2016, 16:39:07) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> 
```


Discussion:
I suspect this has something to do with building inside a virtualenv. Some things seem to work with 3.5 when run inside venv, but not outside it (not the above though). Hard to say.
Python 3.5 on Travis-CI container infrastructure seems happy with resulting pex files, unlike my computer.
A debug print for the `module` object in `pex.py:146` suggests the problem:

```
['/usr/lib/python3.5/distutils']
['/usr/lib/python3.5/logging']
['/usr/lib/python3.5/http']
['/usr/lib/python3.5/encodings']
['/usr/lib/python3.5/collections']
['/home/itamarst/Datawire/pib/dist/pib/.bootstrap/_pex']
_NamespacePath(['/home/itamarst/.local/lib/python3.5/site-packages/virtualenvwrapper'])
```

So the issue is bad interaction with virtualenvwrapper.

I think this should be fixed in `pex`, because the goal is to make universally redistributable packages, and saying "you can distribute it everywhere except where there's old versions of virtualenvwrapper" doesn't seem great.

I will see if I can provide a PR.
https://www.python.org/dev/peps/pep-0420/ discusses `_NamespacePath` objects.

Resolving patch:
diff --git a/pex/pex.py b/pex/pex.py
--- a/pex/pex.py
+++ b/pex/pex.py
@@ -141,6 +141,11 @@ def minimum_sys_modules(cls, site_libs, modules=None):
         new_modules[module_name] = module
         continue
 
+      # Unexpected objects, e.g. namespace packages, should just be dropped:
+      if not isinstance(module.__path__, list):
+        TRACER.log('Dropping %s' % (module_name,), V=3)
+        continue
+
       # Pop off site-impacting __path__ elements in-place.
       for k in reversed(range(len(module.__path__))):
         if cls._tainted_path(module.__path__[k], site_libs):


Test patch:
diff --git a/tests/test_pex.py b/tests/test_pex.py
--- a/tests/test_pex.py
+++ b/tests/test_pex.py
@@ -131,6 +131,22 @@ def test_minimum_sys_modules():
   assert new_modules == modules
   assert tainted_module.__path__ == ['good_path']
 
+  # If __path__ is not a list the module is removed; typically this implies
+  # it's a namespace package (https://www.python.org/dev/peps/pep-0420/) where
+  # __path__ is a _NamespacePath.
+  try:
+    from importlib._bootstrap_external import _NamespacePath
+    bad_path = _NamespacePath("hello", "world", None)
+  except ImportError:
+    bad_path = {"hello": "world"}
+  class FakeModule(object):
+    pass
+  tainted_module = FakeModule()
+  tainted_module.__path__ = bad_path   # Not a list as expected
+  modules = {'tainted_module': tainted_module}
+  new_modules = PEX.minimum_sys_modules(['bad_path'], modules)
+  assert new_modules == {}
+
 
 def test_site_libs():
   with nested(mock.patch.object(PEX, '_get_site_packages'), temporary_dir()) as (