Repository: pypa/pipenv

Issue:
AttributeError: '_NamespacePath' object has no attribute 'sort' when running pipenv on arch
I installed pipenv (with pip) on my arch machine recently, and it's giving me an exception on startup.

Googling around it seems like it might related to this issue: pypa/setuptools#885, but my setuptools is fully updated which seems to be the fix listed in that thread.

## output
```
→ sudo pip install --upgrade pipenv pip setuptools
→ pipenv --help
Traceback (most recent call last):
  File "/usr/bin/pipenv", line 7, in <module>
    from pipenv import cli
  File "/usr/lib/python3.6/site-packages/pipenv/__init__.py", line 13, in <module>
    from .cli import cli
  File "/usr/lib/python3.6/site-packages/pipenv/cli.py", line 20, in <module>
    import pip
  File "/usr/lib/python3.6/site-packages/pip/__init__.py", line 26, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/usr/lib/python3.6/site-packages/pip/utils/__init__.py", line 27, in <module>
    from pip._vendor import pkg_resources
  File "/usr/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3018, in <module>
    @_call_aside
  File "/usr/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3004, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3046, in _initialize_master_working_set
    dist.activate(replace=False)
  File "/usr/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2578, in activate
    declare_namespace(pkg)
  File "/usr/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2152, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2092, in _handle_ns
    _rebuild_mod_path(path, packageName, module)
  File "/usr/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2121, in _rebuild_mod_path
    orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'
```

## system info
```
→ pip -V
pip 9.0.1 from /usr/lib/python3.6/site-packages (python 3.6)

→ pip show pipenv
Name: pipenv
Version: 7.0.5
Summary: Sacred Marriage of Pipfile, Pip, & Virtualenv.
Home-page: https://github.com/kennethreitz/pipenv
Author: Kenneth Reitz
Author-email: me@kennethreitz.org
License: MIT
Location: /usr/lib/python3.6/site-packages
Requires: semver, setuptools, pip-tools, pip, pew, virtualenv, requests

→ pip show setuptools
Name: setuptools
Version: 36.4.0
Summary: Easily download, build, install, upgrade, and uninstall Python packages
Home-page: https://github.com/pypa/setuptools
Author: Python Packaging Authority
Author-email: distutils-sig@python.org
License: UNKNOWN
Location: /usr/lib/python3.6/site-packages
Requires: 

→ uname -a
Linux jubal 4.12.8-2-ARCH #1 SMP PREEMPT Fri Aug 18 14:08:02 UTC 2017 x86_64 GNU/Linux

→ python -V
Python 3.6.2

```


Discussion:
this looks to be a problem with pip, not with pipenv. 
@BenDoan Did you manage to fix this? I'm having the exact same issue.
@hobarrera Unfortunately no.
I looked through https://github.com/pypa/setuptools/issues/885, and I don't entirely follow.

This looks like an issue within `pip`. Specifically, it's due to code vendored in pip. The thing is, I don't quite understand in which sort of scenario pipenv _could_ work, since there seems to be a reliance on a broken upstream feature. 😐
we use a patched pip, so we can fix anything needed!
Ah, good point. I didn't notice pipenv used a patched pip.

If that's the case, then isn't this a `pipenv` issue? The patched/vendored pip seems to be what's failing:
```
Traceback (most recent call last):
  File "/usr/bin/pipenv", line 11, in <module>
    load_entry_point('pipenv==8.0.2', 'console_scripts', 'pipenv')()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 570, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2751, in load_entry_point
    return ep.load()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2405, in load
    return self.resolve()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2411, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3.6/site-packages/pipenv/__init__.py", line 17, in <module>
    from .cli import cli
  File "/usr/lib/python3.6/site-packages/pipenv/cli.py", line 22, in <module>
    import pip
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/__init__.py", line 26, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/utils/__init__.py", line 27, in <module>
    from pip._vendor import pkg_resources
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 3018, in <module>
    @_call_aside
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 3004, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 3046, in _initialize_master_working_set
    dist.activate(replace=False)
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 2578, in activate
    declare_namespace(pkg)
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 2152, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 2092, in _handle_ns
    _rebuild_mod_path(path, packageName, module)
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 2121, in _rebuild_mod_path
    orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'
```
nope, still a pip issue. we just make a few small cosmetic changes to the codebase
do not specify setuptools in your Pipfile. that is bad. 
those are system components. 
you are using it wrong. 
> do not specify setuptools in your Pipfile. that is bad.

I'm not doing that; I don't have a Pipfile.

> you are using it wrong.

I'm not sure how I'm using it wrong. I'm following the instructions [here](https://pipenv.readthedocs.io/en/latest/basics.html#installing-packages-for-your-project):

```
$ mkdir myproject
$ cd myproject
$ pipenv install requests
Traceback (most recent call last):
  File "/usr/bin/pipenv", line 11, in <module>
    load_entry_point('pipenv==8.0.2', 'console_scripts', 'pipenv')()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 570, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2751, in load_entry_point
    return ep.load()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2405, in load
    return self.resolve()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2411, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3.6/site-packages/pipenv/__init__.py", line 17, in <module>
    from .cli import cli
  File "/usr/lib/python3.6/site-packages/pipenv/cli.py", line 22, in <module>
    import pip
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/__init__.py", line 26, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/utils/__init__.py", line 27, in <module>
    from pip._vendor import pkg_resources
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 3018, in <module>
    @_call_aside
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 3004, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 3046, in _initialize_master_working_set
    dist.activate(replace=False)
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 2578, in activate
    declare_namespace(pkg)
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 2152, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 2092, in _handle_ns
    _rebuild_mod_path(path, packageName, module)
  File "/usr/lib/python3.6/site-packages/pipenv/patched/pip/_vendor/pkg_resources/__init__.py", line 2121, in _rebuild_mod_path
    orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'
```
> nope, still a pip issue. we just make a few small cosmetic changes to the codebase

This confuses me. If it's a pip issue, and pip is vendored, I don't quite understand how pipenv would work on any other setup. I'm clearly missing something obvious, but I'm not sure what.

Resolving patch:
diff --git a/pipenv/patched/pip/_vendor/pkg_resources/__init__.py b/pipenv/patched/pip/_vendor/pkg_resources/__init__.py
--- a/pipenv/patched/pip/_vendor/pkg_resources/__init__.py
+++ b/pipenv/patched/pip/_vendor/pkg_resources/__init__.py
@@ -74,13 +74,8 @@
 __import__('pip._vendor.packaging.requirements')
 __import__('pip._vendor.packaging.markers')
 
-
 if (3, 0) < sys.version_info < (3, 3):
-    msg = (
-        "Support for Python 3.0-3.2 has been dropped. Future versions "
-        "will fail here."
-    )
-    warnings.warn(msg)
+    raise RuntimeError("Python 3.3 or later is required")
 
 # declare some globals that will be defined later to
 # satisfy the linters.
@@ -791,7 +786,7 @@ def add(self, dist, entry=None, insert=True, replace=False):
         self._added_new(dist)
 
     def resolve(self, requirements, env=None, installer=None,
-            replace_conflicting=False):
+                replace_conflicting=False, extras=None):
         """List all distributions needed to (recursively) meet `requirements`
 
         `requirements` must be a sequence of ``Requirement`` objects.  `env`,
@@ -807,6 +802,12 @@ def resolve(self, requirements, env=None, installer=None,
         the wrong version.  Otherwise, if an `installer` is supplied it will be
         invoked to obtain the correct version of the requirement and activate
         it.
+
+        `extras` is a list of the extras to be used with these requirements.
+        This is important because extra requirements may look like `my_req;
+        extra = "my_extra"`, which would otherwise be interpreted as a purely
+        optional requirement.  Instead, we want to be able to assert that these
+        requirements are truly required.
         """
 
         # set up the stack
@@ -830,7 +831,7 @@ def resolve(self, requirements, env=None, installer=None,
                 # Ignore cyclic or redundant dependencies
                 continue
 
-            if not req_extras.markers_pass(req):
+            if not req_extras.markers_pass(req, extras):
                 continue
 
             dist = best.get(req.key)
@@ -1009,7 +1010,7 @@ class _ReqExtras(dict):
     Map each requirement to the extras that demanded it.
     """
 
-    def markers_pass(self, req):
+    def markers_pass(self, req, extras=None):
         """
         Evaluate markers for req against each extra that
         demanded it.
@@ -1019,7 +1020,7 @@ def markers_pass(self, req):
         """
         extra_evals = (
             req.marker.evaluate({'extra': extra})
-            for extra in self.get(req, ()) + (None,)
+            for extra in self.get(req, ()) + (extras or (None,))
         )
         return not req.marker or any(extra_evals)
 
@@ -1956,6 +1957,12 @@ def find_eggs_in_zip(importer, path_item, only=False):
             subpath = os.path.join(path_item, subitem)
             for dist in find_eggs_in_zip(zipimport.zipimporter(subpath), subpath):
                 yield dist
+        elif subitem.lower().endswith('.dist-info'):
+            subpath = os.path.join(path_item, subitem)
+            submeta = EggMetadata(zipimport.zipimporter(subpath))
+            submeta.egg_info = subpath
+            yield Distribution.from_location(path_item, subitem, submeta)
+
 
 
 register_finder(zipimport.zipimporter, find_eggs_in_zip)
@@ -2118,6 +2125,10 @@ def position_in_sys_path(path):
         parts = path_parts[:-module_parts]
         return safe_sys_path_index(_normalize_cached(os.sep.join(parts)))
 
+    if not isinstance(orig_path, list):
+        # Is this behavior useful when module.__path__ is not a list?
+        return
+
     orig_path.sort(key=position_in_sys_path)
     module.__path__[:] = [_normalize_cached(p) for p in orig_path]
 
@@ -2304,8 +2315,14 @@ def resolve(self):
     def require(self, env=None, installer=None):
         if self.extras and not self.dist:
             raise UnknownExtra("Can't require() without a distribution", self)
+
+        # Get the requirements for this entry point with all its extras and
+        # then resolve them. We have to pass `extras` along when resolving so
+        # that the working set knows what extras we want. Otherwise, for
+        # dist-info distributions, the working set will assume that the
+        # requirements for that extra are purely optional and skip over them.
         reqs = self.dist.requires(self.extras)
-        items = working_set.resolve(reqs, env, installer)
+        items = working_set.resolve(reqs, env, installer, extras=self.extras)
         list(map(working_set.add, items))
 
     pattern = re.compile(
@@ -3010,9 +3027,11 @@ def _initialize(g=globals()):
     "Set up global resource manager (deliberately not state-saved)"
     manager = ResourceManager()
     g['_manager'] = manager
-    for name in dir(manager):
-        if not name.startswith('_'):
-            g[name] = getattr(manager, name)
+    g.update(
+        (name, getattr(manager, name))
+        for name in dir(manager)
+        if not name.startswith('_')
+    )
 
 
 @_call_aside
@@ -3041,10 +3060,10 @@ def _initialize_master_working_set():
     # ensure that all distributions added to the working set in the future
     # (e.g. by calling ``require()``) will get activated as well,
     # with higher priority (replace=True).
-    dist = None  # ensure dist is defined for del dist below
-    for dist in working_set:
+    tuple(
         dist.activate(replace=False)
-    del dist
+        for dist in working_set
+    )
     add_activation_listener(lambda dist: dist.activate(replace=True), existing=False)
     working_set.entries = []
     # match order


Test patch: