Repository: pypa/setuptools

Issue:
pkg_resources.find_on_path() thinks the zc.recipe.egg wheel is an egg
Originally reported by: **mgedmin (Bitbucket: [mgedmin](http://bitbucket.org/mgedmin), GitHub: [mgedmin](http://github.com/mgedmin))**

---

Steps to reproduce: try to `pip install zc.recipe.egg`.  You'll get an error:

```
zc.recipe.egg is in an unsupported or invalid wheel
```

If you hack pip's sources to expose the real traceback, you'll see [this](https://github.com/pypa/pip/issues/3028#issuecomment-148614984):

```
Traceback (most recent call last):                                                                                                                                                                                                           
  File "/home/qwcode/p/pypa/pip/pip/basecommand.py", line 200, in main                                                                                                                                                                       
    status = self.run(options, args)                                                                                                                                                                                                         
  File "/home/qwcode/p/pypa/pip/pip/commands/install.py", line 302, in run                                                                                                                                                                   
    root=options.root_path,                                                                                                                                                                                                                  
  File "/home/qwcode/p/pypa/pip/pip/req/req_set.py", line 657, in install                                                                                                                                                                    
    **kwargs
  File "/home/qwcode/p/pypa/pip/pip/req/req_install.py", line 805, in install
    version = pip.wheel.wheel_version(self.source_dir)
  File "/home/qwcode/p/pypa/pip/pip/wheel.py", line 559, in wheel_version
    wheel_data = dist.get_metadata('WHEEL')
  File "/home/qwcode/p/pypa/pip/pip/_vendor/pkg_resources/__init__.py", line 1611, in get_metadata
    return self._get(self._fn(self.egg_info, name))
  File "/home/qwcode/p/pypa/pip/pip/_vendor/pkg_resources/__init__.py", line 1722, in _get
    with open(path, 'rb') as stream:
IOError: [Errno 2] No such file or directory: '/tmp/pip-build-ucnzjI/zc.recipe.egg/EGG-INFO/WHEEL'
```

AFAIU the problem is the following check in pkg_resources.find_on_path():

``` python
    if os.path.isdir(path_item) and os.access(path_item, os.R_OK):
        if path_item.lower().endswith('.egg'):
            # unpacked egg
```

This is wrong: the unpacked zc.recipe.egg _wheel_ has a top-level directory called `zc.recipe.egg`.

See https://github.com/pypa/pip/issues/3028 for more background.

---
- Bitbucket: https://bitbucket.org/pypa/setuptools/issue/462



Discussion:
_Original comment by_ **datakurre (Bitbucket: [datakurre](http://bitbucket.org/datakurre), GitHub: [datakurre](http://github.com/datakurre))**:

---

Are zipped eggs just files? If so, maybe

```
#!python

and (os.path.isfile(path)  # is zipped egg
     or os.path.isdir(os.path.join(path, 'EGG-INFO'))
```

_Original comment by_ **jaraco (Bitbucket: [jaraco](http://bitbucket.org/jaraco), GitHub: [jaraco](http://github.com/jaraco))**:

---

It seems that some of those tests for `endswith('.egg')` are also expected to match when the target is a zipped egg, and when that detection fails, it causes problems when installing zipped eggs (setuptools tests themselves fail to run).

_Original comment by_ **jaraco (Bitbucket: [jaraco](http://bitbucket.org/jaraco), GitHub: [jaraco](http://github.com/jaraco))**:

---

Back out 435b1b762fba. The change breaks detection of zip eggs. Reopens #462.

_Original comment by_ **jaraco (Bitbucket: [jaraco](http://bitbucket.org/jaraco), GitHub: [jaraco](http://github.com/jaraco))**:

---

Only detect a path as an unpacked egg if it also has an EGG-INFO directory. Fixes #462.

_Original comment by_ **jaraco (Bitbucket: [jaraco](http://bitbucket.org/jaraco), GitHub: [jaraco](http://github.com/jaraco))**:

---

Extract function for detecting unpacked egg. Ref #462.

_Original comment by_ **jaraco (Bitbucket: [jaraco](http://bitbucket.org/jaraco), GitHub: [jaraco](http://github.com/jaraco))**:

---

If making these changes addresses the issue, I have a patch staged and will try to push tomorrow.

_Original comment by_ **iElectric (Bitbucket: [iElectric](http://bitbucket.org/iElectric), GitHub: Unknown)**:

---

Note that it occurs at multiple places:

```
    substituteInPlace pkg_resources/__init__.py \
      --replace "if path_item.lower().endswith('.egg'):" "if path_item.lower().endswith('.egg') and os.path.isdir(os.path.join(path_item, 'EGG-INFO')):" \
      --replace "if subitem.endswith('.egg'):" "if subitem.endswith('.egg') and os.path.isdir(os.path.join(subitem, 'EGG-INFO')):" \
      --replace "if path.lower().endswith('.egg'):" "if path.lower().endswith('.egg') and os.path.isdir(os.path.join(path, 'EGG-INFO')):" \
      --replace "lower.endswith('.egg')" "lower.endswith('.egg') and os.path.isdir(os.path.join(lower, 'EGG-INFO'))"
```

_Original comment by_ **iElectric (Bitbucket: [iElectric](http://bitbucket.org/iElectric), GitHub: Unknown)**:

---

I've tried that and zc.recipe.egg will still claim to be unsupported wheel.

_Original comment by_ **mgedmin (Bitbucket: [mgedmin](http://bitbucket.org/mgedmin), GitHub: [mgedmin](http://github.com/mgedmin))**:

---

Proposed fix: change the inner check to

``` python
        if path_item.lower().endswith('.egg') and os.path.isdir(os.path.join(path_item, 'EGG-INFO')):
```

The `stable` build of the Zope RTD docs [just now failed due to this bug](https://readthedocs.org/projects/zope/builds/5049069/).
@jaraco Would updating just a bunch of conditionals be enough to fix this?
Would appreciate reviewers for #1116, which I'm proposing to fix this issue.

Resolving patch:
diff --git a/pkg_resources/__init__.py b/pkg_resources/__init__.py
--- a/pkg_resources/__init__.py
+++ b/pkg_resources/__init__.py
@@ -1553,7 +1553,7 @@ def _setup_prefix(self):
         path = self.module_path
         old = None
         while path != old:
-            if _is_unpacked_egg(path):
+            if _is_egg_path(path):
                 self.egg_name = os.path.basename(path)
                 self.egg_info = os.path.join(path, 'EGG-INFO')
                 self.egg_root = path
@@ -1956,7 +1956,7 @@ def find_eggs_in_zip(importer, path_item, only=False):
         # don't yield nested distros
         return
     for subitem in metadata.resource_listdir('/'):
-        if _is_unpacked_egg(subitem):
+        if _is_egg_path(subitem):
             subpath = os.path.join(path_item, subitem)
             for dist in find_eggs_in_zip(zipimport.zipimporter(subpath), subpath):
                 yield dist
@@ -2033,7 +2033,7 @@ def find_on_path(importer, path_item, only=False):
                     yield Distribution.from_location(
                         path_item, entry, metadata, precedence=DEVELOP_DIST
                     )
-                elif not only and _is_unpacked_egg(entry):
+                elif not only and _is_egg_path(entry):
                     dists = find_distributions(os.path.join(path_item, entry))
                     for dist in dists:
                         yield dist
@@ -2221,12 +2221,22 @@ def _normalize_cached(filename, _cache={}):
         return result
 
 
+def _is_egg_path(path):
+    """
+    Determine if given path appears to be an egg.
+    """
+    return (
+        path.lower().endswith('.egg')
+    )
+
+
 def _is_unpacked_egg(path):
     """
     Determine if given path appears to be an unpacked egg.
     """
     return (
-        path.lower().endswith('.egg')
+        _is_egg_path(path) and
+        os.path.isfile(os.path.join(path, 'EGG-INFO', 'PKG-INFO'))
     )
 
 
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -89,7 +89,7 @@ def pypi_link(pkg_filename):
 
 setup_params = dict(
     name="setuptools",
-    version="36.2.5",
+    version="36.2.6",
     description="Easily download, build, install, upgrade, and uninstall "
         "Python packages",
     author="Python Packaging Authority",


Test patch:
diff --git a/pkg_resources/tests/test_find_distributions.py b/pkg_resources/tests/test_find_distributions.py
new file mode 100644
--- /dev/null
+++ b/pkg_resources/tests/test_find_distributions.py
@@ -0,0 +1,65 @@
+import subprocess
+import sys
+
+import pytest
+import pkg_resources
+
+SETUP_TEMPLATE = """
+import setuptools
+setuptools.setup(
+    name="my-test-package",
+    version="1.0",
+    zip_safe=True,
+)
+""".lstrip()
+
+class TestFindDistributions:
+
+    @pytest.fixture
+    def target_dir(self, tmpdir):
+        target_dir = tmpdir.mkdir('target')
+        # place a .egg named directory in the target that is not an egg:
+        target_dir.mkdir('not.an.egg')
+        return str(target_dir)
+
+    @pytest.fixture
+    def project_dir(self, tmpdir):
+        project_dir = tmpdir.mkdir('my-test-package')
+        (project_dir / "setup.py").write(SETUP_TEMPLATE)
+        return str(project_dir)
+
+    def test_non_egg_dir_named_egg(self, target_dir):
+        dists = pkg_resources.find_distributions(target_dir)
+        assert not list(dists)
+
+    def test_standalone_egg_directory(self, project_dir, target_dir):
+        # install this distro as an unpacked egg:
+        args = [
+            sys.executable,
+            '-c', 'from setuptools.command.easy_install import main; main()',
+            '-mNx',
+            '-d', target_dir,
+            '--always-unzip',
+            project_dir,
+        ]
+        subprocess.check_call(args)
+        dists = pkg_resources.find_distributions(target_dir)
+        assert [dist.project_name for dist in dists] == ['my-test-package']
+        dists = pkg_resources.find_distributions(target_dir, only=True)
+        assert not list(dists)
+
+    def test_zipped_egg(self, project_dir, target_dir):
+        # install this distro as an unpacked egg:
+        args = [
+            sys.executable,
+            '-c', 'from setuptools.command.easy_install import main; main()',
+            '-mNx',
+            '-d', target_dir,
+            '--zip-ok',
+            project_dir,
+        ]
+        subprocess.check_call(args)
+        dists = pkg_resources.find_distributions(target_dir)
+        assert [dist.project_name for dist in dists] == ['my-test-package']
+        dists = pkg_resources.find_distributions(target_dir, only=True)
+        assert not list(dists)
diff --git a/tests/requirements.txt b/tests/requirements.txt
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -1,5 +1,6 @@
 importlib; python_version<"2.7"
 mock
 pytest-flake8; python_version>="2.7"
+virtualenv>=13.0.0
 pytest-virtualenv>=1.2.7
 pytest>=3.0.2