Repository: pypa/setuptools

Issue:
Difference in sys.path order, depending on how run
In the coverage.py test suite, I have tests that check various environmental values to see that they are they same for `coverage run myprog.py` as they are for `python myprog.py`.  One of the values is sys.path.  Recently, these tests started failing because a .egg file I installed with easy_install would appear in the middle of sys.path for coverage, but at the end for python.

I'm guessing this has something to do with #674, but I don't understand why.

The actual diff of sys.path for the two environments is:

```
      "path": [
          "/private/var/folders/j2/gr3cj3jn63s5q8g3bjvw57hm0000gp/T/test_cover_52005401",
          "/Users/ned/coverage/trunk/tests/modules",
          "/Users/ned/coverage/trunk/tests/zipmods.zip",
          "/Users/ned/coverage/trunk/.tox/py27/lib/python27.zip",
          "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7",
          "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/plat-darwin",
          "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/plat-mac",
          "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/plat-mac/lib-scriptpackages",
          "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/lib-tk",
          "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/lib-old",
          "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/lib-dynload",
          "/usr/local/pythonz/pythons/CPython-2.7.11/lib/python2.7",
          "/usr/local/pythonz/pythons/CPython-2.7.11/lib/python2.7/plat-darwin",
          "/usr/local/pythonz/pythons/CPython-2.7.11/lib/python2.7/lib-tk",
          "/usr/local/pythonz/pythons/CPython-2.7.11/lib/python2.7/plat-mac",
          "/usr/local/pythonz/pythons/CPython-2.7.11/lib/python2.7/plat-mac/lib-scriptpackages",
-         "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/covtestegg1-0.0.0-py2.7.egg",
          "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages",
          "/Users/ned/coverage/trunk",
+         "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/covtestegg1-0.0.0-py2.7.egg",
      ]
```

Is this expected? Is there something I can do to get the identical behavior in the two environments?



Discussion:
Hi Ned. I am slightly surprised by the behavior too, and I think you're right that #674 is implicated. To confirm, just set SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite, and if the tests pass again, we'll know for sure that the change is implicated.

Assuming that's the case, here's what I think is happening:
- tox creates the environment using an older version of setuptools, with SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite. This causes the `easy-install.pth` file to have the rewrite technique at the bottom, causing eggs to appear earlier in the stack.
- tox installs the covtestegg1, so it appears before site-packages.
- tests are run
- setuptools is upgraded, setting SETUPTOOLS_SYS_PATH_TECHNIQUE=raw, and either that upgrade or the installation of another package causes eggs to appear after site-packages.
- tests are run without coverage, but sys.path is different now.

I have a few questions - is setuptools being upgraded as part of the process? If you ran the coverage tests again at the end, I would expect sys.path to be the same as the second run; is it?

It occurs to me that a more proper workaround, assuming you have Setuptools >= 12 installed from the beginning, is that if you were to set SETUPTOOLS_SYS_PATH_TECHINQUE=raw, it would give forward compatibility and should give consistent results.

I hope that helps. Let me know what you find.

@jaraco running the failing test with `SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite` makes the test pass.

In this case, the test runs python and coverage directly, one after the other within one test, so I'm not sure there's an opportunity for things to change between them.  It seems like something is different about running a program with "python" and running it with a setuptools-created command like "coverage".

Here's the test: https://github.com/nedbat/coveragepy/blob/master/tests/test_process.py#L432-L438

```
    def test_coverage_run_is_like_python(self):
        with open(TRY_EXECFILE) as f:
            self.make_file("run_me.py", f.read())
        out_cov = self.run_command("coverage run run_me.py")
        out_py = self.run_command("python run_me.py")
        self.assertMultiLineEqual(out_cov, out_py)
        self.assert_execfile_output(out_cov)
```

To run one test:

```
COVERAGE_NO_PYTRACER=1 tox -e py27 -- tests/test_process.py:ProcessTest.test_coverage_run_is_like_python
```

I can try other experiments if you direct me :)

Oh, my. I wouldn't have expected it, but the `import pkg_resources` is implicated.

```
$ python -m venv ~/.envs/issue686
$ SETUPTOOLS_SYS_PATH_TECHINQUE=raw ~/.envs/issue686/bin/python -m easy_install --version                                                               
setuptools 20.10.1 from /Users/jaraco/.envs/issue686/lib/python3.5/site-packages (Python 3.5)
$ SETUPTOOLS_SYS_PATH_TECHINQUE=raw ~/.envs/issue686/bin/python -m easy_install -q requests
$ SETUPTOOLS_SYS_PATH_TECHINQUE=raw ~/.envs/issue686/bin/python -c "import pkg_resources, sys, pprint; pprint.pprint(sys.path)" > with-pkg_resources.txt
$ SETUPTOOLS_SYS_PATH_TECHINQUE=raw ~/.envs/issue686/bin/python -c "import sys, pprint; pprint.pprint(sys.path)" > without-pkg_resources.txt            
$ diff with-pkg_resources.txt without-pkg_resources.txt 
6,7c6,7
<  '/Users/jaraco/.envs/issue686/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg',
<  '/Users/jaraco/.envs/issue686/lib/python3.5/site-packages']
---
>  '/Users/jaraco/.envs/issue686/lib/python3.5/site-packages',
>  '/Users/jaraco/.envs/issue686/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg']
```

@minrk: Can you investigate and see if you can identify a way for pkg_resources to honor the ordering implied by `easy-install.pth` per your pull request?

@jaraco I've found the culprit, but I'm not 100% sure what the desired behavior is.

The triggering sequence:
1. requests.egg is already on sys.path, so it is in the initial working set, immediately after its `site-packages`, as `.pth` files are supposed to behave.
2. `_initialize_master_working_set` [activates](https://github.com/pypa/setuptools/blob/v25.1.0/pkg_resources/__init__.py#L2953) all distributions already on sys.path
3. `activate` [triggers](https://github.com/pypa/setuptools/blob/v25.1.0/pkg_resources/__init__.py#L2510) `insert_on`, which [defines its behavior](https://github.com/pypa/setuptools/blob/v25.1.0/pkg_resources/__init__.py#L2588) as "Insert self.location in path before its nearest parent directory"

It seems to me that `activate` shouldn't be touching sys.path if the dist is already on sys.path, at least in this case, but I don't know all of the various cases `dist.activate()` should cover.

Simply removing the `.activate()` call from the startup sequence gives the behavior I want/expect, but I'm guessing that's not the right answer.

The high-level problem seems to be that 
1. `.activate()` triggers a forceful, unconditional modification of sys.path.
2. any eggs on sys.path get `.activated()` on import of pkg_resources.

For my usage, 1. seems to be the problem - I never want pkg_resources to touch sys.path under any circumstances.

I also don't fully understand why pkg_resources is modifying sys.path, because `easy-install.pth` has already done so (or not, with SETUPTOOLS_SYS_PATH_TECHINQUE=raw).

I assume there are some exceptional cases where pkg_resources is actually meant to modify sys.path. Can you enumerate those, so we can limit the condition when sys.path is modified to less than always?


Resolving patch:
diff --git a/pkg_resources/__init__.py b/pkg_resources/__init__.py
--- a/pkg_resources/__init__.py
+++ b/pkg_resources/__init__.py
@@ -947,11 +947,17 @@ def require(self, *requirements):
 
         return needed
 
-    def subscribe(self, callback):
-        """Invoke `callback` for all distributions (including existing ones)"""
+    def subscribe(self, callback, existing=True):
+        """Invoke `callback` for all distributions
+
+        If `existing=True` (default),
+        call on all existing ones, as well.
+        """
         if callback in self.callbacks:
             return
         self.callbacks.append(callback)
+        if not existing:
+            return
         for dist in self:
             callback(dist)
 
@@ -2503,11 +2509,11 @@ def _get_metadata(self, name):
             for line in self.get_metadata_lines(name):
                 yield line
 
-    def activate(self, path=None):
+    def activate(self, path=None, replace=False):
         """Ensure distribution is importable on `path` (default=sys.path)"""
         if path is None:
             path = sys.path
-        self.insert_on(path, replace=True)
+        self.insert_on(path, replace=replace)
         if path is sys.path:
             fixup_namespace_packages(self.location)
             for pkg in self._get_metadata('namespace_packages.txt'):
@@ -2585,7 +2591,24 @@ def get_entry_info(self, group, name):
         return self.get_entry_map(group).get(name)
 
     def insert_on(self, path, loc=None, replace=False):
-        """Insert self.location in path before its nearest parent directory"""
+        """Ensure self.location is on path
+
+        If replace=False (default):
+            - If location is already in path anywhere, do nothing.
+            - Else:
+              - If it's an egg and its parent directory is on path,
+                insert just ahead of the parent.
+              - Else: add to the end of path.
+        If replace=True:
+            - If location is already on path anywhere (not eggs)
+              or higher priority than its parent (eggs)
+              do nothing.
+            - Else:
+              - If it's an egg and its parent directory is on path,
+                insert just ahead of the parent,
+                removing any lower-priority entries.
+              - Else: add it to the front of path.
+        """
 
         loc = loc or self.location
         if not loc:
@@ -2597,9 +2620,16 @@ def insert_on(self, path, loc=None, replace=False):
 
         for p, item in enumerate(npath):
             if item == nloc:
-                break
+                if replace:
+                    break
+                else:
+                    # don't modify path (even removing duplicates) if found and not replace
+                    return
             elif item == bdir and self.precedence == EGG_DIST:
                 # if it's an .egg, give it precedence over its directory
+                # UNLESS it's already been added to sys.path and replace=False
+                if (not replace) and nloc in npath[p:]:
+                    return
                 if path is sys.path:
                     self.check_version_conflict()
                 path.insert(p, loc)
@@ -2947,10 +2977,14 @@ def _initialize_master_working_set():
     run_script = working_set.run_script
     # backward compatibility
     run_main = run_script
-    # Activate all distributions already on sys.path, and ensure that
-    # all distributions added to the working set in the future (e.g. by
-    # calling ``require()``) will get activated as well.
-    add_activation_listener(lambda dist: dist.activate())
+    # Activate all distributions already on sys.path with replace=False and
+    # ensure that all distributions added to the working set in the future
+    # (e.g. by calling ``require()``) will get activated as well,
+    # with higher priority (replace=True).
+    for dist in working_set:
+        dist.activate(replace=False)
+    del dist
+    add_activation_listener(lambda dist: dist.activate(replace=True), existing=False)
     working_set.entries=[]
     # match order
     list(map(working_set.add_entry, sys.path))


Test patch: