Repository: linkedin/shiv

Issue:
The change to sys.path potentially overrides standard library modules.
The change in #48 makes sure that shiv packed content (packages) has the precedence over the vended Python distribution packages or things installed into system Python site-packages. Fair enough. We want to run what we packed into the shiv.

However, it does have a potentially buggy behavior. Example:
* an old deprecated 3rd party package such as *uuid* or *argparse* is pulled in transitively
* these packages are unmaintained and did not get any changes (uuid since Python 2.5, for example)
* the standard library modules have been maintained and added new APIs (uuid did)
* since shiv's site-packages are stored on the sys.path **before** the standard library path, an obsolete 3rd party backward-compatibility package (but not **forward** compatible) will override the corresponding standard library module
* if any other package uses the new APIs from the affected package (e.g., new uuid APIs), the application will break because the old 3rd party package does not have these functions/methods.

I believe this requires changes to what was done in #48 to insert *shiv* site-packages *before* any other site-packages paths, but *after* the standard library path.

Discussion:
coveralls: ## Pull Request Test Coverage Report for [Build 120](https://coveralls.io/builds/18505798)

* **0** of **0**   changed or added relevant lines in **0** files are covered.
* **10** unchanged lines in **1** file lost coverage.
* Overall coverage decreased (**-1.2%**) to **73.93%**

---


|  Files with Coverage Reduction | New Missed Lines | % |
| :-----|--------------|--: |
| [.tox/py36/lib/python3.6/site-packages/shiv/bootstrap/__init__.py](https://coveralls.io/builds/18505798/source?filename=.tox%2Fpy36%2Flib%2Fpython3.6%2Fsite-packages%2Fshiv%2Fbootstrap%2F__init__.py#L103) | 10 | 51.56% |
<!-- | **Total:** | **10** |  | -->

|  Totals | [![Coverage Status](https://coveralls.io/builds/18505798/badge)](https://coveralls.io/builds/18505798) |
| :-- | --: |
| Change from base [Build 116](https://coveralls.io/builds/18505654): |  -1.2% |
| Covered Lines: | 190 |
| Relevant Lines: | 257 |

---
##### 💛  - [Coveralls](https://coveralls.io)


Patch:
diff --git a/src/shiv/bootstrap/__init__.py b/src/shiv/bootstrap/__init__.py
index 82169bf..02488ba 100644
--- a/src/shiv/bootstrap/__init__.py
+++ b/src/shiv/bootstrap/__init__.py
@@ -83,6 +83,12 @@ def extract_site_packages(archive, target_path):
     shutil.move(str(target_path_tmp), str(target_path))
 
 
+def _first_sitedir_index():
+    for index, part in enumerate(sys.path):
+        if Path(part).stem == 'site-packages':
+            return index
+
+
 def bootstrap():
     """Actually bootstrap our shiv environment."""
 
@@ -99,18 +105,18 @@ def bootstrap():
     if not site_packages.exists() or env.force_extract:
         extract_site_packages(archive, site_packages.parent)
 
-    preserved = sys.path[1:]
+    # get sys.path's length
+    length = len(sys.path)
 
-    # truncate the sys.path so our package will be at the start,
-    # and take precedence over anything else (eg: dist-packages)
-    sys.path = sys.path[0:1]
+    # Find the first instance of an existing site-packages on sys.path
+    index = _first_sitedir_index() or length
 
     # append site-packages using the stdlib blessed way of extending path
     # so as to handle .pth files correctly
     site.addsitedir(site_packages)
 
-    # restore the previous sys.path entries after our package
-    sys.path.extend(preserved)
+    # reorder to place our site-packages before any others found
+    sys.path = sys.path[:index] + sys.path[length:] + sys.path[index:length]
 
     # do entry point import and call
     if env.entry_point is not None and env.interpreter is None:


Test patch:
diff --git a/test/test_bootstrap.py b/test/test_bootstrap.py
index 5ece54f..1f77034 100644
--- a/test/test_bootstrap.py
+++ b/test/test_bootstrap.py
@@ -12,7 +12,7 @@ import pytest
 
 from unittest import mock
 
-from shiv.bootstrap import import_string, current_zipfile, cache_path
+from shiv.bootstrap import import_string, current_zipfile, cache_path, _first_sitedir_index
 from shiv.bootstrap.environment import Environment
 
 
@@ -61,6 +61,13 @@ class TestBootstrap:
 
         assert cache_path(mock_zip, Path.cwd(), uuid) == Path.cwd() / f"test_{uuid}"
 
+    def test_first_sitedir_index(self):
+        with mock.patch.object(sys, 'path', ['site-packages', 'dir', 'dir', 'dir']):
+            assert _first_sitedir_index() == 0
+
+        with mock.patch.object(sys, 'path', []):
+            assert _first_sitedir_index() is None
+
 
 class TestEnvironment:
     def test_overrides(self):