Repository: pypa/pipenv

Issue:
Make pipenv sync skip installed dependencies
We are currently using pipenv sync to deploy some of our apps with the versions of packages that we have tested.  The problem we're having is just how long it takes for a sync to run.  We might install just one new package (or update one version), then have to wait several minutes for a sync to complete.  

Using `pipenv sync --verbose` it looks like a pip install is being called for every single dependency.  Wouldn't it be faster to check the existing hashes in the lock file against the dependencies which are already installed to confirm that they match?  I don't actually know what those hashes are calculated over so this is a bit of a naive request.  Ultimately my goal is just to make pipenv sync not spin for several minutes on existing dependencies when they don't need to be updated.

Also of note, we came from using our own virtual environments with a normal requirements file for pip.  Adding a new dependency or updating a version was _much_ quicker then, so it has to be something specific to the way pipenv is running independent installs that takes so long.

Currently using Python 3.6.6 with pip 18.1 and pipenv 2018.10.13




Discussion:
I think there is already an open issue about this. We don’t object, but don’t see it as a priority either. You can implement it if you want it now.
Fyi it’s actually the hash checking which is going to be the slow part here. Pip won’t actually install something that is already up to date, unless we are passing a bad flag somewhere in which case I’d be interested in that. Is it actually installing anything or simply just taking a long time to no-op?
I think it is the overhead of launching no-op pip subprocesses…
@techalchemy  Is hash checking (locking in general) a CPU bound operation or network based?

I was wondering how much of a speedup we can get by using Cython..
Similar with https://github.com/pypa/pipenv/issues/2895
@devxpy sorry for the delay, hash checking will depend, it's partly that we just need to chunk and hash every file in order to actually perform a hash check. We do have a cache for hashes which can speed things up for files that are already downloaded, but files that we download also have to be hashed for hash checking, so it's kind of both.

And launching subprocesses has overhead as well, which we've measured.  However, when I run pipenv with no additional subprocesses it has basically no impact.  I've been curious about cython myself

Resolving patch:
diff --git a/pipenv/core.py b/pipenv/core.py
--- a/pipenv/core.py
+++ b/pipenv/core.py
@@ -735,6 +735,9 @@ def batch_install(deps_list, procs, failed_deps_queue,
 
     deps_to_install = deps_list[:]
     deps_to_install.extend(sequential_deps)
+    deps_to_install = [
+        dep for dep in deps_to_install if not project.environment.is_satisfied(dep)
+    ]
     sequential_dep_names = [d.name for d in sequential_deps]
 
     deps_list_bar = progress.bar(
diff --git a/pipenv/environment.py b/pipenv/environment.py
--- a/pipenv/environment.py
+++ b/pipenv/environment.py
@@ -19,12 +19,14 @@
 import pipenv
 
 from .vendor.cached_property import cached_property
+from .vendor.packaging.utils import canonicalize_name
 from .vendor import vistir
 
 from .utils import normalize_path, make_posix
 
 
 BASE_WORKING_SET = pkg_resources.WorkingSet(sys.path)
+# TODO: Unittests for this class
 
 
 class Environment(object):
@@ -712,6 +714,33 @@ def is_installed(self, pkgname):
 
         return any(d for d in self.get_distributions() if d.project_name == pkgname)
 
+    def is_satisfied(self, req):
+        match = next(
+            iter(
+                d for d in self.get_distributions()
+                if canonicalize_name(d.project_name) == req.normalized_name
+            ), None
+        )
+        if match is not None:
+            if req.editable and req.line_instance.is_local and self.find_egg(match):
+                requested_path = req.line_instance.path
+                return requested_path and vistir.compat.samefile(requested_path, match.location)
+            elif match.has_metadata("direct_url.json"):
+                direct_url_metadata = json.loads(match.get_metadata("direct_url.json"))
+                commit_id = direct_url_metadata.get("vcs_info", {}).get("commit_id", "")
+                vcs_type = direct_url_metadata.get("vcs_info", {}).get("vcs", "")
+                _, pipfile_part = req.as_pipfile().popitem()
+                return (
+                    vcs_type == req.vcs and commit_id == req.commit_hash
+                    and direct_url_metadata["url"] == pipfile_part[req.vcs]
+                )
+            elif req.line_instance.specifiers is not None:
+                return req.line_instance.specifiers.contains(
+                    match.version, prereleases=True
+                )
+            return True
+        return False
+
     def run(self, cmd, cwd=os.curdir):
         """Run a command with :class:`~subprocess.Popen` in the context of the environment
 


Test patch: