Repository: pypa/pipenv

Issue:
Editables from pipfile are installed shallow

Dependencies of editables are not included in the lock file, so they do not get installed during a fresh `pipenv install`, this is due to the `no_deps` switch in https://github.com/kennethreitz/pipenv/blob/023696e46993bf55cd2807eae1801f8884a6357a/pipenv/cli.py#L775

A minimal example is demonstrated in the following gist
https://gist.github.com/PaulVisscher/f20d2d56fff66414b85ed77622a3aa66

##### Describe your environment

1. Debian GNU/Linux 8
1. Python version: Python 3.6.3
1. Pipenv version: 3.8.2

##### Expected result
Dependencies of editable packages are added to the lock file and installed

##### Actual result
Editable packages are installed shallow: dependencies of editable packages are not added to the lock file, and thus not installed.
```
pipenv shell
python
>>> import foo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/dummy/foo/foo/__init__.py", line 1, in <module>
    import jsonschema
ModuleNotFoundError: No module named 'jsonschema'
```

##### Steps to replicate
 - Replicate the structure from the linked [gist](https://github.com/kennethreitz/pipenv/blob/023696e46993bf55cd2807eae1801f8884a6357a/pipenv/cli.py#L775) or download the included test folder from the [test-case.zip](https://github.com/kennethreitz/pipenv/files/1480495/test-case.zip).
- Run
```
pipenv install
pipenv shell
````
- Run
```
python
>>> import foo
````




Discussion:
I'm having the same issue (tested in Debian 8.7)... I'll put together a more helpful writeup shortly, but briefly:

I'm using setup.py to specify install_requires. I'm using Pipfile to reference the setup.py:
```
[packages]
"e1839a8" = {path=".", editable=true}
```

If I do **pipenv install** none of my "setup.py install_requires" stuff gets installed.
If I do: **pipenv install --skip-lock** all packages get installed properly.

And of course, outside of pipenv, the setup.py file works well by doing:
**python setup.py install** or
**python setup.py develop** or
**pip install -e .**
yes, this is indeed an issue.

you don't need to do much more of a writeup unless you can point at the cause (or more helpfully the solution!)

I am kind of thinking this relates to our other dependency resolution issues now that I've spent a bunch of time on that issue...


@PaulVisscher the reason `no_deps` is true there is because the deps of editable files are supposed to be resolved into the lockfile, which is the real problem here.

Resolving patch:
diff --git a/pipenv/project.py b/pipenv/project.py
--- a/pipenv/project.py
+++ b/pipenv/project.py
@@ -56,23 +56,34 @@ def path_to(self, p):
     def _build_package_list(self, package_section):
         """Returns a list of packages for pip-tools to consume."""
         ps = {}
+        # TODO: Separate the logic for showing packages from the filters for supplying pip-tools
         for k, v in self.parsed_pipfile.get(package_section, {}).items():
             # Skip editable VCS deps.
             if hasattr(v, 'keys'):
-                # When a vcs url is given without editable it only appears as a key
-                if is_vcs(v) or is_vcs(k):
-                    # Non-editable VCS entries can't be resolved by piptools
+                # When a vcs url is gven without editable it only appears as a key
+                # Eliminate any vcs, path, or url entries which are not editable
+                # Since pip-tools can't do deep resolution on them, even setuptools-installable ones
+                if (is_vcs(v) or is_vcs(k) or (is_installable_file(k) or is_installable_file(v)) or
+                        any((prefix in v and
+                             (os.path.isfile(v[prefix]) or is_valid_url(v[prefix])))
+                            for prefix in ['path', 'file'])):
+                    # If they are editable, do resolve them
                     if 'editable' not in v:
                         continue
                     else:
                         ps.update({k: v})
                 else:
-                    if not (is_installable_file(k) or is_installable_file(v) or
-                            any(file_prefix in v for file_prefix in ['path', 'file'])):
-                        ps.update({k: v})
+                    ps.update({k: v})
             else:
+                # Since these entries have no attributes we know they are not editable
+                # So we can safely exclude things that need to be editable in order to be resolved
+                # First exclude anything that is a vcs entry either in the key or value
                 if not (any(is_vcs(i) for i in [k, v]) or
+                        # Then exclude any installable files that are not directories
+                        # Because pip-tools can resolve setup.py for example
                         any(is_installable_file(i) for i in [k, v]) or
+                        # Then exclude any URLs because they need to be editable also
+                        # Things that are excluded can only be 'shallow resolved'
                         any(is_valid_url(i) for i in [k, v])):
                     ps.update({k: v})
         return ps
diff --git a/pipenv/utils.py b/pipenv/utils.py
--- a/pipenv/utils.py
+++ b/pipenv/utils.py
@@ -304,7 +304,7 @@ def get_requirement(dep):
         # Only parse if it is a file or an installable dir
         if dep_path.is_file() or (dep_path.is_dir() and pip.utils.is_installable_dir(dep)):
             if dep_path.is_absolute() or dep_path.as_posix() == '.':
-                path = dep
+                path = dep_path.as_posix()
             else:
                 path = get_converted_relative_path(dep)
             dep = dep_path.resolve().as_uri()


Test patch:
diff --git a/tests/test_pipenv.py b/tests/test_pipenv.py
--- a/tests/test_pipenv.py
+++ b/tests/test_pipenv.py
@@ -879,6 +879,29 @@ def test_urls_work(self):
 
             assert 'file' in dep
 
+    @pytest.mark.install
+    @pytest.mark.files
+    @pytest.mark.resolver
+    def test_local_package(self):
+        """This test ensures that local packages (directories with a setup.py)
+        installed in editable mode have their dependencies resolved as well"""
+        file_name = 'tablib-0.12.1.tar.gz'
+        package = 'tablib-0.12.1'
+        # Not sure where travis/appveyor run tests from
+        test_dir = os.path.dirname(os.path.abspath(__file__))
+        source_path = os.path.abspath(os.path.join(test_dir, 'test_artifacts', file_name))
+        with PipenvInstance() as p:
+            # This tests for a bug when installing a zipfile in the current dir
+            copy_to = os.path.join(p.path, file_name)
+            shutil.copy(source_path, copy_to)
+            import tarfile
+            with tarfile.open(copy_to, 'r:gz') as tgz:
+                tgz.extractall(path=p.path)
+            c = p.pipenv('install -e {0}'.format(package))
+            assert c.return_code == 0
+            assert all(pkg in p.lockfile['default'] for pkg in ['xlrd', 'xlwt', 'pyyaml', 'odfpy'])
+
+
     @pytest.mark.install
     @pytest.mark.files
     def test_local_zipfiles(self):