Repository: pypa/pipenv

Issue:
Git submodules in docker
`pipenv, version 11.10.0`

I have a submodule `./dep1` in my git repo and am adding it to pipenv via `pipenv install -e dep1`, which works just fine and adds the line
```
"abc123" = {path = "./dep1", editable = true}
```
to my Pipfile. Then I use a Dockerfile as follows:
```
FROM kennethreitz/pipenv
COPY ./app /app
COPY ./dep1 /app/dep1
CMD python3 start.py
```
where I added the third line, so the submodule is available in the WORKDIR. However, upon `docker build` I am getting the following error 
```
+ pipenv install --deploy --system
Installing -e ./dep1…

Error:  An error occurred while installing -e ./dep1!
./dep1 should either be a path to a local project or a VCS url beginning with svn+, git+, hg+, or bzr+
```
This is unexpected, since ./dep1 *is* a path to a local project, relative to the WORKDIR in the docker image with the Pipfile, which is /app. Am I missing something or is this a bug?


Discussion:
Hi @peter-ki!  Thanks for using pipenv!  I need to update the docker instance to use the most current version of pipenv.  Can you try rewriting your dockerfile to pull from python:3.6 or whichever version you are using and then pip install --upgrade pipenv?  I'm wondering if our latest release fixes this issue.  Additionally could you post your pipfile and Dockerfile?  Also would it be possible to launch an interactive docker session and run the commands requested by the issue template so I can see the output?  I know that's a lot, docker issues can be problematic to troubleshoot.
The dockerfile would be closer to ours in the repo: 
```
FROM ubuntu:18.04

# -- Install Pipenv:
RUN apt update && apt install python3-pip git -y && pip3 install pipenv

ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8

# -- Install Application into container:
RUN set -ex && mkdir /app

WORKDIR /app

# -- Adding Pipfiles
ONBUILD COPY Pipfile Pipfile
ONBUILD COPY Pipfile.lock Pipfile.lock

# -- Install dependencies:
ONBUILD RUN set -ex && pipenv install --deploy --system
```
@erinxocon thank you very much for the fast response! 

Interesting. I tried to work with the Dockerfile you provided, which seems to take the process a step further. I get a different error, though: 
```
+ pipenv install --deploy --system
Installing -e file:///app/data-cache...
Obtaining file:///app/data-cache
Installing collected packages: DataCache
  Running setup.py develop for DataCache
Successfully installed DataCache

Adding -e file:///app/data-cache to Pipfile's [packages]...
Your Pipfile.lock (34a2ab) is out of date. Expected: (6bd758).
Aborting deploy.
```

The Pipfile is 
```
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
kafka = "*"
pandas = "*"
numpy = "*"
"1bc1e69" = {path = "./data-cache", editable = true}

[dev-packages]

[requires]
python_version = "3.6"
```

and the Dockerfile now is
```
FROM ubuntu:18.04

# -- Install Pipenv:
RUN apt update && apt install python3-pip git -y && pip3 install pipenv

ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8

# -- Install Application into container:
RUN set -ex && mkdir /app

COPY ./services /app
COPY ./data-cache /app/data-cache

WORKDIR /app

# -- Adding Pipfiles
COPY Pipfile Pipfile
COPY Pipfile.lock Pipfile.lock

# -- Install dependencies:
RUN set -ex && pipenv install --deploy --system

CMD python3 start.py
```
Hm...

I also could run in the container interactively. But what does *commands requested by the issue template* refer to?
@peter-ki Thanks for the info.  So when you create a bug report we ask you to run the command `python -m pipenv.help` and paste the output, it contains diagnostic info for use in troubleshooting the problem.  And since yours is with a local package it would be helpful to get a capture of that.  So if you could run `python -m pipenv.help` in the interactive session and paste the output that would be appreciated!
@peter-ki Also what vesion of pipenv are you using to generate the `lockfile`?  Can you try updating to our latest release using `pip/pip3 install pipenv --upgrade` and regenerate the lockfile and then rebuild the docker container with the new lockfile? The error is telling you that your lockfile is out of date and can't be used with `--deploy`
Ah sorry, got it. Here you go:

root@36f1e613ddbd:/app# python3 -m pipenv.help
<details><summary>$ python -m pipenv.help output</summary>

Pipenv version: `'2018.6.25'`

Pipenv location: `'/usr/local/lib/python3.6/dist-packages/pipenv'`

Python location: `'/usr/bin/python3'`

Other Python installations in `PATH`:

  - `3.6`: `/usr/bin/python3.6m`
  - `3.6`: `/usr/bin/python3.6`

  - `3.6.5`: `/usr/bin/python3`

PEP 508 Information:

```
{'implementation_name': 'cpython',
 'implementation_version': '3.6.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.9.87-linuxkit-aufs',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Wed Mar 14 15:12:16 UTC 2018',
 'python_full_version': '3.6.5',
 'python_version': '3.6',
 'sys_platform': 'linux'}
```

System environment variables:

  - `LC_ALL`
  - `LS_COLORS`
  - `LESSCLOSE`
  - `LANG`
  - `KAFKA_SERVER_ENDPOINTS`
  - `HOSTNAME`
  - `PWD`
  - `HOME`
  - `TERM`
  - `SHLVL`
  - `PATH`
  - `LESSOPEN`
  - `_`
  - `PYTHONDONTWRITEBYTECODE`
  - `PIP_PYTHON_PATH`

Pipenv–specific environment variables:


Debug–specific environment variables:

  - `PATH`: `/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin`
  - `LANG`: `C.UTF-8`
  - `PWD`: `/app`


---------------------------

Contents of `Pipfile` ('/app/Pipfile'):

```toml
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
kafka = "*"
pandas = "*"
numpy = "*"
"1bc1e69" = {path = "./data-cache", editable = true}
datacache = {editable = true, file = "file:///app/data-cache"}

[dev-packages]

[requires]
python_version = "3.6"

```


Contents of `Pipfile.lock` ('/app/Pipfile.lock'):

```json
{
    "_meta": {
        "hash": {
            "sha256": "599ae093fa031e777034f7217b53754baeca64542ef99125d2603a6b2834a2ab"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "1bc1e69": {
            "editable": true,
            "path": "./data-cache"
        },
        "kafka": {
            "hashes": [
                "sha256:35a49e9a432f6792aad48b2ab09d03867357aaa06eacba2678b7fd0808156e5c",
                "sha256:ffb3348cd918c31344b1a2eb61f692cbc423c83b1b06466a86de3db8a3f6a018"
            ],
            "index": "pypi",
            "version": "==1.3.5"
        },
        "numpy": {
            "hashes": [
                "sha256:07379fe0b450f6fd6e5934a9bc015025bb4ce1c8fbed3ca8bef29328b1bc9570",
                "sha256:085afac75bbc97a096744fcfc97a4b321c5a87220286811e85089ae04885acdd",
                "sha256:2d6481c6bdab1c75affc0fc71eb1bd4b3ecef620d06f2f60c3f00521d54be04f",
                "sha256:2df854df882d322d5c23087a4959e145b953dfff2abe1774fec4f639ac2f3160",
                "sha256:381ad13c30cd1d0b2f3da8a0c1a4aa697487e8bb0e9e0cbeb7439776bcb645f8",
                "sha256:385f1ce46e08676505b692bfde918c1e0b350963a15ef52d77691c2cf0f5dbf6",
                "sha256:4d278c2261be6423c5e63d8f0ceb1b0c6db3ff83f2906f4b860db6ae99ca1bb5",
                "sha256:51c5dcb51cf88b34b7d04c15f600b07c6ccbb73a089a38af2ab83c02862318da",
                "sha256:589336ba5199c8061239cf446ee2f2f1fcc0c68e8531ee1382b6fc0c66b2d388",
                "sha256:5edf1acc827ed139086af95ce4449b7b664f57a8c29eb755411a634be280d9f2",
                "sha256:6b82b81c6b3b70ed40bc6d0b71222ebfcd6b6c04a6e7945a936e514b9113d5a3",
                "sha256:6c57f973218b776195d0356e556ec932698f3a563e2f640cfca7020086383f50",
                "sha256:758d1091a501fd2d75034e55e7e98bfd1370dc089160845c242db1c760d944d9",
                "sha256:8622db292b766719810e0cb0f62ef6141e15fe32b04e4eb2959888319e59336b",
                "sha256:8b8dcfcd630f1981f0f1e3846fae883376762a0c1b472baa35b145b911683b7b",
                "sha256:97fa8f1dceffab782069b291e38c4c2227f255cdac5f1e3346666931df87373e",
                "sha256:9d69967673ab7b028c2df09cae05ba56bf4e39e3cb04ebe452b6035c3b49848e",
                "sha256:9e1f53afae865cc32459ad211493cf9e2a3651a7295b7a38654ef3d123808996",
                "sha256:a4a433b3a264dbc9aa9c7c241e87c0358a503ea6394f8737df1683c7c9a102ac",
                "sha256:baadc5f770917ada556afb7651a68176559f4dca5f4b2d0947cd15b9fb84fb51",
                "sha256:c725d11990a9243e6ceffe0ab25a07c46c1cc2c5dc55e305717b5afe856c9608",
                "sha256:d696a8c87315a83983fc59dd27efe034292b9e8ad667aeae51a68b4be14690d9",
                "sha256:e1864a4e9f93ddb2dc6b62ccc2ec1f8250ff4ac0d3d7a15c8985dd4e1fbd6418"
            ],
            "index": "pypi",
            "version": "==1.14.5"
        },
        "pandas": {
            "hashes": [
                "sha256:211cfdb9f72f26d2ede21c751d27e08fed4434d47fb9bb82ebc8ff753888b8b6",
                "sha256:28fd087514616549a0e3259cd68ac88d7eaed6bd3062017a7f312e27941266bd",
                "sha256:2fb7c63138bd5ead296b18b2cb6abd3a394f7581e5ae052b02b27df8244b03ca",
                "sha256:372435456c349a8d39ff001967b161f6bd29d4c3de145a4cf9b366648defbb1f",
                "sha256:3790a3348ab0f416e58061d21693cb662fbb2f638001b94bf2b2199fedc1b1c2",
                "sha256:437a6e906a6717a9ed2627cf6e7895b63dfaa0172567cbd75a553f55cf78cc17",
                "sha256:50b52af2af2e15f4aeb2fe196da073a8c131fa02e433e105d95ce40016df5690",
                "sha256:720daad75b5d35dd1b446842210c4f3fd447464c9c0884972f3f12b213a9edd1",
                "sha256:b4fb71acbc2709b8f5993cb4b5445d8182864f11c39787e317aae39f21206270",
                "sha256:b704fd73022342cce612996de495a16954311e0c0cf077c1b83d5cf0b9656a60",
                "sha256:cbbecca0c7af6a2160b2d6ba30becc286824a98c61dcc6a41fada664f226424c",
                "sha256:d2a071de755cc8ee7784e1b4c7b9b643d951d35c8adea7d64fe7c57cff9c47a7",
                "sha256:d8154c5c68713a82461aba735832f0b4692be8a45a0a340a303bf90d6f80f36f",
                "sha256:e1b86f7c55467ce1f6c12715f2fd1817f4a909b5c8c39bd4b5d2415ef2b04bd8",
                "sha256:fcc63e8134516e93e16eb4ceac9afaa51f4adc5bf58efddae7cbc562f5b77dd0"
            ],
            "index": "pypi",
            "version": "==0.23.1"
        },
        "python-dateutil": {
            "hashes": [
                "sha256:1adb80e7a782c12e52ef9a8182bebeb73f1d7e24e374397af06fb4956c8dc5c0",
                "sha256:e27001de32f627c22380a688bcc43ce83504a7bc5da472209b4c70f02829f0b8"
            ],
            "version": "==2.7.3"
        },
        "pytz": {
            "hashes": [
                "sha256:65ae0c8101309c45772196b21b74c46b2e5d11b6275c45d251b150d5da334555",
                "sha256:c06425302f2cf668f1bba7a0a03f3c1d34d4ebeef2c72003da308b3947c7f749"
            ],
            "version": "==2018.4"
        },
        "six": {
            "hashes": [
                "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
                "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
            ],
            "version": "==1.11.0"
        }
    },
    "develop": {}
}

```
</details>
The pipenv is the latest. To be sure I did run 
```
pip3 install pipenv --upgrade
pipenv clean
pipenv update
```
But the same issue remains.

Reviewing the diagnostic info I realize the Pipfiles differ. There is the extra line
```
datacache = {editable = true, file = "file:///app/data-cache"}
``` 
in the Pipfile in the docker, which is missing in the Pipfile on the host.
when the pipfile and lockfile are in sync let me know if you can rebuild the docker container!
also try removing the following line:

`"1bc1e69" = {path = "./data-cache", editable = true}`

From:
```
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
kafka = "*"
pandas = "*"
numpy = "*"
"1bc1e69" = {path = "./data-cache", editable = true}
datacache = {editable = true, file = "file:///app/data-cache"}

[dev-packages]

[requires]
python_version = "3.6"
```
I don't think you need them both, the one I'm asking to be removed is an older syntax and we've since changed how we resolve local packages.
Well, the line
```
"1bc1e69" = {path = "./data-cache", editable = true}
```
got created on the host and the line
```
datacache = {editable = true, file = "file:///app/data-cache"}
```
got added upon 
```
RUN set -ex && pipenv install --deploy --system
```
in the docker. So the Pipfiles are in sync before `pipenv install` is called in docker. To test, I omitted `pipenv install` line from the Dockerfile and the image got built. I attached to the container and ran 
```
pipenv install --deploy --system --verbose
```
interactively, while looking at the Pipfile before and after. 

Deleting the first line and keeping the second does not work, because on the host `"file:///app/data-cache"` does not resolve, so pipenv update fails with the following error:
```
Running $ pipenv lock then $ pipenv sync.
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
ne_round
    for dep in self._iter_dependencies(best_match):
  File "/usr/local/Cellar/pipenv/11.10.0/libexec/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 275, in _iter_dependencies
    for dependency in self.repository.get_dependencies(ireq):
  File "/usr/local/Cellar/pipenv/11.10.0/libexec/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 215, in get_dependencies
    legacy_results = self.get_legacy_dependencies(ireq)
  File "/usr/local/Cellar/pipenv/11.10.0/libexec/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 234, in get_legacy_dependencies
    dist = ireq.get_dist()
  File "/usr/local/Cellar/pipenv/11.10.0/libexec/lib/python3.6/site-packages/pipenv/vendor/pip9/req/req_install.py", line 1069, in get_dist
    egg_info = self.egg_info_path('').rstrip('/')
  File "/usr/local/Cellar/pipenv/11.10.0/libexec/lib/python3.6/site-packages/pipenv/vendor/pip9/req/req_install.py", line 483, in egg_info_path
    filenames = os.listdir(base)
FileNotFoundError: [Errno 2] No such file or directory: '/app/data-cache'
```


Sorry don't know what line of thought I was on.  I have thought of something else.  

```
FROM ubuntu:18.04

# -- Install Pipenv:
RUN apt update && apt install python3-pip git -y && pip3 install pipenv

ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8

# -- Install Application into container:
RUN set -ex && mkdir /app

COPY ./services /app
COPY ./data-cache /app/data-cache

WORKDIR /app

# -- Adding Pipfiles
COPY Pipfile Pipfile
COPY Pipfile.lock Pipfile.lock

# -- Install dependencies:
RUN set -ex && pipenv install && pipenv --deploy --system

CMD python3 start.py
```
Can you try that dockerfile with your original pipfile before i asked you to remove that line?  It should relock the pipfile after the install which will allow deploy to work.  I won't be able to create a small test for another hour or so, apologies for asking you to test so many things.
Cool, that works (after fixing the line which should read
```
RUN set -ex && pipenv install && pipenv install --deploy --system
```
, I guess) while 
```
RUN set -ex && pipenv install --deploy --system
```
fails. Could you give a short explanation?
Anyway, thank you very much for resolving the issue!
I don't think my solution is the proper way, I'm working on another, but essentially, when the pipfile references the local package, the lockfile becomes different for each platform, your host machine and the docker instance since they are in difference places.  `pipenv install` creates a virtualenv, and relocks the pipfile and then `pipenv install --deploy --system` uses that new updated lockfile for the install.  So we are essentially creating a virtualenv when it's not needed to sync lockfiles after you install the new local dep...I'm looking into a better Dockerfile solution now.
I am 100% confused about this, @erinxocon and I have been hashing it out all afternoon.  This is definitely not a good thing and we will sort out why it's happening.  Sorry for the issue, `--deploy` should never modify your pipfile!
Oh... I think I understand what happened... the new requirements library resolves the names of python packages.  It must have a bug in it that does two things

1, It resolves to absolute paths and converts to URIs
2. It ignores existing entries with other names

These are not good.

Resolving patch:
diff --git a/pipenv/__version__.py b/pipenv/__version__.py
--- a/pipenv/__version__.py
+++ b/pipenv/__version__.py
@@ -2,4 +2,4 @@
 #   //   ) ) / / //   ) ) //___) ) //   ) ) ||  / /
 #  //___/ / / / //___/ / //       //   / /  || / /
 # //       / / //       ((____   //   / /   ||/ /
-__version__ = '2018.6.25'
+__version__ = '2018.6.26.dev0'
diff --git a/pipenv/core.py b/pipenv/core.py
--- a/pipenv/core.py
+++ b/pipenv/core.py
@@ -617,7 +617,7 @@ def ensure_project(
     # Automatically use an activated virtualenv.
     if PIPENV_USE_SYSTEM:
         system = True
-    if not project.pipfile_exists:
+    if not project.pipfile_exists and not deploy:
         project.touch_pipfile()
     # Skip virtualenv creation when --system was used.
     if not system:
@@ -1268,7 +1268,8 @@ def do_init(
                 cleanup_virtualenv(bare=False)
                 sys.exit(1)
     # Ensure the Pipfile exists.
-    ensure_pipfile(system=system)
+    if not deploy:
+        ensure_pipfile(system=system)
     if not requirements_dir:
         cleanup_reqdir = True
         requirements_dir = TemporaryDirectory(
@@ -1915,7 +1916,8 @@ def do_install(
         package_name = False
     # Install editable local packages before locking - this gives us access to dist-info
     if project.pipfile_exists and (
-        not project.lockfile_exists or not project.virtualenv_exists
+        # double negatives are for english readability, leave them alone.
+        (not project.lockfile_exists and not deploy) or (not project.virtualenv_exists and not system)
     ):
         section = project.editable_packages if not dev else project.editable_dev_packages
         for package in section.keys():
@@ -2569,6 +2571,8 @@ def do_sync(
     unused=False,
     sequential=False,
     pypi_mirror=None,
+    system=False,
+    deploy=False,
 ):
     # The lock file needs to exist because sync won't write to it.
     if not project.lockfile_exists:
@@ -2581,8 +2585,8 @@ def do_sync(
         )
         sys.exit(1)
 
-    # Ensure that virtualenv is available.
-    ensure_project(three=three, python=python, validate=False)
+    # Ensure that virtualenv is available if not system.
+    ensure_project(three=three, python=python, validate=False, deploy=deploy)
 
     # Install everything.
     requirements_dir = TemporaryDirectory(
@@ -2595,6 +2599,8 @@ def do_sync(
         requirements_dir=requirements_dir,
         ignore_pipfile=True,    # Don't check if Pipfile and lock match.
         pypi_mirror=pypi_mirror,
+        deploy=deploy,
+        system=system,
     )
     requirements_dir.cleanup()
     click.echo(crayons.green('All dependencies are now up-to-date!'))
diff --git a/pipenv/patched/piptools/repositories/pypi.py b/pipenv/patched/piptools/repositories/pypi.py
--- a/pipenv/patched/piptools/repositories/pypi.py
+++ b/pipenv/patched/piptools/repositories/pypi.py
@@ -19,7 +19,6 @@
     PyPI,
     InstallRequirement,
     SafeFileCache,
-    InstallationError,
 )
 
 from pipenv.patched.notpip._vendor.packaging.requirements import InvalidRequirement, Requirement
@@ -27,6 +26,7 @@
 from pipenv.patched.notpip._vendor.packaging.specifiers import SpecifierSet, InvalidSpecifier, Specifier
 from pipenv.patched.notpip._vendor.packaging.markers import Marker, Op, Value, Variable
 from pipenv.patched.notpip._vendor.pyparsing import ParseException
+from pipenv.patched.notpip._internal.exceptions import InstallationError
 
 from ..cache import CACHE_DIR
 from pipenv.environments import PIPENV_CACHE_DIR
@@ -278,13 +278,16 @@ def get_legacy_dependencies(self, ireq):
             if ireq.editable:
                 try:
                     dist = ireq.get_dist()
+                except InstallationError:
+                    ireq.run_egg_info()
+                    dist = ireq.get_dist()
+                except (TypeError, ValueError, AttributeError):
+                    pass
+                else:
                     if dist.has_metadata('requires.txt'):
                         setup_requires = self.finder.get_extras_links(
                             dist.get_metadata_lines('requires.txt')
                         )
-                except (TypeError, ValueError, AttributeError):
-                    pass
-
             try:
                 # Pip < 9 and below
                 reqset = RequirementSet(


Test patch: