Repository: pypa/pipenv

Issue:
Pipfile.lock out of date, but installation uses --system... re-building lockfile must happen in isolation.
`pipenv` complains about installation using --system flag and does not update lock file on subsequent package installs.

<details><summary>$ python -m pipenv.help output</summary>

Pipenv version: `'2018.05.18'`

Pipenv location: `'/Users/user/.virtualenvs/quicktest/lib/python3.6/site-packages/pipenv'`

Python location: `'/Users/user/.virtualenvs/quicktest/bin/python'`

Other Python installations in `PATH`:

  - `2.6`: `/opt/local/bin/python2.6`
  - `2.6`: `/opt/local/bin/python2.6`
  - `2.7`: `/opt/local/bin/python2.7`
  - `2.7`: `/opt/local/bin/python2.7`
  - `2.7`: `/usr/bin/python2.7`
  - `3.6`: `/opt/local/bin/python3.6m`
  - `3.6`: `/Users/user/.virtualenvs/quicktest/bin/python3.6`
  - `3.6`: `/opt/local/bin/python3.6`

  - `3.6.5`: `/Users/user/.virtualenvs/quicktest/bin/python`
  - `3.6.5`: `/opt/local/bin/python`
  - `2.7.10`: `/usr/bin/python`
  - `2.7.15`: `/opt/local/bin/python2`
  - `3.6.5`: `/Users/user/.virtualenvs/quicktest/bin/python3`
  - `3.6.5`: `/opt/local/bin/python3`

PEP 508 Information:

```
{'implementation_name': 'cpython',
 'implementation_version': '3.6.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.5.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.5.0: Fri Apr 13 19:32:32 PDT '
                     '2018; root:xnu-4570.51.2~1/RELEASE_X86_64',
 'python_full_version': '3.6.5',
 'python_version': '3.6',
 'sys_platform': 'darwin'}
```

System environment variables:

  - `TERM_SESSION_ID`
  - `SSH_AUTH_SOCK`
  - `COLORFGBG`
  - `ITERM_PROFILE`
  - `XPC_FLAGS`
  - `LANG`
  - `PWD`
  - `SHELL`
  - `TERM_PROGRAM_VERSION`
  - `TERM_PROGRAM`
  - `PATH`
  - `DISPLAY`
  - `COLORTERM`
  - `COMMAND_MODE`
  - `TERM`
  - `HOME`
  - `TMPDIR`
  - `USER`
  - `XPC_SERVICE_NAME`
  - `LOGNAME`
  - `SHLVL`
  - `OLDPWD`
  - `HISTSIZE`
  - `HISTFILE`
  - `SAVEHIST`
  - `WORDCHARS`
  - `EDITOR`
  - `DJROOT`
  - `PYROOT`
  - `PYLROOT`
  - `JAVA_HOME`
  - `FPATH`
  - `VIRTUALENV_PYTHON`
  - `VIRTUALENVWRAPPER_PROJECT_FILENAME`
  - `VIRTUALENVWRAPPER_WORKON_CD`
  - `VIRTUALENVWRAPPER_SCRIPT`
  - `WORKON_HOME`
  - `VIRTUALENVWRAPPER_HOOK_DIR`
  - `DOCBOOK_XSL_URL`
  - `XML_CATALOG_FILES`
  - `ES_HOME`
  - `GOPATH`
  - `NVM_DIR`
  - `PS1`
  - `VIRTUAL_ENV`
  - `_`
  - `PYTHONDONTWRITEBYTECODE`
  - `PIP_PYTHON_PATH`

Pipenv–specific environment variables:


Debug–specific environment variables:

  - `PATH`: `/Users/user/.virtualenvs/quicktest/bin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin`
  - `SHELL`: `/opt/local/bin/zsh`
  - `LANG`: `ru_RU.UTF-8`
  - `PWD`: `/tmp`
  - `VIRTUAL_ENV`: `/Users/user/.virtualenvs/quicktest`


---------------------------

Contents of `Pipfile` ('/private/tmp/Pipfile'):

```toml
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pygments = "*"
fabric = "*"

[dev-packages]

[requires]
python_version = "3.6"

```


Contents of `Pipfile.lock` ('/private/tmp/Pipfile.lock'):

```json
{
    "_meta": {
        "hash": {
            "sha256": "4fef9e909141947fb573b0345f2129099de0f89c3f36a5b8164a15fcc0fea9e8"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "pygments": {
            "hashes": [
                "sha256:78f3f434bcc5d6ee09020f92ba487f95ba50f1e3ef83ae96b9d5ffa1bab25c5d",
                "sha256:dbae1046def0efb574852fab9e90209b23f556367b5a320c0bcb871c77c3e8cc"
            ],
            "index": "pypi",
            "version": "==2.2.0"
        }
    },
    "develop": {}
}

```
</details>

##### Steps to replicate

```
% mkvirtualenv quicktest
Running virtualenv with interpreter /opt/local/bin/python3.6
Using base prefix '/opt/local/Library/Frameworks/Python.framework/Versions/3.6'
New python executable in /Users/user/.virtualenvs/quicktest/bin/python3.6
Also creating executable in /Users/user/.virtualenvs/quicktest/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /Users/user/.virtualenvs/quicktest/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/user/.virtualenvs/quicktest/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/user/.virtualenvs/quicktest/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/user/.virtualenvs/quicktest/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/user/.virtualenvs/quicktest/bin/get_env_details
Requirement already up-to-date: pip in /Users/user/.virtualenvs/quicktest/lib/python3.6/site-packages (10.0.1)
Collecting pipenv
  Using cached https://files.pythonhosted.org/packages/8d/fe/4faa519acfb984015dde4e2973d89e47baf6a9cd81e9d58e2f2d3c47fb6f/pipenv-2018.5.18-py3-none-any.whl
Collecting virtualenv-clone>=0.2.5 (from pipenv)
  Using cached https://files.pythonhosted.org/packages/6d/c2/dccb5ccf599e0c5d1eea6acbd058af7a71384f9740179db67a9182a24798/virtualenv_clone-0.3.0-py2.py3-none-any.whl
Collecting certifi (from pipenv)
  Using cached https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl
Collecting virtualenv (from pipenv)
  Using cached https://files.pythonhosted.org/packages/b6/30/96a02b2287098b23b875bc8c2f58071c35d2efe84f747b64d523721dc2b5/virtualenv-16.0.0-py2.py3-none-any.whl
Requirement not upgraded as not directly required: setuptools>=36.2.1 in /Users/user/.virtualenvs/quicktest/lib/python3.6/site-packages (from pipenv) (39.2.0)
Installing collected packages: virtualenv-clone, certifi, virtualenv, pipenv
Successfully installed certifi-2018.4.16 pipenv-2018.5.18 virtualenv-16.0.0 virtualenv-clone-0.3.0
```

The first installed package is locked just fine:

```
(quicktest) user@otg /tmp!% hash -r

(quicktest) user@otg /tmp % pipenv install pygments
Creating a Pipfile for this project…
Installing pygments…
Collecting pygments
  Using cached https://files.pythonhosted.org/packages/02/ee/b6e02dc6529e82b75bb06823ff7d005b141037cb1416b10c6f00fc419dca/Pygments-2.2.0-py2.py3-none-any.whl
Installing collected packages: pygments
Successfully installed pygments-2.2.0

Adding pygments to Pipfile's [packages]…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (fea9e8)!
Installing dependencies from Pipfile.lock (fea9e8)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:00
```

Subsequent package installs do complain about `--system` flag:

```
(quicktest) user@otg /tmp % pipenv install fabric
Installing fabric…
Collecting fabric
  Using cached https://files.pythonhosted.org/packages/ab/66/c4f9a1f6e50fbf783aab49eef620cb2ec5a8a1eb6cf834347beab4d381a3/fabric-2.1.3-py2.py3-none-any.whl
Collecting invoke<2.0,>=1.0 (from fabric)
  Using cached https://files.pythonhosted.org/packages/6c/66/9e232c59e61f0a0b6552d68419a5c5a5dba368e105fdbfd2b6c74c402234/invoke-1.0.0-py3-none-any.whl
Collecting cryptography>=1.1 (from fabric)
  Using cached https://files.pythonhosted.org/packages/40/87/acdcf84ce6d25a7db1c113f4b9b614fd8d707b7ab56fbf17cf18cd26a627/cryptography-2.2.2-cp34-abi3-macosx_10_6_intel.whl
Collecting paramiko>=2.4 (from fabric)
  Using cached https://files.pythonhosted.org/packages/3e/db/cb7b6656e0e7387637ce850689084dc0b94b44df31cc52e5fc5c2c4fd2c1/paramiko-2.4.1-py2.py3-none-any.whl
Collecting cffi>=1.7; platform_python_implementation != "PyPy" (from cryptography>=1.1->fabric)
  Using cached https://files.pythonhosted.org/packages/8e/be/40b1bc2c3221acdefeb9dab6773d43cda7543ed0d8c8df8768f05af2d01e/cffi-1.11.5-cp36-cp36m-macosx_10_6_intel.whl
Collecting idna>=2.1 (from cryptography>=1.1->fabric)
  Using cached https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl
Collecting six>=1.4.1 (from cryptography>=1.1->fabric)
  Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting asn1crypto>=0.21.0 (from cryptography>=1.1->fabric)
  Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting pynacl>=1.0.1 (from paramiko>=2.4->fabric)
  Using cached https://files.pythonhosted.org/packages/74/8e/a6c0d340972d9e2f1a405aaa3f2460950b4c0337f92db0291a4355974529/PyNaCl-1.2.1-cp36-cp36m-macosx_10_6_intel.whl
Collecting bcrypt>=3.1.3 (from paramiko>=2.4->fabric)
  Using cached https://files.pythonhosted.org/packages/7e/59/d48fd712941da1a5d6490964a37bb3de2e526965b6766273f6a7049ee590/bcrypt-3.1.4-cp36-cp36m-macosx_10_6_intel.whl
Collecting pyasn1>=0.1.7 (from paramiko>=2.4->fabric)
  Using cached https://files.pythonhosted.org/packages/a0/70/2c27740f08e477499ce19eefe05dbcae6f19fdc49e9e82ce4768be0643b9/pyasn1-0.4.3-py2.py3-none-any.whl
Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.1->fabric)
Installing collected packages: invoke, pycparser, cffi, idna, six, asn1crypto, cryptography, pynacl, bcrypt, pyasn1, paramiko, fabric
Successfully installed asn1crypto-0.24.0 bcrypt-3.1.4 cffi-1.11.5 cryptography-2.2.2 fabric-2.1.3 idna-2.6 invoke-1.0.0 paramiko-2.4.1 pyasn1-0.4.3 pycparser-2.18 pynacl-1.2.1 six-1.11.0

Adding fabric to Pipfile's [packages]…
Pipfile.lock (fea9e8) out of date, but installation uses --system... re-building lockfile must happen in isolation. Please rebuild lockfile in a virtualenv. Continuing anyway...
Installing dependencies from Pipfile.lock (fea9e8)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:00
```



Discussion:
No idea if this is related, but I'm getting a version mismatch error when I run `pipenv lock && pipenv install --system --deploy --ignore-pipfile`. I'm running the command this way because I noticed the error when I would `pipenv lock` during development, but the install command would fail in CI, so I juxtaposed them to be sure. Strangely, this error isn't consistent for the same exact source code; sometimes it fails and other times it succeeds, but the hash returned from the `pipenv lock` is always the same. I've also run `cksum` on the lockfile to make sure it's the exact same in all cases.

EDIT: I noticed in the trials that are succeeding, our pipenv is changing its version from 2018.05.18 to 11.10.1 (because someone pinned our pipenv version in the Pipfile to 11.10.1--not sure how it got installed since all of the previous `pipenv install` commands were failing). In short, both versions of pipenv build the same lock file but 2018.05.18 fails to install from that lock file, expecting a different version.
@techalchemy I installed the latest master branch, but the issue is still present.
Ok thanks for the update, guess this one wasn’t as simple. Still need to capture these with test coverage
@max-arnold are you still experiencing this? I can't reproduce this.
Yes

```
user@otg /tmp % which python3.6  # installed via macports

/opt/local/bin/python3.6

user@otg /tmp % mkvirtualenv -p python3.6 quicktest

Running virtualenv with interpreter /opt/local/bin/python3.6
Using base prefix '/opt/local/Library/Frameworks/Python.framework/Versions/3.6'
...
virtualenvwrapper.user_scripts creating /Users/user/.virtualenvs/quicktest/bin/get_env_details


(quicktest) user@otg /tmp % pip install git+https://github.com/pypa/pipenv.git@e326ce6e071c8b93f9b451cbf912284811952122

Collecting git+https://github.com/pypa/pipenv.git@e326ce6e071c8b93f9b451cbf912284811952122
  Cloning https://github.com/pypa/pipenv.git (to revision e326ce6e071c8b93f9b451cbf912284811952122) to /private/var/folders/fh/j4kp2bt97gxb89plh9h4gxx80000gn/T/pip-req-build-mzqcvs0e
...
Successfully installed certifi-2018.4.16 pipenv-2018.5.18 virtualenv-16.0.0 virtualenv-clone-0.3.0


(quicktest) user@otg /tmp % hash -r

(quicktest) user@otg /tmp % pipenv install pygments

Creating a Pipfile for this project…
Installing pygments…
...
Updated Pipfile.lock (fea9e8)!
Installing dependencies from Pipfile.lock (fea9e8)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:00

(quicktest) user@otg /tmp % pipenv install requests

Installing requests…
...
Successfully installed chardet-3.0.4 idna-2.7 requests-2.19.1 urllib3-1.23

Adding requests to Pipfile's [packages]…
Pipfile.lock (fea9e8) out of date, but installation uses --system... re-building lockfile must happen in isolation. Please rebuild lockfile in a virtualenv. Continuing anyway...
Installing dependencies from Pipfile.lock (fea9e8)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:00

(quicktest) user@otg /tmp % pipenv --version

pipenv, version 2018.05.18

(quicktest) user@otg /tmp % grep requests Pipfile.lock

(quicktest) user@otg /tmp!% grep requests Pipfile

requests = "*"

```

Resolving patch:
diff --git a/pipenv/core.py b/pipenv/core.py
--- a/pipenv/core.py
+++ b/pipenv/core.py
@@ -1261,6 +1261,7 @@ def do_init(
     pypi_mirror=None,
 ):
     """Executes the init functionality."""
+    global PIPENV_VIRTUALENV
     if not system:
         if not project.virtualenv_exists:
             try:
@@ -1292,7 +1293,7 @@ def do_init(
                 )
                 requirements_dir.cleanup()
                 sys.exit(1)
-            elif system or allow_global:
+            elif (system or allow_global) and not (PIPENV_VIRTUALENV):
                 click.echo(
                     crayons.red(
                         u'Pipfile.lock ({0}) out of date, but installation '
@@ -1321,7 +1322,7 @@ def do_init(
     if not project.lockfile_exists and not skip_lock:
         # Unless we're in a virtualenv not managed by pipenv, abort if we're
         # using the system's python.
-        if (system or allow_global) and not PIPENV_VIRTUALENV:
+        if (system or allow_global) and not (PIPENV_VIRTUALENV):
             click.echo(
                 '{0}: --system is intended to be used for Pipfile installation, '
                 'not installation of specific packages. Aborting.'.format(
@@ -1793,6 +1794,7 @@ def do_install(
         keep_outdated = project.settings.get('keep_outdated')
     remote = requirements and is_valid_url(requirements)
     # Warn and exit if --system is used without a pipfile.
+    global PIPENV_VIRTUALENV
     if system and package_name and not PIPENV_VIRTUALENV:
         click.echo(
             '{0}: --system is intended to be used for Pipfile installation, '


Test patch: