Repository: pypa/pipenv

Issue:
`pipenv install` fails on v11.10.1.dev[1-4] in docker
If there's no or an outdated Pipfile.lock  `pipenv install --system` fails during docker build with a TypeError.

<details><summary>$ python -m pipenv.help output</summary>

Pipenv version: `'11.10.1.dev4'`

Pipenv location: `'/usr/local/lib/python3.6/site-packages/pipenv'`

Python location: `'/usr/local/bin/python'`

Other Python installations in `PATH`:

  - `2.7`: `/usr/bin/python2.7`
  - `2.7`: `/usr/bin/python2.7`
  - `3.5`: `/usr/bin/python3.5m`
  - `3.5`: `/usr/bin/python3.5`
  - `3.6`: `/usr/local/bin/python3.6m`
  - `3.6`: `/usr/local/bin/python3.6`
  - `3.6`: `/usr/local/bin/python3.6`

  - `3.6.5`: `/usr/local/bin/python`
  - `3.6.5`: `/usr/local/bin/python`
  - `2.7.13`: `/usr/bin/python`
  - `2.7.13`: `/usr/bin/python2`
  - `3.6.5`: `/usr/local/bin/python3`
  - `3.6.5`: `/usr/local/bin/python3`
  - `3.5.3`: `/usr/bin/python3`

PEP 508 Information:

```
{'implementation_name': 'cpython',
 'implementation_version': '3.6.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.15.0-2-amd64',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Debian 4.15.11-1 (2018-03-20)',
 'python_full_version': '3.6.5',
 'python_version': '3.6',
 'sys_platform': 'linux'}
```

System environment variables:

  - `HOSTNAME`
  - `PYTHON_PIP_VERSION`
  - `HOME`
  - `GPG_KEY`
  - `PIPENV_VERSION`
  - `PATH`
  - `LANG`
  - `PYTHON_VERSION`
  - `PWD`
  - `PYTHONDONTWRITEBYTECODE`
  - `PIP_PYTHON_PATH`

Pipenv–specific environment variables:

 - `PIPENV_VERSION`: `v11.10.1.dev4`

Debug–specific environment variables:

  - `PATH`: `/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin`
  - `LANG`: `C.UTF-8`
  - `PWD`: `/code`


---------------------------

Contents of `Pipfile` ('/code/Pipfile'):

```toml
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
requests = "*"

[dev-packages]

[requires]
python_version = "3.6"

```

</details>

------------

##### Expected result

`pipenv install --system` installs packages.

##### Actual result

`pipenv install --system` fails with a TypeError:

```
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/cli.py", line 402, in install
    selective_upgrade=selective_upgrade,
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 1902, in do_install
    requirements_dir=requirements_directory,
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 1340, in do_init
    verbose=verbose,
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 1038, in do_lock
    allow_global=system,
  File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 408, in venv_resolve_deps
    escape_grouped_arguments(which('python')),
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 126, in which
    p = os.sep.join([location] + ['bin/{0}'.format(command)])
TypeError: sequence item 0: expected str instance, NoneType found
```

Same happens if a `Pipfile.lock` exists:

```
Pipfile.lock (b6f97a) out of date, updating to (8e0064)…
Locking [dev-packages] dependencies…
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/cli.py", line 402, in install
    selective_upgrade=selective_upgrade,
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 1902, in do_install
    requirements_dir=requirements_directory,
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 1329, in do_init
    do_lock(system=system, pre=pre, keep_outdated=keep_outdated)
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 1038, in do_lock
    allow_global=system,
  File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 408, in venv_resolve_deps
    escape_grouped_arguments(which('python')),
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 126, in which
    p = os.sep.join([location] + ['bin/{0}'.format(command)])
TypeError: sequence item 0: expected str instance, NoneType found
```

##### Steps to replicate

Create a Dockerfile and a Pipfile in a new directory with the following content:

```Dockerfile
FROM python:3.6-stretch
ARG PIPENV_VERSION
RUN mkdir -p /code

RUN pip install "pipenv==${PIPENV_VERSION}"
# RUN pip install "git+https://github.com/pypa/pipenv.git@master#egg=pipenv"

WORKDIR /code
COPY Pipfile* /code/
RUN python -m pipenv.help && ls -l && pipenv install --system --verbose
```

```toml
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"


[packages]
requests = "*"


[dev-packages]
pytest = "*"


[requires]
python_version = "3.6"
```

Run `docker build --build-arg PIPENV_VERSION="v11.10.1.dev4" .`  (Same for dev1, dev2, and dev3).
To try it with the latest commit in pypa/pipenv@master, use the commented out line in the Dockerfile instead.

To try it with a `Pipfile.lock` you can use the one from the [documentation](https://docs.pipenv.org/basics/#example-pipfile-lock).


Discussion:
This looks like a variant to the old `/bin/python` not found thing, just a tiny bit more explicit. The root cause is the failure to locate `python`.
WIth `v11.10.0`:

```
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…

/bin/sh: 1: /bin/python: not found
```

If I add `RUN ln -s /usr/local/bin/python /bin/python` after `COPY`, it works with `v11.10.0`, but not with `v11.10.1.dev4` (same TypeError).

Resolving patch:
diff --git a/pipenv/__version__.py b/pipenv/__version__.py
--- a/pipenv/__version__.py
+++ b/pipenv/__version__.py
@@ -2,4 +2,4 @@
 #   //   ) ) / / //   ) ) //___) ) //   ) ) ||  / /
 #  //___/ / / / //___/ / //       //   / /  || / /
 # //       / / //       ((____   //   / /   ||/ /
-__version__ = '11.10.1.dev4'
+__version__ = '11.10.1.dev5'
diff --git a/pipenv/core.py b/pipenv/core.py
--- a/pipenv/core.py
+++ b/pipenv/core.py
@@ -115,7 +115,7 @@ def spinner():
 
 
 def which(command, location=None, allow_global=False):
-    if location is None:
+    if not allow_global and location is None:
         location = project.virtualenv_location or os.environ.get('VIRTUAL_ENV')
     if not allow_global:
         if os.name == 'nt':
@@ -127,6 +127,11 @@ def which(command, location=None, allow_global=False):
     else:
         if command == 'python':
             p = sys.executable
+    if not os.path.exists(p):
+        if command == 'python':
+            p = sys.executable or system_which('python')
+        else:
+            p = system_which(command)
     return p
 
 
@@ -1316,6 +1321,20 @@ def do_init(
                 )
                 requirements_dir.cleanup()
                 sys.exit(1)
+            elif system or allow_global:
+                click.echo(
+                    crayons.red(
+                        u'Pipfile.lock ({0}) out of date, but installation '
+                        u'uses {1}... re-building lockfile must happen in '
+                        u'isolation. Please rebuild lockfile in a virtualenv. '
+                        u'Continuing anyway...'.format(
+                            crayons.white(old_hash[-6:]),
+                            crayons.white('--system')
+                        ),
+                        bold=True,
+                    ),
+                    err=True,
+                )
             else:
                 click.echo(
                     crayons.red(
@@ -1329,16 +1348,28 @@ def do_init(
                 do_lock(system=system, pre=pre, keep_outdated=keep_outdated)
     # Write out the lockfile if it doesn't exist.
     if not project.lockfile_exists and not skip_lock:
-        click.echo(
-            crayons.normal(u'Pipfile.lock not found, creating…', bold=True),
-            err=True,
-        )
-        do_lock(
-            system=system,
-            pre=pre,
-            keep_outdated=keep_outdated,
-            verbose=verbose,
-        )
+        if system or allow_global:
+            click.echo(
+                '{0}: --system is intended to be used for Pipfile installation, '
+                'not installation of specific packages. Aborting.'.format(
+                    crayons.red('Warning', bold=True)
+                ),
+                err=True,
+            )
+            click.echo('See also: --deploy flag.', err=True)
+            requirements_dir.cleanup()
+            sys.exit(1)
+        else:
+            click.echo(
+                crayons.normal(u'Pipfile.lock not found, creating…', bold=True),
+                err=True,
+            )
+            do_lock(
+                system=system,
+                pre=pre,
+                keep_outdated=keep_outdated,
+                verbose=verbose,
+            )
     do_install_dependencies(
         dev=dev,
         requirements=requirements,
diff --git a/pipenv/utils.py b/pipenv/utils.py
--- a/pipenv/utils.py
+++ b/pipenv/utils.py
@@ -405,7 +405,7 @@ def venv_resolve_deps(
 
     resolver = escape_grouped_arguments(resolver.__file__.rstrip('co'))
     cmd = '{0} {1} {2} {3} {4} {5}'.format(
-        escape_grouped_arguments(which('python')),
+        escape_grouped_arguments(which('python', allow_global=allow_global)),
         resolver,
         '--pre' if pre else '',
         '--verbose' if verbose else '',


Test patch: