Repository: pypa/pipenv

Issue:
pipenv installs into python installation and not virtualenv
### Issue description

`pipenv install` installs packages like pytest into PYTHONHOME Location and not the virtualenv

### Expected result

pytest is installed into virtualenv and pipenv run pytest works.

### Actual result

`Installing collected packages: pytest 
The scripts py.test.exe and pytest.exe are installed in 'D:\Python36\Scripts' which is not on PATH.    Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pytest-4.0.0.`

`pipenv run pytest` results in command pytest not found

### Steps to replicate

<details><summary>$ pipenv --support</summary>

Pipenv version: `'2018.11.14'`

Pipenv location: `'d:\\python36\\lib\\site-packages\\pipenv'`

Python location: `'d:\\python36\\python.exe'`

Python installations found:

  - `3.6.7`: `D:\Python36\python.exe`
  - `3.6.6`: `D:\Python36-64\python.exe`

PEP 508 Information:

```
{'implementation_name': 'cpython',
 'implementation_version': '3.6.7',
 'os_name': 'nt',
 'platform_machine': 'AMD64',
 'platform_python_implementation': 'CPython',
 'platform_release': '7',
 'platform_system': 'Windows',
 'platform_version': '6.1.7601',
 'python_full_version': '3.6.7',
 'python_version': '3.6',
 'sys_platform': 'win32'}
```

System environment variables:

  - `ALLUSERSPROFILE`
  - `APPDATA`
  - `CGROAMINGPATH`
  - `CLIENTNAME`
  - `COMMONPROGRAMFILES`
  - `COMMONPROGRAMFILES(X86)`
  - `COMMONPROGRAMW6432`
  - `COMPUTERNAME`
  - `COMSPEC`
  - `DAS_HOME`
  - `DSPACE_RTT_TOOLS_2`
  - `FP_NO_HOST_CHECK`
  - `GOPATH`
  - `GOROOT`
  - `HOMEDRIVE`
  - `HOMEPATH`
  - `HTTPS_PROXY`
  - `LM_LICENSE_FILE`
  - `LOCALAPPDATA`
  - `LOGONSERVER`
  - `LSFORCEHOST`
  - `LSHOST`
  - `NUMBER_OF_PROCESSORS`
  - `OPENSSL_CONF`
  - `OS`
  - `PATH`
  - `PATHEXT`
  - `PLMVIS_82_LIBPATH`
  - `PROCESSOR_ARCHITECTURE`
  - `PROCESSOR_ARCHITEW6432`
  - `PROCESSOR_IDENTIFIER`
  - `PROCESSOR_LEVEL`
  - `PROCESSOR_REVISION`
  - `PROGRAMDATA`
  - `PROGRAMFILES`
  - `PROGRAMFILES(X86)`
  - `PROGRAMW6432`
  - `PROMPT`
  - `PSMODULEPATH`
  - `PUBLIC`
  - `RTARCH`
  - `RTHOME`
  - `SECSPECIALMODE`
  - `SESSIONNAME`
  - `SILVER_HOME`
  - `SIL_USER_TYPE`
  - `SYSTEMDRIVE`
  - `SYSTEMROOT`
  - `TEMP`
  - `TERM`
  - `TMP`
  - `TSK_OPTIONS_FILE_SW160800V6_2R1`
  - `UATDATA`
  - `USERDNSDOMAIN`
  - `USERDOMAIN`
  - `USERDOMAIN_ROAMINGPROFILE`
  - `USERNAME`
  - `USERPROFILE`
  - `VS100COMNTOOLS`
  - `VS120COMNTOOLS`
  - `VS140COMNTOOLS`
  - `VS90COMNTOOLS`
  - `WINDIR`
  - `WINDOWS_TRACING_FLAGS`
  - `WINDOWS_TRACING_LOGFILE`
  - `PIP_DISABLE_PIP_VERSION_CHECK`
  - `PYTHONDONTWRITEBYTECODE`
  - `PIP_SHIMS_BASE_MODULE`
  - `PIP_PYTHON_PATH`
  - `PYTHONFINDER_IGNORE_UNSUPPORTED`
  - `PYTHONHOME`

Pipenv specific environment variables:


Debug specific environment variables:

  - `PATH`: `C:\Program Files (x86)\Common Files\Oracle\Java\javapath;%PYTHONHOME%;%PYTHONHOME%\Scripts;D:\Sprachen\Perl\perl\bin\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\Microsoft Windows Performance Toolkit\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SDKs\TypeScript\1.0\;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;l`

</details>



Discussion:
Hey @m-stolle,

We did a quick check and this issue looks very darn similar to

* [#3260 - PipfileNotFound raised for no Pipfile while installing](3260)
* [#3259 - Update pythonfinder for resilient parsing](3259)
* [#3255 - Installation failed due to missing hashes when using --skip-lock](3255)

This could be a coincidence, but if any of these issues solves your problem then I did a good job :smile:

If not, the maintainers will get to this issue shortly.

Cheers,
Your Friendly Neighborhood ProBot

I believe this is fixed on the master branch — @frostming is this covered by your change to environment targets?
Also can you provide the text of the command you executed and the content of your `Pipfile`
Sure,

Pipefile
```
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pyparsing = "*"
pytest = "*"
pytest-mock = "*"
pytest-pythonpath = "*"
pylint = "*"

[dev-packages]

[requires]
python_version = "3.6"

```

Command:
`D:\Python36\scripts\pipenv.exe install -v --python D:\Python36\python.exe `

[pip_output.txt](https://github.com/pypa/pipenv/files/2600272/pip_output.txt)

I guess this line can explain the issue.

> `PYTHONHOME is set.  You *must* activate the virtualenv before using it`

@m-stolle pip will install site packages to PYTHONHOME. You should unset it and Python interpreter will    still be working well.
I compared the behavior with virtualenv.

@techalchemy When PYTHONHOME is set, using `$(VENV_PATH)/bin/pip install ...` won't do the trick. Maybe we should shade the PYTHONHOME temporarily inside virtualenv, shell, run, etc.?
I'm not totally certain whats best here, my initial instinct was yeah to just unset it the way we do with `PIP_USER` -- just check if `allow_global` or `system` is set, and if not, just unset `PYTHONHOME`? I might override it in `pipenv.environment.Environment.activated()` anyway...

Resolving patch:
diff --git a/pipenv/_compat.py b/pipenv/_compat.py
--- a/pipenv/_compat.py
+++ b/pipenv/_compat.py
@@ -13,31 +13,7 @@
 import sys
 import warnings
 import vistir
-from tempfile import _bin_openflags, gettempdir, _mkstemp_inner, mkdtemp
-
-try:
-    from tempfile import _infer_return_type
-except ImportError:
-
-    def _infer_return_type(*args):
-        _types = set()
-        for arg in args:
-            if isinstance(type(arg), six.string_types):
-                _types.add(str)
-            elif isinstance(type(arg), bytes):
-                _types.add(bytes)
-            elif arg:
-                _types.add(type(arg))
-        return _types.pop()
-
-
-if sys.version_info[:2] >= (3, 5):
-    try:
-        from pathlib import Path
-    except ImportError:
-        from .vendor.pathlib2 import Path
-else:
-    from .vendor.pathlib2 import Path
+from .vendor.vistir.compat import NamedTemporaryFile, Path, ResourceWarning, TemporaryDirectory
 
 # Backport required for earlier versions of Python.
 if sys.version_info < (3, 3):
@@ -45,257 +21,14 @@ def _infer_return_type(*args):
 else:
     from shutil import get_terminal_size
 
-try:
-    from weakref import finalize
-except ImportError:
-    try:
-        from .vendor.backports.weakref import finalize
-    except ImportError:
-
-        class finalize(object):
-            def __init__(self, *args, **kwargs):
-                from .utils import logging
-                logging.warn("weakref.finalize unavailable, not cleaning...")
-
-            def detach(self):
-                return False
-
-
-from vistir.compat import ResourceWarning
-
-
 warnings.filterwarnings("ignore", category=ResourceWarning)
 
 
-class TemporaryDirectory(object):
-
-    """
-    Create and return a temporary directory.  This has the same
-    behavior as mkdtemp but can be used as a context manager.  For
-    example:
-
-        with TemporaryDirectory() as tmpdir:
-            ...
-
-    Upon exiting the context, the directory and everything contained
-    in it are removed.
-    """
-
-    def __init__(self, suffix="", prefix="", dir=None):
-        if "RAM_DISK" in os.environ:
-            import uuid
-
-            name = uuid.uuid4().hex
-            dir_name = os.path.join(os.environ["RAM_DISK"].strip(), name)
-            os.mkdir(dir_name)
-            self.name = dir_name
-        else:
-            self.name = mkdtemp(suffix, prefix, dir)
-        self._finalizer = finalize(
-            self,
-            self._cleanup,
-            self.name,
-            warn_message="Implicitly cleaning up {!r}".format(self),
-        )
-
-    @classmethod
-    def _cleanup(cls, name, warn_message):
-        vistir.path.rmtree(name)
-        warnings.warn(warn_message, ResourceWarning)
-
-    def __repr__(self):
-        return "<{} {!r}>".format(self.__class__.__name__, self.name)
-
-    def __enter__(self):
-        return self
-
-    def __exit__(self, exc, value, tb):
-        self.cleanup()
-
-    def cleanup(self):
-        if self._finalizer.detach():
-            vistir.path.rmtree(self.name)
-
-
-def _sanitize_params(prefix, suffix, dir):
-    """Common parameter processing for most APIs in this module."""
-    output_type = _infer_return_type(prefix, suffix, dir)
-    if suffix is None:
-        suffix = output_type()
-    if prefix is None:
-        if output_type is str:
-            prefix = "tmp"
-        else:
-            prefix = os.fsencode("tmp")
-    if dir is None:
-        if output_type is str:
-            dir = gettempdir()
-        else:
-            dir = os.fsencode(gettempdir())
-    return prefix, suffix, dir, output_type
-
-
-class _TemporaryFileCloser:
-    """
-    A separate object allowing proper closing of a temporary file's
-    underlying file object, without adding a __del__ method to the
-    temporary file.
-    """
-
-    file = None  # Set here since __del__ checks it
-    close_called = False
-
-    def __init__(self, file, name, delete=True):
-        self.file = file
-        self.name = name
-        self.delete = delete
-
-    # NT provides delete-on-close as a primitive, so we don't need
-    # the wrapper to do anything special.  We still use it so that
-    # file.name is useful (i.e. not "(fdopen)") with NamedTemporaryFile.
-    if os.name != "nt":
-
-        # Cache the unlinker so we don't get spurious errors at
-        # shutdown when the module-level "os" is None'd out.  Note
-        # that this must be referenced as self.unlink, because the
-        # name TemporaryFileWrapper may also get None'd out before
-        # __del__ is called.
-
-        def close(self, unlink=os.unlink):
-            if not self.close_called and self.file is not None:
-                self.close_called = True
-                try:
-                    self.file.close()
-                finally:
-                    if self.delete:
-                        unlink(self.name)
-
-        # Need to ensure the file is deleted on __del__
-
-        def __del__(self):
-            self.close()
-
-    else:
-
-        def close(self):
-            if not self.close_called:
-                self.close_called = True
-                self.file.close()
-
-
-class _TemporaryFileWrapper:
-
-    """
-    Temporary file wrapper
-    This class provides a wrapper around files opened for
-    temporary use.  In particular, it seeks to automatically
-    remove the file when it is no longer needed.
-    """
-
-    def __init__(self, file, name, delete=True):
-        self.file = file
-        self.name = name
-        self.delete = delete
-        self._closer = _TemporaryFileCloser(file, name, delete)
-
-    def __getattr__(self, name):
-        # Attribute lookups are delegated to the underlying file
-        # and cached for non-numeric results
-        # (i.e. methods are cached, closed and friends are not)
-        file = self.__dict__["file"]
-        a = getattr(file, name)
-        if hasattr(a, "__call__"):
-            func = a
-
-            @functools.wraps(func)
-            def func_wrapper(*args, **kwargs):
-                return func(*args, **kwargs)
-
-            # Avoid closing the file as long as the wrapper is alive,
-            # see issue #18879.
-            func_wrapper._closer = self._closer
-            a = func_wrapper
-        if not isinstance(a, int):
-            setattr(self, name, a)
-        return a
-
-    # The underlying __enter__ method returns the wrong object
-    # (self.file) so override it to return the wrapper
-
-    def __enter__(self):
-        self.file.__enter__()
-        return self
-
-    # Need to trap __exit__ as well to ensure the file gets
-    # deleted when used in a with statement
-
-    def __exit__(self, exc, value, tb):
-        result = self.file.__exit__(exc, value, tb)
-        self.close()
-        return result
-
-    def close(self):
-        """
-        Close the temporary file, possibly deleting it.
-        """
-        self._closer.close()
-
-    # iter() doesn't use __getattr__ to find the __iter__ method
-
-    def __iter__(self):
-        # Don't return iter(self.file), but yield from it to avoid closing
-        # file as long as it's being used as iterator (see issue #23700).  We
-        # can't use 'yield from' here because iter(file) returns the file
-        # object itself, which has a close method, and thus the file would get
-        # closed when the generator is finalized, due to PEP380 semantics.
-        for line in self.file:
-            yield line
-
-
-def NamedTemporaryFile(
-    mode="w+b",
-    buffering=-1,
-    encoding=None,
-    newline=None,
-    suffix=None,
-    prefix=None,
-    dir=None,
-    delete=True,
-):
-    """
-    Create and return a temporary file.
-    Arguments:
-    'prefix', 'suffix', 'dir' -- as for mkstemp.
-    'mode' -- the mode argument to io.open (default "w+b").
-    'buffering' -- the buffer size argument to io.open (default -1).
-    'encoding' -- the encoding argument to io.open (default None)
-    'newline' -- the newline argument to io.open (default None)
-    'delete' -- whether the file is deleted on close (default True).
-    The file is created as mkstemp() would do it.
-    Returns an object with a file-like interface; the name of the file
-    is accessible as its 'name' attribute.  The file will be automatically
-    deleted when it is closed unless the 'delete' argument is set to False.
-    """
-    prefix, suffix, dir, output_type = _sanitize_params(prefix, suffix, dir)
-    flags = _bin_openflags
-    # Setting O_TEMPORARY in the flags causes the OS to delete
-    # the file when it is closed.  This is only supported by Windows.
-    if os.name == "nt" and delete:
-        flags |= os.O_TEMPORARY
-    if sys.version_info < (3, 5):
-        (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags)
-    else:
-        (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
-    try:
-        file = io.open(
-            fd, mode, buffering=buffering, newline=newline, encoding=encoding
-        )
-        return _TemporaryFileWrapper(file, name, delete)
-
-    except BaseException:
-        os.unlink(name)
-        os.close(fd)
-        raise
+__all__ = [
+    "NamedTemporaryFile", "Path", "ResourceWarning", "TemporaryDirectory",
+    "get_terminal_size", "getpreferredencoding", "DEFAULT_ENCODING", "force_encoding",
+    "UNICODE_TO_ASCII_TRANSLATION_MAP", "decode_output", "fix_utf8"
+]
 
 
 def getpreferredencoding():
@@ -366,8 +99,8 @@ def force_encoding():
 UNICODE_TO_ASCII_TRANSLATION_MAP = {
     8230: u"...",
     8211: u"-",
-    10004: u"x",
-    10008: u"Ok"
+    10004: u"OK",
+    10008: u"x",
 }
 
 
@@ -384,13 +117,11 @@ def decode_output(output):
             output = output.translate(UNICODE_TO_ASCII_TRANSLATION_MAP)
         output = output.encode(DEFAULT_ENCODING, "replace")
     return vistir.misc.to_text(output, encoding=DEFAULT_ENCODING, errors="replace")
-    return output
 
 
 def fix_utf8(text):
     if not isinstance(text, six.string_types):
         return text
-    from ._compat import decode_output
     try:
         text = decode_output(text)
     except UnicodeDecodeError:
diff --git a/pipenv/core.py b/pipenv/core.py
--- a/pipenv/core.py
+++ b/pipenv/core.py
@@ -732,6 +732,8 @@ def batch_install(deps_list, procs, failed_deps_queue,
         with vistir.contextmanagers.temp_environ():
             if not allow_global:
                 os.environ["PIP_USER"] = vistir.compat.fs_str("0")
+                if "PYTHONHOME" in os.environ:
+                    del os.environ["PYTHONHOME"]
             c = pip_install(
                 dep,
                 ignore_hashes=any([ignore_hashes, dep.editable, dep.is_vcs]),
@@ -1914,6 +1916,8 @@ def do_install(
             with vistir.contextmanagers.temp_environ(), create_spinner("Installing...") as sp:
                 if not system:
                     os.environ["PIP_USER"] = vistir.compat.fs_str("0")
+                    if "PYTHONHOME" in os.environ:
+                        del os.environ["PYTHONHOME"]
                 try:
                     pkg_requirement = Requirement.from_line(pkg_line)
                 except ValueError as e:


Test patch:
diff --git a/tests/integration/test_pipenv.py b/tests/integration/test_pipenv.py
--- a/tests/integration/test_pipenv.py
+++ b/tests/integration/test_pipenv.py
@@ -98,7 +98,7 @@ def mocked_mkdtemp(suffix, prefix, dir):
             prefix = '-dir-with-leading-dash'
         return mkdtemp(suffix, prefix, dir)
 
-    with mock.patch('pipenv._compat.mkdtemp', side_effect=mocked_mkdtemp):
+    with mock.patch('pipenv.vendor.vistir.compat.mkdtemp', side_effect=mocked_mkdtemp):
         with temp_environ(), PipenvInstance(chdir=True) as p:
             del os.environ['PIPENV_VENV_IN_PROJECT']
             p.pipenv('--python python')
diff --git a/tests/pytest-pypi/pytest_pypi/app.py b/tests/pytest-pypi/pytest_pypi/app.py
--- a/tests/pytest-pypi/pytest_pypi/app.py
+++ b/tests/pytest-pypi/pytest_pypi/app.py
@@ -4,6 +4,8 @@
 
 import requests
 from flask import Flask, redirect, abort, render_template, send_file, jsonify
+from zipfile import is_zipfile
+from tarfile import is_tarfile
 
 app = Flask(__name__)
 session = requests.Session()
@@ -51,7 +53,7 @@ def __init__(self, name):
     def __repr__(self):
         return "<Artifact name={0!r} files={1!r}".format(self.name, len(self.files))
 
-    def add_dir(self, path):
+    def add_file(self, path):
         path = os.path.abspath(path)
         base_path, fn = os.path.split(path)
         self.files[fn] = path
@@ -63,11 +65,20 @@ def prepare_fixtures(path):
     if not (os.path.exists(path) and os.path.isdir(path)):
         raise ValueError("{} is not a directory!".format(path))
     for root, dirs, files in os.walk(path):
+        package_name, _, _ = os.path.relpath(root, start=path).partition(os.path.sep)
+        if package_name not in ARTIFACTS:
+            ARTIFACTS[package_name] = Artifact(package_name)
         for file in files:
-            package_name = os.path.basename(root)
-            if package_name not in ARTIFACTS:
-                ARTIFACTS[package_name] = Artifact(package_name)
-            ARTIFACTS[package_name].add_dir(os.path.join(root, file))
+            file_path = os.path.join(root, file)
+            rel_path = os.path.relpath(file_path, start=path)
+            _, _, subpkg = rel_path.partition(os.path.sep)
+            subpkg, _, _ = subpkg.partition(os.path.sep)
+            pkg, ext = os.path.splitext(subpkg)
+            if not (is_tarfile(file_path) or is_zipfile(file_path) or ext == ".git"):
+                continue
+            if subpkg not in ARTIFACTS[package_name].files:
+                ARTIFACTS[package_name].add_file(os.path.join(root, file))
+            ARTIFACTS[package_name].add_file(os.path.join(root, file))
 
 
 def prepare_packages(path):