Repository: pypa/pipenv

Issue:
Pipenv cannot create virtualenv
When I try to create a new env using `pipenv --two` or `pipenv --three`,  pipenv enters an infinite loop repeatedly printing 

```
Virtualenv already exists!
Removing existing virtualenv…
```

If I try `pipenv install`, pipenv tries to install the packages to `/usr/lib/python3.6` and the output of `pipenv --venv` is empty.

##### Describe you environment

1. OS Type: Archlinux
1. Python version: Python 3.6.2
1. Pipenv version:  pipenv, version 8.2.7


Discussion:
sounds like a path issue.  Can you echo out your PATH variable and paste it here?
`/home/jck/.local/bin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl`

pipenv is in `/usr/bin`
@jck where is your Pipfile located or if it doesn't exist what directory are you running the command in? Is there a Pipfile or requirements.txt anywhere above this directory?
@nateprewitt That was my next question.  @kennethreitz and I ran into a problem with this on windows and it was because there was a Pipfile somewhere higher in my file tree.
I've tried running the command in a folder with a `requirements.txt` and another with a `Pipfile`. There are no Pipfile or requirements.txt anywhere above this directory.
Does it happen no matter what directory you are in?  I can't replicate this short of installing Arch and trying.
@jck does pipenv --where return anything or is it also blank?
It happens in any directory. `pipenv --where` correctly shows the path of the folder.
can you do a `--verbose` when you try and `pipenv --two/--three`?
The `--verbose` option does not seem to work with `--two` or `--three`

`Error: no such option: --verbose`
what about `pipenv install --two --verbose`? 
No extra information, just an infinite loop of 
```
Virtualenv already exists!
Removing existing virtualenv…
```
Can you provide the full output of `env` from your shell?
I just retried everything here on my personal machine(also Arch) which did not have pipenv installed previously. And encountered all the same issues. I think it might be an arch specific issue.

This is the output of env:
```
LS_COLORS=no=00:fi=00:di=34:ow=34;40:ln=35:pi=30;44:so=35;44:do=35;44:bd=33;44:cd=37;44:or=05;37;41:mi=05;37;41:ex=01;31:*.cmd=01;31:*.exe=01;31:*.com=01;31:*.bat=01;31:*.reg=01;31:*.app=01;31:*.txt=32:*.org=32:*.md=32:*.mkd=32:*.h=32:*.c=32:*.C=32:*.cc=32:*.cpp=32:*.cxx=32:*.objc=32:*.sh=32:*.csh=32:*.zsh=32:*.el=32:*.vim=32:*.java=32:*.pl=32:*.pm=32:*.py=32:*.rb=32:*.hs=32:*.php=32:*.htm=32:*.html=32:*.shtml=32:*.erb=32:*.haml=32:*.xml=32:*.rdf=32:*.css=32:*.sass=32:*.scss=32:*.less=32:*.js=32:*.coffee=32:*.man=32:*.0=32:*.1=32:*.2=32:*.3=32:*.4=32:*.5=32:*.6=32:*.7=32:*.8=32:*.9=32:*.l=32:*.n=32:*.p=32:*.pod=32:*.tex=32:*.bmp=33:*.cgm=33:*.dl=33:*.dvi=33:*.emf=33:*.eps=33:*.gif=33:*.jpeg=33:*.jpg=33:*.JPG=33:*.mng=33:*.pbm=33:*.pcx=33:*.pdf=33:*.pgm=33:*.png=33:*.PNG=33:*.ppm=33:*.pps=33:*.ppsx=33:*.ps=33:*.svg=33:*.svgz=33:*.tga=33:*.tif=33:*.tiff=33:*.xbm=33:*.xcf=33:*.xpm=33:*.xwd=33:*.xwd=33:*.yuv=33:*.aac=33:*.au=33:*.flac=33:*.mid=33:*.midi=33:*.mka=33:*.mp3=33:*.mpa=33:*.mpeg=33:*.mpg=33:*.ogg=33:*.ra=33:*.wav=33:*.anx=33:*.asf=33:*.avi=33:*.axv=33:*.flc=33:*.fli=33:*.flv=33:*.gl=33:*.m2v=33:*.m4v=33:*.mkv=33:*.mov=33:*.MOV=33:*.mp4=33:*.mp4v=33:*.mpeg=33:*.mpg=33:*.nuv=33:*.ogm=33:*.ogv=33:*.ogx=33:*.qt=33:*.rm=33:*.rmvb=33:*.swf=33:*.vob=33:*.webm=33:*.wmv=33:*.doc=31:*.docx=31:*.rtf=31:*.dot=31:*.dotx=31:*.xls=31:*.xlsx=31:*.ppt=31:*.pptx=31:*.fla=31:*.psd=31:*.7z=1;35:*.apk=1;35:*.arj=1;35:*.bin=1;35:*.bz=1;35:*.bz2=1;35:*.cab=1;35:*.deb=1;35:*.dmg=1;35:*.gem=1;35:*.gz=1;35:*.iso=1;35:*.jar=1;35:*.msi=1;35:*.rar=1;35:*.rpm=1;35:*.tar=1;35:*.tbz=1;35:*.tbz2=1;35:*.tgz=1;35:*.tx=1;35:*.war=1;35:*.xpi=1;35:*.xz=1;35:*.z=1;35:*.Z=1;35:*.zip=1;35:*.ANSI-30-black=30:*.ANSI-01;30-brblack=01;30:*.ANSI-31-red=31:*.ANSI-01;31-brred=01;31:*.ANSI-32-green=32:*.ANSI-01;32-brgreen=01;32:*.ANSI-33-yellow=33:*.ANSI-01;33-bryellow=01;33:*.ANSI-34-blue=34:*.ANSI-01;34-brblue=01;34:*.ANSI-35-magenta=35:*.ANSI-01;35-brmagenta=01;35:*.ANSI-36-cyan=36:*.ANSI-01;36-brcyan=01;36:*.ANSI-37-white=37:*.ANSI-01;37-brwhite=01;37:*.log=01;32:*~=01;32:*#=01;32:*.bak=01;33:*.BAK=01;33:*.old=01;33:*.OLD=01;33:*.org_archive=01;33:*.off=01;33:*.OFF=01;33:*.dist=01;33:*.DIST=01;33:*.orig=01;33:*.ORIG=01;33:*.swp=01;33:*.swo=01;33:*,v=01;33:*.gpg=34:*.gpg=34:*.pgp=34:*.asc=34:*.3des=34:*.aes=34:*.enc=34:*.sqlite=34:
GREP_COLOR=37;45
GREP_COLORS=mt=37;45
SSH_AUTH_SOCK=/tmp/jck/ssh-agent.sock
SSH_AGENT_PID=22437
MANPATH=:/home/jck/.vim/plugged/fzf/man
```
I think the root cause of the issue is that `project.virtualenv_location` is an empty string. I can dig into the source and investigate during the weekend.

My initial guess is that `delegator` is screwing up while creating the virtualenv and the `virtualenv_location` property ignores the exit code of `pew dir` and accepts and empty string as the virtualenv location.
An extra datapoint: when I first experimented with pipenv about a month ago, I was able to successfully use it.
I’m guessing the root cause is actually that your shell isn’t configured in any capacity at all. You need to be setting `WORKON_HOME`, `LANG`, `LC_ALL`, whatever python / path / virtualenv settings are required (normally at least adding `~/.local/bin` to $PATH). Then instead of using `sudo pip install` you can use `pip install —user` for pipenv virtualenv etc. Also I noticed $SHELL isn’t set. Unrelated but @erinxocon @nateprewitt this is the second arch user in as many days where I’ve seen that. Is this a ‘feature’ of arch?

Anyway we don’t really need you to guess too much since there isn’t much code involved in this. Try setting up your shell and let me know if that helps, otherwise I’ll grab an arch image from work
* I accidentally didn't paste the complete output of `env`, sorry.
*  `~/.local/bin` is in my PATH. But I don't see how this is relevant, I installed pipenv using my system's package manager. You'll be hard pressed to find an arch user who would `sudo pip install` OR `pip install --user` anything.
* I use virtualenv and pew without any issues. pew does not require `WORKON_HOME` to be set and defaults to `{$XDG_DATA_HOME:~/.local/share}/virtualenvs`

@jck its really hard to troubleshoot when you are giving us only partial debugging information.  Rather than providing actual command outputs you are describing what behaviors are like, and taking guesses as to what the cause of those behaviors is.  As long as you are curating debugging information it is going to be hard to help.  

Would you mind providing the complete output of `env`, `which pipenv`, `which python`, `pipenv --version`, and `python --version`?

Also, I don't like to generalize regarding the behavior of users based on distribution flavor or anything else.  Making judgments about what users do isn't relevant, and it doesn't help diagnose bugs.  There is also nothing inherently wrong with user installs using pip.

As a sidenote, you are talking about the system distribution of pipenv from the arch repository.  Your mention of it just now is the first time I have ever heard that such a thing even exists.  We certainly didn't build that and as far as a I know were not aware of it.  At a glance, I can see that it comes from the community arch repo, not from the official system repository.  Not that different from `sudo pip install` 
FWIW, I had already provided all the information you wanted(except complete `env`) but here it is again.

* `which pipenv`: `/usr/bin/pipenv`
* `which python` `/usr/bin/python`
* `pipenv --version`: `pipenv, version 8.2.7`
* `python --version`: `Python 3.6.2`
* `env`: 
```
XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
LANG=en_US.UTF-8
DISPLAY=:0
XDG_VTNR=1
LOGNAME=jck
PWD=/home/jck/a
TMPDIR=/tmp/jck
AUTOJUMP_SOURCED=1
SHLVL=2
HG=/usr/bin/hg
MOZ_PLUGIN_PATH=/usr/lib/mozilla/plugins
XAUTHORITY=/home/jck/.Xauthority
XDG_SESSION_CLASS=user
QT_QPA_PLATFORMTHEME=qt5ct
COLORTERM=truecolor
XDG_SESSION_ID=c2
DESKTOP_SESSION=/usr/share/xsessions/i3
XDG_SESSION_DESKTOP=i3
AUTOJUMP_ERROR_PATH=/home/jck/.local/share/autojump/errors.log
PAM_KWALLET5_LOGIN=/run/user/1000/kwallet5.socket
LESS=-F -g -i -M -R -w -X -z-4
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
VTE_VERSION=5000
MAIL=/var/spool/mail/jck
_=/usr/bin/env
OLDPWD=/home/jck
SHELL=/bin/zsh
GRADLE_HOME=/usr/share/java/gradle
XDG_SESSION_TYPE=x11
WINDOWID=50331651
TERM=xterm-termite
GTK_MODULES=canberra-gtk-module
XDG_CURRENT_DESKTOP=i3
PAGER=less
EDITOR=nvim
PATH=/home/jck/.local/bin:/home/jck/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
HOME=/home/jck
XDG_SEAT=seat0
USER=jck
XDG_RUNTIME_DIR=/run/user/1000
XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session1
LESS_TERMCAP_mb=[01;31m
LESS_TERMCAP_md=[0;33m
LESS_TERMCAP_me=[0m
LESS_TERMCAP_se=[0m
LESS_TERMCAP_so=[1;30;43m
LESS_TERMCAP_ue=[0m
LESS_TERMCAP_us=[4;32m
LS_COLORS=no=00:fi=00:di=34:ow=34;40:ln=35:pi=30;44:so=35;44:do=35;44:bd=33;44:cd=37;44:or=05;37;41:mi=05;37;41:ex=01;31:*.cmd=01;31:*.exe=01;31:*.com=01;31:*.bat=01;31:*.reg=01;31:*.app=01;31:*.txt=32:*.org=32:*.md=32:*.mkd=32:*.h=32:*.c=32:*.C=32:*.cc=32:*.cpp=32:*.cxx=32:*.objc=32:*.sh=32:*.csh=32:*.zsh=32:*.el=32:*.vim=32:*.java=32:*.pl=32:*.pm=32:*.py=32:*.rb=32:*.hs=32:*.php=32:*.htm=32:*.html=32:*.shtml=32:*.erb=32:*.haml=32:*.xml=32:*.rdf=32:*.css=32:*.sass=32:*.scss=32:*.less=32:*.js=32:*.coffee=32:*.man=32:*.0=32:*.1=32:*.2=32:*.3=32:*.4=32:*.5=32:*.6=32:*.7=32:*.8=32:*.9=32:*.l=32:*.n=32:*.p=32:*.pod=32:*.tex=32:*.bmp=33:*.cgm=33:*.dl=33:*.dvi=33:*.emf=33:*.eps=33:*.gif=33:*.jpeg=33:*.jpg=33:*.JPG=33:*.mng=33:*.pbm=33:*.pcx=33:*.pdf=33:*.pgm=33:*.png=33:*.PNG=33:*.ppm=33:*.pps=33:*.ppsx=33:*.ps=33:*.svg=33:*.svgz=33:*.tga=33:*.tif=33:*.tiff=33:*.xbm=33:*.xcf=33:*.xpm=33:*.xwd=33:*.xwd=33:*.yuv=33:*.aac=33:*.au=33:*.flac=33:*.mid=33:*.midi=33:*.mka=33:*.mp3=33:*.mpa=33:*.mpeg=33:*.mpg=33:*.ogg=33:*.ra=33:*.wav=33:*.anx=33:*.asf=33:*.avi=33:*.axv=33:*.flc=33:*.fli=33:*.flv=33:*.gl=33:*.m2v=33:*.m4v=33:*.mkv=33:*.mov=33:*.MOV=33:*.mp4=33:*.mp4v=33:*.mpeg=33:*.mpg=33:*.nuv=33:*.ogm=33:*.ogv=33:*.ogx=33:*.qt=33:*.rm=33:*.rmvb=33:*.swf=33:*.vob=33:*.webm=33:*.wmv=33:*.doc=31:*.docx=31:*.rtf=31:*.dot=31:*.dotx=31:*.xls=31:*.xlsx=31:*.ppt=31:*.pptx=31:*.fla=31:*.psd=31:*.7z=1;35:*.apk=1;35:*.arj=1;35:*.bin=1;35:*.bz=1;35:*.bz2=1;35:*.cab=1;35:*.deb=1;35:*.dmg=1;35:*.gem=1;35:*.gz=1;35:*.iso=1;35:*.jar=1;35:*.msi=1;35:*.rar=1;35:*.rpm=1;35:*.tar=1;35:*.tbz=1;35:*.tbz2=1;35:*.tgz=1;35:*.tx=1;35:*.war=1;35:*.xpi=1;35:*.xz=1;35:*.z=1;35:*.Z=1;35:*.zip=1;35:*.ANSI-30-black=30:*.ANSI-01;30-brblack=01;30:*.ANSI-31-red=31:*.ANSI-01;31-brred=01;31:*.ANSI-32-green=32:*.ANSI-01;32-brgreen=01;32:*.ANSI-33-yellow=33:*.ANSI-01;33-bryellow=01;33:*.ANSI-34-blue=34:*.ANSI-01;34-brblue=01;34:*.ANSI-35-magenta=35:*.ANSI-01;35-brmagenta=01;35:*.ANSI-36-cyan=36:*.ANSI-01;36-brcyan=01;36:*.ANSI-37-white=37:*.ANSI-01;37-brwhite=01;37:*.log=01;32:*~=01;32:*#=01;32:*.bak=01;33:*.BAK=01;33:*.old=01;33:*.OLD=01;33:*.org_archive=01;33:*.off=01;33:*.OFF=01;33:*.dist=01;33:*.DIST=01;33:*.orig=01;33:*.ORIG=01;33:*.swp=01;33:*.swo=01;33:*,v=01;33:*.gpg=34:*.gpg=34:*.pgp=34:*.asc=34:*.3des=34:*.aes=34:*.enc=34:*.sqlite=34:
GREP_COLOR=37;45
GREP_COLORS=mt=37;45
SSH_AUTH_SOCK=/tmp/jck/ssh-agent.sock
SSH_AGENT_PID=1259
VISUAL=nvim
```

I've also tried installing `pipenv` using `pipsi` and I have the same issues.
> SHLVL=2

You appear to be running in a subshell.

You also don't have `LC_ALL` set.

I thought LC_ALL was meant to be set only to override all other locale settings? https://wiki.archlinux.org/index.php/locale#LC_ALL:_troubleshooting

All my terminals seem to start with SHLVL>1, so I went to a virtual console (where SHLVL==1) and `export LC_ALL=en_US.UTF-8` and had all the same issues.

So I downloaded an archlinux image and built a VM completely from scratch, even using the same desktop environment as you are using (i3) and the same shell (zsh).  I then installed pipenv from the community repo and did zero additional configuration of anything (no special environment variables, no path manipulations, just got far enough to get a terminal open on this ridiculous DE which reminds me of fluxbox from a billion years ago).  Then I went to `/tmp` and made a directory called `test`, went there and did `pipenv install tablib` -- screenshot instead of copy since I didn't get the bidirectional clipboard working in virtualbox:

![image](https://user-images.githubusercontent.com/954310/32137200-4c5ae1b6-bbe9-11e7-831f-6b2f377a0657.png)

There's no need to resort to making personal attacks. My choice of window manager would obviously have no affect on Pipenv's behaviour.

The issue still exists for me and I've provided all the information you've asked. At the very least, don't close the bug for the sake of other users who might run into the same issue.
Looking at `project.py` in master, it seems like the `virtualenv_exists` property would never return false since the `virtualenv_location` property always returns the stdout of `pew dir ..` irrespective of pew's exit code. 
The reason this bug was triggered on my system is that `/bin/activate` exists on my system.

Pipenv checks `project.virtualenv_exists` before deciding whether to create a venv. It does so by checking if `project.virtualenv_location`/bin/activate exists. When a virtualenv does not exist, `project.virtualenv_location` is empty(stdout of `pew dir`), so `virtualenv_exists` essentially checks the existence of `/bin/activate`.
It turns out that my original hunch about the root cause was correct. Irrespective of that, I feel compelled to say that my interactions with @techalchemy  has left a bad taste in my mouth and turned me off this project.

I provided all the information I was asked for and additionally, voiced my technical observations about what I thought the causes of the issue could be. However, what I got in return was vocal disdain for my distro and dev environment(it is OK to have those opinions, but this bug report is not the place to get into that), unnecessary focus on extraneous details and complete disregard for potentially useful technical comments.

On top of that, closing the bug just because you couldn't reproduce it on an Arch VM despite of the fact that there was no way to know if this was an arch specific issue in the first place.

> There's no need to resort to making personal attacks. My choice of window manager would obviously have no affect on Pipenv's behaviour.

@jck I didn't make any attacks, beyond pointing out that your window manager was hard to configure (but I did it, in an attempt to replicate the bug you reported).  Also, I don't think it's so obvious  that your window manager or any other configuration parameter can be easily ruled out when troubleshooting, which is why it was frustrating that you were only providing selective output from the commands requested for troubleshooting.

> Looking at `project.py` in master, it seems like the `virtualenv_exists` property would never return false since the `virtualenv_location` property always returns the stdout of `pew dir ..` irrespective of pew's exit code.

>The reason this bug was triggered on my system is that `/bin/activate` exists on my system.

> Pipenv checks `project.virtualenv_exists` before deciding whether to create a venv. It does so by checking if `project.virtualenv_location/bin/activate` exists. When a virtualenv does not exist, `project.virtualenv_location` is empty(stdout of `pew dir`), so `virtualenv_exists` essentially checks the existence of `/bin/activate`.

If you had been this detailed when you first suspected this as the culprit, we would have resolved this issue two weeks ago because I would have done this verification and just fixed it:

```python
>>> from pipenv.vendor import delegator
>>> c = delegator.run('pew dir "asdf"')
>>> c.out
''
>>> c.err
"ERROR: Environment 'asdf' does not exist. Create it with 'pew new asdf'.\n"
```

>It turns out that my original hunch about the root cause was correct. Irrespective of that, I feel compelled to say that my interactions with @techalchemy has left a bad taste in my mouth and turned me off this project.

I'm sorry that you feel turned off to this project.  Communicating with you on this issue was difficult, because whenever I asked for outputs you responded with a curated selection or just with your own analysis. That's frustrating for a lot of reasons, but mostly because I can quickly scan outputs and identify issues.  

>I provided all the information I was asked for and additionally, voiced my technical observations about what I thought the causes of the issue could be. However, what I got in return was vocal disdain for my distro and dev environment(it is OK to have those opinions, but this bug report is not the place to get into that), unnecessary focus on extraneous details and complete disregard for potentially useful technical comments.

I have no disdain for your distro at all.  I'm not really sure where you are getting that from.  I used to use slackware and fluxbox many years ago and it reminded me of that.  Arch is wildly popular and I enjoyed setting it up.  I haven't really used it much myself but I certainly have nothing against it.  Similarly I don't really know anything about your dev environment so I don't have anything against that either.  Apart from that I don't know how continuously asking you to provide configuration outputs can be classified as focusing on extraneous details.  If you look through the many issues we have encountered, you will find that a large majority of them have been resolved by addressing shell misconfigurations among other things.

> On top of that, closing the bug just because you couldn't reproduce it on an Arch VM despite of the fact that there was no way to know if this was an arch specific issue in the first place.

No, there is no way to know if a bug is specific to an OS or a user's configuration or some other thing.  That's why I attempted to replicate as many pieces as I was aware of, including your OS and your environment, and I couldn't replicate the issue you reported (which turned out to be due to having a python installation in a system path interacting with an uncaught race condition in our code as you pointed out).  I suspect we could have caught this sooner if, when you were providing general commentary about what you thought was causing the problem. you included the relevant code and output to make your point, or submitted a PR with a fix.

My point is that when you say something like 'I think its because of pew and delegator not returning a virtualenv location' it's not immediately obvious how that relates to an infinite loop of attempted creation-deletions.  So, given an inability to reproduce and a lack of response for 10 days I closed the issue.  It's pretty standard, not a personal thing.

Resolving patch:
diff --git a/pipenv/project.py b/pipenv/project.py
--- a/pipenv/project.py
+++ b/pipenv/project.py
@@ -103,7 +103,7 @@ def requirements_exists(self):
     @property
     def virtualenv_exists(self):
         # TODO: Decouple project from existence of Pipfile.
-        if self.pipfile_exists:
+        if self.pipfile_exists and os.path.exists(self.virtualenv_location):
             if os.name == 'nt':
                 extra = ['Scripts', 'activate.bat']
             else:


Test patch: