Repository: pypa/virtualenv

Issue:
"executable py33/bin/python3.3 is not functioning" with mac framework build
It looks like virtualenv and python 3.3's built-in venv support interact badly in a mac framework build.  Virtualenv creation fails with the message "executable not functioning".  I've traced through it enough to figure out what's happening:

1) "virtualenv -p python3.3" re-execs itself under /opt/local/bin/python3.3
2) In a framework build,  $prefix/bin/pythonX.Y is a "pythonw" executable, which sets `__PYVENV_LAUNCHER__` then executes the "real" binary (Resources/Python.app/Contents/MacOS/Python).  The real binary uses the launcher environment variable (if present) to determine sys.prefix
3) virtualenv copies the real binary (not pythonw) into the environment as $env/bin/pythonX.Y
4) When virtualenv runs the env's python to test it, the launcher variable from the pythonw process that started virtualenv is still present, so the env's python uses the global value for sys.prefix.
5) If the env's python is run in a clean environment (without `__PYVENV_LAUNCHER__`), it finds the right prefix.

It looks like virtualenv needs to ensure that the env's python is always run with a correct (or missing) `__PYVENV_LAUNCHER__` variable, and never inherits one from the environment.  One way to do this would be to copy the pythonw shim into $env/bin instead of the underlying python, but I don't know what other ramifications this would have.

Even though virtualenv is not needed as much since 3.3 has equivalent functionality built in, it's still important that virtualenv work with 3.3 because tools like tox do not yet support 3.3-native venvs.

My environment: OSX Mountain Lion, python 3.3rc1 installed via macports, virtualenv installed from the "develop" branch today.

```
(venv_tip)[bdarnell@gonzo ~]$ virtualenv -v -p python3.3 py33
Running virtualenv with interpreter /opt/local/bin/python3.3
Creating py33/lib/python3.3
Symlinking Python bootstrap modules
  Symlinking py33/lib/python3.3/config-3.3m
  Symlinking py33/lib/python3.3/lib-dynload
  Symlinking py33/lib/python3.3/os.py
  Ignoring built-in bootstrap module: posix
  Symlinking py33/lib/python3.3/posixpath.py
  Cannot import bootstrap module: nt
  Symlinking py33/lib/python3.3/ntpath.py
  Symlinking py33/lib/python3.3/genericpath.py
  Symlinking py33/lib/python3.3/fnmatch.py
  Symlinking py33/lib/python3.3/locale.py
  Symlinking py33/lib/python3.3/encodings
  Symlinking py33/lib/python3.3/codecs.py
  Symlinking py33/lib/python3.3/stat.py
  Cannot import bootstrap module: UserDict
  Cannot import bootstrap module: copy_reg
  Symlinking py33/lib/python3.3/types.py
  Symlinking py33/lib/python3.3/re.py
  Cannot import bootstrap module: sre
  Symlinking py33/lib/python3.3/sre_parse.py
  Symlinking py33/lib/python3.3/sre_constants.py
  Symlinking py33/lib/python3.3/sre_compile.py
  Cannot import bootstrap module: _abcoll
  Symlinking py33/lib/python3.3/warnings.py
  Symlinking py33/lib/python3.3/linecache.py
  Symlinking py33/lib/python3.3/abc.py
  Symlinking py33/lib/python3.3/io.py
  Symlinking py33/lib/python3.3/_weakrefset.py
  Symlinking py33/lib/python3.3/copyreg.py
  Symlinking py33/lib/python3.3/tempfile.py
  Symlinking py33/lib/python3.3/random.py
  Symlinking py33/lib/python3.3/__future__.py
  Symlinking py33/lib/python3.3/collections
  Symlinking py33/lib/python3.3/keyword.py
  Symlinking py33/lib/python3.3/tarfile.py
  Symlinking py33/lib/python3.3/shutil.py
  Symlinking py33/lib/python3.3/struct.py
  Symlinking py33/lib/python3.3/copy.py
  Symlinking py33/lib/python3.3/tokenize.py
  Symlinking py33/lib/python3.3/token.py
  Symlinking py33/lib/python3.3/functools.py
  Symlinking py33/lib/python3.3/heapq.py
  Symlinking py33/lib/python3.3/bisect.py
  Symlinking py33/lib/python3.3/weakref.py
  Symlinking py33/lib/python3.3/reprlib.py
  Symlinking py33/lib/python3.3/base64.py
  Symlinking py33/lib/python3.3/_dummy_thread.py
  Symlinking py33/lib/python3.3/hashlib.py
  Symlinking py33/lib/python3.3/hmac.py
  Symlinking py33/lib/python3.3/rlcompleter.py
Creating py33/lib/python3.3/site-packages
Writing py33/lib/python3.3/site.py
Writing py33/lib/python3.3/orig-prefix.txt
Writing py33/lib/python3.3/no-global-site-packages.txt
Creating parent directories for py33/include
Symlinking py33/include/python3.3m
Creating py33/bin
New python executable in py33/bin/python3.3
Changed mode of py33/bin/python3.3 to 0o755
Also creating executable in py33/bin/python
Changed mode of py33/bin/python to 0o755
Symlinking py33/.Python
Testing executable with py33/bin/python3.3 -c "import sys;out=sys.stdout;getattr(out, "buffer", out).write(sys.prefix.encode("utf-8"))"
ERROR: The executable py33/bin/python3.3 is not functioning
ERROR: It thinks sys.prefix is '/opt/local/Library/Frameworks/Python.framework/Versions/3.3' (should be '/Users/bdarnell/py33')
ERROR: virtualenv is not compatible with this system or executable
```



Discussion:
Reproduced locally - thanks for the detailed report.

This gets further with the below diff but fails with

```
OSError: Command /Users/pnasrat/Devel...env/33/bin/python3.3 -x /Users/pnasrat/Devel.../33/bin/easy_install /Users/pnasrat/Devel...pport/pip-1.2.tar.gz failed with error code 2
```

Running with `-vvv`

```
  Extracting distribute-0.6.28-py3.3.egg to /Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages
  distribute 0.6.28 is already the active version in easy-install.pth
  Installing easy_install script to /Library/Frameworks/Python.framework/Versions/3.3/bin
  Installing easy_install-3.3 script to /Library/Frameworks/Python.framework/Versions/3.3/bin
```

``` diff
diff --git a/virtualenv.py b/virtualenv.py
index 2316d7c..f16dcdd 100755
--- a/virtualenv.py
+++ b/virtualenv.py
@@ -1257,6 +1257,10 @@ def install_python(home_dir, lib_dir, inc_dir, bin_dir, site_packages, clear):
     mkdir(bin_dir)
     py_executable = join(bin_dir, os.path.basename(sys.executable))
     if 'Python.framework' in prefix:
+        # OS X framework builds cause validation to break
+        # https://github.com/pypa/virtualenv/issues/322
+        if os.environ.get('__PYVENV_LAUNCHER__'):
+          os.unsetenv('__PYVENV_LAUNCHER__')
         if re.search(r'/Python(?:-32|-64)*$', py_executable):
             # The name of the python executable is not quite what
             # we want, rename it.
```


Resolving patch:
diff --git a/virtualenv.py b/virtualenv.py
--- a/virtualenv.py
+++ b/virtualenv.py
@@ -489,7 +489,7 @@ def _install_req(py_executable, unzip=False, distribute=False,
         source = None
     else:
         setup_fn = None
-        source = 'distribute-0.6.27.tar.gz'
+        source = 'distribute-0.6.28.tar.gz'
         project_name = 'distribute'
         bootstrap_script = DISTRIBUTE_SETUP_PY
 
@@ -511,7 +511,7 @@ def _install_req(py_executable, unzip=False, distribute=False,
     if unzip:
         cmd.append('--always-unzip')
     env = {}
-    remove_from_env = []
+    remove_from_env = ['__PYVENV_LAUNCHER__']
     if logger.stdout_level_matches(logger.DEBUG):
         cmd.append('-v')
 
@@ -1257,6 +1257,10 @@ def install_python(home_dir, lib_dir, inc_dir, bin_dir, site_packages, clear):
     mkdir(bin_dir)
     py_executable = join(bin_dir, os.path.basename(sys.executable))
     if 'Python.framework' in prefix:
+        # OS X framework builds cause validation to break
+        # https://github.com/pypa/virtualenv/issues/322
+        if os.environ.get('__PYVENV_LAUNCHER__'):
+          os.unsetenv('__PYVENV_LAUNCHER__')
         if re.search(r'/Python(?:-32|-64)*$', py_executable):
             # The name of the python executable is not quite what
             # we want, rename it.


Test patch: