Repository: pypa/virtualenv

Issue:
RuntimeError: 'Exec format error' when using pipenv+pyenv-installed python 3.8.1 to install environment
Thanks for submitting an issue!

If submitting a BUG please provide:

- [X ] Minimal reproducible example or detailed descriptions
```
/Users/mhjeon/.local/share/virtualenvs/nowwow-0D8RInBf/bin/python3 -mvirtualenv /tmp/env -p /Users/mhjeon/.local/share/virtualenvs/nowwow-0D8RInBf/bin/python3
```
This gives me an output of
```
RuntimeError: failed to query /Users/mhjeon/.pyenv/versions/3.8.1/Python.framework/Versions/3.8/python with code 8 err: 'Exec format error'
```
- [ ] OS and `pip list` output
using virtualenv 20.0.0b2
 
Please see [this issue for more detail](https://github.com/pre-commit/pre-commit/issues/1321)


Discussion:
What's the content of the executable in question? Says it's not executable 🤔 Can you post please the output by passing the -vvv flag? Please also attach the ls -al of the executable in question. Thanks!
Please also attach the command you've used to install your pyenv python in question 😊
This issue still needs a small reproducible script, I'm failing to do so. A run with ``-vvv`` would be a good start.
A reproducible script for me was

```
mkdir -p /tmp/vtest && cd /tmp/vtest
virtualenv --python python3.7 py37
. ./py37/bin/activate
pip install virtualenv
./py37/bin/python -m virtualenv --no-download --python ./py37/bin/python py37
```

and I get the following output:
```
RuntimeError: failed to query /opt/local/Library/Frameworks/Python.framework/Versions/3.7/python with code 8 err: 'Exec format error'
```

Can you please post the output by passing -vvv to the virtualenv command.
```
➜ ./py37/bin/python -m virtualenv -vvv --no-download --python ./py37/bin/python py37
118 setup logging to NOTSET [DEBUG report:43]
128 find interpreter for spec PythonSpec(path=/private/tmp/vtest/py37/bin/python) [INFO builtin:44]
128 discover system for PythonInfo(spec=CPython3.7.4.final.0-64, exe=/private/tmp/vtest/py37/bin/python, platform=darwin, version='3.7.4 (default, Oct 16 2019, 11:14:51) \n[Clang 11.0.0 (clang-1100.0.33.8)]', encoding_fs_io=utf-8-UTF-8) in /opt/local/Library/Frameworks/Python.framework/Versions/3.7 [DEBUG py_info:332]
129 filesystem is not case-sensitive [DEBUG info:28]
130 Attempting to acquire lock 4446500432 on /Users/ksmithson/Library/Application Support/virtualenv/py-info/20.0.1/c405059ba5cab83b4c9edb814600b939aab4bb299cf178dce99ee642f825d112.lock [DEBUG filelock:270]
130 Lock 4446500432 acquired on /Users/ksmithson/Library/Application Support/virtualenv/py-info/20.0.1/c405059ba5cab83b4c9edb814600b939aab4bb299cf178dce99ee642f825d112.lock [INFO filelock:274]
130 get interpreter info via cmd: /opt/local/Library/Frameworks/Python.framework/Versions/3.7/python -s /private/tmp/vtest/py37/lib/python3.7/site-packages/virtualenv/discovery/py_info.py [DEBUG cached_py_info:107]
132 Attempting to release lock 4446500432 on /Users/ksmithson/Library/Application Support/virtualenv/py-info/20.0.1/c405059ba5cab83b4c9edb814600b939aab4bb299cf178dce99ee642f825d112.lock [DEBUG filelock:315]
132 Lock 4446500432 released on /Users/ksmithson/Library/Application Support/virtualenv/py-info/20.0.1/c405059ba5cab83b4c9edb814600b939aab4bb299cf178dce99ee642f825d112.lock [INFO filelock:318]
132 done in 129ms [INFO __main__:23]
132 RuntimeError: failed to query /opt/local/Library/Frameworks/Python.framework/Versions/3.7/python with code 8 err: 'Exec format error' [ERROR __main__:30]
```
I'm a little sad we still released while we had identified this during the beta period -- would have been nice to work this out instead of having users pester downstream packages about this https://github.com/pre-commit/pre-commit/issues/1327
here's what I'm getting:

```console
$ virtualenv -vvv vvv -p python2.7
69 setup logging to NOTSET [DEBUG report:43]
75 find interpreter for spec PythonSpec(implementation=CPython, major=2, minor=7) [INFO builtin:44]
75 discover system for PythonInfo(spec=CPython3.7.2.final.0-64, exe=/private/tmp/venv/bin/python3, platform=darwin, version='3.7.2 (v3.7.2:9a3ffc0492, Dec 24 2018, 02:44:43) \n[Clang 6.0 (clang-600.0.57)]', encoding_fs_io=utf-8-UTF-8) in /Library/Frameworks/Python.framework/Versions/3.7 [DEBUG py_info:332]
77 filesystem is not case-sensitive [DEBUG info:28]
79 Attempting to acquire lock 4549774472 on /Users/asottile/Library/Application Support/virtualenv/py-info/20.0.1/fb7ec9939e66e8ed189e2fe772cae61fcc992510c23be22bd7dadeab5367beb4.lock [DEBUG filelock:270]
79 Lock 4549774472 acquired on /Users/asottile/Library/Application Support/virtualenv/py-info/20.0.1/fb7ec9939e66e8ed189e2fe772cae61fcc992510c23be22bd7dadeab5367beb4.lock [INFO filelock:274]
79 get interpreter info via cmd: /Library/Frameworks/Python.framework/Versions/3.7/python -s /private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py [DEBUG cached_py_info:107]
81 Attempting to release lock 4549774472 on /Users/asottile/Library/Application Support/virtualenv/py-info/20.0.1/fb7ec9939e66e8ed189e2fe772cae61fcc992510c23be22bd7dadeab5367beb4.lock [DEBUG filelock:315]
82 Lock 4549774472 released on /Users/asottile/Library/Application Support/virtualenv/py-info/20.0.1/fb7ec9939e66e8ed189e2fe772cae61fcc992510c23be22bd7dadeab5367beb4.lock [INFO filelock:318]
82 done in 80ms [INFO __main__:26]
82 RuntimeError: failed to query /Library/Frameworks/Python.framework/Versions/3.7/python with code 8 err: 'Exec format error' [ERROR __main__:33]
Traceback (most recent call last):
  File "/private/tmp/venv/bin/virtualenv", line 8, in <module>
    sys.exit(run_with_catch())
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/__main__.py", line 31, in run_with_catch
    run(args)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/__main__.py", line 19, in run
    run_via_cli(args)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/run/__init__.py", line 21, in run_via_cli
    session = session_via_cli(args)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/run/__init__.py", line 28, in session_via_cli
    parser = build_parser(args)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/run/__init__.py", line 41, in build_parser
    parser._interpreter = interpreter = discover.interpreter
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/discover.py", line 44, in interpreter
    self._interpreter = self.run()
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/builtin.py", line 33, in run
    return get_interpreter(self.python_spec)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/builtin.py", line 46, in get_interpreter
    for interpreter, impl_must_match in propose_interpreters(spec):
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/builtin.py", line 63, in propose_interpreters
    yield PythonInfo.current_system(), True
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 268, in current_system
    cls._current_system = cls.from_exe(sys.executable, raise_on_error=True, resolve_to_host=True)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 290, in from_exe
    proposed = proposed._resolve_to_system(proposed)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 318, in _resolve_to_system
    target = target.discover_exe(prefix=prefix, exact=False)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 341, in discover_exe
    info = self.from_exe(exe_path, resolve_to_host=False)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 287, in from_exe
    proposed = from_exe(cls, exe, raise_on_error=raise_on_error, ignore_cache=ignore_cache)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/cached_py_info.py", line 37, in from_exe
    raise result
RuntimeError: failed to query /Library/Frameworks/Python.framework/Versions/3.7/python with code 8 err: 'Exec format error'
```

I had to adjust `__main__.py` to `raise` instead of `sys.exit(1)` to get the stacktrace

```diff
$ diff -u venv/lib/python3.7/site-packages/virtualenv/__main__.py{.old,}
--- venv/lib/python3.7/site-packages/virtualenv/__main__.py.old	2020-02-10 15:43:05.000000000 -0800
+++ venv/lib/python3.7/site-packages/virtualenv/__main__.py	2020-02-10 15:41:28.000000000 -0800
@@ -28,6 +28,7 @@
         run(args)
     except (KeyboardInterrupt, Exception) as exception:
         logging.error("%s: %s", type(exception).__name__, exception)
+        raise
         sys.exit(1)
 
 
```
note that I am not using pyenv, my python is acquired from homebrew
@asottile did not need the stack trace, knew exactly where things fail, but needed to see the discovery trace to see where things go wrong. Thanks.
@asottile how did you install virtualenv? 

Resolving patch:
diff --git a/src/virtualenv/discovery/py_info.py b/src/virtualenv/discovery/py_info.py
--- a/src/virtualenv/discovery/py_info.py
+++ b/src/virtualenv/discovery/py_info.py
@@ -338,7 +338,9 @@ def discover_exe(self, prefix, exact=True):
             for name in possible_names:
                 exe_path = os.path.join(folder, name)
                 if os.path.exists(exe_path):
-                    info = self.from_exe(exe_path, resolve_to_host=False)
+                    info = self.from_exe(exe_path, resolve_to_host=False, raise_on_error=False)
+                    if info is None:  # ignore if for some reason we can't query
+                        continue
                     for item in ["implementation", "architecture", "version_info"]:
                         found = getattr(info, item)
                         searched = getattr(self, item)
@@ -411,6 +413,9 @@ def _find_possible_folders(self, inside_folder):
 
         # or at root level
         candidate_folder[inside_folder] = None
+        if self.executable.startswith(self.prefix):
+            binary_within = os.path.relpath(os.path.dirname(self.executable), self.prefix)
+            candidate_folder[os.path.join(inside_folder, binary_within)] = None
 
         return list(candidate_folder.keys())
 


Test patch:
diff --git a/tests/unit/discovery/py_info/test_py_info.py b/tests/unit/discovery/py_info/test_py_info.py
--- a/tests/unit/discovery/py_info/test_py_info.py
+++ b/tests/unit/discovery/py_info/test_py_info.py
@@ -205,7 +205,12 @@ def _make_py_info(of):
     target_py_info = _make_py_info(target)
     mocker.patch.object(target_py_info, "_find_possible_exe_names", return_value=names)
     mocker.patch.object(target_py_info, "_find_possible_folders", return_value=[str(tmp_path)])
-    mocker.patch.object(target_py_info, "from_exe", side_effect=lambda k, resolve_to_host: discovered_with_path[k])
+
+    # noinspection PyUnusedLocal
+    def func(k, resolve_to_host, raise_on_error):
+        return discovered_with_path[k]
+
+    mocker.patch.object(target_py_info, "from_exe", side_effect=func)
     target_py_info.real_prefix = str(tmp_path)
 
     target_py_info.system_executable = None