Repository: pypa/pip

Issue:
pip freeze returns "Permission denied: 'hg'" after using pip install -e
**Environment**

* pip version: 19.3.1 and above
* Python version: 3.7.7
* OS: macOS 10.14.6

**Description**
After using `pip install -e` to install a local package, `pip freeze` returns an error: `PermissionError: [Errno 13] Permission denied: 'hg'`.

I do not use Mercurial and do not have an `hg` executable on my machine.

It happens with pip 19.3.1 and above, when using `-e`.

It doesn't happen with pip 19.2.3, and it doesn't happen if I don't use `-e`. It doesn't happen with `pip list`.

**Expected behavior**
`pip freeze` should return the list of packages.

**How to Reproduce**
Create a simple package, install it using `pip install -e`, then run `pip freeze`.
It works with pip 19.2.3 but in 19.3.1 or later versions, `pip freeze` returns an error.

You can just copy/paste the following, which creates a virtual environment and does the `pip install -e` and the `pip freeze` on two different versions of pip:
```
cd
mkdir testdir
cd testdir/
virtualenv testenv
source testenv/bin/activate
mkdir my_plugin
echo "
from setuptools import setup

setup(
    name='my_plugin',
    py_modules=['my_plugin'],
    version='1.0.0',
)
" > my_plugin/setup.py
pip install pip==19.2.3
pip -V
pip install -e my_plugin/
pip freeze
pip uninstall -y my_plugin
pip freeze
pip install pip==19.3.1
pip -V
pip install -e my_plugin/
pip freeze
pip uninstall -y my_plugin
pip freeze
```

**Output**
Here's the output of the above commands, starting at the first `pip -V`:
```
$ pip -V
pip 19.2.3 from /Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip (python 3.7)
$ pip install -e my_plugin/
Obtaining file:///Users/<username>/testdir/my_plugin
Installing collected packages: my-plugin
  Running setup.py develop for my-plugin
Successfully installed my-plugin
WARNING: You are using pip version 19.2.3, however version 20.2b1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ pip freeze
# Editable install with no version control (my-plugin==1.0.0)
-e /Users/<username>/testdir/my_plugin
$ pip uninstall -y my_plugin
Uninstalling my-plugin-1.0.0:
  Successfully uninstalled my-plugin-1.0.0
$ pip freeze
$ pip install pip==19.3.1
Collecting pip==19.3.1
  Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 19.2.3
    Uninstalling pip-19.2.3:
      Successfully uninstalled pip-19.2.3
Successfully installed pip-19.3.1
$ pip -V
pip 19.3.1 from /Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip (python 3.7)
$ pip install -e my_plugin/
Obtaining file:///Users/<username>/testdir/my_plugin
Installing collected packages: my-plugin
  Running setup.py develop for my-plugin
Successfully installed my-plugin
WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ pip freeze
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 153, in _main
    status = self.run(options, args)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/commands/freeze.py", line 100, in run
    for line in freeze(**freeze_kwargs):
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/operations/freeze.py", line 70, in freeze
    req = FrozenRequirement.from_dist(dist)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/operations/freeze.py", line 249, in from_dist
    req, editable, comments = get_requirement_info(dist)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/operations/freeze.py", line 188, in get_requirement_info
    vcs_backend = vcs.get_backend_for_dir(location)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 234, in get_backend_for_dir
    if vcs_backend.controls_location(location):
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/vcs/mercurial.py", line 149, in controls_location
    log_failed_cmd=False)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 632, in run_command
    log_failed_cmd=log_failed_cmd)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/utils/subprocess.py", line 190, in call_subprocess
    stdout=subprocess.PIPE, cwd=cwd, env=env,
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: 'hg'
$ pip uninstall -y my_plugin
Uninstalling my-plugin-1.0.0:
  Successfully uninstalled my-plugin-1.0.0
$ pip freeze
$ 
```

`pip list` still works. In fact, if I add `pip list` just before the problematic `pip freeze` in the chain of commands above, there is no error:
```
$ pip list
Package    Version
---------- -------
my-plugin  1.0.0  
pip        19.3.1 
setuptools 47.1.1 
wheel      0.34.2 
WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ pip freeze
my-plugin==1.0.0
```


Discussion:
Is `my_plugin` managed by mercurial, and does your project root contain a `.hg` directory? I suspect this is a side effect of #7065; the Mercurial detection code path is much less trotted than Git due to popularity, and this may be hitting an edge case where Mercurial is expected but not actually being available.

What does it show if you run `hg` in the command line?
So I don't have `hg` at all:
```
$ hg
-bash: hg: command not found
$ type hg
-bash: type: hg: not found
```

In my repro steps, there is no `.hg`folder, either in `~/testdir/` or in `~`.

I'm not sure what makes pip expect to find Mercurial.

Oh, I also found #7072. It's about Mercurial and pip freeze and was merged between 19.2.3 and 19.3.1, so it seems likely that the problem started there. I'll look more.
It’d be awesome if you could `git bisect` the exact commit that introduced the breakage. This is quite difficult to reproduce locally 😢
I'd never heard of that tool! This is awesome.

I'm running it between [a5b1a57](https://github.com/pypa/pip/commit/a5b1a57d6707ca9a8ea28c3c1d96b266ed88fbcb) (19.2.3) and [a5b1a57](https://github.com/pypa/pip/commit/6500e595601b8cb4ce597b9f7a71aaf82441c1f8) (19.3.1), which I know as good and bad respectively.

This is the result:
```
c0809f4183cbf40e3fd69245095537456d74b97f is the first bad commit
commit c0809f4183cbf40e3fd69245095537456d74b97f
Author: Tony Beswick <tonybeswick@orcon.net.nz>
Date:   Tue Sep 24 11:46:02 2019 +1200

    Added a `controls_location` implementation to the `Mercurial` class, so it porperly detects that subdirectories are under mercurial control.

:040000 040000 6a58155df7ff4b3457d40c849c2c4f659df01db6 3d487b2c2fb13982d89e0291b91b964d9be8416a M	src
```
It points to commit [c0809f4](https://github.com/pypa/pip/commit/c0809f4183cbf40e3fd69245095537456d74b97f).

I'm not sure why this code would run since I don't have Mercurial. But it makes sense to me that if pip is trying to run that code, then the `cls.run_command()` fails. I think I'll need to dig into why pip thinks it needs to run code for Mercurial.
Another interesting thing is that the implementation does try to catch the error when hg is not installed (the `BadCommand` block) but your environment raises something it does not expect. I wonder why your system raises `PermissionError` instead of `FileNotFoundError`, which is the expected exception for unavailable commands. 

Yes that's really odd.
I'll do my best to track this down and understand what's wrong with my environment and see if pip can detect that. Thanks for your help so far!
Is there anything actionable here? I'm sorry but I can't tell.
I'm experiencing the same problem with my project. @uranusjr were you able to resolve this?

I noticed that an egg was being made with a different name than my source code, due to an uneditted setup.py 

Removing the egg allowed pip freeze to run properly.
I haven't taken the time to figure out why my environment behaves in this way. I still have the problem but am still working around it by pinning pip to an earlier version.

I'm okay closing the ticket for now (if that's what you're asking), if I can reopen it when I have more information about the problem.
I encountered this issue recently. I worked around the issue by downgrading pip to 19.2.3.
It may make sense if we just treat `PermissionError` the same as `FileNotFoundError` in the version control code. The two exceptions both signal that the version control tool is not available, which is what we really care about, after all.

Another choice would be to perform our own command resolution logic (via `shutil.which()`) instead of relying the operating system to emit a `FileNotFoundError` for the subprocess. But I’m not quite sure if that’s feasible for pip right now (because Python 2).
I had the same issue, pip-20.2.4

```
$ pip freeze
ERROR: Exception:
Traceback (most recent call last):
  File "/home/ec2-user/st-prod/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 228, in _main
    status = self.run(options, args)
  File "/home/ec2-user/st-prod/lib/python3.6/site-packages/pip/_internal/commands/freeze.py", line 101, in run
    for line in freeze(**freeze_kwargs):
  File "/home/ec2-user/st-prod/lib/python3.6/site-packages/pip/_internal/operations/freeze.py", line 67, in freeze
    req = FrozenRequirement.from_dist(dist)
  File "/home/ec2-user/st-prod/lib/python3.6/site-packages/pip/_internal/operations/freeze.py", line 252, in from_dist
    req, editable, comments = get_requirement_info(dist)
  File "/home/ec2-user/st-prod/lib/python3.6/site-packages/pip/_internal/operations/freeze.py", line 187, in get_requirement_info
    vcs_backend = vcs.get_backend_for_dir(location)
  File "/home/ec2-user/st-prod/lib/python3.6/site-packages/pip/_internal/vcs/versioncontrol.py", line 333, in get_backend_for_dir
    repo_path = vcs_backend.get_repository_root(location)
  File "/home/ec2-user/st-prod/lib/python3.6/site-packages/pip/_internal/vcs/mercurial.py", line 147, in get_repository_root
    log_failed_cmd=False,
  File "/home/ec2-user/st-prod/lib/python3.6/site-packages/pip/_internal/vcs/versioncontrol.py", line 774, in run_command
    log_failed_cmd=log_failed_cmd)
  File "/home/ec2-user/st-prod/lib/python3.6/site-packages/pip/_internal/vcs/versioncontrol.py", line 119, in call_subprocess
    cwd=cwd
  File "/usr/lib64/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib64/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: 'hg'
```

I had to install hg-git to fix it

`sudo yum install hg-git -y`


I’m even more confused to hear your fix worked. What on earth is going on.
Hi, I am having the same issue with `pip freeze` on the following configuration:

**Environment**

- Pip version: 20.3.1 (2020-12-03)
- Python version: 2.7.6
- OS: macOS 10.14.6

**Diagnostic**

I have two user profiles for development (named "first" and "second" here). On the first profile, `pip` works without problem; on the second profile, I have the issue: when I run `hg` on the command line, I get:

```shell
second@imac-de-laurent-1 ~> hg
fish: Unknown command 'hg'
```

So, I took a look on the first profile, it appears that `hg` works!

```shell
first@imac-de-laurent-1 ~> hg
Mercurial Distributed SCM
[…]
```

The problem is that the `hg` command in a symbolic link to a **user-level** installation of [SourceTree](https://www.sourcetreeapp.com/):

```shell
second@iMac-de-Laurent ~> ls -l /usr/local/bin/hg 
lrwxr-xr-x  1 first  admin  93 13 oct  2016 /usr/local/bin/hg -> /Users/first/Applications/SourceTree.app/Contents/Resources/mercurial_local/hg_local
```

So, if I launch this command from the second profile, I get a "permission denied" error:

```shell
second@iMac-de-Laurent ~> /usr/local/bin/hg
second@iMac-de-Laurent ~ [126]> 
```

=> exit code is 126

**How to fix?**

From the Pip's point of view, the `hg` command should be considered unavailable. So you should take into account that Permission denied `[Errno 13]` is a valid exception, and should catch it. But, it could be useful for the user (the developer) to emit a warning and alert him that he doesn't have access to this tool even if it is installed.

In my situation, I will try to uninstall SourceTree and reinstall it "globally".

Once SourceTree uninstalled (and the broken symbolic link removed), I tried `pip freeze` again.
This time, I get:

```shell
second@imac-de-laurent-1 ~/w/my_project(262|M?)> pip freeze --verbose
Created temporary directory: /private/var/folders/lr/hjtkqzxd189dw_x7dk0493h80000gp/T/pip-ephem-wheel-cache-9hFWsh
Checking in /Users/second/workspace/my_project/src for .svn (svn)...
Checking in /Users/second/workspace/my_project/src for .git (git)...
Checking in /Users/second/workspace/my_project/src for .bzr (bzr)...
Checking in /Users/second/workspace/my_project/src for .hg (hg)...
could not determine if /Users/second/workspace/my_project/src is under hg control because hg is not available
No VCS found for editable requirement "my_project==3.1.26" in: '/Users/second/workspace/my_project/src'
...
[list of dependencies]
```

It's curious because my project is under Subversion, so the checking of `.svn` should have worked. Is it another problem?
> rom the Pip's point of view, the `hg` command should be considered unavailable.

This is arguably a user issue because `hg` *is* available, but corrupted. pip can definitely catch `PermissionError` and emit a better error message, but probably should not pretend the command does not exist at all.
I’ll mark this as “awaiting PR” since it is pretty clear what we want to do. Feel free to try your hands on this and we can continue the discussion from there!
OK @uranusjr , I'm working on it.

I will use an error message suggesting the developper to reinstall the 'hg' (or whatever) tool for all users (globally).

For instance: "Command hg is available, but you don't have permission to execute it - Can you ask an admin to reinstall hg for all users?"

Resolving patch:
diff --git a/src/pip/_internal/vcs/versioncontrol.py b/src/pip/_internal/vcs/versioncontrol.py
--- a/src/pip/_internal/vcs/versioncontrol.py
+++ b/src/pip/_internal/vcs/versioncontrol.py
@@ -684,6 +684,18 @@ def run_command(
             raise BadCommand(
                 f'Cannot find command {cls.name!r} - do you have '
                 f'{cls.name!r} installed and in your PATH?')
+        except PermissionError:
+            # errno.EACCES = Permission denied
+            # This error occurs, for instance, when the command is installed
+            # only for another user. So, the current user don't have
+            # permission to call the other user command.
+            raise BadCommand(
+                f"No permission to execute {cls.name!r} - install it "
+                f"locally, globally (ask admin), or check your PATH. "
+                f"See possible solutions at "
+                f"https://pip.pypa.io/en/latest/reference/pip_freeze/"
+                f"#fixing-permission-denied."
+            )
 
     @classmethod
     def is_repository_directory(cls, path):


Test patch:
diff --git a/tests/unit/test_vcs.py b/tests/unit/test_vcs.py
--- a/tests/unit/test_vcs.py
+++ b/tests/unit/test_vcs.py
@@ -303,6 +303,27 @@ def test_version_control__get_url_rev_and_auth__no_revision(url):
     assert 'an empty revision (after @)' in str(excinfo.value)
 
 
+@pytest.mark.parametrize("vcs_cls", [Bazaar, Git, Mercurial, Subversion])
+@pytest.mark.parametrize(
+    "exc_cls, msg_re",
+    [
+        (FileNotFoundError, r"Cannot find command '{name}'"),
+        (PermissionError, r"No permission to execute '{name}'"),
+    ],
+    ids=["FileNotFoundError", "PermissionError"],
+)
+def test_version_control__run_command__fails(vcs_cls, exc_cls, msg_re):
+    """
+    Test that ``VersionControl.run_command()`` raises ``BadCommand``
+    when the command is not found or when the user have no permission
+    to execute it. The error message must contains the command name.
+    """
+    with patch("pip._internal.vcs.versioncontrol.call_subprocess") as call:
+        call.side_effect = exc_cls
+        with pytest.raises(BadCommand, match=msg_re.format(name=vcs_cls.name)):
+            vcs_cls.run_command([])
+
+
 @pytest.mark.parametrize('url, expected', [
     # Test http.
     ('bzr+http://bzr.myproject.org/MyProject/trunk/#egg=MyProject',