Repository: pypa/pip

Issue:
pip freeze returns "Permission denied: 'hg'" after using pip install -e
**Environment**

* pip version: 19.3.1 and above
* Python version: 3.7.7
* OS: macOS 10.14.6

**Description**
After using `pip install -e` to install a local package, `pip freeze` returns an error: `PermissionError: [Errno 13] Permission denied: 'hg'`.

I do not use Mercurial and do not have an `hg` executable on my machine.

It happens with pip 19.3.1 and above, when using `-e`.

It doesn't happen with pip 19.2.3, and it doesn't happen if I don't use `-e`. It doesn't happen with `pip list`.

**Expected behavior**
`pip freeze` should return the list of packages.

**How to Reproduce**
Create a simple package, install it using `pip install -e`, then run `pip freeze`.
It works with pip 19.2.3 but in 19.3.1 or later versions, `pip freeze` returns an error.

You can just copy/paste the following, which creates a virtual environment and does the `pip install -e` and the `pip freeze` on two different versions of pip:
```
cd
mkdir testdir
cd testdir/
virtualenv testenv
source testenv/bin/activate
mkdir my_plugin
echo "
from setuptools import setup

setup(
    name='my_plugin',
    py_modules=['my_plugin'],
    version='1.0.0',
)
" > my_plugin/setup.py
pip install pip==19.2.3
pip -V
pip install -e my_plugin/
pip freeze
pip uninstall -y my_plugin
pip freeze
pip install pip==19.3.1
pip -V
pip install -e my_plugin/
pip freeze
pip uninstall -y my_plugin
pip freeze
```

**Output**
Here's the output of the above commands, starting at the first `pip -V`:
```
$ pip -V
pip 19.2.3 from /Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip (python 3.7)
$ pip install -e my_plugin/
Obtaining file:///Users/<username>/testdir/my_plugin
Installing collected packages: my-plugin
  Running setup.py develop for my-plugin
Successfully installed my-plugin
WARNING: You are using pip version 19.2.3, however version 20.2b1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ pip freeze
# Editable install with no version control (my-plugin==1.0.0)
-e /Users/<username>/testdir/my_plugin
$ pip uninstall -y my_plugin
Uninstalling my-plugin-1.0.0:
  Successfully uninstalled my-plugin-1.0.0
$ pip freeze
$ pip install pip==19.3.1
Collecting pip==19.3.1
  Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 19.2.3
    Uninstalling pip-19.2.3:
      Successfully uninstalled pip-19.2.3
Successfully installed pip-19.3.1
$ pip -V
pip 19.3.1 from /Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip (python 3.7)
$ pip install -e my_plugin/
Obtaining file:///Users/<username>/testdir/my_plugin
Installing collected packages: my-plugin
  Running setup.py develop for my-plugin
Successfully installed my-plugin
WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ pip freeze
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 153, in _main
    status = self.run(options, args)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/commands/freeze.py", line 100, in run
    for line in freeze(**freeze_kwargs):
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/operations/freeze.py", line 70, in freeze
    req = FrozenRequirement.from_dist(dist)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/operations/freeze.py", line 249, in from_dist
    req, editable, comments = get_requirement_info(dist)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/operations/freeze.py", line 188, in get_requirement_info
    vcs_backend = vcs.get_backend_for_dir(location)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 234, in get_backend_for_dir
    if vcs_backend.controls_location(location):
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/vcs/mercurial.py", line 149, in controls_location
    log_failed_cmd=False)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 632, in run_command
    log_failed_cmd=log_failed_cmd)
  File "/Users/<username>/testdir/testenv/lib/python3.7/site-packages/pip/_internal/utils/subprocess.py", line 190, in call_subprocess
    stdout=subprocess.PIPE, cwd=cwd, env=env,
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: 'hg'
$ pip uninstall -y my_plugin
Uninstalling my-plugin-1.0.0:
  Successfully uninstalled my-plugin-1.0.0
$ pip freeze
$ 
```

`pip list` still works. In fact, if I add `pip list` just before the problematic `pip freeze` in the chain of commands above, there is no error:
```
$ pip list
Package    Version
---------- -------
my-plugin  1.0.0  
pip        19.3.1 
setuptools 47.1.1 
wheel      0.34.2 
WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ pip freeze
my-plugin==1.0.0
```

Discussion:
sbidoul: Thanks for addressing this.

By chance do you know why the issue does not surface in other commands that would attempt to use `hg` (such as `pip install` or `pip wheel`) ? It is somewhat surprising that only `pip freeze` shows the stack trace in such case.

Regarding the documentation update, it feels somewhat out of place in the pip freeze page, which otherwise does not talk about VCS. The place that talks bout VCS is the pip install page which is not ideal either if the error occurs in pip freeze. Perhaps we should not go that far and consider it is not the purpose of the pip documentation to explain how to resolve VCS installation issues ?
laurent-laporte-pro: ## Why `hg` fails but not the other commands?

Here's how I got the problem with Mercurial:

The `hg` command wasn't available on my Mac, so I downloaded [SourceTree](https://www.sourcetreeapp.com/) and installed it locally, in my user account. Then I just added a symbolic link in `/usr/local/bin`.

Subsequently, I created a new user account on my Mac. On this account, the `hg` command is accessible because it is a symbolic link installed globally in `/usr/local/bin`, but the target of the link wasn't. This is why I got the "permission denied" error.

When you download SourceTree, you have a simple ZIP file that you install as you want, there is no installer that would prompt you to install the application globally. I think there are a lot of Mac (or Windows) users who want to "try" Mercurial, and therefore install it locally, not globally.

We don't have this kind of problem with Git or Subversion because these two tools are already installed on Mac, even if the version is a bit old. Regarding Bazaar, I didn't get an error because this command is simply not installed.

As a conclusion, the issue with `pip freeze` can also happen with `bzr` (on Mac), or the other commands in Windows since those commands are not pre-installed.

## Why `pip freeze` fails but `pip install` or `pip wheel` don't?

Actually, when I posted my pull request I didn't know why `pip install` and `pip wheel` work. I quickly tested these commands before posting, and only `pip freeze` has an issue.

Further analysis shows:

- that only the `freeze.get_requirement_info` function calls the `VcsSupport.get_backend_for_dir` function: this function searches for the VCS that best corresponds to the directory passed as a parameter.
- the 'pip freeze' and 'pip wheel' functions determine which VCS to use based on the download url.

## Documentation

The documentation update was suggested by Tzu-ping Chung [(See message)](https://github.com/pypa/pip/issues/8418#issuecomment-757317449). Since it is a `pip freeze`-only issue, I decided to write it in `docs/man/commands/freeze.rst`.

If you prefer, I can move it under _Installing from local packages_ section.


laurent-laporte-pro: @sbidoul @uranusjr any news about thaïs PR?
faph: Could I just check, when is this likely to be released?
uranusjr: The next feature release is scheduled in April, so don't hold your breath for it just yet.
BrownTruck: Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the ``master`` branch into this pull request or rebase this pull request against ``master`` then it will be eligible for code review and hopefully merging!

Patch:
diff --git a/docs/html/cli/pip_freeze.rst b/docs/html/cli/pip_freeze.rst
index 352f7d321..3533db793 100644
--- a/docs/html/cli/pip_freeze.rst
+++ b/docs/html/cli/pip_freeze.rst
@@ -72,3 +72,21 @@ Examples
 
          env1\bin\python -m pip freeze > requirements.txt
          env2\bin\python -m pip install -r requirements.txt
+
+
+Fixing "Permission denied:" errors
+==================================
+
+The purpose of this section of documentation is to provide practical
+suggestions to users seeing a `"Permission denied" error <https://github.com/pypa/pip/issues/8418>`__ on ``pip freeze``.
+
+This error occurs, for instance, when the command is installed only for another
+user, and the current user doesn't have the permission to execute the other
+user's command.
+
+To solve that issue, you can try one of the followings:
+
+- Install the command for yourself (e.g. in your home directory).
+- Ask the system admin to allow this command for all users.
+- Check and correct the PATH variable of your own environment.
+- Check the `ACL (Access-Control List) <https://en.wikipedia.org/wiki/Access-control_list>`_ for this command.
diff --git a/news/8418.bugfix.rst b/news/8418.bugfix.rst
new file mode 100644
index 000000000..1bcc9b787
--- /dev/null
+++ b/news/8418.bugfix.rst
@@ -0,0 +1,1 @@
+Fix ``pip freeze`` permission denied error in order to display an understandable error message and offer solutions.
diff --git a/news/8418.doc.rst b/news/8418.doc.rst
new file mode 100644
index 000000000..6634f6cd6
--- /dev/null
+++ b/news/8418.doc.rst
@@ -0,0 +1,1 @@
+Add a section in the documentation to suggest solutions to the ``pip freeze`` permission denied issue.
diff --git a/src/pip/_internal/vcs/versioncontrol.py b/src/pip/_internal/vcs/versioncontrol.py
index 7600f6fef..cd6213bb1 100644
--- a/src/pip/_internal/vcs/versioncontrol.py
+++ b/src/pip/_internal/vcs/versioncontrol.py
@@ -684,6 +684,18 @@ class VersionControl:
             raise BadCommand(
                 f'Cannot find command {cls.name!r} - do you have '
                 f'{cls.name!r} installed and in your PATH?')
+        except PermissionError:
+            # errno.EACCES = Permission denied
+            # This error occurs, for instance, when the command is installed
+            # only for another user. So, the current user don't have
+            # permission to call the other user command.
+            raise BadCommand(
+                f"No permission to execute {cls.name!r} - install it "
+                f"locally, globally (ask admin), or check your PATH. "
+                f"See possible solutions at "
+                f"https://pip.pypa.io/en/latest/reference/pip_freeze/"
+                f"#fixing-permission-denied."
+            )
 
     @classmethod
     def is_repository_directory(cls, path):


Test patch:
diff --git a/tests/unit/test_vcs.py b/tests/unit/test_vcs.py
index 8fe5d3e71..6d82e139a 100644
--- a/tests/unit/test_vcs.py
+++ b/tests/unit/test_vcs.py
@@ -303,6 +303,27 @@ def test_version_control__get_url_rev_and_auth__no_revision(url):
     assert 'an empty revision (after @)' in str(excinfo.value)
 
 
+@pytest.mark.parametrize("vcs_cls", [Bazaar, Git, Mercurial, Subversion])
+@pytest.mark.parametrize(
+    "exc_cls, msg_re",
+    [
+        (FileNotFoundError, r"Cannot find command '{name}'"),
+        (PermissionError, r"No permission to execute '{name}'"),
+    ],
+    ids=["FileNotFoundError", "PermissionError"],
+)
+def test_version_control__run_command__fails(vcs_cls, exc_cls, msg_re):
+    """
+    Test that ``VersionControl.run_command()`` raises ``BadCommand``
+    when the command is not found or when the user have no permission
+    to execute it. The error message must contains the command name.
+    """
+    with patch("pip._internal.vcs.versioncontrol.call_subprocess") as call:
+        call.side_effect = exc_cls
+        with pytest.raises(BadCommand, match=msg_re.format(name=vcs_cls.name)):
+            vcs_cls.run_command([])
+
+
 @pytest.mark.parametrize('url, expected', [
     # Test http.
     ('bzr+http://bzr.myproject.org/MyProject/trunk/#egg=MyProject',