Repository: pypa/pip

Issue:
'METADATA' crash during 'pip install'
* pip version:20.2
* Python version:3.8.4
* Operating system:windows 10

```
C:\Users\elcot>pip install scipy
Requirement already satisfied: scipy in c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages (1.4.1)
Requirement already satisfied: numpy>=1.13.3 in c:\users\elcot\appdata\roaming\python\python38\site-packages (from scipy) (1.19.0)
ERROR: Error while checking for conflicts. Please file an issue on pip's issue tracker: https://github.com/pypa/pip/issues/new
Traceback (most recent call last):
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3021, in _dep_map
    return self.__dep_map
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3012, in _parsed_pkg_info
    return self._pkg_info
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _pkg_info

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\commands\install.py", line 535, in _determine_conflicts
    return check_install_conflicts(to_install)
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\operations\check.py", line 108, in check_install_conflicts
    package_set, _ = create_package_set_from_installed()
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\operations\check.py", line 50, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2736, in requires
    dm = self._dep_map
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3023, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3032, in _compute_dependencies
    for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3014, in _parsed_pkg_info
    metadata = self.get_metadata(self.PKG_INFO)
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1420, in get_metadata
    value = self._get(path)
  File "c:\users\elcot\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1616, in _get
    with open(path, 'rb') as stream:
PermissionError: [Errno 13] Permission denied: 'c:\\users\\elcot\\appdata\\roaming\\python\\python38\\site-packages\\astroid-2.4.2.dist-info\\METADATA'
```


Discussion:
```
PermissionError: [Errno 13] Permission denied:
'c:\\users\\elcot\\appdata\\roaming\\python\\python38\\site-packages\\astroid-2.4.2.dist-info\\METADATA'
```

Are you able to open this file by hand in Notepad.exe (or any other editor)? It seems weird that this file cannot be read; this may be an environment issue, not a pip bug (although we should catch it and improve the error message).
I wonder why we’re suddenly seeing this in 20.2; there’s nothing new in the check operation AFAICT. Did something change in `pkg_resources`? Can this somehow be related to #8054?
> I wonder why we’re suddenly seeing this in 20.2; there’s nothing new in the check operation AFAICT. Did something change in `pkg_resources`? Can this somehow be related to #8054?

Hi @KishoreKumarRV ,

As per the above comment, could you also try installing pip from the latest master branch, (`pip install https://github.com/pypa/pip/archive/master.zip`) and see if that resolves the error. 

There was a regression bug which might have been introduced by the above PR which was fixed by https://github.com/pypa/pip/pull/8659, and could be the cause of this bug.
I get the same error too. I get it with basically any package, here I tried to install pyinstaller

```
PS C:\Users\user> pip install pyinstaller
Requirement already satisfied: pyinstaller in c:\users\user\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages (3.6)
Requirement already satisfied: altgraph in c:\users\user\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages (from pyinstaller) (0.17)
Requirement already satisfied: pywin32-ctypes>=0.2.0 in c:\users\user\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages (from pyinstaller) (0.2.0)
Requirement already satisfied: setuptools in c:\program files\windowsapps\pythonsoftwarefoundation.python.3.8_3.8.1520.0_x64__qbz5n2kfra8p0\lib\site-packages (from pyinstaller) (47.1.0)
Requirement already satisfied: pefile>=2017.8.1 in c:\users\user\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages (from pyinstaller) (2019.4.18)
Requirement already satisfied: future in c:\users\user\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages (from pefile>=2017.8.1->pyinstaller) (0.18.2)
ERROR: Error while checking for conflicts. Please file an issue on pip's issue tracker: https://github.com/pypa/pip/issues/new
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3021, in _dep_map
    return self.__dep_map
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3012, in _parsed_pkg_info
    return self._pkg_info
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _pkg_info

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_internal\commands\install.py", line 534, in _determine_conflicts
    return check_install_conflicts(to_install)
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_internal\operations\check.py", line 108, in check_install_conflicts
    package_set, _ = create_package_set_from_installed()
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_internal\operations\check.py", line 50, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2736, in requires
    dm = self._dep_map
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3023, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3032, in _compute_dependencies
    for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3014, in _parsed_pkg_info
    metadata = self.get_metadata(self.PKG_INFO)
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1420, in get_metadata
    value = self._get(path)
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1616, in _get
    with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: 'c:\\users\\user\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\\localcache\\local-packages\\python38\\site-packages\\pypykatz-0.3.12.dist-info\\METADATA
```
> Are you able to open this file by hand in Notepad.exe (or any other editor)? It seems weird that this file cannot be read; this may be an environment issue, not a pip bug (although we should catch it and improve the error message).

Can *anyone* here saying you have this problem try to do this?
Hi @uranusjr 

I have the same problem, this is my bug
```
C:\Users\ar_hv>pip install https://github.com/pypa/pip/archive/master.zip
Collecting https://github.com/pypa/pip/archive/master.zip
  Using cached https://github.com/pypa/pip/archive/master.zip
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied (use --upgrade to upgrade): pip==20.3.dev0 from https://github.com/pypa/pip/archive/master.zip in c:\users\ar_hv\anaconda3\lib\site-packages
Building wheels for collected packages: pip
  Building wheel for pip (PEP 517) ... done
  Created wheel for pip: filename=pip-20.3.dev0-py2.py3-none-any.whl size=1503758 sha256=01687db60f3b36becccc2fb2e5aaea14119f22e3e11f21249feb9a88d8b7a7c8
  Stored in directory: C:\Users\ar_hv\AppData\Local\Temp\pip-ephem-wheel-cache-wbew0v2i\wheels\b2\f0\ae\286fb76d950bd0a0d20bcabbda0f56531389ed5030f038f6b9
Successfully built pip
ERROR: Error while checking for conflicts. Please file an issue on pip's issue tracker: https://github.com/pypa/pip/issues/new
Traceback (most recent call last):
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3021, in _dep_map
    return self.__dep_map
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3012, in _parsed_pkg_info
    return self._pkg_info
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _pkg_info

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_internal\commands\install.py", line 534, in _determine_conflicts
    return check_install_conflicts(to_install)
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_internal\operations\check.py", line 108, in check_install_conflicts
    package_set, _ = create_package_set_from_installed()
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_internal\operations\check.py", line 50, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2736, in requires
    dm = self._dep_map
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3023, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3032, in _compute_dependencies
    for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3014, in _parsed_pkg_info
    metadata = self.get_metadata(self.PKG_INFO)
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1420, in get_metadata
    value = self._get(path)
  File "c:\users\ar_hv\anaconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1616, in _get
    with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: 'c:\\users\\ar_hv\\anaconda3\\lib\\site-packages\\s3transfer-0.3.3.dist-info\\METADATA'
```

I tried opening the file by hand and realized that inside anaconda3 I don't have a folder named "lib" just one named "Lib". This is the path for the file `C:\Users\ar_hv\anaconda3\Lib\site-packages\s3transfer-0.3.3.dist-info\metadata.json` . I don't know if that's supposed to be the file but is the only one I found.

I appreciate your help
Case does not matter in Windows paths, so `Lib` and `lib` are equivalent. So you only have `metadata.json`, but not `METADATA` (no extension)?
> > Are you able to open this file by hand in Notepad.exe (or any other editor)? It seems weird that this file cannot be read; this may be an environment issue, not a pip bug (although we should catch it and improve the error message).
> 
> Can _anyone_ here saying you have this problem try to do this?

Sorry, I've been on vacation. That file doesn't even exist in my case. The path only exists till "site-packages".  It's probably not a pip error, it's my fault.
If @Adrianahuante’s observation is correct, this is probably pip’s fault, not yours (i.e. if pip correctly installs some files for the distribution but not all). I wonder if this has something to do with the change in 20.2 that wheels are now installed directly from the archive.
Yes @uranusjr , in that folder I have two files: DESCRIPTION.rst and metadata.json. I don't have the METADATA file (with no extention) 

![image](https://user-images.githubusercontent.com/48062038/90145777-e9032200-dd45-11ea-9610-75008bb00f02.png)

Hmm, that does look a lot like a wheel installation failed half-way. So there are three issues here:

1. pip should avoid this from happening. There is already a rollback feature that cleans files up if installation fails, so we need to investigate why it did not kick in for you.
2. Why does the installation fail? Wheel installation is simply ZIP extraction, and there’s very little chance it could fail unexpectedly. Tagging @chrahunt since you’re the one most familiar with the new implementation.
3. pip should be more resilient when this happens. There happens to be a [discussion about making `METADATA` optional in `.dist-info`](https://discuss.python.org/t/pep-627-updating-pep-376-making-record-optional-in-installed-dist-info/4126/16) (for entirely unrelated reasons), and this would be a good chance to implement it. But the logic is in `pkg_resources`, and since upstream fixes probably won’t land soon (new setuptools is Python 3-only, pip is currently vendoring an old version of it), we’ll need to do vendor-patching.
I don't know what went wrong in the installation but I unistalled anaconda and installed it again. That solved the problem, although now I don't have the folder `C:\Users\ar_hv\anaconda3\Lib\site-packages\s3transfer-0.3.3.dist-info`

Thanks!
To folks facing this issue, could you provide some context on how to reproduce this failure?

Does this require interrupting the installation in an earlier run to cause failures in subsequent runs? 
It is always preferred to install TF using conda and not 'pip' because it causes dependencies issues.
I faced the same problem when I tried to install TF using 'pip' inside a 'conda' env.

For a temporary fix, I created the missing files inside the file named 'METADATA' (with no extension) and it ran without any error.
Idk how reliable the solution is, but yes it worked for me. 
  
I have this error on arch GNU/Linux 
doing pip install salt

```
 pip3 install salt
Defaulting to user installation because normal site-packages is not writeable
Processing ./.cache/pip/wheels/a7/12/bc/b4f43921fe4940f3fa5bf478951dea3fb4add1f79033a9e73e/salt-3001.1-py3-none-any.whl
Requirement already satisfied: distro in /usr/lib/python3.8/site-packages (from salt) (1.5.0)
Requirement already satisfied: requests>=1.0.0 in /usr/lib/python3.8/site-packages (from salt) (2.24.0)
Requirement already satisfied: pycryptodomex>=3.9.7 in /usr/lib/python3.8/site-packages (from salt) (3.9.7)
Requirement already satisfied: msgpack!=0.5.5,>=0.5 in /usr/lib/python3.8/site-packages (from salt) (1.0.0)
Requirement already satisfied: pyzmq>=17.0.0 in /usr/lib/python3.8/site-packages (from salt) (19.0.1)
Requirement already satisfied: Jinja2 in /usr/lib/python3.8/site-packages (from salt) (2.11.2)
Requirement already satisfied: PyYAML in /usr/lib/python3.8/site-packages (from salt) (5.3.1)
Requirement already satisfied: MarkupSafe in /usr/lib/python3.8/site-packages (from salt) (1.1.1)
Requirement already satisfied: chardet>=3.0.2 in /usr/lib/python3.8/site-packages (from requests>=1.0.0->salt) (3.0.4)
Requirement already satisfied: idna>=2.5 in /usr/lib/python3.8/site-packages (from requests>=1.0.0->salt) (2.10)
Requirement already satisfied: urllib3>=1.21.1 in /usr/lib/python3.8/site-packages (from requests>=1.0.0->salt) (1.25.10)
ERROR: Error checking for conflicts.
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2720, in _dep_map
    return self.__dep_map
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2835, in __getattr__
    raise AttributeError(attr)
AttributeError: _Distribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/packaging/requirements.py", line 98, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/usr/lib/python3.8/site-packages/pyparsing.py", line 1955, in parseString
    raise exc
  File "/usr/lib/python3.8/site-packages/pyparsing.py", line 3814, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pyparsing.ParseException: Expected stringEnd, found ','  (at char 1), (line:1, col:2)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 512, in _warn_about_conflicts
    package_set, _dep_info = check_install_conflicts(to_install)
  File "/usr/lib/python3.8/site-packages/pip/_internal/operations/check.py", line 114, in check_install_conflicts
    package_set, _ = create_package_set_from_installed()
  File "/usr/lib/python3.8/site-packages/pip/_internal/operations/check.py", line 53, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2756, in requires
    dm = self._dep_map
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2722, in _dep_map
    self.__dep_map = self._filter_extras(self._build_dep_map())
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2751, in _build_dep_map
    dm.setdefault(extra, []).extend(parse_requirements(reqs))
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3109, in parse_requirements
    yield Requirement(line)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3119, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/usr/lib/python3.8/site-packages/packaging/requirements.py", line 100, in __init__
    raise InvalidRequirement(
packaging.requirements.InvalidRequirement: Parse error at "','": Expected stringEnd
Installing collected packages: salt
```
> It is always preferred to install TF using conda and not 'pip' because it causes dependencies issues.

@immohann, have you tried `pip --use-feature=2020-resolver`?

@r3k2, could you please check if the problem persists using the latest version of pip from PyPI?  Something might went wrong during your distro's devendoring process.
> > Are you able to open this file by hand in Notepad.exe (or any other editor)? It seems weird that this file cannot be read; this may be an environment issue, not a pip bug (although we should catch it and improve the error message).
> 
> Can _anyone_ here saying you have this problem try to do this?

I try with another path and get it:
![image](https://user-images.githubusercontent.com/37663144/92068049-78ab5780-ed6b-11ea-8168-4a53f8f44828.png)

![image](https://user-images.githubusercontent.com/37663144/92068034-6e895900-ed6b-11ea-8ca9-f2d2ce3c2b45.png)

@KishoreKumarRV Did you have `prompt-toolkit` installed before `pip install --upgrade scipy`? If so, did `prompt-toolkit` install successfully?
@uranusjr I tried `pip install --upgrade scipy` and it is still giving me the same error


Resolving patch:
diff --git a/src/pip/_internal/operations/check.py b/src/pip/_internal/operations/check.py
--- a/src/pip/_internal/operations/check.py
+++ b/src/pip/_internal/operations/check.py
@@ -45,8 +45,8 @@ def create_package_set_from_installed(**kwargs):
         name = canonicalize_name(dist.project_name)
         try:
             package_set[name] = PackageDetails(dist.version, dist.requires())
-        except RequirementParseError as e:
-            # Don't crash on broken metadata
+        except (OSError, RequirementParseError) as e:
+            # Don't crash on unreadable or broken metadata
             logger.warning("Error parsing requirements for %s: %s", name, e)
             problems = True
     return package_set, problems


Test patch: