Repository: pypa/pip

Issue:
pip19.0.1 list error"AttributeError: _version"
**Environment**

* pip version:19.0.1
* Python version:3.7
* OS:MacOS

<!-- Feel free to add more information about your environment here -->
I update pip from 10.1 to 19.0.1, and I wanna check my python packages, I input "pip list" then two errors occurred.
**Description**
<!-- A clear and concise description of what the bug is. -->

**Expected behavior**
<!-- A clear and concise description of what you expected to happen. -->
Exception:
Traceback (most recent call last):
  File "/usr/local/var/pyenv/versions/3.7.0/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2584, in version
    return self._version
  File "/usr/local/var/pyenv/versions/3.7.0/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2691, in __getattr__
    raise AttributeError(attr)
AttributeError: _version

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/var/pyenv/versions/3.7.0/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/usr/local/var/pyenv/versions/3.7.0/lib/python3.7/site-packages/pip/_internal/commands/list.py", line 148, in run
    self.output_package_listing(packages, options)
  File "/usr/local/var/pyenv/versions/3.7.0/lib/python3.7/site-packages/pip/_internal/commands/list.py", line 205, in output_package_listing
    data, header = format_for_columns(packages, options)
  File "/usr/local/var/pyenv/versions/3.7.0/lib/python3.7/site-packages/pip/_internal/commands/list.py", line 271, in format_for_columns
    row = [proj.project_name, proj.version]
  File "/usr/local/var/pyenv/versions/3.7.0/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2589, in version
    raise ValueError(tmpl % self.PKG_INFO, self)
ValueError: ("Missing 'Version:' header and/or METADATA file", Unknown [unknown version] (/usr/local/var/pyenv/versions/3.7.0/lib/python3.7/site-packages))
**How to Reproduce**
<!-- Describe the steps to reproduce this bug. -->

1. Get package from '...'
2. Then run '...'
3. An error occurs.

**Output**

```
Paste the output of the steps above, including the commands themselves and
pip's output/traceback etc.
```



Discussion:
there seems to be completely broken metadata in your python installation - can you try to run pip in pdb and get the project name/location, perhaps the metadata folder to see whats messing things up?
@zhouyu328 What's the output of the following command?
```bash
find /usr/local/var/pyenv/versions/3.7.0/lib/python3.7/site-packages/ -name METADATA
```
so i have been having the same probem so i experimented a bit and i think this could help:
i edited:
~/.local/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py
and added just before line 2589  a print so it looks like this
`
@property
    def version(self):
        try:
            return self._version
        except AttributeError:
            version = _version_from_file(self._get_metadata(self.PKG_INFO))
            if version is None:
                print(self._provider.__dict__) #this
                tmpl = "Missing 'Version:' header and/or %s file"
                raise ValueError(tmpl % self.PKG_INFO, self)
            return version`
the self is a object filled with None and Unknown data:
`{'project_name': 'Unknown', 'py_version': None, 'platform': None, 'location': '~/.local/lib/python3.6/site-packages', 'precedence': -1, '_provider': <pip._vendor.pkg_resources.PathMetadata object at 0x7f0faa49d4e0>, '_key': 'unknown'}
{'project_name': 'Unknown', 'py_version': None, 'platform': None, 'location': '~/.local/lib/python3.6/site-packages', 'precedence': -1, '_provider': <pip._vendor.pkg_resources.PathMetadata object at 0x7f0faa49d4e0>, '_key': 'unknown'}`

but the _provider returns this:
`{'module_path': '~/.local/lib/python3.6/site-packages', 'egg_info': '~/.local/lib/python3.6/site-packages/-rpcio-1.18.0.dist-info'}
{'module_path': '~/.local/lib/python3.6/site-packages', 'egg_info': '~/.local/lib/python3.6/site-packages/-rpcio-1.18.0.dist-info'}`

no idea why it's printed twice in both cases
i checked with
sudo apt install python3-grpcio
and i have it installed
> @zhouyu328 What's the output of the following command?
> 
> ```shell
> find /usr/local/var/pyenv/versions/3.7.0/lib/python3.7/site-packages/ -name METADATA
> ```

in my ~/.local/lib/python3.6/site-packages, but there are more folders han that and many have the samename but without the version.dist-info:

./pip-19.0.1.dist-info/METADATA
./lxml-4.3.0.dist-info/METADATA
./pickleshare-0.7.5.dist-info/METADATA
./asn1crypto-0.24.0.dist-info/METADATA
./keyrings.alt-3.1.1.dist-info/METADATA
./certifi-2018.10.15.dist-info/METADATA
./autopep8-1.4.3.dist-info/METADATA
./Pygments-2.3.1.dist-info/METADATA
./cligj-0.5.0.dist-info/METADATA
./jsonschema-2.6.0.dist-info/METADATA
./labelImg-1.8.1.dist-info/METADATA
./contextlib2-0.5.5.dist-info/METADATA
./pymacaroons-0.13.0.dist-info/METADATA
./six-1.10.0.dist-info/METADATA
./Glances-3.1.0.dist-info/METADATA
./cryptography-2.5.dist-info/METADATA
./mccabe-0.6.1.dist-info/METADATA
./jsoncomment-0.3.0.dist-info/METADATA
./ipykernel-5.1.0.dist-info/METADATA
./enum34-1.1.6.dist-info/METADATA
./docopt-0.6.2.dist-info/METADATA
./nbformat-4.4.0.dist-info/METADATA
./sympy-1.3.dist-info/METADATA
./cloudpickle-0.7.0.dist-info/METADATA
./pydocstyle-3.0.0.dist-info/METADATA
./wheel-0.32.3.dist-info/METADATA
./absl_py-0.7.0.dist-info/METADATA
./autokeras-0.3.6.dist-info/METADATA
./keyring-17.1.1.dist-info/METADATA
./grpcio_tools-1.10.1.dist-info/METADATA
./SecretStorage-3.1.1.dist-info/METADATA
./lws-1.2.dist-info/METADATA
./pyquery-1.4.0.dist-info/METADATA
./jeepney-0.4.dist-info/METADATA
./Keras_Preprocessing-1.0.2.dist-info/METADATA
./yarg-0.1.9.dist-info/METADATA
./blinker-1.4.dist-info/METADATA
./psutil-5.5.0.dist-info/METADATA
./libmagic-1.0.dist-info/METADATA
./h5py-2.9.0.dist-info/METADATA
./python_magic-0.4.15.dist-info/METADATA
./jupyter_console-6.0.0.dist-info/METADATA
./Click-7.0.dist-info/METADATA
./mistune-0.8.4.dist-info/METADATA
./pyflakes-2.0.0.dist-info/METADATA
./audioread-2.1.6.dist-info/METADATA
./inflect-2.1.0.dist-info/METADATA
./EbookLib-0.17.1.dist-info/METADATA
./networkx-2.2.dist-info/METADATA
./click_plugins-1.0.4.dist-info/METADATA
./grpcio-1.0.0.dist-info/METADATA
./unicode_slugify-0.1.3.dist-info/METADATA
./six-1.12.0.dist-info/METADATA
./scipy-1.2.0.dist-info/METADATA
./macaroonbakery-1.2.1.dist-info/METADATA
./tensorboard-1.12.0.dist-info/METADATA
./llvmlite-0.27.0.dist-info/METADATA
./qtconsole-4.4.3.dist-info/METADATA
./wcwidth-0.1.7.dist-info/METADATA
./requests-2.21.0.dist-info/METADATA
./ptyprocess-0.6.0.dist-info/METADATA
./pipreqs-0.4.9.dist-info/METADATA
./torchvision-0.2.1.dist-info/METADATA
./rope-0.11.0.dist-info/METADATA
./testpath-0.4.2.dist-info/METADATA
./pystache-0.5.4.dist-info/METADATA
./olefile-0.46.dist-info/METADATA
./Werkzeug-0.14.1.dist-info/METADATA
./defusedxml-0.5.0.dist-info/METADATA
./jupyter-1.0.0.dist-info/METADATA
./scikit_learn-0.20.2.dist-info/METADATA
./soupsieve-1.7.3.dist-info/METADATA
./pyRFC3339-1.1.dist-info/METADATA
./docker-3.7.0.dist-info/METADATA
./librosa-0.6.2.dist-info/METADATA
./cycler-0.10.0.dist-info/METADATA
./singledispatch-3.4.0.3.dist-info/METADATA
./python_language_server-0.22.0.dist-info/METADATA
./imageio-2.4.1.dist-info/METADATA
./influxdb-5.2.1.dist-info/METADATA
./pyflakes-2.1.0.dist-info/METADATA
./nbconvert-5.4.0.dist-info/METADATA
./backcall-0.1.0.dist-info/METADATA
./traitlets-4.3.2.dist-info/METADATA
./python_pptx-0.6.17.dist-info/METADATA
./bleach-3.1.0.dist-info/METADATA
./Keras-2.2.4.dist-info/METADATA
./cssselect-1.0.3.dist-info/METADATA
./joblib-0.13.1.dist-info/METADATA
./PyQt5_sip-4.19.13.dist-info/METADATA
./termcolor-1.1.0.dist-info/METADATA
./gast-0.2.2.dist-info/METADATA
./PyQt5-5.10.1.dist-info/METADATA
./argcomplete-1.9.4.dist-info/METADATA
./pluggy-0.8.1.dist-info/METADATA
./tensorboard-1.10.0.dist-info/METADATA
./urllib3-1.24.1.dist-info/METADATA
./Mako-1.0.7.dist-info/METADATA
./jupyter_core-4.4.0.dist-info/METADATA
./ipywidgets-7.4.2.dist-info/METADATA
./pyxdg-0.26.dist-info/METADATA
./simplejson-3.16.0.dist-info/METADATA
./Markdown-3.0.1.dist-info/METADATA
./tornado-5.1.1.dist-info/METADATA
./pandas-0.23.4.dist-info/METADATA
./louis-1.3.dist-info/METADATA
./pyinotify-0.9.6.dist-info/METADATA
./chardet-2.3.0.dist-info/METADATA
./netifaces-0.10.9.dist-info/METADATA
./yapf-0.25.0.dist-info/METADATA
./simplegeneric-0.8.1.dist-info/METADATA
./python_debian-0.1.34.dist-info/METADATA
./chardet-3.0.4.dist-info/METADATA
./httplib2-0.12.0.dist-info/METADATA
./python_jsonrpc_server-0.0.2.dist-info/METADATA
./widgetsnbextension-3.4.2.dist-info/METADATA
./prometheus_client-0.5.0.dist-info/METADATA
./sip-4.19.8.dist-info/METADATA
./mock-2.0.0.dist-info/METADATA
./pytesseract-0.2.5.dist-info/METADATA
./requests_file-1.4.3.dist-info/METADATA
./Keras_Preprocessing-1.0.5.dist-info/METADATA
./ipython-7.2.0.dist-info/METADATA
./lazr.uri-1.0.3.dist-info/METADATA
./requests_unixsocket-0.1.5.dist-info/METADATA
./notebook-5.7.4.dist-info/METADATA
./pytesseract-0.2.6.dist-info/METADATA
./Flask-1.0.2.dist-info/METADATA
./pycocotools-2.0.0.dist-info/METADATA
./pocketsphinx-0.1.15.dist-info/METADATA
./distro-1.3.0.dist-info/METADATA
./isort-4.3.4.dist-info/METADATA
./pysnmp-4.4.8.dist-info/METADATA
./pyparsing-2.3.1.dist-info/METADATA
./pycups-1.9.74.dist-info/METADATA
./PyJWT-1.7.1.dist-info/METADATA
./resampy-0.2.1.dist-info/METADATA
./xlrd-1.2.0.dist-info/METADATA
./snowballstemmer-1.2.1.dist-info/METADATA
./autopep8-1.4.2.dist-info/METADATA
./pycairo-1.18.0.dist-info/METADATA
./lazr.restfulclient-0.14.2.dist-info/METADATA
./tldextract-2.2.0.dist-info/METADATA
./snuggs-1.4.2.dist-info/METADATA
./launchpadlib-1.10.6.dist-info/METADATA
./jedi-0.13.2.dist-info/METADATA
./scour-0.37.dist-info/METADATA
./futures-3.1.1.dist-info/METADATA
./sshuttle-0.78.4.dist-info/METADATA
./Cython-0.29.3.dist-info/METADATA
./websocket_client-0.54.0.dist-info/METADATA
./SpeechRecognition-3.8.1.dist-info/METADATA
./GPUtil-1.4.0.dist-info/METADATA
./python_pptx-0.6.5.dist-info/METADATA
./configparser-3.5.0.dist-info/METADATA
./ply-3.11.dist-info/METADATA
./PyOpenGL-3.1.0.dist-info/METADATA
./pyzmq-17.1.2.dist-info/METADATA
./decorator-4.3.2.dist-info/METADATA
./pytz-2018.9.dist-info/METADATA
./binwalk-2.1.0.dist-info/METADATA
./PyYAML-3.13.dist-info/METADATA
./terminado-0.8.1.dist-info/METADATA
./python_jsonrpc_server-0.1.2.dist-info/METADATA
./torch-1.0.0.dist-info/METADATA
./pyqtgraph-0.10.0.dist-info/METADATA
./tensorflow-1.12.0.dist-info/METADATA
./prompt_toolkit-2.0.7.dist-info/METADATA
./cffi-1.11.5.dist-info/METADATA
./numpy-1.16.0.dist-info/METADATA
./youtube_dl-2019.1.24.dist-info/METADATA
./selenium-3.141.0.dist-info/METADATA
./wheel-0.32.2.dist-info/METADATA
./XlsxWriter-1.1.2.dist-info/METADATA
./pyparsing-2.3.0.dist-info/METADATA
./tensorflow-1.10.0.dist-info/METADATA
./Unidecode-1.0.23.dist-info/METADATA
./Jinja2-2.10.dist-info/METADATA
./bleach/_vendor/html5lib-1.0.1.dist-info/METADATA
./pbr-5.1.1.dist-info/METADATA
./textract-1.6.1.dist-info/METADATA
./docx2txt-0.7.dist-info/METADATA
./pysmi-0.3.3.dist-info/METADATA
./pocketsphinx-0.1.3.dist-info/pocketsphinx-0.1.3.dist-info/METADATA
./PyWavelets-1.0.1.dist-info/METADATA
./jupyter_client-5.2.4.dist-info/METADATA
./tqdm-4.29.1.dist-info/METADATA
./webencodings-0.5.1.dist-info/METADATA
./setuptools-40.6.3.dist-info/METADATA
./selenium_requests-1.3.dist-info/METADATA
./lightgbm-2.2.2.dist-info/METADATA
./oauthlib-3.0.1.dist-info/METADATA
./numba-0.42.0.dist-info/METADATA
./PyNaCl-1.3.0.dist-info/METADATA
./Keras_Applications-1.0.4.dist-info/METADATA
./defer-1.0.4.dist-info/METADATA
./xlrd-1.0.0.dist-info/METADATA
./ipython_genutils-0.2.0.dist-info/METADATA
./rasterio-1.0.10.dist-info/METADATA
./dask-1.1.0.dist-info/METADATA
./reportlab-3.5.13.dist-info/METADATA
./distro_info-0.0.0.dist-info/METADATA
./setuptools-39.1.0.dist-info/METADATA
./-rpcio-1.18.0.dist-info/grpcio-1.18.0.dist-info/METADATA
./rasterio-1.0.13.dist-info/METADATA
./MarkupSafe-1.1.0.dist-info/METADATA
./protobuf-3.6.1.dist-info/METADATA
./wadllib-1.3.3.dist-info/METADATA
./affine-2.2.2.dist-info/METADATA
./testresources-2.0.1.dist-info/METADATA
./future-0.17.1.dist-info/METADATA
./pyxattr-0.6.1.dist-info/METADATA
./docker_pycreds-0.4.0.dist-info/METADATA
./certifi-2018.11.29.dist-info/METADATA
./matplotlib-3.0.2.dist-info/METADATA
./beautifulsoup4-4.5.3.dist-info/METADATA
./mpmath-1.1.0.dist-info/METADATA
./Send2Trash-1.5.0.dist-info/METADATA
./itsdangerous-1.1.0.dist-info/METADATA
./Keras_Applications-1.0.6.dist-info/METADATA
./SpeechRecognition-3.6.3.dist-info/METADATA
./html5lib-1.0.1.dist-info/METADATA
./idna-2.8.dist-info/METADATA
./toolz-0.9.0.dist-info/METADATA
./pycparser-2.19.dist-info/METADATA
./kiwisolver-1.0.1.dist-info/METADATA
./entrypoints-0.3.dist-info/METADATA
./tensorboard-1.12.2.dist-info/METADATA
./pycodestyle-2.4.0.dist-info/METADATA
./astor-0.7.1.dist-info/METADATA
./pycryptodomex-3.7.3.dist-info/METADATA
./python_language_server-0.21.2.dist-info/METADATA
./PyGObject-3.30.4.dist-info/METADATA
./scikit_image-0.14.2.dist-info/METADATA
./pycrypto-2.6.1.dist-info/METADATA
./opencv_python-4.0.0.21.dist-info/METADATA
./bottle-0.12.16.dist-info/METADATA
./attrs-18.2.0.dist-info/METADATA
./pandocfilters-1.4.2.dist-info/METADATA
./pexpect-4.6.0.dist-info/METADATA
./pyasn1-0.4.5.dist-info/METADATA
./parso-0.3.2.dist-info/METADATA
./numpy-1.14.5.dist-info/METADATA
./nltk-3.4.dist-info/METADATA
./Pillow-5.4.1.dist-info/METADATA
./beautifulsoup4-4.7.1.dist-info/METADATA
./python_dateutil-2.7.5.dist-info/METADATA
These don't look right...
```
./pocketsphinx-0.1.3.dist-info/pocketsphinx-0.1.3.dist-info/METADATA
./-rpcio-1.18.0.dist-info/grpcio-1.18.0.dist-info/METADATA
```
@0000matteo0000 Does this fix your issue?
```bash
mv pocketsphinx-0.1.3.dist-info p; mv p/pocketsphinx-0.1.3.dist-info .; rm -r p
mv -rpcio-1.18.0.dist-info r; mv r/grpcio-1.18.0.dist-info .; rm -r r
```
> @0000matteo0000 Does this fix your issue?
> 
> ```shell
> mv pocketsphinx-0.1.3.dist-info p; mv p/pocketsphinx-0.1.3.dist-info .; rm -r p
> mv -rpcio-1.18.0.dist-info r; mv r/grpcio-1.18.0.dist-info .; rm -r r
> ```

yup thanks, i moved those and i had to also move pip itself but that fixed it it seems, could there be a bug in pip so that packages are put in the wrong folders? because i never moved anything around there
@uranusjr Do you know why packages might be getting installed with a leading hyphen? See the comment above (`-rpcio` instead of `grpcio`). Here is another example  (`-onfigparser`): https://github.com/pypa/pip/issues/6169#issuecomment-457537807
Hmm, no, nothing from the top of my head. I also tried but failed to reproduce this on various environments (Python 2.7/3.7, Mac/Windows, pip 19.0.0/19.0.1, venv/global). This seems to be a very specific problem :|
might be related, i use this script to update all pip packages

#!/bin/bash
#pip install --user --upgrade --force-reinstall --ignore-installed pip
pip install --user --upgrade pip
p=$(pip list | awk 'NR>2, ORS=" " {print $1;}')
#pip install --user --upgrade --force-reinstall --ignore-installed $p
pip install --user --upgrade $p

but if it mangled package names pip would just stop say package not found
and used the force reinstall and ignore installed commands on multiple packages trying to fix the "failed to import main from package pip" when i try to run it as sudo.
no problem running pip from the root user though
I am also getting some error using pip freeze. I am using windows 10, python 3.6.0 and pip version is 19.0.1. Please take a look:
```
`Error checking for conflicts.
Traceback (most recent call last):
  File "c:\users\rohit\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2584, in version
    return self._version
  File "c:\users\rohit\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2691, in __getattr__
    raise AttributeError(attr)
AttributeError: _version

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\rohit\appdata\local\programs\python\python36\lib\site-packages\pip\_internal\commands\install.py", line 503, in _warn_about_conflicts
    package_set, _dep_info = check_install_conflicts(to_install)
  File "c:\users\rohit\appdata\local\programs\python\python36\lib\site-packages\pip\_internal\operations\check.py", line 108, in check_install_conflicts
    package_set, _ = create_package_set_from_installed()
  File "c:\users\rohit\appdata\local\programs\python\python36\lib\site-packages\pip\_internal\operations\check.py", line 47, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "c:\users\rohit\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2589, in version
    raise ValueError(tmpl % self.PKG_INFO, self)
ValueError: ("Missing 'Version:' header and/or METADATA file", Unknown [unknown version] (c:\users\rohit\appdata\local\programs\python\python36\lib\site-packages))`
```
Any help will be appreciated. 
A couple comments:

* @zooba Is this related to #6169 which you are fixing? See https://github.com/pypa/pip/issues/6194#issuecomment-457716967 [link fixed] for the paths this issue is breaking on. Notice in particular the leading "-" in (`-rpcio-1.18.0.dist-info`).

* `pkg_resources` should probably be including the location of the errant METADATA (e.g. path to the file) in the `ValueError` message to make troubleshooting easier. Maybe someone can file an issue there.


The nested .dist-info directories is I think the last issue I'm working on in my PR.

Is `pkg_resources` allowing leading hyphen in package names? That's easily fixed by removing it from the "valid/invalid" character list.
> Is pkg_resources allowing leading hyphen in package names? 

I was guessing that issue might be on the pip side (e.g. pip deciding when to call `pkg_resources`). (I also meant to mention this point in my previous comment.) Or perhaps it could even be both maybe. Either way, it's another thing to look into as a follow-up to this issue. 
I too am getting the initial error mentioned by @zhouyu328. On inspection, the following packages of mine have a leading hyphen introduced:

```
-_pycache__
-ecorator-4.3.0.dist-info
-ix-1.11.0.dist-info
-plotly_utils
-uartz
-ynput-1.3.10.dist-info
-ytz-2018.5.dist-info
```

I, too, use a script to update all packages:
`sudo -H /Users/sterlingbutters/anaconda3/bin/pip freeze --local | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 /Users/sterlingbutters/anaconda3/bin/pip install -U`

Would appreciate info on fixing the now-broken configuration (I'm scared mostly to mess with the `__pycache__` dir)
the worst thing is terminals and pip itself will fail with packages starting with an hyphen because they will treat it as a parameter.
could some of the parameters given with double hyphen be badly interpreted so that one of them ends up in the package name?
also some packages got moved inside their own directory for some reason
and to me this happened just after the update to pip 19 from 18, if there are scripts managing the update of pip's environment from one to the other maybe the problem could lie there
> the worst thing is terminals and pip itself will fail with packages starting with an hyphen because they will treat it as a parameter.

That's a _very_ good point that I hadn't thought of. I'll drop the hyphen as a potential replacement character.

Resolving patch:
diff --git a/src/pip/_internal/req/req_uninstall.py b/src/pip/_internal/req/req_uninstall.py
--- a/src/pip/_internal/req/req_uninstall.py
+++ b/src/pip/_internal/req/req_uninstall.py
@@ -15,7 +15,7 @@
 from pip._internal.utils.logging import indent_log
 from pip._internal.utils.misc import (
     FakeFile, ask, dist_in_usersite, dist_is_local, egg_link_path, is_local,
-    normalize_path, renames,
+    normalize_path, renames, rmtree,
 )
 from pip._internal.utils.temp_dir import AdjacentTempDirectory
 
@@ -241,7 +241,10 @@ def _stash(self, path):
             best = AdjacentTempDirectory(os.path.dirname(path))
             best.create()
             self._save_dirs.append(best)
-        return os.path.join(best.path, os.path.relpath(path, best.original))
+        relpath = os.path.relpath(path, best.original)
+        if not relpath or relpath == os.path.curdir:
+            return best.path
+        return os.path.join(best.path, relpath)
 
     def remove(self, auto_confirm=False, verbose=False):
         """Remove paths in ``self.paths`` with confirmation (unless
@@ -265,6 +268,13 @@ def remove(self, auto_confirm=False, verbose=False):
                     new_path = self._stash(path)
                     logger.debug('Removing file or directory %s', path)
                     self._moved_paths.append((path, new_path))
+                    if os.path.isdir(path) and os.path.isdir(new_path):
+                        # If we're moving a directory, we need to
+                        # remove the destination first or else it will be
+                        # moved to inside the existing directory.
+                        # We just created new_path ourselves, so it will
+                        # be removable.
+                        os.rmdir(new_path)
                     renames(path, new_path)
                 for pth in self.pth.values():
                     pth.remove()
@@ -311,9 +321,13 @@ def rollback(self):
         logger.info('Rolling back uninstall of %s', self.dist.project_name)
         for path, tmp_path in self._moved_paths:
             logger.debug('Replacing %s', path)
+            if os.path.isdir(tmp_path) and os.path.isdir(path):
+                rmtree(path)
             renames(tmp_path, path)
         for pth in self.pth.values():
             pth.rollback()
+        for save_dir in self._save_dirs:
+            save_dir.cleanup()
 
     def commit(self):
         """Remove temporary save dir: rollback will no longer be possible."""
diff --git a/src/pip/_internal/utils/temp_dir.py b/src/pip/_internal/utils/temp_dir.py
--- a/src/pip/_internal/utils/temp_dir.py
+++ b/src/pip/_internal/utils/temp_dir.py
@@ -98,7 +98,11 @@ class AdjacentTempDirectory(TempDirectory):
 
     """
     # The characters that may be used to name the temp directory
-    LEADING_CHARS = "-~.+=%0123456789"
+    # We always prepend a ~ and then rotate through these until
+    # a usable name is found.
+    # pkg_resources raises a different error for .dist-info folder
+    # with leading '-' and invalid metadata
+    LEADING_CHARS = "-~.=%0123456789"
 
     def __init__(self, original, delete=None):
         super(AdjacentTempDirectory, self).__init__(delete=delete)
@@ -117,8 +121,8 @@ def _generate_names(cls, name):
             if name[i] in cls.LEADING_CHARS:
                 continue
             for candidate in itertools.combinations_with_replacement(
-                    cls.LEADING_CHARS, i):
-                new_name = ''.join(candidate) + name[i:]
+                    cls.LEADING_CHARS, i - 1):
+                new_name = '~' + ''.join(candidate) + name[i:]
                 if new_name != name:
                     yield new_name
 


Test patch:
diff --git a/tests/unit/test_utils.py b/tests/unit/test_utils.py
--- a/tests/unit/test_utils.py
+++ b/tests/unit/test_utils.py
@@ -573,8 +573,10 @@ def names():
         assert not any(n == name for n in names())
 
         # Check the first group are correct
-        assert all(x == y for x, y in
-                   zip(some_names, [c + name[1:] for c in chars]))
+        expected_names = ['~' + name[1:]]
+        expected_names.extend('~' + c + name[2:] for c in chars)
+        for x, y in zip(some_names, expected_names):
+            assert x == y
 
 
 class TestGlibc(object):