Repository: pypa/pip

Issue:
Upgrading pip fails on Windows when install path is too long
Received the following log on https://github.com/Microsoft/PTVS/issues/782.

```
Virtual environment is being created at 'C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env'
Virtual environment was successfully created at 'C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env'
----- Installing 'pip' -----
You are using pip version 6.0.8, however version 7.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-7.1.2-py2.py3-none-any.whl#md5=5ff9fec0be479e4e36df467556deed4d
  Using cached pip-7.1.2-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 6.0.8
    Uninstalling pip-6.0.8:
      Exception:
      Traceback (most recent call last):
        File "C:\Python34\lib\shutil.py", line 523, in move
          os.rename(src, real_dst)
      FileNotFoundError: [WinError 3] The system cannot find the path specified: 'c:\\users\\trevorsullivan\\source\\repos\\pythonapplication11\\pythonapplication11\\env\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\_implementation.cpython-34.pyc' -> 'C:\\Users\\TREVOR~1\\AppData\\Local\\Temp\\pip-h64zdfhc-uninstall\\users\\trevorsullivan\\source\\repos\\pythonapplication11\\pythonapplication11\\env\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\_implementation.cpython-34.pyc'

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\basecommand.py", line 232, in main
          status = self.run(options, args)
        File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\commands\install.py", line 347, in run
          root=options.root_path,
        File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\req\req_set.py", line 543, in install
          requirement.uninstall(auto_confirm=True)
        File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\req\req_install.py", line 667, in uninstall
          paths_to_remove.remove(auto_confirm)
        File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\req\req_uninstall.py", line 126, in remove
          renames(path, new_path)
        File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\utils\__init__.py", line 316, in renames
          shutil.move(old, new)
        File "C:\Python34\lib\shutil.py", line 535, in move
          copy2(src, real_dst)
        File "C:\Python34\lib\shutil.py", line 245, in copy2
          copyfile(src, dst, follow_symlinks=follow_symlinks)
        File "C:\Python34\lib\shutil.py", line 109, in copyfile
          with open(dst, 'wb') as fdst:
      FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\TREVOR~1\\AppData\\Local\\Temp\\pip-h64zdfhc-uninstall\\users\\trevorsullivan\\source\\repos\\pythonapplication11\\pythonapplication11\\env\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\_implementation.cpython-34.pyc'
----- Failed to install 'pip' -----
```

The problem seems to be that the entire install path is replicated beneath `TEMP`, which very quickly exceeds the maximum path length supported by Windows. I guess the aim is to be able to rollback a failed uninstall (which also fails here, and leaves corrupt state), but we may need an alternative to including the full path - maybe generate some sort of map file as well?



Discussion:
Related to #2892

I am still experiencing the same issue even though I am using pip 7.1.2 and python 3.5.1.

Are there any updates available regarding this issue?

Just ran into this under pip 8.1.1 and Python 3.5.2 on Windows 10.

I am wondering if a work around for Windows users might not be to check for paths that are likely to exceed the limit, (260 chars total), and using `subst` to mount a drive at a suitable point, i.e. identify a spare drive letter and mount that letter to as the `TEMP` - of course this would not truly fix the problem and deeply nested packages could still hit problems - that is down to Microsoft actually addressing the MAXPATH issue once and for all.  If the installation succeeds then the temporary area could be deleted and the `subst` deleted. In the case of a `TEMP` location if it fails then the path on the "new" drive could be used for recovery before deletion and the `subst` deleted.  Of course it cannot help someone who already has 23 drive letters mapped _A: B: & C: are reserved!_

In the case of a virtual environment facing the issue I think that the best that can be done is to issue the user an error with a suggestion that they make the virtual environment with `--relocatable` and either move it higher up the directory tree or use `subst` to put it at the top of a "new" drive.

I have just done a quick test on a Windows 10 (Anniversary Update) 64 bit machine and it would quite happily allow me to create a directory 217 characters long, (at which point the next `mkdir` failed and then mount it with 'subst' to a new drive letter and create a path 117 characters long on that drive - all that I had time for.

FWIW, Windows 10 allows the option to disable MAX_PATH limits for "self-certified" applications, which will include Python 3.6. Currently there's a machine policy that needs to be enabled, and the Python 3.6 installer will prompt users to do that (if they can - need to be admin).

So the issue will go away in the future, and anything else we do now is mitigation for people on existing setups (which obviously has a lot of value, since it's going to be a long time before _everyone_ is on Python 3.6 on Windows 10).

@zooba - My reading so far suggests that the disable MAX_PATH limits option is only planned to be available _or possibly only accessible_ in Widows 10 Pro and Enterprise editions not the home edition.

@GadgetSteve Where are you reading that? It's highly unlikely that such core APIs would be different between Windows SKUs, though of course there's no support included in the home editions for managing group policy. Shouldn't prevent the registry edits from working though.

@zooba It looks like the only way to enable the change for the user is via the Group Policy Editor which is missing from the Home editions - I have had too many changes that should work via registry edits fail to work, revert on a MS Update or work erratically, _(e.g. I cannot get the numb-lock to survive a reboot on Win 10 even after the registry edits),_ to be completely happy with the idea of relying on them.

@pfmoore Any ideas on what could be a possible workaround for this?


On 21/08/2017 11:29, Pradyun Gedam wrote:
> @pfmoore <https://github.com/pfmoore> Any ideas on what could be a 
> possible workaround for this?
>
If you are on Windows 10 applying the changes detailed at 
https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/ 
is a very good option.

Otherwise options include:

Installing python to a short path, e.g. C:\Py36 for Python 3.6 and 
setting temp to c:\tmp can sometimes help.

If the path that is too long is the cashe path you can use `pip 
--cache-dir=C:\PyCashe` and the --root and --prefix options can also be 
helpful and sometimes you can make a long path into a short one with the 
command line subst and mklink commands.

Of course there is always the option to switch to Linux...
-- 
Steve (Gadget) Barnes
Any opinions in this message are my personal opinions and do not reflect 
those of my employer.

---
This email has been checked for viruses by AVG.
http://www.avg.com




On 31/10/2016 16:52, Steve Dower wrote:
> @GadgetSteve <https://github.com/GadgetSteve> Where are you reading 
> that? It's highly unlikely that such core APIs would be different 
> between Windows versions, though of course there's no support included 
> in the home editions for managing group policy. Shouldn't prevent the 
> registry edits from working though.
> 
The registry edits do work the early story that I was reading claimed 
that the fix would not be available other than via the group policy editor.

-- 
Steve (Gadget) Barnes
Any opinions in this message are my personal opinions and do not reflect 
those of my employer.

---
This email has been checked for viruses by AVG.
http://www.avg.com


Thanks @GadgetSteve! I'll try to summarise what I've understood. Here's how one can workaround this issue as of today:

- If you can use Python 3.6 and Windows 10, use them. Along with everything else, it has support for the long paths - since the Anniversary Update of Windows 10.
  - You might have to enable long path support on Windows 10 though. [[instructions](https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/)]
- If you don't have the ability to use Windows 10 and Python 3.6, you'll have to workin the restrictions set by Windows:
  - You can use the `--cache-dir` option to set a cache directory that's shorter.
  - Set `TEMP` as a shorter path - like `C:\Temp`
- You can try making a long path into a short one with the command line tools `subst` and `mklink` commands. (but don't)

Since this is basically fixed with the latest versions of the software involved (pip 9.0.1, Python 3.6.1, Windows 10 with Anniversary) - should there be changes in pip to workaround this issue?
I wouldn't suggest going down the subst or mklink routes at all. pip should just name its temp directory something other than the full path to the eventual/original install location.

The fix in Python 3.6 and Windows 10 is fine, but will never apply to all users.
Of course some deeply nested libraries may well end up hitting the problem with the actual files final location for user installs of python which is problematic, even if it never gets that far with the current  process, maybe we also need a separate ticket for getting pip to auto-install  to wheel with no override if platform is Windows and any of the resulting paths would exceed the limit.
I agree with @zooba - that's the form I'd expect such a patch to take.

If pip can't install to the final location (as @GadgetSteve suggests) that's not an issue for pip - we'll get an OS error and roll back the install, and it's then the user's issue to solve.
> should there be changes in pip to workaround this issue?

Yes - a patch is needed that shortens pip's temporary directory paths.

---

FTR, "awaiting PR" is essentially for indicating that further discussion related to this issue should be deferred until someone comes around to make a PR.

Ah, OK. I've removed the label in that case, as there's no real need for any discussion. Do we not have a label for "Agreed to be a reasonable request, but won't go further until someone steps up with a PR"?
@pfmoore Nope.

Now that I think of it, the only reason I don't wanna use that label is
because I have used it in a different context. I've mostly used a comment
after labelling to describe the state of the issue; so, stating that you
just need to make the PR would probably be fine here too. So, I dent mind
if this gets labeled as awaiting PR.

Maybe we should rename the current label to "deferred till PR" and add a
new label with the current name for this situation?

On Tue, Aug 22, 2017, 03:01 Paul Moore <notifications@github.com> wrote:

> Ah, OK. I've removed the label in that case, as there's no real need for
> any discussion. Do we not have a label for "Agreed to be a reasonable
> request, but won't go further until someone steps up with a PR"?
>
> —
> You are receiving this because you commented.
>
>
> Reply to this email directly, view it on GitHub
> <https://github.com/pypa/pip/issues/3055#issuecomment-323858485>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/ADH7SdnLsuxCOYel6O8YFtPxhmeFtp5Iks5safcsgaJpZM4FxHtJ>
> .
>

> Maybe we should rename the current label to "deferred till PR" and add a
> new label with the current name for this situation?

I've gone ahead and done this.
> pip should just name its temp directory something other than the full path to the eventual/original install location.

Noting for whomsoever comes around to making a PR for this, the behaviour change would be to shorten pip's temporary directories by not being the full path to the install location.



On 27/10/2016 17:48, Steve Dower wrote:
> FWIW, Windows 10 allows the option to disable MAX_PATH limits for 
> "self-certified" applications, which will include Python 3.6. Currently 
> there's a machine policy that needs to be enabled, and the Python 3.6 
> installer will prompt users to do that (if they can - need to be admin).
> 
> So the issue will go away in the future, and anything else we do now is 
> mitigation for people on existing setups (which obviously has a lot of 
> value, since it's going to be a long time before /everyone/ is on Python 
> 3.6 on Windows 10).
> 
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub 
> <https://github.com/pypa/pip/issues/3055#issuecomment-256702823>, or 
> mute the thread 
> <https://github.com/notifications/unsubscribe-auth/AFVaWjBjtClaJLPN89fSDCv0NVrIdOB9ks5q4NXlgaJpZM4FxHtJ>.
> 

Note that selecting this option enables long paths for everything, 
including python 2.7, and AFAIK it remains enabled even if the version 
of python that enabled it is subsequently removed.

-- 
Steve (Gadget) Barnes
Any opinions in this message are my personal opinions and do not reflect 
those of my employer.

[Path Length Limitation](https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx) 

```"\\?\D:\very long path"```

> Note  The maximum path of 32,767 characters is approximate, because the "\\?\" prefix may be expanded to a longer string by the system at run time, and this expansion applies to the total length.
Question for @pradyunsg @pfmoore @dstufft: would you consider a change to try and use a random name in the target directory instead of TEMP? This has the nice advantage that we can simply rename a directory and then try to delete it, rather than doing multiple copies. When used on install, we can extract/install to the random name and then just rename the directory when it looks good, which drastically simplifies the process of copying permissions/etc., as they'll be inherited normally. And if the user doesn't have permissions to install, it'll fail much sooner.

In both cases, the random name can be the same length as the target name, which will ensure that path length issues don't get any worse than when the package is installed. I guess the downside is potentially leftover files in the install directory when pip hard crashes, but the upside is less crashing and significantly faster installs (I've prototyped some parts of this on Windows and we're talking at least 2x faster for big packages like Django).

Thoughts? Given one-off installs into virtual environments are becoming the norm, I think the risk of cruft being left behind in install directories is worth the other benefits.
Seems reasonable to me, I'd just say we should use a name that *isn't* importable as well.
Substituting the first character with a digit is probably an easy algorithm, at least for the first 10 attempts :)
Yea, or add a prefix like ``-pip-tmp`` or something.
I want to avoid generating a longer name, if possible. If we cross the 260 char barrier here, install will fail when the package would otherwise work (apart from caching pyc files... but oh well).
Ah right, that makes sense. Okay.
I think I'd use a leading ``-`` or something instead of a digit, just because it feel more purposeful to me? But that's kind of nitpicky, so just a suggestion.
I think it'll have to be an invalid character in a `dist-info` folder name as well, since those will need the same treatment. So probably I'm going to end up with a list of invalid package name characters that are valid directory name characters and swap them in until there's a free one. Trying `-` (or `~`?) first is fine by me.
Presumably another option here is to just not put the *entire* path inside of the temporary directory as well? Like if there's some common prefix here, we should be able to just exclude that from the temporary directory path that we generate so we're not nesting things nearly as far.

That would turn something like:

```
C:\\Users\\TREVOR~1\\AppData\\Local\\Temp\\pip-h64zdfhc-uninstall\\users\\trevorsullivan\\source\\repos\\pythonapplication11\\pythonapplication11\\env\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\_implementation.cpython-34.pyc
```

into

```
C:\\Users\\TREVOR~1\\AppData\\Local\\Temp\\pip-h64zdfhc-uninstall\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\_implementation.cpython-34.pyc
```

It doesn't have the other benefits though (simple renames, etc) and it means it could still fail when this strategy works in the case the ``TMPDIR`` path is longer than the common prefix we'd remove. On the flip side, it does mean that it's probably a smaller delta in our current behavior which may be a "safer" change to make.

I don't feel strongly one way or another, but the idea popped into my head so figured I'd mention it.
Yeah, that is going to deal with the original problem here (most of the time). I think it was suggested a few times higher on the thread, and it's definitely a much simpler change.

That said, I think the uninstall case is going to be just as simple to rename and then rmtree, so the delta doesn't get much bigger. And the perf benefit to wheel extraction is worth doing in-place as well. Maybe I'll start with uninstall though and send two separate PRs.
I'm OK with the idea of a random name in the target. Just to be clear, we're talking about the temporary install directory that gets moved into place for the final install? Not the build directory?
Sounds OK to me as well.
> Just to be clear, we're talking about the temporary install directory that gets moved into place for the final install? Not the build directory?

Correct. IIUC, eventually all installs will come from/via wheels, right? So the install side will only affect the wheel extraction directory, and shouldn't touch build at all.
Yep, under PEP 517 (now in master, but not yet used for all builds) we set up a build directory (which will still be in `$TEMP`) and call the backend to build a wheel. We then extract the wheel and move the extracted files. If we extract to a random-named directory in the target location and move, that will skip a whole copy step, which is a definite performance improvement[1]

Legacy non-PEP517 installs go direct via `setup.py install`, so there's no wheel or wheel extraction step, so they won't change (which is fine by me, as you say that code path is destined for removal anyway)

[1] We might get a double improvement - I don't know much about how anti-virus software works, but if it can recognise that a move doesn't require a new scan, that could mean that the proposed approach will remove an extra unneeded virus scan as well.
Pretty sure we'll get that double improvement :) I've been profiling with AV scanning enabled, and it seems to be triggered totally differently between `copyfile` (open/read/close/open/write/close) and rename.

Resolving patch:
diff --git a/src/pip/_internal/req/req_uninstall.py b/src/pip/_internal/req/req_uninstall.py
--- a/src/pip/_internal/req/req_uninstall.py
+++ b/src/pip/_internal/req/req_uninstall.py
@@ -17,7 +17,7 @@
     FakeFile, ask, dist_in_usersite, dist_is_local, egg_link_path, is_local,
     normalize_path, renames,
 )
-from pip._internal.utils.temp_dir import TempDirectory
+from pip._internal.utils.temp_dir import AdjacentTempDirectory
 
 logger = logging.getLogger(__name__)
 
@@ -86,16 +86,54 @@ def compact(paths):
     sep = os.path.sep
     short_paths = set()
     for path in sorted(paths, key=len):
-        should_add = any(
+        should_skip = any(
             path.startswith(shortpath.rstrip("*")) and
             path[len(shortpath.rstrip("*").rstrip(sep))] == sep
             for shortpath in short_paths
         )
-        if not should_add:
+        if not should_skip:
             short_paths.add(path)
     return short_paths
 
 
+def compress_for_rename(paths):
+    """Returns a set containing the paths that need to be renamed.
+
+    This set may include directories when the original sequence of paths
+    included every file on disk.
+    """
+    case_map = dict((os.path.normcase(p), p) for p in paths)
+    remaining = set(case_map)
+    unchecked = sorted(set(os.path.split(p)[0]
+                           for p in case_map.values()), key=len)
+    wildcards = set()
+
+    def norm_join(*a):
+        return os.path.normcase(os.path.join(*a))
+
+    for root in unchecked:
+        if any(os.path.normcase(root).startswith(w)
+               for w in wildcards):
+            # This directory has already been handled.
+            continue
+
+        all_files = set()
+        all_subdirs = set()
+        for dirname, subdirs, files in os.walk(root):
+            all_subdirs.update(norm_join(root, dirname, d)
+                               for d in subdirs)
+            all_files.update(norm_join(root, dirname, f)
+                             for f in files)
+        # If all the files we found are in our remaining set of files to
+        # remove, then remove them from the latter set and add a wildcard
+        # for the directory.
+        if len(all_files - remaining) == 0:
+            remaining.difference_update(all_files)
+            wildcards.add(root + os.sep)
+
+    return set(map(case_map.__getitem__, remaining)) | wildcards
+
+
 def compress_for_output_listing(paths):
     """Returns a tuple of 2 sets of which paths to display to user
 
@@ -153,7 +191,7 @@ def __init__(self, dist):
         self._refuse = set()
         self.pth = {}
         self.dist = dist
-        self.save_dir = TempDirectory(kind="uninstall")
+        self._save_dirs = []
         self._moved_paths = []
 
     def _permitted(self, path):
@@ -193,9 +231,17 @@ def add_pth(self, pth_file, entry):
             self._refuse.add(pth_file)
 
     def _stash(self, path):
-        return os.path.join(
-            self.save_dir.path, os.path.splitdrive(path)[1].lstrip(os.path.sep)
-        )
+        best = None
+        for save_dir in self._save_dirs:
+            if not path.startswith(save_dir.original + os.sep):
+                continue
+            if not best or len(save_dir.original) > len(best.original):
+                best = save_dir
+        if best is None:
+            best = AdjacentTempDirectory(os.path.dirname(path))
+            best.create()
+            self._save_dirs.append(best)
+        return os.path.join(best.path, os.path.relpath(path, best.original))
 
     def remove(self, auto_confirm=False, verbose=False):
         """Remove paths in ``self.paths`` with confirmation (unless
@@ -215,12 +261,10 @@ def remove(self, auto_confirm=False, verbose=False):
 
         with indent_log():
             if auto_confirm or self._allowed_to_proceed(verbose):
-                self.save_dir.create()
-
-                for path in sorted(compact(self.paths)):
+                for path in sorted(compact(compress_for_rename(self.paths))):
                     new_path = self._stash(path)
                     logger.debug('Removing file or directory %s', path)
-                    self._moved_paths.append(path)
+                    self._moved_paths.append((path, new_path))
                     renames(path, new_path)
                 for pth in self.pth.values():
                     pth.remove()
@@ -251,20 +295,21 @@ def _display(msg, paths):
         _display('Would remove:', will_remove)
         _display('Would not remove (might be manually added):', will_skip)
         _display('Would not remove (outside of prefix):', self._refuse)
+        if verbose:
+            _display('Will actually move:', compress_for_rename(self.paths))
 
         return ask('Proceed (y/n)? ', ('y', 'n')) == 'y'
 
     def rollback(self):
         """Rollback the changes previously made by remove()."""
-        if self.save_dir.path is None:
+        if not self._save_dirs:
             logger.error(
                 "Can't roll back %s; was not uninstalled",
                 self.dist.project_name,
             )
             return False
         logger.info('Rolling back uninstall of %s', self.dist.project_name)
-        for path in self._moved_paths:
-            tmp_path = self._stash(path)
+        for path, tmp_path in self._moved_paths:
             logger.debug('Replacing %s', path)
             renames(tmp_path, path)
         for pth in self.pth.values():
@@ -272,7 +317,8 @@ def rollback(self):
 
     def commit(self):
         """Remove temporary save dir: rollback will no longer be possible."""
-        self.save_dir.cleanup()
+        for save_dir in self._save_dirs:
+            save_dir.cleanup()
         self._moved_paths = []
 
     @classmethod
diff --git a/src/pip/_internal/utils/temp_dir.py b/src/pip/_internal/utils/temp_dir.py
--- a/src/pip/_internal/utils/temp_dir.py
+++ b/src/pip/_internal/utils/temp_dir.py
@@ -1,5 +1,6 @@
 from __future__ import absolute_import
 
+import itertools
 import logging
 import os.path
 import tempfile
@@ -58,7 +59,7 @@ def __exit__(self, exc, value, tb):
             self.cleanup()
 
     def create(self):
-        """Create a temporary directory and store it's path in self.path
+        """Create a temporary directory and store its path in self.path
         """
         if self.path is not None:
             logger.debug(
@@ -80,3 +81,62 @@ def cleanup(self):
         if self.path is not None and os.path.exists(self.path):
             rmtree(self.path)
         self.path = None
+
+
+class AdjacentTempDirectory(TempDirectory):
+    """Helper class that creates a temporary directory adjacent to a real one.
+
+    Attributes:
+        original
+            The original directory to create a temp directory for.
+        path
+            After calling create() or entering, contains the full
+            path to the temporary directory.
+        delete
+            Whether the directory should be deleted when exiting
+            (when used as a contextmanager)
+
+    """
+    # The characters that may be used to name the temp directory
+    LEADING_CHARS = "-~.+=%0123456789"
+
+    def __init__(self, original, delete=None):
+        super(AdjacentTempDirectory, self).__init__(delete=delete)
+        self.original = original.rstrip('/\\')
+
+    @classmethod
+    def _generate_names(cls, name):
+        """Generates a series of temporary names.
+
+        The algorithm replaces the leading characters in the name
+        with ones that are valid filesystem characters, but are not
+        valid package names (for both Python and pip definitions of
+        package).
+        """
+        for i in range(1, len(name)):
+            if name[i] in cls.LEADING_CHARS:
+                continue
+            for candidate in itertools.combinations_with_replacement(
+                    cls.LEADING_CHARS, i):
+                new_name = ''.join(candidate) + name[i:]
+                if new_name != name:
+                    yield new_name
+
+    def create(self):
+        root, name = os.path.split(self.original)
+        for candidate in self._generate_names(name):
+            path = os.path.join(root, candidate)
+            try:
+                os.mkdir(path)
+            except OSError:
+                pass
+            else:
+                self.path = os.path.realpath(path)
+                break
+
+        if not self.path:
+            # Final fallback on the default behavior.
+            self.path = os.path.realpath(
+                tempfile.mkdtemp(prefix="pip-{}-".format(self.kind))
+            )
+        logger.debug("Created temporary directory: {}".format(self.path))


Test patch:
diff --git a/tests/unit/test_req_uninstall.py b/tests/unit/test_req_uninstall.py
--- a/tests/unit/test_req_uninstall.py
+++ b/tests/unit/test_req_uninstall.py
@@ -6,7 +6,7 @@
 import pip._internal.req.req_uninstall
 from pip._internal.req.req_uninstall import (
     UninstallPathSet, compact, compress_for_output_listing,
-    uninstallation_paths,
+    compress_for_rename, uninstallation_paths,
 )
 from tests.lib import create_file
 
@@ -90,9 +90,24 @@ def in_tmpdir(paths):
         "lib/mypkg/support/would_be_skipped.skip.py",
     ])
 
+    expected_rename = in_tmpdir([
+        "bin/",
+        "lib/mypkg.dist-info/",
+        "lib/mypkg/would_be_removed.txt",
+        "lib/mypkg/__init__.py",
+        "lib/mypkg/my_awesome_code.py",
+        "lib/mypkg/__pycache__/",
+        "lib/mypkg/support/support_file.py",
+        "lib/mypkg/support/more_support.py",
+        "lib/mypkg/support/__pycache__/",
+        "lib/random_other_place/",
+    ])
+
     will_remove, will_skip = compress_for_output_listing(sample)
+    will_rename = compress_for_rename(sample)
     assert sorted(expected_skip) == sorted(compact(will_skip))
     assert sorted(expected_remove) == sorted(compact(will_remove))
+    assert sorted(expected_rename) == sorted(compact(will_rename))
 
 
 class TestUninstallPathSet(object):
diff --git a/tests/unit/test_utils.py b/tests/unit/test_utils.py
--- a/tests/unit/test_utils.py
+++ b/tests/unit/test_utils.py
@@ -4,6 +4,7 @@
 util tests
 
 """
+import itertools
 import os
 import shutil
 import stat
@@ -29,7 +30,7 @@
     split_auth_from_netloc, untar_file, unzip_file,
 )
 from pip._internal.utils.packaging import check_dist_requires_python
-from pip._internal.utils.temp_dir import TempDirectory
+from pip._internal.utils.temp_dir import AdjacentTempDirectory, TempDirectory
 
 
 class Tests_EgglinkPath:
@@ -547,6 +548,34 @@ def test_create_and_cleanup_work(self):
         assert tmp_dir.path is None
         assert not os.path.exists(created_path)
 
+    @pytest.mark.parametrize("name", [
+        "ABC",
+        "ABC.dist-info",
+        "_+-",
+        "_package",
+    ])
+    def test_adjacent_directory_names(self, name):
+        def names():
+            return AdjacentTempDirectory._generate_names(name)
+
+        chars = AdjacentTempDirectory.LEADING_CHARS
+
+        # Ensure many names are unique
+        # (For long *name*, this sequence can be extremely long.
+        # However, since we're only ever going to take the first
+        # result that works, provided there are many of those
+        # and that shorter names result in totally unique sets,
+        # it's okay to skip part of the test.)
+        some_names = list(itertools.islice(names(), 10000))
+        assert len(some_names) == len(set(some_names))
+
+        # Ensure original name does not appear
+        assert not any(n == name for n in names())
+
+        # Check the first group are correct
+        assert all(x == y for x, y in
+                   zip(some_names, [c + name[1:] for c in chars]))
+
 
 class TestGlibc(object):
     def test_manylinux1_check_glibc_version(self):