Repository: pypa/pip

Issue:
Infinite loop when using caching on filesystems that can't hard link
When running pip 6.1.1 on a system where the home directory is mounted from a VMware host-guest fileshare, a pip install loops infinitely.

This would occur with any similar filesystem. The workaround is to disable the pip version check with `--disable-pip-version-check`.  Since the same mechanism is used by caching, that must be disabled too with `--no-cache-dir`

Lines 27-52 in the method `acquire()` are the culprit where the code doesn't track the reason for the OSError exception returned, and blindly retries.

Here's a `sysdig` trace showing the infinite loop:

```
17060 14:54:32.102163823 3 modelA_venv_test (e95bad8864c3) pip (119666:48) > link oldpath=/home/townsen/.cache/pip/http/f/e/d/0/e/e95bad8864c3-c5cfe740.487463445631603138057 newpath=/home/townsen/.cache/pip/http/f/e/d/0/e/fed0ed508030b766d5b0c2792132c8bf197804464765b46b361f93a4.lock 
17061 14:54:32.102601573 3 modelA_venv_test (e95bad8864c3) pip (119666:48) < link res=-1(EPERM) 
17062 14:54:32.102626589 3 modelA_venv_test (e95bad8864c3) pip (119666:48) > stat 
17063 14:54:32.102634519 3 modelA_venv_test (e95bad8864c3) pip (119666:48) < stat res=0 path=/home/townsen/.cache/pip/http/f/e/d/0/e/e95bad8864c3-c5cfe740.487463445631603138057 
17064 14:54:32.102647613 3 modelA_venv_test (e95bad8864c3) pip (119666:48) > select 
17065 14:54:32.102651540 3 modelA_venv_test (e95bad8864c3) pip (119666:48) > switch next=11 pgft_maj=0 pgft_min=8390 vm_size=86116 vm_rss=26416 vm_swap=0 
17165 14:54:32.203997692 3 modelA_venv_test (e95bad8864c3) pip (119666:48) < select res=0 
17166 14:54:32.204023467 3 modelA_venv_test (e95bad8864c3) pip (119666:48) > link oldpath=/home/townsen/.cache/pip/http/f/e/d/0/e/e95bad8864c3-c5cfe740.487463445631603138057 newpath=/home/townsen/.cache/pip/http/f/e/d/0/e/fed0ed508030b766d5b0c2792132c8bf197804464765b46b361f93a4.lock 
17167 14:54:32.204199133 3 modelA_venv_test (e95bad8864c3) pip (119666:48) < link res=-1(EPERM) 
```

fixes pip install freezing when using cache dir which doesn't support hard links
This fixes a bug when a user specifies a cache directory in a filesystem, that does not support creating hard links. 
LockFile only checks if OS has hard link support, but not the target filesystem.
This leads to pip freezing forever under specified conditions.
Fixes below solve the problem:
- don't use LinkLockFile for caching as we don't know if the target filesystem supports hard links
- if someone will ever use LinkLockFile - make sure it raises exeption for unsupported file system, but not fall into infinite loop



Discussion:

Also fixes https://github.com/pypa/pip/issues/1969

+1 :+1: 

Hey,

Thanks for the contribution, however we have a strict policy that the only modifications we make to the bundled dependencies are those required to actually make bundling work. The best way to handle this is to go to the relevant upstreams and make PRs against them.


Resolving patch:
diff --git a/pip/_vendor/cachecontrol/__init__.py b/pip/_vendor/cachecontrol/__init__.py
--- a/pip/_vendor/cachecontrol/__init__.py
+++ b/pip/_vendor/cachecontrol/__init__.py
@@ -4,7 +4,7 @@
 """
 __author__ = 'Eric Larson'
 __email__ = 'eric@ionrock.org'
-__version__ = '0.11.2'
+__version__ = '0.11.5'
 
 from .wrapper import CacheControl
 from .adapter import CacheControlAdapter
diff --git a/pip/_vendor/cachecontrol/caches/file_cache.py b/pip/_vendor/cachecontrol/caches/file_cache.py
--- a/pip/_vendor/cachecontrol/caches/file_cache.py
+++ b/pip/_vendor/cachecontrol/caches/file_cache.py
@@ -1,7 +1,8 @@
 import hashlib
 import os
 
-from pip._vendor.lockfile import FileLock
+from pip._vendor.lockfile import LockFile
+from pip._vendor.lockfile.mkdirlockfile import MkdirLockFile
 
 from ..cache import BaseCache
 from ..controller import CacheController
@@ -49,11 +50,23 @@ def _secure_open_write(filename, fmode):
 
 class FileCache(BaseCache):
     def __init__(self, directory, forever=False, filemode=0o0600,
-                 dirmode=0o0700):
+                 dirmode=0o0700, use_dir_lock=None, lock_class=None):
+
+        if use_dir_lock is not None and lock_class is not None:
+            raise ValueError("Cannot use use_dir_lock and lock_class together")
+
+        if use_dir_lock:
+            lock_class = MkdirLockFile
+
+        if lock_class is None:
+            lock_class = LockFile
+
         self.directory = directory
         self.forever = forever
         self.filemode = filemode
         self.dirmode = dirmode
+        self.lock_class = lock_class
+
 
     @staticmethod
     def encode(x):
@@ -83,7 +96,7 @@ def set(self, key, value):
         except (IOError, OSError):
             pass
 
-        with FileLock(name) as lock:
+        with self.lock_class(name) as lock:
             # Write our actual file
             with _secure_open_write(lock.path, self.filemode) as fh:
                 fh.write(value)
diff --git a/pip/download.py b/pip/download.py
--- a/pip/download.py
+++ b/pip/download.py
@@ -342,7 +342,7 @@ def __init__(self, *args, **kwargs):
         # require manual evication from the cache to fix it.
         if cache:
             secure_adapter = CacheControlAdapter(
-                cache=SafeFileCache(cache),
+                cache=SafeFileCache(cache, use_dir_lock=True),
                 max_retries=retries,
             )
         else:


Test patch: