Repository: pypa/pip

Issue:
remove the distribute to setuptools upgrade workarounds
Since v1.4 (when the setuptools merger occurred), pip has supported a set of "hacks" (see #1122) to allow distribute to conflict with setuptools, so that `pip install -U setuptools` would work to upgrade distribute.

They were always intended to be temporary to get users through the merger easier.  But they're fragile and a maintenance headache.

This removes the hacks.

Note that the following will still work to upgrade from distribute to setuptools:  `pip install -U distribute`.  The reason this works is that a final version of distribute (v0.7.3) was released that was simple wrapper with: `install_requires=['setuptools>=0.7']`.  I left a test (a slight mod to one of the old tests) that confirms this works.  

Note the cost:
- users on older systems or in older virtualenvs can potentially install setuptools on top of distribute with unknown side effects?
- users who may have become accustomed to `pip install -U setuptools` working to upgrade distribute may be surprised.

`pip install --ignore-installed --user setuptools` still tries to uninstall system version
Hi guys, running of the 1.6dev1 version of pip when I try

``` bash
pip install --user setuptools
```

it still tries to uninstall the system version of `distribute` if its too old. The situation is not resolved by adding either (or both) of `--upgrade` or `--ignore-installed`.

I think it boils down to the hack here: https://github.com/pypa/pip/blob/develop/pip/req/req_set.py#L563 which doesn't check for `--ignore-installed`.

I'm not so familiar with the pip code or workflow, so I'll happily be told I'm completely wrong.



Discussion:
@dstufft pretty sure we want this for 7.0, just don't want to merge if more 6 releases are coming? I guess it can sit til 7.0 is imminent?

@qwcode as far as 6.1 vs 7.0 goes, basically my plan is the next release should be whatever is ready for that point and the version number will go based on what kind of changes we've landed. 

@dstufft 
concerns about the costs I mention?
this is the kind of change that will break build systems somewhere, but I think it's inevitable

Merge conflicts

+1 these hacks break things for me often enough to be annoying. I have to work around them by adding a fake .egg-info for distutils. Time for them to go.

This needs to be rebased, however the develop branch is now on 7.0 so it can be committed.

the title says  `--ignore-installed`, but your snippet says `--user`.  which is it?

Apologies for the confusion, I've edited the original issue post to reflect that using one or both of `--ignore-installed` or `--user` doesn't make any difference to the error I'm getting.

the issue does boil down to the line mentioned in the description, but I'm wary of fixing it, knowing that we planned on removing the HACKS most likely in v1.7 (see #1122)

I dropped this out of 6.0 and moved it to 7.0. I want to push forward on 6.0 and I believe that removing the hacks in 7.0 will fix this issue and it's likely best to just wait until then.


Resolving patch:
diff --git a/pip/req/req_install.py b/pip/req/req_install.py
--- a/pip/req/req_install.py
+++ b/pip/req/req_install.py
@@ -378,18 +378,6 @@ def run_egg_info(self):
             )
 
         with indent_log():
-            # if it's distribute>=0.7, it won't contain an importable
-            # setuptools, and having an egg-info dir blocks the ability of
-            # setup.py to find setuptools plugins, so delete the egg-info dir
-            # if no setuptools. it will get recreated by the run of egg_info
-            # NOTE: this self.name check only works when installing from a
-            #       specifier (not archive path/urls)
-            # TODO: take this out later
-            if (self.name == 'distribute' and not
-                    os.path.isdir(
-                        os.path.join(self.source_dir, 'setuptools'))):
-                rmtree(os.path.join(self.source_dir, 'distribute.egg-info'))
-
             script = self._run_setup_py
             script = script.replace('__SETUP_PY__', repr(self.setup_py))
             script = script.replace('__PKG_NAME__', repr(self.name))
@@ -989,17 +977,7 @@ def check_if_exists(self):
         if self.req is None:
             return False
         try:
-            # DISTRIBUTE TO SETUPTOOLS UPGRADE HACK (1 of 3 parts)
-            # if we've already set distribute as a conflict to setuptools
-            # then this check has already run before.  we don't want it to
-            # run again, and return False, since it would block the uninstall
-            # TODO: remove this later
-            if (self.req.project_name == 'setuptools' and
-                    self.conflicts_with and
-                    self.conflicts_with.project_name == 'distribute'):
-                return True
-            else:
-                self.satisfied_by = pkg_resources.get_distribution(self.req)
+            self.satisfied_by = pkg_resources.get_distribution(self.req)
         except pkg_resources.DistributionNotFound:
             return False
         except pkg_resources.VersionConflict:
diff --git a/pip/req/req_set.py b/pip/req/req_set.py
--- a/pip/req/req_set.py
+++ b/pip/req/req_set.py
@@ -599,20 +599,6 @@ def install(self, install_options, global_options=(), *args, **kwargs):
         """
         to_install = self._to_install()
 
-        # DISTRIBUTE TO SETUPTOOLS UPGRADE HACK (1 of 3 parts)
-        # move the distribute-0.7.X wrapper to the end because it does not
-        # install a setuptools package. by moving it to the end, we ensure it's
-        # setuptools dependency is handled first, which will provide the
-        # setuptools package
-        # TODO: take this out later
-        distribute_req = pkg_resources.Requirement.parse("distribute>=0.7")
-        for req in to_install:
-            if (req.name == 'distribute' and
-                    req.installed_version is not None and
-                    req.installed_version in distribute_req):
-                to_install.remove(req)
-                to_install.append(req)
-
         if to_install:
             logger.info(
                 'Installing collected packages: %s',
@@ -621,33 +607,6 @@ def install(self, install_options, global_options=(), *args, **kwargs):
 
         with indent_log():
             for requirement in to_install:
-
-                # DISTRIBUTE TO SETUPTOOLS UPGRADE HACK (1 of 3 parts)
-                # when upgrading from distribute-0.6.X to the new merged
-                # setuptools in py2, we need to force setuptools to uninstall
-                # distribute. In py3, which is always using distribute, this
-                # conversion is already happening in distribute's
-                # pkg_resources. It's ok *not* to check if setuptools>=0.7
-                # because if someone were actually trying to ugrade from
-                # distribute to setuptools 0.6.X, then all this could do is
-                # actually help, although that upgade path was certainly never
-                # "supported"
-                # TODO: remove this later
-                if requirement.name == 'setuptools':
-                    try:
-                        # only uninstall distribute<0.7. For >=0.7, setuptools
-                        # will also be present, and that's what we need to
-                        # uninstall
-                        distribute_requirement = \
-                            pkg_resources.Requirement.parse("distribute<0.7")
-                        existing_distribute = \
-                            pkg_resources.get_distribution("distribute")
-                        if existing_distribute in distribute_requirement:
-                            requirement.conflicts_with = existing_distribute
-                    except pkg_resources.DistributionNotFound:
-                        # distribute wasn't installed, so nothing to do
-                        pass
-
                 if requirement.conflicts_with:
                     logger.info(
                         'Found existing installation: %s',


Test patch:
diff --git a/tests/functional/test_install_upgrade.py b/tests/functional/test_install_upgrade.py
--- a/tests/functional/test_install_upgrade.py
+++ b/tests/functional/test_install_upgrade.py
@@ -298,12 +298,27 @@ def test_upgrade_vcs_req_with_dist_found(script):
     assert "pypi.python.org" not in result.stdout, result.stdout
 
 
-class TestUpgradeSetuptools(object):
+class TestUpgradeDistributeToSetuptools(object):
     """
-    Tests for upgrading to setuptools (using pip from src tree)
-    The tests use a *fixed* set of packages from our test packages dir
-    note: virtualenv-1.9.1 contains distribute-0.6.34
-    note: virtualenv-1.10 contains setuptools-0.9.7
+    From pip1.4 to pip6, pip supported a set of "hacks" (see Issue #1122) to
+    allow distribute to conflict with setuptools, so that the following would
+    work to upgrade distribute:
+
+     ``pip install -U  setuptools``
+
+    In pip7, the hacks were removed.  This test remains to at least confirm pip
+    can upgrade distribute to setuptools using:
+
+      ``pip install -U distribute``
+
+    The reason this works is that a final version of distribute (v0.7.3) was
+    released that is simple wrapper with:
+
+      install_requires=['setuptools>=0.7']
+
+    The test use a fixed set of packages from our test packages dir. Note that
+    virtualenv-1.9.1 contains distribute-0.6.34 and virtualenv-1.10 contains
+    setuptools-0.9.7
     """
 
     def prep_ve(self, script, version, pip_src, distribute=False):
@@ -328,82 +343,18 @@ def prep_ve(self, script, version, pip_src, distribute=False):
             expect_stderr=True,
         )
 
-    @pytest.mark.skipif("sys.version_info >= (3,0)")
-    def test_py2_from_setuptools_6_to_setuptools_7(
-            self, script, data, virtualenv):
-        self.prep_ve(script, '1.9.1', virtualenv.pip_source_dir)
-        result = self.script.run(
-            self.ve_bin / 'pip', 'install', '--no-binary=:all:', '--no-index',
-            '--find-links=%s' % data.find_links, '-U', 'setuptools'
-        )
-        assert (
-            "Found existing installation: setuptools 0.6rc11" in result.stdout
-        )
-        result = self.script.run(self.ve_bin / 'pip', 'list')
-        assert "setuptools (0.9.8)" in result.stdout
-
-    def test_py2_py3_from_distribute_6_to_setuptools_7(
+    def test_from_distribute_6_to_setuptools_7(
             self, script, data, virtualenv):
         self.prep_ve(
             script, '1.9.1', virtualenv.pip_source_dir, distribute=True
         )
         result = self.script.run(
             self.ve_bin / 'pip', 'install', '--no-index',
-            '--find-links=%s' % data.find_links, '-U', 'setuptools'
+            '--find-links=%s' % data.find_links, '-U', 'distribute'
         )
         assert (
             "Found existing installation: distribute 0.6.34" in result.stdout
         )
         result = self.script.run(self.ve_bin / 'pip', 'list')
         assert "setuptools (0.9.8)" in result.stdout
-        assert "distribute (0.7.3)" not in result.stdout
-
-    def test_from_setuptools_7_to_setuptools_7(self, script, data, virtualenv):
-        self.prep_ve(script, '1.10', virtualenv.pip_source_dir)
-        result = self.script.run(
-            self.ve_bin / 'pip', 'install', '--no-index',
-            '--find-links=%s' % data.find_links, '-U', 'setuptools'
-        )
-        assert "Found existing installation: setuptools 0.9.7" in result.stdout
-        result = self.script.run(self.ve_bin / 'pip', 'list')
-        assert "setuptools (0.9.8)" in result.stdout
-
-    def test_from_setuptools_7_to_setuptools_7_using_wheel(
-            self, script, data, virtualenv):
-        self.prep_ve(script, '1.10', virtualenv.pip_source_dir)
-        result = self.script.run(
-            self.ve_bin / 'pip', 'install', '--no-index',
-            '--find-links=%s' % data.find_links, '-U', 'setuptools'
-        )
-        assert "Found existing installation: setuptools 0.9.7" in result.stdout
-        # only wheels use dist-info
-        assert 'setuptools-0.9.8.dist-info' in str(result.files_created)
-        result = self.script.run(self.ve_bin / 'pip', 'list')
-        assert "setuptools (0.9.8)" in result.stdout
-
-    # disabling intermittent travis failure:
-    #   https://github.com/pypa/pip/issues/1379
-    @pytest.mark.skipif("hasattr(sys, 'pypy_version_info')")
-    def test_from_setuptools_7_to_setuptools_7_with_distribute_7_installed(
-            self, script, data, virtualenv):
-        self.prep_ve(
-            script, '1.9.1', virtualenv.pip_source_dir, distribute=True
-        )
-        result = self.script.run(
-            self.ve_bin / 'pip', 'install', '--no-index',
-            '--find-links=%s' % data.find_links, '-U', 'setuptools'
-        )
-        result = self.script.run(
-            self.ve_bin / 'pip', 'install', '--no-index',
-            '--find-links=%s' % data.find_links, 'setuptools==0.9.6'
-        )
-        result = self.script.run(self.ve_bin / 'pip', 'list')
-        assert "setuptools (0.9.6)" in result.stdout
-        assert "distribute (0.7.3)" not in result.stdout
-        result = self.script.run(
-            self.ve_bin / 'pip', 'install', '--no-index',
-            '--find-links=%s' % data.find_links, '-U', 'setuptools'
-        )
-        assert "Found existing installation: setuptools 0.9.6" in result.stdout
-        result = self.script.run(self.ve_bin / 'pip', 'list')
-        assert "setuptools (0.9.8)" in result.stdout
+        assert "distribute (0.7.3)" in result.stdout