Repository: pypa/pip

Issue:
installing from a wheel puts platform files into purelib, ignoring WHEEL Root-Is-Pure: False
When installing a package with C extensions, the platform specific shared libraries should be placed in the 'platform library' install path install_platlib.  On 64 bit linux, this would be /usr/lib64/python2.7/<...>

pip 7.0.3 incorrectly installs these platform specific shared binaries to the 'pure library' install path install_purelib.

This only happens when installing from a wheel file.  The wheel file contains a WHEEL configuration file which indicates the package is not pure (Root-Is-Pure: False), but pip appears to ignore that.

For example, if I were to:

  pip install python-qpid-proton

pip builds the extensions, generates a wheel in the cache, and installs from that wheel.
the shared libraries generated by the extension are placed here:
## $ pip show python-qpid-proton

Metadata-Version: 2.0
Name: python-qpid-proton
Version: 0.9.1.1
Summary: An AMQP based messaging library.
Home-page: http://qpid.apache.org/proton/
Author: Apache Qpid
Author-email: proton@qpid.apache.org
License: Apache Software License
Location: /usr/lib/python2.7/site-packages   <----------------------------

However, if I disable the cache (which skips the wheel building/install) the shared objects are correctly installed:

$ sudo pip --no-cache-dir install python-qpid-proton
## $ pip show python-qpid-proton

Metadata-Version: 1.1
Name: python-qpid-proton
Version: 0.9.1.1
Summary: An AMQP based messaging library.
Home-page: http://qpid.apache.org/proton/
Author: Apache Qpid
Author-email: proton@qpid.apache.org
License: Apache Software License
Location: /usr/lib64/python2.7/site-packages    <-------------------------------------------
Requires: 

It appears that the install_platlib value in the distribution scheme is being incorrectly overridden here

https://github.com/pypa/pip/blob/develop/pip/locations.py#L196

The Distribution instance created here:
https://github.com/pypa/pip/blob/develop/pip/locations.py#L181

is a dummy distribution in the case of a wheel.  It is not the actual distribution as defined by the packages setup.py file.  As such, it will not have any extensions configured.  

But when finalize_options is run on the dummy install instance, distutils checks to see if 'install_lib' is unset, and _if there are no extensions_ it set install_lib to the value of install_purelib.

Which means install_lib will never be set to platlib, even if the package contains binary extensions.
So when the installation path for a platform library is picked here:

https://github.com/pypa/pip/blob/develop/pip/wheel.py#L248

it has been incorrectly set to the pure lib directory.

https://github.com/pypa/pip/blob/develop/pip/wheel.py#L193



Discussion:
I believe this same issue is hitting us in [this bug](https://bugs.launchpad.net/devstack/+bug/1484870) in devstack with fedora22

There we pre-build a wheel of cffi, which later gets installed.  However, an earlier rpm-provided .so is left in /usr/lib64/python2.7/site-packages, which gets picked up by the interpreter first.  When the interface changes enough (as it now has :) it blows up

This is with pip 7.1.0


Resolving patch:
diff --git a/pip/locations.py b/pip/locations.py
--- a/pip/locations.py
+++ b/pip/locations.py
@@ -237,8 +237,12 @@ def distutils_scheme(dist_name, user=False, home=None, root=None,
     for key in SCHEME_KEYS:
         scheme[key] = getattr(i, 'install_' + key)
 
-    if i.install_lib is not None:
-        # install_lib takes precedence over purelib and platlib
+    # install_lib specified in setup.cfg should install *everything*
+    # into there (i.e. it takes precedence over both purelib and
+    # platlib).  Note, i.install_lib is *always* set after
+    # finalize_options(); we only want to override here if the user
+    # has explicitly requested it hence going back to the config
+    if 'install_lib' in d.get_option_dict('install'):
         scheme.update(dict(purelib=i.install_lib, platlib=i.install_lib))
 
     if running_under_virtualenv():


Test patch:
diff --git a/tests/unit/test_locations.py b/tests/unit/test_locations.py
--- a/tests/unit/test_locations.py
+++ b/tests/unit/test_locations.py
@@ -181,6 +181,9 @@ def test_distutils_config_file_read(self, tmpdir, monkeypatch):
         scheme = distutils_scheme('example')
         assert scheme['scripts'] == '/somewhere/else'
 
+    # when we request install-lib, we should install everything (.py &
+    # .so) into that path; i.e. ensure platlib & purelib are set to
+    # this path
     def test_install_lib_takes_precedence(self, tmpdir, monkeypatch):
         f = tmpdir.mkdir("config").join("setup.cfg")
         f.write("[install]\ninstall-lib=/somewhere/else/")