Repository: spack/spack

Issue:
Bootstrapping from sources is broken on develop
### Steps to reproduce the issue

Trying to bootstrap from sources on `develop` I got the following error cause by #24095:
```console
$ spack solve -d zlib
[ ... ]
[+] /home/culpo/.spack/bootstrap/store/linux-ubuntu18.04-x86_64/gcc-11.1.0/clingo-bootstrap-spack-4xc3gf3j73rmmevzoozp3cx3e7tysifh
==> [2021-07-21-12:11:02.289096] '/usr/bin/python3.6' '-c' 'from distutils.sysconfig import get_python_lib; print(get_python_lib(plat_specific=False, standard_lib=False, prefix=""))'
==> [2021-07-21-12:11:02.327841] Error: cannot import module "clingo" from spec "clingo-bootstrap@spack%gcc@11.1.0~docs~ipo+python build_type=Release arch=linux-ubuntu18.04-x86_64 ^bison@3.0.4%gcc@11.1.0 patches=b72914fe38e54a6fc25f29019e0a0786705c4f61ce20d414cc2010c8d62448c7 arch=linux-ubuntu18.04-x86_64 ^cmake@3.21.0%gcc@11.1.0~doc+ncurses+openssl+ownlibs~qt build_type=Release arch=linux-ubuntu18.04-x86_64 ^python@3.6%gcc@11.1.0+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib arch=linux-ubuntu18.04-x86_64 ^re2c@1.2.1%gcc@11.1.0 arch=linux-ubuntu18.04-x86_64
```
i.e. Spack is not able to import the `clingo-bootstrap` that was just installed. The installation is correct and using `git bisect` the first commit that broke the bootstrapping procedure is:
```console
$ git bisect bad
c37df94932260fe3e5f22381af03587662569f7e is the first bad commit
commit c37df94932260fe3e5f22381af03587662569f7e
Author: Adam J. Stewart <ajstewart426@gmail.com>
Date:   Fri Jul 16 10:28:00 2021 -0500

    Python: query distutils to find site-packages directory (#24095)
    
    Third-party Python libraries may be installed in one of several directories:
    
    1. `lib/pythonX.Y/site-packages` for Spack-installed Python
    2. `lib64/pythonX.Y/site-packages` for system Python on RHEL/CentOS/Fedora
    3. `lib/pythonX/dist-packages` for system Python on Debian/Ubuntu
    
    Previously, Spack packages were hard-coded to use the (1). Now, we query the Python installation itself and ask it which to use. Ever since #21446 this is how we've been determining where to install Python libraries anyway.
    
    Note: there are still many packages that are hard-coded to use (1). I can change them in this PR, but I don't have the bandwidth to test all of them.
    
    * Python: handle dist-packages and site-packages
    * Query Python to find site-packages directory
    * Add try-except statements for when distutils isn't installed
    * Catch more errors
    * Fix root directory used in import tests
    * Rely on site_packages_dir property

 lib/spack/spack/bootstrap.py                       |   4 -
 lib/spack/spack/build_systems/python.py            |  43 +++---
 lib/spack/spack/build_systems/sip.py               |  28 ++--
 var/spack/repos/builtin/packages/python/package.py | 160 +++++++++++++++------
 4 files changed, 145 insertions(+), 90 deletions(-)
```

### Information on your system

* **Spack:** 0.16.2-3579-887820ecb5
* **Python:** 3.6.9
* **Platform:** linux-ubuntu18.04-broadwell
* **Concretizer:** original

### Additional information

- [x] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have searched the issues of this repo and believe this is not a duplicate
- [x] I have run the failing commands in debug mode and reported the output



Discussion:
The tree for `clingo-bootstrap` on the system where it fails is:
```console
$ tree /home/culpo/.spack/bootstrap/store/linux-ubuntu18.04-x86_64/gcc-11.1.0/clingo-bootstrap-spack-4xc3gf3j73rmmevzoozp3cx3e7tysifh/lib/
/home/culpo/.spack/bootstrap/store/linux-ubuntu18.04-x86_64/gcc-11.1.0/clingo-bootstrap-spack-4xc3gf3j73rmmevzoozp3cx3e7tysifh/lib/
├── cmake
│   └── Clingo
│       ├── ClingoConfig.cmake
│       ├── ClingoConfigVersion.cmake
│       ├── ClingoTargets.cmake
│       └── ClingoTargets-release.cmake
├── libclingo.so -> libclingo.so.4
├── libclingo.so.4 -> libclingo.so.4.0
├── libclingo.so.4.0
├── python3
│   └── dist-packages
└── python3.6
    └── site-packages
        ├── clingo
        │   ├── ast.pyi
        │   ├── __init__.pyi
        │   ├── py.typed
        │   └── types.pyi
        └── clingo.cpython-36m-x86_64-linux-gnu.so

7 directories, 12 files
```
Before we were looking in the `python3.6/site-packages` directory, while after that commit we are looking into `python3/dist-packages` which is empty.
can confirm using the `spack/ubuntu-bionic` docker image (and installing python3-dev first :roll_eyes:)

Resolving patch:
diff --git a/lib/spack/spack/bootstrap.py b/lib/spack/spack/bootstrap.py
--- a/lib/spack/spack/bootstrap.py
+++ b/lib/spack/spack/bootstrap.py
@@ -93,15 +93,14 @@ def make_module_available(module, spec=None, install=False):
 
     for ispec in installed_specs:
         # TODO: make sure run-environment is appropriate
-        module_path = os.path.join(ispec.prefix,
-                                   ispec['python'].package.site_packages_dir)
+        module_path = ispec['python'].package.get_python_lib(prefix=ispec.prefix)
         try:
             sys.path.append(module_path)
             __import__(module)
             return
         except ImportError:
             tty.warn("Spec %s did not provide module %s" % (ispec, module))
-            sys.path = sys.path[:-2]
+            sys.path = sys.path[:-1]
 
     def _raise_error(module_name, module_spec):
         error_msg = 'cannot import module "{0}"'.format(module_name)
@@ -118,14 +117,13 @@ def _raise_error(module_name, module_spec):
         spec.concretize()
     spec.package.do_install()
 
-    module_path = os.path.join(spec.prefix,
-                               spec['python'].package.site_packages_dir)
+    module_path = spec['python'].package.get_python_lib(prefix=spec.prefix)
     try:
         sys.path.append(module_path)
         __import__(module)
         return
     except ImportError:
-        sys.path = sys.path[:-2]
+        sys.path = sys.path[:-1]
         _raise_error(module, spec)
 
 


Test patch: