Repository: spack/spack

Issue:
Installation issue: sz+python when using external python
### Steps to reproduce the issue

packages.yaml
```
packages:
  python:
    externals:
    - spec: 'python@3.8'
      # configure for module/path to system python paths not writable by user
    buildable: false
```

Install `sz+python` with spack
```
spack install sz+python
```

Error paraphrased
```
Cannot install sz python to <path/to/system/python/modules>, no write permissions
```

### Information on your system

ORNL Spock

* **Spack:** 0.16.2-4028-d27e0bff5a
* **Python:** 3.6.13
* **Platform:** cray-sles15-zen2
* **Concretizer:** clingo


### Additional information

maintainers: @disheng222 @robertu94 

[spack-build-env.txt](https://github.com/spack/spack/files/7172102/spack-build-env.txt)
[spack-build-out.txt](https://github.com/spack/spack/files/7172103/spack-build-out.txt)


### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have run `spack maintainers <name-of-the-package>` and **@mentioned** any maintainers
- [X] I have uploaded the build log and environment files
- [X] I have searched the issues of this repo and believe this is not a duplicate


Discussion:
@kwryankrattiger unfortunately this error is to be expected, but spack doesn't provide the tools to mark external python as as a conflicted configuration.  The underlying problems is that CMake doesn't have a predictable way to knowing where to install the python modules, so it relies on ${Python3_SITELIB} which is defined by the python interpreter that is linked against.  Since your system python library isn't mutable for the user account running spack, this is the only thing that it can reasonably do.  I think alot of other python packages with compiled modules have this problem too.  The work around has a pretty low cost of using a spack compiled python.

You should also be aware that `sz+python` is deprecated.  -- another thing that I am not sure how to indicate in spack other than a comment.  The build system prints out a message to this effect, but you probably didn't see it.  The recommended alternative is to use `libpressio+python+sz`  You can find `libpresio` in this [third party repo](https://github.com/robertu94/spack_packages).  Once you clone it, you should run `spack repo add ${PATH_TO_THE_THIRD_PARTY_REPO)` and the libpressio package will appear.

If one of the spack dev's wants to comment on a possible fix for the install directory issue, please let me know (@adamjstewart  I think you use a lot of spack and python).  I would be happy to implement it in SZ.
Even if we're using Spack-installed python, sz shouldn't be installing to the python installation prefix. In other packages, we either tell CMake where to install the modules or patch CMake to force it to install to the correct installation prefix. I would consider this a bug in our sz recipe that we should fix.
@adamjstewart is that done with a cmake variable like `-DPATH_TO_INSTALL_PYTHON_MODULES`?  Or is there some other way that this is handled?  Is there a good example too look at ?
Yes, either with a CMake variable if one exists or a patch function if one does not. If you grep for packages that use `site_packages_dir` you should find a few examples. 
On the question of deprecation, you can specify different depends/options depending on the version of the package specified in the spec.

```python
# Use libpressio for 
depends_on('libpressio+python+sz', when='@1.2.3:+python')
#...

if spec.satisfies('@:1.2.2'):
   # Configure to install python versions older than 1.2.3
```

This is broken because it creates a cycle between `libpressio` and `sz`, and also `libpressio` is not available with default spack so I don't think this is really going to pass muster without that package being ported over too.
@kwryankrattiger I think I understand what you are saying.  Are you suggesting that if someone requests `sz+python` above a certain version, that spack should bring in `libpressio+python+sz`?  If LibPressio and SZ had exactly the same python API, I might agree that a something like that might make sense.  However, the APIs are not compatible like that.  In fact, that is a major reason why the SZ python API is deprecated.  The SZ python API is broken semantically in some pretty subtle ways that crash if you aren't careful.  Some people still use it in ways that are safe so we leave it in, but I would strongly caution against using it in new development.

I will however submit a fix to SZ (and also to LibPressio) to fix this specific issue about the python installation directory.  After I submit the fix to the build system, it may take a while to get a release cut and then mainlined into spack.

There is a plan to submit LibPressio to spack's upstream repo, however, there are a number of other optional dependencies that libpresiso has support for that would also need to have their packages merged.  For example, some updates to dlib, Nvidia's NVCOMP library, MGARD, etc... 
@adamjstewart I applied the fix to reference `site_packages_dir` locally to libpressio and SZ (patch forthcoming), however both of these depend on py-numpy which in turn depends on py-cython.  When I try to use a system python with py-numpy and py-cython, I get the following errors on Fedora 34. I get the same errors with a `spack install` instead of `spack dev-build`

[error_log.txt](https://github.com/spack/spack/files/7174226/error_log.txt)

@robertu94 what version of Spack are you using? That should be fixed by #24095. Can you share your `spack-build-env.txt` file too? Also, if you can locate what directory the cython module lives in. You may need to reinstall cython if it was installed before #24095.
@adamjstewart  I am using commit `995684133124f70365ef326fdf5b64d649218ca7`  I just installed Cython today for the first time today since I just reinstalled my whole OS recently.  

Here is the spack-build-env.txt for py-cython [spack-build-env.txt](https://github.com/spack/spack/files/7174331/spack-build-env.txt)

Here is the build tree for cython [cython-tree.log](https://github.com/spack/spack/files/7174346/cython-tree.log)

Here is the spack-build-env.txt for py-numpy [spack-build-env.txt](https://github.com/spack/spack/files/7174341/spack-build-env.txt)

A spack-build-env.txt is not created for libpressio since py-numpy fails to install.



Odd, Cython gets installed to `lib64`, but `lib` gets appended to the `PYTHONPATH`. Can you upload the `spack-build-out.txt` for Cython?
@adamjstewart I did see previous comment
That's the `spack-build-env.txt`, not the `spack-build-out.txt`, right?
Ah. sorry. I’ll send it in the morning 
Here is the build out [spack-build-out.txt](https://github.com/spack/spack/files/7177904/spack-build-out.txt)

Okay, so it looks like Cython is one of the only packages that uses platform-specific site package dirs. Let me submit a PR to try to fix this.
@kwryankrattiger  as soon as #25998 gets merged, libpressio@0.70.8: works with system installed python.  SZ will require a small patch once @disheng222 merges the patch to the cmake files for SZ.  You can find the PR for it https://github.com/szcompressor/SZ/pull/75.  I'm linking  [this patch file](https://github.com/spack/spack/files/7180261/szpackage.txt) that you can apply to your spack instance as a temporary measure to instruct spack to install this PR.  You can apply it with `git apply ./path/to/szpackage.txt` 

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/sz/package.py b/var/spack/repos/builtin/packages/sz/package.py
--- a/var/spack/repos/builtin/packages/sz/package.py
+++ b/var/spack/repos/builtin/packages/sz/package.py
@@ -91,6 +91,7 @@ def cmake_args(self):
 
         if "+python" in self.spec:
             args.append("-DBUILD_PYTHON_WRAPPER=ON")
+            args.append("-DSZ_PYTHON_SITELIB={0}".format(site_packages_dir))
         else:
             args.append("-DBUILD_PYTHON_WRAPPER=OFF")
 


Test patch: