Repository: spack/spack

Issue:
Installation issue: julia@1.7.3
### Steps to reproduce the issue

```console
$ spack spec -I julia@1.7.3
==> Error: curl 'platform=darwin' conflicts with '+libssh2'
```


### Error message

<details><summary>Error message</summary><pre>
==> Error: curl 'platform=darwin' conflicts with '+libssh2'
</pre></details>


### Information on your system

* **Spack:** 0.19.0.dev0 (4f57430ef36dab4e84532ac705c6fefa38965257)
* **Python:** 3.9.13
* **Platform:** darwin-monterey-m1
* **Concretizer:** clingo

### Additional information

@glennpj @haampie @vchuravy Trying to install julia on an M1 Mac, I'm getting an error before even getting to the build stage:
```
==> Error: curl 'platform=darwin' conflicts with '+libssh2'
```
The curl conflict is well documented [in this issue](https://github.com/spack/spack/issues/7777).

In a (perhaps naïve) attempt to work around the conflict, I get the same error when attempting to specify a the `~libssh2` variant of curl:
```
spack spec -I julia@1.7.3 ^curl~libssh2
==> Error: curl 'platform=darwin' conflicts with '+libssh2'
```
`curl~libssh2` installs fine on its own. I assume the libssh2 dependency of julia is propagating to curl despite the dependency spec (I'm still a spack beginner, so may be specifying it incorrectly). 

This seems like a roadblock to installing julia with spack on darwin; are there any known workarounds?

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have run `spack maintainers <name-of-the-package>` and **@mentioned** any maintainers
- [X] I have uploaded the build log and environment files
- [X] I have searched the issues of this repo and believe this is not a duplicate


Discussion:
Right, this would require some digging into how julia builds curl for macos. The spack recipe explicitly depends on `curl +libssh2`, that's why you get a concretization failure. Can you check if by magic `curl+libssh2` builds fine on macos now (you have to update the recipe: `spack edit curl` and remove the line `conflicts('platform=darwin', when='+libssh2')`)? I don't think anybody really tried to fix the issue reported in #7777.

(Btw, I don't think Julia 1.7 officially supports aarch64, @vchuravy knows this better than me)

Digging:

- https://github.com/JuliaPackaging/Yggdrasil/blob/61181704ad73472918d878c44cdb007ead02ae13/L/LibCURL/common.jl#L38 seems like libssh2 is enabled for all platforms, so you'd think it's not impossible to build it...
> (Btw, I don't think Julia 1.7 officially supports aarch64, @vchuravy knows this better than me)

We officially discourage aarch64-darwin on 1.7.6 to many aarch64 bugs that were only fixed in Julia 1.8
@haampie Can confirm the same failure as #7777 still exists:
```
==> Error: ProcessError: Command exited with status 1:
    '/var/folders/ng/0znrbqd97zd77svrss84wgpm0000gn/T/bosmacs/spack-stage/spack-stage-curl-7.84.0-wr46aiukpx6u7p5mnf2kibq4aqt3mmwe/spack-src/configure' '--prefix=/opt/spack/opt/spack/darwin-monterey-m1/apple-clang-13.0.0/curl-7.84.0-wr46aiukpx6u7p5mnf2kibq4aqt3mmwe' '--with-zlib=/opt/spack/opt/spack/darwin-monterey-m1/apple-clang-13.0.0/zlib-1.2.12-4rhqrnwnwhexh4bstplg5advofhnpn7q' '--without-brotli' '--without-libgsasl' '--without-libpsl' '--without-zstd' '--enable-shared' '--enable-static' '--without-gssapi' '--without-gnutls' '--without-mbedtls' '--without-nss' '--without-openssl' '--with-secure-transport' '--without-libidn2' '--without-librtmp' '--without-nghttp2' '--with-libssh2' '--without-libssh' '--disable-ldap'

1 error found in build log:
     253    checking for size_t... yes
     254    checking for sys/types.h... (cached) yes
     255    checking for sys/time.h... (cached) yes
     256    checking for sys/socket.h... (cached) yes
     257    checking for struct timeval... yes
     258    checking run-time libs availability... failed
  >> 259    configure: error: one or more libs available at link-time are not available run-time. Libs used at link-time: -lssh2 -L/opt/spack/opt/spack/darwin-monter
            ey-m1/apple-clang-13.0.0/libssh2-1.10.0-bnm6auvwfzd3wcq66x2t6kfeuvtseocu/lib -lssh2 -lmbedcrypto  -lz
```

@vchuravy Thanks for clarifying about 1.7/1.8 on aarch64, I wasn't aware. 1.8.0-rc1 has the issue with curl.
Okay lol, that's a weird configure script check.
@bosmacs can you try #31816 
@haampie I'm getting essentially the same error with that patch, though I see the linker paths have changed:
```
==> Error: ProcessError: Command exited with status 1:
    '/var/folders/ng/0znrbqd97zd77svrss84wgpm0000gn/T/bosmacs/spack-stage/spack-stage-curl-7.84.0-klh4lebvilh2be774hatgvcf5jqv5yzi/spack-src/configure' '--prefix=/opt/spack/opt/spack/darwin-monterey-m1/apple-clang-13.0.0/curl-7.84.0-klh4lebvilh2be774hatgvcf5jqv5yzi' '--with-zlib=/opt/spack/opt/spack/darwin-monterey-m1/apple-clang-13.0.0/zlib-1.2.12-4rhqrnwnwhexh4bstplg5advofhnpn7q' '--without-brotli' '--without-libgsasl' '--without-libpsl' '--without-zstd' '--enable-shared' '--enable-static' '--without-gssapi' '--without-gnutls' '--without-mbedtls' '--without-nss' '--without-openssl' '--with-secure-transport' '--without-libidn2' '--without-librtmp' '--without-nghttp2' '--with-libssh2=/opt/spack/opt/spack/darwin-monterey-m1/apple-clang-13.0.0/libssh2-1.10.0-bnm6auvwfzd3wcq66x2t6kfeuvtseocu' '--without-libssh' '--disable-ldap'

1 error found in build log:
     251    checking for size_t... yes
     252    checking for sys/types.h... (cached) yes
     253    checking for sys/time.h... (cached) yes
     254    checking for sys/socket.h... (cached) yes
     255    checking for struct timeval... yes
     256    checking run-time libs availability... failed
  >> 257    configure: error: one or more libs available at link-time are not available run-time. Libs used at link-time: -lssh2 -lssh2 -lz
```
@haampie I tried again, this time using the `--fresh` argument, e.g. `spack install --fresh curl +libssh2`, and it appears to have worked. Now installing `julia@1.8.0-rc1` to confirm.

However, if I build with the complete set of options Julia requires, I still get a similar configure error:

```
> spack install --fresh curl@7.78 tls=mbedtls +nghttp2 +libssh2 ^libssh2 crypto=mbedtls ^mbedtls libs=shared
...
1 error found in build log:
     271    checking for size_t... yes
     272    checking for sys/types.h... (cached) yes
     273    checking for sys/time.h... (cached) yes
     274    checking for sys/socket.h... (cached) yes
     275    checking for struct timeval... yes
     276    checking run-time libs availability... failed
  >> 277    configure: error: one or more libs available at link-time are not available run-time. Libs used at link-time: -lnghttp2  -lssh2 -lssh2 -lmbedtls -lmbedx5
            09 -lmbedcrypto -lz
```
I suppose this has to do with mbedtls.
I can reproduce this... finally.
The issue here is a combination of things: mbedtls does not set a proper `install_name` (not an absolute path, nor `@rpath/libname.dylib`, but simply `libname.dylib`), and this causes libraries not to be found at runtime, which curl checks.

Spack should rewrite `<filename>` type install names to `@rpath/<filename>`, but that's not done in the compiler wrapper.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/mbedtls/package.py b/var/spack/repos/builtin/packages/mbedtls/package.py
--- a/var/spack/repos/builtin/packages/mbedtls/package.py
+++ b/var/spack/repos/builtin/packages/mbedtls/package.py
@@ -137,3 +137,8 @@ def build(self, spec, prefix):
 
     def install(self, spec, prefix):
         make("install", "DESTDIR={0}".format(prefix))
+
+    @run_after("install")
+    def darwin_fix(self):
+        if self.spec.satisfies("platform=darwin"):
+            fix_darwin_install_name(self.prefix.lib)
diff --git a/var/spack/repos/builtin/packages/xz/package.py b/var/spack/repos/builtin/packages/xz/package.py
--- a/var/spack/repos/builtin/packages/xz/package.py
+++ b/var/spack/repos/builtin/packages/xz/package.py
@@ -53,3 +53,8 @@ def determine_version(cls, exe):
         output = Executable(exe)("--version", output=str, error=str)
         match = re.search(r"xz \(XZ Utils\) (\S+)", output)
         return match.group(1) if match else None
+
+    @run_after("install")
+    def darwin_fix(self):
+        if self.spec.satisfies("platform=darwin"):
+            fix_darwin_install_name(self.prefix.lib)


Test patch: