Repository: rust
Issue: Building host tools for Solaris
Labels: A-cross,O-SPARC,O-solaris,C-discussion

I'm not quite sure if this is the right place to track these things, but I've been trying to cross build the native host tools for Solaris (both x86 and SPARC).

## x86_64

### Environment

Cross: `x86_64-apple-darwin` → `x86_64-pc-solaris`
Sysroot: see below
Compiler: gcc 11.2.0
binutils: GNU binutils 2.40 + ar LLVM 16 (todo: test with GNU ar)
rust 1.70.0
Local changes to rust: #114078

### Known issues
- [ ] The rust build process sets `-nodefaultlibs` for reasons that aren't entirely clear to me.  To get the build working successfully I had to wrap `gcc` and `g++` with a small script to ensure that `CFLAGS`/`CXXFLAGS` always included `--sysroot=${SYSROOT} -L${SYSROOT}/lib/amd64 -L${SYSROOT}/lib -Wl,-rpath-link,${SYSROOT}/lib/amd64`
- [x] ~~Linking to OpenSSL does not work.  Headers seem to be from ≥ 1.1.0, linker finds libraries for 1.0.2.  The build is stopped at cargo.  Linker errors are all missing symbols (e.g. `OPENSSL_init_ssl`, `BIO_meth_new`).~~ Solved by building sysroot from Solaris packages. Note: Ensure that the build picks up the `pkg-config` info at `${SYSROOT}/usr/openssl/3/lib/64/pkgconfig/libssl.pc` *or* set `OPENSSL_LIB_DIR="${SYSROOT}/usr/openssl/3/lib/64"` and `OPENSSL_INCLUDE_DIR="${SYSROOT}/usr/openssl/3/include"`
- [ ] The installer complains of various uninitialized variables and panics as the script is run with `set -u`.  Related:
  * rust-lang/rust-installer#124
- [x] ~~LLVM does not like the Solaris headers[[1](https://discourse.llvm.org/t/postgresql-12-build-issues-on-solaris-11-4/53278/2)].  There appears to be a workaround in LLVM but it's not being triggered (at least during a cross compile).  Manually undefining the registers in the Solaris headers fixed the build for me, but this seems like it should work out of the box.~~  Solved by building sysroot from Solaris packages.
- [x] `ar` and `ranlib` need to be modern-ish.  When using an older `ar` the static LLVM support libraries are empty and the build fails with a load of undefined references to LLVM symbols.  For the x86_64 build I used an LLVM `ar` from macports via `CMAKE_AR` and resolved the issue.  For the sparc build I used `ar` and `ranlib` from GNU binutils 2.40. Setting `ar` and `ranlib` in `config.toml` should fix @3eka's issue with undefined symbols. This is set per-target e.g. `target.x86_64-pc-solaris.ar` or `target.sparcv9-sun-solaris.ar` and must be an *absolute* path.
- [ ] `rust-analyzer` does not compile due to `nix` incorrectly assuming `flock` exists in Solaris (apparently it hasn't since at least 5.x) [[1](https://www.perkin.org.uk/posts/solaris-portability-flock.html), [2](https://stackoverflow.com/questions/59033269/using-flock-on-solaris-5-10)].  Related:
  * nix-rust/nix#2082
  * rust-lang/libc#3310

### Status

With this in place I can build these artifacts from 1.70.0:
```
cargo-nightly-x86_64-pc-solaris.tar.xz
rust-analysis-nightly-x86_64-pc-solaris.tar.xz
rust-analyzer-nightly-x86_64-pc-solaris.tar.xz
rust-demangler-nightly-x86_64-pc-solaris.tar.xz
rust-dev-nightly-x86_64-pc-solaris.tar.xz
rust-nightly-x86_64-pc-solaris.tar.xz
rust-std-nightly-x86_64-pc-solaris.tar.xz
rustc-dev-nightly-x86_64-pc-solaris.tar.xz
rustc-nightly-x86_64-pc-solaris.tar.xz
rustfmt-nightly-x86_64-pc-solaris.tar.xz
```

Manually installing the `rust-std-nightly` and `rustc-nightly` tarballs got me an install that successfully compiled a "Hello, world" program with the Oracle provided gcc as the linker.

## SPARCv9

### Environment

Cross: `x86_64-apple-darwin` → `sparcv9-sun-solaris`
Sysroot: see below
Compiler: gcc 11.2.0
binutils: GNU binutils 2.40
rust 1.70.0
Local changes to rust: #114078

### Known issues
- [x] `ar` and `ranlib` need to be modern-ish. See above.
- [ ] Due to using `gcc` as the cross compiler(?) I had to explicitly set `LLVM_TOOL_LLVM_LIBTOOL_DARWIN_BUILD` to `OFF` in `src/bootstrap/llvm.rs` because Apple's `libtool(1)` doesn't seem to know what to do with ELF/SPARC objects.  This should probably be part of a larger discussion on whether cross compiling from an Apple → non-Apple target is a worthwhile goal.
- [ ] `llvm-wrapper/PassWrapper.cpp` calls `llvm::sys::getHostCPUName` which on Solaris/SPARC calls into `libkstat`.  `libLLVMTargetParser` which contains `getHostCPUName` does not appear to link against `libkstat` Somewhere along the way we should be linking against `libkstat`. Related:
  * #114224
  * llvm/llvm-project#64186.

### Status

I can build these artifacts from 1.70.0:

```
cargo-nightly-sparcv9-sun-solaris.tar.xz
rust-analysis-nightly-sparcv9-sun-solaris.tar.xz
rust-analyzer-nightly-sparcv9-sun-solaris.tar.xz
rust-demangler-nightly-sparcv9-sun-solaris.tar.xz
rust-dev-nightly-sparcv9-sun-solaris.tar.xz
rust-nightly-sparcv9-sun-solaris.tar.xz
rust-std-nightly-sparcv9-sun-solaris.tar.xz
rustc-dev-nightly-sparcv9-sun-solaris.tar.xz
rustc-nightly-sparcv9-sun-solaris.tar.xz
rustfmt-nightly-sparcv9-sun-solaris.tar.xz
```

## Sysroot

Package information from Oracle is available here:

http://pkg.oracle.com/solaris/release/en/catalog.shtml

Solaris packages contain files for both x86 and SPARC architectures.  The following packages should be sufficient to create a usable sysroot for cross compiling:

```
# system headers
system/header@11.4,5.11-11.4.42.0.0.111.1:20211203T213657Z

# Core system libraries
system/library@11.4,5.11-11.4.42.0.0.111.1:20211203T214210Z

# ld.so, crt*
system/linker@11.4-11.4.0.0.1.15.0:20180817T003258Z

# libc
system/library/libc@11.4-11.4.42.0.0.111.1:20211203T214042Z

# libm
system/library/math@11.4-11.4.0.1.0.17.0:20200221T005425Z

# libucrypto
system/library/security/crypto@11.4,5.11-11.4.42.0.0.111.1:20211203T214125Z

# libssl v3
library/security/openssl-3@3,11.4-11.4.42.0.0.111.0:20211203T203706Z

# libz
library/zlib@1.2.11,11.4-11.4.42.0.0.111.0:20211203T203816Z
```