Repository: rust
Issue: Backtraces do not work for binaries loaded through ld.so
Labels: E-needs-test,P-medium,T-compiler,C-bug

When a Rust binary that is executed through ld.so — the [Linux dynamic linker](https://unix.stackexchange.com/questions/400621/what-is-lib64-ld-linux-x86-64-so-2-and-why-can-it-be-used-to-execute-file) — triggers a panic, the backtraces are useless. I've included a discussion of why executing through ld.so is valuable below, but first, to reproduce, build a simple binary that panics on execution:

```console
$ cargo new --bin ld-so-backtraces
     Created binary (application) `ld-so-backtraces` package
$ cd ld-so-backtraces
$ echo 'fn main() { panic!() }' > src/main.rs
$ cargo b
   Compiling ld-so-backtraces v0.1.0 (/local/home/jongje/ld-so-backtraces)
    Finished dev [unoptimized + debuginfo] target(s) in 0.46s
```

Backtraces work fine when executing the binary directly:

```console
$ env RUST_BACKTRACE=1 $PWD/target/debug/ld-so-backtraces
thread 'main' panicked at 'explicit panic', src/main.rs:1:13
stack backtrace:
   0: rust_begin_unwind
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panicking.rs:142:14
   2: core::panicking::panic
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panicking.rs:48:5
   3: ld_so_backtraces::main
             at ./src/main.rs:1:13
   4: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
```

But when executing through `ld.so`, the debug symbols vanish:

```console
$ /usr/lib/"ld-linux-$(uname -m).so.1" $PWD/target/debug/ld-so-backtraces
thread 'main' panicked at 'explicit panic', src/main.rs:1:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
$ env RUST_BACKTRACE=1 /usr/lib/"ld-linux-$(uname -m).so.1" $PWD/target/debug/ld-so-backtraces
thread 'main' panicked at 'explicit panic', src/main.rs:1:13
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
$ env RUST_BACKTRACE=full /usr/lib/"ld-linux-$(uname -m).so.1" $PWD/target/debug/ld-so-backtraces
thread 'main' panicked at 'explicit panic', src/main.rs:1:13
stack backtrace:
   0:     0xffffab78c468 - stpcpy
   1:     0xffffab7a3b70 - tunable_list
   2:     0xffffab78a7b4 - __getcwd
   3:     0xffffab78da00 - <unknown>
   4:     0xffffab78d754 - <unknown>
   5:     0xffffab78df4c - <unknown>
   6:     0xffffab78ddf8 - <unknown>
   7:     0xffffab78c910 - getauxval
   8:     0xffffab78db80 - <unknown>
   9:     0xffffab779e60 - _dl_map_object_from_fd
  10:     0xffffab779d7c - _dl_map_object_from_fd
  11:     0xffffab77a0d0 - _dl_map_object_from_fd
  12:     0xffffab77a2e0 - open_verify.constprop.7
  13:     0xffffab77a170 - _dl_map_object_from_fd
  14:     0xffffab77a21c - _dl_map_object_from_fd
  15:     0xffffab788470 - __tunable_set_val
  16:     0xffffab77a1e8 - _dl_map_object_from_fd
  17:     0xffffab77a0fc - _dl_map_object_from_fd
  18:     0xffffab584da4 - __libc_start_main
  19:     0xffffab779fe4 - _dl_map_object_from_fd
```

I suspect this happens because backtraces attempt to load debug symbols through `/proc/self/exe`, but `/proc/self/exe` points to `ld.so` in this case, _not_ the real binary. I don't know of any _good_ solutions to this, but one solution is to prefer `argv[0]` if `/proc/self/exe` is `ld.so`. It won't be perfect (callers need to use `exec -a` among other things), but it'll be better than what happens today. I suspect this happens with other execution wrappers than just `ld.so` (does Valgrind do this for certain kinds of execution for example?), but I don't have any concrete examples.

### Why ld.so?

Where I work, we execute binaries through the dynamic linker so that we can explicitly set the shared library search path to ensure that the same shared libraries used to build a given binary are used when it is executed. It has a similar effect as setting `LD_LIBRARY_PATH`, but with the added benefit that it does not _also_ set the shared library search path of any programs executed down the line. There is a lot of underlying complexity here that is probably not worth digging too much into, but the idea is that a binary should ship "with its environment" (think Nix-style binaries), and so if the built Rust binary and some binary it in turn invokes are built separately, they should use different shared library search paths, namely the ones that map to exactly what they were built with. Executing through the dynamic linker achieves that, `LD_LIBRARY_PATH` does not. (This is also done by things like [go-appimage](https://github.com/probonopd/go-appimage/issues/49)).

Cargo has a [similar issue](https://github.com/rust-lang/cargo/issues/10113), though its solution is going to be a little different since it can also be used a library: https://github.com/rust-lang/cargo/issues/10119.

rust-analyzer also run into [issues with ld.so](https://github.com/rust-lang/rust-analyzer/issues/13159), though from what I can tell there are fixes in the pipeline there already.

### Meta

`rustc --version --verbose`:
```
rustc 1.63.0 (4b91a6ea7 2022-08-08)
binary: rustc
commit-hash: 4b91a6ea7258a947e59c6522cd5898e7c0a6a88f
commit-date: 2022-08-08
host: aarch64-unknown-linux-gnu
release: 1.63.0
LLVM version: 14.0.5
```