Repository: rust
Issue: As of 1.37.0, `dylib` shared libraries no longer support interpositioning of functions defined in C
Labels: A-linkage,P-medium,T-compiler,regression-from-stable-to-stable,C-bug

I encountered a stable-to-stable linking regression while working with a project that compiles to an x86-64 ELF shared library for Linux.  The library exports a Rust interface, but also includes some wrappers of libc functions, written in C, that need to shadow the system implementations via interpositioning.  I perform the final linking step using rustc, which works correctly under rustc 1.36.0 but subtly fails under 1.37.0: the libc wrapper functions are not exported as dynamic symbols, causing the program to behave differently at runtime.  The offending patchset appears to be https://github.com/rust-lang/rust/pull/59752, and I've managed to construct a minimal example to illustrate the problem...

# Minimal example
The library consists of two files:
* `interpose.rs` defines a single-function Rust API:
```
#![crate_type = "dylib"]

pub fn relax() {
	println!("Relax said the night guard");
}
```
* `exit.c` defines an implementation of `exit()` that should shadow the one in libc:
```
#include <stdio.h>
#include <stdlib.h>

void exit(int ign) {
	(void) ign;
	puts("We are programmed to receive");
}
```
The following short program, `hotel_california.rs`, will be used to test its behavior:
```
extern crate interpose;

use interpose::relax;
use std::os::raw::c_int;

extern {
	// NB: Deliberately returns () instead of ! for the purpose of this example.
	fn exit(_: c_int);
}

fn main() {
	relax();
	unsafe { exit(1); }
	println!("You can check out any time you like but you can never leave");
}
```

# Expected behavior (past stable releases)
This is how the program used to behave when built with a stable compiler:
```
$ rustc --version
rustc 1.36.0 (a53f9df32 2019-07-03)
$ rustc -Cprefer-dynamic -Clink-arg=exit.o interpose.rs
$ rustc -L. -Crpath hotel_california.rs
$ ./hotel_california
Relax said the night guard
We are programmed to receive
You can check out any time you like but you can never leave
$ echo $?
0
```
Notice that the call to `exit()` gets intercepted and does not, in fact, exit the program.

# Broken behavior (as of 1.37.0 stable)
Newer versions of the compiler result in different program output:
```
$ rustc --version
rustc 1.37.0
$ rustc -Cprefer-dynamic -Clink-arg=exit.o interpose.rs
$ rustc -L. -Crpath hotel_california.rs
$ ./hotel_california
Relax said the night guard
$ echo $?
1
```

# Discussion: symbol table entries
The problem is evident upon examining the static and dynamic symbol tables of the `libinterpose.so` file.  When built with rustc 1.36.0, we see that `exit` is exported in the dynamic symbol table (indicated by the `D`):
```
$ objdump -tT libinterpose.so | grep exit$
000000000000118c g     F .text  000000000000001a exit
000000000000118c g    DF .text  000000000000001a exit
```
In contrast, the output from rustc 1.37.0 doesn't list `exit` in the dynamic symbol table because the static symbol table lists it as a local symbol (`l`) rather than a global one (`g`):
```
$ objdump -tT libinterpose.so | grep exit$
000000000000118c l     F .text  000000000000001a exit
```

# Discussion: linker invocation
I was curious to see how rustc was invoking cc to link the program, so I traced the command-line arguments by substituting the fake linker `false`.  Here's with rustc 1.36.0:
```
$ rustc -Clinker=false -Cprefer-dynamic -Clink-arg=exit.o interpose.rs
error: linking with `false` failed: exit code: 1
  |
  = note: "false" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/solb/Desktop/rust-1.36/lib/rustlib/x86_64-unknown-linux-gnu/lib" "interpose.interpose.3a1fbbbh-cgu.0.rcgu.o" "interpose.interpose.3a1fbbbh-cgu.1.rcgu.o" "-o" "libinterpose.so" "in
terpose.54bybojgvbim5uqh.rcgu.o" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/home/solb/Desktop/rust-1.36/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-L" "/home/solb/Desktop/rust-1.36/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-lst
d-9895e8982b0a79e7" "-Wl,--end-group" "-Wl,-Bstatic" "/tmp/user/1000/rustchJWjrY/libcompiler_builtins-38e90baf978bc428.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil" "-shared" "exit.o"
  = note: 

error: aborting due to previous error
```
And with rustc 1.37.0:
```
$ rustc -Clinker=false -Cprefer-dynamic -Clink-arg=exit.o interpose.rs
error: linking with `false` failed: exit code: 1
  |
  = note: "false" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib" "interpose.interpose.3a1fbbbh-cgu.0.rcgu.o" "interpose.interpose.3a1fbbbh-cgu.1.rcgu.o" "-o" "libinterpose.so" "-Wl,--version-script=/tmp/
user/1000/rustc7Re7af/list" "interpose.54bybojgvbim5uqh.rcgu.o" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-L" "/usr/lib/x86_64-linux-gnu" "-lstd-6c8733432f42c6a2" "-Wl,--end-group"
 "-Wl,-Bstatic" "/tmp/user/1000/rustc7Re7af/libcompiler_builtins-67541964815c9eb5.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil" "-shared" "-Wl,-soname=libinterpose.so" "exit.o"
  = note: 

error: aborting due to previous error
```
Notice the newly-added `-Wl,--version-script` flag, which has no knowledge of the symbols from the `exit.o` object file.

# Discussion: static library instead of bare object file
One might be tempted to work around the problem by telling rustc about the object file so it can keep the symbols it defines global.  I tried this on rustc 1.36.0:
```
$ ar rs libexit.a exit.o
ar: creating libexit.a
$ rustc -Cprefer-dynamic -L. interpose.rs -lexit
$ rustc -L. -Crpath hotel_california.rs
```
This has a very surprising result: the exit symbol is not present at all in `libinterpose.so`, but it does exist somewhere (the LLVM bitcode for monomorphization, maybe?) that allows the compiler to statically link it into the **executable**:
```
$ objdump -tT libinterpose.so | grep exit$
$ objdump -tT hotel_california | grep exit$
0000000000001337 g     F .text  000000000000001a              exit
0000000000001337 g    DF .text  000000000000001a  Base        exit
```
This is no good either because it leads to subtly different interposition behavior.  For example:
* Before, `exit()` could be further shadowed by libraries loaded via the `LD_PRELOAD` environment variable.  Building it directly into the executable breaks this.
* The `cc` and `ld` apply very different optimizations to `exit()` because it is now part of a PIE instead of a PIC object; depending on how wrapping is implemented, this can break it and even result in infinite recursion.
* If a C program links against `libinterpose.so`, it will no longer get the interposed version of `exit()`.  This is a very real situation for my project, because it also exports a C API via Rust's FFI.

# Possible mitigation: expose a `-Climit_rdylib_exports` command-line switch
The simplest way to allow users to work around this would be to allow invokers of rustc to opt out of the change introduced by https://github.com/rust-lang/rust/pull/59752.  However, the change is likely to have broken other use cases as well, so perhaps it needs to be revisited in more detail.

# See also
The same changeset seems to be causing problems with inline functions, as observed at https://github.com/rust-lang/rust/issues/65610.