Repository: rust
Issue: wasm-panic-small test has been failing for a while, CI doesn't seem to realise?
Labels: A-testsuite

The test `tests/run-make/wasm-panic-small` has been failing for some time now, at least when I run it. I had a quick skim through some Continuous Integration logs which seem to [report it as "ok"](https://github.com/rust-lang-ci/rust/actions/runs/6119116689/job/16608418655#step:24:2755) (Github is janky with big logs, you might need to search "wasm-panic-small" or scroll to line 2755).

My reproduction steps are this:
1. Clone the rust repo
2. place this config.toml in the repo directory:
```
# Includes one of the default files in src/bootstrap/defaults
profile = "compiler"
changelog-seen = 2

[build]
target = ["wasm32-unknown-unknown","x86_64-unknown-linux-gnu"]

[rust]
# need this, or wasm-panic-small test gives "error: linker `rust-lld` not found"
lld = true
```
3. Run `./x test tests/run-make/wasm-panic-small`
4. Gives this output
```
--- stdout -------------------------------
LD_LIBRARY_PATH="/home/matt/rust/build/x86_64-unknown-linux-gnu/test/run-make/wasm-panic-small/wasm-panic-small:/home/matt/rust/build/x86_64-unknown-linux-gnu/stage1/lib:/home/matt/rust/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/home/matt/rust/build/x86_64-unknown-linux-gnu/stage0/lib" '/home/matt/rust/build/x86_64-unknown-linux-gnu/stage1/bin/rustc' --out-dir /home/matt/rust/build/x86_64-unknown-linux-gnu/test/run-make/wasm-panic-small/wasm-panic-small -L /home/matt/rust/build/x86_64-unknown-linux-gnu/test/run-make/wasm-panic-small/wasm-panic-small  -Ainternal_features foo.rs -C lto -O --target wasm32-unknown-unknown --cfg a
wc -c < /home/matt/rust/build/x86_64-unknown-linux-gnu/test/run-make/wasm-panic-small/wasm-panic-small/foo.wasm
7361
[ "`wc -c < /home/matt/rust/build/x86_64-unknown-linux-gnu/test/run-make/wasm-panic-small/wasm-panic-small/foo.wasm`" -lt "1024" ]
------------------------------------------
```

The test compiles some small functions and tests that the size of the binary output is smaller than the expected values. The test fails because the output is several times larger than it expects. It's strange that CI doesn't seem to have caught this?

# lto
Edit: I think I understand this one now, see edit below.
Another strange thing, that should maybe have its own issue, happens when you add `lto = "thin-local"` to the end of the aforementioned config.toml.

The lto option is described in config.example.toml like this:
```
# Select LTO mode that will be used for compiling rustc. By default, thin local LTO
# (LTO within a single crate) is used (like for any Rust crate). You can also select
# "thin" or "fat" to apply Thin/Fat LTO to the `rustc_driver` dylib, or "off" to disable
# LTO entirely.
#lto = "thin-local"
```
So Thin Link Time Optimisation should only be applied when building rustc itself.
However, when I ran `./x test tests/run-make/wasm-panic-small` again, I got
```
... snip ...
wc -c < /home/matt/rust/build/x86_64-unknown-linux-gnu/test/run-make/wasm-panic-small/wasm-panic-small/foo.wasm
7050
[ "`wc -c < /home/matt/rust/build/x86_64-unknown-linux-gnu/test/run-make/wasm-panic-small/wasm-panic-small/foo.wasm`" -lt "1024" ]
------------------------------------------
```
It says **7050** now, the binary has changed size! I might be missing something, but this seems fishy to me. It seems maybe options for compiling the compiler are leaking through to compiling the tests? That doesn't seem right, the tests should always build with the same environment and output the same binary, regardless of whether the compiler used is a debug one or was linked in a certain way etc.
EDIT: I realise now that this is probably because the *standard library* was compiled with LTO on, and then linked into the final binary. We should change the comment in config.example.toml to make that a little clearer.

I've reproduced these bugs on two different Ubuntu machines, so I don't think its just my configuration.
Thanks for reading!