Change:
Hi all, my usual test: Can Rust compile itself after a compiler upgrade. Rust 1.43 dies at this stage: I didn't identify the problematic code yet, I just wanted to create a tracking issue first. Rust 1.43 can compile other crates fine, I quickly tested rust-zip or rust-askama.
Do you need this for anything? This is not a supported configuration
yes, Linux distributions often need to recompile a (compiler) package with some patches applied or after a major LLVM / glibc / other major update. See for the discussion on the Rust 1.42.0 release.
Really? I thought this has been an attempted guarantee since years ago. (Even though it was broken for various previous releases.) Debian certainly needs this guarantee and I will have to figure out how to fix this when packaging 1.43 for Debian. This one seems like a new case not similar to the previous cases, gah.
Well, we don't test this at all, so it's not surprising that it breaks almost every release. Why can't Debian just use the previous release to bootstrap?
We do, but the porters (for tier-2 architectures like riscv64) and other people occasionally need to rebuild stuff afterwards, when the old version is no longer available (has been moved to archives).
It's not really unsupported so much as not tested. I would ask that if Debian (or others) want this to work we find out before the stable release -- ideally when beta is cut, or earlier -- that way we can do much more to fix it. I'll try to take a look at this specific failure soon and see if there's a simple patch.
Bootstrapping rustc twice would make merging PRs much slower than it's today. However, maybe one could add it only to the beta branch's CI. Or, alternatively, add rustc to crater. The "old state" would serve as check whether it can bootstrap normally, the "new state" would check whether it can bootstrap itself. There's a proposal to add servo already:
I don't think there's been any proposal that we test this in PR CI? To be clear, I don't currently believe that's something we want to do, though we'd generally be willing to accept PRs that make self-bootstrapping possible, which is why I suggested testing on beta releases (instead of stable releases, where we can't do much so all patches must be downstream).
It wasn't in response to any proposal or such, just about how to stop regressions like this from happening. It has been mentioned in the thread that it's not being tested. So naturally one would wonder how the testing would look like so I wrote down my thoughts. Also, yes, optimally the users would test beta releases (question to have you tested 1.44 beta yet?), and I've filed a pr for rav1e today to do precisely this, but the entire premise of CI as well as crater is to be proactive about breakage.
Understood, I will make some more effort to test the beta releases! We had done so a few times in the past but not really made it a regular part of the process.
I don't think there's been any proposal that we test this in PR CI? I agree it would be too onerous for regular PRs, but perhaps it could be done when anointing nightly as beta, and then on all PRs to beta, that sounds not too expensive.
Have you found this yet? Maybe something wrong with your ? I also don't see "messages will be sent to 'stderr'" anywhere in the Rust sources, but googling indicates this often comes from syslog. I don't know why that would be fed into your stdin though... FWIW, the automatic rebuild in Fedora rawhide was just fine:
Yesterday evening I continued with the investigation. I managed to recompile the Fedora 31 source rpm on my Fedora 31 workstation. So it must be something triggered by my "custom distro" setup. The plan for today is to limit the number of parallel build jobs to one and run "forkstat" in the background. I want to isolate the exact command invocation that triggers the error. The /dev/null device should be fine, the build process is not running inside a container or chroot.
I think we might be onto something here. After I had another failed Rust 1.43.0 re-compile attempt, I noticed /dev/null was broken: (tmux crashed on startup, that's how I noticed) I've rebooted the devbox, /dev/null is fine again and I will now compile the previous Rust 1.42.0 twice. If /dev/null stays fine, I'll re-try with Rust 1.43.0 and see if /dev/null gets replaced.
the only remotely related change to /dev/null handling from 1.42.0 to 1.43.0 is this in It looks totally unsuspicious to me.
That kind of error usually occurs when one uses something like , versus redirecting , since usually writes temp names first and then renames over the target. I did find this in one test, new in 1.42 via :
When I run that test manually as root, I do indeed get a clobbered : warning: unused import: --src/test/ui/non-ice-error-on-worker-io- | 34 | use crate::task::Waker; | ^^^^^^^^^^^^^^^^^^ | = note: on by default -rw-r--r--. 1 root root 3562 Apr 28 09:48 /dev/null
Running tests as root is indeed very trusting :) But it seems like in this case we should probably write to e.g. /dev/rust-nonexistent-path instead of null to avoid the accidental breakage? Alternatively maybe we can move this test to a make test and chmod our permissions away...
I don't know if a nonexistent path will exercise the intended failure mode, but a simple chmod is insufficient to stop root.
There's the immutable xattr, even root can't delete those files. This is how Android malware persists across factory resets: But requiring filesystem xattrs just for one unit test is a no no. Another option would be a read only bind mount, but again, the setup is overkill. May be a read only place in proc like /proc/version would do: I've started a build without the unit test in question for the night, results will be in tomorrow.
with the mentioned unit test disabled, I can finally build Rust 1.43.0 with itself. Rustception all the way, thanks! I've verified again it didn't blow up with Rust 1.42.0, even though the unit test was part of it already. May be the way rustc handles output file creation changed in 1.43.0, so these two in combination led to failure. I've tested the /proc/version approach: Using an existing directory as output file gives a different error message:
I think will be fine, even though it's a different IO error. It still triggers the old ICE on 1.41.0, and is caught as a proper fatal error on 1.42+, even as root.
I'd be happy to review a PR here, FWIW, it sounds like it should be a pretty simple update to the UI test.
Just wanted to say thank you for the quick fix and happy birthday to Rust! Also Rust 1.43.1 compiled itself without problems.

Discussion:
// // An attempt to `-o` into a directory we cannot write into should indeed // be an error; but not an ICE. <ins> // // However, some folks run tests as root, which can write `/dev/` and end // up clobbering `/dev/null`. Instead we'll use a non-existent path, which // also used to ICE, but even root can't magically write there. </ins> <del> // compile-flags: -o /dev/null </del> <ins> // compile-flags: -o /does-not-exist/output </ins> // The error-pattern check occurs *before* normalization, and the error patterns // are wildly different between build environments. So this is a cop-out (and we

// error-pattern: error // On Mac OS X, we get an error like the below <del> // normalize-stderr-test "failed to write bytecode to /dev/null.non_ice_error_on_worker_io_fail.*" -> "io error modifying /dev/" </del> <ins> // normalize-stderr-test "failed to write bytecode to /does-not-exist/output.non_ice_error_on_worker_io_fail.*" -> "io error modifying /does-not-exist/" </ins> // On Linux, we get an error like the below <del> // normalize-stderr-test "couldn't create a temp dir.*" -> "io error modifying /dev/" </del> <ins> // normalize-stderr-test "couldn't create a temp dir.*" -> "io error modifying /does-not-exist/" </ins> // ignore-tidy-linelength // ignore-windows - this is a unix-specific test

warning: ignoring --out-dir flag due to -o flag <del> error: io error modifying /dev/ </del> <ins> error: io error modifying /does-not-exist/ </ins> error: aborting due to previous error; 1 warning emitted