Change:
} } <ins> /// If a tool does not pass its tests, don't ship it. /// Right now, we do this only for Miri. fn check_toolstate(&mut self) { // Get the toolstate for this rust revision. let rev = self.rust_git_commit_hash.as_ref().expect("failed to determine rust git hash"); let toolstates = reqwest::get(TOOLSTATE).expect("failed to get toolstates"); let toolstates = BufReader::new(toolstates); let toolstate = toolstates.lines() .find_map(|line| { let line = line.expect("failed to read toolstate lines"); let mut pieces = line.splitn(2, 't'); let commit = pieces.next().expect("malformed toolstate line"); if commit != rev { // Not the right commit. return None; } // Return the 2nd piece, the JSON. Some(pieces.next().expect("malformed toolstate line").to_owned()) }) .expect("failed to find toolstate for rust commit"); let toolstate: HashMap<String, String> = serde_json::from_str(&toolstate).expect("toolstate is malformed JSON"); // Mark some tools as missing based on toolstate. if toolstate.get("miri").map(|s| &*s as &str) != Some("test-pass") { println!("Miri tests are not passing, removing component"); self.miri_version = None; self.miri_git_commit_hash = None; } } /// Hash all files, compute their signatures, and collect the hashes in `self.digests`. </ins> fn digest_and_sign(&mut self) { for file in t!(self.input.read_dir()).map(|e| t!(e).path()) { let filename = file.file_name().unwrap().to_str().unwrap();

Discussion:
Looking at https://rust-, it seems like Miri is getting shipped as a rustup component even though the tests fail. Unfortunately, Miri is currently entirely usable, so it'd probably be better if it did not get shipped in this state. Is that something we can do? Cc
maybe we can make the miri toolstate always go to even if it technically is just
what would be the best place to change this? I think I'd prefer Miri not to be shipped at all when the tests fail; I'd rather people stick to an old-but-working Miri than to a new-but-broken one. I found but that does not seem to decide if a component gets built for distribution or not.
seems to suggest that it is supposed to work as you describe that it should be working. Since it obviously doesn't and I can't see how that comment could be true from looking at the source, I'm guessing this is some refactoring fallout. I'll dig a little into git blame
I found it. The comment was for rls in but then the code making the comment true was removed later
See for why this is hard to fix: the builders don't even run the tool tests, so they cannot know if the tests pass. :/ Also, that's probably for the better actually, as the "tools" builder sets --cfg miri to be more Miri-friendly and we don't want that in the release artifacts.
One (crazy?) thing we could so is have the dist builders query the toolstate repo to figure out if the tests are passing...
That will either retrieve an outdated toolstate or will need to block until the toolstate is uploaded ...
Without blocking it would lag behind by 1 merged PR, yes.
would it be possible to add some kind of build assert so Miri fails on builders when Rust internals change too much?
Well builders already know when Miri fails to build because internals change too much. The issue is things that make Miri still compile but break the test suite.
would it be possible to at least change the logic that opens issues such that for Miri, it opens an issue like when the tests fail? I could also try to implement that if you point me to the right place. So far we have had zero cases of Miri tests spuriously failing.

That should be easy to do:
Gave it a shot at
had an interesting idea: can't we do the "don't ship broken tools" as part of a post-processing step? We anyway do some little amount of work when a commit hits (as opposed to ), such as actually updating the toolstate repo. So maybe that could also either delete the broken components from the S3 bucket, or else generate a file to be put into the S3 bucket that says which of the components are really there so that the broken ones can be considered "not there"? do you think something like that would be possible?
Thinking about this again, probably the most natural place to do this is in the . That could "just" also check the toolstate database for the commit it is promoting, and then (possibly only for certain tools) not add them to the nightly if their toolstate is not green.
This is biting us again right now, where CI knows that Miri is broken but we ship the component anyway, leading to CI failures in downstream projects that have Miri in their CI.
For another case where this issue got quite painful, see
My PR is because I was incorrectly assuming that all commits have their toolstate recorded in the toolstate repo. Reopening.
FTR, for testing , I downloaded some artifacts: then I edited my : and finally I ran This produces stuff in .
So looks like we have logic in that specifically does not commit to the toolstate repo if nothing changed: Unfortunately, this means that we cannot easily fetch the toolstate of the to-be-produced release in build-manifest, which caused my to fail. Would it be okay for you to change this logic to record the toolstate of all commits in the toolstate repo? Or would you prefer if I try to find another approach for solving this issue? Right now I am somewhat out of ideas though if we cannot even use the toolstate repo to communicate this information.
Wouldn't it be simpler to save list of "good" components to something like and upload it alongside tarballs? Running would show what you need.
I guess we could upload the two JSON files from the two tool runners somehow? Not sure which would be the right places to hook for that, though.
Status: blocked on which is blocked on
This seems to be finally fixed!

Today nightly was shipped with , and everything looks normal :tada:
Miri is "test-fail" now, so with the next nightly we should be able to see if indeed it gets removed from rustup as it should.
Confirmed it's working! On miri is missing.