Code change:
I think this is the fourth beta I'm having to re-figure-out how all this works and figure out how to ignore their failures on beta. can we just exclude these tools from toolstate until this is figured out?
I'm slightly confused, why would these tools need to be disabled/their submodule update if they were building on master? Isn't the beta just the current master branch + some tweaks for --version output and no unstable features? The release in March actually prevented the merge of any PRs on master that would break a tool. So I thought the idea was to fix all tools in that week. Or is this about beta backports breaking tools?
I don't think we've had a beta yet whether either clippy or miri were passing tests and fails the build if any tool fails to build on beta. We also do not want to release either of these tools to stable/beta yet.
neither of these tools has a step, so there's no danger in that (the last beta) didn't have clippy issues. Since we have grace period of a week, I can easily get in a working clippy within that week (which then won't be broken until the beta). My issue is mainly that I don't really know when it begins ;) Maybe we can ping the tool authors once a day that their tool needs to be fixed for the upcoming beta?
My point with this issue is that we shouldn't automate ourselves into a hole where tools we do not want to ship block releases. That should not require authors to fix tools, we should simply ignore failures or have an easy way of disabling them. Right now producing a beta is pain as you have to resend it to bors multiple times after re-learning that these are blocking the release. We do not ship miri/clippy but a successful clippy compilation affects how rls is compiled, which should not be happening on beta/stable.
The code that influences rls is explicitly skipped on beta/stable, so the rls is completely free of clippy on beta/stable for miri I agree, but for clippy, aren't we trying to move to a model where we actually dist it on stable, so I'd call having it ready for stable some form of proof of concept for the distributing part.
Ok sure yes rls is protected but I feel like this is missing my point. Miri isn't ready. Clippy isn't ready. They're blocking beta releases. That shouldn't happen.

Related discussion:
text: fi } <ins> # deduplicates the submodule check and the assertion that on beta some tools MUST be passing check_dispatch() { if [ "$1" = submodule_changed ]; then # ignore $2 (branch id) verify_status $3 $4 elif [ "$2" = beta ]; then echo "Requiring test passing for $3..." if grep -q '"'"$3"'":"(test|build)-fail"' "$TOOLSTATE_FILE"; then exit 4 fi fi } # list all tools here status_check() { check_dispatch $1 beta book src/doc/book check_dispatch $1 beta nomicon src/doc/nomicon check_dispatch $1 beta reference src/doc/reference check_dispatch $1 beta rust-by-example src/doc/rust-by-example check_dispatch $1 beta rls src/tool/rls check_dispatch $1 beta rustfmt src/tool/rustfmt # these tools are not required for beta to successfully branch check_dispatch $1 nightly clippy-driver src/tool/clippy check_dispatch $1 nightly miri src/tool/miri } </ins> # If this PR is intended to update one of these tools, do not let the build pass # when they do not test-pass. <del> verify_status book src/doc/book verify_status nomicon src/doc/nomicon verify_status reference src/doc/reference verify_status rust-by-example src/doc/rust-by-example verify_status rls src/tool/rls verify_status rustfmt src/tool/rustfmt verify_status clippy-driver src/tool/clippy verify_status miri src/tool/miri </del> <ins> status_check "submodule_changed" </ins> if [ "$RUST_RELEASE_CHANNEL" = nightly -a -n "${TOOLSTATE_REPO_ACCESS_TOKEN+is_set}" ]; then . "$(dirname $0)/repo.sh"