Code change:
That is, appears to lose incremental artifacts. cc
Yes, I believe this is because rustc/cargo will otherwise error on "found a new std" or something like that? I don't quite see how incremental is different from the normal compilation here?
incremental would still work even if dependencies changed. I think the fundamental problem here is that deletes everything, whereas (which, IIRC, does not remove incremental artifacts by default) would be appropriate.
Hm, I thought was equivalent to essentially -- could you clarify that? Would it be better to use cargo clean in rustbuild?
Regardless of what does I don't think we should execute it here. Rustbuild is overly conservative in what it deletes, and it can get away probably with just deleting the folder instead of the entire folder. That would leave around the incremental cache. I believe, however, that is like
Maybe the benchmarks I remember having for incremental had the incremental dir elsewhere than . I sure was confused by it, since I'd expect to remove all artifacts. If it does do that, then I agree should just remove a subset of the contents of target dirs.
Looking at the code, these calls appear to be relevant: The problem is that the directory refers to is non-trivially structured. Instead of removing , we need to remove all the files (or at least the ones that are artifacts) and the directory in .
I keep hitting this, specifically it keeps rebuilding and , every time I change rustc, which cause them to take much longer than any of the rustc crates, and this issue has become my main bottleneck (now that I'm building on a machine).
Nominating for discussion (specifically, allocating resources towards this) at the next relevant team meeting (tagged both and as I'm not sure which has "jurisdiction" here).
It'd be helpful to know how much rustbuild can "trust" the compiler here. Can we keep incremental artifact directories around from previous versions of the compiler?
The compiler can only work with incremental artifacts that it created itself. For any other version of the compiler it has to assume that it's incompatible. It usually just ignores such incompatible artifacts, but in the bootstrapping scenario, it might not always be smart enough to do that.
I think the question here is will rustc's incremental artifacts correctly deal with crates changing (including adding and removing crates) in the sysroot. If so, we could keep the folder around. Cargo assumes the sysroot is unchanging, which is why we remove the directory in the first place.
Incremental doesn't care about vs , so I'd expect it to be pretty resilient here.
triage: P-medium. (this inefficiency in incremental comp. would be nice to resolve, but is not P-high IMO.)

Related discussion:
text: let mut cargo = Command::new(&self.initial_cargo); let out_dir = self.stage_out(compiler, mode); <del> // command specific path, we call clear_if_dirty with this let mut my_out = match cmd { "build" => self.cargo_out(compiler, mode, target), // This is the intended out directory for crate documentation. "doc" | "rustdoc" =>  self.crate_doc_out(target), _ => self.stage_out(compiler, mode), }; // This is for the original compiler, but if we're forced to use stage 1, then // std/test/rustc stamps won't exist in stage 2, so we need to get those from stage 1, since // we copy the libs forward. let cmp = self.compiler_for(compiler.stage, compiler.host, target); let libstd_stamp = match cmd { "check" | "clippy" | "fix" => check::libstd_stamp(self, cmp, target), _ => compile::libstd_stamp(self, cmp, target), }; let libtest_stamp = match cmd { "check" | "clippy" | "fix" => check::libtest_stamp(self, cmp, target), _ => compile::libtest_stamp(self, cmp, target), }; let librustc_stamp = match cmd { "check" | "clippy" | "fix" => check::librustc_stamp(self, cmp, target), _ => compile::librustc_stamp(self, cmp, target), }; </del> <ins> // Codegen backends are not yet tracked by -Zbinary-dep-depinfo, // so we need to explicitly clear out if they've been updated. for backend in self.codegen_backends(compiler) { self.clear_if_dirty(&out_dir, &backend); } </ins> if cmd == "doc" || cmd == "rustdoc" { <del> if mode == Mode::Rustc || mode == Mode::ToolRustc || mode == Mode::Codegen { </del> <ins> let my_out = match mode { </ins> // This is the intended out directory for compiler documentation. <del> my_out = self.compiler_doc_out(target); } </del> <ins> Mode::Rustc | Mode::ToolRustc | Mode::Codegen => self.compiler_doc_out(target), _ => self.crate_doc_out(target), }; </ins> let rustdoc = self.rustdoc(compiler); self.clear_if_dirty(&my_out, &rustdoc); <del> } else if cmd != "test" { match mode { Mode::Std => { self.clear_if_dirty(&my_out, &self.rustc(compiler)); for backend in self.codegen_backends(compiler) { self.clear_if_dirty(&my_out, &backend); } }, Mode::Test => { self.clear_if_dirty(&my_out, &libstd_stamp); }, Mode::Rustc => { self.clear_if_dirty(&my_out, &self.rustc(compiler)); self.clear_if_dirty(&my_out, &libstd_stamp); self.clear_if_dirty(&my_out, &libtest_stamp); }, Mode::Codegen => { self.clear_if_dirty(&my_out, &librustc_stamp); }, Mode::ToolBootstrap => { }, Mode::ToolStd => { self.clear_if_dirty(&my_out, &libstd_stamp); }, Mode::ToolTest => { self.clear_if_dirty(&my_out, &libstd_stamp); self.clear_if_dirty(&my_out, &libtest_stamp); }, Mode::ToolRustc => { self.clear_if_dirty(&my_out, &libstd_stamp); self.clear_if_dirty(&my_out, &libtest_stamp); self.clear_if_dirty(&my_out, &librustc_stamp); }, } </del> } cargo