Code change:
That is, appears to lose incremental artifacts. cc
Yes, I believe this is because rustc/cargo will otherwise error on "found a new std" or something like that? I don't quite see how incremental is different from the normal compilation here?
incremental would still work even if dependencies changed. I think the fundamental problem here is that deletes everything, whereas (which, IIRC, does not remove incremental artifacts by default) would be appropriate.
Hm, I thought was equivalent to essentially -- could you clarify that? Would it be better to use cargo clean in rustbuild?
Regardless of what does I don't think we should execute it here. Rustbuild is overly conservative in what it deletes, and it can get away probably with just deleting the folder instead of the entire folder. That would leave around the incremental cache. I believe, however, that is like
Maybe the benchmarks I remember having for incremental had the incremental dir elsewhere than . I sure was confused by it, since I'd expect to remove all artifacts. If it does do that, then I agree should just remove a subset of the contents of target dirs.
Looking at the code, these calls appear to be relevant: The problem is that the directory refers to is non-trivially structured. Instead of removing , we need to remove all the files (or at least the ones that are artifacts) and the directory in .
I keep hitting this, specifically it keeps rebuilding and , every time I change rustc, which cause them to take much longer than any of the rustc crates, and this issue has become my main bottleneck (now that I'm building on a machine).
Nominating for discussion (specifically, allocating resources towards this) at the next relevant team meeting (tagged both and as I'm not sure which has "jurisdiction" here).
It'd be helpful to know how much rustbuild can "trust" the compiler here. Can we keep incremental artifact directories around from previous versions of the compiler?
The compiler can only work with incremental artifacts that it created itself. For any other version of the compiler it has to assume that it's incompatible. It usually just ignores such incompatible artifacts, but in the bootstrapping scenario, it might not always be smart enough to do that.
I think the question here is will rustc's incremental artifacts correctly deal with crates changing (including adding and removing crates) in the sysroot. If so, we could keep the folder around. Cargo assumes the sysroot is unchanging, which is why we remove the directory in the first place.
Incremental doesn't care about vs , so I'd expect it to be pretty resilient here.
triage: P-medium. (this inefficiency in incremental comp. would be nice to resolve, but is not P-high IMO.)

Related discussion:
text: deps.push((path_to_add.into(), false)); } <del> // Now we want to update the contents of the stamp file, if necessary. First // we read off the previous contents along with its mtime. If our new // contents (the list of files to copy) is different or if any dep's mtime // is newer then we rewrite the stamp file. </del> deps.sort(); <del> let stamp_contents = fs::read(stamp); let stamp_mtime = mtime(&stamp); </del> let mut new_contents = Vec::new(); <del> let mut max = None; let mut max_path = None; </del> for (dep, proc_macro) in deps.iter() { <del> let mtime = mtime(dep); if Some(mtime) > max { max = Some(mtime); max_path = Some(dep.clone()); } </del> new_contents.extend(if *proc_macro { b"h" } else { b"t" }); new_contents.extend(dep.to_str().unwrap().as_bytes()); new_contents.extend(b"0"); } <del> let max = max.unwrap(); let max_path = max_path.unwrap(); let contents_equal = stamp_contents .map(|contents| contents == new_contents) .unwrap_or_default(); if contents_equal && max <= stamp_mtime { builder.verbose(&format!("not updating {:?}; contents equal and {:?} <= {:?}", stamp, max, stamp_mtime)); return deps.into_iter().map(|(d, _)| d).collect() } if max > stamp_mtime { builder.verbose(&format!("updating {:?} as {:?} changed", stamp, max_path)); } else { builder.verbose(&format!("updating {:?} as deps changed", stamp)); } </del> t!(fs::write(&stamp, &new_contents)); deps.into_iter().map(|(d, _)| d).collect() }