Change:
CC On today's git () seems to get stuck if you are building docs. In : will get stuck at: The underlying command is: Adding this to the config seems to work around the hang: Thanks!
How long did you let the command run before killing it? I remember installing the docs taking a long time last time I did it.
I left it for half an hour or so IIRC. I'll run it for longer and see if it goes away. How long do you think it should take to install on a modern SSD?
I don't remember exactly how long it takes, but it doesn't depend on the speed of the disk. is just really inefficient. You can see if it's actually hanging by checking the contents of this command while it's installing:
The log file is empty, but I can see new files appearing on the disk (using and ) so progress must be occurring. Slowly.
It's been about 40 minutes now. It seems to be adding files to is that right?
So it's just very slow.
I took a look into this. The main issue is that the generated is very inefficient; it takes about 5 minutes to copy 2500 files (there are more than 2500, but I got bored and cancelled it). what do you think about rewriting this inner loop in Rust so it's not launching so many external processes?
This issue is compounded on macOS, where it seems to be even more inefficient to launch lots of processes from a shell. Sorry I don't have timings, but it takes tens of minutes at least for this step [edit: over an hour]; long enough that I would expect most users who are trying to build our software from source in macOS (it's Zcash, which installs a pinned version of Rust in order to build) are likely to think it's hanging and give up.
I think the problem may largely be that the generated for rust-docs component (at least) is a file: list instead of using the directory: command; directory commands should be much more efficient. This change may be a recent(ish) regression; I suspect that fixing that is likely the right solution here.
Oops I forgot to link back here: we discussed this briefly and it was difficult to do in a cross-platform way.
We solved our problem for Zcash by just not installing rust-docs (passing to ). This reduced our total build time on macOS ! Obviously it doesn't solve the underlying issue.
FYI, addresses this (not the underlying cause of the inefficiency but the majority of its extent).
I bisected the underlying issue to “bootstrap: convert rust-docs to use Tarball” (). Before this commit, rust-docs installs in half a second, and is 1 line: After this commit, rust-docs installs in some twenty minutes, and is lines:
Proposed fix in . (We should also do the optimization in rust-lang/rust-installer of course.)

Discussion:
let mut tarball = Tarball::new(builder, "rust-docs", &host.triple); tarball.set_product_name("Rust Documentation"); <del> tarball.add_dir(&builder.doc_out(host), dest); </del> <ins> tarball.add_bulk_dir(&builder.doc_out(host), dest); </ins> tarball.add_file(&builder.src.join("src/doc/robots.txt"), dest, 0o644); Some(tarball.generate()) }

let mut tarball = Tarball::new(builder, "rustc-docs", &host.triple); tarball.set_product_name("Rustc Documentation"); <del> tarball.add_dir(&builder.compiler_doc_out(host), "share/doc/rust/html/rustc"); </del> <ins> tarball.add_bulk_dir(&builder.compiler_doc_out(host), "share/doc/rust/html/rustc"); </ins> Some(tarball.generate()) } }

temp_dir: PathBuf, image_dir: PathBuf, overlay_dir: PathBuf, <ins> bulk_dirs: Vec<PathBuf>, </ins> include_target_in_component_name: bool, is_preview: bool,

temp_dir, image_dir, overlay_dir, <ins> bulk_dirs: Vec::new(), </ins> include_target_in_component_name: false, is_preview: false,

self.builder.cp_r(src.as_ref(), &dest); } <ins> pub(crate) fn add_bulk_dir(&mut self, src: impl AsRef<Path>, dest: impl AsRef<Path>) { self.bulk_dirs.push(dest.as_ref().to_path_buf()); self.add_dir(src, dest); } </ins> pub(crate) fn generate(self) -> GeneratedTarball { let mut component_name = self.component.clone(); if self.is_preview {

.arg("--image-dir") .arg(&this.image_dir) .arg(format!("--component-name={}", &component_name)); <ins> if let Some((dir, dirs)) = this.bulk_dirs.split_first() { let mut arg = dir.as_os_str().to_os_string(); for dir in dirs { arg.push(","); arg.push(dir); } cmd.arg("--bulk-dirs").arg(&arg); } </ins> this.non_bare_args(cmd); }) }