Change:
Since , has been distributed in the component in two locations: (presumably as a runtime dependency of ) (not sure, maybe for linking custom drivers?) likely belongs in the component, if the duplication can't be avoided <hr/The difference can be seen in these consecutive nightlies' components: has just the second maybe it's used via relative by ? separate codegen backends are gone, maybe we should just try removing it? first nightly to include has both copies All 3 of those files are identical (even across nightlies, yay for deterministic builds): One fascinating aspect is that the size only increases from to , despite alone being uncompressed, so compression is really helping. Even the only goes from to . (I'm using archive sizes from the and directory listings in , FWIW) <hr/This part of the build system seems relevant, but I'm not sure which of the two this creates: cc
cc -- you've touched code related to this a bunch recently I think, so maybe would have some time to investigate fixing this. I suspect we want to only ship one LLVM per tarball, at least, though ideally we would only have just the one LLVM library total as well.
Half-way through writing up this issue I realized the connection. I'd suggest we try to get rid of the copy and see what breaks, it's possible it's been dead weight for a while now. And only if we have to keep both for some odd reason, we'd move it to .
The purposely orders the tarred files so that similar filenames are adjacent in the compression stream, especially in hope of helping with such duplicate files. I think we may still want the copy for the odd cross-compiled tool using , but that definitely belongs in .
Is that... even possible? Is a per-target component in the same way is?
Yes it is -- try -- but I don't know if anyone really needs that...
Actually, can you try something like cross-compiling a copy of to , after deleting the per-target ? I suspect that it would actually run just fine on a host, because 's s should only really contain the information and maybe some object code that is missing (which we could get rid of if we kept MIR for everything but idk how feasible that is). That is, the result of linking against should still use at runtime, so in is unnecessary. But maybe I'm missing something. Like running the cross-compiled on a host without anything more than the component?
Hmm, I can't actually get cross-builds to work with . I this to : ... so it would pick that up from the sysroot. Trying to cross-compile i686, I get: I do have both i686 and x8664 builds of in the sysroot. I suspect the error is because the i686 build has different metadata than my host build, so it can't find the one it wants for the host. That may mean that is really tied to being used on the same host. It works if I just compile native x86_64 though. Then I tried removing the , and it gets essentially the same error as : AFAICS, building normal crates still works fine with that extra removed though. In short: Cross-compiling with seems broken -- maybe it should be shipped native-only. The only difference would be in the manifest created for rustup. Linking to crates will inherit , so that needs to be in the library path.
Ah, fascinating. What happens if you build such an executable and then remove the component entirely? Can it pick up the dylibs outside or does it hardcode the paths inside?
The final executable still isn't actually linked to LLVM, since it didn't need any of those symbols: I can run this with , which set , or even if I set that path myself. But doesn't work if we've removed that library, since needs itself.
Okay, it being dependent on makes sense. Overall, probably the safe thing to do would be to move the duplicate to , oh well.
Oh, also needs LLVM, and those binaries are installed in with RUNPATH . I guess they might be able to find the LLVM in anyway?

Discussion:
// Ensure that `libLLVM.so` ends up in the newly build compiler directory, // so that it can be found when the newly built `rustc` is run. <del> dist::maybe_install_llvm_dylib(builder, target_compiler.host, &sysroot); </del> <ins> dist::maybe_install_llvm_runtime(builder, target_compiler.host, &sysroot); dist::maybe_install_llvm_target(builder, target_compiler.host, &sysroot); </ins> // Link the compiler binary itself into place let out_dir = builder.cargo_out(build_compiler, Mode::Rustc, host);

// components like the llvm tools and LLD. LLD is included below and // tools/LLDB come later, so let's just throw it in the rustc // component for now. <del> maybe_install_llvm_dylib(builder, host, image); </del> <ins> maybe_install_llvm_runtime(builder, host, image); </ins> // Copy over lld if it's there if builder.config.lld_enabled {

} } <del> // Maybe add libLLVM.so to the lib-dir. It will only have been built if // LLVM tools are linked dynamically. // // We add this to both the libdir of the rustc binary itself (for it to load at // runtime) and also to the target directory so it can find it at link-time. // // Note: This function does no yet support Windows but we also don't support //       linking LLVM tools dynamically on Windows yet. pub fn maybe_install_llvm_dylib(builder: &Builder<'_>, target: Interned<String>, sysroot: &Path) { </del> <ins> /// Maybe add libLLVM.so to the given destination lib-dir. It will only have /// been built if LLVM tools are linked dynamically. /// /// Note: This function does not yet support Windows, but we also don't support ///       linking LLVM tools dynamically on Windows yet. fn maybe_install_llvm(builder: &Builder<'_>, target: Interned<String>, dst_libdir: &Path) { </ins> let src_libdir = builder.llvm_out(target).join("lib"); <del> let dst_libdir1 = sysroot.join("lib/rustlib").join(&*target).join("lib"); let dst_libdir2 = sysroot.join(builder.sysroot_libdir_relative(Compiler { stage: 1, host: target })); t!(fs::create_dir_all(&dst_libdir1)); t!(fs::create_dir_all(&dst_libdir2)); </del> if target.contains("apple-darwin") { let llvm_dylib_path = src_libdir.join("libLLVM.dylib"); if llvm_dylib_path.exists() { <del> builder.install(&llvm_dylib_path, &dst_libdir1, 0o644); builder.install(&llvm_dylib_path, &dst_libdir2, 0o644); </del> <ins> builder.install(&llvm_dylib_path, dst_libdir, 0o644); </ins> } return; }

panic!("dist: Error calling canonicalize path `{}`: {}", llvm_dylib_path.display(), e); }); <del> builder.install(&llvm_dylib_path, &dst_libdir1, 0o644); builder.install(&llvm_dylib_path, &dst_libdir2, 0o644); </del> <ins> builder.install(&llvm_dylib_path, dst_libdir, 0o644); </ins> } } <ins> /// Maybe add libLLVM.so to the target lib-dir for linking. pub fn maybe_install_llvm_target(builder: &Builder<'_>, target: Interned<String>, sysroot: &Path) { let dst_libdir = sysroot.join("lib/rustlib").join(&*target).join("lib"); maybe_install_llvm(builder, target, &dst_libdir); } /// Maybe add libLLVM.so to the runtime lib-dir for rustc itself. pub fn maybe_install_llvm_runtime(builder: &Builder<'_>, target: Interned<String>, sysroot: &Path) { let dst_libdir = sysroot.join(builder.sysroot_libdir_relative(Compiler { stage: 1, host: target })); maybe_install_llvm(builder, target, &dst_libdir); } </ins> #[derive(Clone, Debug, Eq, Hash, PartialEq)] pub struct LlvmTools { pub target: Interned<String>,

builder.install(&exe, &dst_bindir, 0o755); } <ins> // Copy libLLVM.so to the target lib dir as well, so the RPATH like // `$ORIGIN/../lib` can find it. It may also be used as a dependency // of `rustc-dev` to support the inherited `-lLLVM` when using the // compiler libraries. maybe_install_llvm_target(builder, target, &image); </ins> // Prepare the overlay let overlay = tmp.join("llvm-tools-overlay"); drop(fs::remove_dir_all(&overlay));