Code change:
Trying to build Miri against latest rustc artifacts fails: Regression range: Likely cause: Cc
How does miri link against rust?
Just a bunch of :
Given that rustc CI still works and only builds against the distributed rustc toolchain are affected, it seems likely that the issue is related to the rustc-dev component -- maybe that doesn't contain all the required files any more?
Reproducing instructions: checkout out to install the right toolchain (that branch uses ) -
The necessary symlink is indeed not part of rustc-dev (only rust-dev). We could add it, but it's not obvious to me why it is needed. I'm not familiar with how rustc generates that linker invocation, but the fact that it explicitly includes in it seems wrong to me ( is a dependency of and just linking against should be sufficient).
There is no component? At least doesn't list one and comes back empty. I don't know anything about this linking magic either.^^ Cc
But evidently the symlink is needed... so in the mean time it'd be good to unstuck this by adding the symlink I think. I expect this will break out-of-tree builds of Clippy as well, and rustfmt -- basically every tool that links against rustc.
Ah, it's not a real component, just the tarball that download-ci-llvm uses. Main problem is that we have checks against shipping symlinks, and I expect they exist for good reason...
Ah, yeah probably Windows doesn't like them. Why can't rustc_driver link against the correct .so file, i.e., why is a symlink needed in the first place?
I think this is the reason why it gets to the linker line: This is because llvm-config returns something like and the symlink is needed to resolve that to . And it seems like rustc just embeds the original name in the dylib, rather than the symlink-resolved variant, so the symlink is still needed. Possibly that's a bug? Similar to how shared objects embed the resolved name, maybe rustc should do the same for its own metadata. Though independent of general rustc behavior, I guess we could explicitly resolve the symlink in the for rustcllvm. That might be the most straightforward fix.
If llvm-config inherently relies on a symlink, how do they support Windows?
That would break compiling an rlib which states that it links against a certain dylib when said dylib is not available on the host, right? Currently you only need dylibs to be available when actually linking (that is using a crate type other than rlib or staticlib). It includes , not . Even if just was passed, has a for and thus needs to be present in . Edit: Forgot it is supposed to be in llvm-tools-preview, not in rustc-dev. By the way has never been a symlink afaik. Instead a separate copy was shipped in both the rustc and llvm-tools-preview components. The first for rustc itself to link against and the second for user code to link against. Just like we ship in both the rustc and rustc-dev components. Both copies just happened to be identical.
I just looked at the latest nightly and is present in both and as it should and has the correct filename for it. also has as .
I got confused about when got merged. I though it was already included in the latest nightly. I just looked at the llvm-preview component for that PR and it includes rather than the that is expected. Is it possible to rename it back and patch up the ? There is no way to link against with all linkers without a symlink, while for that is not a problem.
Why can't rustc_driver include ?
Because the linker will prefix the name with and postfix it with , so would cause the linker to look for rather than . (notice the different location of ) The only portable way to link against is to create a symlink from to , set the to and then tell the linker to link against . But on Windows we can't use symlinks. While on linux is possible, this is not the case on most other targets.
Windows doesn't have a concept of sonames, so I don't think a symlink would be used there. I don't know what the dylib is called there. We use static linking there anyway. So I think we wouldn't actually ship a symlink on anything but Linux. More problematic is the other part of this comment: If rustup-toolchain-install-master can't deal with symlinks, that's also a problem for Linux.
Any platform except Windows you mean?
As far as I can tell, dist-x86_64-linux is the only configuration where we link LLVM dynamically. (Well and the llvm-16/llvm-17 images, but that's non-dist and a different situation anyway.)
CI also starts with this exact same issue.
So to summarize, I see a few options here: Undo the library name change by carrying a patch in our LLVM fork. Implies carrying a long-term patch to LLVM, though a fairly small one. Undo the library name change after the fact. As pointed out, this isn't just a matter of moving the files, we also have to fix up the DTSONAME, which doesn't appear straightforward. Would require adding a dependency on patchelf? Ship the symlink in rustup components like rustc-dev, on Linux only. May require updates to tooling like rustup-toolchain-install-master. Change the way rustc links against upstream dylib dependencies, by resolving library names. Requires replicating linker search path logic and may break other other things. Not sure if this is possible. Same as 4, but only do this in rustllvms Again, not sure about the consequences, but at least this is more limited. I think 3 is probably ideal long term, but depends on just how broken symlink support is right now. Is this a matter of "we'll copy the file instead of symlinking" or "rustup-toolchain-install-master is going to abort"? Maybe knows. The easiest fix would be 1.
There's another fix, which is to ship a with content
Oooh, that's a great idea!
I got klint CI working with the fix:
Option 6, convince LLVM that this way of naming libraries is a bad idea and causes problems? I can't tell to what extent this is caused by Rust idiosyncrasies vs LLVM doing something strange. Wait, a .so file can be just a text file...?!?
Basically, LLVM used to do it's own thing here, and now follows the standard convention for shared object naming on Linux. This turns out to be quite inconvenient for us, but I don't think it would be reasonable to ask LLVM to undo the change. (Though, imho, this change really should not have been done in an rc3 release...) It can be a linker script. Somewhat unusual, but things like or are often linker scripts.
Well, if that linker script solution works that would be great. :)
This is now beginning to block Miri development. If someone reading along knows enough about linkers to approve that would be great. :)

Related discussion:
text: // If we have a symlink like libLLVM-18.so -> libLLVM.so.18.1, install the target of the // symlink, which is what will actually get loaded at runtime. builder.install(&t!(fs::canonicalize(source)), destination, 0o644); <ins> let full_dest = destination.join(source.file_name().unwrap()); </ins> if install_symlink { <del> // If requested, also install the symlink. This is used by download-ci-llvm. let full_dest = destination.join(source.file_name().unwrap()); </del> <ins> // For download-ci-llvm, also install the symlink, to match what LLVM does. Using a // symlink is fine here, as this is not a rustup component. </ins> builder.copy(&source, &full_dest); <ins> } else { // Otherwise, replace the symlink with an equivalent linker script. This is used when // projects like miri link against librustc_driver.so. We don't use a symlink, as // these are not allowed inside rustup components. let link = t!(fs::read_link(source)); t!(std::fs::write(full_dest, format!("INPUT({})n", link.display()))); </ins> } } else { builder.install(&source, destination, 0o644);