Change:
Apologies for not having a minimal reproduction, but this was an extremely difficult bug to even isolate occurring inside some complicated AVX2 code. The bug is causing the wrong values to be computed. Whether or not it occurs depends on the following conditions: With unset, the bug does NOT occur in debug builds, but DOES occur with With , the bug does NOT occur in builds and both debug and release builds are OK I can attempt to further isolate and reduce the problem, but there's a lot of spooky-action-at-a-distance happening making that rather difficult. EDIT: I've deleted the branch as there is now a , but so long as GitHub hasn't GC'd it here's the original commit: NOTE: if you from here, I've included lots of notes in the latest commit about the bug in the commit message. The commit also contains comments indicating lines you can comment or uncomment to make the tests succeed or fail. - NOTE: as this is a bug in the AVX2 backend, you'll need to run it on an AVX2-capable host to trigger the bug. This test fails with a miscomputed result (as do all of the tests across the board if you run the whole suite): The aforementioned will pass if any of the following things which are documented in the commit () message and comments introduced in that commit are changed: A commented out statement near the first observation of the miscompilation is uncommented (heisenbug!) The attribute on the function is commented out. This function is in a completely different module, hence my descriptions of "spooky action at a distance" (the function in which the bug is occurring is annotated , but the bug still occurs if that attribute is commented out) This bug is easily reproducible and occurs on all versions of the Rust compiler and all operating systems I've tried. I've reproduced it locally on macOS and it also occurred on Linux/Ubuntu via GitHub Actions. Here are some compiler versions I've tried: Latest nightly as of opening this ticket: It also broke in CI which tests it under the MSRV of 1.41.0.
If this is helpful, this is the code closest to where the problem is occurring:
ping llvm
Hey LLVM ICE-breakers! This bug has been identified as a good "LLVM ICE-breaking candidate". In case it's useful, here are some [instructions] for tackling these sorts of bugs. Maybe take a look? Thanks! <3 [instructions]: https://rustc-dev- cc
I found a workaround for this issue which suffices for my purposes, but also hopefully helps track down the bug. The miscompilation was occurring inside of a lambda. I extracted the lambda(s) into named functions and that worked around the problem successfully:
This is unsound and it impacts cryptographic code, so we decided on . Assigning and removing as in the prioritization working group.
meta: I thought was more about language issues -- are all LLVM codegen bugs going to be flagged this way?
I think is generally used for miscompilations as well.
I've removed , , , , , and , it's not exactly minimal (900 lines) but here you go I may have inadvertently removed a couple target features specific-code or attributes, or too much of the cpu autodetection, as it now doesn't build with RUSTFLAGS="-C target-feature=+avx2" ? The error is still the same as the OP (and reproduces on the playground) so maybe it is useful to I haven't looked at the issue much though, and am not sure what to expect here (especially if code using target features is built without them enabled).
ping cleanup Would be nice to get an MCVE for this one.
Hey Cleanup Crew ICE-breakers! This bug has been identified as a good "Cleanup ICE-breaking candidate". In case it's useful, here are some [instructions] for tackling these sorts of bugs. Maybe take a look? Thanks! <3 [instructions]: https://rustc-dev- cc
Edit: down to 55 lines
This is the smallest version I can get
Looks like this still reproduces on nightly, so not fixed by LLVM 12.
I believe this is an ABI mismatch problem. Argument promotion converts the by-pointer arguments into by-value arguments, so we pass by value across and avx2 ABI boundary. The caller passes two yym registers, the callee expects four xmm registers. cc
This sounds very similar to why we are passing the SIMD arguments by-pointer in the first place – in attempt to avoid this exact kind of ABI mismatch.
It looks like ArgPromotion does check for ABI compatibility: Either it's not doing that right, or there's some more complex interaction here.
produces: And then will generate these two call-sites: Which clearly don't follow the same ABI. It's not really obvious to me how this should be resolved though. This breaks the current model that inlining is safe if the caller has a superset of the callee features. Requiring the same features would be rather restrictive. One could argue that lowering should be responsible for doing ABI adjustments in calls, but I'm not sure how feasible that is from a legalization perspective.
Well, inlining a function if the caller has a superset of features is still not wrong. Even without any passes something like is already encountering ABI mismatches – the argument is passed in via , but the callee expects to find them in and . I don't think there's a right place to fix these kinds of ABI mismatches anywhere in LLVM, not without a major refactor somewhere. Even if you adjust the codegen for instruction to take callee's features into account, it would still have potential to go wrong with, say, function declarations. In context of IR generated by Rust, I would say, promothing these arguments to be by val, rather than indiract (Rust produces IR to pass these by a pointer, right?) is wrong. Rust operates under presumption that LLVM won't make these potentially ABI-breaking argument promotions. In that context the entire arg promotion pass is invalid on x86 for vector types.
I don't think we can really blame argument promotion for this. What argument promotion does is perfectly legal in isolation, as it is only promoting arguments of calls with compatible ABI -- these calls just end up in an incompatible context as a result of inlining. Assuming that it is technically infeasible to adjust call codegen, I think the right place to address this would be in inlining, by scanning calls inside the function, rather than just checking the target features on the function itself.
Ah, I see what you're saying. Yeah, I think that would resolve the bug in the strictest sense of the word, but I imagine it would also make quite a few people unhappy about sub-optimal codegen due to missed inlining opportunities (see e.g. which, I think, would occur more broadly were promotion get to work its magic before inlining gets through this sort of code)
Interestingly, the behavior on nightly seems to be different. On stable and beta, with prints but on nightly it prints So it seems that LLVM 12 may have changed something, but didn't fix the bug.
Currently on both stable and nightly: At last, consistent behavior between versions.
This issue shares some of the reason for its problems with
This is fixed upstream (cf ) and will be pulled in with the LLVM 14 update.
Fantastic! Thank you!
This is fixed by the LLVM 14 upgrade on beta/nightly.
I think we still want a regression test for this issue, right?

Discussion:
<ins> // run-pass // only-x86_64 // compile-flags: -C opt-level=3 // Regression test for issue #79865. // The assertion will fail when compiled with Rust 1.56..=1.59 // due to a LLVM miscompilation. use std::arch::x86_64::*; fn main() { if is_x86_feature_detected!("avx") { let res: [f64; 4] = unsafe { std::mem::transmute::<_, _>(first()) }; assert_eq!(res, [22.0, 44.0, 66.0, 88.0]); } } #[target_feature(enable = "avx")] unsafe fn first() -> __m256d { second() } unsafe fn second() -> __m256d { let v0 = _mm256_setr_pd(1.0, 2.0, 3.0, 4.0); let v1 = _mm256_setr_pd(10.0, 20.0, 30.0, 40.0); // needs to be called twice to hit the miscompilation let (add, _) = add_sub(v0, v1); let (add, _) = add_sub(add, add); add } #[inline(never)] // needed to hit the miscompilation unsafe fn add_sub(v1: __m256d, v0: __m256d) -> (__m256d, __m256d) { let add = _mm256_add_pd(v0, v1); let sub = _mm256_sub_pd(v0, v1); (add, sub) } </ins>