Change:
#[inline] pub unsafe fn lock(&self) { <ins> #[cold] #[inline(never)] fn fail(r: i32) -> ! { let error = Error::from_raw_os_error(r); panic!("failed to lock mutex: {error}"); } </ins> let r = libc::pthread_mutex_lock(raw(self)); <del> debug_assert_eq!(r, 0); </del> <ins> // As we set the mutex type to `PTHREAD_MUTEX_NORMAL` above, we expect // the lock call to never fail. Unfortunately however, some platforms // (Solaris) do not conform to the standard, and instead always provide // deadlock detection. How kind of them! Unfortunately that means that // we need to check the error code here. To save us from UB on other // less well-behaved platforms in the future, we do it even on "good" // platforms like macOS. See #120147 for more context. if r != 0 { fail(r) } </ins> } #[inline]

Discussion:
Hi! We've run into a subtle Mutex bug on a POSIX platform, and it's led me to propose a small change to the -based PAL code. Shout-out to for noticing this and coming up with a reduction test case. The current lock routine, for instance, for the generic non-Futex Unix PAL is here: Since we're in std, that will be compiled out in almost all cases. This presents a soundness problem on non-Futex POSIX platforms where a non-error-checked call may fail. Since the PAL is specifying a (a non-error-checked mutex), you might expect to be unable to fail. However, POSIX platforms in the wild have interpreted 's semantics more loosely, and provide deadlock detection on for non-checked mutexes. This specifically affects versions of Solaris dating back to before the open source release (the point at which my ability to follow history ends), and various Solaris-descended operating systems, including Illumos (where this was noticed). This means that on Solaris, Illumos, and kin, the current Rust Mutex implementation is unsound, because it can be used to generate aliasing s: To be clear -- my interpretation of the POSIX standard says that these platforms are violating it. We have filed a bug reflecting this against Illumos, so that with any luck some future versions will behave differently. But there are a number of systems in the wild that may never see that fix, and it'd be good (IMO) to have Mutex be correct there too. I look forward to the world where I only have to write code for correct operating systems, but we're not there yet. :-) Because the correctness of the POSIX lock operation is critical for soundness in Rust, I propose that the PAL check the return code from , under the banner of "wouldn't you want to know if this assumption were violated?" Concretely, I'd like to switch that check to a full-fledged rather than a . I'm writing this bug to open a discussion and see what folks would want to see before this happens, or if anyone has a good alternative. For instance, would folks feel strongly about this change having a platform-specific to enable it, assuming that only a named subset of POSIX platforms will ever use this interpretation of the standard? (I'd be more comfortable opting specific POSIX platforms out of the check, personally, or making it unconditional -- but let's talk!)

As far as I can tell, the posix PAL is used primarily, in terms of numbers, by Darwin. Linux and the common BSDs do not rely on it. Darwin's mutex does not behave the same way as the Solaris-descended ones, so the soundness issue does not affect Darwin. As a result I could imagine opting Darwin out of the test, if folks are concerned about a performance hit. I'm not sure if the current platform support has any kind of shared set of "POSIX interpretation quirks" that generates features I could use instead of targetos; if that's a thing, point me at it. Thanks in advance!
If it causes unsoundness then I think the best thing to do is to have an allow list of known good systems and assume the worst for everything else. The list of "bad" targets is potentially unbounded but we know which targets we've checked and are ok.
That'd be my preference as well,