Change:
<ins> #![feature(alloc_error_hook, allocator_api)] use std::{ alloc::{set_alloc_error_hook, AllocError, Allocator, Layout, System}, collections::VecDeque, panic::{catch_unwind, AssertUnwindSafe}, ptr::NonNull, }; #[test] fn test_shrink_to_unwind() { // This tests that `shrink_to` leaves the deque in a consistent state when // the call to `RawVec::shrink_to_fit` unwinds. The code is adapted from #123369 // but changed to hopefully not have any UB even if the test fails. struct BadAlloc; unsafe impl Allocator for BadAlloc { fn allocate(&self, l: Layout) -> Result<NonNull<[u8]>, AllocError> { // We allocate zeroed here so that the whole buffer of the deque // is always initialized. That way, even if the deque is left in // an inconsistent state, no uninitialized memory should be accessed. System.allocate_zeroed(l) } unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: Layout) { unsafe { System.deallocate(ptr, layout) } } unsafe fn shrink( &self, _ptr: NonNull<u8>, _old_layout: Layout, _new_layout: Layout, ) -> Result<NonNull<[u8]>, AllocError> { Err(AllocError) } } set_alloc_error_hook(|_| panic!("alloc error")); let mut v = VecDeque::with_capacity_in(15, BadAlloc); v.push_back(1); v.push_front(2); // This should unwind because it calls `BadAlloc::shrink` and then `handle_alloc_error` which unwinds. assert!(catch_unwind(AssertUnwindSafe(|| v.shrink_to_fit())).is_err()); // This should only pass if the deque is left in a consistent state. assert_eq!(v, [2, 1]); } </ins>

Discussion:
<!-- Thank you for filing a bug report! Please provide a short summary of the bug, along with any information you feel relevant to replicating the bug. --I tried this code: I expected to see this happen: causes an alloc error, the thread unwinds, and are dropped in some order. Instead, this happened: (the exact value printed differs between runs, which makes sense if it's reading uninit memory) This occurs because of the way is implemented: It first moves the memory around to fit the new capacity, then calls with no regards to potential unwinding. Specifically, this would look something like this: initialization, the deque buffer looks like this: copies the head to just behind the tail: calls calls which returns an alloc error, the error handler unwinds, leaving the buffer looking like this: impl of drops its elements, reading the uninit memory. <!-- If you're using the stable version of the compiler, you should also check if the bug also exists in the beta or nightly versions. --While this specific setup requires nightly to manually set the alloc error handle hook and to trigger the alloc error in , this same bug could occur on stable if OOMs. The docs for also explicitly state that it may either abort or unwind. I'm not sure what would be the best way to fix this. Maybe just the call to and manually abort the process would work well enough?
label +T-libs +I-unsound
And we can't even run it in miri (cc
Yeah the Miri allocator never fails so we never even tried running the allocerrorhandler... filed to track that.
This is nightly-only since it requires an unstable flag .
Could there hypothetically be a scenario (on stable) where you're not linking against std, but you still have an unwinding panic handler? In that case I think this UB could still occur (because panics when not linked against std).
No, all of the machinery related to unwinding depends on unstable implementation details.
In that case, would it be possible to just back out of allowing to unwind? It seems that it always aborts on stable (which evidently programs rely on), and allowing it to unwind feels like just another unsafe footgun to me.

Looks like the docs were changed to allow unwinding in based on the arguments in that the custom alloc error hook is allowed to unwind and that 1.68 release notes stated that future versions of std may choose to call the panic handler on OOM. To me these points don't seem mutually exclusive to always aborting; even if either a custom handler or the default handler panics and unwinds, we could just catch those unwinds and abort unconditionally.
Unwinding on OOM was introduced in RFC 2116 (), and unsafe code not handling this correct was a known issue. I think we should keep this feature as unstable until we have at least audited the entire standard library for incorrect handling of unwinds on OOM.
Running this in Miri works now, and does report "Undefined Behavior: using uninitialized data, but this operation requires initialized memory".