Change:
I tried this code: I expected to see this happen: The drop implementation for and is called. Instead, this happened: Everything that was mapped and the allocation was leaked. <!-- If you're using the stable version of the compiler, you should also check if the bug also exists in the beta or nightly versions. --: I discovered this issue when I learned that re-uses a allocation if possible. Which prompted me to read the code. Running the example with also reports the memory leak. And changing to no longer displays the memory leak. This is what I think happens: While dropping the call in panics. Then is being dropped, but honestly I don't really understand why. My understanding is that should unwind. If anyone can explain this, please do so. Then as part of unwinding is dropped, but it has its fields set to an empty buffer of len 0, so it does nothing. Note, I suspect more optimal code gen could be achieved by using to avoid generating code for the drop that will never do anything after we cleared the fields. The two created will never get dropped, thus leading to the memory leak. Plus the main allocation, previously owned by as stored in will be leaked too. Because it was cleared in and assumed to be taken over by but we never reached that point. Proposed solution, a drop guard in that guards the call and would drop the head in this case and the backing allocation if panics.
Tagging because of your work on and the associated PRs.
Nice find! Technically this is a regression. label regression-from-stable-to-stable, T-libs. It works properly until 1.47, it double-frees and then aborts (thread panicked while panicking; presumably because of double-freeing , too) in 1.48 through 1.51, and the current behavior is since 1.52 (all tested on godbolt, with the direct usage of variables in format strings fixed).
I haven’t looked at the code yet myself to figure out what’s going on. In the meantime, I think that bisecting what changes were relevant for the regression 1.47 to 1.48 on one hand, and the point between 1.51 and 1.52 that appeared to have fixed the double-free (but introduced or kept (hard to tell with the abort) the memory leak) could be interesting. label E-needs-bisection. ^^
Doing the bisection. First one is second one is label -E-needs-bisection
So that’s likely and . Let me tag who might be more familiar with the code at hand since they were involved with both PRs :-)
I’m not sure if I’m reading the PR description of correctly as in that this leak might be intentional? (If it is intentional, I haven’t understood the rationale yet though…) I mean… who knows, maybe this is talking about something entirely different than what this issue is about ^^
Looking at the code, this leak seems relatively easy to avoid. Assuming that dropping the elements while unwinding will itself not panic. I'm guessing the comment you reference relates to something else. Also any idea why and how gets dropped?
The relevant piece of code seems to be this one: Where is defined here: So what happens is: all the elements to collect have been written however there could be more element in the original / moreover we want to give ownership of the allocation to the new so we remove any reference to the original allocation from the and we drop the remaining elements finally, we create the that logically owns both the and the allocation The problem is that a panic can occur while dropping the remaining elements, and in that case we didn't yet create the final , so nothing is responsible for dropping the allocation (which was already removed from the New`s. The simpliest fix would be to create the before dropping the remaining elements, however I'm not sure if that's correct with respect to aliasing/stacked borrows. For example if is ever implemented using internally then it would claim unique access to the allocation, which invalidates the pointer used to drop the remaining elements. I think the most robust solution would be a drop guard that frees the allocation and the s on drop (and is forgotten before creating the actual that's returned). Dropping the old elements is done with which will try to drop all the elements even if one panics while being dropped. If another element's destructor panics then that's a panic while panicking and the program aborts.
keeps going and then rethrows the panic. That's because it replicates the . Yeah, to fix the double-drop in , this was the simplest solution that didn't require any extra parts. And if you're panicing in drop then leaks are allowed behavior. Avoiding them is more a QoI thing.
A bigger concern imo is... well... panic-safety. If the and the exist at the same time then you need to be really sure that panics can only occur after the IntoIter has given up the allocation. If a panic occurs between materializing the and before calling then you get double-frees again.
The idea was to create it right before calling
That could work. At least I'm not seeing anything that would go wrong with that given current implementations. The question is whether it's worth it adding yet another unsafe wrinkle to this goldberg machine.
~I think the issue will be type confusion. Which is what I initially looked for. If dropping those remaining Old elements panics, it would drop Olds as News. I think a simple drop guard before forget_ should do the trick.~ Never mind I got confused.
I think the issue will be type confusion. Which is what I initially looked for. If dropping those remaining Old elements panics, it would drop Olds as News. I think a simpel drop guard before forget_ should do the trick.

Discussion:
//! This is handled by the [`InPlaceDrop`] guard for sink items (`U`) and by //! [`vec::IntoIter::forget_allocation_drop_remaining()`] for remaining source items (`T`). //! <ins> //! If dropping any remaining source item (`T`) panics then [`InPlaceDstBufDrop`] will handle dropping //! the already collected sink items (`U`) and freeing the allocation. //! </ins> //! [`vec::IntoIter::forget_allocation_drop_remaining()`]: super::IntoIter::forget_allocation_drop_remaining() //! //! # O(1) collect

use core::mem::{self, ManuallyDrop}; use core::ptr::{self}; <del> use super::{InPlaceDrop, SpecFromIter, SpecFromIterNested, Vec}; </del> <ins> use super::{InPlaceDrop, InPlaceDstBufDrop, SpecFromIter, SpecFromIterNested, Vec}; </ins> /// Specialization marker for collecting an iterator pipeline into a Vec while reusing the /// source allocation, i.e. executing the pipeline in place.

); } <del> // Drop any remaining values at the tail of the source but prevent drop of the allocation // itself once IntoIter goes out of scope. // If the drop panics then we also leak any elements collected into dst_buf. </del> <ins> // The ownership of the allocation and the new `T` values is temporarily moved into `dst_guard`. // This is safe because `forget_allocation_drop_remaining` immediately forgets the allocation // before any panic can occur in order to avoid any double free, and then proceeds to drop // any remaining values at the tail of the source. </ins> // // Note: This access to the source wouldn't be allowed by the TrustedRandomIteratorNoCoerce // contract (used by SpecInPlaceCollect below). But see the "O(1) collect" section in the // module documenttation why this is ok anyway. <ins> let dst_guard = InPlaceDstBufDrop { ptr: dst_buf, len, cap }; </ins> src.forget_allocation_drop_remaining(); <ins> mem::forget(dst_guard); </ins> let vec = unsafe { Vec::from_raw_parts(dst_buf, len, cap) };

} } } <ins> // A helper struct for in-place collection that drops the destination allocation and elements, // to avoid leaking them if some other destructor panics. pub(super) struct InPlaceDstBufDrop<T> { pub(super) ptr: *mut T, pub(super) len: usize, pub(super) cap: usize, } impl<T> Drop for InPlaceDstBufDrop<T> { #[inline] fn drop(&mut self) { unsafe { super::Vec::from_raw_parts(self.ptr, self.len, self.cap) }; } } </ins>

} /// Drops remaining elements and relinquishes the backing allocation. <ins> /// This method guarantees it won't panic before relinquishing /// the backing allocation. </ins> /// /// This is roughly equivalent to the following, but more efficient /// /// ``` /// # let mut into_iter = Vec::<u8>::with_capacity(10).into_iter(); <ins> /// let mut into_iter = std::mem::replace(&mut into_iter, Vec::new().into_iter()); </ins> /// (&mut into_iter).for_each(core::mem::drop); <del> /// unsafe { core::ptr::write(&mut into_iter, Vec::new().into_iter()); } </del> <ins> /// std::mem::forget(into_iter); </ins> /// ``` /// /// This method is used by in-place iteration, refer to the vec::in_place_collect

self.ptr = self.buf.as_ptr(); self.end = self.buf.as_ptr(); <ins> // Dropping the remaining elements can panic, so this needs to be // done only after updating the other fields. </ins> unsafe { ptr::drop_in_place(remaining); }

mod set_len_on_drop; #[cfg(not(no_global_oom_handling))] <del> use self::in_place_drop::InPlaceDrop; </del> <ins> use self::in_place_drop::{InPlaceDrop, InPlaceDstBufDrop}; </ins> #[cfg(not(no_global_oom_handling))] mod in_place_drop;

} #[test] <del> fn test_from_iter_specialization_panic_during_drop_leaks() { static mut DROP_COUNTER: usize = 0; </del> <ins> fn test_from_iter_specialization_panic_during_drop_doesnt_leak() { static mut DROP_COUNTER_OLD: [usize; 5] = [0; 5]; static mut DROP_COUNTER_NEW: [usize; 2] = [0; 2]; </ins> #[derive(Debug)] <del> enum Droppable { DroppedTwice(Box<i32>), PanicOnDrop, } </del> <ins> struct Old(usize); </ins> <del> impl Drop for Droppable { </del> <ins> impl Drop for Old { </ins> fn drop(&mut self) { <del> match self { Droppable::DroppedTwice(_) => { unsafe { DROP_COUNTER += 1; } println!("Dropping!") } Droppable::PanicOnDrop => { if !std::thread::panicking() { panic!(); } } </del> <ins> unsafe { DROP_COUNTER_OLD[self.0] += 1; } if self.0 == 3 { panic!(); </ins> } <ins> println!("Dropped Old: {}", self.0); </ins> } } <del> let mut to_free: *mut Droppable = core::ptr::null_mut(); let mut cap = 0; </del> <ins> #[derive(Debug)] struct New(usize); impl Drop for New { fn drop(&mut self) { unsafe { DROP_COUNTER_NEW[self.0] += 1; } println!("Dropped New: {}", self.0); } } </ins> let _ = std::panic::catch_unwind(AssertUnwindSafe(|| { <del> let mut v = vec![Droppable::DroppedTwice(Box::new(123)), Droppable::PanicOnDrop]; to_free = v.as_mut_ptr(); cap = v.capacity(); let _ = v.into_iter().take(0).collect::<Vec<_>>(); </del> <ins> let v = vec![Old(0), Old(1), Old(2), Old(3), Old(4)]; let _ = v.into_iter().map(|x| New(x.0)).take(2).collect::<Vec<_>>(); </ins> })); <del> assert_eq!(unsafe { DROP_COUNTER }, 1); // clean up the leak to keep miri happy unsafe { drop(Vec::from_raw_parts(to_free, 0, cap)); } </del> <ins> assert_eq!(unsafe { DROP_COUNTER_OLD[0] }, 1); assert_eq!(unsafe { DROP_COUNTER_OLD[1] }, 1); assert_eq!(unsafe { DROP_COUNTER_OLD[2] }, 1); assert_eq!(unsafe { DROP_COUNTER_OLD[3] }, 1); assert_eq!(unsafe { DROP_COUNTER_OLD[4] }, 1); assert_eq!(unsafe { DROP_COUNTER_NEW[0] }, 1); assert_eq!(unsafe { DROP_COUNTER_NEW[1] }, 1); </ins> } // regression test for issue #85322. Peekable previously implemented InPlaceIterable,