Change:
return; } <del> /* INVARIANT: vec.len() > read >= write > write-1 >= 0 */ </del> <ins> // Check if we ever want to remove anything. // This allows to use copy_non_overlapping in next cycle. // And avoids any memory writes if we don't need to remove anything. let mut first_duplicate_idx: usize = 1; let start = self.as_mut_ptr(); while first_duplicate_idx != len { let found_duplicate = unsafe { // SAFETY: first_duplicate always in range [1..len) // Note that we start iteration from 1 so we never overflow. let prev = start.add(first_duplicate_idx.wrapping_sub(1)); let current = start.add(first_duplicate_idx); // We explicitly say in docs that references are reversed. same_bucket(&mut *current, &mut *prev) }; if found_duplicate { break; } first_duplicate_idx += 1; } // Don't need to remove anything. // We cannot get bigger than len. if first_duplicate_idx == len { return; } /* INVARIANT: vec.len() > read > write > write-1 >= 0 */ </ins> struct FillGapOnDrop<'a, T, A: core::alloc::Allocator> { /* Offset of the element we want to check if it is duplicate */ read: usize,

Discussion:
It seems like the implementation makes two passes over the data, making many more copies than are strictly necessary. Doing this allows the implementation to panic without causing a leak. I wouldn't call this bad, but it was a surprise to see that it was doing the extra work. Mimicking the way handles this and temporarily shortening the vector allows us to cut down on the copies (and also conveniently removes a panicking branch). I wonder if it makes sense to document the trade-off that makes here? Imo, it'd make it easier to use the library correctly in the way that
I wonder if it would be even faster to split it into two loops since once it will always be hitting that branch in the loop to do ? Maybe it could help reduce branch misprediction? I feel like this could be applied to the same thing right now. I wonder why are we not using but instead use ? And do we really need to , from what I see it does not do that right now? cc who may be interested in this.
The leaking behavior is unnecessary. A drop guard handle the tail.
is being called in in the current implementation. If you''re asking why I'm using , it's because it's faster than , and we don't need to use in this implementation. I'm pretty sure you're right My mistake, I wound myself up trying to justify the call. I think that means this would be a direct upgrade to the current implementation? (Note: My implementation seems to be broken in a couple ways, it's just for reference)
I tried to flesh out the implementation a little, and make sure it's working properly. The behaviour of the still needs to be tested, and it'd be good to do a more thorough benchmark if this was being considered for addition. I also implemented the nested loop version. I can't tell if it helped with performance yet, since my computer gives the benchmarks too much noise.
Do you think separating the branch into a different loop is a good idea in case they are not in the same bucket?
I've used a second (and third) loop in the implementation to avoid the branch, did I misunderstand your recommendation?
It still does seemed to be in the loop , I think making that in the inner loop is the same as the one currently since that branch will always be there.

I didn't benchmark and I am not sure if it will be faster, just a guess avoiding that branch may end up reducing branch mispredictions.
:thinking: I honestly don't know how I'd implement that - it seems like we need to branch somewhere to verify the the copies are legal. I'd welcome a PR showing what you mean.
I mean we are branching it in another loop since once the dedup and is different, it will always copy stuff when it hit the other branch, so instead of having 2 branches when the bucket is different, we could keep it to the other loop to remove the branch. Not sure if it's faster though.
proposed additional optimizations for this issue. Details here: