Change:
The following example (): produces the following LLVM-IR: and machine code: where 128 bytes are copied every time a value of type A is moved/copied/read/... However, one actually only has to copy a single byte, since all other bytes are trailing padding. The expected machine code is (): cc
The least gross and special-cased way I can think of achieving this would be extending (and related intrinsics) with some extra info about bytes which don't need to be copied, e.g. a bitset in metadata. It would discarded when lowering to a call but could hypothetically be used when expanding to a sequence of loads and stores inline. That's still a lot of implementation work for what appears to be a somewhat artificial case, though. A slightly more hacky, and more limited, fix for this specific case would be to trim the size copied by the amount of trailing padding (in this case 128 -1) if we can see statically that it's a typed copy of a single element ( call with constant 1 as length, MIR /). However, this might regress performance of e.g. because it would at first glance disallow copying it with a 32 bit load/store (similar issue exists with the proposed stride/size distinction).
Instead of extending memcpy, could the Rust front end generate single loads for each of the fields, and would LLVM be able to merge them as appropriate? That is, for the OP, I'd expect Rust to emit an 8-bit load. For something like I'd expect Rust to emit two loads (8-bit and 16-bit), and for (u8, u8, u8) I'd expect Rust to emit three 8-bit loads. For (u8, u16) and for (u8, u8, u8) I do however expect LLVM to merge the loads into a single 32-bit load (EDIT: if possible).
I don't think that can work. Leaving aside the huge inefficiencies of emitting so many instructions and praying that LLVM merges them, if we just emit loads and stores for the non-padding bytes and don't say anything about the padding, then LLVM has no indication that it's even allowed to clobber those parts.
LLVM already supports specifying padding through metadata: We currently don't use it because we generally aren't interested in TBAA. I also haven't checked whether the padding specified there is actually taking into account.
I am not at all convinced that this is a legal optimization, and I think making it legal makes the operational semantics much more annoying than it should be. says To me this looks like the description of an untyped, -like operation that just copies a bunch of bytes. I feel fairly strongly that behavior of should not depend on the type of the pointers. If you had used , that would be a totally different situation.
In particular, the docs also say that is equivalent to , which in turn : Given this spec, I see no way to justify not copying padding bytes. As in, I think we have to copy all bytes to comply with the spec, and the proposed optimization is illegal.
Not for copy_nonoverlapping possibly, but we should be able to elide padding copies for implicit memcpy's, as in the example (copy of return value into sret).
Ah, yes, I agree can be optimized as described in the OP. This can be explained, for example, by .
(A C call to does not need to call the function).
I did use in my original comment, e.g., see .
I never claimed it had to. Of course the compiler is allowed to inline . But I claim it has to copy all the bytes, including padding bytes. I don't understand how this comment responds to anything I said. Agreed, I had missed that. But if I read your OP correctly, you are saying the optimization should happen for (that's ). I vehemently disagree.
Yep, for as specified, that cannot be done. For is ok. TIL that works like . I expected it to work like in C++, which is also generic over T and does not copy padding when profitable. I think I've never needed with those semantics for a that is not , and when I needed it with memcpy semantics, i've casted the pointers to u8 manually. I am very surprised that there isn't a typed copy intrinsic that does the sane thing and avoids copying padding when profitable.
Likewise, I think we should add such a function (or feel out whether we can get away with redefining the existing functions that way).
I think we would need to survey how users are using it in the wild. The API does say that all bytes are copied and it does not require the memory to be "valid" at (e.g. the API docs do not suggest that s are read or written via the pointers), so users can use any that fits for the operation as long as and the size argument matches whatever memory they want to copy. I don't recall any use in libcore/liballoc/libstd where changing the semantics would break things. It can't imagine why would it make sense for some code to pick an arbitrary T with different padding, instead of the T that you actually want the code to copy.
Just to be sure we are on the same side here, the semantics of that would basically to do repeated ? So, this does not need any new mechanism in the Abstract Machine and can be explained with more primitive operations that we already have? If it doesn't change the Abstract Machine, I am fine with whatever. ;)
A typed copy would also imply that this operation is UB if the copied value(s) does not satisfy the validity invariant.
Yes. Yes, I think we can just give this define the operational semantics of such a function as equivalent to: However, that sounds like you want to make a primitive operation in the Abstract Machine. Is that so? I think the operational semantics of can be just defined as: and that should work independently of whether is a valid bit-pattern for . (note: the function might be special in other ways, e.g., the Rust run-time currently requires a symbol to actually exist, so this is a lang item, and the Rust frontend generates via the lang item, but this is just a valid optimization that follows from its semantics). Arguably, if the s are invalid having lying around is a foot-gun. Today, I would find it clearer if the code would do the copy via a instead. For older code that just wanted to perform a byte-wise copy, I just always used since I found that clearer as well.
will run drop glue, you need (in the first function). But with that change applied, I agree.
Turns out is actually an intrinsic (), so that would make sense. (I thought it would be implemented using . ;) Or (for the spec) we just say we use the (Pseudo-)MIR-level , which does not drop implicitly. I don't want to do anything, I was just trying to figure out what it is that you want to do. ;) Personally I'd feel best about just keeping our current semantics. Do you mean the current operational semantics? Because that sure looks like it copies the padding bytes. Now I am confused when you are talking about the old and when you are talking about the new (maybe ?). Using seems reasonable except it opens the door to all our discussions around padding in unions. :/
Yes and yes (the current semantics do copy the padding bytes).
I called the new copynonoverlapping just .
There though...
Does this imply that should be an intrinsic too? (Right now it is a copynonoverlapping, and thus generates that, not a load.)
Maybe. How we spec those (copying bytes vs. typed copy) and how they are implemented is orthogonal in principle, though if we actually want to exploit that they are typed we likely need intrinsics.
I think that, at least to fix this bug, the answer is yes. The current implementation does an untyped copy instead of a typed one. That's correct, but copies too much. One can implement an untyped copy on top of a typed one, but the opposite is not true.
Well, we first need a lang team decision that this is indeed a bug -- i.e., that these operations should act like typed copies.
To that end, could someone make a summary for why it should be considered a bug or better yet why a typed version needs to exist? (and also why not?) A pros & cons would be helpful. :)
I think we should separate the three questions at hand here. First of all, this function from the OP: unarguably performs typed copy. That it currently results in machine code that copies padding is an implementation detail of the current compiler not to be relied upon any more than e.g. in a statement. This should not be controversial let alone need a new T-lang decision. People working on the compiler just need to decide whether the demonstrated inefficency is worth prioritizing (as discussed above, fixing it is non-trivial and/or may regress other programs), but it's pretty clearly compatible with currently agreed-upon semantics of Rust to not copy the padding in this program. The second question (raised by the second code example in the OP) is whether and friends should perform typed copies, or gain new variants that perform typed copies. This is a trickier question which I currently don't care about enough to write the sort of summary asked for. A third question, which came up in the last couple comments, is whether and should commit to being typed copies or raw-byte-copies (I think it's currently unclear both ways, unlike clearly saying it copies bytes in its docs). I strongly think they should be typed copies and will try to write a summary of that situation soon, possibly in a different issue to avoid diluting this one further.
Agreed. So there should likely be some issue just tracking the lost optimization potential here. Maybe that should be this one, after removing the other example from the OP. Actually, thinking about this again -- both of them pass the input/output by value. So they already* do a typed copy for that. Doing two typed copies in a row is indistinguishable from doing a typed copy and a byte-wise copy, so actually we can do what we want (between these two options) for / and (conforming) code cannot tell the difference.
I think we all agree that changing the semantics of to do typed copies would be a breaking change, but beyond that, I think there is not much to summarize. Whether we can make this breaking change or whether that's a change worth making are unresolved questions. It is my personal opinion that typed copy semantics are a better default and they might enable some optimizations, but I wouldn't risk breaking the world for them when we can just add two new APIs without issues. While we could test how much code this change would break using , I don't think we can do the same test using crater, e.g., because it is ok to implement typed copies as untyped ones, so nothing might actually break during the crater run, but everything might break later when we update the LLVM version. The same applies to assessing the performance impact of this change, e.g., it might be none today. This makes it a high-risk change with unknown value. I also think that resolving this is fairly low priority, but I'd be ok with someone implementing these behind a feature gate, and with with people experimenting with using approach - we'll probably learn something useful from doing that.
There is definitely some analysis to be done here (lay out the implications for unsafe code using them & for codegen), if someone wants to push it further. Though FWIW since it's a library API change/expansion it seems more like a libs team thing than a lang team thing (though of course it falls in the subject matter of UCG, too). If by " approach" you mean TBAA metadata on memcpys, one unpleasant thing you'll learn is that it will miscompile Rust programs -- I am reasonably certain there's no way to express padding with the TBAA metadata without making type punning UB.
Yep; agreed. (The behavior of intrinsics is mostly a lang team thing since the whole point of them is about their "intrinsicness".)
What implications for unsafe code do you have in mind? One can already write a in plain Rust by using in a loop to perform typed copies, and I would be surprised if a lot of unsafe code doesn't do this already. The authors might have not rationalized why they chose typed vs untyped copies (they probably didn't knew existed). From a documentation point-of-view, we should just document that users should prefer typed copies over untyped ones, with the simple example, that they don't necessarily copy padding, and have a larger optimization potential. They should only use untyped copies when they need to copy all raw bytes, including padding, and we should warn them there again that padding bytes can change the moment a typed copy happens, e.g., when returning a value from a function, performing a typed assignment, etc. I think that experimenting with optimizations for typed copies is worth doing, and that we should try to find ways to make them faster, but this is an orthogonal issue to whether we add the semantic APIs for allowing users to easily perform typed copies. We could implement typed copies as untyped ones forever and that would be fine (miri would, however, implement typed copies appropriately and detect misuses here). Ouch.
I don't think that's the case. TBAA metadata is generic and you don't need to model C++ semantics in particular with it. Making everything alias everything is also possible. (Of course, it's still a much bigger gun than what we actually need here.)
The implications for unsafe code are exactly the implication of choosing typed copies vs copying bytes (which may be invalid for the type in question!) including padding, and the wider impact of recommending a different default. You're also missing my point re: codegen but I won't be tricked into writing the summary I already decided I don't want to take the time to write ;) Oops, I think you're right, I forgot that there can be multiple distinct "roots". (Though I am worried about the explosion in metadata this may cause, and how the AA infrastructure will weigh the MayAlias from TBAA with a Must/NoAlias from other AA implementations.)
True. Note however that right now, while Miri does check the validity invariant on typed copies, it . That will be non-trivial to implement, I think. That's clearly a deficiency in Miri, just one I thought you should be aware of. :)
FWIW the same question arises for . I would think the answer is the same as for the functions: we haven't properly documented this, and by now probably too many programmers rely on these being untyped operations, so that's what we should go for. Surprisingly, however, that is currently not the case -- is actually currently implemented as a typed copy! This changed with , Cc Is that really what we want?
Here's a little for checking that all these operations are untyped. Currently, it fails in Miri.
I have no strong feeling about typed vs untyped copies in swap_nonoverlapping. So long as both sides are doing the same kind of copy, that's fine. (It was confusing when L-R was typed but R-L was untyped, so let's not do that again.)
In I am proposing to document that these functions are all doing untyped copies.

Discussion:
/// `copy_nonoverlapping` is semantically equivalent to C's [`memcpy`], but /// with the argument order swapped. /// <ins> /// The copy is "untyped" in the sense that data may be uninitialized or otherwise violate the /// requirements of `T`. The initialization state is preserved exactly. /// </ins> /// [`memcpy`]: https://en.cppreference.com/w/c/string/byte/memcpy /// /// # Safety

/// order swapped. Copying takes place as if the bytes were copied from `src` /// to a temporary array and then copied from the array to `dst`. /// <ins> /// The copy is "untyped" in the sense that data may be uninitialized or otherwise violate the /// requirements of `T`. The initialization state is preserved exactly. /// </ins> /// [`memmove`]: https://en.cppreference.com/w/c/string/byte/memmove /// /// # Safety

/// Swaps the values at two mutable locations of the same type, without /// deinitializing either. /// <del> /// But for the following two exceptions, this function is semantically </del> <ins> /// But for the following exceptions, this function is semantically </ins> /// equivalent to [`mem::swap`]: /// /// * It operates on raw pointers instead of references. When references are

///   overlapping region of memory from `x` will be used. This is demonstrated ///   in the second example below. /// <ins> /// * The operation is "untyped" in the sense that data may be uninitialized or otherwise violate ///   the requirements of `T`. The initialization state is preserved exactly. /// </ins> /// # Safety /// /// Behavior is undefined if any of the following conditions are violated:

/// Swaps `count * size_of::<T>()` bytes between the two regions of memory /// beginning at `x` and `y`. The two regions must *not* overlap. /// <ins> /// The operation is "untyped" in the sense that data may be uninitialized or otherwise violate the /// requirements of `T`. The initialization state is preserved exactly. /// </ins> /// # Safety /// /// Behavior is undefined if any of the following conditions are violated:

if mem::align_of::<T>() >= mem::align_of::<$ChunkTy>() && mem::size_of::<T>() % mem::size_of::<$ChunkTy>() == 0 { <del> let x: *mut MaybeUninit<$ChunkTy> = x.cast(); let y: *mut MaybeUninit<$ChunkTy> = y.cast(); </del> <ins> let x: *mut $ChunkTy = x.cast(); let y: *mut $ChunkTy = y.cast(); </ins> let count = count * (mem::size_of::<T>() / mem::size_of::<$ChunkTy>()); // SAFETY: these are the same bytes that the caller promised were // ok, just typed as `MaybeUninit<ChunkTy>`s instead of as `T`s. // The `if` condition above ensures that we're not violating // alignment requirements, and that the division is exact so // that we don't lose any bytes off the end. <del> return unsafe { swap_nonoverlapping_simple(x, y, count) }; </del> <ins> return unsafe { swap_nonoverlapping_simple_untyped(x, y, count) }; </ins> } }; }

} // SAFETY: Same preconditions as this function <del> unsafe { swap_nonoverlapping_simple(x, y, count) } </del> <ins> unsafe { swap_nonoverlapping_simple_untyped(x, y, count) } </ins> } /// Same behaviour and safety conditions as [`swap_nonoverlapping`]

/// `swap_nonoverlapping` tries to use) so no need to manually SIMD it. #[inline] #[rustc_const_unstable(feature = "const_swap", issue = "83163")] <del> const unsafe fn swap_nonoverlapping_simple<T>(x: *mut T, y: *mut T, count: usize) { </del> <ins> const unsafe fn swap_nonoverlapping_simple_untyped<T>(x: *mut T, y: *mut T, count: usize) { let x = x.cast::<MaybeUninit<T>>(); let y = y.cast::<MaybeUninit<T>>(); </ins> let mut i = 0; while i < count { <del> let x: &mut T = // SAFETY: By precondition, `i` is in-bounds because it's below `n` unsafe { &mut *x.add(i) }; let y: &mut T = // SAFETY: By precondition, `i` is in-bounds because it's below `n` // and it's distinct from `x` since the ranges are non-overlapping unsafe { &mut *y.add(i) }; mem::swap_simple(x, y); </del> <ins> // SAFETY: By precondition, `i` is in-bounds because it's below `n` let x = unsafe { &mut *x.add(i) }; // SAFETY: By precondition, `i` is in-bounds because it's below `n` // and it's distinct from `x` since the ranges are non-overlapping let y = unsafe { &mut *y.add(i) }; mem::swap_simple::<MaybeUninit<T>>(x, y); </ins> i += 1; }

} #[test] <ins> fn swap_copy_untyped() { // We call `{swap,copy}{,_nonoverlapping}` at `bool` type on data that is not a valid bool. // These should all do untyped copies, so this should work fine. let mut x = 5u8; let mut y = 6u8; let ptr1 = &mut x as *mut u8 as *mut bool; let ptr2 = &mut y as *mut u8 as *mut bool; unsafe { ptr::swap(ptr1, ptr2); ptr::swap_nonoverlapping(ptr1, ptr2, 1); } assert_eq!(x, 5); assert_eq!(y, 6); unsafe { ptr::copy(ptr1, ptr2, 1); ptr::copy_nonoverlapping(ptr1, ptr2, 1); } assert_eq!(x, 5); assert_eq!(y, 5); } #[test] </ins> fn test_const_copy() { const { let ptr1 = &1;