Change:
The following code fails on stable: When compiled in release mode, the assert is optimized out by LLVM even though it checks the exact same condition as the other assert, as can be verified by testing in debug mode. Note that it's theoretically possible for the stack to be aligned correctly such that the bug is suppressed, but that's not likely. The 256's can be replaced with larger alignments if that happens. The bug is caused by the impl of copying to the stack with the default alignment of 16. ()
and seem related
This is a soundness hole.
So... I guess the problem is that the that rustc emits for the unsized local doesn't use ? That's... kind of bad, because it doesn't look like even supports dynamically setting the alignment: If this is correct, the issue should be renamed to something like "unsized locals do not respect alignment".
A way to fix this would be to change the ABI for dynamic trait calls that take by value to always pass by address. When the trait object is created is where the trampoline function would be generated, so there is no additional cost for static calls.
This can also be fixed by not allowing dynamically-sized local variables with an alignment more than some maximum value and having all dynamically-sized locals use that maximum alignment unless optimizations can prove that a smaller alignment suffices.
Messing with the ABI wouldn't help with unsized locals in general (assuming is correct), since e.g. will also need an alloca with dynamic alignment in general. Given only a value, there is no static way to say anything about its alignment, so how would you distinguish whether calling a method on it is OK?
Can we call and then align the pointer for the local ourselves as a workaround until we come up with a good solution?
In clang there's a function which might or might not be useful in this case.
requires a constant alignment. Non constant alignments are . This matches semantics of LLVM instruction, so it's not helping, as alignment of value in is unknown at compile time when moving from .
I think that overallocating for unsized locals of unknown alignment may make sense. You can and then make use of to compute the offset. Sure, it increases stack requirements, but it seems acceptable for , and in most cases you end up with extra 7 bytes on stack or less. 256 alignment seems like an edge case where wasting 255 bytes doesn't matter.
if we keep setting as the default alignment we can skip the overallocating and offset dance for types with an alignment below that.
My idea was that rustc would prohibit coercing overaligned types to a dyn trait that takes by value. So:
The alignment is dynamically determined though, not sure if saving 15 bytes of stack space is worth the conditional branch?
Oh right. Yea that's probably not helpful.
Since , this would be backwards incompatible (at latest) once trait object upcasting is finally implemented: and trait objects would have to have the same alignment restriction (because you can upcast to ), but today you can freely create such objects with any alignment.
I was under the impression that calling didn't require dynamic , is that related to missing MIR optimizations, meaning we have unnecessary moves which end up duplicating the data on the stack using a dynamic ?
Correctness shouldn't rely on optimizations to fire...
Sure, I was just hoping we weren't relying on "unsized locals", only unsized parameters (i.e. ), for what we're exposing as stable. And I was hoping an optimization wouldn't need to be involved to get there. I wonder if it would even be possible to restrict "unsized locals" to the subset where we can handle it entirely through ABI, and not actually have dynamic s anywhere, since that would make me far more comfortable with 's implementation or even stabilization. Ironically, if that works, it's also the subset we could've implemented years earlier than we did, since most of the complexity, unknowns (and apparently bugs) are around dynamic s, not calling trait object methods on a dereference of .
That should be possible. I think everything that is necessary is performing the deref of the box as argument () instead of moving the box contents to a local first () That would also make it possible to call in cg_clif. Cranelift doesn't have alloca support, so the current MIR is untranslatable.
Hmm, but that's not trivially sound because it could lead to MIR calls where argument and return places overlap, which is UB. We'd potentially have to special-case it to derefs, but even then what's to stop someone from doing ? Maybe we could introduce a temporary for the return value instead, for these kinds of virtual calls? That should remove any potential overlap while allowing the call. (Unrelatedly, I just realize we use a copying-out shim for vtable entries that take by value, even when would be passed as at the call ABI level. We would probably have to make computing the call ABI a query for this microopt to not be expensive)
That would result in being moved to a temp first before dropping and calling the boxed closure. If it wasn't moved first before the drop of , then borrowck would give an error. (I meant for this change to be done at mir construction level, not as an optimization pass, so borrowck should catch any soundness issues)
We can try it, I guess, but right now a lot relies on many redundant copies/moves being created by MIR building and may not catch edge cases like this. But also I don't think moving out of a moves the itself today. Me too, but I was going with the "correct by construction" approach of creating a redundant copy/move, just on the return side, instead of the argument side, of the call.
After revisiting on triage, removing nomination.
pre-triage: nominated for discussion in the next weekly meeting.
I just checked and, a function call roughly this (touched up for clarity): Some of my previous comments assumed the wasn't there and the call would use the destination directly, but given that there is a move, that simplifies my suggestion. <hr/For virtual by-value calls, we can skip one move when building the MIR, specifically the move out of the dereference of a for the argument. Given that we already move the result of the call, there should be no aliasing caused by this, but if we decide to change anything about how we build MIR calls, we'd need to be extra careful. We can also make this pattern be gated by a separate feature-gate (rather than ), and make sure the standard library only uses that feature-gate and not . This should solve any alignment issues might have, because we'd not be using dynamic unless the broader feature is used.
OK, if I understand what's being proposed here, it is to excerpt a subset of the "original unsized-locals plan" -- specifically, to permit locals with unsized types, but only in contexts where we can be guaranteed that we can point to the original storage and don't actually have to move any value (and hence we don't require variable size stack frames). I'm in favor of this proposal, but I do think we want to be slightly careful to ensure we're not violating soundness. The main concern would be if there is some way to access or re-use the variable that was moved. I attempted to do so with this example () but it fails to compile: The error here arises because the closure would need to capture a to do anything, which prevents a later move. But maybe there is some kind of tricky example that can arise where we re-assign the value during argument evaluation?
compiles and does before , where is the result of .
Borrowck should prevent that, right?
borrowck may rely on MIR building creating extraneous copies, so we can't just assume it helps us. IMO we should limit it to functions taking unsized parameters and function calls passing such parameters or dereferences of es (any other pointer/reference doesn't support moving out of).
Well, I think is right that borrowck would catch violations of soundness in some strict sense, but I guess what I meant is that we might be breaking backwards compatibility.
The example that feels pretty relevant. If I understood your proposal, it would no longer compile. If I recall (thinking back), I had a plan to handle this by save the to an alternate location in the actual call, we pass ownership of and then do a shallow free of the This still seems like it would work, though I'm not sure how easy/elegant it is to formulate. In a sense we'd be rewriting to .
Isn't example dependent on doing a direct virtual call? My understanding is that you can't do this on stable with because the virtual call exists in one place, in libstd, and it doesn't do anything weird, so it would work with a restricted ruleset. EDIT: if it can work on stable, it's because calling 's will move the whole before the assignment is evaluated, which is fine, because the shallow drop is actually inside that , not its caller.
I agree that this code doesn't compile on stable today, but I'm not sure if that's the point. I guess you're saying it's ok for it to stop compiling? I think what I would like is that our overall behavior is as consistent as possible between Sized and Unsized parameters. I think we could actually do a more general change that doesn't special case Sized vs Unsized or anything else. I think I am basically proposing the same thing as earlier, but with a twist: Today, when building the parameters, we introduce a temporary for each parameter. So if we had we might do: What I am saying is that we could say: If the argument is moved (not copied), and the "place" has outer derefs (e.g., we are passing where is some place), then we assign the place to a temporary and we make the argument to the function be : I don't believe this is observable to end-users, except potentially for when the destructor on runs. I believe that it accepts all the same code we used to accept. It also has nothing specific to or types at all. Does that make sense? I'm very curious to know if anyone sees any problems with it. (One obvious thing: if we ever add , its design would have to accommodate this requirement, but I think that's fine, in fact the complications around getting moves right for precisely this case are one of the reasons we've held off on thus far.)
Can you not reinitialize today? If you move the whole , it is indeed more conservative (and should let us skip unnecessary moves even in cases). But also, do you want to move the locals check from typeck to borrowck? I don't have an intuition for when your approach wouldn't create unsized MIR locals, from the perspective of typeck. However, this is only relevant if we want a stricter feature-gate. At the very least it seems like it would be easier to implement a change only to MIR building, and it should fix problems with for the time being, even without a separate feature-gate.
Hmm, that's a good point, I think do that. Sigh. I knew that seemed too easy. I remember we used to try and prevent that, but we never did it fully, and in the move to NLL we loosened those rules back up, so this compiles: Still, you could do my proposal for unsized types, though it would introduce an incongruity. UPDATE: Oh, wait, I remember now. For unsized types, reinitialization is not possible, because you can't do with an unsized rvalue (we wouldn't know how many bytes to move). So in fact there isn't really the same incongruity.
To be clear, I wasn't proposing changing anything apart from MIR building either, I don't think. IIUC, what you proposed is that would be compiled to: whereas under my proposal (limited to apply to unsized parameters, instead of moved parameters) would yield: I think I mis-stated the problem earlier. It's not so much that your proposal will introduce compilation errors, it's that I think it changes the semantics of code like this in what I see as a surprising way: I believe that this code would compile to the following under your proposal: which means that would be invoked with the new value of , right? Under my proposal, the code would still compile, but it would behave in what I see as the expected fashion. Specifically, it would generate:
Isn't there a way to "lock" that value through the borrow-checker, so further accesses would be disallowed, until the call returns? I'd rather be stricter here. <hr/My point was that if your proposal affects sized moves, you're breaking stable reinitialization. If it doesn't affect sized moves, I don't see how this applies anymore:
Let me just chime in here and say that this is a pain in Miri. Unsized locals are pretty different from sized locals, I had to introduce a bunch of special hacks to make them work. MIR locals even needed a new state just for them: Lazily allocating locals seems odd to me, but at least we can consistently do that for all locals. Elsewhere however to make this lazy initialization actually work out and catch mistakes where an unsized local is "overwritten" with a value of different size: This all feels really unprincipled, and it would be great if at some point we could come up with a coherent semantics for unsized locals in MIR. The worst part is that the size of a local is implicitly determined on the first write to it. It would be much better to be explicit about that, e.g. at time.
I don't think we have anything like this today, but presumably we could add such a thing, but I'm not quite sure what you have in mind. When does this prohibition on further accesses take effect? In particular, the IR for under your proposal, if I understood, would be But the access to that (I believe) we wish to prevent takes place in the "evaluate remaining arguments" part of things. So I think we'd have to add some sort of "pseudo-move" that tells borrowck that should not be assignable from that point forward, right? Well, it depends on your perspective. It's true that, given the need to permit reinitialization, the check is now specific to unsized values -- i.e., MIR construction will be different depending on whether we know the type to be sized or not. That's unfortunate. However, I think we still maintain a degree of congruence, from an end-user perspective. In particular, users could not have reinitialized a anyhow. However, under my proposal, users can write things like and the code works as expected (versus either having an unexpected result or getting an error). In any case, it seems like we're sort of "narrowing in" on a proposal here: when we see a function call with a parameter that is , and the value being passed is a deref, we want to modify the IR to not introduce a temporary and instead pass that parameter directly from its location, either we do some sort of "no more assignments" lock, or we move the pointer and introduce a temporary (and then pass in a deref of that temporary) Does that sound right so far?
If it's easier to do the more flexible approach (by moving the pointer) than the "locking" approach (which is a bit like borrowing the pointer until all arguments are evaluated, then releasing the borrow and doing a move instead), I don't mind it, I was only arguing with the "uniformity" aspect. Something important here: your approach is stabilizable, while mine is meant more for that one in the standard library, and not much else.
Yes, I was kind of shooting for something that we could conceivably stabilize.
Removing nomination, this was already discussed in triage meeting and it's under control.
So should we maybe try out the approach I proposed and see how well it works?
yes, going to try that out.
For the record, the "new plan" is this. When generating the arguments to a call, we ordinarily will make a temporary containing the value of each argument: But we will modify this such that, for arguments that meet the following conditions: The argument is moved (not copied), and its type is not known to be , and the "place" has outer derefs (e.g., we are passing P where P is some place), then we assign the place P (instead of ) to a temporary and we make the argument to the function be (instead of ). This means that given where , we used to generate but now we generate We are now moving the box itself, which is slightly different than the semantics for values known to be sized. In particular, it does not permit of the box after having moved its contents. But that is not supported for unsized types anyway. The change does permit modifications to the callee during argument evaluation and they will work in an analogous way to how they would work with sized values. So given something like: we will find that we (a) save the old value of to a temporary and then (b) overwrite while evaluating the arguments and then (c) invoke the with the old value of .
Update: There is also
is a perfectly normal operand, I don't know what you mean. It's the same kind of as we currently have in (on the RHS)
Oh sorry, somehow I thought operands could only refer directly to locals, not also dereference them. I think I am beginning to see the pieces here -- together with function argument handling, this entirely avoids having to dynamically determine the size of a local. I have to agree that is quite elegant, even though baking "pass-by-reference" into MIR semantics still seems like a heavy hammer to me. On the other hand this places a severe restriction on now unsized locals may be used, basically taking back large parts of Is there some long-term plan to bring that back (probably requires with dynamic alignment in LLVM)? Or should we amend the RFC or at least the tracking issue with the extra restrictions?
I would keep the more general feature but use two different feature-gates (if we can). And just like with a few other feature-gates () we could warn that (the general one) is incomplete and could lead to misaligned variables (or something along those lines). We could also add runtime asserts that the (dynamic) alignment is respected, once we know could never trigger them (e.g. from stable code).
I was wondering about that too. It seems like it'd be good to review the more general feature in any case. This is partly why I brought up the idea of wanting a guaranteed "no alloca" path -- this was something I remember us discussing long ago, and so in some sense I feel like the work we're doing here isn't just working around LLVM's lack of dynamic alignment support (though it is doing that, too) but also working to complete the feature as originally envisioned.
I two notes to the tracking issue so we don't overlook this in the future.
This is probably expected, but the fix failed to actually make sound:

Discussion:
/// Returns an operand suitable for use until the end of the current /// scope expression. /// <del> /// The operand returned from this function will *not be valid* after /// an ExprKind::Scope is passed, so please do *not* return it from /// functions to avoid bad miscompiles. </del> <ins> /// The operand returned from this function will *not be valid* /// after the current enclosing `ExprKind::Scope` has ended, so /// please do *not* return it from functions to avoid bad /// miscompiles. </ins> crate fn as_local_operand<M>(&mut self, block: BasicBlock, expr: M) -> BlockAnd<Operand<'tcx>> where M: Mirror<'tcx, Output = Expr<'tcx>>,

self.as_operand(block, local_scope, expr) } <ins> /// Returns an operand suitable for use until the end of the current scope expression and /// suitable also to be passed as function arguments. /// /// The operand returned from this function will *not be valid* after an ExprKind::Scope is /// passed, so please do *not* return it from functions to avoid bad miscompiles.  Returns an /// operand suitable for use as a call argument. This is almost always equivalent to /// `as_operand`, except for the particular case of passing values of (potentially) unsized /// types "by value" (see details below). /// /// The operand returned from this function will *not be valid* /// after the current enclosing `ExprKind::Scope` has ended, so /// please do *not* return it from functions to avoid bad /// miscompiles. /// /// # Parameters of unsized types /// /// We tweak the handling of parameters of unsized type slightly to avoid the need to create a /// local variable of unsized type. For example, consider this program: /// /// ```rust /// fn foo(p: dyn Debug) { ... } /// /// fn bar(box_p: Box<dyn Debug>) { foo(*p); } /// ``` /// /// Ordinarily, for sized types, we would compile the call `foo(*p)` like so: /// /// ```rust /// let tmp0 = *box_p; // tmp0 would be the operand returned by this function call /// foo(tmp0) /// ``` /// /// But because the parameter to `foo` is of the unsized type `dyn Debug`, and because it is /// being moved the deref of a box, we compile it slightly differently. The temporary `tmp0` /// that we create *stores the entire box*, and the parameter to the call itself will be /// `*tmp0`: /// /// ```rust /// let tmp0 = box_p; call foo(*tmp0) /// ``` /// /// This way, the temporary `tmp0` that we create has type `Box<dyn Debug>`, which is sized. /// The value passed to the call (`*tmp0`) still has the `dyn Debug` type -- but the way that /// calls are compiled means that this parameter will be passed "by reference", meaning that we /// will actually provide a pointer to the interior of the box, and not move the `dyn Debug` /// value to the stack. /// /// See #68034 for more details. crate fn as_local_call_operand<M>( &mut self, block: BasicBlock, expr: M, ) -> BlockAnd<Operand<'tcx>> where M: Mirror<'tcx, Output = Expr<'tcx>>, { let local_scope = self.local_scope(); self.as_call_operand(block, local_scope, expr) } </ins> /// Compile `expr` into a value that can be used as an operand. /// If `expr` is a place like `x`, this will introduce a /// temporary `tmp = x`, so that we capture the value of `x` at

self.expr_as_operand(block, scope, expr) } <ins> /// Like `as_local_call_operand`, except that the argument will /// not be valid once `scope` ends. fn as_call_operand<M>( &mut self, block: BasicBlock, scope: Option<region::Scope>, expr: M, ) -> BlockAnd<Operand<'tcx>> where M: Mirror<'tcx, Output = Expr<'tcx>>, { let expr = self.hir.mirror(expr); self.expr_as_call_operand(block, scope, expr) } </ins> fn expr_as_operand( &mut self, mut block: BasicBlock,

} } } <ins> fn expr_as_call_operand( &mut self, mut block: BasicBlock, scope: Option<region::Scope>, expr: Expr<'tcx>, ) -> BlockAnd<Operand<'tcx>> { debug!("expr_as_call_operand(block={:?}, expr={:?})", block, expr); let this = self; if let ExprKind::Scope { region_scope, lint_level, value } = expr.kind { let source_info = this.source_info(expr.span); let region_scope = (region_scope, source_info); return this.in_scope(region_scope, lint_level, |this| { this.as_call_operand(block, scope, value) }); } let tcx = this.hir.tcx(); if tcx.features().unsized_locals { let ty = expr.ty; let span = expr.span; let param_env = this.hir.param_env; if !ty.is_sized(tcx.at(span), param_env) { // !sized means !copy, so this is an unsized move assert!(!ty.is_copy_modulo_regions(tcx, param_env, span)); // As described above, detect the case where we are passing a value of unsized // type, and that value is coming from the deref of a box. if let ExprKind::Deref { ref arg } = expr.kind { let arg = this.hir.mirror(arg.clone()); // Generate let tmp0 = arg0 let operand = unpack!(block = this.as_temp(block, scope, arg, Mutability::Mut)); // Return the operand *tmp0 to be used as the call argument let place = Place { local: operand, projection: tcx.intern_place_elems(&[PlaceElem::Deref]), }; return block.and(Operand::Move(place)); } } } this.expr_as_operand(block, scope, expr) } </ins> }

use crate::build::expr::category::{Category, RvalueFunc}; use crate::build::{BlockAnd, BlockAndExtension, BlockFrame, Builder}; use crate::hair::*; <del> use rustc_middle::mir::*; use rustc_middle::ty::{self, CanonicalUserTypeAnnotation}; </del> use rustc_data_structures::fx::FxHashMap; use rustc_hir as hir; <ins> use rustc_middle::mir::*; use rustc_middle::ty::{self, CanonicalUserTypeAnnotation}; </ins> use rustc_span::symbol::sym; use rustc_target::spec::abi::Abi;

} else { let args: Vec<_> = args .into_iter() <del> .map(|arg| unpack!(block = this.as_local_operand(block, arg))) </del> <ins> .map(|arg| unpack!(block = this.as_local_call_operand(block, arg))) </ins> .collect(); let success = this.cfg.start_new_block();

<ins> // run-pass #![feature(unsized_locals)] #![allow(dead_code)] #[repr(align(256))] struct A { v: u8, } impl A { fn f(&self) -> *const A { self } } fn f2(v: u8) -> Box<dyn FnOnce() -> *const A> { let a = A { v }; Box::new(move || a.f()) } fn main() { let addr = f2(0)(); assert_eq!(addr as usize % 256, 0, "addr: {:?}", addr); } </ins>

error[E0382]: borrow of moved value: `x` --> $DIR/borrow-after-move.rs:39:24 | <ins> LL |         let x = "hello".to_owned().into_boxed_str(); |             - move occurs because `x` has type `std::boxed::Box<str>`, which does not implement the `Copy` trait </ins> LL |         x.foo(); |         - value moved here LL |         println!("{}", &x); <del> |                        ^^ value borrowed here after partial move | = note: move occurs because `*x` has type `str`, which does not implement the `Copy` trait </del> <ins> |                        ^^ value borrowed here after move </ins> error: aborting due to 5 previous errors

LL |         y.foo(); |         ^ value used here after move <del> error[E0382]: use of moved value: `*x` </del> <ins> error[E0382]: use of moved value: `x` </ins> --> $DIR/double-move.rs:45:9 | LL |         let _y = *x; |                  -- value moved here LL |         x.foo(); <del> |         ^ value used here after move </del> <ins> |         ^ value used here after partial move </ins> | = note: move occurs because `*x` has type `str`, which does not implement the `Copy` trait error[E0382]: use of moved value: `*x` --> $DIR/double-move.rs:51:18 | <ins> LL |         let x = "hello".to_owned().into_boxed_str(); |             - move occurs because `x` has type `std::boxed::Box<str>`, which does not implement the `Copy` trait </ins> LL |         x.foo(); |         - value moved here LL |         let _y = *x; |                  ^^ value used here after move <del> | = note: move occurs because `*x` has type `str`, which does not implement the `Copy` trait </del> error: aborting due to 6 previous errors

error[E0508]: cannot move out of type `[u8]`, a non-copy slice <del> --> $DIR/unsized-exprs2.rs:22:19 </del> <ins> --> $DIR/unsized-exprs2.rs:22:5 </ins> | LL |     udrop::<[u8]>(foo()[..]); <del> |                   ^^^^^^^^^ |                   | |                   cannot move out of here |                   move occurs because value has type `[u8]`, which does not implement the `Copy` trait </del> <ins> |     ^^^^^^^^^^^^^^^^^^^^^^^^ |     | |     cannot move out of here |     move occurs because value has type `[u8]`, which does not implement the `Copy` trait </ins> error: aborting due to previous error