jiebi/CodeConvo huggingface dataset:
To be clear, I wasn't proposing changing anything apart from MIR building either, I don't think. IIUC, what you proposed is that would be compiled to: whereas under my proposal (limited to apply to unsized parameters, instead of moved parameters) would yield: I think I mis-stated the problem earlier. It's not so much that your proposal will introduce compilation errors, it's that I think it changes the semantics of code like this in what I see as a surprising way: I believe that this code would compile to the following under your proposal: which means that would be invoked with the new value of , right? Under my proposal, the code would still compile, but it would behave in what I see as the expected fashion. Specifically, it would generate:
Isn't there a way to "lock" that value through the borrow-checker, so further accesses would be disallowed, until the call returns? I'd rather be stricter here. <hr/My point was that if your proposal affects sized moves, you're breaking stable reinitialization. If it doesn't affect sized moves, I don't see how this applies anymore:
Let me just chime in here and say that this is a pain in Miri. Unsized locals are pretty different from sized locals, I had to introduce a bunch of special hacks to make them work. MIR locals even needed a new state just for them: Lazily allocating locals seems odd to me, but at least we can consistently do that for all locals. Elsewhere however to make this lazy initialization actually work out and catch mistakes where an unsized local is "overwritten" with a value of different size: This all feels really unprincipled, and it would be great if at some point we could come up with a coherent semantics for unsized locals in MIR. The worst part is that the size of a local is implicitly determined on the first write to it. It would be much better to be explicit about that, e.g. at time.
I don't think we have anything like this today, but presumably we could add such a thing, but I'm not quite sure what you have in mind. When does this prohibition on further accesses take effect?
In particular, the IR for under your proposal, if I understood, would be But the access to that (I believe) we wish to prevent takes place in the "evaluate remaining arguments" part of things. So I think we'd have to add some sort of "pseudo-move" that tells borrowck that should not be assignable from that point forward, right? Well, it depends on your perspective. It's true that, given the need to permit reinitialization, the check is now specific to unsized values -- i.e., MIR construction will be different depending on whether we know the type to be sized or not. That's unfortunate. However, I think we still maintain a degree of congruence, from an end-user perspective. In particular, users could not have reinitialized a anyhow. However, under my proposal, users can write things like and the code works as expected (versus either having an unexpected result or getting an error). In any case, it seems like we're sort of "narrowing in" on a proposal here: when we see a function call with a parameter that is , and the value being passed is a deref, we want to modify the IR to not introduce a temporary and instead pass that parameter directly from its location, either we do some sort of "no more assignments" lock, or we move the pointer and introduce a temporary (and then pass in a deref of that temporary) Does that sound right so far?
If it's easier to do the more flexible approach (by moving the pointer) than the "locking" approach (which is a bit like borrowing the pointer until all arguments are evaluated, then releasing the borrow and doing a move instead), I don't mind it, I was only arguing with the "uniformity" aspect. Something important here: your approach is stabilizable, while mine is meant more for that one in the standard library, and not much else.
Yes, I was kind of shooting for something that we could conceivably stabilize.
Removing nomination, this was already discussed in triage meeting and it's under control.
So should we maybe try out the approach I proposed and see how well it works?
yes, going to try that out.
For the record, the "new plan" is this.
When generating the arguments to a call, we ordinarily will make a temporary containing the value of each argument: But we will modify this such that, for arguments that meet the following conditions: The argument is moved (not copied), and its type is not known to be , and the "place" has outer derefs (e.g., we are passing P where P is some place), then we assign the place P (instead of ) to a temporary and we make the argument to the function be (instead of ). This means that given where , we used to generate but now we generate We are now moving the box itself, which is slightly different than the semantics for values known to be sized. In particular, it does not permit of the box after having moved its contents. But that is not supported for unsized types anyway. The change does permit modifications to the callee during argument evaluation and they will work in an analogous way to how they would work with sized values. So given something like: we will find that we (a) save the old value of to a temporary and then (b) overwrite while evaluating the arguments and then (c) invoke the with the old value of .
Update: There is also
is a perfectly normal operand, I don't know what you mean. It's the same kind of as we currently have in (on the RHS)
Oh sorry, somehow I thought operands could only refer directly to locals, not also dereference them. I think I am beginning to see the pieces here -- together with function argument handling, this entirely avoids having to dynamically determine the size of a local. I have to agree that is quite elegant, even though baking "pass-by-reference" into MIR semantics still seems like a heavy hammer to me. On the other hand this places a severe restriction on now unsized locals may be used, basically taking back large parts of Is there some long-term plan to bring that back (probably requires with dynamic alignment in LLVM)? Or should we amend the RFC or at least the tracking issue with the extra restrictions?
I would keep the more general feature but use two different feature-gates (if we can). And just like with a few other feature-gates () we could warn that (the general one) is incomplete and could lead to misaligned variables (or something along those lines).
We could also add runtime asserts that the (dynamic) alignment is respected, once we know could never trigger them (e.g. from stable code).
I was wondering about that too. It seems like it'd be good to review the more general feature in any case. This is partly why I brought up the idea of wanting a guaranteed "no alloca" path -- this was something I remember us discussing long ago, and so in some sense I feel like the work we're doing here isn't just working around LLVM's lack of dynamic alignment support (though it is doing that, too) but also working to complete the feature as originally envisioned.
I two notes to the tracking issue so we don't overlook this in the future.
This is probably expected, but the fix failed to actually make sound:
The feature gate for the issue is . Tracking issues are used to record the overall progress of implementation. They are also uses as hubs connecting to other relevant issues, e.g., bugs or open design questions. A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature. Instead, open a dedicated issue for the specific matter and add the relevant feature gate label. [ ] Adjust documentation ([see instructions on rustc-dev-guide][doc-guide]) [ ] Stabilization PR ([see instructions on rustc-dev-guide][stabilization-guide]) [stabilization-guide]: https://rustc-dev- [doc-guide]: https://rustc-dev- Initial implementation ()
FWIW, here is a full-featured API provided by an external crate:
I wonder how this API relates to . It would be great if there was some safe API that wrapped around the use of to access a 's spare capacity, then updated based on the s .
Maybe we should follow that example and generalize to return ?
has implemented that, albeit as a private function. Sounds like a good idea to me.
Since exists now I think is redundant.
That's not actually true due to pointer invalidation, see and
I think it would be better to have this feature sooner rather than later. I'm starting to notice instances where people are doing things that are probably or definitely UB because they want this functionality. So I think this API falls squarely into the standard library's mission to provide pre-built and correct implementations of things that are subtle or difficult to get right.   (originally posted on but per Ralf's comments there, it is probably better to have this method for the above use cases)
I agree. I've been using this in my code and found it to be very useful. Even though is more general, I've never had a need for it and I feel that is useful on its own. The API being stabilized is: merge
Team member has proposed to merge this. The next step is review by the rest of the tagged team members: [x] [x] [x] [x] [x] [ ] No concerns currently listed. Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!
See for info about what commands tagged team members can give me.
while waiting for this feature to be stabilized, you should suggest that and use one of the convenience methods in:
:bell: This is now entering its final comment period, as per the . :bell:
Ralf mentioned this on , mostly rewording with my own opinion: I would prefer not to recommend that function, because even if you just want access to the spare region, under SB, these interfaces invalidate all pointers to the initialized region. I think that's too much of a footgun for codebases where people are already writing code that is rejected by Miri.
I wonder how many people do keep outstanding pointers to the initialized part while toying with the extra part 's implementation is already better than their 0-elems slices that get -ly converted into non-empty slices (and currently better than the stdlib one since it doesn't involve a 12-week delay for stable Rust). Anyhow, I've just released a new version () with a dedicated adapter: , for extra good measure.
To be clear, my concern is that the splitting interfaces are adding a landmine for the future. If I'm sending someone a PR I'll make sure it passes Miri at the time, but that's a hard condition to maintain. Especially because the two aforementioned codebases have dependencies that do int-ptr casts, an operation that Miri just doesn't understand at the moment, so it would be hard to advocate Miri in CI.
The final comment period, with a disposition to merge, as per the , is now complete. As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed. This will be merged soon.
I tried to document this code: I expected to see this happen: charstd::char` points to module page of char type points to char primitive page Instead, this happened: type points to module page of char :
I think the problem is that modules are in the type namespace, so is doing exactly what you are asking it for (the module, since it has higher priority). Since the built-in types scope is the lowest precedence, perhaps there needs to be a way to access it directly? Maybe something like or ? Or maybe should skip over modules, and keep looking deeper in the scope hierarchy? (cc )
Oh hmm I see what you mean. There are : Types, values, and macros. Since modules and types are in the same namespace, it uses the first one it finds. Maybe we could work around this by first resolving in an empty module (so it has nothing in scope), then only falling back to the current scope if we don't find something there?
I'm surprised this hasn't caused trouble before now. Does the root of never use the primitive?
I don't know how it works, but the resolver is smart enough to know that in a type context won't match a module (like , the won't match a module named "char". Maybe it has something to do with late-resolution  which has a to give it some context?
Adding that would be more of area ... I can give it a shot though.
It looks like doesn't even return a resolution and only records it for later use ... I'm going to go with the empty module workaround and we can fix it later if it turns out to cause issues.
The double resolve seems hacky. We could probably make type@ specifically not resolve to a module, perhaps? Also, note that modules are in all namespaces
How would that work? We could skip any module resolutions, but since modules have precedence over types I don't know how we'd get the primitive type to resolve afterwards.
Oh hold on we have a function already. We can look at directly.
This doesn't have anything to do with primitives, this has to do with glob imports vs non-glob imports. You can get the same issue with the following:
This also doesn't have to do with modules being "resolved first" or anything. Things that are non-glob in-scope are resolved first.
Actually, my bad: This does have to do with primitives in the sense that primitives are the only thing that still work when shadowed because of some special code in checking the primitive tables. I think when the string matches one of the primitive table entries for TypeNS lookups that were explicitly annotated with type@ and resolve to a module we can hack this in. In the future we might want to tighten up this code so that e.g. can't resolve to an enum, etc.
do you mind if I fix at the same time and resolve primitives even before modules? That seems like the less surprising behavior to me, the module can still be disambiguated with .
I'm a bit wary of doing this because having the char module in scope is not a common situation, and it will get weirder for people who have a different char module. I think we should resolve to whatever is in scope, but if is explicitly mentioned we can then force resolve to the primitive.
https://crater-
I'll need to re-check later but this seems to bisect to somehow.
The UI test changes from that PR appear to back this up, as confusing as it is.
I'm not sure what's going on in that crate, if it built successfully by mistake, or how that PR interacts with it, so let's cc the PR author and reviewer for validation.
After , no longer implements and , because the context now contains a which doesn't implement those traits. So this is an issue with the standard library and not the crate being built. Repro: label -T-compiler +T-libs +S-has-mcve
WG-prioritization assigning priority (). Noticing this on which says that was meant to be confined on nightly. Unsure about the fallout of this regression but for now marking as critical for tracking purposes. label -I-prioritize +P-critical
Revert up:
Isn't not unwind safe?
That line was by your PR, so while what you say is technically true, talking about what "already happened" when we are traveling back and forth through time via version control may be more confusing than helpful.
That line was already there. I the line below it though. Original PR:
Apologies, I guess I can't count numbers today? Hm, so, the full error of the regression is this: Note that it's trying to make a pointer implement . And for what it's worth, stable does indeed document is . Thus unfortunately that line you pointed to, as opposed to the one I thought you were referring to, is effectively just documenting that all are , and that will include regardless of whether it is is . And confusingly, , unlike .
I have opened as an alternative.
The following code () panics the compiler in stable 1.38.0, beta 1.39.0-beta.6, and nightly 2019-10-17: Removing the bound or filling it with makes it compile just fine. Error + backtrace:
cc
Hmm, I didn't know that adding a "syntactically trivial bound" could cause something like this. Is there some implicit bound that gets in cases like this?
minimized test case ():
The minimized test worked in previous stable Rust versions (at least in 1.33 under 2015 edition; though 2018 edition breaks there, which indicates this is with 99.999% certainty injected by MIR-borrowck...)
triage: P-high. Adding self to assignee list; removing I-nominated.
The following snippet works fine before but now it reports a warning: Steps to reproduce: clone rustup override set nightly-2017-08-11 cargo test Is this an expected behavior, or a bug? Thank you!
cc -- probably related to the new lint implementation
Yes this was caused by The bug is on . The check for whether is in scope is not checking the current set of attributes being linted. For example this does not warn: but this warns: (as you've noticed) The fix is to just take the local variable into account which has the up-to-that-point set of lint attributes collected.
I've opened a PR for this at
<!-- Thank you for filing a regression report! A regression is something that changed between versions of Rust but was not supposed to. Please provide a short summary of the regression, along with any information you feel is relevant to replicate it. --I tried this code: I expected to see this happen: same codegen or similar in performance Instead, this happened: array has way worse codegen <details<summaryAssembly</summary<p</p</details<!-- Provide the most recent version this worked on, for example: It most recently worked on: Rust 1.47 --It most recently worked on: 1.64 <!-- Provide the version you are using that has the regression. --65 till now. Regressed in nightly-2022-08-13, maybe LLVM 15 ?: : <!-- Did the compiler crash? If so, please provide a backtrace. --<!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summaryBacktrace</summary<p</p</details<!-- If you know when this regression occurred, please add a line like below, replacing with one of stable, beta, or nightly. -- modify labels: +regression-from-stable-to-stable -regression-untriaged <!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOTEND --
WG-prioritization assigning priority (). cc since this regressed in the LLVM 15 update label -I-prioritize +P-medium +T-compiler
I've seen llvm doing these strange 1-byte-at-a-time "vectorizations" in other places too. This might be a more general problem. In this case we can probably paper over it by implementing TrustedRandomAccess for the array iter.
Unfortunately, the array iterator is fundamentally worse right now, because indexing into it -- a fundamental part of being a by-value array iterator -- keeps it from SRoAing and thus doesn't optimize well. This is why when I was making better I ended up making to not have to use . Sadly, for the foreseeable future you're better off using the "weaker" slice iterator when you can.
My guess is that the SROA is missing something in the opaque pointer mode.
<!-- [31mICE[0m: Rustc '-Zunstable-options --edition=2024 -ooutputfile -Zdump-mir-dir=dir' 'error: internal compiler error: impossible case reached', 'error: internal compiler error: impossible case reached' File: --auto-reduced (treereduce-rust): original: Version information ` Command: <!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summary<strongProgram output</strong</summary<p</p</details<!-- query stack: [coroutinebymovebodydefid] looking up the coroutine by-move body for [analysis] running analysis passes on this crate -- label +F-asyncclosure
As a step of reducing your code examples, could you please ensure that is necessary to reproduce this ICE before mentioning it in the issue? This ICE reproduces on edition 2021, which is the current stable edition.
My project fails on nightly also while using edition 2021.
Yep, a fix is already up and it's waiting for approval. I recommend downgrading to a slightly older nightly until this is fixed.
I'm opening this up to serve as a tracking issue for enabling multiple codegen units in release mode by default. I've written up a before but the tl;dr; is that multiple codegen units enables us to run optimization/code generation in parallel, making use of all available computing resources often speeding up compilations by more than 2x. Historically this has not been done due to claims of a loss in performance, but the is intended to assuage such concerns. The most viable route forward seems to be to enable multiple CGUs and ThinLTO at the same time in release mode. Blocking issues: [x] Enable for libstd on all platforms - - - - Potential blockers/bugs: - -
cc
Presumably this will remain a stable compiler flag, but what else will change by default? Is the plan to make this only affect , or is the plan to also make this affect ? Do we want to make continue to use only a single CGU, to hedge against regressions for people who are already willing to trade off compiler time for runtime?
I would specifically propose that any opt level greater than 1 uses 16 codegen units and ThinLTO enabled by default for those 16 codegen units.
This is very interesting. One would think that ThinLTO-driven inlining should always have to do less work when our much more conservative pre-LLVM inlining. But that doesn't always seem to be the case, as in the . Most tests on the seem to profit though. Overall it's still not a clear picture to me.
was that comment meant for a different thread? I forget which one as well though, so I'll respond here! As a "quick benchmark" I compiled the regex test suite with 16 CGUs + ThinLTO and then toggled inlining in all CGUs on/off. Surprisingly inlining in all CGUs was 5s faster to compile, and additionally . In those timings is where inlining is only in one CGU, where is inlining in all CGUs. One benchmark, , got twice as slow!
Yeah, the comment was in response to (rust-doom) but since that was closed already, I put it here.
Regarding the compilation time difference between pre- and post-trans inlining, my hypothesis would be that sometimes pre-trans inlining will lead to more code being eliminated early on (as in the case of regex apparently) and sometimes it will have the opposite effect. I suspect that there's room for improvement by tuning which LLVM passes we run specifically for ThinLTO. Or do we do that already?
Heh it's true yeah, I'd imagine that there's always room for improvement in pass tuning in Rust :). Right now we perform (afaik) 0 customization of any pass manager in LLVM. All of the normal optimization passes, LTO optimization passes, and ThinLTO optimization passes are all the same as what's in LLVM itself.
I wanted to also take the time and tabulate all the results from the to make sure it's all visibile in one place. Note that all the timings below are comparing a release build to a release build with 16 CGUs and ThinLTO enabled Improvements All of the following compile times improved, sorted by most improved to least improved Regressions The following crates regressed in compile times, sorted from smallest regression to largest alexcrichton's attempt to reproduce the regressions Here I attempt to reproduce the regressions on my own machine with Unfortunately the only regression I was able to reproduce was the rust-belt regression. I'll be looking more into that.
Looking into , one interesting thing I've found is that the crate takes longer to compile with ThinLTO than it would otherwise. Looking at it appears that one codegen unit in this crate takes 99% of the time in LLVM. This codegen unit appears to be basically entirely dominated by . Almost all of the benefit of multiple codegen units is spreading out the work across all CPUs. Enabling ThinLTO to avoid losing any perf is fundamentally doing more work than what's already happening at , but we get speedups across the board in most cases. If we have one huge CGU, split it in two, and work on those in parallel then we've got 50% of the original time to run ThinLTO (in a perfect world). If ThinLTO takes more than 50% of the time then we'd have a build time regression, but that's almost never happening.