jiebi/CodeConvo huggingface dataset:
root: updns-0.0.6: v.
triage: P-high. Nominating for discussion.
self-assigned for investigation in short term (as in, next two or three days). However I suspect someone from will be a better choice to investigate this.
We should try to bisect this
used . Determined: searched nightlies: from nightly-2019-11-26 to nightly-2020-01-03 regressed nightly: nightly-2019-12-01 searched commits: from to  regressed commit:
is a rollup commit of the following PR's: I'm guessing this is due to PR
I'm still working on reducing this down to something that I could put into the playpen. However, one interesting thing I've found locally during my reduction: This version of the will signal an analogous compilation error: but if I lift the first argument out to a separate -binding (i.e. change it to use above, or the sub-expression out to a separate -binding (i.e. change the argument expression to use above), then the error goes away. I'll admit that I don't know all the details of how the fn-transformation works to support , but I strongly suspect that if this kind of local code change can make an error go away, then that represents a weakness somewhere (either in the fn-transformation, or in the borrowck).
Okay in the details block is a standalone version suitable for the : <details</detailsand the error it generates:
So this does seem like somewhat clear (expected?) fallout from PR : a use of a in a generator witness will be recorded as a unsafe pointer , and then is never , but the method requires that the task you give it is . I believe we can readily update the crate in question to avoid using the in the expression under the RHS of . But I do wonder whether we can/should improve the diagnostics here to try to guide the user towards such a solution ...?
I might have worded this too strongly, and/or over-reduced my example. In particular, its easy to move around the access to . But the borrow of may be trickier to deal with.
I'll have a look at addressing this with another approach I considered for . If that doesn't work out then I'll have a look at the error message.
The patch linked is enough to fix the crate in question, but I'm all for finding an in-compiler solution if we can do it soundly.
OK. I'm going to write some things up because I've changed my mind a bunch of times about how/if this can be fixed in MIR building and I don't want to keep forgetting things. Generators store values that if their value needs to be preserved across a yield. These values need to be visible for auto-traits, for example a generator may be resumed on a different thread after it yields, and so cannot be if any of theses values isn't . There are two passes that determine which values can be included in the generator: A pass that runs as part of typeck that walks the HIR to find the types of expressions and then checks for yields in the values scope (as determined by the RegionScopeTree). The types recorded here are stored in the GeneratorWitnesss type contained in the generator, which is then used for auto trait checking. This notably uses the types of expressions that HIR sees. The MIR transform that lowers the generator to a state machine. This determines the values that the generator actually stores. As a check for the compiler, we assert that any types that we capture here are included in the witness type (we don't know all auto traits, so can't check only for them). After statics no longer have a special representation in MIR. They are instead lowered to a dereference of a pointer to the given static: is lowered as if it were: Which becomes: The example above is somewhat deceptive in that the temporary that holds the pointer is used immediately after it's assigned. In this case it cannot end up in the generator since it's not live across any user code. However this isn't true in all cases, index and match expressions allow running user code while the pointer is live: In these cases MIR has to put the temporary pointer into the generator state and promptly ICEs. The obvious solution now is make type checking record the new pointer types in the generator witness in these cases. Unfortunately HIR is not a great tool for specifying "these cases". The methods that the region scope tree provides in HIR are either usually far too large (the "temporary scope") or occasionally too small (the "enclosing scope"). went for the sound but rather approximate approach, leaving us here.
One idea might be to try changing MIR building so that the temporary is never live across user code. For example, one might change the lowering as follows However working out how to do this for in a reasonable to implement way is harder. MIR actually already contains an escape hatch for values that are in the MIR but aren't in the HIR in the form of a flag (called ) on the variable. We currently use it for drop flags and boxes created by expressions. It's unclear what exactly the difference is between is. I guess one could argue that auto traits are for layout introspection and this would be further breaking that. I think that's everything for now.
Visiting from triage. were you planning to work on this further?
It's definitely tempting to take advantage of the constraints around statics to just declare these equivalent, and use the flag to get out of this. While I don't see any soundness issues around that, I also don't quite trust myself to get this right :) Can you expand on this? I'm not sure I follow.
Another solution that comes to mind relies on the fact that these pointers never need to be stored in a generator. They are consts, and the temporaries containing them shouldn't ever be mutated. We could add semantics for "const temporaries" to MIR, and never store them inside generators. The semantics would (hopefully) be simple: defined once and assigned a value that doesn't depend on any other temporaries; never mutated or borrowed mutably. We can move the statements defining them to the top of the function inside the generator transform. But that would be adding more semantics and special casing to MIR to fix a change which removed extra semantics and special casing, so I'm not sure if that's what we want :)
The argument is that the given the following auto trait we guarantee that if then there are no raw pointers directly stored in an instance of . If we make that guarantee then using internal in generators would not be OK. I would argue that we're not making such (or really any) guarantees with user defined auto traits.
Yeah, I don't think auto traits should expose layout implementation details of generators. And this really does seem like a pure implementation detail -- it doesn't have any bearing on what kinds of (unsafe) operations are allowed on a generator, like a self-reference would.
cc
So, I've been putting off commenting here to let these ideas sit, but let me leave a comment. I think I've reached a conclusion. TL;DR. I think we should mark these types as internal, but I think there are some better options for fixing this in the medium term, and this interaction should be taken into account as part of the "MIR 2.0" effort to rework places. Auto traits weren't really intended as a tool for probing layout, but rather semantic considerations. This is why traits like can be implemented by users. So I don't really think that auto traits give any sort of guarantee about their layout. Rather, this question is about is simply "what are the constituent types exposed by a generator", and this should correspond to the types that user code can directly access. The raw pointers here are not directly accessible. The values that are accessible are the "live values" that the user's code may later use -- and of course the code currently makes the constituent types be a superset of those, based on the AST structure. Of course, the real root cause of this error is a simplifying change to the definition of in MIR. It used to be that a Place had this grammar: but now we changed to just , which is why we are required to introduce the new temporary (and that holds the address of the ). As we've been talking about this change, I've been saying that we should be considering the older definition of as the one that is used by various analyses (and in particular by borrow check). This particular transformation of removing statics doesn't affect borrow check, but we failed to take into account the fact that the set of live variables can effect the definition of what is live in a generator. In particular, compiling for some expression requires a temporary unless corresponds to a place -- those places used to include statics like , but they no longer do. But I think that for analysis purposes, it should appear as if they do. We could fix this in two ways. We could write some code motion / duplication that takes temporaries like that are live across a yield and creates a fresh variable after the yield. The idea would be to make them never* live across a yield.
We can do this because that operation is a compile-time constant, really, and thus the value doesn't need to be live across a yield. This might be worth doing as a size optimization regardless. Secondly, some of the proposals we've been kicking around for going further with simplifying include the idea of a distinct sort of temporaries that explicitly represent a "partially constructed that hasn't been used by the user". The idea was to allow borrowck (and other analyses) to trivially reconstruct the . We might want to add this interaction with auto-traits into the consideration for that design, and maybe it takes us some place (e.g., these special temporaries are not considered live for the purposes of auto traits). Having written out both of those options, though, I sort of lean towards the first, at least for this specific issue.
I think these references and raw pointers created by MIR construction should be marked as . They are not visible at the language level. They can be viewed as just prefetching the address of a static, so the compiler could pick any type it likes for it (, , , some compiler internal type) so it must not affect auto trait inference. Drop flags are ignored for similar reasons. You cannot use an auto trait like this to detect them: So I think that should be reverted and the temporary static references should be marked as .
Note: opened instituting this change and I r+'d it, since fixing the regression seems pretty important and time is of the essence; there is general consensus here on the thread; this is only restoring older behavior. In particular, there were some suggestions that this might be a lang team issue; I think there is some 'intersection' and I don't want the perception that I'm trying to shortcircuit discussion. That said, I also think this is primarily a compiler team matter, since it's a deviation from the old behavior, and further this deviation was caused by altering the MIR away from the "natural, Rust-based" definition and towards a lower-level definition.
A simple malformed macro makes the compiler crash. Code: Playpen error: Playpen:
Another case: Playpen:
I had a crash in a bad macro which hits the same bug as Note that his/her error has the following which is a different stack trace: I'm on the latest beta; not nightly.
I think you and I are getting a different panic from this bug. I've opened since the panic is in a different location in the code.
<!-- Thank you for finding an Internal Compiler Error! If possible, try to provide a minimal verifiable example. You can read "Rust Bug Minimization Patterns" for how to create smaller examples. --<!-- If you're using the stable version of the compiler, you should also check if the bug also exists in the beta or nightly versions. --: <!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summary<strongBacktrace</strong</summary<p</p</details
label +F-dyn_star
:laughing: the code thinks that this is a upcast, I think, rather than an unsize coercion. shouldn't be too hard to fix, but I may leave it for someone else. edit: nvm I'll just fix it.
I'm trying to build my crate on nightly, and I'm getting a compiler panic. I bisected several nightly builds to try and narrow down when the problem started. The last known good nightly was , and the first failing was The panic also occurs in . The crates causing the panic are available here:
cc
triage: P-high, removing nomination label.
Minimal repro:
As some info, I've narrowed down the offending commit to –– most likely the changes in .
I tried this code: When I run it under , I get this: Creating a reference from such pointer looks suspicious indeed. I could send a pull request that adds a condition there, but I want to ask first if this is being done because „we are , we can“, or if this is something that really should be fixed. : (but I've looked at the git sources and the thing is still there) <!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOTEND --
I think it should only be UB to dereference the pointer you get from , so I'm not sure why Miri thinks there's UB happening. Just in case, prioritize
Because that pointer is being turned into a reference to feed to internally. And references are supposed to be always valid.
Looks like fixed this for but missed it in .
Yes, seems this needs to use .
I renamed the issue to reflect that the bug is actually in (which calls).
Cc
Assuming you use the default flags for Miri, that change should not be required to make Miri pass, so I'd be surprised if doing the same change for would make any difference here. If it does, that seems like a bug in Miri that I should investigate.
Ah, but the in is indeed a problem (the PR was mostly about so I got side-tracked; the is what should not have any effect here). Miri ensures that references are at least dereferencable, so yeah that would explain the problem here. Good catch!
Yep, I apparently missed this somehow in . Fixing the indicated line to use should be enough to fix it; I'm preparing the patch currently. claim
Hm, Miri still complains, even after the fix landed... it seems Miri doesn't like being called with dangling pointers. I'll have to look into this.
IIRC, when , I changed the type of the intrinsic to take a raw pointer rather than a reference, but miri wasn't touched, so that would make sense miri's still expecting a proper reference.
Patch arrived really fast, we are assigning "post-mortem" a priority of as discussed as part of the and removing .
We need to add an epoch lint for It will lint , suggesting removal in root mods, and replacement with in non-root mods. This is blocked on the ability to elide extern crate in the first place.
It is also blocked on a feature to force linkage of a crate, see this comment and the discussion afterwards:
cc
I think the lint should be by default off in cases with , such as: This is because of the ergonomics problems with macros and the new macro import system, expressed here: It's bewildering for the migrating users if a lint tells them not to use the syntax, but then finding out that many of the macros out there have downright bad ergonomics with the import syntax. (For clarification: I think this is an undesirable state of affairs, and we should be linting against all , but if we do that, the ergonomics problem with the macro import should be fixed somehow.)
I don't think we want at all, really. If we want to import macros, we should have another means of doing it. Otherwise we have an inconsistency. So yes, I think I agree with your last paragraph above all,
I've discussed this a bit in the tracking issue for the macros import system:
Thanks, you expressed the exact concern I had in a concise way.
Another thing this needs to lint is unlisted crates that are included in that clash with existing modules. I'm not quite sure on what the story needs to be here cc
Does it? I've thought one of the things the path clarity RFC was about was to have separate namespaces for top level modules and extern crates.
The current plan has changed a bit so I'm not sure if that is still true. (If stuff does change it will probably be RFCd again, there's a bit of experimentation going on)
Sigh I hope we won't discuss all of this crap all over again... I don't enjoy that but it is not me who wants to change things.
I've hidden your last comment, which I found hostile and unconstructive.
IDK what has been hearing, but for me, this is the latest state of consensus on that issue, and it seems to have separate namespaces: is that the case? You seem unresponsive on IRC that's why I'm using github.
That's mostly the latest state of consensus, yeah. Some tweaks have been made iirc. Everything has been implemented.
We need to find a way to implement Clone for where . In user code, "cloning" such a box manually is simple: will do it with a short roundtrip through which should be unproblematic. In the rust distribution, we can't impl Clone easily since the tools to do so are in libcollections but the Box type lives in liballoc.
You do not need vec for the implementation. Something like the following would work: Note that because can panic it is required to keep track of the partial result so the correct stuff is dropped.
Yes, that's nice that it can be carefully solved by essentially extracting the needed parts from what Vec does. But the Box code is otherwise so clean, it doesn't feel like the right way to do it.
You're welcome to suggest an alternative but I doubt it can be done in a manner that is both correct and significantly cleaner :)
Your fundamental analysis is great: just isn't an exception safe representation while filling it with elements. It needs to keep track of the fill level, so a representation like Vec or BoxBuild is needed...
Clone is also wanted for , but that can be much simpler: There is no panic issue, and it can use a simple memcpy of the data.
Instead it should use the order that appear in the crate file. Even better, it should allow adding section headers (with a description) between them. Hypothetically: And get something like this: This would make both the source code and the documentation more accessible as the modules would be presented in a comprehensible order, with some description of each related group of modules. Edit: Rename issue to reflect that this isn't an either-or. Alphabetical order is useful, but so is programmer-specified order (to make the code accessible for newcomers) and Rustdoc should support both - see the thread below.
Shouldn't related modules be nested in submodules rather than logically ordered in the source? Finding things in long non-ordered lists is a real pain.
Yeah, I'm going to vote "alphabetical".
Hmmm. I pretty sure I put a comment here, and now its gone. Strange. Anyway. Here it goes again. Alphabetical order == random order. That's great when you know what you are looking for and just want to find it. When coming to a new non-trivial code base, being presented the content in random order is just about the worst possible way to access it. Nested modules allow us to group related modules together, and that's a good thing. If you have a long list of modules, certainly, group them. But this doesn't solve the problem. You now have a TOC where each level is randomized, which is still inaccessible. Searching for a function or module by name is an important use case, sure. But being able to access a new code base is as important, if not more so. Looking up a name in a short non-alphabetized list (at each level) isn't too hard, and one can always compensate for the lack of an alphabetical index by using ^F. Trying to understand a system presented in random order is next to impossible, and one can not compensate for the lack of a comprehensible order in a large system. If there was a way to specify the order of the entries in each level, which we agree should only contain "a few" entries, then the TOC would become accessible. If one could even add a bit of annotation for each level, so much the better.
The annotated TOC would serve as a very accessible way for a newcomer to understand the code base. And, this doesn't rule out also generating an alphabetically sorted reference index, beside the TOC. All good books do both - a TOC in reasonable order and an index for looking up data by name. We shouldn't have to choose, we should be able to get both. So, I misnamed the issue. It should be "Rustdoc shouldn't list modules only alphabetically".
Agreed. Can someone rename the title?
Rename done.
Visiting for triage. Minor, but still an issue.
Triage, still listed alphabetically. Here's a sample source file: !
Rustc used to be able to replace with a constant when optimizing, not at the moment. It simply looks like a Layout method that is not inlinable. (This is a regression strictly speaking — stable & beta can do this optimization, but not nightly.) Code to reproduce:
Are there any ways to write automated tests to check for "regressions" like this? Assertions against LLVM bitcode?
Yes can be used for this, e.g. Assuming this is run against the bitcode after optimization, that is.
cc -- seems LLVM related
Nope. Just () that is not marked as inline.
cc
I didn't confirm that making Layout::repeat as inline fixes this, so please confirm that/add a test before closing.
<!-- Thank you for filing a bug report! Please provide a short summary of the bug, along with any information you feel relevant to replicating the bug. --I tried this code: I expected to see this happen: The three printed TypeIds should be identical, as with the behavior in Rust <=1.70 Instead, this happened: switch between rust versions. I could not find reference in the that could explain such behavioral changes.
if my bisection is correct, this surfaced in commit cc (makes sense?)
I do not believe that is the correct PR to point at. Bisecting anything related to MIR opts can be tricky because you need to stabilize the MIR opt pipeline against changes like in that PR which don't actually implement any new behavior, they just change the meaning of MIR opt levels. Try bisecting with ? The above PR just changes the opt level 3 inliner behavior to be the default.
Reproduces since before with
This has to do with the fact that this impl: ... applies even when . Therefore in a polymorphic function like , we're eagerly inlining from that impl even though during monomorphization we'd prefer the built-in candidate instead. This seems problematic.
WG-prioritization assigning priority (). label -I-prioritize +P-high
Passing rustc the flag and passing may produce different files (that is, outputs). This is true of rust-core (see below), though was not the case for a simpler ("Hello, world!") test file. Test script: Diff output indicates binary files differ. Tested with , rust-core on . rustc compiled from master on Arch Linux. If this is, in fact, expected behavior- sorry for the trouble, unexpected to me.
Can you provide a diff between the outputs? This is likely a known issue, but I would like to be sure.
.bc is a binary format, so just gives "Binary files and differ" as its output. On my machine, is 19K and is 62K. I'm not familiar enough with the format to extract much more data.
I note the difference persists when the second line is (two separate flags), though both produce both artifacts.
Can you pass and diff those outputs?
With emit flags as and respectively, there is no difference in and (according to diff.)
It seems that every bytecode is invalid only when a rlib is produced. It stopped working between rust-git-0.9.1301 and 1362 (shortly before 2014-02-23 15:37:05 -0800). Going back thorugh the log, I found I believe that creates deflated bytecode. Perhaps should be stored with extension or .