jiebi/CodeConvo huggingface dataset:
Code change:
fn after_krate(&mut self) -> Result<(), Error> { debug!("Done with crate"); <del> debug!("Adding Primitive impls"); for primitive in Rc::clone(&self.cache).primitive_locations.values() { self.get_impls(*primitive); } </del> let e = ExternalCrate { crate_num: LOCAL_CRATE }; let index = (*self.index).clone().into_inner(); debug!("Constructing Output");

Related discussion:
text: Generated rustdoc JSON for every one of dozens of crates I've looked at seems to contain unexpected inlined items that look like an block and its contents. I expected to see this happen: rustdoc JSON file contains only the items defined or re-exported by the corresponding crate. Instead, this happened: the rustdoc JSON file contains items like this: In this example, is and here are the names of the items contained in that impl (also part of the rustdoc JSON): Similarly, you can find items from : Invocation for generating rustdoc JSON: : You can find a variety of pre-generated rustdoc JSON files with this issue here: Briefly discussed in Zulip: https://rust- I'll have to work around this behavior to shield , which uses the same codebase as the playground link above. I can hold off on updating the playground for, say, another week, but at some point I'll need to upgrade it to include my workaround which means those items will cease being shown. label +T-rustdoc +A-rustdoc-json
Possibly duplicate of
Might be! Definitely at least some overlap, not entirely sure if duplicate. Do you mind if I ship the workaround for this in cargo-semver-checks and the playground, which will hide these items from our lints and from queries in the playground?
Yeah, it's easy enough to reproduce in raw json, I'm not worried about the repro breaking.
Code change:
<ins> use core::fmt; </ins> use core::iter::FusedIterator; <ins> use core::marker::PhantomData; use core::mem::{self, MaybeUninit}; </ins> use core::ptr::{self, NonNull}; <del> use core::{fmt, mem}; </del> use crate::alloc::{Allocator, Global}; <del> use super::{count, Iter, VecDeque}; </del> <ins> use super::{count, wrap_index, VecDeque}; </ins> /// A draining iterator over the elements of a `VecDeque`. ///

Related discussion:
text: liballoc's fails when run in Miri. The error occurs in the call: is called with a as argument. contains contains a shared reference to a slice; that slice is thus marked as "must not be mutated for the entire duration of this function call". calls calls calls calls , which eventually drops the . calls to re-arrange stuff (I have not fully understood this yet), and in some cases this will end up writing to memory that the slice in points to. I am not sure what the best way to fix this is. We have to fix holding (indirectly) a shared reference to something that it'll mutate during its . The mutation is likely there for a reason, so I guess the shared reference has to go. (FWIW, this shared reference already caused other trouble, but that was fixed by ) Cc <!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOTEND --
yeah I guess just change it to and toss in a PhantomData for the borrow. shrug
You mean, change this in as well? Sure, that should work. In principle this reduces the precision of compiler analyses on .
ahh misread. maybe would just not use Iter then. depends on what the impl looks like now
But the first thing it does is which should exhaust the such that its is an empty slice. I suppose its pointer will still be somewhere in the deque's memory, but with length 0 it can't actually read anything. Is that not good enough to appease Miri?
No. This code still matches the pattern There is no notion of "exhausting" or "consuming" a shared reference. When you create a shared reference with a certain lifetime, you promise that until that lifetime ends, no mutation happens. here is mutable so you can change what the slice points to, but that has no effect at all on the guarantees that have been made about the value that was initially passed to . Now, lifetimes are an entirely static concept, so for Stacked Borrows a "lifetime" generally is "until the pointer is used for the last time".
Code change:
<ins> use core::fmt; </ins> use core::iter::FusedIterator; <ins> use core::marker::PhantomData; use core::mem::{self, MaybeUninit}; </ins> use core::ptr::{self, NonNull}; <del> use core::{fmt, mem}; </del> use crate::alloc::{Allocator, Global}; <del> use super::{count, Iter, VecDeque}; </del> <ins> use super::{count, wrap_index, VecDeque}; </ins> /// A draining iterator over the elements of a `VecDeque`. ///

Related discussion:
text: However, for the special case of a reference being passed to a function, Stacked Borrows asserts that the "lifetime" will last at least as long as the function the reference is pointed to. (This is enforced by the "barriers" explained .) This is extremely useful for optimizations.
forward to . I wouldn't want to copy those.
Oh, I also didn't realize that doesn't actually shrink down to an empty slice -- it only updates its indexes. It would otherwise have to use two slices for the wrapped head and tail. So that full shared reference does still exist after the iterator is exhausted. (This also raises the validity question of having references to uninitialized data.) Could we base on instead? We could arrange to work through that mutable reference too, so there's no question of mutable aliasing.
That might work. However, must not access in that case -- that would be in conflict with the unique reference in .
Hmm, does need to be covariant over ? I guess would break that.
Actually this idea of "consuming a shared ref" is far from unique to ... the following code also errors in Miri:
As I understand it, Miri and stacked borrows should be "more powerful" than the Rust borrow checker, right? So, is this an example where Miri does not match what Rust allows or is this a bug in Rust that could lead to undefined behaviour in safe code? ()
-- I believe is admitting that example as something that should be allowed. But this issue with involves raw pointers, which rustc doesn't check at all, so miri is in uncharted waters here.
This is up for us all to decide. I can't tell you. ;) Not necessary. I have argued that / are problematic long before Stacked Borrows. The lifetimes there are, strictly speaking, wrong. I think it would be reasonable to rule that they have to use raw pointers.
FWIW here is a stand-alone version of the testcase to reproduce this with Miri (but it needs access to some private internals):
See for the issue about whether Stacked Borrows should allow such code or not. For now, with , Miri accepts this code again. So I am going to close the issue here. It still represents an interesting data point to inform what Stacked Borrows rules should be.
Code change:
<ins> use core::fmt; </ins> use core::iter::FusedIterator; <ins> use core::marker::PhantomData; use core::mem::{self, MaybeUninit}; </ins> use core::ptr::{self, NonNull}; <del> use core::{fmt, mem}; </del> use crate::alloc::{Allocator, Global}; <del> use super::{count, Iter, VecDeque}; </del> <ins> use super::{count, wrap_index, VecDeque}; </ins> /// A draining iterator over the elements of a `VecDeque`. ///

Related discussion:
text: nods wisely just gotta stand still long enough for the spec to bend around you
;-) I don't think the decision is made either way, but for now with several counterexamples in libstd, it is not helpful for Miri to complain about this. Plus, this matches what we formalized in Coq; the basic optimizations that we considered so far are all fine with this change.
See there actually is a bug here right now, because we do emit for passed to a function. So what Miri found is actually UB in the LLVM IR that we are generating right now. Reopening for now, but maybe it makes more sense to treat this as a duplicate of
Smaller reproduction that I found in (a now closed dupe of this issue) that doesn't rely on VecDeque internals, that fails now with
re-enables the part of Stacked Borrows that got disabled (in parts) to work around the issue, before I knew that we actually have here and are risking real miscompilations.
claim
Code change:
use std::collections::BTreeMap; use std::env; <del> use std::fs; </del> <ins> use std::fs::{self, File}; </ins> use std::io::{self, Read, Write}; use std::path::{PathBuf, Path}; use std::process::{Command, Stdio}; <ins> use std::collections::HashMap; </ins> static HOSTS: &[&str] = &[ "aarch64-unknown-linux-gnu",

Related discussion:
text: Looking at https://rust-, it seems like Miri is getting shipped as a rustup component even though the tests fail. Unfortunately, Miri is currently entirely usable, so it'd probably be better if it did not get shipped in this state. Is that something we can do? Cc
maybe we can make the miri toolstate always go to even if it technically is just
what would be the best place to change this? I think I'd prefer Miri not to be shipped at all when the tests fail; I'd rather people stick to an old-but-working Miri than to a new-but-broken one. I found but that does not seem to decide if a component gets built for distribution or not.
seems to suggest that it is supposed to work as you describe that it should be working. Since it obviously doesn't and I can't see how that comment could be true from looking at the source, I'm guessing this is some refactoring fallout. I'll dig a little into git blame
I found it. The comment was for rls in but then the code making the comment true was removed later
See for why this is hard to fix: the builders don't even run the tool tests, so they cannot know if the tests pass. :/ Also, that's probably for the better actually, as the "tools" builder sets --cfg miri to be more Miri-friendly and we don't want that in the release artifacts.
One (crazy?) thing we could so is have the dist builders query the toolstate repo to figure out if the tests are passing...
That will either retrieve an outdated toolstate or will need to block until the toolstate is uploaded ...
Without blocking it would lag behind by 1 merged PR, yes.
would it be possible to add some kind of build assert so Miri fails on builders when Rust internals change too much?
Well builders already know when Miri fails to build because internals change too much. The issue is things that make Miri still compile but break the test suite.
would it be possible to at least change the logic that opens issues such that for Miri, it opens an issue like when the tests fail? I could also try to implement that if you point me to the right place. So far we have had zero cases of Miri tests spuriously failing.
Code change:
use std::collections::BTreeMap; use std::env; <del> use std::fs; </del> <ins> use std::fs::{self, File}; </ins> use std::io::{self, Read, Write}; use std::path::{PathBuf, Path}; use std::process::{Command, Stdio}; <ins> use std::collections::HashMap; </ins> static HOSTS: &[&str] = &[ "aarch64-unknown-linux-gnu",

Related discussion:
text: That should be easy to do:
Gave it a shot at
had an interesting idea: can't we do the "don't ship broken tools" as part of a post-processing step? We anyway do some little amount of work when a commit hits (as opposed to ), such as actually updating the toolstate repo. So maybe that could also either delete the broken components from the S3 bucket, or else generate a file to be put into the S3 bucket that says which of the components are really there so that the broken ones can be considered "not there"? do you think something like that would be possible?
Thinking about this again, probably the most natural place to do this is in the . That could "just" also check the toolstate database for the commit it is promoting, and then (possibly only for certain tools) not add them to the nightly if their toolstate is not green.
This is biting us again right now, where CI knows that Miri is broken but we ship the component anyway, leading to CI failures in downstream projects that have Miri in their CI.
For another case where this issue got quite painful, see
My PR is because I was incorrectly assuming that all commits have their toolstate recorded in the toolstate repo. Reopening.
FTR, for testing , I downloaded some artifacts: then I edited my : and finally I ran This produces stuff in .
So looks like we have logic in that specifically does not commit to the toolstate repo if nothing changed: Unfortunately, this means that we cannot easily fetch the toolstate of the to-be-produced release in build-manifest, which caused my to fail. Would it be okay for you to change this logic to record the toolstate of all commits in the toolstate repo? Or would you prefer if I try to find another approach for solving this issue? Right now I am somewhat out of ideas though if we cannot even use the toolstate repo to communicate this information.
Wouldn't it be simpler to save list of "good" components to something like and upload it alongside tarballs? Running would show what you need.
I guess we could upload the two JSON files from the two tool runners somehow? Not sure which would be the right places to hook for that, though.
Status: blocked on which is blocked on
This seems to be finally fixed!
Code change:
use std::collections::BTreeMap; use std::env; <del> use std::fs; </del> <ins> use std::fs::{self, File}; </ins> use std::io::{self, Read, Write}; use std::path::{PathBuf, Path}; use std::process::{Command, Stdio}; <ins> use std::collections::HashMap; </ins> static HOSTS: &[&str] = &[ "aarch64-unknown-linux-gnu",

Related discussion:
text: Today nightly was shipped with , and everything looks normal :tada:
Miri is "test-fail" now, so with the next nightly we should be able to see if indeed it gets removed from rustup as it should.
Confirmed it's working! On miri is missing.
Code change:
pub mod fun_treemap; pub mod list; pub mod priority_queue; <del> pub mod rope; </del> pub mod smallintmap; pub mod sort;

Related discussion:
text: In commit I had to disable one of the rope tests. The purpose of that commit was to fix up a bug in the way operator [] works on strings. In particular it used to let you index up to and including the trailing null; now it considers access to the null byte an index overrun and fails. I tried to figure out exactly what was going on in the rope code for quite a while, and tried adjusting a number of obvious-looking things, but to no avail. I'm not really good with boundary-case arithmetic (are any of us?) and the rope module is a maze of such arithmetic. Sharper minds requested to find the bug. Meanwhile I marked the rope test in question () as ignored. The others seem to work.
Paging
I'll take a look.
There is some discussion on IRC that maybe we should just pitch the rope code as it exists now.
agreed. The code is pretty outdated and the work required to modernize it would probably be similar to the work needed to just re-write it. When I was looking through it, it seemed to be doing some very strange things that I don't think are even close to being safe, so there's that too.
Code change:
[`std::env`]: https://doc.rust-lang.org/stable/std/env/index.html#functions [strikethrough]: https://github.github.com/gfm/#strikethrough-extension- [tables]: https://github.github.com/gfm/#tables-extension- <ins> [task list extension]: https://github.github.com/gfm/#task-list-items-extension- </ins>

Related discussion:
text: These include: The reasoning for all of these extensions is that they all have wide use not just in GitHub, but in many applications of Markdown.
I'm not sure if automatically creating links from bare domain names is a good idea, so if there's an option to disable that specifically I'd suggest it. The rest all seems fine to me.
Just like said for Autolinks. I'm also not sure to see the point of the task list one. However, adding strikethrough would be nice.
Tasklist would be useful for things like this: [x] Implemented thing [x] Some other implemented thing [ ] Unimplemented thing [ ] Some other unimplemented thing or possibly in the crate root [x] Webhooks [x] Websockets [ ] Websites [ ] Other
Another noteworthy extension to implement is . This would prevent the following tags from being used for safety reasons: - - - - -
We decided in not to add this. Instead rustdoc will now suggest to turn raw links into automatic links (). So I think the only thing left is adding task list parsing.
Code change:
} else { // `sret` parameter thus one less integer register available arg.make_indirect(); <ins> // NOTE(eddyb) return is handled first, so no registers // should've been used yet. assert_eq!(int_regs, MAX_INT_REGS); </ins> int_regs -= 1; } } Ok(ref cls) => { // split into sized chunks passed individually <del> int_regs -= needed_int; sse_regs -= needed_sse; </del> if arg.layout.is_aggregate() { let size = arg.layout.size; arg.cast_to(cast_target(cls, size))

Related discussion:
text: We encountered this with WebRender, where we exposed a method in Rust to the Gecko, and the last parameters supplied appeared random on the Rust side. I made a reduced test case adapted from it that you can grab at: It is a very simple test case where the C++ program puts all the parameters on the stack, prints the expected values of some of them, and calls the Rust method. The Rust method prints the values it received. We have enough parameters to fill up the registers used in the calling convention, and once we get to pushing things on the stack, it goes wrong. You can see from the output that they mismatch. ./build- && ./run- outputs c++ expect width=7 height=8 c++ expect slice=9 10 11 12 c++ expect outset=13. 13. 13. 13. c++ expect repeat h=0 v=0 got rust image 6 width 7 height 8 got rust slice SideOffsets { top: 9, right: 10, bottom: 11, left: 12 } got rust outset LayoutGeometry { top: 0.0, left: 0.0, width: 0.0, height: 0.0 } got rust repeat h=26 v=58 Note that the earlier parameters are correct, but the last 3 (outset, repeat h and repeat v) are all incorrect.
Is there an upstream issue in WebRender you can link? What platform is this happening on?
The Gecko side integrating with WebRender was disabled, and I have been working to re-enable this in You can find the code in question at: I intend to just work around this using structs for the moment as part of the aforementioned bug.
We're seeing this on all of our x64 platforms (win, mac, linux; all using clang). It looks like a pretty clear bug in our sysv callee ABI for x64 when there are lots of arguments.
<details<summaryrustc callee llvm-ir </summary</details<details<summaryclang caller llvm-ir </summary</detailsJust the signatures:
Note to self: can go negative from scalar integer arguments after the 6 integer registers have been exhausted and then is always true. Will try to fix soon (also I wonder if this is a stable-to-stable regression from years ago?).
Code change:
} else { // `sret` parameter thus one less integer register available arg.make_indirect(); <ins> // NOTE(eddyb) return is handled first, so no registers // should've been used yet. assert_eq!(int_regs, MAX_INT_REGS); </ins> int_regs -= 1; } } Ok(ref cls) => { // split into sized chunks passed individually <del> int_regs -= needed_int; sse_regs -= needed_sse; </del> if arg.layout.is_aggregate() { let size = arg.layout.size; arg.cast_to(cast_target(cls, size))

Related discussion:
text: Looks like this was working circa 1.0, but has been broken at least since 1.12. Oof.
Do you need this fixed on beta as well, btw?
it would certainly be great if you uplifted this, but we have to work around it for the next few weeks anyway
Code change:
Err(CharTryFromError(())) } else { // SAFETY: checked that it's a legal unicode value <del> Ok(unsafe { from_u32_unchecked(i) }) </del> <ins> Ok(unsafe { transmute(i) }) </ins> } } }

Related discussion:
text: The following program causes UB: Miri says: The problem is this code: This calls unless the new code point is in , but what about surrogates in ? This code is unchanged since its introduction in I am not sure what the intended safety contract of is. That method is not marked but clearly should be -- it calls . So my guess is the safety precondition is that must not be part of a surrogate pair, but the thing is, calls it without actually ensuring that condition. The condition it does ensure is that the codepoint is not in , but that does not help.
is excluded from Unicode scalar values a.k.a. , but it’s fine as a Unicode code point. I think this bug is not in the contract of (which is private anyway) but in its implementation, namely going through . The fix is to either duplicate the logic of into , or (to avoid duplication) move that logic to a new function in libcore that takes a parameter and that is called by both and . This function would need to be public because is in a different crate, but it should be prema-unstable and .
I would classify the priority of this bug as low. Although this call to is UB, I expect Miri checking for it explicitly is the only case where that has any consequence in today’s implementation: behaves as expected (for the purpose of WTF-8) in the surrogate range and does not exploit this UB The "layout" of in rustc excludes values beyond but not surrogates:
By the way, the docs at feel insufficient: Which values are invalid? Similarly, and say that represents Unicode Scalar values, but I couldn’t find it documented anywhere that constructing an out of range is Undefined Behavior rather than "merely" a logic bug. Is this specified elsewhere? What is the closest we have to a written down normative resource that specifies what is or isn’t UB in the the Rust language? (For APIs I assume this is the responsibility of their respective doc-comments.)
I agree the impact is low, but it's not just Miri -- this blocks , which is how I discovered the problem. That would be
But then what is "unchecked" about it? Elsewhere in that file the code is careful not to construct a from a without checking for surrogates: That made me assume the author was aware that surrogates in are UB.
Code change:
Err(CharTryFromError(())) } else { // SAFETY: checked that it's a legal unicode value <del> Ok(unsafe { from_u32_unchecked(i) }) </del> <ins> Ok(unsafe { transmute(i) }) </ins> } } }

Related discussion:
text: The type contains a (generalized-UTF-8-encoded) sequence of code point (including potentially surrogates) that doesn’t form a surrogate pair. doesn’t check for surrogate pairs. is a public method that is documented as returning a scalar value, therefore excluding all surrogates. Ok I looked into it, there are three different people involved. The original code in 2014 was YOLO Later in 2014, moved to duplicating logic instead. In 2015, PR which first imported WTF-8 support in libstd deduplicated that logic by adding a function that takes . (I knew that approach sounded familiar…) In 2016, PR changed the signature of and in passing removed . It made use + instead, presumably in the middle of updating many callers other.
Okay, so the fix would be to revert the part of that removed ?
That or copy into . I find them both not great, so meh.
Even with that done, there's still UB in another testcase: This is converting something to a before calling -- so probably the same issue as the above, just for a different encoding.
Code change:
} fn run_ui_cargo(config: &mut compiletest::Config) { <ins> if cargo::is_rustc_test_suite() { return; } </ins> fn run_tests( config: &compiletest::Config, filter: &Option<String>,

Related discussion:
text: The following sample crashed rustc: Removing the , or unwrapping the (leaving only ), or replacing the predicate (leaving ), solves the crash. Reproduced on Windows 64-bit, Fedora 32-bit, Ubuntu 64-bit. Full crash log (thanks to ) @ Short crash message:
Fixed pending
Fixed in