jiebi/CodeConvo huggingface dataset:
They should be: rustc 1.24.0-nightly ( 2017-12-20) binary: rustc commit-hash: commit-date: 2017-12-20 host: x8664-pc-windows-gnu release: 1.24.0-nightly LLVM version: 4.0 rustc 1.24.0-nightly ( 2017-12-21) binary: rustc commit-hash: commit-date: 2017-12-21 host: x8664-pc-windows-gnu release: 1.24.0-nightly LLVM version: 4.0 Mixing up things seems quite easy here.
Ah, that's why I asked, so the range is .... Still not seeing anything clearly relevant, but maybe someone else from does?
The --emit=metadata times are computed as the minimum of four tries. Their variance is very small (on the order of +-0.05) so I think they are reliable. The binary run-time is the minimum of 5 runs, their variance is higher perhaps because they do some I/O, despite using a SSD, about +-0.12. The compilation is slow so there I've done only one time, so those values are the less reliable, but differences above 1.5 seconds should be reliable.
For the compilation time issue, maybe there was a bug in NLL that made it active without it being enabled. For the run-time issue, we need to actually investigate this.
As in, investigate the cause of the run-time fluctuations and how it unregressed?
This is my good old friend the torpedo. LLVM optimizes into this: This is basically equivalent to this pseudo-Rust code After the "non-constant-address" write is generated, LLVM is sunk - LLVM has no way of getting rid of the non constant address, and no way of optimizing accesses to the target with that non-constant address.
Upstream LLVM (that's the yet-to-be-released LLVM 6) has this patch, which avoids the SimplifyCfg sink problem by not running sinking before inlining I have confirmed that - e.g. this code is optimized correctly with the new Until then, I think representing small enums using a struct (rather than an array) should work around this problem.
With the 2017-12-22 Nightly all the problems are back: Rustc version --emit=metadata Opt build Binary size Run-time 1.24.0-nightly 2017-12-22 5.36 53.57 13.96
Coud you open a PR that turns small unions into LLVM structs (rather than arrays)? That should fix the problem.
The binaries generated by the last Nightly (1.24.0-nightly 2017-12-25) is further few percents slower (perhaps because of thinLTO?), the compilation times can't be compared now.
Is there more stable measure besides compile time / run time? They can be affected too easily by the environment especially when the run time is expected to take less than 1 second. The generated assembly size seems to be a good measure.
Please re-test next nightly and reopen if not fixed.
The run-time of the binary is now OK. The compilation time with --emit=metadata is still quite large.
Can you open a separate one for compilation times? Seems to be a separate issue.
, I've tried here:
I was looking at the generated asm for to see what happens to the array, and was surprised to see a large function with 5(!) call sites for . The generated code doesn't seem to vary with the number of elements in the array. That got me to compare with the functionally equivalent code: which doesn't yield the same assembly at all. The first difference being that the latter doesn't even store the array (only the strs) while the former does. That got me even further in comparing with equivalent loops, up to the extreme (and stupid): vs. The latter generates the simplest code possible: The former not so much:
Documenting my thinking in the hopes that it will be useful, or at least that someone will correct me via Campbell's Law. We know that the ASM outputs are different, so let's go back a few steps until we can find where in the toolchain we want to add our improvement. I compiled this on 1.19 with , but the important thing to note is that while iterany is quite bloated, it looks like the for loop has already been optimized: So is just as trivial in LLVM bytecode as it is in assembler - whatever optimizations we needed to do have already happened. We need to go back farther. . Notice that the tables have turned! is more concise than , so the code we need to add should change the MIR output. Note that . So the problem here seems to be that we aren't inlining before we generate LLVM bytecode. But that answer doesn't satisfy me either, because neither the nor the has any calls to , and in fact . So... one of these things is happening: 're failing to inline , and we haven't noticed because LLVM's been inlining this code for us. LLVM doesn't have as much knowledge as we do, so it can't do as good a job as we theoretically could. 've been looking at unoptimized MIR this whole time. This hypothesis seems to be borne out by the fact that when I rerun rustc without optimizations, I get exactly the same MIR output. But , and ? It's possible that both of these hypotheses are correct; maybe this is unoptimized MIR, because the MIR optimizations are doing absolutely nothing to this file!
I'm still cloning rustc, but once I do I'll comment out the MIR optimizations and see where that gets us.
Well, I can oblige that in one aspect :) You put far too much hope into MIR optimizations. They do very little right now&mdash;inlining, for example, isn't even enabled by default (only with ). Even in the future, when all the MIR optimizations we've ever wanted are implemented, they will only cover a small subset of what LLVM does (there's no point in duplicating all the work that went into LLVM, and that would be impractical anyway). This code looks sufficiently messy that I don't expect MIR optimizations to ever fully optimize it away. For example, that would probably require loop unrolling, which hasn't ever been proposed AFAIK (and if it was, I'd be skeptical of it). In any case, the more interesting question is why LLVM can't optimize the iterator version as well as the alternative. Whatever goes awry in optimizing this, it happens in one of LLVM's many passes.
The LLVM IR you posted is very weird. It has multiple branches on constants, and presumably dead code consequently. There's also several branches on values?!? These should all have been cleaned up. After running it through again, I get this IR (names are all messed up cause I did it on , which insists on applying C++ name demangling): <details</detailsThat's... less code, but still very stupid. Why are the adjacent loads and stores not folded away?! A third pass (just at this point) gets rid of them, constant folds the rest, and winds up with . Looks like a nasty phase ordering issue. But what about this IR confuses LLVM so much?
Neat discovery. Should we move this discussion to LLVM's bug tracker and close this task, then?
Not necessarily, no. In the past people have tweaked the definitions of iterator methods to help the optimizer. rustc might also be able to help by changing the pass pipeline or by generating better/different IR in the first place.
This did optimize out in LLVM 3.9, and was broken by LLVM 4.0.
In more detail, LLVM 3.9 generates the following small code for ( metadata stripped - it doesn't matter, with this function substituted in both 3.9 and 4.0 manage to optimize everything out): While LLVM 4.0 manages to screw up and generate this monstrosity: I think the somehow prevents SROA.
Going closer to the root: LLVM 3.9 optimizes the closure in to this: While LLVM 4.0 for some reason generates this:
caused by
The initial nonequivalence in the filing of the bug report is due to the slice iterator's explicit manual unrolling by 4 in its implementation of any. That might seem bad but as soon as llvm learns to loop optimize a loop with two exits (either the element was found or the iterator reached the end), the explicit unrolling can be removed.
is the PR that fixed this intended to be a permanent solution, or is it just a workaround? IOW, should we open a new bug to track reevaluating this fix when LLVM is resolved?
Hmmm, mentioned this bug, but you're right, this is older so me "fixing it" is probably more coincidental than anything else.
That LLVM bug is now tracked at and appears to have had no motion on it in the past 5 years. Assembly in the initial comparison cases remains identical per No other changes, as far as I can tell.
This was nominally resolved on our end, but was left open in the possibility that the LLVM issue was going to be resolved, so as to remove the "temporary" workaround. However, if we wish to track LLVM-side bugs that are not currently an impact on us, but may allow upgrades later, then I believe we may wish to consider tracking them in another form than an issue on the GH tracker, as they are "not an issue", so to speak. Alternatively, if we want to track them on this tracker, we may wish to track them in a more organized fashion (e.g. a metabug). I am going to close this issue. Do feel free to reopen this if it regresses again, or it becomes otherwise currently actionable, or if there is a particular reason we must have an open issue on our end to track this LLVM issue for potential improvements as opposed to any of the other currently-open LLVM issues we could be potentially waiting on in order to make an improvement.
This ensures that someone looking at the documentation who searches for the word "length" to find the method which returns the length of the method will be able to find the method, and also explains the origin of the name if someone is unaware. I bring this up because one of my coworkers came to me asking for the name of the method, because they were having trouble finding it in the documentation.
The problem with this is that it's generally bad to include this in the summary, as it's just repeating what's already been said. So I didn't do this on purpose. I'm not sure what exact text rustdoc searches though.... i'd be against putting this in the summary, but for putting it in the longer description
We seem to be very inconsistent about what summaries to use for methods: Some say "length" and some say "number of elements". Perhaps changing them all to say something like "Returns the number of elements in / length of the slice." would help here.
It doesn't need to be in the summary(?), it can be in the body of the doc. already finds and and so on, so the focus is simply in Ctrl+F search inside the doc page itself (if I understand correctly?).
This is unfortunate because: gives the impression that documentation fixes should not be submitted here, or are not welcome. prevents you from gathering commonly requested information (link to docs, rust version number, suggested fixes) in an automatic fashion. prevents automatic labelling for documentation-relevant issues. To fix this, . <!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOT_END --
label A-meta
claim
Run with this in : and inspect . There is an item for the of the blanket impl  and it's default value is . There is an item but the default value is missing. It looks like this: When using instead of the information about the default value is present. It looks like this: This change, which in practice is a regression, is caused by But we can't revert it of course because that will result in ICEs which is even worse. So we need some kind of follow-up fix. labels +A-rustdoc-json +T-rustdoc +C-bug +requires-nightly
It seems like you already suggested a fix. Want to open a PR for it?
Yep I will do that as soon as I get time. Probably today.
It's not in a hurry. Don't hesitate to ask if you need help.
Hello, this is your friendly neighborhood mergebot. After merging PR rust-lang/rust, I observed that the tool miri has failing tests. A follow-up PR to the repository is needed to fix the fallout. cc do you think you would have time to do the follow-up work? If so, that would be great! cc the PR reviewer, and nominating for compiler team prioritization.
This test is failing:
might give a hint what needs to be updated. I haven't gotten a chance to look closely yet.
Yeah, we'll need to emulate that new static thing by Unfortunately I don't entirely understand the static magic that goes on there... there's just a fn ptr stored in a particular section and somehow that is enough?!?
Yes, all platforms have sections that they treat as an array of pointers, filling the entire section, which makes it easy to append more entries, as linkers will happily concatenate sections with the same name. This is how C++ global constructors are implemented everywhere. So I guess there's two levels to this: miri should collect the platform-specific global ctors/dtors (on Linux I think there's also init/fini which work similarly) and execute them before C main (and after, in the case of dtors) when is , the ctors (or just the init flavor?) can also take 1, 2 or 3 arguments, not only 0 arguments (I believe this is also how C main technically works)
Okay. This is basically I hoped we'd never have to do this... For dtors, we already run TLS dtors, but those are entirely separate (we hook the pthread calls where dtors are set, and then run those functions later). There's no way those can be unified to one mechanism, is there? Also, this begs the question in which order stuff is called before/after main.
C++ asks itself the same question and I personally haven't heard of there being an answer. The safe thing to do is to make sure any order is safe (in Rust you have to initialize global state before any runtime initialization, so it's safer than C++ where before the constructor runs, the fields are uninitialized). The implementations just go through the sections in order, so the unknown quantity is what order the linker concatenates all of these sections.
In miri you don't have the linker to think about, but you do need a mechanisms to access all statics from all crates.
Wait, why can't libstd initialize the args from the start lang item as well? Surely there's an inexpensive way to do so?
That's what I did originally, but said not to.
Reverting should fix this.
We could either adjust the 's to no use the new method when is set, or we could implement those initializers. Seems like we'll want them eventually anyway... What would be the best way to do that?
In the short term, excluding from the new mechanism seems like a reasonable solution to unbreak miri. That depends on two factors: what order functions get placed into the sections, and what order they get called. The default linker scripts will collect all sections in sorted order, followed by the section itself, into a single section. Likewise for and . The sort order here uses the linker's , which takes a numeric section suffix as a numeric priority, and sorts sections with non-numeric suffixes asciibetically. (There are also a couple of special cases to make sure that glibc can guarantee its own functions run first/last in a couple of those. And there are some very careful special cases to handle the old , , , and sections, which get translated into corresponding , , , and sections respectively in the final binary to preserve the behavior of those sections, keeping in mind that ctors sections get run in reverse order and the dtors sections get run in forwards order. translates the priority in numbered ctors and dtors sections by subtracting it from .) When loading a program or library, glibc calls the function pointers in in order, then the (single) function (which executes the contents of the section), then the function pointers in in order. At shutdown, glibc calls the function pointers in in reverse order, then the (single) function (which executes the contents of the section). Does that help?
It is a great reference, thanks! So far I have no idea how to wire up all these sections in Miri, but I guess we'll see about that. ;) On a separate note, also broke Miri by removing the getter from . I am not sure yet if there is a replacement that we can use instead.
Leaving that here just so that I hopefully can find it later.
I don't see any way to actually do that.
I would prefer to avoid duplicate initialization of arguments. I'd like to make Rust executables smaller, not larger.
I agree, I would accept this change.
I opened a PR for that:
Done. I set it to close this issue since we already have to track the more elaborate approach.
Well, actually closing the issue will require also bumping the submodule to include
I don't understand, this would only be a few bytes with ? To unbreak miri it's probably fine but I'm increasingly worried about , as it allows divergence to exist in ways which are not controlled within miri itself.
I was talking about a user of these facilities, not miri implementation. To expand a bit (although it's kind of offtopic): Rust forces you to mutate a that started out with a constant value (e.g. ), which means nothing else would observe an uninitialized value, just the initial constant value, if it happened to ran before the runtime initialization (which would e.g. replace the with a ). This makes Rust + safer in general than C++ with global constructors.
Ack.
Aand now we also need to wait for to land. That's the third independent Miri-breaking PR in 24h, impressive. ;)
<!-- Thank you for finding an Internal Compiler Error! If possible, try to provide a minimal verifiable example. You can read "Rust Bug Minimization Patterns" for how to create smaller examples. --<!-- If you're using the stable version of the compiler, you should also check if the bug also exists in the beta or nightly versions. --: <!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summary<strongBacktrace</strong</summary<p</p</detailsAdding an suffix fixes the issue: The error first appears with the toolchain. It works fine with .
The offset intrinsic does not support doing this, it requires isize or usize. ICEs using intrinsics wrongly are expected and not a bug. Thanks for the report anyways.
Shouldn't a occur for the argument?
There is nothing to be coerced to, since the intrinsics accept any type on the type level.
Is this specific to the intrinsic? I tried a couple of other intrinsics, and the compiler complained about mismatched types: : : Doing the same for results in an ICE, e.g.:
the intrinsic is generic, that's why it passes type checking
It's only generic in the first parameter. And has the exact same signature, but the issue doesn't occur for it.
Hi what is the motivation to keep this check as an ICE instead of a user friendly error since the error arises from misusage of the API?
Giving better errors for intrinsics and other internal-only features increases maintenance burden, so decided to not support it. Invalid usage of intrinsics may lead to ICEs (since intrinsics should not be used - I see for your case here it makes sense to use them in the tests, but I think you can live with ICEs). I guess it's not ideal that the ICE message says to report it - but fixing that is hard.
The Any trait is not in release condition at all, considering that is still uses slow virtual calls that could be replaced with an associated static in the near future but stabilizing with would mean that it could not be removed. Normally, this isn't an issue, but since Any is almost exclusively used through trait objects this blocks possible optimizations to layout that could allow to be represented as instead of containing a full vtable.
It would need to be , where the is the destructor (btw).
From : It's easy, according to implement my own ` implementation for free. In retrospect it makes sense, but it was not obvious to me at all just from reading the docs. It is not mentioned in , or the docs for . And as the link in my post suggests, trying to work my way around the ownership of the APIs led me quite far afield. Admittedly it's not the most common case, but I think it would save others time if the standard library at least mentioned this somewhere. EDIT: fixed forum link which changed its name when I marked it solved.
A section about "you should consider overloading these things" definitely sounds good. should also be on that list. (In the future or in nightly is in that list too, .)
Your forum link is broken.
The docs do say However, I do like 's suggestion, about talking about which methods you may want to override.
Such hints should be non-normative though to allow the implementations, especially specializations, to use different approaches in the future.
Agreed, I was picturing more which methods to consider overloading, not specifically which things use them. (Or, contrapositively, which things there's usually no reason to overload, like , because the default implementation is generally as good as it gets because it's a trivial wrapper.)
So basically this issue is about adding some documentation on which methods you might want to override right? Which methods do you think are best to mention? Or did I misinterpret the discussion?
That's correctly, probably under or near For now my inclination would be to mention and -- once stable it should be instead, but IIRC we don't mention unstable in the broader docs until it stabilizes.
What about adding this line of text at the end of the "Implementing Iterator" section? which call internally. However, it is also possible to write a custom implementation of methods such as and if an iterator is able to compute them more efficiently without calling . Feel free to adjust the text or make something up yourself :) If you agree I will create a PR for this.
I think it would also be nice to add something to the documentation. How about this? Rather than overriding this method directly, instead override the method.
in case it wasn't clear or you are not watching your GitHub notifications, I approve of both your and suggestions. I look forward to the PR!
<!-- Thank you for finding an Internal Compiler Error! If possible, try to provide a minimal verifiable example. You can read "Rust Bug Minimization Patterns" for how to create smaller examples. --I don't have a minimal example yet, but here's . Sample: The ICE happens on the and macros, which are both from the crate. <!-- If you're using the stable version of the compiler, you should also check if the bug also exists in the beta or nightly versions. --This began with commit . <!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summary<strongBacktrace</strong</summary<p</p</details
FYI, I think that link is only accessible to Googlers: !
Sorry, .
Minimized: Caused by
When we run on the AST, we create a definition for the desugared closure body inside an : As a result, the entry for our macro call will have the of the desugared closure. However, this desugared closure does not actually exist in the AST (it is created during lowering), so we will never check for lints with its when we emit buffered early lints.
Nightly has been broken for 3 days, if there's disagreement on how to fix the issue, can we revert while the fix is being worked on?
This will be fixed by PR
I'm still hitting this bug with the latest nightly.
The following function can drop any object arbitrary number of times, using only safe code: Probably bad :) Found while researching
Nominating for P-high
cc
Too bad this doesn't just cancel out the Leakocalypse :stuckouttongue:
(well, any object that implements . But yes, I see the point being made here.)
look, no required. I guess it doesn't work for types that are already unsized.
to be clear, I am looking at this now as well. :)
I guess is related.
I hope it is unrelated. In particular I wrote this on that ticket:
Yeah, it's unrelated. The call happens for , which does need drop.