jiebi/CodeConvo huggingface dataset:
The common fix for , , and is to rewrite the module based on the spec at and the tests at (Note: the tests assume the API, designed for JavaScript in web browsers. We do not necessarily want the same API for Rust outside of testing code.)
I'm interested in this
I have a work-in-progress implementation. I’ll publish it later today so you can pick it up if you want.
Here it is: Note that I’m targeting Rust , the version used in Servo. (Though there is an upgrade being worked on.) IDNA and the actual URL parser are not there yet, but everything else (host/domain/ipv6, , percent encoding, test harness) is there. This should start being useful as soon as the URL parser is . The host parser returns an error on non-ASCII domains. This is good enough for now, IDNA support can be later. also made an in-progress implementation: It has some known bugs ( is incorrect with non-ASCII input, at least) but the parser code can probably be reused.
Also, be aware that the URL spec itself is still moving and has a number of open bugs:
I Punycode encoding and decoding (with tests) to but not Nameprep, which is the other big part of IDNA.
is now (IMHO) ready to replace . It is not "done" yet (some bugs remain, including in the spec!), but at this point it has more than feature-parity and is probably correct in more corner cases. Given the current tendency to move things out of , rust-url probably shouldn’t move in. It could be distributed with rustc like the rest of the modules moving out of extra, but this has the downside of tying rust-url upgrades to compiler upgrades, and only weak advantages IMO. So, whenever the Rust team decides it’s appropriate, I suggest removing and recommending rust-url as a replacement.
I would generally be in favor of a in the distribution for now. This would certainly not be a package in long-term, but for now putting everything in this repo is the ad-hoc solution we have until we get a real package manager.
In that case, I’d like to iterate a bit on rust-url (port rust-http and Servo to it, …) before it moves into the distribution. Also, rust-url depends on rust-encoding, which I guess would have to move as well.
I’d say rust-url is now in "beta" state. I have branches of rust-http and Servo using it: It builds with Cargo. It depends on rust-encoding (which also builds with Cargo), but for something non-essential that could be deactivated if avoiding the additional dependency helps. I’d like people to start using it and give feedback. I think the best way to do that is to get rid of the old liburl. I’ll go through the RFC process, but before that, what would you prefer? Replace liburl with snapshots[] of rust-url. You were in favor of this in February, but Cargo has reached alpha since then. Just remove liburl, and tell people to use rust-url through Cargo. Something else? [] In any case, I’d like to keep a separate rust-url repository that I can update in Servo independently of language upgrades.
+1 on removing liburl. Thanks
Perhaps we can deprecate all of liburl for one release cycle.
This seems like a great place to let cargo shine.
Deprecating sounds reasonable. That means emitting warnings by default, right?
awesome work! I agree with and that I think this is where cargo shines, and we can probably just remove liburl entirely without re-adding rust-url. We can also try to grow more official support for it in terms of documentation and automation (not present currently). We can mark liburl as with a message to use instead which should ferry everyone along quickly.
<!-- Thank you for finding an Internal Compiler Error! If possible, try to provide a minimal verifiable example. You can read "Rust Bug Minimization Patterns" for how to create smaller examples. --Github: Repro: Reproduces in: Stable Linux - rustc 1.61.0 ( 2022-05-18) Nightly Linux - rustc 1.63.0-nightly ( 2022-06-02) <!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summary<strongBacktrace</strong</summary<p</p</details
I'll see if I can make a smaller repro case
So, the minimal repro case is MUCH smaller: in : In : Which points to these code chunks: This seems to work in the tests for , so it seems like the ICE only triggers when used from an external trait. Additionally, this ICE does NOT trigger in .
CC
Yeah, this seems to occur specifically when trying to compile an external crate that calls this function. Lib tests inside the crate that call the function don't ICE, but adding an integration test (outside of the crate's compilation unit) does trigger the ICE:
If the function is made , the ICE no longer occurs. For example, this commit does not ICE:
prioritize
, if the functions are made but placed in a private module, they get culled again, triggering the ICE:
It also appears that if you move the functions out of the , AND mark them as , the ICE is also abated. See this PR for addressing:
For folks joining now/as a TL;DR, the offending code is basically this: The function produces a value, that gets created in a static: Which gets loaded into a static in user code: It LOOKS like somehow the two original functions (declared inside a private, const fn; inside a private type), are getting culled, which means that when the static (in another compilation unit) goes to be created, the MIR for those functions are missing.
This is likely a disagreement between . Though I don't know much about codegen.
Anyways I think I have a fix, though I am not certain it's the right one. In any case, I at least have a minimized UI test for rustc devs who know codegen better than me. PR incoming.
Found with The attribute applies to The was initially lexed as part of a token This seems to be exactly the case described by the : <details<summary<strongBacktrace</strong</summary<p</p</details<!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOTEND --
claim
Do you have any suggestion for this case? The code path is going like this: Parser eat the first one of '' at here and set : fall pass at here: So this assertion is failed: The result is: and is true. Seems the assertion missed this scenario, do we need to add a TrailingToken::Gt ?
The assertion I wrote is overly conservative - it really just needs to check that we don't simultaneously break a token and 'really' capture a trailing token. Specifically, we only need to panic if we actually incremented in the match block. I think the simplest fix would be to add a variable, which gets set to in the parts of the block that increment . The assertion can then be changed to breaklasttoken
<!-- Thank you for finding an Internal Compiler Error! If possible, try to provide a minimal verifiable example. You can read "Rust Bug Minimization Patterns" for how to create smaller examples. --<!-- If you're using the stable version of the compiler, you should also check if the bug also exists in the beta or nightly versions. --: <!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summary<strongBacktrace</strong</summary<p</p</details<!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOTEND --
cc is this fixed by one of your PRs? This looked like it might be , but while was bisected to this one was bisected to , not sure what to make of it :thinking:
Yes, this issue is fixed in (ice → errors): <details<summarystderr</summary</detailsI'm not sure why the bisection points differ, they all share the same root cause. It doesn't really matter though. Btw, I saw about another new reproducer but I dismissed it as irrelevant since I assumed it required , too (I did run it but didn't notice I could omit that flag). Sorry about that. Thanks for opening a separate issue because now we actually have 2 beautiful regression tests for the bug fixed in ! So let's not close this issue then but wait for to get merged and then I will open a separate PR with the regression tests & assign you. claim
Obviously a corner case. This would never actually be used for anything, this is just testing the bounds of what the compiler is capable of. Running this Ruby method with numarguments or greater generates a Rust file which can be used to reproduce the output. (via ) : </p</details<!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOTEND --
claim
label +regression-from-stable-to-beta
Nothing that Ruby code can achieve that Rust macros couldn't do as well: ()
Haha, just tried out a different direction of “many arguments”. Apparently this results in a segfault, AFAICT It’s explicitly a stack overflow on stable (and AFAIK segfaults can happen on stack overflow in sound rust code, so maybe it isn’t some memory unsafety). <hrEdit: Opened this as a separate issue
Assigning priority as discussed in the of the Prioritization Working Group. label -I-prioritize +P-low
<!-- Thank you for creating a tracking issue! Tracking issues are for tracking a feature from implementation to stabilization. Make sure to include the relevant RFC for the feature if it has one. If the new feature is small, it may be fine to skip the RFC process. In that case, you can use use in your initial implementation PR. The reviewer will ask you to open a tracking issue if they agree your feature can be without an RFC. --Feature gate: This is a tracking issue for (and friends for the other unsigned types). Like (), this generates simpler code than the underlying integer type because it can avoid a separate zero check. <!-- Include a short description of the feature. --<!-- For most library features, it'd be useful to include a summarized version of the public API. (E.g. just the public function signatures without their doc comments or implementation.) --<!-- For larger features, more steps might be involved. If the feature is changed later, please add those PRs here as well. --- [x] Implementation: [ ] Final commenting period (FCP) [ ] Stabilization PR <!-- Once the feature has gone through a few release cycles and there are no unresolved questions left, the feature might be ready for stabilization. If this feature didn't go through the RFC process, a final commenting period (FCP) is always needed before stabilization. This works as follows: A library team member can kick off the stabilization process, at which point the rfcbot will ask all the team members to verify they agree with stabilization. Once enough members agree and there are no concerns, the final commenting period begins: this issue will be marked as such and will be listed in the next This Week in Rust newsletter. If no blocking concerns are raised in that period of 10 days, a stabilzation PR can be opened by anyone. --<!-- Include any open questions that need to be answered before the feature can be stabilised. If multiple (unrelated) big questions come up, it can be a good idea to open a separate issue for each, to make it easier to keep track of the discussions. It's useful to link any relevant discussions and conclusions (whether on GitHub, Zulip, or the internals forum) here. --- None yet.
Great, something else that just breaks my code. Sigh.
With , I would like to propose  for stabilization as well. (I think this in the tracking issue is the right thing to do now, rather than filing a PR?)
shall we stabilize ? merge
Team member has proposed to merge this. The next step is review by the rest of the tagged team members: [x] [x] [x] [x] [x] [ ] No concerns currently listed. Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See for info about what commands tagged team members can give me.
:bell: This is now entering its final comment period, as per the . :bell:
The final comment period, with a disposition to merge, as per the , is now complete. As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed. This will be merged soon.
we should change the following function to use a instead of Using a walker causes the control-flow to be a lot more complicated and given that this code is soundness-critical, we should make sure it's as easy to understand as possible <!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOTEND --
claim
This enhancement suggestion looks a bit silly to me, but I think it's worth asking for an opinion in this bug tracker. I think this is a reasonable usage pattern of and . I've used it with a larger SomeData to avoid both an useless initialization and the use of that in those cases is slightly slower, the disadvantage is that this code contains and it's a bit fiddly: An API that removes the unsafety could be: is similar to , but it doesn't include any growing strategy (and doesn't include such growing/realloc code in the binary), so it just uses the capacity. If the capacity isn't enough it panics (just like the first code panics if the capacity is finished at ).
label C-enhancement
label -C-enhancement C-feature-request
It might be better to return a instead to let the user handle the failure case.
An extra note. Currently on this code: I think LLVM12 isn't able to see that code will never realloc and doesn't need a bound test:
The naive implementation of already optimizes out the reallocation case with . Since (parts of) the reallocation code is inlined into , this would give smaller code size (). Here is a use case that I encounter fairly frequently: I want to construct a , so I preallocate one and push elements into it. However, if there is an off-by-one error in the preallocation, then it would reallocate every time. Having a function (followed by unwrap) would help catch these errors.
I suggest creating a fixed-capacity type wrapper instead, so that it can also be used with and many more methods without realloc code bloat.
I suggest creating a fixed-capacity Vec type wrapper instead, so that it can also be used with Extend and many more methods without realloc code bloat. A question: such wrapper doesn't cause bloat itself?
No, I don't think it'd be much different from deref that gives you the as a slice.
What do you envision to do when the vector runs out of space?
Panic. I also hope that the wrapper type with a fixed-size storage would be easier for optimizer to reason about (similarly how taking a slice of a vec before a loop helps remove bounds checks in the loop). If the optimizer is able to remove capacity checks, then failure handling code doesn't matter, because it will be optimized out too.
Interestingly, it can't be as simply implemented outside of , because the field can't be directly borrowed, but if you need to use reference and , then you can't have the slice at the same time, because it'd violate mutable aliasing rules. If necessary, the capacity error behavior could be configurable via a type parameter, e.g. or (that silently ignores excess elements). The extra argument would allow compile-time specialized code (so no extra runtime check), and inflexibility of a type argument wouldn't get in the way, because it's a temporary object that you'd create on demand, so you can create a different one for each use.
why not add unchecked versions of functions with capacity check (, , , , , ...)?
Error: That's this: The GVN MIR diff is this: searched nightlies: from nightly-2023-06-01 to nightly-2023-11-08 regressed nightly: nightly-2023-10-30 searched commit range: regressed commit: <details<summarybisected with <a href=''cargo-bisect-rustc</av0.6.7</summaryHost triple: x86_64-unknown-linux-gnu Reproduce with: </details
This issue is a bit old but I've now minimized the reproducer, it's in the description. the assertion that's failing here looks to me like it is simply too picky, and it should be checking if the alignment of the allocation is the alignment of the type we're reading from it, instead of exactly equal. Do you agree that this is just a too-picky assertion?
I agree the assertion is just too picky. I expect the subtlety of needing to pass the larger alignment for the llvm constant.
I can't figure out what you mean here. What LLVM constant? Why is there subtelty? Maybe I just don't understand what's going on with GVN here, I'm assuming that GVN has merged two constants together so that we're now loading part of a more-aligned constant to get a less-aligned one. Is that wrong?
I mean, this is codegen code. It creates a constant for llvm. That constant has some alignment. We need to pick the right one, between the lhs and rhs of the assert.
Where are we picking an alignment? Is it down here?
smaller repro: <!-- ICE: Rustc '-Zmir-opt-level=5 -Cdebuginfo=2 -ooutputfile -Zdump-mir-dir=dir' 'thread 'rustc' panicked at 'assertion failed'', 'thread 'rustc' panicked at 'assertion failed'' File: --auto-reduced (treereduce-rust): &lt;details&lt;summary&lt;strongoriginal code&lt;/strong&lt;/summary&lt;poriginal: </p</detailsVersion information ` Command: <!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summary<strongProgram output</strong</summary<p</p</details<!-- query stack: --
I don't really know.
<!-- Thank you for finding an Internal Compiler Error! If possible, try to provide a minimal verifiable example. You can read "Rust Bug Minimization Patterns" for how to create smaller examples. --<!-- If you're using the stable version of the compiler, you should also check if the bug also exists in the beta or nightly versions. --: <!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summary<strongBacktrace</strong</summary<p</p</details
glacier ""
Error: Parsing glacier command in failed: ...'cier https' ''... Please let know if you're having trouble with this bot.
Compiler error on but ICE on label +regression-from-stable-to-stable
cc
A previous (now-fixed) ICE with the same error message is
Thanks for the report! Would be good to get a better backtrace; this one seems to be missing many frames and there are no line numbers.
, I updated the backtrace in my first post. I used rustup to get the latest trunk version, and the one I have been using does not have line numbers. If there is a flag or configuration for rustup to get rustc binaries with line numbers, I would be happy to change the settings on my testing machine.
I think getting line numbers requires a debug build of the compiler, so it's not quite as easy as installing something via rustup.
Labeling .
Here's a backtrace with debug info: <details</details
Unfortunately, this bottoms out ; more digging will be needed to determine the true underlying cause of the error.
Here is the actual backtrace of the underlying CTFE/Miri error: <details</detailsIt points to this line: can handle pointer values just fine, so I think what is happening is that we are reading parts of a pointer value here, which indeed is not supported in CTFE.
So... I guess the fix is to make this kind of error also be permitted during validation and turn it into a proper validation error?
Permitting it at the top-level is no good, since then we don't know "where in the value" it arose. We need a version of that catches this error and turns it into a validation error with the current "path".
Oh... right, we should fix this in general, and not just wrap in .
Maybe we should not have any automatic way to bubble up errors and require all sites to use ?
I think we should wrap it but put the wrapped thing into a helper function sow e can also use it for the other places. So basically, you want to use a different error type for validation, one that excludes all the other errors? I am open to refactorings here.
Let's start out with wrapping, and then I can experiment with the refactoring so we can have a look at the usability effect of it Instructions: wrap the call at in a macro invocation and capture the error. Emit a validation message about the fact that reading parts of a pointer is not possible during CTFE. Don't forget to add a test (the failing example from this issue works great as a test) and run the test with
There are quite a few calls in that will all have this problem, so ideally they should all be treated this way. To avoid code duplication, it will probably make sense to move this into a helper function.
Reopening this to track that is still called unguarded a few times in , so e.g. the issue can still arise with .
At line 393, the guard object's field is incremented by the value returned from a read implementation. If a questionable returns a value larger than the buffer size, it will take that value and set the length of the vector over the boundary. This bug is reachable from and . Here is a that demonstrates the bug. It segfaults with .
This is definitely broken - nice find. A simple fix would be to just add an in the drop impl: You can still get weird behavior in readtoendwithreservation, but it should be memory safe and it seems best to avoid a bunch of extra logic in the main loop.
It looks like the bug was introduced in
Assigning as and removing .
Since the function is tagged the exact name presumably has to exactly match something that another system is expecting, which may not be snake case. For example, JNI bindings will look something like
Modifying lints requires an RFC now, as it can impact a LOT of code.
(By modifying I mean "major changes" not bugfixes)
Hm, while I think new lints definitely want an RFC, modifications to existing ones can probably go through the lang team's nomination process, leading to either an immediate decision or request for RFC.
I'm fine with that, just following (what I thought was) procedure. :smile:
<!-- Thank you for finding an Internal Compiler Error! If possible, try to provide a minimal verifiable example. You can read "Rust Bug Minimization Patterns" for how to create smaller examples. --<!-- If you're using the stable version of the compiler, you should also check if the bug also exists in the beta or nightly versions. --: <!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summary<strongBacktrace</strong</summary<p</p</details
searched nightlies: from nightly-2023-02-17 to nightly-2023-02-19 regressed nightly: nightly-2023-02-18 searched commit range: regressed commit: - -cc <details<summarybisected with <a href=''cargo-bisect-rustc</av0.6.6</summaryHost triple: x86_64-unknown-linux-gnu Reproduce with: </details
Right.. is my own PR; I'm noting this as an issue, mainly so I won't forget this, because I won't do this myself right now / today. Or someone else can do this if they want. I'm not 100% certain where the best place for such a test to go is, and/or whether it makes sense to have them in one place or separated and more close to the respective type's definition. A test to prevent any future regression removing the new auto trait implementations introduced by can look about as follows <s` </sEdit: This code is needlessly complicated, see my comment below. label C-enhancement, A-testsuite, E-easy, E-needs-test, T-libs <!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOTEND --
claim
FYI, somehow I missed the more straightforward way of writing these.
<!-- [31mICE[0m: Rustc '' 'thread 'rustc' panicked at 'assertion failed: expect 1 RPITITs in call to '', 'thread 'rustc' panicked at 'assertion failed: expect 1 RPITITs in call to '' File: --auto-reduced (treereduce-rust): original: Version information ` Command: <!-- query stack: [collectreturnpositionimpltraitintraittys] comparing an impl and trait method signature, inferring any hidden types in the process [typeof] computing type of [checkmodimplwf] checking that impls are well-formed in top-level module [analysis] running analysis passes on this crate --
<!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summary<strongProgram output</strong</summary<p</p</details
fyi