jiebi/CodeConvo huggingface dataset:
An error message indicating that a full (or at least longer) git history is required. The correct link would include a commit hash inbetween the . But the required commit hash is not available because there is no full history. <!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOTEND --
Hm, can I do this? label +A-rustbuild
Mentoring instructions: find what this command outputs when the repository is shallow: If it's an error, see if you can check for a shallow repository somehow and give a hard error instead of continuing onward just to fail.
Note that this shouldn't need to the full history of the repo, just the past few days, since PRs are constantly being merged.
We should only need one (bors merge) commit, I believe - each bors merge has the full set of artifacts.
The command prints nothing in my shallow clone. There is a bors merge commit in my shallow clone though: The command before used to work for me -- I was able to enable in my shallow clone. I tried to fetch more commits, and then started to work: But still prints nothing.
cc I would have expected the git log and git rev-list to be 'identical', in some sense, so perhaps that's something worth filing upstream as well. But regardless, I am inclined to revert to fix this issue; it was primarily a 'hopefully more robust' solution and to my knowledge wasn't actually fixing any known bugs, so since it caused a bug seems good to revert it for now...
I think it's possible that the shallow clone is causing the predicate to fail. Note the decoration in The git documentation implies that shallow clones use something akin to the (fake parentage for commits) facility for shallow clones. Maybe removing is sufficient? I'm a bit wary because bors seems to create non-merge commits on occasion (for squashes), but maybe it never does so on the first-parent history. I haven't had a chance to poke at a shallow clone in detail yet.
I'm seeing a similar problem for git worktrees, not just shallow clones. EDIT: this may have been a separate reason, it just had the right timing
Would love to see movement on this.
I don't want to allocate ~20GB of laptop space to a deep clone of the repository in order to simply be able to build the compiler. When i was messing around, removing "--merges" as suggested did the trick.
feel free to make a PR with that change :)
I wasn't sure if there was some reason not to do that: I'm a bit wary because bors seems to create non-merge commits on occasion (for squashes), but maybe it never does so on the first-parent history. I don't really grok most of what said there, but if you think that's a safe change, then sure thing!
As long as we keep first parent, dropping merges should be ok, any commit in the first parent chain should have artifacts (if it's recent enough).
I would appreciate knowing details of what behavior you're seeing with git worktrees, if you're willing to provide them.
Sorry, I'm not seeing any related problem when trying to do a simple test with worktrees. I do see a problem with shallow clones. Could you please describe what you're seeing? Does it still happen if you delete the options from the command lines in ?
I don't think we need to exhaustively catalogue the issues - even if removing --merges doesn't fix all the issues it will fix some of them, and we can make more improvements in follow-ups.
claim
Someone should probably file a bug about failing to see merge commits in shallow clones. I don't feel like I understand the issue well enough to do so.
<!-- Thank you for filing a bug report! Please provide a short summary of the bug, along with any information you feel relevant to replicating the bug. --I tried this code: I expected to see this happen: Instead, this happened: Apparently, taking a shared reference of will suppress the lint. This seems like a false negative. While false negatives can't be avoided entirely, this case seems to be avoidable as taking a shared reference shouldn't change the type of (which is ) or modify any data in this example. Note that will trigger the lint properly. : <!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOT_END --
See also .
claim
Note this is a problem because which is supposed to fail to compile but this bug makes it fail at runtime instead
In case no one has dug into it yet, the error is suppressed in but not in and before:
label +regression-from-stable-to-stable
WG-prioritization assigning priority (). label -I-prioritize +P-medium
This is caused by . It only ever worked because the lint ran after optimizations. I made a fix . It specializes the length of arrays as constant, even if the array isn't. I'm going to open a PR when I have time, probably tomorrow. In the long term, I'd like to implement some more extensive tracking of "what parts of a value are constant". For example, the compiler currently assumes that anything that had its reference taken can change, but actually, this can only happen inside (as I understand it).
Labels from the closed duplicate , label A-diagnostics, A-lint, A-mir-opt, T-compiler.
Fixed on stable label E-needs-test
I claim the add test part
This lint only appears on full builds (not check), but this will be fixed once lands.
Reopening because I feel this is not yet solved, see on . As per the , should nail down this issue?
MRE: Also, duplicate of
Another variant of this same issue:
The behavior of .consume when the buffer is empty is weird : while :
Currently the contract of is that you can only pass a number the number of bytes you receive from . In this case the method is never being called, causing the weird behavior. I believe this is a documentation issue as the documentation for the method is pretty curt.
Maybe adding an assert_debug to check that could be useful too ?
What's the intended use of if it must always be passed a value the return value of ?
Oh sorry, I meant ! The function is intended to tell the buffer how many bytes you've consumed from the return value of
thread 'rustc' panicked at 'index out of bounds: the len is but the index is ', ../src/libserialize kind of expected, after switching toolchain from 1.15 back to 1.14. cargo clean && cargo build "repaired it"
Do you have some more information on how to reproduce this?
I hit this compiling hyper with 1.13 stable.
Which version of hyper is that? I can't reproduce with latest .
It was with master. I tried and now it works. So rustc/cargo were using libs compiled when I tried this with 1.15nightly?
Sounds a bit like cargo was re-using libraries that it should have re-compiled because the compiler has changed. cc
Sorry if I did not save the full backtrace - It was my first issue report. I switched away from 1.15 nightly back to 1.14 nightly in order to solve a serde macro issue. Cargo cleaning and rebuilding fixed the issue. I will make sure to run again and capture the output if it happens again. On 11/16/2016 4:54 PM, Michael Woerister wrote:
Sounds suspiciously like a Cargo bug! If there's a reproduction I'd be more than willing to dig in.
I believe I just reproduced this in Cargo itself: When executing with the last few lines are: So what's happening here is: is populating the entire crate cache with a bunch of libraries on the nightly compiler. switching compilers, Cargo decides to recompile everything. doesn't delete everything as it recompiles, rather incrementally builds up a new crate cache compiler, given to that directory, will read crates in that directory. compiler panics when reading a crate compiled by a future compiler. So I take back what I said earlier about this likely being a Cargo bug, it seems like this may be a bug in the compiler trying to read stale metadata? I thought we had like 10 checks in place to ensure we didn't do that, but maybe it's still leaking through?
I'm going to tag this as a stable-to-nightly regression as this appears to have been introduced in nightly. If I build cargo with beta and then switch to beta, the compiler doesn't ICE.
The panic on stable happens in we check version information. Did we change how we encode version information?
cc
I incremented the header version number though... It should be impossible for older rustc to get that far.
on it looks the same on . Perhaps we just need a PR to bump that?
Oh it looks like my changes have reached stable. I wonder if perhaps the is in a precarious spot and/or has been moved or the serialization otherwise changed. This requires further investigation.
Why is this even calling leb128? Are we using variable-length integers in the metadata header?
I assumed leb128 was the crate being built, not one used by the compiler.
Sure enough. Now every change to requires us to update the LLVM version number, because checking the rustc version number requires parsing . I think we should go back to JSON version guard.
Ahh that's what it was. We can put it at the start then, before/after the absolute address. I'd forgotten that the whole struct would have to be decoded.
What was the PR that modified the versioning scheme? I gather that something has changed about it. Can somebody explain more clearly what is going wrong here?
I mistakenly put the version string inside a larger , somehow thinking that it being at the start helped, but if the whole (that is, ) can't be decoded (e.g. because its definition changed) then you don't get to see the version string, whereas before it was separate. The fix would be to separate it again, put it back at the start and leave it there. Or just encode/decode it before instead of as part of it. Either will work as long as it can't be affected by changes to the metadata schema. The PR that introduced the new schema was , although breakage didn't happen until changed later, for procedural macros and/or the name resolution changes ( would know).
removed the field from for macro modularization.
I believe I have a fix ready. Does this affect beta too now, or just nightly?
AFAIK it's just nightly, not beta or stable
This can be replicated with a minimal : Compile it to WASM with Then, check if the generated WASM uses reference types: <!-- Provide the most recent version this worked on, for example: It most recently worked on: Rust 1.47 --It most recently worked on: (and previous stable rust versions) On those versions, compiling the above and then running with doesn't fail. <!-- Provide the version you are using that has the regression. --: <!-- If you know when this regression occurred, please add a line like below, replacing with one of stable, beta, or nightly. modify labels: +regression-from-stable-to-{channel} -regression-untriaged -- modify labels: +regression-from-stable-to-nightly -regression-untriaged This seems to have been caused by The compatibility note there says: However, the generated code seems to be indeed affected by default. Interestingly, reference types seem to be used unnecessarily, as using to turn the generated into wat and then to turn it back into wasm removes the reference types. As-is, this breaks compatibility with the WebAssembly 1.0 spec.
cc Looks like this causes issues in practice and we might want to do after all?
WG-prioritization assigning priority (). label -I-prioritize +P-high
I was unable to confirm the presence of any reference type instructions in the generated Wasm, even though I got the same error pointing towards a that looks perfectly valid to me. Interestingly enough, doing the back and forth conversion with and remove the issue, even though the output of both files is exactly the same, apart from the missing debug information. I'm assuming that this is another example of a tool making use of the , so this was already anticipated. Unfortunately (?) this isn't the first time LLVM has enabled features by default, in the past they even did affect code generation. From what I've read in the various WebAssembly specification groups, this was always intended. In conclusion, I agree this is a form of breakage, it seems to be intended, I believe if Rust wants to do anything about it the solution needs to a bit more comprehensive. E.g. actually disable all target features by default (I don't know if this is feasible with LLVM, would need research). See and .
I just spend some time digging into this error message that spits out.
Locally I've produced exactly the same message, but it really doesn't make any sense to me. The location pointed out by the error message does indeed contain a call to , but the reserved value is , so I don't know what the issue is. Further, after using the -trick, it doesn't produce this error anymore, but the code location contains the exact same bytes. I also checked if reads the target feature section. While I confirmed it does, I don't believe it validates it from what I can see in the code. So maybe there is some sort of codegen difference, but I am unable to actually find it.
Could it be the encoding difference discussed in the description of this PR?
Yes, I just confirmed that's the issue. So it does affect codegen, even if in a minor way!
Yes the problem is the LEB-encoding of the table index immediate. Using on the output of rustc I see: which shows that 's type and table index are both using a 5-byte encoding of a zero value. This is not valid in MVP WebAssembly which required the table index to be a literal zero byte, not a LEB. Given I doubt LLVM will do anything here so it sounds like yes, we'll have to solve this in Rust. I believe this is due to the the binary encoding of the LEB. This isn't reflected in the text format and most other tools don't handle this well, so it's a subtle error. That's also why convert-to-text-and-back produces a working binary. To clarify, I don't believe that this is the case. I believe modifications to the target feature section are unrelated. Ok so the question now is what to do. I see a few possible routes: this change. That would entail updating the release notes for the LLVM upgrade PR to indicate that wasm targets are indeed affected. This would probably also detail the workaround of . Note that may not be enough given in . 't accept this change. That would require something along the lines of , perhaps modified to use then adding on features afterwards. "Fix" this change. This would require coordinating with upstream LLVM to change how this works. My guess is that given the existence of this seems unlikely to happen.
There's not really a governing body per se around the Rust WebAssembly targets so there's not really a group of people we can point to and say "ok please tell us what to do". In lieu of that I would personally be tempted to go with route (1) because that keeps LLVM/C/C++ aligned with Rust and prevents divergence in the wasm ecosystem. Given the age of the proposal and the general intention for enabling more features by default over time as popular engines implement them this will also continue to change. So, concretely, I would propose: Do not change code as a result of this issue, leave it as-is. Update the release notes for LLVM 19 to indicate that WebAssembly targets are affected Update the WebAssembly documentation to explicitly document how to achieve and "MVP binary" by using and .
in the compiler team triage meeting. We think plan is reasonable at present but if there are additional issues or complications (such as it being more difficult to create an MVP binary than just using ), then option 2 (use by default) seems to be the way to go.
Looks like there may be complications. I am endeavouring to add a test to CI which asserts that a particular compiler invocation will generate MVP features. So far roadblocks are: Using is not sufficient. The attribute is not rewritten for IR of dependent libraries, like the standard library. That means that is only applied to the local crate and the standard library still codegens newer features. One can fix the above issue to additionally use but you'd have to pass for all active features. Not great. Even with there are still bulk-memory instructions in at least for example. These are coming from wasi-libc for the target which is what rustc tests in CI. This is not applicable to this issue as users typically use for this but would make adding a test difficult. Trying out the target locally I can't even get something to work it seems. LLVM is producing an invalid binary with alone when there's a memcpy in the binary. There's... a lot going on here apparently. Given the above I don't think that (2) I described above is a silver bullet by any means, even that seems doubtful to work. I need to investigate more what's going on here with all these features slinging all over the place.
Testing shows that plus Cargo's is sufficient.
and WebAssembly book () In Rust some requirements for WebAssembly targets. This makes Wasm binaries compiled with Rust (at least with default options/out of the box std) incompatible with older runtime environments, in particular not-so-old iOS safari (). Here is for example the support for Safari iOS 15.5 simulator: ! I searched but did not find any place where these requirements are written down. I think that this is a problem.
In principle Rust can produce binaries that only target the core subset of wasm 1.0 just fine. You do need to set some additional options with and build your own standard libraries though. But point taken, is missing dedicated pages on the wasm target that could contain this information.
The 2nd sentence right after the one you quoted outlines exactly what you need to do to take advantage of rustc’s capability to emit outputs to the wasm 1.0 spec (or any superset of it:) Make sure to do this and report back if you're still seeing the unexpected instructions/types/other entities after rustc is told to produce wasm binaries the way you need them to be. AFAIK, not for WASM.
Sorry, I realized I misunderstood your answer and deleted my post while you were answering it. Do you know how to show which features are enabled by default for a given target? I searched and did not find. I'm trying to discover whether is enabled by default on nowadays. As it seems to be because I am seeing types in the emitted wasm, even though I disabled the feature by "" in an environment variable RUSTFLAGS. It would be surprising to me that is built with that feature by default on wasm32, but maybe it is.
Hm, I'm actually not sure why you might be seeing simd128 in your modules. As far as I can tell it is not enabled by default for any wasm target. In order to print out features used you should look for lines in: For what it is worth I’ve been using this exact target at my $DAYJOB with a pretty primitive runtime and it has been working largely OOB if my memory serves me right. So it indeed could have something to do with your dependencies possibly using something like ? Not sure!
It seems that the was built with as I see this WASM function: What I am not sure is whether 1. this was integrated with the standard library and then imported indirectly to my project, or 2. somehow one of the dependency managed to enable it by itself. I tend to favor 1 but I'm not sure. That is why it would be great for such things to be documented :-)
For reference, I asked on the about how the std lib is built.
I confirm that as of version 1.75, the standard library is not using any WebAssembly SIMD instructions. Here is my test: which shows nothing, so they get pulled in by the memchr package through some of my dependencies.
You yourself pointed out which crate produces the v128 types: . It uses to enable emission of these types, regardless of the global configuration it looks like. This is usually fine for other architectures, but rarely for wasm, so it might make sense to file an issue over there.
I fully agree and I just opened an issue there: BurntSushi/memchr
As a heads up I've proposed that close this issue with documentation of how to target MVP wasm.
<!-- Thank you for filing a bug report! Please provide a short summary of the bug, along with any information you feel relevant to replicating the bug. --I tried to build a wasm binary with MVP opcode only, so I disabled the features: I expected to see no in the wasm binary. Instead, this happened: <!-- If you're using the stable version of the compiler, you should also check if the bug also exists in the beta or nightly versions. --: <!-- Include a backtrace in the code block by setting in your environment. E.g. . --<details<summaryBacktrace</summary<p</p</details
You can not deactivate default features, because std is prebuilt. You need to build std yourself. Also these features are considered the default by LLVM ever since the recent upgrade to LLVM 16: This should possibly be mentioned in the Rust 1.70 release notes though?
cc to confirm that we're fine with inheriting this change to default target features in Rust.
More context for this change can be found in these locations: - discussion about enabling new features in both Emscripten and LLVM. - the original LLVM patch for this - more follow-up discussion on LLVM In that sense this is an expected change, but at the same time the change was made with the expectation that most users are running on an engine that implements sign-extension. could you expand a bit more on your motivation for excluding these sign-extension instructions? I do agree, though, that this should be mentioned in the detailed release notes. It's unfortunately a bit opaque since not all users of wasm are intimately familiar with proposals and whether or not engines support them, but it's a change nonetheless worth mentioning.
I have tried to use , still not work: Our blockchain platform has used Wasm as a virtual machine running smart contracts and has been running steadily for many years. The MVP opcodes is fully sufficient for our use case. I hope there are ways to turn off these default extension features to avoid frequent upgrades every time a new feature is introduced.
Could you share the code used to reproduce this? These upgrades will not be frequent. This is the first time it's changed in 5 years.
The use case I have for wanting to build without these extensions isn't critical (servers for a game which don't support the extensions in the old version of nodejs they're running), but a way to successfully disable them to be able to build for such environments would be appreciated!
It would be helpful to have a reproduction of this to run to test out fixes. I tested something locally and was sufficient to disable sign extension instructions. I don't know why isn't working and that may be an LLVM bug. I can't confirm whether this works for others though without a means of reproduction.
I just tried the following: compiled with: to make sure std being precompiled isn't causing problems, and it totally still contains the sign-ext instructions: So you are right, this is some sort of LLVM bug.
Confirming that results in a file without the sign-ext opcodes, while does not, in my testing as well.
would you perhaps know more about ? I would expect that with on the function then even without sign-extension instructions wouldn't get generated, but I'm not sure.
I wonder if this is a consequence of It looks like that might be unioning the module-level features along with the per-function features to form the effective feature set to use.
It can also be reproduced with:
FWIW, the above flags did not work for me with - the resulting binary still had sign-extension opcodes (namely ) - but this served as a workaround (): Command: ^ With the wasm-opt pass, sign-extension ops were gone; without the pass, still there. From the thread above, it sounds like those flags work for some people but not for others. My use case is (was) supporting old Apple devices stuck on iOS 12 and thus Safari 12. Now past this, the binary can load, but performance is enough of a problem to suggest dropping support anyway.
A slightly related issue is that the WebAssembly extensions used by default by the Rust compiler are not documented (besides in the change log, but new users won't look at it), I opened a specific issue for that () and just want to link it here.
As a heads up I've documentation about features to which I've currently got as flagged to close this issue. The documented solution for targetting MVP wasm is: which should solve the original OP of this issue.