jiebi/CodeConvo huggingface dataset:
I'm not a giant fan of the syntax because it looks odd when rendered by anything other than rustdoc, which I'd rather avoid, one of the major design concerns for intra-doc links was compatibility with other markdown parsers:
So, I agree, but I'm not convinced that the warning class is something that needs to be surfaced in the JSON backend as anything special. Also, bear in mind, HTML is a part of markdown: valid HTML (well a subset of HTML) is valid markdown. Users already can , and they do that in many cases. Markdown parsers already need to be able to handle HTML. There's nothing new there. Whatever consumes the JSON backend will need to be able to handle that too. I mean, yes, in theory, in practice people don't do this because of the barrier to entry (you have to get your tools to use that, including )
Also, this is incorrect, pulldown absolutely parses HTML. It has to, like I said valid HTML is valid Markdown. It doesn't parse it thoroughly, I believe you can mess it up with sufficiently complex HTML, but it parses HTML. What this changes is subtler: up till now you did not need any particular stylesheet to acceptably render rustdoc markdown from the JSON backend on your own (just use your favorite markdown renderer's stylesheet), but now you will need other css classes to handle rendering warnings as warnings.
I do not agree that this is a problem. I've used Phabricator's implementation of this approach happily and it is by design that warning banners are one paragraph. It is not the intention that you would put entire sections of documentation into a warning banner. They are for a brief callout. It's fine to refer to a subsequent section of the documentation from the callout if needed. I suppose this is a matter of taste but it's one where I believe the opinionatedness of the syntax is a positive thing.
So I discussed this a bit with on the side, we agree that the effect on the JSON backend isn't major since HTML is already a subset of markdown and docs use HTML already (especially for images and tables). My position is as follows: I strictly prefer HTML over some other scoped syntax with nuances people have to learn. I don't think there's as much value in unscoped syntax which makes my position "HTML or nothing", basically.
I'm not wholly against unscoped single-line syntax. We and syntax in W3C specs and you invariably need to add more stuff to it (there's an HTML class in these specs for this reason). Furthermore, Rust has a strong culture of wrapping text, so this means we need to work well while wrapped, and that means we're again introducing some form of scoping mechanism (e.g. indenting the text), which editors will have to learn, as well.
I don't know about admonitions specifically but there seems to be something of a community habit to use "magic tags" on fenced code blocks for this sort of features (e.g. github has a tag), maybe that could be an option? I don't know if pulldown_cmark has a hook for manipulating code blocks but… So a warning would be or maybe the tag would be to leave room for things like note/todo/danger/… Alternatively, take a gander at how other markdown implementations extend the language for this sort of features? e.g. seems to be using for admonitions.
Hmm, but is still for code, even if it has some extra behavior. Aside from the design concerns, this would be feasible to implement: we just extract the language string from the code block with pulldown.
Could you use something that's already built into markdown? For example, rustdoc could take and render it as an admonition.
That's what suggested: to use markdown and make some changes on rustdoc side. I think the codeblock approach is the best personally.
My bad, I didn't see that.
I wonder if it makes more sense to add this as a proc-macro. Rustdoc can still provide the CSS classes and things, and you could use them with HTML directly, but you could use or something and it would expand to the correct HTML tags and classes.
Might be better to have a then, no?
Is that going to be a special header similar to the "unsafe"/"experimental"/etc ones, or is that going to also be used within docs in an order-sensitive way? If the latter we should probably just use a CSS class; mixing and is ugly
Agreed.
Among the suggested ideas, my favourite one remains the code block with "warning" as tag:
Yeah, but triple backticks is also for preformatted text without further markup, so you won't be able to e.g. use text formatting (or other code) in there
True... Finding the right syntax is much more complicated than expected...
That's the issue of using markdown, for all that it has a pretty nice baseline syntax it really was not designed for extensibility.
We can always use : it'd allow to have text formatting inside it.
I would use , it's not quite kosher to invent arbitrary HTML tags (though you can use Custom Elements, which is more heavyweight)
Sounds good to me. It's funny because we kinda came back to the . :rofl:
I just thought: we could use tag in the markdown and then transform it into in the generated HTML?
My main objection to all of these is that rustdoc shouldn't be adding custom markdown syntax that no one else supports. Any new syntax we use should look reasonable even if rendered with the default renderer.
Well, it remains valid markdown since it's HTML (even though it's custom tag). I'm completely fine with , but it might be simpler/better for users to simply write .
Coming back to this (because of ), it seems like the conversation kinda agreed on adding a CSS class directly and then it would look like this in the docs: . I'll send a PR in the next days and then we can open an FCP to see if everyone is ok with it (and if not, we'll have code to help the debate).
This indeed seems like a better solution than the HTML block. It also plays better with something said about not linking it too much too HTML. This will look like a proper warning or note, mostly independent of the HTML generation.
They also seemed to not be too favourable to anything that needs some extra parsing from rustdoc iuc. I'm fine with bot the github solution and the CSS class.
Maybe one more comment to make here. The note-taking app, which uses Markdown also has a similar syntax to create these blocks. It is called . It uses a syntax similar to the syntax that is used for doc intra-links currently, and it is fully compatible with them.
I am uncertain if this is any better than what suggested from the GitHub syntax, but because of the similarity to the doc intra-link syntax. I do feel like this might be interesting because it should go well with backwards compatibility. However, this may look worse than the GitHub syntax solution when not going through cargo doc.
I thought I'd already left this comment, but I'd really like to avoid adding more markdown syntax where possible. This feature works fine without special syntax, I don't think we need it. edit: copying over the reasoning from the :
Although, I might not agree fully, I can fully appreciate and understand the reasoning you left in the corresponding PR. Still, I do think it might be worth to standardize some kind of syntax for some kind of warning block. This does not actually require customized markdown. Alternatively, I can suggest adding the GitHub flavor with emojis to the . This may help in standardizing short form warnings and notes within documentation. We could for example propose to standardize: which should look like the following I do value people's opinion on this. Is this a possible compromise?
That's a valid alternative, I think the yellow color is pretty important though, and it's useful for themes to be able to make styling decisions about warnings so I do think that having some way of marking it is important. I just don't think the way of marking it should be some magic invocation as opposed to CSS classes. (And I think that the bar for a feature that is adding new semantics to the Rustdoc markdown parser is much, much higher than that of one stabilizing a CSS class)
100% agree. I will look at the Rust API guidelines issue and possibly an RFC to rust to track the adoption in GitHub and possibly eventually merge. Thank you :smile:
Beyond all this discussion of third-party tools, I'd really like to make sure that whatever new syntax and presentation we add to rustdoc can also be to [mdBook]. That way, when the standard library starts using callouts, we can make sure the same style also gets used in The Book. Mostly, The Book seems to just use blockquotes, often with headings like [this one about ownership]. mdBook doesn't do anything special with it: it's [just a block quote with a header].
[mdBook]: [this one about ownership]: [just a block quote with a header]:
This is a good point. Once we have something merged here, I'll send a PR there to have an equivalent.
CC
I'd be happy to add some CSS rules to mdbook if that is what you are proposing.
Yeah, it is.
FYI, using a is not semantic and Microsoft GitHub is setting a bad precedence. — CommonMark Spec v0.30, denotes a in Markdown and is rendered as such. — W3C HTML spec, Unless you are quoting a source, this is not the correct element to use. While I disagree with even using altogether, at least some tools out there are at least translating the blockquote in admonition context into a (div has no semantic meaning which is unfortunately the best HTML option for admonitions). It would be ill-advised to follow Microsoft GitHub’s Markdown syntax fork&mdash;especially with their disregard for semantics. Similarly using a block would also be breaking semantics as an admonition is not preformatted text. CommonMark does have that could be used if the solution was preferred. There’s also other reasonable options noted here that don’t break semantics.
I agree that block quotes for callouts are bad, and would like to work with mdBook to roll out a common syntax (probably ) that will replace the ad-hoc, unsemantic markup that they use right now.
Using is what we agreed upon in The debate now is on the look itself.
Unfortunately isn't great because markdown has a "boundary" when HTML tags are used. I wanted to add a warning indicating that users probably wanted another trait, but intra-doc links don't work (which is the correct behavior per markdown spec).
Works perfectly fine though (well, minus the UI bug I just discovered haha): ! I used this code: I'll send a PR to improve documentation.
Ah. Well I suppose the issue wasn't with intra-doc links, but with inline code blocks. What I tried was that but with bar`, and it failed.
Works too: !
I opened which adds extra explanation on how to use markdown inside html tags. That should solve your issue.
That note is exactly what I was just looking in to. I had it inline, given my familiarity with HTML. TIL that matters.
https://crater-
searched nightlies: from nightly-2024-02-01 to nightly-2024-03-20 regressed nightly: nightly-2024-02-10 searched commit range: regressed commit: <details<summarybisected with <a href=''cargo-bisect-rustc</av0.6.8</summaryHost triple: x86_64-unknown-linux-gnu Reproduce with: </detailsprobably
WG-prioritization assigning priority (). label -I-prioritize +P-high
MCVE
The following code fails on stable: When compiled in release mode, the assert is optimized out by LLVM even though it checks the exact same condition as the other assert, as can be verified by testing in debug mode. Note that it's theoretically possible for the stack to be aligned correctly such that the bug is suppressed, but that's not likely. The 256's can be replaced with larger alignments if that happens. The bug is caused by the impl of copying to the stack with the default alignment of 16. ()
and seem related
This is a soundness hole.
So... I guess the problem is that the that rustc emits for the unsized local doesn't use ? That's... kind of bad, because it doesn't look like even supports dynamically setting the alignment: If this is correct, the issue should be renamed to something like "unsized locals do not respect alignment".
A way to fix this would be to change the ABI for dynamic trait calls that take by value to always pass by address. When the trait object is created is where the trampoline function would be generated, so there is no additional cost for static calls.
This can also be fixed by not allowing dynamically-sized local variables with an alignment more than some maximum value and having all dynamically-sized locals use that maximum alignment unless optimizations can prove that a smaller alignment suffices.
Messing with the ABI wouldn't help with unsized locals in general (assuming is correct), since e.g. will also need an alloca with dynamic alignment in general. Given only a value, there is no static way to say anything about its alignment, so how would you distinguish whether calling a method on it is OK?
Can we call and then align the pointer for the local ourselves as a workaround until we come up with a good solution?
In clang there's a function which might or might not be useful in this case.
requires a constant alignment. Non constant alignments are . This matches semantics of LLVM instruction, so it's not helping, as alignment of value in is unknown at compile time when moving from .
I think that overallocating for unsized locals of unknown alignment may make sense. You can and then make use of to compute the offset. Sure, it increases stack requirements, but it seems acceptable for , and in most cases you end up with extra 7 bytes on stack or less. 256 alignment seems like an edge case where wasting 255 bytes doesn't matter.
if we keep setting as the default alignment we can skip the overallocating and offset dance for types with an alignment below that.
My idea was that rustc would prohibit coercing overaligned types to a dyn trait that takes by value. So:
The alignment is dynamically determined though, not sure if saving 15 bytes of stack space is worth the conditional branch?
Oh right. Yea that's probably not helpful.
Since , this would be backwards incompatible (at latest) once trait object upcasting is finally implemented: and trait objects would have to have the same alignment restriction (because you can upcast to ), but today you can freely create such objects with any alignment.
I was under the impression that calling didn't require dynamic , is that related to missing MIR optimizations, meaning we have unnecessary moves which end up duplicating the data on the stack using a dynamic ?
Correctness shouldn't rely on optimizations to fire...
Sure, I was just hoping we weren't relying on "unsized locals", only unsized parameters (i.e. ), for what we're exposing as stable. And I was hoping an optimization wouldn't need to be involved to get there. I wonder if it would even be possible to restrict "unsized locals" to the subset where we can handle it entirely through ABI, and not actually have dynamic s anywhere, since that would make me far more comfortable with 's implementation or even stabilization. Ironically, if that works, it's also the subset we could've implemented years earlier than we did, since most of the complexity, unknowns (and apparently bugs) are around dynamic s, not calling trait object methods on a dereference of .
That should be possible. I think everything that is necessary is performing the deref of the box as argument () instead of moving the box contents to a local first () That would also make it possible to call in cg_clif. Cranelift doesn't have alloca support, so the current MIR is untranslatable.
Hmm, but that's not trivially sound because it could lead to MIR calls where argument and return places overlap, which is UB. We'd potentially have to special-case it to derefs, but even then what's to stop someone from doing ? Maybe we could introduce a temporary for the return value instead, for these kinds of virtual calls? That should remove any potential overlap while allowing the call.
(Unrelatedly, I just realize we use a copying-out shim for vtable entries that take by value, even when would be passed as at the call ABI level. We would probably have to make computing the call ABI a query for this microopt to not be expensive)
That would result in being moved to a temp first before dropping and calling the boxed closure. If it wasn't moved first before the drop of , then borrowck would give an error. (I meant for this change to be done at mir construction level, not as an optimization pass, so borrowck should catch any soundness issues)
We can try it, I guess, but right now a lot relies on many redundant copies/moves being created by MIR building and may not catch edge cases like this. But also I don't think moving out of a moves the itself today. Me too, but I was going with the "correct by construction" approach of creating a redundant copy/move, just on the return side, instead of the argument side, of the call.
After revisiting on triage, removing nomination.
pre-triage: nominated for discussion in the next weekly meeting.
I just checked and, a function call roughly this (touched up for clarity): Some of my previous comments assumed the wasn't there and the call would use the destination directly, but given that there is a move, that simplifies my suggestion. <hr/For virtual by-value calls, we can skip one move when building the MIR, specifically the move out of the dereference of a for the argument. Given that we already move the result of the call, there should be no aliasing caused by this, but if we decide to change anything about how we build MIR calls, we'd need to be extra careful. We can also make this pattern be gated by a separate feature-gate (rather than ), and make sure the standard library only uses that feature-gate and not . This should solve any alignment issues might have, because we'd not be using dynamic unless the broader feature is used.
OK, if I understand what's being proposed here, it is to excerpt a subset of the "original unsized-locals plan" -- specifically, to permit locals with unsized types, but only in contexts where we can be guaranteed that we can point to the original storage and don't actually have to move any value (and hence we don't require variable size stack frames).
I'm in favor of this proposal, but I do think we want to be slightly careful to ensure we're not violating soundness. The main concern would be if there is some way to access or re-use the variable that was moved. I attempted to do so with this example () but it fails to compile: The error here arises because the closure would need to capture a to do anything, which prevents a later move. But maybe there is some kind of tricky example that can arise where we re-assign the value during argument evaluation?
compiles and does before , where is the result of .
Borrowck should prevent that, right?
borrowck may rely on MIR building creating extraneous copies, so we can't just assume it helps us. IMO we should limit it to functions taking unsized parameters and function calls passing such parameters or dereferences of es (any other pointer/reference doesn't support moving out of).
Well, I think is right that borrowck would catch violations of soundness in some strict sense, but I guess what I meant is that we might be breaking backwards compatibility.
The example that feels pretty relevant. If I understood your proposal, it would no longer compile. If I recall (thinking back), I had a plan to handle this by save the to an alternate location in the actual call, we pass ownership of and then do a shallow free of the This still seems like it would work, though I'm not sure how easy/elegant it is to formulate. In a sense we'd be rewriting to .
Isn't example dependent on doing a direct virtual call? My understanding is that you can't do this on stable with because the virtual call exists in one place, in libstd, and it doesn't do anything weird, so it would work with a restricted ruleset. EDIT: if it can work on stable, it's because calling 's will move the whole before the assignment is evaluated, which is fine, because the shallow drop is actually inside that , not its caller.
I agree that this code doesn't compile on stable today, but I'm not sure if that's the point. I guess you're saying it's ok for it to stop compiling? I think what I would like is that our overall behavior is as consistent as possible between Sized and Unsized parameters. I think we could actually do a more general change that doesn't special case Sized vs Unsized or anything else.
I think I am basically proposing the same thing as earlier, but with a twist: Today, when building the parameters, we introduce a temporary for each parameter. So if we had we might do: What I am saying is that we could say: If the argument is moved (not copied), and the "place" has outer derefs (e.g., we are passing where is some place), then we assign the place to a temporary and we make the argument to the function be : I don't believe this is observable to end-users, except potentially for when the destructor on runs. I believe that it accepts all the same code we used to accept. It also has nothing specific to or types at all. Does that make sense? I'm very curious to know if anyone sees any problems with it. (One obvious thing: if we ever add , its design would have to accommodate this requirement, but I think that's fine, in fact the complications around getting moves right for precisely this case are one of the reasons we've held off on thus far.)
Can you not reinitialize today? If you move the whole , it is indeed more conservative (and should let us skip unnecessary moves even in cases). But also, do you want to move the locals check from typeck to borrowck? I don't have an intuition for when your approach wouldn't create unsized MIR locals, from the perspective of typeck. However, this is only relevant if we want a stricter feature-gate. At the very least it seems like it would be easier to implement a change only to MIR building, and it should fix problems with for the time being, even without a separate feature-gate.
Hmm, that's a good point, I think do that. Sigh. I knew that seemed too easy. I remember we used to try and prevent that, but we never did it fully, and in the move to NLL we loosened those rules back up, so this compiles: Still, you could do my proposal for unsized types, though it would introduce an incongruity. UPDATE: Oh, wait, I remember now. For unsized types, reinitialization is not possible, because you can't do with an unsized rvalue (we wouldn't know how many bytes to move). So in fact there isn't really the same incongruity.