Change:
In the interest of improving error/diagnostic quality for macros I'd like to try and get some of the feature stabilized. While some of these APIs don't seem to have consensus on their effect on IDE/language server support, the API seems both well-tested and to have consensus that its result only being invalidated by edits to the contents of the macro makes it a non-issue for IDE support (as in ). label +T-libs-api +A-macros +A-proc-macros
Nominating to consider for partial stabilization. I remember we had a lot of conversation about some of the proc_macro APIs in the past. I don't remember this one in particular being of concern. Does anyone on know of any blockers for stabilizing this?
Most of the discussion was related to either user-defined diagnostics or the line/column information being exposed. I see no reason this can't be stabilized.
merge
Team member has proposed to merge this. The next step is review by the rest of the tagged team members: [x] [ ] [x] [x] [x] [x] Concerns: resolved by resolved by Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See for info about what commands tagged team members can give me.
Do you have thoughts here?
It would be nice to have an example in the docs for this API, if possible. One other thing that pops out at me is the return type of . My guess is that perf isn't generally a huge concern in this context, and therefore it's okay. But I don't know that for sure. Otherwise, this return type is going to force a clone/allocation on every call. I don't know enough about proc macro internals to judge whether that's reasonable or not.
Yeah, I think getting the text of what's passed into a macro shouldn't be problematic. What could problematic is the macro somehow getting a span which "escapes" the macro (so, eg, the span for the whole file containing the macro) and querying the source of that, but I assume that's not possible. There's also a question of line-endings. Does the returned text preserve original line-endings, or is it allowed to normalize to ? I'd prefer to not expose original line endings.
:bell: This is now entering its final comment period, as per the . :bell:
concern a question of line-endings
Not the whole file, but it can certainly escape a single macro call, and can include totally unrelated items. It sounds similarly problematic as getting a span for the whole file. concern span which escapes the macro Is supposed to return here? Is supposed to return ?
I would personally expect that to fail at the source_text call (good find), unless we want to make more stringent guarantees about external spans being impossible to construct. (Personally my expectations of Span::join are that failures are about spans impossible to represent (from incompatible sources), not ones we choose not to let the user do things with)
I wonder whether one should do  style removals of indentation before returning the source text, or whether to not do that. Ideally, adding indentation to a macro invocation / struct declaration should not issue a re-run of the macro. Edit: I'm wondering about this for two reasons, first because it looks weird, and second for the use case of extending incremental compilation to parsing. just iterates all lines of the input and takes the minimum of all indentations. This is not precisely what I had in mind, it was more something that integrates the start of the macro invocation. So if you have: For a macro: Then it currently outputs: Which isn't that nice. It's not just not what shows up in the code, as the is aligned with the (my first concern) but also if you increase the level of indentation by one you are getting: I wonder if there is some way of processing that we could arrive at something like:
(FTR, 's example above is using which is unstable, but there are hacks to do as well)
I think modifying whitespace outside of newline normalization is a bad idea. I'm not sure what usecase the API would be useful for if that were the case. Even for diagnostics/errors, changing the code formatting is detrimental? Or well, I guess for indoc-style it is consistent across the contents of the macro so it's maybe fine? I would be worried about how it'd effect mapping spans to the contents of source_text (such as pointing to specific tokens in an error)
I don't think we should do any modification to the source text, even newline normalization. I think we should just return the source text verbatim. If a user wants to normalize it, they can do so (or use any number of crates to do so); however, a user can't un-normalize it if we've normalized it.
To clarify why normalizing might be a good idea: What line endings are used is already somewhat ill-defined. The source code of the same git project checked out on Linux and on Windows would typically have different line-endings. If we don't normalize line-endings and expose them to proc-macros, that opens another avenue why, eg, might give different results depending on the host OS the project is compiled on. I don't think that in practice there will be a lot of these sorts of problems, but if we are committed to reproducible builds in principle, it seems worthwhile to not add fundamental non-determinism, if we can help it.
I think that differing newlines will already cause issues with reproducible builds since all of the spans will have different offsets, the debuginfo will be different, etc. I think it's reasonable to require configuring git to preserve line endings if you want reproducible builds.
... and I now recall that we actually do normalize newlines already?
I don't feel that either of these warrants blocking the stabilization of : resolve a question of line-endings resolve span which escapes the macro For line endings, either normalizing or not is okay with me. For spans that cross boundaries of a macro invocation, I'd probably prefer if would return in this situation but it's pretty far down the list of things wrong/broken with macros already, so I don't find it concerning for this stabilization.
:bell: This is now entering its final comment period, as per the . :bell:
The final comment period, with a disposition to merge, as per the , is now complete. As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed. This will be merged soon.
how would you deal with join being already exposed on stable, without an Option like interface? I've pointed that out above in the thread, linking to a comment that links to a . Given that the API is already stable and doesn't return an Option, it should IMO be that returns a None if a span is unhygienic. Maybe spans could contain a bool param on whether they are hygienic or not, which is set by join as well as join-like primitives. Then checks that bool and makes its descision based on that. This stuff can I think be done after stabilization, too.
That hack doesn't result in a joined . That hack creates a using both the start and the end span such that the resulting compiler error will join the two spans, but it doesn't give your proc_macro a joined .
good point. I've tried building a similar primitive that joins spans in a similar fashion, but it doesn't work: If you try that on the example from dtolnay given above, you get as output: It's I think because the group wholly ignores the spans of its children and instead assumes the span of the macro invocation that created it. I guess this combination trick can only be used to issue compilation errors. So things work differently than I thought they would work. I'm sorry for having brought this up. Case rested :).
Stabilization PR:

Discussion:
/// Note: The observable result of a macro should only rely on the tokens and /// not on this source text. The result of this function is a best effort to /// be used for diagnostics only. <del> #[unstable(feature = "proc_macro_span", issue = "54725")] </del> <ins> #[stable(feature = "proc_macro_source_text", since = "CURRENT_RUSTC_VERSION")] </ins> pub fn source_text(&self) -> Option<String> { self.0.source_text() }