Change:
<ins> // Copyright 2017 The Rust Project Developers. See the COPYRIGHT // file at the top-level directory of this distribution and at // http://rust-lang.org/COPYRIGHT. // // Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or // http://www.apache.org/licenses/LICENSE-2.0> or the MIT license // <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your // option. This file may not be copied, modified, or distributed // except according to those terms. #![feature(macro_vis_matcher)] //{{{ issue 40569 ============================================================== macro_rules! my_struct { ($(#[$meta:meta])* $ident:ident) => { $(#[$meta])* struct $ident; } } my_struct!(#[derive(Debug, PartialEq)] Foo40569); fn test_40569() { assert_eq!(Foo40569, Foo40569); } //}}} //{{{ issue 26444 ============================================================== macro_rules! foo_26444 { ($($beginning:ident),*; $middle:ident; $($end:ident),*) => { stringify!($($beginning,)* $middle $(,$end)*) } } fn test_26444() { assert_eq!("a , b , c , d , e", foo_26444!(a, b; c; d, e)); assert_eq!("f", foo_26444!(; f ;)); } macro_rules! pat_26444 { ($fname:ident $($arg:pat)* =) => {} } pat_26444!(foo 1 2 5...7 =); pat_26444!(bar Some(ref x) Ok(ref mut y) &(w, z) =); //}}} //{{{ issue 40984 ============================================================== macro_rules! thread_local_40984 { () => {}; ($(#[$attr:meta])* $vis:vis static $name:ident: $t:ty = $init:expr; $($rest:tt)*) => { thread_local_40984!($($rest)*); }; ($(#[$attr:meta])* $vis:vis static $name:ident: $t:ty = $init:expr) => {}; } thread_local_40984! { // no docs #[allow(unused)] static FOO: i32 = 42; /// docs pub static BAR: String = String::from("bar"); // look at these restrictions!! pub(crate) static BAZ: usize = 0; pub(in foo) static QUUX: usize = 0; } //}}} //{{{ issue 35650 ============================================================== macro_rules! size { ($ty:ty) => { std::mem::size_of::<$ty>() }; ($size:tt) => { $size }; } fn test_35650() { assert_eq!(size!(u64), 8); assert_eq!(size!(5), 5); } //}}} //{{{ issue 27832 ============================================================== macro_rules! m { ( $i:ident ) => (); ( $t:tt $j:tt ) => (); } m!(c); m!(t 9); m!(0 9); m!(struct); m!(struct Foo); macro_rules! m2 { ( $b:expr ) => (); ( $t:tt $u:tt ) => (); } m2!(3); m2!(1 2); m2!(_ 1); m2!(enum Foo); //}}} //{{{ issue 39964 ============================================================== macro_rules! foo_39964 { ($a:ident) => {}; (_) => {}; } foo_39964!(_); //}}} //{{{ issue 34030 ============================================================== macro_rules! foo_34030 { ($($t:ident),* /) => {}; } foo_34030!(a, b/); foo_34030!(a/); foo_34030!(/); //}}} //{{{ issue 24189 ============================================================== macro_rules! foo_24189 { ( pub enum $name:ident { $( #[$attr:meta] )* $var:ident } ) => { pub enum $name { $( #[$attr] )* $var } }; } foo_24189! { pub enum Foo24189 { #[doc = "Bar"] Baz } } macro_rules! serializable { ( $(#[$struct_meta:meta])* pub struct $name:ident { $( $(#[$field_meta:meta])* $field:ident: $type_:ty ),* , } ) => { $(#[$struct_meta])* pub struct $name { $( $(#[$field_meta])* $field: $type_ ),* , } } } serializable! { #[allow(dead_code)] /// This is a test pub struct Tester { #[allow(dead_code)] name: String, } } macro_rules! foo_24189_c { ( $( > )* $x:ident ) => { }; } foo_24189_c!( > a ); fn test_24189() { let _ = Foo24189::Baz; let _ = Tester { name: "".to_owned() }; } //}}} //{{{ some more tests ========================================================== macro_rules! test_block { (< $($b:block)* >) => {} } test_block!(<>); test_block!(<{}>); test_block!(<{1}{2}>); macro_rules! test_ty { ($($t:ty),* $(,)*) => {} } test_ty!(); test_ty!(,); test_ty!(u8); test_ty!(u8,); macro_rules! test_path { ($($t:path),* $(,)*) => {} } test_path!(); test_path!(,); test_path!(::std); test_path!(std::u8,); test_path!(any, super, super::super::self::path, X<Y>::Z<'a, T=U>); macro_rules! test_meta_block { ($($m:meta)* $b:block) => {}; } test_meta_block!(windows {}); //}}} fn main() { test_26444(); test_40569(); test_35650(); test_24189(); } </ins>

Discussion:
A macro with two cases that match identdepending on the order of the cases. This works: But this doesn't: the error message: Why would it expect an there? If ident, shouldn't it just move on to the second macro case and match that?
Annoyingly, the macro expander doesn't attempt to handle this kind of ambiguity at all. It won't back out of and retry the next rule, it just bails. It can only use "constants" (like or some other explicit token that isn't a macro variable) to disambiguate.
I don't think this is really a direct issue since this basically resolves to the macro expander being unable to handle ambiguity between macro arms. As such, I'm going to close.

Here is a minimum example of the problem. Compilation fails with following error. Which means that the fragment qualifier is not taken into account to choose which variant of the macro is to be used.
Yeah, macros don't do backtracking. You'll need to adjust the matchers to provide some other way to disambiguate.
Yes, I obviously know that I could just put brackets or whatever around one of the patterns to make it distinct. But the whole point of having fragment qualifiers is precisely to distinguish between two otherwise identical patterns. If macros cannot do it although they have the perfect tool for it, then they are stupid and ought to be improved.
It wasn't obvious to me that you knew the workaround -- sorry for assuming! Also see .
Thanks for pointing the PR. It is blocked for now, but I suppose there is still hope. Sorry for being harsh. I have been raging for months over how annoyingly limited and stupid Rust macros are, and I finally found a case general and obvious enough to justify my complaining…
triage: Macros still don't backtrack. I can't find an issue for that, though I do suspect that one exists in some form. Labeling as A-macros, though.
here is (closed).
Closing in favor of -- macro rules do not backtrack.

In this case, it should be looking for an ident or a , in which there is no ambiguity. All channels are affected. This is affecting my ability to write macros accepting generics using angle brackets, e.g. , with a supporting pattern to turn into . May be related to .
In theory there is no ambiguity because an ident can't start with . As far as I can tell the macro evaluator doesn't reason about this at all. It sure is annoying and I hope it can be fixed. I suppose it introduces a forwards compatibility consideration, just like follow sets: if we let macros distinguish between and , then can never be to the ident grammar.
the error is spurious; this is definitely a bug. The issue is purely when the repeating group is empty. Hence working.

at least that's how much I currently understand, since macros are really counterintuitive sometimes. I don't think this is supposed to happen. Even if it is, the exact rules used for macro matching should definitely be documented somewhere (I think there's not even an RFC).
It seems like the parser in particular is very greedy, or something. For example, eliminates the error (as does , but I assume this is reduced from something where you can't do that). However, fixing this (if it can be fixed) is probably a breaking change :(
Found the culprit: Do you really think this would break something? I actually don't think it would, since a fix for this should only accept more macro invocations.
On second thought... I think that's not the direct cause, since all other fragments behave roughly the same. Oh well, back to the drawing board
Okay, so as it turns out, all NT parsers introduce this bug (except , which you can't really test). In this particular case, changing from to only worked because idents and integral literals are both valid expressions. For example, this doesn't work: Now that this confusion is out of the way, I think I see what happens: When parsing as an expression, libsyntax panics because of a syntax error, which aborts compilation (the macro docs state that the parser fully commits to parsing such a nonterminal, so the fact that the third invocation doesn't work is expected behaviour). However, when parsing , it works. But since the token wasn't eaten, the second arm is tried (this is the bug!). It matches, of course, and the macro is accepted. So, fixing this would indeed be a breaking change, since the intended behaviour is (at least as far as I know), to reject the invocation, but this isn't happening.
I disagree that attempting the second arm is a bug. Ideally, should attempt each rule until it finds one that matches. From my perspective, the bug is that has no way to recover from failed parse attempts. Ideally, your last example should go something like this (using to represent cursor positions): - → → matched - → → input too long; next rule → → → matched - → syntax error; next rule. → → → matched
That would indeed be useful!

But I think this comment implies that that isn't the intended behaviour: (stating that the macro parser fully commits to NTs implies to me that it doesn't backtrack to try other arms)
I believe that's referring to how it parses within a rule. Earley parsers can deal with ambiguities by tracking multiple potential parse forests (if I remember correctly; my understanding is a little vague). What it's saying is that it has to commit to parsing a non-terminal (i.e. higher-level productions like expressions) because the parser doesn't have any way to back out of a partial parse. So when it encounters one, it has to parse it, come hell or high water. Having the macro system not check successive rules once a rule starts matching would be apocalyptic: it would kill damn near every useful, non-trivial macro. We're talking mass hysteria, cats and dogs living together.
Fair enough. In that case the bug is just that the macro expander will panic when it can't parse an NT, so it can't backtrack. I also managed to dig up , which was closed as "not a bug", but this definitely feels like one.
Sorry to dig this up again, but isn't an a terminal, so your comments about NTs aren't applicable to this bug?
Correct! That's why I mentioned : // this could be handled like a token, since it is one (AFAIK, token == terminal)
I got bitten by this bug too. Note that I see two different issues here: The parser accepts , which according to is a bug. Indeed, according to the docs I would have expected the parser to refuse, but I can live with it either way and I can see where fixing the issue would cause code to break.The parser refuses , which should be a no-brainer: `` is not an ident, so the first rule should not even begin to match and no backtracking is required to move on to the next rule. Moreover I don't see where fixing this would break anything, since it would only allow code that currently does not compile (although I have no idea how hard fixing this may turn out to be).

This contains a macro with two *-repeating arguments, plus a non-repeating argument sandwiched between them. It works fine when the repeating arguments are supplied, but when I try to leave out the first one, rustc claims the syntax is ambiguous. But I can't see why.
I have this problem with = too, in the pattern . I get "ambiguous syntax" for the = symbol.
It's a known limitation of macros, see
What's strange is that it's perfectly capable of using to terminate the repeating ident matcher, because clearly isn't an ident. But it won't use that fact to disambiguate the match at the first token.
Also, the example at that point in the book doesn't make any sense. Neither nor can follow .
Current output:
The problem is not that is considered as an identifier (see e.g. ), but rather that once we start parsing a "fragment" (like ), parsing must succeed or macro parsing aborts. Fixing this turned out to be quite complex, and I expect we'll only address it in macros 2.0. Closing.

I’m writing a to work around , and I’ve discovered that it can’t handle attributes on the variants like I thought it would. Specifically, the matcher fails to match , complaining “error: local ambiguity: multiple parsing options: built-in NTs ident ('var') or 1 other options.” This doesn’t seem to make sense, since an cannot start with . Small test case:
Did you ever find a workaround for this? I'm trying to write a macro that parses structs and I'm running into this: Yields:
Actually, this issue looks like the exact same as
I managed to implement a workaround while writing a builder macro, which is to have two parts: keep matching on the front token as a token, save that into a separate place, and recursively call yourself, passing the rest of the tokens have another rule that matches on the final pattern that no longer has meta attributes See code at (sorry for ugliness, this was a proof of concept which evolved a lot). I'll convert that to a proper crate and document it this weekend. Example output for
has idents conflicting with other idents; those conflicts are “legitimate”, given Rust’s left-to-right parsing strategy. In contrast, this is about idents conflicting with tokens that cannot possibly be idents. Simpler example, showing that rustc can’t determine that (e.g.) is not an ident: It seems like the same thing happens with any token in place of .
I'm running into an error in enum_primitive that according to the bug report is blocked on this issue
So this issue is solved? (at least on nightly)
It’s marked as closed, isn’t it? But you’ll have to wait for the 2017-07-11 nightly, which hasn’t been built yet.