rust-lang/rust — pull request #19103 by huonw (2014-11-19 10:23:30+00:00)
Title: Tokenise suffixes on all literals

The Rust compiler tokenises a literal followed immediately (no whitespace) by an identifier as a single token: (for example) the text sequences `"foo"bar`, `1baz` and `1u1024` are now a single token rather than the pairs `"foo"` `bar`, `1` `baz` and `1u` `1024` respectively.

The compiler rejects all such suffixes in the parser, except for the 12 numeric suffixes we have now.

I'm fairly sure this will affect very few programs, since it's not currently legal to have `` in a Rust program, except in a macro invocation. Any macro invocation relying on this behaviour can simply separate the two tokens with whitespace: `foo!("bar"baz)` becomes `foo!("bar" baz)`.

This implements [RFC 463](https://github.com/rust-lang/rfcs/blob/master/text/0463-future-proof-literal-suffixes.md), and so closes https://github.com/rust-lang/rust/issues/19088.