atom/atom — pull request #2475 by klorenz (2014-06-01 05:28:19+00:00)
Title: Emit tokenized-lines signal if lines tokenized

Enables grammar developers to inject scope names on a higher level.
This would enable grammars for like restructured text and markdodwn
to insert correct scopenames for constructs like:

Headline
========

Usually it would be tokenized as

[ [ value: "Headline" scopes: [ 'text.restructuredtext', 'meta.paragraph.restructuredtext' ],
[ value: "========", scopes: ["text.restructuredtext", "meta.paragraph.restructuredtext", "markup.heading.restructuredtext", "punctuation.definition.heading.restructuredtext"] ]

Having the opportunity to look at tokenization on a multiline-base, you can insert "markup.heading.restructuredtext" at
headlines marked as paragraphs.