luarocks: bump stdlib requirement to v40.

* rockspec.conf (dependencies): Bump stdlib requirement to v40.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>
syntax: add support for basic syntax highlighting.

* lib/zile/grammar/lua-bundle.lua: New syntax grammar for Lua
language.
* lib/zile/syntax.lua: New file. Return a sparse array of
attributes for a line of text according to that buffer's grammar.
* build-aux/config.ld.in (file): Add lib/zile/syntax.lua.
* local.mk (dist_modules_DATA): Add doc/modules/zile.syntax.html.
* lib/zile/bundle.lua: Use lrex onigurama engine for easier
porting of TextMate grammar files.
(load_grammar): Initial massaging of a grammar file for fast
access to attributes.
* lib/zile/zile.mk (dist_pkgdata_DATA): Add lib/zile/syntax.lua
and lib/zile/grammar/lua-bundle.lua.
* lib/zile/buffer.lua (buffer_new): Initialise syntax member.
* lib/zile/file.lua (find_file): Load the grammar when opening a
new file.
* lib/zz/main.lua (main): Highlight the scratch buffer correctly.
* lib/zile/term_redisplay.lua (draw_line): Use it to set
attributes as the line is displayed.
* lib/zile/lib.lua (stack): Stack object place holder.
* rockspec.conf (dependencies): Add lrexlib-oniguruma.
* NEWS: Update.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>
syntax: select the appropriate mode according to file extension

* lib/zile/grammar/diff-bundle.lua: New file, with the syntax
grammar for diff and patch files.
* lib/zile/zile.mk (dist_pkgdata_DATA): Add diff-bundle.lua.
* lib/zz/default-color-bundle.lua: Set highlight styles for new
diff grammar.
* lib/zile/bundle.lua (load_file_associations): Load a map of
file extension to grammar basename using the fileTypes key in
each grammar file.
* lib/zile/term_redisplay.lua (draw_status_line): Display the
name of the loaded grammar for the current buffer in its mode-
line.
* NEWS: Update.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>
refactor: centralise parser state in a table.

* lib/zile/syntax.lua (state.new): New function. Return a new
parser state...
(syntax_attrs): ...refactored from here.
(highlight): Change to accept a single state table parameter.
(parse): Likewise, but save long derefrence chains into locals
for better access speed.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>
refactor: add parser state metamethods and simplify.

* lib/zile/syntax.lua (push_op): Moved from here...
(metatable.push_op): ...to here.  Adjust all callers.
(state.new): Set the metatable on the new parser state.
(highlight): Cache parser attrs and ops in locals for faster
access.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>
syntax: calculate attributes for grammar capture expressions.

* lib/zile/bundle.lua (scopetoattr): Search for most complete
dot-delimited scope name match in colors table.
(load_grammar): Simplify calculation of match expression
attribute calculations using scopetoattr.
Also add capture expression attributes to loaded grammar table.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>
syntax: support colored capture expressions.

* lib/zile/syntax.lua (state:push_op): Don't push nil valued
operations.
(rex_exec): Adjust 1-based onig capture indices to 0-based
zile buffer offsets.
(leftmost_match): Return capture indices of matched pattern.
(parse): Between beginning and end of a matched pattern, queue
operations to set attributes for captured sub-expressions.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>
syntax: support begin and end expressions on a single line.

* lib/zile/bundle.lua (capturestoattr): Call scopetoattr for each
element of captures.
(load_grammar): Factor out pattern compilation from here...
(compile_patterns): New function.  ...to here.
(compile_rex): Precompile onig regular expressions.
* lib/zile/syntax.lua (leftmost_match): Account for matching
against expressions in .rex and .finish.
(parse): keep a pattern stack so that we can push a new pattern
list onto the stack for matching subexpressions, and then pop
back to the original when the finish expression is matched.
Similarly, save the active color to a color stack when a new
pattern list is pushed, and use that to color the whole begin/
end pair when the finish expression is matched.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>
syntax: support back-references in grammar end expressions.

Somewhat trickier than expected because we do want to match the
likes of \2, while not accidentally expanding \\2 (not a back-
reference, just an escaped backslash followed by a plain 2):
* lib/zile/bundle.lua (compile_rex): Now a global function.
(has_backrefs): New function to check for back-references, being
careful of unbalanced escapes.
(compile_grammar): Don't precompile `end' expressions with un-
resolved back-references.
* lib/zile/syntax.lua (expand): New function to expand back-
references in expressions using a reference string and capture
offsets, and then compile on demand; again being careful of
unbalanced escapes.
(parse): Pass capture offsets to expression matches, and use
them to expand back-references on demand.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>
syntax: support grammar repository and include references.

* lib/zile/grammar/lua-bundle.lua (string.quoted.single.lua)
(string.quoted.double.lua): Use repository to reduce repetition
of escaped character patterns.
(string.quoted.single.lua, string.quoted.double.lua): Add
repository pattern for % placeholders in format strings.
* lib/zile/bundle.lua (compile_pattern): Factored out from the
inner loop of compile_patterns.
(load_grammar): Use compile_patterns on repository entries
too.
* lib/zile/syntax.lua (leftmost_match): Look-up include
references in the grammar repository, and perform matches against
any expressions or pattern list found there.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>
syntax: only recalculate edited line and onwards.

Keep track of the currently edited line, and only recalculate
highlighting for that line and those following.
* lib/zile/buffer.lua (replace_estr): Mark newly edited line as
dirty.
* lib/zile/syntax.lua (state.new): Bump dirty.  Store bp and lnum.
(metadata.get_lnum): New accessor method.
(syntax_attrs): Unless this line has been edited since the last
time it was displayed, or comes after the last edited line, just
return the highlighting calculated last time.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>
syntax: support multi-line syntax matches.

* lib/zile/syntax.c (state.init): Store the initial state for the
parser in the first line of the file when first called.
(state.new): Copy the stored parser state for this line into the
new state table.
(metatable.store): After highlighting, save the current state to
the next line in the buffer so that we can pick up highlighting
from there without re-parsing the entire file.
* NEWS: Update.

Signed-off-by: Gary V. Vaughan <gary@gnu.org>