extras/lex: fix whitespace and a spelling error

extras/lex/lex.m:
    As above.
extras/lex: add support for Java and C#

extras/lex/lex.m:
    As above.
extras/lex improvements

    + Add a charset type that can be used to represent sets of characters.
      These are useful when constructing lexers.

    + Start adding support for wider character sets.

    + Add support for matching on ranges of characters.

extras/lex/lex.m:
    As above.

    anybut/1 now uses unicode as a basis.

extras/lex/samples/lex_demo.m:
    Demonstrate the new features.
extras/lex: minor changes following Sebastian's changes

extras/lex/lex.m:
    Document some caveats with the charset functions.

    Add a range check to the two-argument charset function.

    Simplify the anybut function.
extras/lex: fix lex.lexeme char 8-bit cutoff

In lex.lexeme, all char codepoints where downcast to 8bit, thus rendering
Unicode matching impossible.  This change removes this limitation.

extras/lex/lex.lexeme.m:
    As above.
extras/lex: enable full Unicode support.

The NFA is now using the sparse_bitset to compile the state transitions.
This significantly reduces the compile time for the regular expression, thus
enabling the use of a much larger Unicode range for character recognition.

The helper function anybut/1 has been made aware of unicode, i.e.
anybut/1 now recognises [`0x1` .. `0xffff`] - [but chars].

In addition, the regexp(string) instance now accepts Unicode chars,
the previous code was only valid for ASCII chars (when using the UTF-8
encoded C-strings in the C-based grades)

extras/lex/lex.convert_NFA_to_DFA.m:
    Use the sparse_bitset ADT to build the tables used by the library.

extras/lex/lex.m:
    Add a new constructor to the regexp type allowing regexps to be build
    directly from character sets.

extras/lex/lex.regexp.m:
    Build regexps from character sets.

extras/lex/samples/lex_demo.m:
    Demonstrate the use of wide characters in the lexer demo.

extras/lex/lex.automata.m:
extras/lex/lex.lexeme.m:
    Conform to above changes.
extras/lex: Use a more concise datastructure for character ranges

Lists of Unicode characters can be very long as there are many characters.
This change represents ranges of characters by storing only the first and
last character in each range.

extras/lex/lex.m:
    As above.
extras/lex: don't use deprecated functions

extras/lex/regex.m:
    Don't use deprecated standard library functions.
Merge remote-tracking branch 'upstream/master'