ni_get_next_token now uses external buffer

This change is a preparation to consolidate the future interface on
lexers, with support for a cache for peeking. It's needed because
were the lexer to return already built tokens upon peek(), it could
happen that a caller could peek a token another already had destroyed.

With this change, peek is now able to create a new version of the
token to each caller, copying from the same internal buffer.
Renamed the Lexers test suite to Tokens.
Lexers and their tests.

Lexers now are supported: they encapsulate a stream upon which to
call the tokenizer, presenting the interface for peeking (returning
the current token under the stream cursor) and advancing (moving
the stream cursor ahead for the next token available).
Dynamic error handling system. Untested.