Lexer table sizes reduction

Completely covers issue #24 and partially covers issue #17. The solid regular expression named WORD is replaced with several lexer rules under a new exclusive state TOKEN. The latter significantly decreases the required lexer table sizes so that they can fit into minimal limits mandatory for conforming implementation of the lex(1) utility.