style: make programming style consistent

Signed-off-by: Peter Williams <pwil3058@gmail.com>
LexicalAnalyser: add a "skip list" and remove automatic skipping of white space.
Give the user full control of what gets matched and whether it gets returned.

Signed-off-by: Peter Williams <pwil3058@gmail.com>