Leaving recursive construct recognition to the parser

Removes all exclusive conditions from the lexer by leaving recognition of possibly recursive constructs (parameter expansions and command quotes) to the parser. Rules for the TOKEN state are merged into one action. The MAIN state is no more needed thanks to leading newlines allowed grammatically.