Tokenize only non-recursive constructs

This pull-request contains changes similar to those in #28 (leaves possibly recursive constructs recognition to the parser) which was reverted by #30 (aligns to Shell Grammar Rules in IEEE 1003.1-2013 and resolves all of 5 shift/reduce conflicts). Thanks to the changes in #30, #31, and #32, the resulting grammar now accepts an arbitrary Shell program without any conflicts when processed by `yacc(1)`. The latter is regarding the issue #17 and essential to safely introduce grammar rules for pseudo-token `WORD` which may include recursive constructs that the lexer cannot handle by itself.