Add support for $-expressions in the lexer
Misc. readability improvements to lexer
Fix bug in word tokenization

If words started with alphanumeric characters and contained command
substitutions, the command substitution would not be accumulated and
the () would be treated as operators.
Leave quotes in tokens

The specification wasn't clear (to me) on this point. It didn't
explicitly say to include the quotation marks, but I can't see how later
stages can work properly unless they're left in. I wouldn't have even
thought to remove them, but in the specification it explicitly says when
to include characters in operator tokens, so I figured it would also
explicitly say to include quote characters, if that's what was
intended. Maybe this will make more sense to me later on.
Add support for comments to lexer