Only export tokenize from the Tokenize module.
Refactored Tokenize to use a data type that makes pattern matching easier.
Renamed internal types in Tokenize
Removed unneeded imports.
Parse basic identifiers. Ugly.
Replace emptyToken function with an Empty type constructor.
Tiny cleanup, remove unused name.
Added test showing numbers are treated like identifiers.
Reduce duplication by filtering out empty identifiers.
Refactored to remove a case statement.
Renamed variables to shorten lines.
Added "var" and fixed a bug where the javascript couldn't end in an identifier token.
Create some data types to make the signature of lookahead more understandable, hopefully.
Tried to prettify the code in Tokenize
Added beautify function - simple cases right now.
Wrote a test showing we can parse simple strings.
Don't get fooled by tokens occurring inside quotes.
Simplified types a little bit.
Reducing need for parentheses by re-ordering parameters in tokenize
Re-order function parameters to reduce parentheses.