Issue #8: Improved documentation, error handling and code analysis issues.
Made Tokenizer more readable

Refactored the original parse loop into a state machine, extracted
if-expressions into named methods, made variable names more expressive.