Added comments to the tokenizer
Merge pull request #38 from cesanta/tokenzier_comments

Added comments to the tokenizer