Tokenizer changes
Tokenization: Minor bug fix
Missed tokenizer rules
Minor change and explanation added
Spelling mistake fix
Merge pull request #1 from nicklondhe/master

Merging the latest changes