Added tokenizer class and implementation. Regex probably not correct.
Merged.