Start to add some tests. Added tokenization tests and some token
processing tests.