Updated tokenizing code to use new API
Added broken MaximalMunchTokenizer & a test suite

Trying to fix the tokenizer by adding tests until the problem becomes apparent.