Fixed a few bugs, implemented tokenizer base and added some tests