Finalised tokenizer

Tokenizer unit tests working, operators are now successfully identified.