Changing to a different type of tokeniser
Tokeniser finished for now. Fixed the tests.