Tokeniser: changes to main method