Added more tests for tokenisation.