change tokeniser to allow rewind