add tests for next_token