Added get_next_token function. Untested.