add tests for token stream methods