unit testing token stream