add tests for new token look up code