fixed token generation/refreshing that got broken due to changes for enabling Japanese search queries