GitHub repository: tiktoken-go/tokenizer
Primary language: Go
Created: 2023-04-06T16:56:56Z
Description: Pure Go implementation of OpenAI's tiktoken tokenizer