GitHub repository: Kedzia00/tiktoken
Primary language: Python
Description: tiktoken is a fast BPE tokeniser for use with OpenAI's models.