GitHub repository: matyasosvath/hutoken
Primary language: C
Created: 2024-12-03T13:46:26Z
Description: A fast byte pair encoding (BPE) for language models. Work in progress.