GitHub repository: BBuf/tokenizers-cpp
Primary language: C++
Description: Universal cross-platform tokenizers binding to HF and sentencepiece