GitHub repository: project-vajra/tokenizers-cpp
Primary language: C++
Description: Universal cross-platform tokenizers binding to HF and sentencepiece