GitHub repository: mlc-ai/tokenizers-cpp
Primary language: C++
Description: Universal cross-platform tokenizers binding to HF and sentencepiece