GitHub repository: DonghakPark/tokenizers-cpp
Primary language: C++
Description: Universal cross-platform tokenizers binding to HF and sentencepiece