GitHub repository: shivendrra/shredword
Primary language: C++
Description: Fast & efficient BPE tokenizer written in C & python for LLM tranining