GitHub repository: legalforce-research/tokenizer-speed-bench
Primary language: Rust
Description: Comparison code of various tokenizers