GitHub repository: rojas-diego/tokengeex
Primary language: Rust
Description: TokenGeeX is a tokenizer for CodeGeeX aimed at code and Chinese