Kimi-K2.6 fast tokenizer (no custom code)

Faithful conversion of the moonshotai/Kimi-K2.6 tiktoken tokenizer to a HF fast tokenizer (PreTrainedTokenizerFast), loadable without trust_remote_code. Built with transformers.convert_slow_tokenizer.TikTokenConverter from tiktoken.model (163584 base ranks + 256 special tokens, ids preserved). Verified token-for-token identical to the reference tokenizer on code/CJK/special-token/long-text samples.

Purpose: benchmarking harnesses (genai-bench etc.) that cannot enable remote code. Maintained by Baseten FDE; not an official Moonshot AI artifact.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support