moonshotai

/

Kimi-K3

like 8.45k

Follow

Moonshot AI 15.3k

Summary

Kimi K3 ships no Jinja chat template: there is no chat_template.jinja and no chat_template key in tokenizer_config.json. The prompt format lives in encoding_k3.py and is reachable only through trust_remote_code=True.

That works for callers who go through tokenization_kimi.py, but every stack that renders prompts from tokenizer.chat_template gets nothing, so llama.cpp and minja, GGUF metadata, and OpenAI-compatible servers that do not import the model's Python.

The gap is already being filled downstream, which is the reason for this PR rather than a discussion thread: unsloth/Kimi-K3 uploaded a community port hours after release, and measured against encoding_k3.py it is not byte-exact on any conversation that declares tools (details and reproduction in a PR filed there). A template in this repo is the only thing that makes the format authoritative instead of guessed.

Kimi K2 shipped a Jinja template. This PR restores that for K3 without changing the format: the template is a faithful port of build_chat_segments, not a reinterpretation.

It renders under transformers and under minja, so the one file covers llama.cpp and the GGUF conversions as well as vLLM and SGLang. That constrained the implementation as minja has no sort filter, no tojson keyword arguments and no replace filter, so the reorder pass is a selection sort over range and the compact-JSON tool declaration is a recursive macro over dictsort. Both are verified byte-exact against encoding_k3.py, under both engines.

What it reproduces

apply_chat_template's behaviour exactly, including thinking_effort defaulting to "max", the recursive key sort and compact separators in the tool declaration, &-before-" attribute escaping, the reasoning_contentto reasoning fallback, the JSON type names on per-argument tags, tool_choice and response_format blocks, and the run-scoped tool-result matching in normalize_xtml_tool_result_messages, a run of consecutive tool messages is reordered into call order and renamed from the matched call only when the whole run matches by id.

Equivalence

verify.py drives build_chat_segments from this repo as the oracle and compiles the template with transformers' own _compile_jinja_template, so both sides are the real code paths:

oracle:       moonshotai/Kimi-K3@9f62e4e9fffb/encoding_k3.py (sha256 verified)
candidate:    chat_template.jinja
compiled by:  transformers.utils.chat_template_utils._compile_jinja_template

ChatLint shapes:  292/296 renders byte-exact across 49 shapes
json encoding:    29/29 scalar encodings byte-exact
randomised:       24000/24000 conversations byte-exact
RESULT: equivalent within the documented limitation

The 24,000 randomised conversations cover adversarial content (injected <|open|> markers, unicode, quotes and ampersands inside tool names, argument keys and tool descriptions), every JSON value type as an argument, flat and function-wrapped tool calls, name fields, dynamic tool declarations via a system message with tools, unknown roles, shuffled and partially matched tool-result runs, thinking on and off, and every tool_choice / response_format combination.

The json encoding line covers the one place the template reimplements json.dumps in Jinja, since minja cannot pass sort_keys or separators: 1e30, 1e-7, -0.0, integers past 2^53, control characters, U+2028, emoji and empty containers, all byte-exact.

Under minja, checked by running llama.cpp's engine rather than assuming:

engine:     minja (llama.cpp)
shapes:     292/296 renders byte-exact across 49 shapes
randomised: 300/300 conversations byte-exact

The one documented limitation

If tool_calls[].function.arguments arrives as a JSON string, normalize_tool_arguments parses it and the renderer expands it into per-argument tags. A Jinja template cannot do this: transformers exposes only tojson, raise_exception and strftime_now to templates, inside an ImmutableSandboxedEnvironment, so no JSON parser is reachable.

The template falls back to the <|open|>json type="object" block that encoding_k3.py itself emits for unparseable arguments, so the content is valid JSON in a form the model has seen rather than something corrupt. Every other argument encoding, dict, missing, {}, empty string, whitespace, unparseable string - is byte-identical.

This is worth flagging in the model card either way, since a JSON string is what the OpenAI API returns: servers that hand raw OpenAI payloads to a template should parse arguments first.

Two small things noticed while porting

Neither is addressed by this PR.

1. The thinking-effort system block tells the model supported values "include low, medium, high, and max", but _VALID_THINKING_EFFORTS is {"low", "high", "max"}, thinking_effort="medium" trips the assert. The prompt advertises a value the renderer rejects.
2. build_chat_segments silently skips messages whose role is not user, system, tool or assistant. A developer message, which the OpenAI API now prefers over system, disappears with no error.

Provenance

Written and verified with ChatLint, a conformance suite for chat templates. Kimi K3's Python renderer passes it with no errors and no warnings, as does this template; Moonshot ships 5 of the 14 clean templates in its 786-template corpus.