leonsarmiento/KAT-Coder-V2.5-Dev-6bit-XL-mlx

KAT-Coder-V2.5-Dev by Kwaipilot — a 35B-A3B sparse MoE agentic coding model achieving 69.40 SWE-bench Verified, quantized for Apple Silicon using the BaseQuant_XL 6/8-bit recipe.

KAT-Coder-V2.5-Dev is a post-trained MoE built on Qwen3.6-35B-A3B via two-stage SFT (127K examples) + RL (10 epochs) with token-level consistency (TITO), truncated importance sampling (TIS), and hierarchical reward shaping from harness execution feedback. It ships text-only weights (no vision tower) despite the Qwen3_5MoeForConditionalGeneration architecture — the MLX conversion uses mlx_vlm with strict=False to skip the 333 missing vision_tower.* parameters.

Quickstart

pip install -U mlx-lm

python -m mlx_lm.generate --model leonsarmiento/KAT-Coder-V2.5-Dev-6bit-XL-mlx --max-tokens 256 --temperature 1.0 --top-p 0.95 --prompt "Implement an LRU cache in Python with O(1) get/put."

Works with LM Studio. Thinking mode is on by default (emits <think>...</think>). Pass enable_thinking=False for direct answers.

Quantization Strategy

BaseQuant_XL recipe — precision is allocated by layer importance, not applied uniformly:

Layers	Bits	Rationale
mlp.gate (router), shared_expert_gate, lm_head, shared_expert	bf16	Routing decisions and output projection — any quantization noise here causes expert misrouting or output degradation
embed_tokens, self_attn, linear_attn	8-bit	Every-token layers — near-lossless, attention quality preserved
switch_mlp (routed experts)	6-bit	Bulk parameters — 256 experts with only 8 active per token; redundancy absorbs quantization noise. 6-bit is the sweet spot for routed experts (higher bits can cause overthinking)

• Bits per weight: ~6.9 · Total size: ~28 GB (6 shards) · Group size: 64

Recommended Inference Parameters

Thinking Mode (default)

Parameter	Value
temperature	1.0
top_p	0.95
top_k	20
presence_penalty	1.5
reasoning_parser	qwen3

Instruct Mode (non-thinking)

Parameter	Value
temperature	0.7
top_p	0.8
top_k	20
presence_penalty	1.5
enable_thinking	false

preserve_thinking is supported — set chat_template_kwargs: {"preserve_thinking": true} to retain thinking traces from historical messages (beneficial for agentic scenarios).

Model Overview

Property	Value
Architecture	Qwen3.6 MoE (35B-A3B) — text-only weights
Parameters	35B total / ~3B active per token
Experts	256 (8 routed + 1 shared)
Context window	262,144 tokens native (~1M with YaRN)
Attention	Hybrid linear + full (full every 4th layer)
Thinking	On by default (<think>...</think>)
Tool calling	XML-style via qwen3_coder parser
License	Apache-2.0

MLX Benchmarks (n=50, instruct mode)

Benchmark	KAT-Coder-V2.5-Dev 6bit-XL	Qwen3.6-35B 6bit-XL	Gemma-4 26B 6bit-XL
MMLU	70%	64%	76%
MMLU_PRO	64%	64%	82%
ARC_CHALLENGE	90%	90%	90%
HUMANEVAL	92%	78%	98%
MBPP	78%	78%	82%
Average	78.8%	74.8%	85.6%

KAT-Coder leads Qwen3.6 on 4 of 5 benchmarks — the RL training for agentic coding shows in HUMANEVAL (+14 points). MMLU_PRO trails Gemma-4 significantly (64% vs 82%); KAT-Coder is specialized for SWE, not broad knowledge. Coding is strong but Gemma-4 remains overall champion.

MATHQA Thinking Efficiency (n=10, seed=42, 8192-token budget)

3-stage methodology: initial run → answer extraction (regex cascade) → re-test failures. Verified accuracy separates sampling/extraction variance from genuine capability gaps.

Model	Verified Acc	Reason chars/correct	Time/correct
Gemma-4 26B 6bit-XL	10/10 (100%)	2,420	39.8s
Qwen3.6-35B 6bit-XL	10/10 (100%)	3,562	32.1s
Salience-1.5-Pro 6bit-XL	10/10 (100%)	5,350	47.7s
KAT-Coder-V2.5-Dev 6bit-XL	9/10 (90%)	265	7.5s

KAT-Coder is the reasoning efficiency champion — 9× less reasoning per correct answer than Gemma-4, and 20× less than Salience. Its SWE-focused RL training produces extremely terse, decisive reasoning chains. The one genuine failure (Q636) was an algebra error, not a reasoning deficit. KAT doesn't overthink — it computes, verifies, and answers.

Preserve Thinking

KAT-Coder is a Qwen3.6 model and supports preserve_thinking for agentic multi-turn workflows. Set chat_template_kwargs: {"preserve_thinking": true} to retain thinking traces from historical messages — essential for long-horizon SWE tasks where the model needs to recall its own reasoning from prior turns.

Verification (number-guessing game, 4 turns): With preserve_thinking enabled, KAT stays perfectly consistent — picking a secret number in turn 1 and giving correct higher/lower hints in all subsequent turns (1.4–1.6s each). The model reads its own prior <think> blocks rather than re-deriving the answer. Without preserve_thinking, the model re-picks a new secret number each turn and reconstructs consistency from visible conversation history — which works for simple games but risks drift in complex agentic tasks.

Source Model Benchmarks

From Kwaipilot's evaluation (vLLM/SGLang, BF16):

Benchmark	KAT-Coder-V2.5-Dev	Qwen3.6-35B-A3B (base)
SWE-bench Verified	69.40	64.40
SWE-bench Multilingual	63.00	57.00
SWE-bench Pro	45.96	40.63
Terminal-Bench 2.1	41.02	32.02
KAT-Code-Bench	46.21	42.76
Scicode	44.20	37.53

Paper: arXiv:2607.05471.

Capabilities

• Agentic coding — multi-step tool use, repo-scale edits, long-horizon SWE tasks
• SWE-bench SOTA — 69.40 Verified at this scale class
• Deep reasoning — structured <think>...</think> chains with one-token switch to disable
• Tool calling — RL-trained for reliable, non-degenerate tool-use patterns

Flight Simulator — Coded by This Model

A real-world coding benchmark: each model was prompted to write a complete flight simulator from scratch. The resulting code was rendered and recorded.

KAT-Coder made by Kwaipilot (KwaiKAT Team). MLX quantization by leonsarmiento.