KAT-Coder-V2.5-Dev-FTW

A FreeToken-ready FTW build of the KAT-Coder-V2.5-Dev coding fine-tune.

This repository packages the 35B-A3B KAT-Coder-V2.5-Dev Mixture-of-Experts fine-tune in FreeToken's native FTW fast-load format. It is intended to load directly in FreeToken for local, heterogeneous GPU/CPU inference.

KAT-Coder-V2.5-Dev was created by Kwaipilot by post-training Qwen3.6-35B-A3B with supervised fine-tuning and reinforcement learning for coding and agentic software-engineering tasks. This repository does not claim a new fine-tuning run: it makes that existing fine-tune usable in FreeToken.

Model lineage

1. Qwen3.6-35B-A3B — base architecture.
2. Kwaipilot/KAT-Coder-V2.5-Dev — 35B total / approximately 3B active coding fine-tune, trained with SFT and RL.
3. sahilchachra/KAT-Coder-V2.5-Dev-NVFP4A16 — data-free, weight-only NVFP4A16 quantization.
4. remixie/KAT-Coder-V2.5-Dev-FTW — FreeToken FTW conversion and packaging.

The model has 40 hybrid Qwen3.5/3.6-style layers, 256 routed experts plus a shared expert, and 8 routed experts active per token. It supports reasoning, coding, agent workflows, and tool calling through the Qwen chat template.

What is FTW?

FTW is FreeToken's self-contained, sharded fast-load checkpoint format. The conversion pre-packs weights and routed-expert banks so FreeToken does not need to reinterpret the original Hugging Face safetensors every time the model is started.

This repository contains:

• freetoken_weight.json — FTW tensor manifest and shard map
• freetoken-00000.ftw
• freetoken-00001.ftw
• freetoken-00002.ftw
• model configuration, tokenizer, chat template, and generation metadata

The three FTW shards contain 22,898,896,896 bytes (approximately 21.3 GiB) of tensor data. Keep the manifest and all three shards together.

Runtime compatibility: .ftw files are not safetensors. This repository is for FreeToken and is not directly loadable by Transformers, vLLM, SGLang, llama.cpp, Ollama, or LM Studio. Use the linked upstream repositories for those runtimes.

Precision layout

A16 means that NVFP4 weights are multiplied by 16-bit activations on the GPU; it does not mean the stored weights are 16-bit.

Component	Storage / execution
Routed expert weights	NVFP4 E2M1, group size 16
Shared expert weights	NVFP4 E2M1, group size 16
GatedDeltaNet / linear-attention projections	BF16
Full self-attention projections	BF16 in this FTW build
Routers, gates, embeddings, LM head, and norms	BF16
GPU NVFP4 activation path	W4A16: FP4 weights × BF16 activations
CPU expert path in FreeToken hybrid mode	W4A8 for CPU-computed expert work
KV cache quantization	None declared by the source checkpoint; runtime-managed by FreeToken

The source A16 checkpoint stores the full-attention projections as weight-only NVFP4 and the GatedDeltaNet projections as BF16. FreeToken currently expects a uniform attention representation for this hybrid architecture, so the ten full-attention layers were dequantized to BF16 during conversion. The expert and shared-expert FP4 payloads remain packed NVFP4.

Download

Install the Hugging Face CLI if needed:

pip install -U "huggingface_hub[cli]"

Windows / FreeToken Desktop

hf download remixie/KAT-Coder-V2.5-Dev-FTW `
  --local-dir "$env:USERPROFILE\.freetoken\models\KAT-Coder-V2.5-Dev-FTW"

Then restart or refresh FreeToken Desktop, open Models, and start KAT-Coder-V2.5-Dev-FTW.

Linux

hf download remixie/KAT-Coder-V2.5-Dev-FTW \
  --local-dir ~/.freetoken/models/KAT-Coder-V2.5-Dev-FTW

Serve with FreeToken

The command below matches the FreeToken engine version used for validation:

ft serve --model-path "$env:USERPROFILE\.freetoken\models\KAT-Coder-V2.5-Dev-FTW" `
  --host 127.0.0.1 `
  --port 8000

FreeToken automatically chooses its attention and MoE backends. On the tested 24 GB GPU it selected Triton attention plus the hybrid GPU/CPU MoE backend.

OpenAI-compatible API

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="KAT-Coder-V2.5-Dev-FTW",
    messages=[
        {
            "role": "user",
            "content": "Write a Python function that returns the n-th Fibonacci number.",
        }
    ],
    temperature=0.7,
    top_p=0.8,
    max_tokens=1024,
    extra_body={
        "top_k": 20,
        "chat_template_kwargs": {"enable_thinking": False},
    },
)

print(response.choices[0].message.content)

Thinking is enabled by the model template by default. Remove "enable_thinking": False to allow a reasoning trace. FreeToken automatically selects the qwen3 reasoning parser and qwen3_coder tool-call parser for this model family.

Conversion details

The conversion was performed from sahilchachra/KAT-Coder-V2.5-Dev-NVFP4A16 without additional training, calibration, or another quantization pass.

The conversion process:

• validated all 13 source safetensor shards;
• adapted compressed-tensors NVFP4 tensor names to FreeToken's expected layout;
• converted compressed-tensors global-scale divisor semantics to FreeToken's multiplier convention;
• fused shared-expert gate/up projections into FreeToken's layout;
• dequantized the ten full-attention layers to BF16 so all attention layers use a representation supported uniformly by FreeToken;
• preserved routed and shared-expert NVFP4 packed weights and their FP8 block scales;
• packed all 40 routed-expert layers into FTW expert banks; and
• copied the tokenizer, chat template, generation config, and relevant model metadata.

No fine-tuning or benchmark claims originate from the FTW conversion itself.

Validation

This artifact was smoke-tested end to end with:

• FreeToken Desktop 0.2.0-beta.13
• FreeToken engine 0.1.1+g30aa89115
• Windows
• NVIDIA GeForce RTX 3090, 24 GB VRAM
• FreeToken's automatically selected hybrid GPU/CPU MoE backend

The model loaded all dense weights and 40 expert banks, completed CUDA-graph and prefill warmup, served the OpenAI-compatible API, followed exact-response instructions, and generated syntactically correct Python code.

These are functional smoke tests, not a formal quality, throughput, perplexity, or benchmark evaluation. Refer to the base model card for the authors' benchmark results. Those results should not automatically be assumed for this quantized FTW build.

Context length and memory

The architecture declares a maximum context length of 262,144 tokens. Actual usable context in FreeToken is limited by the KV pages allocated after model and expert-cache placement. It can therefore be much lower on consumer hardware. Check the Allocating ... tokens for KV cache line in the FreeToken startup log before attempting very long contexts.

The model uses heterogeneous GPU, CPU, and system-memory execution. Hardware requirements depend on FreeToken's selected backend and cache settings. The successful validation above used a 24 GB GPU with CPU/RAM expert offload; it did not require all experts to fit in VRAM.

Known limitations

• FreeToken-specific: use the upstream safetensors model for other runtimes.
• Text only: the architecture config contains vision metadata, but the open KAT-Coder checkpoint ships language-model weights only.
• Hybrid activation precision: GPU NVFP4 execution is W4A16, while expert work assigned to FreeToken's CPU hybrid executor uses W4A8.
• Windows page locking: if loading reaches the expert-bank stage and fails with WinError 1314, Windows denied the optional page-locking privilege. Use a FreeToken release with a pageable/locked-bank fallback or configure the Windows Lock pages in memory privilege. This error does not indicate corrupt model weights and reconversion is unnecessary.
• First-start compilation: the first launch may take longer while Triton kernels are compiled and cached.
• No multimodal input: image/video inputs are unsupported by this artifact.

Sampling

The copied generation config recommends:

• temperature=1.0
• top_p=0.95
• top_k=20

For direct, non-thinking instruction following, the upstream card recommends temperature=0.7, top_p=0.8, and enable_thinking=False.

License and credits

Apache 2.0, following the upstream KAT-Coder release. Users are responsible for reviewing and complying with the license and terms of the upstream model and its dependencies.

• Fine-tuning and original KAT-Coder release: Kwaipilot
• NVFP4A16 quantization: sahilchachra
• FTW conversion and packaging: remixie
• Runtime: FlashML-org/FreeToken

Citation

If you use the model, cite the original KAT-Coder work:

@misc{katcoder_v25_2026,
  title={{KAT-Coder-V2.5 Technical Report}},
  author={{KwaiKAT Team}},
  year={2026},
  month={July},
  eprint={2607.05471},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/pdf/2607.05471}
}