vultr/VultronCoderAtlas-Kimi-K3-2T

VultronCoderAtlas

VultronCoderAtlas is a 2.10 trillion parameter sparse mixture-of-experts model derived from Kimi K3. It targets coding, agentic, tool-use, and multimodal workloads.

VultronCoderAtlas applies Router-weighted Expert Activation Pruning (REAP) to Kimi K3. Cerebras Research introduced the method in REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression.

The model retains 672 routed experts in each of Kimi K3's 92 MoE layers. It preserves top-16 routing, the native vision path, and the 1,048,576-token context limit. Its packed checkpoint is 1.199 TB, down from 1.561 TB for the source checkpoint.

Model details

Field	VultronCoderAtlas
Base model	moonshotai/Kimi-K3 at revision 9f62e4e9fffbd0a83ddd60e1c209d828994b3569
Architecture	93 transformer blocks: 1 dense block and 92 MoE blocks
Attention layout	23 MLA blocks and 69 KDA blocks
Routed experts per MoE layer	672
Routed experts selected per token	16
Shared experts per MoE layer	2
Total parameters	2.10 T
Active parameters per token	Approximately 107 B
Expert weight format	Native MXFP4, 4-bit, group size 32
Packed checkpoint size	1,199,059,386,443 bytes
Context limit	1,048,576 tokens
Input modalities	Text and image
Primary output	Text

The total and active parameter counts are engineering estimates.

Construction

REAP scores each routed expert by averaging its post-renormalization router weight multiplied by its output norm over the tokens routed to it. Atlas measured these signals through Kimi K3's packed MXFP4 serving path using a decontaminated calibration mixture of code, agentic and tool-use trajectories, reasoning, technical prose, and vision.

Each MoE layer retains its 672 highest-scoring experts. The checkpoint compacts their packed tensors and permutes the corresponding router gate and correction-bias rows into the same order. Shared experts, attention, embeddings, and the vision tower preserve their source tensors byte-for-byte.

Deterministic half-splits of the calibration corpus produced a median per-layer Spearman correlation of 0.971, a fifth-percentile correlation of 0.801, a minimum sampling density of 0.981, and mean pooled saliency retention of 0.860.

Intended use

VultronCoderAtlas is intended for:

• repository-scale coding and software maintenance;
• coding agents that use shell, file, and structured tools;
• long-context technical analysis;
• image-and-text tasks supported by the retained Kimi K3 vision path;
• OpenAI-compatible inference with preserved reasoning history.

The model always uses thinking. Multi-turn clients must pass the complete prior assistant message back to the model, including reasoning_content, content, and tool_calls. Dropping the reasoning history changes the protocol used for release evaluation.

Evaluation

The release evaluation used the immutable 672-expert checkpoint. Speculative decoding was disabled for the final DeepSWE, Terminal-Bench 2.1, HumanEval+, MBPP+, and MMMU-Pro results. Prefix caching was enabled for DeepSWE, Terminal-Bench 2.1, HumanEval+, and MBPP+.

Local-harness rows run Kimi K3 and VultronCoderAtlas with the same task revision and protocol. Moonshot-published rows report Kimi K3 values from the pinned upstream model card.

Benchmark	Comparison basis	Kimi K3	VultronCoderAtlas
DeepSWE v1.1	Moonshot published	67.5	64.6
MMMU-Pro, no Python, max	Moonshot published	81.6	79.5
Terminal-Bench 2.1	Local Kimi Code Harness	76.4	75.3
HumanEval+	Local Kimi Code Harness	78.7	82.9
MBPP+	Local Kimi Code Harness	74.3	76.5
SWE-bench Verified	Local Kimi Code Harness	93.8	95.2
LiveCodeBench v6	Local Kimi Code Harness	91.1	92.1
BFCL v4 multi-turn	Local Kimi Code Harness	63.8	63.4

Deployment

The checkpoint requires an inference engine with Kimi K3 architecture support, native MXFP4 kernels, and Kimi K3 reasoning and tool-call parsers. Release qualification used vLLM development builds with tensor parallelism across eight GPUs.

The following command shows the common vLLM interface. Engine version, image, KV-cache allocation, attention backend, and GPU-specific flags must be pinned for the target hardware before production use.

vllm serve vultr/VultronCoderAtlas-Kimi-K3-2T \
  --served-model-name VultronCoderAtlas \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --max-model-len 1048576 \
  --reasoning-parser kimi_k3 \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3

Qualify the requested context length against available KV-cache memory, concurrency, image inputs, output budget, and the selected KV-cache dtype.

OpenAI-compatible client

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="unused",
)

messages = [
    {
        "role": "user",
        "content": "Inspect this repository and identify the cause of the failing test.",
    }
]

response = client.chat.completions.create(
    model="VultronCoderAtlas",
    messages=messages,
    reasoning_effort="max",
    max_tokens=4096,
)

assistant_message = response.choices[0].message
messages.append(assistant_message.model_dump(exclude_none=True))

Preserve the complete assistant_message in subsequent requests.

Limitations

• The packed checkpoint is approximately 1.2 TB. Deployment requires multi-GPU hardware and a Kimi K3-capable inference stack.
• The one-million-token value is the model context limit. Effective request length depends on KV-cache allocation, concurrency, image inputs, and output budget.
• Agentic evaluations show trial variance. Reported means and trial counts are included where available.
• Release qualification covers English workloads.
• The model can emit incorrect code or invalid tool calls. It can leave fixes incomplete. Run generated changes through the repository's tests and review process.

License

The release uses the Kimi K3 License. Distributed copies and substantial portions of the model must include the upstream copyright and permission notices.