# Inkling vs Kimi K3

Cross-model study note for the monorepo.  
**Sources:** each package’s `derived/SPEC.md` and primary references — not invented parity.

Also: **[Laguna S 2.1 · Inkling · Kimi](laguna-s-vs-inkling-vs-kimi.md)** (tri-compare, includes TB2.1 peer table).

| Axis | Inkling | Kimi K3 | Notes |
|---|---|---|---|
| **Org** | Thinking Machines Lab | Moonshot AI (Kimi) | Different product theses |
| **Released** | 2026-07-15 (weights) | API ~2026-07-16; **weights ETA 2026-07-27** | K3 study is pre-weights |
| **Total params** | ~**975B** | **2.8T** | Both open-class flagships |
| **Active params** | **~41B** | **undisclosed** | K3 sparsity known without FLOP count |
| **MoE** | 256 routed, top-**6**, **2** shared; shared-expert **sink** | Stable LatentMoE **16 / 896**; **Quantile Balancing** | K3 denser expert pool, higher top-k |
| **Context** | **1M** | **1M** | Same headline window |
| **Attention (length)** | Relative PE (not RoPE) + hybrid **SWA/global** | **Kimi Delta Attention (KDA)** hybrid linear | Different long-context bets |
| **Attention (depth)** | Standard residual + **SConv** branches | **Attention Residuals (AttnRes)** | Inkling local conv vs K3 depth retrieve |
| **Layer schedule** | 66 layers; **55 SWA / 11 global**; L0–1 dense FFN | TBD (tech report) | Inkling fully config-backed |
| **Multimodal** | Native **image + audio** (hMLP + dMel) early fusion | Native **vision + video** (API); audio not emphasized | Different modality priorities |
| **Thinking control** | Continuous **effort** 0–0.99 in chat template | Always-on thinking; API `reasoning_effort` (**max** only at launch) | Both “controllable compute,” different UX |
| **Serving numerics** | BF16 / NVFP4 / MXFP8 mentioned | QAT **MXFP4 W / MXFP8 A** from SFT | K3 trains quantized earlier |
| **Open implementation** | Day-0 **transformers** + SGLang/vLLM/… | KDA → **vLLM** cache work; full OSS stack with weights | Inkling study includes vendored HF code |
| **Customization wedge** | **Tinker** fine-tune platform | Kimi Code / Work / Enterprise products | Lab vs product-suite framing |
| **Positioning** | Broad customizable base; not closed #1 | Open frontier; still trails Fable 5 / GPT 5.6 Sol | Both honest about not owning closed peak |
| **Critical footgun** | Residual jailbreak / app-layer safety | **Preserved thinking history** required or quality collapses | Different production risks |

---

## Architectural mental models

### Inkling — “sparse decoder + early fusion”

```
text|image|audio → shared residual d=6144
  → hybrid SWA/global attention + relative R + SConv
  → MoE 6/256+2
  → text out (+ effort dial)
```

Study depth lives in **config-backed** layer schedules and HF modular code.

### Kimi K3 — “linear length + residual depth + extreme MoE”

```
text|vision|video → KDA (sequence) + AttnRes (depth)
  → Stable LatentMoE 16/896
  → always-on thinking → text/tools
```

Study depth is **docs/API + KDA paper** until weights/config ship.

---

## When to study which first

| Goal | Prefer |
|---|---|
| Read real open weights + transformers path | **Inkling** |
| Long-horizon coding agents + 1M product API now | **Kimi K3** |
| MoE routing variants (sink vs quantile) | Both |
| Linear / delta attention theory | **Kimi K3** (+ arXiv:2510.26692) |
| Relative PE + SWA hybrid practice | **Inkling** |
| Multimodal audio | **Inkling** |
| Multimodal video product path | **Kimi K3** |

---

## Open gaps that block a fair “who wins”

1. K3 **active params** and full layer table  
2. K3 public **numeric** bench table (blog is figure-heavy)  
3. Independent harness-matched agentic coding comparison  
4. Self-host cost at 1M for both (Inkling has VRAM guides; K3 says ≥64 accelerators)  

Track K3 gaps in [`models/kimi-k3/derived/open_questions.md`](../../models/kimi-k3/derived/open_questions.md).

---

## Package entrypoints

| | |
|---|---|
| Inkling | [`models/inkling/`](../../models/inkling/) |
| Kimi K3 | [`models/kimi-k3/`](../../models/kimi-k3/) |
| Live table | [`derived/DASHBOARD.md`](../../derived/DASHBOARD.md) |