# Glossary

Shared vocabulary across packages. Prefer package `derived/SPEC.md` for numbers.

| Term | Meaning |
|---|---|
| **MoE** | Mixture of Experts — sparse FFN; only some experts run per token |
| **Active params** | Parameters used for a typical forward (not total stored weights) |
| **Shared expert** | Expert(s) always on, often combined with routed experts |
| **Shared-expert sink** | Shared experts share normalized routing mass with top-k (Inkling) |
| **Shared-expert add** | Shared expert output **added** after routed experts (Laguna) — not sink mass |
| **Sigmoid router** | Token-choice scores via σ(logits), not softmax (Laguna `LagunaTopKRouter`) |
| **Softplus attention gate** | Per-head (or per-element) softplus gate on attn output before `o_proj` (Laguna) |
| **QK-norm** | RMSNorm on Q and K per head before RoPE (Laguna) |
| **YaRN** | Yet another RoPE extensioN — long-context RoPE scaling (Laguna global layers) |
| **OpenMDW** | Open Model Deep Wide license family (Laguna S 2.1: OpenMDW-1.1) |
| **pool** | Poolside agent harness; native training/eval environment for Laguna |
| **DFlash** | Poolside speculative draft model pair for Laguna serving |
| **poolside_v1** | Tool-call + reasoning parser name for vLLM/SGLang Laguna serve |
| **Quantile Balancing** | Expert load balance from router-score quantiles (Kimi K3 claim) |
| **Stable LatentMoE** | K3 named MoE framework at 16/896 sparsity |
| **SWA** | Sliding-window attention — local context only |
| **MLA** | Multi-Head Latent Attention — compressed KV (DeepSeek-style; K2.x) |
| **KDA** | Kimi Delta Attention — linear/hybrid attention for long sequences |
| **AttnRes** | Attention Residuals — selective depth retrieval (K3) |
| **Relative PE** | Position via attention bias vs distance, not RoPE (Inkling) |
| **RoPE** | Rotary positional embeddings (common baseline) |
| **SConv** | Short causal convolution on residual branches (Inkling) |
| **hMLP** | Hierarchical MLP patch encoder for images (Inkling) |
| **dMel** | Discrete mel-bin audio tokens (Inkling) |
| **MTP** | Multi-token prediction / speculative draft heads |
| **μP / width multiplier** | Logit scaling by model width factor (Inkling `logits_mup_width_multiplier`) |
| **QAT** | Quantization-aware training |
| **Thinking effort** | Controllable test-time compute / CoT budget |
| **Preserved thinking** | Multi-turn must return prior reasoning content (K3 / Laguna `preserve_thinking`) |
| **Harness overfitting** | Model recalls native tool schemas instead of third-party definitions (Laguna limit) |
| **Tinker** | Thinking Machines fine-tune platform |
| **Primary source** | Official blog, model card, Hub config, paper — not secondary recaps |