Inkling-975B-Alis-MLX-Dynamic-3.7bpw

The quality / golden-spot tier of the Inkling · Alis MLX Dynamic family — siblings: the ~2.7 bpw size-optimal build and the ~6.6 bpw two-box Q6 performance build. Full family: Inkling 975B · Alis MLX Dynamic collection.

Apple Silicon (MLX) mixed-precision quantization of thinkingmachines/Inkling — a 975B-class multimodal Mixture-of-Experts model (66 hybrid decoder layers, 256 routed experts (top-6) + 2 shared per MoE layer, hidden 6144, sliding-window attention + short-convolution hybrid, vision + audio front-ends, 201K vocab).

This build targets the golden spot for a single 512 GB M3 Ultra: near-teacher quality in one box — ~3.71 bits/weight, with the entire non-expert skeleton kept at exact BF16, and quantization scales certified by a fully receipt-sealed, layer-local ALIS-DWQ pass run as a distributed two-Mac pipeline.

---

Quality-first recipe

Bits go where the parameters are; exactness stays where the control flow is. ~96% of weights (the routed expert bank) take the cheap bits; everything a token's routing, attention, or modality path depends on stays BF16 — a stricter split than most mixed builds, which quantize attention too.

Component	Precision	Share	Why
Routed experts w13/w2 (layers 3–65)	3-bit affine, g128	345 GiB	the parameter bulk
Attention (all 5 proj., hybrid + sconv)	BF16	—	every token's critical path
Router / gates	BF16	—	discrete top-6 routing — never quantized
Shared experts	BF16	—	on every token
Embeddings · LM head · norms	BF16	—	distribution-sensitive
Vision + audio towers	BF16	—	modality front-ends kept exact
Layers 0–2 (entirely)	BF16	—	early-layer protection

Dropped: the source's MTP head (model.mtp.*, 5.26B params) — a speculative-decoding accessory not used by the MLX runtime. All three family tiers drop it identically, which is why they report 947B logical params vs the source's 952B.

Base model	thinkingmachines/Inkling (975B-class; 952.4B source / 947.1B logical params)
Bits/weight	3.706 effective (439 GB total)
On-disk size	438.8 GB (408.7 GiB), 108 shards
Format	MLX safetensors (inkling_mm_model)
Modalities	text + image + audio inputs
Attention	sliding-window (512) hybrid + short conv — KV stays small at long context

---

What makes this build different: a sealed, audited DWQ pass

Most public quants ship converted weights. This one ships converted weights plus a machine-checkable certification trail. The quantization scales/biases were passed through layer-local ALIS-DWQ (alis-dwq) against exact BF16 teacher activations, under a guard-and-receipt harness originally built for reproducible two-box runs:

• BF16 teacher boundaries, not logits-only: the full-precision teacher was run as a distributed pipeline across two 512 GB Macs (layers 0–32 / 33–65), dumping per-layer input/target activations (h₀…h₆₆) for 72 calibration batches across text, image, and audio — so every decoder layer trains against its own exact teacher boundary, per modality.
• Layer-local, memory-bounded training: each layer is strict-loaded alone (never the full model) and its affine scales/biases tuned with stop-gradient teacher boundaries and valid-token NMSE. Differentiating a 256-expert quantized gather naively materializes ~150 GiB of dequantized workspace; this pass uses an expert-group × token-block serialized backward that provably matches the fused gradients (cosine ≥ 0.9985) at a ~23 GiB peak — the whole optimization ran inside a watchdog envelope of min 90% system RAM free, zero swap growth.
• Do-no-harm acceptance, per layer: a layer's new scales are kept only if held-out boundary NMSE does not regress at all (allowed regression: 0.0); otherwise the layer rolls back to baseline, byte-exact. Of 63 tuned layers, 62 certified neutral and layer 40 committed a genuine improvement — that accepted delta is what distinguishes these weights from the raw conversion.
• Every step is evidence: each layer ran as a sealed process chain (frozen runtime bundle → launch pin → guarded watchdog → atomic no-clobber receipts), and the 63 terminal receipts link into a hash chain closed by an advanced-completion receipt (status: pass). Nothing in this repo was produced by an unaudited script run.

The result is conservative by construction: you get provably-not-worse-than-baseline weights with a certified improvement where one was found — not a quant that traded unknown regressions for a benchmark bump. (A higher-lr DWQ tier on the same evidence rails is planned as a separate revision.)

---

Sibling builds

Build	bpw	Size	For
this — quality / golden spot	3.71	409 GiB	single 512 GB Mac, best quality in one box
capacity / size-optimal	2.72	299 GiB	single Mac with generous headroom / smaller boxes
Q6 teacher (two-box)	6.60	728 GiB	maximum fidelity, 2 × 512 GB pipeline serving

---

Usage

The inkling_mm_model architecture is served by the project's MLX port (hybrid sliding-window + short-conv attention, 256-expert gather MoE, multimodal towers); upstream mlx-lm support is not yet merged. Until the port lands upstream, treat this repo as weights + provenance, loadable with the Inkling MLX runtime from the alis pipeline:

# Inkling MLX runtime (alis pipeline port)
from inkling_mlx.model import load_model
model = load_model("avlp12/Inkling-975B-Alis-MLX-Dynamic-3.7bpw")

Sliding-window attention keeps the KV cache small, so long contexts are memory-cheap relative to dense-attention peers; the binding constraint on a 512 GB box is the 409 GiB of weights, which fit with room for activations and a long-context cache.

Hardware

Built for 512 GB Apple Silicon (M3 Ultra). For ≤ 320 GB machines, use the upcoming 2.7 bpw capacity build.

Validation status

The DWQ schedule itself is complete and certified (63/63 sealed layer receipts + advanced-completion pass). The remaining project gates — two-box candidate held-out evaluation, Q6 final gates, runtime verification, and final quality validation — are running now; this card will be updated with their measurements (held-out NMSE table, perplexity harness numbers) as they land. Until then, treat quality claims as certified-not-worse than the baseline conversion rather than benchmarked.

Credits

• Base model: Thinking Machines — Inkling (Apache-2.0).
• MLX: Apple ml-explore.
• Two-box BF16 boundary pipeline, layer-local streamed-backward DWQ, sealed evidence harness, and the MLX inkling_mm_model port: Alis (avlp12).

Citation

Alis (avlp12) (2026). Inkling-975B-Alis-MLX-Dynamic-3.7bpw — 3.7 bpw certified layer-local DWQ MLX quantization of Inkling. https://huggingface.co/avlp12/Inkling-975B-Alis-MLX-Dynamic-3.7bpw