Inkling-975B-Alis-MLX-Dynamic-6.6bpw

The performance / maximum-fidelity tier of the Inkling · Alis MLX Dynamic family — a two-box pipeline build shipped as ready-to-serve stage artifacts. Siblings: the 3.7 bpw quality build (single 512 GB Mac) and the 2.7 bpw capacity build. Full family: Inkling 975B · Alis MLX Dynamic collection.

Apple Silicon (MLX) Q6 quantization of thinkingmachines/Inkling — a 975B-class multimodal Mixture-of-Experts model (66 hybrid decoder layers, 256 routed experts (top-6) + 2 shared per MoE layer, hidden 6144, sliding-window attention + short-convolution hybrid, vision + audio front-ends, 201K vocab).

This build answers the question the other tiers can't: what does Inkling sound like when you stop trading anything away? ~6.60 bits/weight, 728 GiB across two 512 GB Macs — routed experts at a near-transparent 6-bit, and every other tensor in the model byte-exact BF16. In the Alis pipeline this artifact is the maximum-fidelity reference tier: the closest distributable approximation of the BF16 model.

---

Recipe: one rule, everything else exact

Component	Precision	Tensors	Why
Routed experts w13/w2 (layers 3–65)	6-bit affine, g128	126	the parameter bulk, at near-transparent width
Everything else — attention, routers/gates, shared experts, dense MLP, embeddings, LM head, norms, vision + audio towers, layers 0–2	BF16 (byte-exact)	1,266	nothing on any control or modality path is quantized

Dropped: the source's MTP head (model.mtp.*, 5.26B params) — a speculative-decoding accessory not used by the MLX runtime. All three family tiers drop it identically, which is why they report 947B logical params vs the source's 952B.

Quantized with the alis-clip quantizer (per-group clipping search from the alis-dwq toolchain) rather than plain rounding, with per-shard checksum verification receipts at write time.

Base model	thinkingmachines/Inkling (975B-class; 952.4B source / 947.1B logical params)
Bits/weight	6.60 effective (781.2 GB total)
On-disk size	781.2 GB (727.6 GiB) — stage0 363 GiB + stage1 365 GiB
Format	MLX safetensors (inkling_mm_model), two pipeline-stage artifacts
Modalities	text + image + audio inputs
Serving	2 × 512 GB Apple Silicon, MLX pipeline parallelism

---

Two-box layout: stage artifacts, not a monolith

This repo ships the model the way a two-box pipeline actually consumes it:

stage0/   # pipeline rank 1 — layers 0–33 + embeddings + vision/audio towers (103 shards)
stage1/   # pipeline rank 0 — layers 33–66 + final norm + LM head (109 shards)

The repo-root config.json is the full-model reference config (partition-free); the loadable artifacts are the stage folders. Each stage folder is a self-contained MLX artifact (own config.json + weight index) whose config declares its pipeline partition. The loader prunes to stage-owned tensors and strict-checks the partial index — each host downloads and verifies only its ~365 GiB half:

# host A (rank 1)                      # host B (rank 0)
from inkling_mlx.model import load_inkling_model
model, cfg = load_inkling_model(       model, cfg = load_inkling_model(
    ".../stage0",                          ".../stage1",
    pipeline_group=group)                  pipeline_group=group)

with group = mx.distributed.init() over the two hosts (the Inkling MLX runtime's pipeline serving path; upstream mlx-lm support for inkling_mm_model is not yet merged — treat this repo as weights + provenance until the port lands upstream).

---

Role in the family

The Inkling · Alis DWQ tiers were trained against exact BF16 teacher activations dumped by a distributed two-Mac pipeline; this Q6 build is the same pipeline's maximum-fidelity distributable tier, and serves as the high-precision reference in the project's held-out evaluation gates. Unlike its siblings it carries no DWQ pass — at 6-bit with clip-search and a fully BF16 skeleton, the conversion is already near the noise floor the DWQ contract measures against.

Build	bpw	Size	For
quality / golden spot	3.71	409 GiB	single 512 GB Mac, best quality in one box
capacity / size-optimal	2.72	299 GiB	320 GB+ Macs, maximum headroom
this — performance / two-box Q6	6.60	728 GiB	2 × 512 GB pipeline, maximum fidelity

Validation status

Both stage conversions are complete with sealed per-shard receipts (conversion_state.json: complete, alis-clip, per-shard checksum verification at write). The project's remaining gates — two-box candidate held-out evaluation (which uses this build as the Q6 reference) and final quality validation — are in progress; this card will be updated with their measurements as they land.

Credits

• Base model: Thinking Machines — Inkling (Apache-2.0).
• MLX: Apple ml-explore.
• Two-box BF16 boundary pipeline, alis-clip quantizer, sealed evidence harness, and the MLX inkling_mm_model port: Alis (avlp12).

Citation

Alis (avlp12) (2026). Inkling-975B-Alis-MLX-Dynamic-6.6bpw — two-box Q6 MLX quantization of Inkling. https://huggingface.co/avlp12/Inkling-975B-Alis-MLX-Dynamic-6.6bpw