PureDrive-35B-A3B-MLX — a 35B-A3B MoE coder at 110 tok/s on a single Apple M4 Pro, lossless (MTP speculation + custom Metal attention)

This repository contains documentation only: no model weights. It describes and evidences an inference recipe (patches, kernels, benchmark code) published on GitHub: https://github.com/SerOv66/PureDrive-35B-A3B-MLX. The checkpoint used is peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP (MIT), an MLX "oQ6e" quantization with a native MTP head of an abliterated derivative of Ornith-1.5-35B-A3B (MIT); it is not redistributed here. The license above (Apache-2.0) covers this repository's documentation and recipe only. See Model credits / licenses below.

Install it with your AI coding agent: tell it to read AGENTS.md (also in the GitHub repository) and install the project on your Mac.

Summary

PureDrive is a patched oMLX / MLX runtime for exact, fast local inference on Apple silicon. Single-stream decoding of a 35B-A3B mixture-of-experts coder model (40 layers: 30 Gated-DeltaNet + 10 full-attention; 256 experts, top-8 + shared expert; 6-bit experts, 8-bit attention/shared/embedding/lm_head; native multi-token-prediction head) on one Apple M4 Pro (20-core GPU, 48 GB) with MLX / oMLX: 110.3 tok/s at 2K context and 62.8 tok/s at 128K, +26...+82% over the first lossless baseline. The method is not tied to this checkpoint: it applies to any mixture-of-experts model that ships an MTP head and to any Apple-silicon Mac with enough memory (see Beyond this checkpoint below). Everything is lossless: original weight precision, bf16 KV cache, exact speculative decoding (greedy output identical to non-speculative decoding; sampling exact in distribution), full top-8 experts, dense attention over all keys.

Decode tok/s, median of 3 × 512 tokens, greedy / default sampling	2K	4K	16K	32K	40K	64K	128K	256K*
This work	110.3 / 111.9	107.3 / 107.5	102.6 / 100.9	95.8 / 94.6	90.2 / 86.7	78.3 / 79.0	62.8 / 62.2	45.9 / 44.8
First lossless baseline (oMLX + row-exact MTP)	85.8 / 82.8	85.1 / 78.7	70.8 / 70.0	62.8 / 62.0	60.4 / 57.9	50.6 / 49.1	34.6 / 34.2	-
Speed-up (greedy)	+29%	+26%	+45%	+53%	+49%	+55%	+82%	-

Measured on one Apple M4 Pro, macOS 27.0.1, MLX 0.32.3, oMLX 0.7.0 (commit cc1fdc9) with the published patch, using a frozen benchmark harness (exact context lengths, real code prompts, KV filled with real tokens, independent audit). 128K constancy test (2,048 tokens) 63.2 tok/s; thermal Nominal; swap +0 MB after 10 minutes of continuous generation. * 256K: measured with the previous certified configuration (cand4, EXP-105/106) in a separate harness, outside the frozen benchmark harness.

Quality

Check	Result
Greedy output with vs without speculation	identical: 57/57 coding problems, 6/6 fixed prompts + one 32K prompt, long generations at 8K-110K context
Sampling	exact in distribution (chi-square tests)
Coding (HumanEval+ 28 + MBPP+ 29)	48 / 57 (stock runtime without speculation 47 / 57)
Reasoning (30 GSM8K problems)	28 / 30 (stock 28 / 30)
Tool calling (14 strict cases)	14 / 14 (stock 14 / 14)
Needle retrieval at 32K / 64K / 131K	3 / 3 (stock 3 / 3); at 262K (cand4): needle, multi-key, multi-value 3/3 each
Attention accuracy vs fp32 reference (KL)	0.004-0.054 nats, same band as the stock kernel (0.006-0.032)

Other runtimes on the same weights and machine (single-run, 256-token reference measurements)

Setup	2K	16K
mlx-vlm 0.7.6, no speculation	60.9	54.3
oMLX main, speculation off	72.95	63.74
llama.cpp, 5-bit GGUF variant, MTP off / on	52.27 / 55.83	47.03 / 59.89
This work (matrix medians, greedy / default)	110.3 / 111.9	102.6 / 100.9

How it works

• Exact speculative decoding with the checkpoint's own MTP head: the head drafts 2 tokens; the target verifies 3 rows in one pass, reading its weights once. About 2.3 tokens are emitted per cycle.
• Row-exact verification: every row of the multi-row verify pass is bit-identical to a one-row decode at the same position, so MTP-on equals MTP-off for greedy decoding. (The stock runtime's multi-row kernels were not: they diverged at token 4-5.)
• Custom Metal GQA attention kernel: K/V read once for all 8 query heads and all verify rows; split-K ("flash-decoding" style) with split points and summation order that depend only on the row's position; wave-aligned split sizes for the 20-core GPU. 2-3× faster than MLX SDPA on 3-8 row verifies.
• Sampling mode uses standard speculative-sampling acceptance, a lower draft temperature and block verification (Sun et al., ICLR 2025); exactness is verified by χ² tests.
• Scheduling and fusions: verify graph pipelining, pre-queued draft, compiled draft sampler, fused add+RMSNorm, fused q/k-norm + RoPE, GDN recurrence and projection-tile retunes — each bit-identical or exact in distribution.
• Draft-side only: a 32K-token draft vocabulary and a 4096-token attention window on the MTP head; the target always verifies with the full lm_head and full attention.

Beyond this checkpoint: any MoE model with an MTP head

The speed-up does not depend on this particular checkpoint. Speculative decoding with the model's own multi-token-prediction (MTP) head, made exact by a row-exact verify pass, needs no separate draft model and no training, so the method applies to every mixture-of-experts model that ships an MTP head, and the same frozen-benchmark and bit-identity methodology certifies the result on each new model.

Scope	What carries over	What to do
Same architecture (qwen3_5_moe, 35B-A3B): Qwen3.6-35B-A3B, Qwen3.5-35B-A3B, Ornith-1.5-35B-A3B and their fine-tunes	patch, kernels and configuration as they are	use an MLX checkpoint that keeps the MTP head (language_model.mtp.*). The original bf16 weights include it; the public MLX conversions checked on 2026-10-11 leave it out, so quantize from the original weights. Then re-run the identity tests and the benchmark harness.
Other MoE models with a native MTP head (for example larger Qwen3.5 MoE models, Qwen3-Next, the DeepSeek-V3 family, GLM-4.5)	the exact speculative loop (row-exact verification, exact greedy and sampling acceptance with block verification, pre-queued draft, verify pipelining), row-exact quantized mat-vec, the benchmark and audit methodology	adapt the model-specific parts: the attention kernel is compiled for 16 query heads / 2 KV heads / head dim 256 (other GQA shapes need re-instantiating and re-tuning; multi-head latent attention needs its own kernel), the MTP-head wiring, and the Gated DeltaNet kernels (used only by hybrid linear-attention models)

Hardware.

• Any Apple-silicon Mac with enough memory. The runtime (oMLX) supports M1–M5 Macs on macOS 15 or later; the numbers here were measured on one M4 Pro (20-core GPU, 48 GB, 273 GB/s). This model needs about 28 GB for the weights plus the bf16 KV cache (about 20 KB per token: 2.7 GB at 128K, 5.4 GB at 256K), so 48 GB Macs and larger hold it; 64–512 GB Macs can hold the larger MTP models above. Decode speed follows memory bandwidth and GPU size. The attention split sizes are tuned for 20 GPU cores: they stay exact on any chip, and re-tuning only changes speed. On other chip generations, re-run the bundled kernel and identity tests first.
• Other unified-memory machines. NVIDIA DGX Spark (GB10, 128 GB, 273 GB/s) and AMD Ryzen AI Max+ 395 "Strix Halo" (up to 128 GB, 256 GB/s) are in the same memory-bandwidth class as the M4 Pro, with more memory, which makes them natural targets for the same approach. The method is hardware-independent; this code is Apple-specific (oMLX runs on macOS, the kernels are Metal), so those machines need a port: on DGX Spark through MLX's CUDA backend or a CUDA runtime with MTP support, on Strix Halo through a ROCm or Vulkan runtime.

How to use

The recipe is a patch to oMLX at commit cc1fdc9 plus a Metal kernel module and a set of environment-flag configurations. Instructions, benchmark code, the frozen benchmark harness, audit tools and raw results are in the GitHub repository: https://github.com/SerOv66/PureDrive-35B-A3B-MLX (start with AGENTS.md). Download the checkpoint peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP (6-bit experts, MTP layer stored as language_model.mtp.*). Another checkpoint of the same architecture and quantization layout also works, but acceptance rates, and therefore speed, depend on the fine-tune.

git clone https://github.com/SerOv66/PureDrive-35B-A3B-MLX PureDrive-35B-A3B-MLX && cd PureDrive-35B-A3B-MLX
scripts/setup_omlx.sh        # oMLX @ cc1fdc9 + certified patch + Metal kernel + venv
uvx --from huggingface_hub hf download peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP --local-dir models/Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP
mkdir -p models/omlx-models && ln -sfn "$PWD/models/Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP" models/omlx-models/coder-q6
./serve.sh                   # OpenAI-compatible API: http://127.0.0.1:8000/v1, model "coder-q6"

Full guide with checks and troubleshooting: AGENTS.md. Reproducing the measurements: docs/REPRODUCE.md in the GitHub repository.

Agentic coding

Pi 0.73.1 is the recommended command-line coding agent on this server (evaluated together with Hermes Agent v0.20.4). On 4 short coding tasks × 3 repetitions it succeeded 12/12 with a median wall time of 15-26 s per task (1.8-3.1× lower than the comparison harness, which also succeeded 12/12), prefix-cache hit 77-95%, and it completed a long session (>= 64K context) in both runs. Decode rates seen in this phase (about 105-126 tok/s) are server-reported counters on short completions, indicative only and not comparable to the table above. Details, tasks and scripts: harness/ in the GitHub repository.

Terminal-Bench 2.1

Evaluated with the vendor's harness (Harbor + Terminus-2), run entirely on the Mac:

Model	Setup	Terminal-Bench 2.1 (Terminus-2)
This work (6-bit, one M4 Pro)	30-task pre-registered subset, 1 run per task	73.3% (22/30, 95% CI 55.6–85.8)
Ornith-1.5-35B-A3B (original, vendor-reported)	89 tasks, mean of 5 runs	67.8%
Qwen3.6-35B-A3B (reported on the Ornith-1.5 card)	89 tasks	52.5%

In line with the original model (67.8 lies inside our confidence interval). Subset evaluation, not a leaderboard submission: 30 tasks drawn with a fixed seed from the 72 this machine can run, one run each, time limits × 3 (max 90 min), containers under x86-64 emulation. Decode during the run: about 110 tok/s below 8K context, 85 tok/s at 32–64K, 63 tok/s at 96–132K. Protocol and per-task table: docs/TERMINAL_BENCH.md in the GitHub repository.

FAQ

How fast does a 35B mixture-of-experts model run on an Apple M4 Pro? With PureDrive, single-stream decode of the 6-bit 35B-A3B coder on an M4 Pro (20-core GPU, 48 GB) runs at 110.3 tok/s at 2K context, 102.6 at 16K, 95.8 at 32K, 78.3 at 64K and 62.8 at 128K (greedy medians of 3 × 512 tokens, frozen benchmark harness), and 45.9 tok/s at 256K (previous certified configuration). During a real agentic workload (Terminal-Bench, 884 requests) decode stayed at 110 tok/s below 8K context and 63 tok/s at 96–132K.

Does the speed-up change the model's output? No. Greedy output is byte-identical to non-speculative decoding (57/57 coding problems, long generations up to 110K context), and sampling is exact in distribution (speculative-sampling acceptance plus block verification, checked with χ² tests). Weights keep their original precision, the KV cache is bf16, attention is full over all keys, and all 8 routed experts run.

Which model does it run? Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP, a 6-bit MLX ("oQ6e") checkpoint with a native multi-token-prediction head, derived from Ornith-1.5-35B-A3B. Architecture: Qwen3.5/3.6-35B-A3B (qwen3_5_moe: 40 layers = 30 Gated DeltaNet + 10 full attention, 256 experts top-8 plus a shared expert, about 3B active parameters per token).

Can it speed up other MoE models, such as Qwen3.6-35B-A3B or Ornith-1.5? Yes, that is the point of the method: it uses the model's own MTP head, so it applies to any mixture-of-experts model that ships one. Checkpoints of the same architecture (qwen3_5_moe 35B-A3B: Qwen3.6, Qwen3.5, Ornith-1.5 and fine-tunes) use the code as it is, given an MLX conversion that keeps the MTP head; other MoE models with MTP heads (Qwen3-Next, the DeepSeek-V3 family, GLM-4.5, larger Qwen3.5 models) reuse the exact speculative loop and the methodology, with the attention kernel and MTP wiring adapted. Speed on each model depends on how often its MTP drafts are accepted, and is measured with the same benchmark harness.

Does it run on other Macs, or on DGX Spark / Ryzen AI Max machines? Any Apple-silicon Mac on macOS 15 or later with enough unified memory (48 GB or more for this model) runs it; speed follows memory bandwidth and GPU size. NVIDIA DGX Spark and AMD Ryzen AI Max+ 395 have the same bandwidth class as the M4 Pro and 128 GB of unified memory, so the method fits them too, but the code (oMLX, Metal kernels) is Apple-specific and needs a port there.

What Mac do I need? Apple silicon. The numbers were measured on an M4 Pro (14-core CPU, 20-core GPU, 48 GB, macOS 27.0.1); the weights take about 28 GB, and 48 GB covers contexts up to 128K as is and 256K with a raised GPU wired-memory limit. Other chips run it at different speeds (kernel split sizes are tuned to 20 GPU cores); machines with less than 48 GB are untested.

How do I connect my coding agent? ./serve.sh exposes an OpenAI-compatible API at http://127.0.0.1:8000/v1 (model coder-q6, any API key, tool calling, 262,144-token context window). Tested with Pi (12/12 coding tasks), Hermes Agent and Terminus-2. See AGENTS.md.

How does it compare with llama.cpp, mlx-vlm and stock oMLX? On the same Mac and weights: mlx-vlm 60.9 tok/s at 2K, stock oMLX 72.95, llama.cpp with a 5-bit GGUF and MTP 55.83; PureDrive 110.3 (the other runtimes are single 256-token runs, shown for orientation).

How good is it at agentic coding? 73.3% on Terminal-Bench 2.1 (22 of 30 pre-registered tasks, 95% CI 55.6–85.8, Terminus-2 agent), with the model and every task container running on the Mac; the original model's vendor-reported score is 67.8%.

What makes it fast? Exact speculative decoding with the model's own MTP head and a row-exact verify pass; a custom Metal split-K GQA attention kernel that reads the KV cache once for all heads and verify rows; verify pipelining, a pre-queued draft and bit-identical kernel fusions.

Can an AI agent install it for me? Yes. Tell your coding agent to read AGENTS.md and install the project: it contains the commands, the expected output of every step and troubleshooting.

Notes on measurement

• Numbers are for one machine (M4 Pro, 20 GPU cores, 48 GB), single stream, one request at a time. Kernel split sizes are tuned to a 20-core GPU; other chips will differ.
• The custom attention kernel changes the summation order relative to stock MLX SDPA; accuracy against an fp32 reference is in the same band as the stock kernel (KL 0.004-0.054 nats depending on context, noise-dominated), and the quality gate uses a "numeric" profile (at most 2 coding/reasoning problems may drop) for that reason. Losslessness (MTP-on == MTP-off) is certified with the prefix cache off; with it on, prompt chunking can change greedy text even without speculation.
• Speed varies with content: acceptance is higher on repetitive code. The three runs per cell use three different real coding prompts; spreads are given in the repository.
• 2K / 4K cells vary by ±4-5% with prefix-cache state.
• The work was carried out by autonomous AI coding agents with independent AI auditors under a written specification; all claims are backed by the recorded measurements and tests in the repository.

Citation

@misc{serov66_puredrive_2026,
  title        = {PureDrive-35B-A3B-MLX: exact fast decoding of a 35B-A3B mixture-of-experts coder model on Apple M4 Pro: MTP speculation and a custom Metal attention kernel},
  author       = {SerOv66},
  year         = {2026},
  howpublished = {\url{https://github.com/SerOv66/PureDrive-35B-A3B-MLX}},
  note         = {Model card: SerOv66/PureDrive-35B-A3B-MLX}
}

Model credits / licenses

The measurements use peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP (MIT), downloaded from Hugging Face; its weight files are byte-identical (sha256) to the checkpoint measured here. Lineage: Ornith-1.5-35B-A3B (ornith-ai, MIT) → Huihui-Ornith-1.5-35B-A3B-abliterated (huihui-ai, MIT; refusal-direction removal) → Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP (peculiar-ragdoll, MIT; Ornith's bf16 MTP head grafted back, oMLX oQ6e requantization). The architecture is the Qwen3.5/3.6-35B-A3B family (Qwen, Apache-2.0).

Component	Author	License
Ornith-1.5-35B-A3B (base model)	ornith-ai	MIT
Huihui-Ornith-1.5-35B-A3B-abliterated	huihui-ai	MIT
Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP (the checkpoint used)	peculiar-ragdoll	MIT
Qwen3.5/3.6-35B-A3B (architecture family)	Qwen	Apache-2.0
oMLX (inference runtime the patches apply to)	oMLX contributors (source)	Apache-2.0
Terminal-Bench 2.1, Harbor, Terminus-2 (agentic benchmark and harness)	harbor-framework (source)	Apache-2.0
"Sharp" chat template v22.5.0 (embedded in the checkpoint, used as shipped)	froggeric	as distributed with the checkpoint
This repository (patches, kernels, benchmark code, docs)	see the GitHub repository	Apache-2.0

All results in this repository are for this derivative served by the runtime described here; scores published by the model vendors for other models are not ours and are not claimed.

Built on: oMLX (Apache-2.0), MLX, mlx-lm, mlx-vlm; speculative sampling (Leviathan et al. 2023; Chen et al. 2023), block verification (Sun et al., ICLR 2025), Gated DeltaNet (Yang et al., ICLR 2025). Full references in the GitHub README.