infosave/Qwen3.6-35B-A3Bcmf

Qwen3.6-35B-A3B — CMF q4tp (with the MTP head)

One 18.7 GB file, mmap-run, CPU/GPU/UMA from the same bytes.

Converted from Qwen/Qwen3.6-35B-A3B (Apache-2.0) to the CMF format in the q4tp layout — 4-bit tiles with predicted per-row scale ladders, ~7% smaller than plain tiled q4 at near-identical quality. Unlike most conversions this file keeps the model's multi-token-prediction head (one full MoE layer; 35.51B parameters total, MTP: 1 block(s) at load).

Architecture: 40 layers (10 full attention + 30 GatedDeltaNet linear), 256 routed experts top-8 + shared expert, vocab 248 320, head_dim 256.

Requires cortiq 0.5.42+ (0.5.43 for the speed)

cortiq 0.5.41 and earlier cannot load this file — their loader predates MoE MTP heads and their GPU graph declines the q4tp layout. Use 0.5.43 for the speeds below (0.5.42 loads the file but runs the older kernels):

cargo install cortiq-cli
hf download infosave/Qwen3.6-35B-A3Bcmf qwen36-35b-a3b-q4tp.cmf --local-dir .   # pip install -U huggingface_hub

Run it

# Since cortiq 0.5.48: no flags. The GPU, its VRAM budget and the
# device/host split are detected and sized automatically.
cortiq run qwen36-35b-a3b-q4tp.cmf

# fast prompt ingest              # long context, O(1) attention on GPU
CMF_BATCH_K=32 CMF_MTP=0 ...      CMF_O1_GPU=1 ... --o1 all

# OpenAI-compatible server (works with Cline/Roo-style clients)
cortiq serve qwen36-35b-a3b-q4tp.cmf

Any VRAM size (cortiq 0.5.48+, dynamic expert loading)

The graph is no longer all-or-nothing: as many leading layers as fit stay on the card, the host finishes the rest — one boundary crossing per token. Measured on the RTX PRO 6000 with the budget capped to what each card size would auto-detect (CPU reference 32.6 tok/s, 48 cores, 0.5.50):

VRAM	4 GB	6 GB	8 GB	12 GB	16 GB	24 GB+
tok/s	26.9	38.3	43.6	58.8	86.9	122.4 (full graph)
layers on GPU	4/40	9/40	14/40	24/40	34/40	all

(0.5.49's pipelined submission lifted the whole ladder — the full card went 98.2 → 122.4 with bit-identical greedy output.)

Perplexity is identical to the printed digit on every rung, on the CPU and on the full graph (3.852 over the reference text) — the split moves layers, never the answer. CMF_GPU_VRAM_MB still overrides the budget if you want to cap it by hand.

Measured (cortiq 0.5.43, July 2026)

	hardware	decode	prompt ingest
Vulkan	RTX PRO 6000 Blackwell	122.4 tok/s (0.5.49)	99 tok/s (CMF_BATCH_K=32)
CPU	48-core server	32.6 tok/s (0.5.50)	33 tok/s
CPU	Apple M4 Air 24 GB	18.2 tok/s (CMF_THREADS=8, pre-0.5.50 — expect roughly double)	—

0.5.43's kernel work took this model from 67.8 to 99.6 tok/s on the same card and the same bytes; an RTX 5090 measured 64.5 on the old kernels.

Long context with --o1 all + CMF_O1_GPU=1: flat 53.8 tok/s at ctx 16 384 where exact attention has fallen to 37.8. GPU output is token-for-token identical to the CPU at short context; on very long prompts greedy decoding may pick a different-but-coherent continuation at probability ties (float ordering, as with any GPU offload). o1 is an approximation and carries no bit-parity claim.

Full walkthrough with every flag explained: docs/QWEN36_MOE.md.

File

• qwen36-35b-a3b-q4tp.cmf — 18 678 996 782 bytes, size-verified against the converter output after upload.
• Reproduce (cortiq 0.5.42+): cortiq convert --model Qwen/Qwen3.6-35B-A3B --quant q4tp --output qwen36-35b-a3b-q4tp.cmf — streaming, peak disk = the output file. Converters before 0.5.42 drop the mtp.* tensors and produce a smaller, different file.

Weights derive from Qwen's release and remain under its Apache-2.0 terms. The CMF container and the cortiq runtime are Apache-2.0 as well (see the repository's LICENSE and PATENTS.md).