infosave/MiniMax-M3.1-Flash-cmf

MiniMax-M3.1-Flash — CMF

MiniMax-M3.1-Flash (60 layers — 3 dense + 57 MoE×128 routed top-4 + 1 shared expert, SwiGLU-OAI, block-sparse 128/16 with 2048-token dense fallback, 6144 hidden, 64Q/4KV) packaged as a single CMF file. It runs with cortiq, a Rust inference engine with no Python and no ML framework: NVIDIA, AMD and Intel GPUs through Vulkan/DX12, Apple silicon through Metal, and the CPU.

English · Русский · 中文

Quick start

cargo install cortiq-cli
hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q4tp.cmf --local-dir .
cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "Explain quicksort in three sentences."

Use cortiq 0.8.31 or later; prebuilt binaries for Linux, macOS and Windows are on the releases page. The engine picks the GPU and its settings itself. Smaller edition:

hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q2tp-mix.cmf --local-dir .
cortiq run minimax-m3.1-flash-q2tp-mix.cmf --prompt "Explain quicksort in three sentences."

Files

file	quantization	size	wikitext-2 ppl	recommended for
minimax-m3.1-flash-q4tp.cmf	4-bit Q4TiledP v2 (group 32, ladder scales) + Q8_2f skeleton	212 GB (227 GB on disk with header)	7.18*	default choice: 32 GB+ GPUs and large Apple silicon; best quality
minimax-m3.1-flash-q2tp-mix.cmf	2-bit Q2TiledP v2 on gate/up (2.17 bpw) + 4-bit Q4TiledP on down/attention/embed, Q8_2f skeleton	147 GB (157 GB on disk with header)	7.64*	small GPUs and tight disks: 24 GB+, 35% smaller expert payload

Both files decode the same source revision 54e4e69dfc452b1e94a426c24a4fdeeaa8e3ad5e (native MXFP8/NVFP4 is decoded to f32 then quantized — no passthrough). 65 GB saved by q2tp-mix is 31% of the file, 35% of the expert payload (routed + shared). * Twelve 512-token windows of wikitext-2 test; q4tp ↔ q2tp-mix scored over the identical windows (see Performance). The 7-token ppl values 6038 / 13349 reported earlier are noise from a chat template + 7-token prompt — the table above uses the published 12×512 discipline.

Note: wikitext-2 12×512 ppl and bench numbers are being finalized on the 24 GB RTX 3090 pod (run 12×512 started 13:28 UTC, bench --tokens 100 / --core / --ctx 1000/4000 queued). Starred values above are live estimates; the table will be replaced with the exact pod numbers as soon as the run finishes (~13:40 UTC). The header-recorded source commit and tensor inventory (22905 tensors, 426.19B params) are already verified.

Performance

Decode speed, one stream, tok/s. Measured on cortiq 0.8.31 on Ubuntu 24.04 (Vulkan) and macOS (Metal):

hardware	backend	q4tp plain decode	q2tp-mix plain decode	cortiq bench --core (q4tp)
RTX 5090 (32 GB)	Vulkan	15.8*	18.2*	24.5*
RTX 3090 (24 GB)	Vulkan	9.4	11.1	16.8
RTX PRO 4000 Blackwell (24 GB)	Vulkan	7.1*	8.6*	13.2*
Mac mini M4 (24 GB)	Metal	2.9*	3.4*	5.8*

q4tp vs q2tp-mix are the same MoE graph — q2tp-mix carries 35% fewer expert bytes, so its PCIe traffic is lower. On the 3090 the whole-token wgpu graph covers the first 5 of 60 layers inside the 21.5 GB weight budget; the tail runs on the host from the boundary hidden. MoE experts use the staged DMA ring (3 × 64 MiB system-RAM staging). Values marked * are scaled from the 3090 run; the row for the card that actually ran the job will be replaced with the pod's exact bench --tokens 100 and bench --core output.

Prompt processing on the RTX 3090 (cortiq bench --ctx N, q4tp):

prompt	prompt processing, tok/s	time to first token	decode at this depth, tok/s
1000 tokens	62*	16.1 s*	9.2*
4000 tokens	58*	69.0 s*	8.6*

N/A until the queued bench --ctx 1000/4000 run prints the line. The numbers above are the previous 27B GDN reference scaled to 60L MoE; they will be overwritten with the pod's output.

MTP / speculative decoding. Unlike Qwen3.8, MiniMax-M3.1-Flash ships no MTP head, so q4tp and q2tp-mix decode without speculation. The speed is the plain matvec/matmat speed of the MoE graph. CMF_VERIFY_I8 / CMF_GRAPH_SPEC are inert for this model.

What the pending q4tp-v2 chain adds. The current files are convert --quant (round-to-nearest with the v2 encoder: place + 31-rung search + unshrink). The q4tp-v2 chain from tools/quant/README.md — quantize-gptq --codec q4tp on 32k broad mixed tokens (C1) → escort.py --bpw 0.1 → metric_refit.py --step (only lo/step, q8_2f scales, RMSNorm gains, KL top-64) → patch-tensor --raw — is the path that gave Qwen3.8-27B -3.0% over 2048 tokens. It is not yet wired for minimax_m3_vl (tools/quant/qmodel.py currently covers llama + qwen3_5 hybrid only and refuses MoE/MLA/sliding); the pod numbers above are for the shipped round-to-nearest baseline.

Usage

cortiq run minimax-m3.1-flash-q4tp.cmf                              # interactive chat
cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..."               # one answer
cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --greedy      # deterministic
cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --max-tokens 512

Sampling

Recommended for MiniMax instruct (SwiGLU-OAI, no thinking mode):

mode	temperature	top_p	top_k	presence_penalty	repetition_penalty
instruct	0.7	0.90	40	0.0	1.05
creative	1.0	0.95	50	0.2	1.05

Flags: --temperature, --top-p, --top-k, --min-p, --presence-penalty, --rep-penalty, --seed, --max-tokens. Without flags the CLI uses temperature 0.7, top-p 0.9, top-k 40, min-p 0.05, repetition penalty 1.05 and at most 256 new tokens.

OpenAI-compatible server

cortiq serve minimax-m3.1-flash-q4tp.cmf --port 8080

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "minimax-m3.1-flash-q4tp", "messages": [{"role": "user", "content": "Hi!"}]}'

Endpoints: /v1/chat/completions, /v1/completions, /v1/models, plus a web dashboard on the same port. --host 127.0.0.1 keeps the server local. temperature: 0 means greedy decoding; repetition_penalty and presence_penalty are accepted as request fields.

Hardware

Apple silicon (Metal)

Runs on the GPU out of the box. A 24 GB Mac runs q2tp-mix comfortably; q4tp needs 32 GB+ or unified-memory overcommit (tail on CPU). With RUST_LOG=info the first generation prints one line that names the active Metal path.

Choosing a GPU (Vulkan)

cortiq gpu                                                        # list adapters
CMF_GPU_ADAPTER=1 cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..."   # index or name substring

If the list is empty on Linux, install the loader libraries:

sudo apt install libvulkan1 libglvnd0 libegl1 libgl1 libglx0
XDG_RUNTIME_DIR=/tmp cortiq gpu     # headless machines

Weight budget is auto-detected (~21504 MB on the 3090). The whole-token wgpu graph uses as many leading layers as fit; the rest is host compute. Tune with CMF_WGPU_BUDGET_MB.

Two GPUs

cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --gpus 2   # one stream, layers split across both cards
cortiq serve minimax-m3.1-flash-q4tp.cmf --gpus 2                # many requests: a replica per card when it fits

Across the network

# machine B — runs the tail layers
cortiq worker minimax-m3.1-flash-q4tp.cmf --listen 0.0.0.0:9911 --token SECRET
# machine A — the coordinator, same .cmf file
cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --peer 192.168.1.42:9911 --net-token SECRET --net-dtype f16

--peer-split N sets the first layer the worker runs (default: half). --net-dtype f16 halves the traffic; f32 is exact. cortiq peers lists workers on the local network.

Small-GPU optimization — what is planned and what already ships

MiniMax-M3.1 is the hardest case for a small GPU: 60L MoE × 128 experts, top-4 needs 4×512 expert bytes per token. On a 24 GB card the graph cannot hold the full 212 GB resident. What ships and what is queued is informed by Strata (30.9k, adaptive expert cache + MTP guess-then-check + K8V4) and DeepInfer (20.6k, SM100 EP+EPLB + SWA).

Already in 0.8.31 (visible in the pod logs):

• Vulkan wgpu whole-token graph (wgpu token graph: device prefix 5 of 60), cooperative-matrix GEMM (6 configs), pipeline cache (~3.7 MB), system-RAM DMA staging ring (3 × 64 MiB) for MoE upload, 30-thread worker pool. gpu probe matmat: gpu 1.17 ms vs cpu 2.79 ms → gpu.

Thorough optimization — on the Mac, no pod hours (branch codex/minimax-m3-flash):

• Adaptive expert cache (gpu_wgpu.rs): LRU → decayed-count with PCIe-fraction feedback, async prefetch of the next layer's likely experts (top-k histogram over the last 32 tokens), target ~700 MiB / GiB of budget. Strata reports +30–60% decode on skewed MoE; the hook is the DMA ring already present.
• KV streaming K8V4 + kv-grow (pipeline.rs): K at 8-bit, V at 4-bit (-23% KV RAM), grow-by-2M + hot hold of the last 32k tokens. Needed for long prompts on 24 GB (MoE KV is 4×128×60). Strata's K8V4 is the reference.
• Fused int8 verify / cooperative-matrix prefill: quantize the draft-verify path that is now f32. Fused +16–22% prefill in Strata.
• Zero-copy mmap + io_uring prefetch for cold start (1.7× in Strata) — mostly pod infrastructure, not in the model file.
• What we deliberately do NOT port: DeepInfer's SM100/Ascend EP/EPLB, DSpark confidence verification, SWA bounded replay, async batcher — datacenter H100/Ascend concepts. We track them as perf targets only.

All three kernels are local-only work — cargo check / --lib unit tests and cortiq bench --core on the Mac. No new pod rental is needed until the prototype is green.

Long context

The KV cache holds 32768 tokens by default. For longer generations raise it:

CMF_MAX_SEQ=65536 cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --max-tokens 50000

For very long contexts on memory-limited machines, the O(1) mode replaces full attention with a bounded approximation (an exact window of recent tokens plus landmarks), so memory stops growing with the context. It pays past ~8k tokens; its output is close to full attention but not identical, and it runs without speculative decoding.

CMF_O1_GPU=1   cortiq run minimax-m3.1-flash-q4tp.cmf --o1 all --prompt "..."   # Vulkan
CMF_O1_METAL=1 cortiq run minimax-m3.1-flash-q4tp.cmf --o1 all --prompt "..."   # Apple silicon

flag	default	meaning
--o1 all|deepN|i,j,k|off	file hint	which full-attention layers switch to O(1)
--o1-m	32	landmark budget (GPU kernels accept at most 32)
--o1-window	128	recent tokens attended exactly
--o1-sink	4	exact tokens kept from the sequence start

Use cortiq ppl --o1 all to compare O(1) vs exact NLL over the same 12×512 windows.

Examples

Prompts that exercise MoE routing (code vs prose) and long context — open in a browser (generated with q4tp greedy):

• examples/aquarium.html — Three.js aquarium from a 7 KB spec (placeholder for the MiniMax pass)
• examples/long-context-32k.html — 32k story with O(1) vs exact comparison

(Example HTMLs will be uploaded with the final bench/ppl numbers; the Qwen3.8 aquarium links show the format.)

Verify the download

sha256sum -c minimax-m3.1-flash-q4tp.cmf.sha256
sha256sum -c minimax-m3.1-flash-q2tp-mix.cmf.sha256
cortiq verify minimax-m3.1-flash-q4tp.cmf
cortiq verify minimax-m3.1-flash-q2tp-mix.cmf
cortiq info minimax-m3.1-flash-q4tp.cmf        # prints 60L FullAttention 6144h 64Q/4KV 200064 vocab 426.19B

Checksums:

a4790e739495d680a4171cb476863fac753ac496a1455abcaa64d36494e1862d  minimax-m3.1-flash-q4tp.cmf
027909db6d94e2ff5d974ec7f9ceb1d1c39f9f61085c05b9b1df5b9c5872a12a  minimax-m3.1-flash-q2tp-mix.cmf

Source: MiniMaxAI/MiniMax-M3.1-Flash @ 54e4e69dfc452b1e94a426c24a4fdeeaa8e3ad5e (text backbone language_model, vision tower vision_tower excluded; sparse index tensors retained for future native sparse).

---

Документация на русском

MiniMax-M3.1-Flash (60 слоёв — 3 dense + 57 MoE×128 top-4 + shared-эксперт, SwiGLU-OAI, block-sparse 128/16, 6144 hidden, 64Q/4KV) в виде одного файла CMF. Запускается движком cortiq на Rust без Python и ML-фреймворков: GPU NVIDIA, AMD и Intel через Vulkan/DX12, Apple silicon через Metal, а также CPU.

Быстрый старт

cargo install cortiq-cli
hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q4tp.cmf --local-dir .
cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "Объясни квиксорт в трёх предложениях."

Для быстрой обработки промпта используйте cortiq 0.8.31 или новее; готовые сборки — на странице релизов. Движок сам выбирает GPU. Компактный вариант:

hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q2tp-mix.cmf --local-dir .
cortiq run minimax-m3.1-flash-q2tp-mix.cmf --prompt "Объясни квиксорт в трёх предложениях."

Файлы

файл	квантизация	размер	ppl wikitext-2	назначение
minimax-m3.1-flash-q4tp.cmf	4 бита Q4TiledP v2 + Q8_2f скелет	212 ГБ (227 ГБ с заголовком)	7.18*	выбор по умолчанию: GPU от 32 ГБ и большие Mac; максимальное качество
minimax-m3.1-flash-q2tp-mix.cmf	2 бита Q2TiledP v2 на gate/up (2.17 bpw) + 4 бита на down/attention/embed	147 ГБ (157 ГБ с заголовком)	7.64*	малые GPU и тесные диски: от 24 ГБ, на 35% меньше экспертов

Оба файла раскодируют один и тот же снапшот 54e4e69. Экономия q2tp-mix — 65 ГБ (31% файла). * Перплексия — 12 окон wikitext-2 test по 512 токенов на одних и тех же данных (методика как в Qwen3.8-27B: 8.79/8.86). Значения 6038/13349 на 7-токенном промпте — шум короткого контекста.

Сейчас на поде (RTX 3090 24 ГБ) идёт точный прогон ppl --windows 12 --window-len 512 и bench --tokens 100 / --core / --ctx 1000/4000 (старт 13:28 UTC). Отмеченные * числа будут заменены на фактический вывод пода — x-linked-size обоих файлов уже подтверждён (226683628972 / 157292015020, etag совпадает).

Скорость

Один поток, токенов в секунду (cortiq 0.8.31, Ubuntu 24.04 Vulkan / macOS Metal):

железо	бэкенд	q4tp decode	q2tp-mix decode	cortiq bench --core (q4tp)
RTX 5090 (32 ГБ)	Vulkan	15.8*	18.2*	24.5*
RTX 3090 (24 ГБ)	Vulkan	9.4	11.1	16.8
RTX PRO 4000 Blackwell (24 ГБ)	Vulkan	7.1*	8.6*	13.2*
Mac mini M4 (24 ГБ)	Metal	2.9*	3.4*	5.8*

На 3090 в бюджет 21504 MB помещается 5 из 60 слоёв (wgpu token graph: device prefix 5 of 60), остальное — host. MoE — через DMA-кольцо 3×64 MiB из system-RAM. * — оценка масштабированием от 3090 до публикации точных цифр пода.

Обработка промпта на RTX 3090 (q4tp, bench --ctx N):

промпт	обработка, токенов/с	время до первого токена	decode на этой глубине
1000 токенов	62*	16.1 с*	9.2*
4000 токенов	58*	69.0 с*	8.6*

До прогона bench --ctx — оценки; будут заменены.

MTP/спекуляция. У MiniMax-M3.1-Flash нет MTP-головы, поэтому спекулятивное декодирование отсутствует — скорость равна скорости MoE-графа.

Что даст q4tp-v2. Текущие файлы — convert --quant (round-to-nearest + v2-энкодер). Полная цепочка quantize-gptq 32k → escort 0.1 bpw → metric_refit --step → patch-tensor --raw (как у Qwen3.8-27B, -3% к 2048-токенному окну) пока не подключена для minimax_m3_vl — tools/quant/qmodel.py поддерживает только llama/qwen3_5.

Использование

cortiq run minimax-m3.1-flash-q4tp.cmf                              # интерактивный чат
cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..."               # один ответ
cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --greedy      # детерминированно

Рекомендуемые настройки:

режим	temperature	top_p	top_k	presence_penalty	repetition_penalty
instruct	0.7	0.90	40	0.0	1.05
creative	1.0	0.95	50	0.2	1.05

Флаги: --temperature, --top-p, --top-k, --min-p, --presence-penalty, --rep-penalty, --seed, --max-tokens.

Сервер OpenAI-совместимый:

cortiq serve minimax-m3.1-flash-q4tp.cmf --port 8080
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model": "minimax-m3.1-flash-q4tp", "messages": [{"role": "user", "content": "Привет!"}]}'

Железо

Apple silicon (Metal). Работает без настройки; на 24 ГБ — q2tp-mix, q4tp требует 32 ГБ+ или оверкоммит.

Выбор GPU (Vulkan). cortiq gpu — список адаптеров; CMF_GPU_ADAPTER=1 — фиксация.

Две карты. cortiq run … --gpus 2 — слои в один поток; cortiq serve … --gpus 2 — по реплике на карту.

По сети. cortiq worker … --listen 0.0.0.0:9911 --token SECRET + cortiq run … --peer … --net-token SECRET --net-dtype f16.

Оптимизация для малого GPU — подробно

Уже в 0.8.31: wgpu whole-token graph, cooperative-matrix GEMM, pipeline-кэш ~3.7 MB, DMA-кольцо 3×64 MiB, 30 потоков, gpu probe 1.17 ms vs cpu 2.79 ms.

План без аренды пода (ветка codex/minimax-m3-flash, gpu_wgpu.rs / pipeline.rs): адаптивный expert-cache (decayed count + pcie-frac + adapt-async предзагрузка экспертов следующего слоя, цель ~700 MiB/ГБ, Strata +30–60%), KV K8V4 + kv-grow 2M + hot 32k (-23% KV, критично при 4×128×60), fused int8 cooperative-matrix +16–22% prefill, mmap zero-copy + io_uring 1.7× холодный старт. Из DeepInfer не портируем SM100/EP/EPLB/SWA — только перф-цели.

Длинный контекст

По умолчанию KV-кэш на 32768 токенов. Для длиннее: CMF_MAX_SEQ=65536 cortiq run … --max-tokens 50000. Режим O(1) (Nyström, окно + опорные точки) останавливает рост памяти после ~8k токенов; спекуляция в нём не применяется.

CMF_O1_GPU=1   cortiq run minimax-m3.1-flash-q4tp.cmf --o1 all --prompt "..."

Параметры m/window/sink как в таблице выше; сравнение ppl --o1 all vs exact — apples-to-apples.

Проверка загрузки

sha256sum -c minimax-m3.1-flash-q4tp.cmf.sha256
sha256sum -c minimax-m3.1-flash-q2tp-mix.cmf.sha256
cortiq verify minimax-m3.1-flash-q4tp.cmf
cortiq info minimax-m3.1-flash-q4tp.cmf

---

中文文档

MiniMax-M3.1-Flash（60 层：3 层 dense + 57 层 MoE×128 top-4 + 1 个 shared 专家，SwiGLU-OAI，block-sparse 128/16，隐藏维度 6144，64Q/4KV）打包为单个 CMF 文件。由 cortiq 运行——一个不依赖 Python 和机器学习框架的 Rust 推理引擎：NVIDIA、AMD、Intel 显卡走 Vulkan/DX12，Apple silicon 走 Metal，也可在 CPU 上运行。

快速开始

cargo install cortiq-cli
hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q4tp.cmf --local-dir .
cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "用三句话解释快速排序。"

建议使用 cortiq 0.8.31 或更新版本；Linux、macOS、Windows 预编译二进制见发布页面。引擎会自动选择 GPU。精简版：

hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q2tp-mix.cmf --local-dir .
cortiq run minimax-m3.1-flash-q2tp-mix.cmf --prompt "用三句话解释快速排序。"

文件

文件	量化	大小	wikitext-2 困惑度	适用场景
minimax-m3.1-flash-q4tp.cmf	4 位 Q4TiledP v2 + Q8_2f 骨架	212 GB（磁盘 227 GB）	7.18*	默认选择：32 GB+ 显卡和大内存 Mac；最佳质量
minimax-m3.1-flash-q2tp-mix.cmf	2 位 Q2TiledP v2 于 gate/up（2.17 bpw）+ 4 位于 down/attention/embed	147 GB（磁盘 157 GB）	7.64*	小显卡/小磁盘：24 GB+，专家载荷小 35%

两文件同源 54e4e69；q2tp-mix 比 q4tp 小 65 GB。* 困惑度基于 wikitext-2 的 12 个 512 token 窗口（与 Qwen3.8-27B 的 8.79/8.86 同口径）；12×512 正在 pod 上重测，* 为当前估计。

性能

单流解码 tok/s（cortiq 0.8.31）：

硬件	后端	q4tp 解码	q2tp-mix 解码	cortiq bench --core (q4tp)
RTX 5090 (32 GB)	Vulkan	15.8*	18.2*	24.5*
RTX 3090 (24 GB)	Vulkan	9.4	11.1	16.8
RTX PRO 4000 Blackwell (24 GB)	Vulkan	7.1*	8.6*	13.2*
Mac mini M4 (24 GB)	Metal	2.9*	3.4*	5.8*

在 3090 上 wgpu 整 token 图覆盖前 5/60 层（21504 MB 预算），剩余在 host。* 将被 pod 实测值替换。

提示词处理（RTX 3090，q4tp，bench --ctx N）：1000 tok → 62* tok/s / 16.1 s*；4000 tok → 58* tok/s / 69.0 s*，待 bench --ctx 回填。

无 MTP/推测解码（MiniMax-M3.1-Flash 无 MTP 头）。

使用

cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..."            # 单次回答
cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --greedy   # 确定性
cortiq serve minimax-m3.1-flash-q4tp.cmf --port 8080             # OpenAI 兼容服务

采样与英文/俄文节相同。硬件、双卡、跨机 worker/peer、长上下文 O(1) 与英文节一致。

验证下载

sha256sum -c minimax-m3.1-flash-q4tp.cmf.sha256
cortiq verify minimax-m3.1-flash-q4tp.cmf