brandonmusic

/

MiniMax-M3-NVFP4

like 10

if you want to run minimax on two rtx pro 6ks, i'd recommend minimax 2.7 instead. a REAP at 50 percent is going to lobotimize, except for maybe gthigns like coding if you do proper calibration on a strong enough dataset (over 10k examples).
If you want great reasoning, as opposed to just parametric knowledge, dsv4 flash is by far the best option. You can get a million kv cache or close to it and decode of over 220 t/s at c1, and well over 11k t/s prefill. look here for https://github.com/local-inference-lab/rtx6kpro/blob/master/models/ds4dspark-v9.md if you want to stay on the bleeding edge of sm120 (and sm121) join our discord here https://discord.gg/FXeGaYeN . Lots of guys tinkering on it fron 2x to 16x rtx pro 6ks. Bunch of people with connections to nvidia, deepseek, and tons of guys contributing to vllm, flashinfer, and tons of other software for this GPU.

Hi — first, thank you for the b12x stack. As far as I can tell it's the only path today to native MSA + NVFP4 M3 on SM120, and after fixing the issues below it does boot and serve on 2x RTX PRO 6000, TP=2 (with the sparkarena REAP50 checkpoint). Reporting two image bugs with root cause and working fixes, plus findings that may be useful for the next tag.

Bug 1: PYTHONPATH=/port/serving (path does not exist in the image) → engine core crash at TP>1

Symptom: with TP=2, engine core workers die at spawn with:

TypeError: _LazyConfigMapping.__init__() missing 1 required positional argument: 'mapping'
RuntimeError: Engine core initialization failed.

Root cause: serve_m3.py sets PYTHONPATH=/port/serving, but the code lives at /opt/m3/serving in the published image. Because the registration module is not importable by name in the worker, vLLM's maybe_register_config_serialize_by_value falls back to cloudpickle by-value serialization of the config, which embeds the process-global transformers CONFIG_MAPPING (_LazyConfigMapping) — and that object cannot be reconstructed on unpickle. On a dev machine where /port/serving exists the import works, serialization goes by-reference, and TP=4 runs fine — which is presumably why this never reproduced for you.

Fix (verified): correct the path (one line in the image), or as a runtime workaround: a mounted sitecustomize.py that fixes sys.path and performs the registration in every interpreter (we also added a defensive copyreg for the lazy mapping). With this in place, spawn at TP=2 works.

Bug 2: MSA_B12X_WIRE_DIR defaults to /port/b12x_wire (does not exist)

Symptom: boot succeeds, first inference dies with ModuleNotFoundError: No module named 'indexer_score'. The module exists in the image at /opt/m3/b12x_wire/indexer_score.py — the default wire dir just points at the wrong place.

Fix (verified): MSA_B12X_WIRE_DIR=/opt/m3/b12x_wire (env). After this, native fused MSA kernels compile in warm_jit and the first forward runs.

Minor: tools API returns HTTP 400

OpenAI-style tools=[...] requests consistently return 400 on this image (chat completions without tools work). Didn't dig into the parser — flagging it because it blocks agentic evaluation.

FYI: patch list needed for the sparkarena REAP50 checkpoint

In case you want the next tag to accept it out of the box, the REAP50 export differs from your full-model layout in ways that required, on top of the two fixes above: config facade (vision fields are nested under img_token_compression_config, your code reads them top-level), weight-name remap (language_model.model.*→model.*, block_sparse_moe→mlp.experts), shard-aware routing of separate w1/w3 into fused gate_up_proj/w13, a BF16 sidecar for the NVFP4-quantized indexer projections, and w13 global-scale normalization at fusion time (that checkpoint's w1/w3 global scales differ per-expert by up to ~35%; the current [:, 0] selection drops the w3 scale). With all of it: boots, serves, MSA native path active, KV bf16 — generation quality issues remain but appear to be calibration-side (reported here).

Happy to share diffs, boot logs, or the numeric scale-verification script. And genuinely looking forward to the SM120 port writeup + updated image mentioned in the recipe README — if you want a tester on 2x RTX PRO 6000 / TP=2 before you tag it, I can turn results around within a day.

Thanks for the work you've put into this stack.

if you want help with testing and debugging, the discord is the way to go. We are a whole community there. hundreds of us.

Thanks — that confirms our end-state conclusion: after fixing everything stack-side (paths, remaps, shard fusion, indexer dtype, w13 scale normalization — each verifiably improved the weights), bounded generation stayed at 0/9, so the remaining blocker had to be REAP50 + v0 calibration. Good to hear it from the source.

And fair point on M2.7 — it's literally been my production model on this box throughout (SGLang TP=2). The M3 run was an upgrade attempt, so nothing lost, plenty learned.

DSv4 Flash DSpark is going on my bench next — those TP2 numbers are hard to ignore. Joining the Discord now; happy to be a same-day tester for future b12x tags on 2x RTX PRO 6000.

if you want to try an attempt at calbiration the reap yourself, i've got calibration data in a separte sub folder here on this one. I did a couple of REAPS, and i rented some b200s to do so.
One other option you might consider is a hybrid nvfp4-n3 for the model. a buddy on the discord did it for this model madeby561/GLM-5.2-NVFP4-NF3-Hybrid. made an 8x card model run on 4x cards. mm3 might do the same from just a pure eyeball of the math.

in the discord weve' got in running on nvfp4 cache, taht's pretty rock solid. and the model doesn't loop like REAPs have a tendency to do, and it's very close to the nvfp4 model on all the benchmarks we've run on it.