avlp12

/

MiniMax-M3-Alis-MLX-Dynamic

like 0

Fixed — please re-download the t512 branch (or main).

You did nothing wrong; this was on our side. Root cause:

The published builds kept the always-on shared expert at 8-bit while the routed experts were 6-bit (t512) / 3-bit (main), using an unpacked MoE layout. oMLX (and stock mlx-vlm) unconditionally re-pack the shared expert into the 129-wide routed SwitchLinear, and an MLX packed bank can only hold one bit-width — so it tried to concatenate a 6-bit routed bank (128, 6144, 1152) with an 8-bit shared expert (1, 6144, 1536) and the packed widths didn't match. That is the exact error you saw. It's not a config mistake and not really an oMLX bug — the unpacked mixed-bit layout simply requires a patched loader.

What changed: main, t512, and t512ref are re-shipped with the shared expert packed at routed bits. They now load on stock mlx-vlm and oMLX with no fork or patch:

pip install -U mlx-vlm
mlx_vlm.generate --model avlp12/MiniMax-M3-Alis-MLX-Dynamic --revision t512 \
  --prompt "안녕하세요" --max-tokens 128

Verified loading + generating on stock mlx-vlm 0.6.7 (128 routed + 1 packed shared = 129 experts, image path intact). The tiny quality cost of packing the shared expert is negligible at t512 (routed already 6-bit) and bounded at main (3-bit). If you want the 8-bit-shared variant, it can be rebuilt with pack_shared_expert=false on the patched mlx-vlm (Blaizzy/mlx-vlm#1544), but it is not hosted.

Thanks for the clear report with the exact shapes — that pinned it immediately.

Thank you very much for this, is there any chance you could upload the other two as well? I am only seeing the T256, and I really need the T512 and T512REF.

Much appreciated.

Thank you very much for this, is there any chance you could upload the other two as well? I am only seeing the T256, and I really need the T512 and T512REF.

Much appreciated.

Files and versions -> click main -> select t512

Resolved: the packed compat builds (main, t512, t512ref) load on stock mlx-vlm and oMLX. Closing — please reopen if it recurs.

Thank you, got the T512. It downloaded fine and ran fine, then I started getting random errors on VSCode. I tried just about every possible setting, it happens randomly, about 6k-7k into thinking and after having processed a solid 100k token or so. Everything else works great.

If you could figure out what the cause is or resolve it, please let me know. My best guess is something to do with the template?

The error mid thinking:
"Reason: Response contained no choices.: Error: Response contained no choices."

Thanks for the follow-up — glad T512 loads and runs. This new symptom ("Response contained no choices" mid-thinking, random, deep into a ~100k-token session) is a different beast from the original load failure: it's the stream ending without content, which the VSCode agent extension then renders as that error. It is almost certainly not the chat template — a template problem would corrupt content deterministically, not kill the transport randomly.

First, the one thing that changes everything: which serving stack are you running T512 on? (oMLX? mlx_vlm server? something else?) The suspects differ by stack:

If oMLX: prime suspect is its memory guard aborting the request mid-generation (engine_core.py: "Request aborted: process memory limit exceeded … loosen memory_guard_tier (safe → balanced → aggressive)"). T512 is 351 GB of weights, and at ~100k context the KV + sparse-index caches add ~14 GB and keep growing during those 6–7k thinking tokens — whether you cross the watermark depends on what else is using RAM at that moment, which is exactly why it feels random. Fixes are config-side: loosen memory_guard_tier, close RAM-heavy apps, cap context, or use main (T256, 192 GB) for maximum long-context headroom.

Regardless of stack, the generic versions of the same suspects apply:

• server-side abort: macOS memory pressure at 351 GB + growing KV on a 512 GB machine can stall/kill generation mid-stream even without an explicit guard;
• client side: a 100k-prefill + 7k-thinking response is a single 10-minute-plus request — some VSCode agent extensions time out, or mis-handle a final "choices": [] usage chunk.

Three things would discriminate immediately:

1. The server log at the moment of failure. One line settles it: an explicit abort/memory message → server guard; a Python traceback → server bug (please paste); a Metal/GPU error → long-context kernel issue (please paste); nothing unusual server-side → the extension is the problem.
2. Which VSCode extension + version (Cline / Roo / Kilo / other), and its API-timeout setting if any.
3. Machine RAM + what else was running, and (if oMLX) your memory_guard_tier.

Reopening this thread until we've pinned it down.

Thanks for letting me know and the explanation. I do not use the guard. I decided to download and use the T512REF and surprisingly, it does not have the same issue I reported. It works great on a single 512GB M3, I have set my CTX to 280K and everything works, 500GB is total usage at 230K context, getting 11 tokens per second and it gets through 110K load in 10 minutes.

Thank you.