Qwen3.6-35B-A3B-AntiLoop-NVFP4 for MLX-LM

This is the text-only, lossless-weight MLX-LM conversion of N8Programs/Qwen3.6-35B-A3B-AntiLoop-NVFP4. It deliberately omits the vision tower so the mixed NVIDIA ModelOpt weights can be preserved without a second quantization pass or an installed-package patch.

Lossless MLX representation

All source quantized weight payloads and scales are retained:

• ModelOpt FP8 E4M3 bytes are packed directly into MLX uint32 tensors with unit E8M0 carrier scales; the original FP32 tensor scale is applied after MLX's native MXFP8 matrix multiplication.
• ModelOpt NVFP4 E2M1 nibbles and E4M3 block-scale bytes remain bit-identical; the original FP32 tensor or per-expert scale is applied after MLX's native NVFP4 matrix multiplication.
• BF16 and FP32 language-model tensors remain in their source precision.
• No weight tensor is dequantized or requantized during conversion.

The package contains 1,475 tensors across 41 language shards: 130 scaled MXFP8 dense modules, 121 scaled NVFP4 dense modules, and 120 scaled NVFP4 expert projections. weight_bytes_requantized=false is recorded in mlx_conversion_manifest.json.

“Lossless” here means no additional error is introduced into the source quantized weights. Activations remain in the model dtype instead of using the source checkpoint's stored activation scales, so execution is not expected to be bit-identical to NVIDIA ModelOpt inference.

Use with MLX-LM

Install the tested runtime:

pip install -U "mlx-lm==0.31.3"

Generate with an unmodified MLX-LM installation:

python -m mlx_lm generate \
  --model mlx-community/Qwen3.6-35B-A3B-AntiLoop-NVFP4 \
  --trust-remote-code \
  --prompt "Solve: 27 * 43" \
  --max-tokens 512

The packaged interactive defaults are temperature=0.6, top_p=0.95, and top_k=20. The Qwen chat template retains its enable_thinking control, so clients such as LM Studio can expose the usual thinking toggle.

MLX-LM natively loads the included modeling_mlx_qwen36_modelopt_hybrid.py through its model-local model_file architecture hook; the loader does not patch or modify the installed package. Review that runtime file before loading this repository. In MLX-LM 0.31.3, --trust-remote-code controls tokenizer loading rather than the model_file hook; it is included above for tokenizer compatibility.

This repository is intentionally text-only. It does not contain the 333 BF16 vision tensors, image processor, or video processor from the upstream model.

Original model card

This is a mixed-precision NVIDIA ModelOpt deployment checkpoint for Qwen3.6-35B-A3B-AntiLoop, a narrow fine-tune intended to recover from pathological self-verification and enumeration loops while preserving ordinary long-form reasoning.

No PEFT adapter is required at inference time. This MLX-LM derivative retains the language architecture, tokenizer, chat template, and 262,144-token native context configuration. It does not package the vision tower or MTP draft weights.

Training data

The exact 178 masked supervised targets used for the final AntiLoop training round are published in the Qwen3.6-35B-A3B-AntiLoop-SFT dataset. The dataset preserves each loss_start_char boundary so the pathological loop prefix remains conditioning context rather than a supervised target. It intentionally excludes the separately generated KL-regularization anchors.

Training procedure

The AntiLoop adapter was trained on the 178 masked supervised targets using a standard supervised fine-tuning procedure, but regularized via KL-loss on separately generated non-loop anchors from the base model on everyday prompts.

Benchmark results

LoopHard

LoopHard is our held-out set of 285 enumeration prompts designed to elicit futile recall, recounting, and self-verification loops. The primary metric is judged loops: whether the model's reasoning trace remains stuck in a futile cycle when generation ends.

Model	Judged loops	Loop rate
NVIDIA NVFP4	72 / 285	25.26%
AntiLoop NVFP4	10 / 285	3.51%
NVIDIA NVFP4 + presence_penalty=1.5	30 / 285	10.53%
AntiLoop NVFP4 + presence_penalty=1.5	1 / 285	0.35%

The matched presence_penalty=1.5 comparison converted all 30 control loops to clean completions while introducing one different loop. Exact two-sided McNemar p = 2.98e-8.

Generation used thinking mode, temperature=0.7, top_p=0.95, top_k=20, a 6,144-token completion limit, and concurrency 24. The two presence_penalty=1.5 arms used the exact original and AntiLoop NVFP4 checkpoints with the same vLLM build and serving configuration: TP1, FP8 KV cache, FlashInfer attention, Marlin NVFP4 MoE, and MTP speculative decoding with three draft tokens.

LoopHard is judged by GLM-5.2 using a convergence-aware rubric: systematic reasoning and verification that reaches a conclusion are not loops, and a trace that notices its own circling and exits is classified as recovered. The calibration set contained 42 manually labeled traces. Across three judge runs, accuracy was 88.1%, 92.9%, and 95.2%; all three runs identified all 17 labeled loops, with 2–5 false positives among the 25 non-loop traces.

The 285 prompts, metadata, and GLM-5.2 evaluation code are published in the LoopHard dataset on Hugging Face.

Capability preservation

The capability checks below compare the same round-2 AntiLoop adapter against its FP8 reference model under a matched runtime-LoRA setup. These runs used the default presence penalty and should not be interpreted as evaluations of the exact mixed-precision artifact at presence_penalty=1.5.

GPQA Diamond

Model	Accuracy
Qwen3.6-35B-A3B official model card	86.0%
FP8 reference, our matched harness	167 / 198 (84.34%)
AntiLoop FP8, our matched harness	166 / 198 (83.84%)

The official-model-card number is included for context and was not produced by our harness.

Our GPQA run used thinking mode, paired per-question seeds, temperature=0.7, top_p=0.95, top_k=20, MTP3 speculative decoding, a 65,536-token reasoning budget, and 4,096 tokens of answer headroom. The difference was not significant.

Source for the published 86.0% result: Qwen/Qwen3.6-35B-A3B model card.

GSM8K

Model	Accuracy
FP8 reference	1273 / 1319 (96.51%)
AntiLoop FP8	1270 / 1319 (96.29%)

The GSM8K run used the exact 1,319-example openai/gsm8k main test split, thinking mode, paired seeds, temperature=0.7, top_p=0.95, top_k=20, MTP3, an 8,192-token reasoning budget and 1,024 tokens of answer headroom. The difference was not significant.

Taken together, the matched GPQA and GSM8K results show no material or statistically detectable capability loss at these sample sizes. They do not establish equivalence across other tasks, modalities, or sampling settings.

Original NVIDIA ModelOpt usage

Use a recent vLLM build with ModelOpt mixed-precision support:

vllm serve N8Programs/Qwen3.6-35B-A3B-AntiLoop-NVFP4 \
  --quantization modelopt \
  --trust-remote-code \
  --max-model-len 262144 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3

MTP speculative decoding can be enabled on a compatible build with:

--speculative-config '{"method":"mtp","num_speculative_tokens":3}'

For the measured LoopHard setting, send the following sampling parameters:

{
  "temperature": 0.7,
  "top_p": 0.95,
  "top_k": 20,
  "presence_penalty": 1.5
}

The capability-preservation results above used the default presence penalty; presence_penalty=1.5 has not yet been evaluated on GPQA or GSM8K.

Follow the nvidia/Qwen3.6-35B-A3B-NVFP4 model card for deployment requirements and the Qwen/Qwen3.6-35B-A3B model card for chat templating, thinking controls, multimodal inputs, and base-model limitations.

Limitations

• This is a narrow behavioral fine-tune, not a general alignment or safety model.
• This MLX-LM artifact is text-only; image and video inputs are unsupported.
• The conversion preserves quantized weight values, but uses model-dtype activations rather than the source ModelOpt activation scales.
• LoopHard is a task-specific, judge-based benchmark; its loop rate should not be interpreted as a general safety, truthfulness, or factuality score.
• The GPQA and GSM8K checks used runtime LoRA on an FP8 base, not this exact mixed-precision artifact.
• Capability preservation has not been tested at presence_penalty=1.5.
• The source checkpoint's fixed-scale FP8 merge approximates the exact BF16 LoRA merge; small adapter updates can round away or clip at the original E4M3 range. This MLX conversion does not add another weight-quantization pass.
• Runtime validation used a 65,536-token configured context, not the full native 262,144-token context.
• Outputs may still be incorrect, overconfident, repetitive, biased, toxic, or unsafe.

License

Apache 2.0, following both the underlying Qwen checkpoint and NVIDIA's quantized derivative. See the pinned Qwen license, the Qwen model card, and the NVIDIA ModelOpt checkpoint card.

(co-written with GPT-5.6-Sol)