Vontra/Qwen3.8-Flash-Next-MLX-oQ2

Qwen3.8 Flash Next — oQ2 MLX

An oQ2 Apple-silicon conversion of Qwen/Qwen3.8-Flash-Next, quantised directly from the official BF16 checkpoint.

Original model · Qwen overview · MLX-VLM · Qwen Community License 1.0

Quality hold: this first oQ2 build loads and runs, but live testing found incoherent output. Do not use it as a quality checkpoint while the mixed-precision recipe is being revised and revalidated from the original BF16 weights.

About this conversion

This repository contains an oQ2 mixed-precision MLX conversion produced directly from Qwen's BF16 weights. Calibration measured layer sensitivity using a load-tested 4-bit proxy; the final tensor quantisation reread the original BF16 checkpoint. Group size 32 covers the model's unusual 160-wide hashed n-gram embedding tables.

Item	Value
Base model	Qwen/Qwen3.8-Flash-Next
Format	MLX safetensors
Quantisation	oQ2 mixed precision; 2-bit affine base, protected layers at 5/8-bit
Base group size	32
Weight shards	14
Weight size	67.67 GB (63.02 GiB)
Configured context	262,144 tokens
Architecture	qwen4_exp vision-language sparse MoE

The upstream tokenizer, chat template, vision processor, and generation configuration are preserved. The optional upstream MTP head is not included.

oQ2 is an extreme-compression checkpoint. It trades output fidelity for memory savings and can degrade instruction following, reasoning, factual accuracy, and visual understanding. Prefer the 4-bit or 8-bit variants when quality matters more than footprint.

Qwen3.8 Flash Next uses the new qwen4_exp architecture. Use an oMLX or MLX-VLM build that explicitly lists qwen4_exp support. Older MLX-VLM releases cannot load this checkpoint.

Quick start

hf download Vontra/Qwen3.8-Flash-Next-MLX-oQ2 \
  --local-dir Qwen3.8-Flash-Next-MLX-oQ2

With a compatible MLX-VLM runtime:

python -m mlx_vlm.generate \
  --model Qwen3.8-Flash-Next-MLX-oQ2 \
  --prompt "Explain sparse mixture-of-experts routing." \
  --max-tokens 512

Architecture

Qwen3.8 Flash Next combines Gated DeltaNet, Qwen Sparse Attention, sparse mixture-of-experts layers, widened gated residual streams, and hashed bigram/trigram embeddings.

Architecture detail	Upstream value
Language-model parameters	125B total / 6B active
N-gram embedding	51B parameters
Layers	48
Routed / active experts	512 / 10, plus 1 shared
Attention heads / KV heads	24 / 2
Hidden size	2,560
Native configured context	262,144 tokens

For upstream evaluations, intended use, limitations, safety guidance, and the complete architecture discussion, see the original model card.

Conversion and validation

• Source: official BF16 checkpoint.
• Data-driven sensitivity calibration completed before final BF16 tensor quantisation.
• All 3,671 converted tensors and 14 indexed shards were checked locally.
• The checkpoint passed an end-to-end Apple-silicon generation smoke test.
• The release payload was scanned for credentials, personal contact details, private paths, private network information, logs, caches, and private organisation data.
• Repeatable performance results will be added after benchmarking.

This is a community conversion, not an official Qwen release.

License and attribution

The upstream model is released under the Qwen Community License 1.0. The required licence text is included in this repository.

Model design, training, evaluations, and upstream documentation belong to Qwen and the original contributors. The MLX conversion, Apple-silicon validation, and packaging are provided by Vontra.