Instructions to use Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP")
config = load_config("Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8 Flash Next — MLX oQ8 with native MTP

A near-uniform 8-bit MLX conversion of Qwen/Qwen3.8-Flash-Next, quantised directly from the official BF16 checkpoint with its native MTP draft block preserved.

Original model · Qwen overview · MLX-VLM · Qwen Community License 1.0

About this conversion

This repository contains an oQ8 MLX conversion produced directly from Qwen's official BF16 weights. Every final tensor, including the native MTP head, was rebuilt from that checkpoint. Group size 32 supports the model's 160-wide hashed n-gram embedding tables.

Item	Value
Repository	`Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP`
Base model	`Qwen/Qwen3.8-Flash-Next`
Format	MLX safetensors
Quantisation	oQ8, near-uniform 8-bit affine
Base group size	32
Native MTP	Included, one Qwen4Exp draft block
Weight tensors	3,747 total, including 76 converted MTP tensors
Weight shards	37
Weight size	202.75 GB / 188.82 GiB
Configured context	262,144 tokens
Architecture	`qwen4_exp` vision-language sparse MoE

The upstream tokenizer, chat template, vision processor, generation configuration, native MTP configuration, licence, and attribution are preserved.

This checkpoint requires an oMLX or MLX-VLM runtime with explicit qwen4_exp native-MTP support. Stock runtimes that do not construct the Qwen4Exp MTP module may reject the 76 MTP tensors during strict weight loading.

Do not attach a Qwen3.8 27B drafter to this model. Flash Next has different hidden dimensions and includes its own matching MTP block.

Download and use

hf download Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP \
  --local-dir Qwen3.8-Flash-Next-MLX-oQ8-MTP

Add the downloaded directory to an MTP-capable oMLX installation, select the model, and enable native MTP in its model settings.

Apple M3 Studio performance

Validated in oMLX on an Apple M3 Studio using a deterministic 128-token chat generation:

Runtime mode	Output tokens	Speed
Native MTP disabled	128	19.5 tokens/s
Native MTP enabled, three draft tokens	128	35.7 tokens/s

Native MTP delivered an 83% throughput uplift on this measured prompt. The MTP-enabled run accepted 69 of 110 draft proposals (62.7%) and produced coherent, non-repetitive prose. Separate deterministic gates returned exactly hello, identified Paris correctly, and answered a basic arithmetic prompt with 5 both before and after enabling MTP.

The first request after loading includes model and kernel warm-up and is not used as a steady-state throughput figure. Results vary with prompt length, cache state, sampling settings, runtime version, and memory pressure.

Architecture

Qwen3.8 Flash Next combines Gated DeltaNet, Qwen Sparse Attention, sparse mixture-of-experts layers, widened gated residual streams, hashed bigram and trigram embeddings, and a native next-token prediction block for speculative decoding.

Architecture detail	Upstream value
Language-model parameters	125B total / 6B active
N-gram embedding	51B parameters
Layers	48
Routed / active experts	512 / 10, plus 1 shared
Attention heads / KV heads	24 / 2
Hidden size	2,560
Native configured context	262,144 tokens
Native MTP draft blocks	1

For upstream evaluations, intended use, limitations, safety guidance, and the complete architecture discussion, see the original model card.

Conversion and validation

The converter read the official BF16 checkpoint directly.
Structural validation checked all 3,747 indexed tensors and all 37 shards.
MTP validation confirmed one configured draft layer and 76 converted MTP tensor entries.
Live generation validation ran with MTP disabled and enabled, covering deterministic exact instruction, factual recall, arithmetic, coherent long generation, MTP telemetry, and cache rollback.

This is a community conversion, not an official Qwen release.

License and attribution

The upstream model is released under the Qwen Community License 1.0. The required licence text is included in this repository.

Model design, training, evaluations, and upstream documentation belong to Qwen and the original contributors. The MLX conversion and packaging are provided by Vontra.

Downloads last month: 1,554

Safetensors

Model size

180B params

Tensor type

U32

BF16

I64

MLX

Hardware compatibility

8-bit

Model tree for Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP

Base model

Qwen/Qwen3.8-Flash-Next

Quantized

(169)

this model