Instructions to use VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP")
config = load_config("VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8 Flash-Next — MLX, experts in 3-bit (e3bit), native MTP

The reference checkpoint of qwen38-mlx-swift, published so that nobody has to rebuild it: it is Vontra/Qwen3.8-Flash-Next-MLX-4bit-MTP with the routed experts re-quantized from 4-bit (group 32) to 3-bit (group 64). Everything else — n-gram embedding tables, attention, shared expert, router gates, norms, vision tower, native MTP block — is copied unchanged.

	Vontra 4-bit (source)	This pack
Size on disk	113 GB	84 GB
Routed experts (`switch_mlp`)	4-bit, group 32	3-bit, group 64
Other tensors	as converted by Vontra	identical
Weight shards	22	22 (same names, same tensor-to-shard layout)

Why: on a 96 GB Mac the 4-bit checkpoint barely fits; this one runs resident with room to spare, with the same greedy output on the project's regression guard.

Measured (M3 Max, 96 GB, qwen38-mlx-swift)

	Value
Greedy decode	13.0–13.5 tok/s
Prefill, varied prose, 4.8 k tokens	112 tok/s
Peak process memory, resident	57.6–58.3 GB
First request (all layers loaded)	48–63 s
Teacher-forced regression guard	10/28 · −4.8003182, identical to the 4-bit source

Full conditions: BENCHMARKS.md. Requirements: Apple Silicon with 96 GB of unified memory, macOS 15+.

Use

hf download VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP \
  --local-dir ~/models/local/Qwen3.8-Flash-Next-MLX-e3bit-MTP

# with qwen38-mlx-swift
qwen38 serve --model-path ~/models/local/Qwen3.8-Flash-Next-MLX-e3bit-MTP --enable-thinking

The loader must support qwen4_exp and per-module quantization (the experts are 3-bit/g64, the rest 4-bit/g32, as declared in config.json).

How it was made

Scripts/qwen4-exp-requantize-experts.py, with mlx-vlm==0.6.17: shard by shard, every *.mlp.switch_mlp.{gate_proj,up_proj,down_proj} tensor is dequantized from 4-bit/g32 and quantized to 3-bit/g64; every other tensor is copied byte for byte into the same output shard. Run it yourself to reproduce this pack from the Vontra source.

Licence

Derived from Qwen3.8 Flash-Next under the Qwen Community License 1.0 (included as LICENSE, unchanged from the source). Credit: Qwen for the model, Vontra for the MLX conversion with native MTP. Note the licence's conditions for very large commercial products and for "Model as a Service" or "AI Work Assistant" businesses.

Downloads last month: 388

Safetensors

Model size

149B params

Tensor type

U32

BF16

I64

MLX

Hardware compatibility

4-bit

Model tree for VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP

Base model

Qwen/Qwen3.8-Flash-Next

Quantized

TensorFold/Qwen3.8-Flash-Next-MLX-4bit-MTP

Quantized

(1)

this model