Instructions to use pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0

Use Docker

docker model run hf.co/pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0

LM Studio
Jan

vLLM

How to use pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0

Ollama
How to use pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF with Ollama:
```
ollama run hf.co/pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0
```
Unsloth Desktop

How to use pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF with Docker Model Runner:
```
docker model run hf.co/pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0
```

Lemonade

How to use pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0

Run and chat with the model

lemonade run user.Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF-Q8_0

List all available models

lemonade list

Hermes Agent

How to use pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-Flash-Next MTP draft head — head-only ROCmFP4 requant (GGUF)

The jointly-trained MTP (NextN) draft head of Qwen/Qwen3.8-Flash-Next (arch qwen4exp), requantized head-only for ROCmFPX engines: the draft LM head (output.weight) is converted to Q4_0_ROCMFP4 (644.14 → 341.02 MiB), while all other 33 tensors are byte-identical to the community Q8_0 extraction of the official head.

This is the sidecar served in our production engine since 2026-09-10.

About the filename: the file is named …-MTP-Q8_0-HEAD-ROCMFP4.gguf because the base file type is Q8_0 — only the head tensor is ROCmFP4. The per-tensor types are the authoritative ones (same convention as our model cards).

Why a matched-quant draft head

The draft head re-reads its entire LM head (248k vocab × 2560) for every drafted token — a per-token matmul over the full vocabulary. On a bandwidth-limited APU that read dominates the draft traffic. Quantizing the drafter like the target has a measurable side benefit beyond the byte saving: the two quantized models are wrong in the same places, so acceptance goes up, not down (first observed by agentionai on the same hardware class).

Measured

Radeon 8060S (gfx1151), Vulkan, target = our ROCmFP4 STRIX_LEAN build, draft-mtp n-max 6 / p-min 0.75, interleaved A/B in a production-style container. Full protocol in the lab wave-6 note.

sidecar	decode t/s	acceptance
Q8_0 head (community)	23.92	0.652
this head	24.99 (+4.5%)	0.667

Greedy fidelity: the engine release this head ships in passed the 12-prompt battery bit-identical to the pinned canon (sha 6cc4f583…) — the head-only requant does not change generated text in the greedy regime.

Runtime

The Q4_0_ROCMFP4 tensor type needs a ROCmFPX-capable build — stock llama.cpp cannot load this file. Recommended runtime: our lab build (pugant/strix-nebulosa, main) — full source, Dockerfile and guides; upstream format definition: charlie12345/ROCmFPX.

llama-server -m Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN-*.gguf \
  -md Qwen3.8-Flash-Next-MTP-Q8_0-HEAD-ROCMFP4.gguf \
  --spec-type draft-mtp --spec-draft-ngl all \
  --spec-draft-n-max 6 --spec-draft-p-min 0.75

Pairs naturally with the LEAN FP4 target above (same quant family on both sides of the speculative pair — that is the point of the matched quant).

Provenance & reproduction

Base: the community Q8_0 extraction of the official head (quimmedes/Qwen3.8-Flash-Next-MTP-GGUF), itself converted from the official checkpoint via the llama.cpp NextN export (PR #27836 lineage).
One tensor converted (output.weight Q8_0 → Q4_0_ROCMFP4); the 33 non-head tensors are sha256-identical to the base; all metadata values unchanged (file size 3.82 GB).
Reproduce (native quantizer flags, no patches needed — quantize flags go first):
```
llama-quantize --allow-requantize --pure \
  --output-tensor-type Q4_0_ROCMFP4 \
  mtp-Qwen3.8-Flash-Next-Q8_0.gguf \
  Qwen3.8-Flash-Next-MTP-Q8_0-HEAD-ROCMFP4.gguf Q8_0
```
Script with dry-run pre-flight and stat checks: scripts/w6t3-drafter-head-requant.sh in the lab snapshot; it regenerates this artifact bit-identically.
sha256: fa30a89fa06cd1bcc40810dcf1fed73d69c5ff33afb2a08b76333c6f9629ea34

Credits & license

Qwen team — the MTP head itself, trained jointly with the target model.
quimmedes — the Q8_0 GGUF extraction this file derives from, and the cafe-llama.cpp fork that runs it.
charlie12345 (ROCmFP4 format / ROCmFPX) and ciru-ai (ROCmFPX) — the tensor types.
danielhanchen (qwen4exp support, PR #27742) and JJJYmmm (MTP draft graph, PR #27739).
agentionai — the matched-quant observation.

Derivative of Qwen/Qwen3.8-Flash-Next, distributed under the Qwen Community License 1.0 (see LICENSE; note the license's Model-as-a-Service clause before commercial serving). Requantization and measurements by pugant; work assisted by GLM by z.ai.

Everything here is experimental and provided as-is, at your own risk.

Downloads last month: 52

GGUF

Model size

4B params

Architecture

qwen4exp

Hardware compatibility

8-bit

Model tree for pugant/Qwen3.8-Flash-Next-MTP-DRAFT-HEAD-ROCMFP4-GGUF

Base model

Qwen/Qwen3.8-Flash-Next

Quantized

(207)

this model