Instructions to use True2456/Qwen3.8-Flash-Next-ANE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use True2456/Qwen3.8-Flash-Next-ANE with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("True2456/Qwen3.8-Flash-Next-ANE")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use True2456/Qwen3.8-Flash-Next-ANE with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "True2456/Qwen3.8-Flash-Next-ANE"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "True2456/Qwen3.8-Flash-Next-ANE"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use True2456/Qwen3.8-Flash-Next-ANE with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "True2456/Qwen3.8-Flash-Next-ANE"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "True2456/Qwen3.8-Flash-Next-ANE"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "True2456/Qwen3.8-Flash-Next-ANE",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use True2456/Qwen3.8-Flash-Next-ANE with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "True2456/Qwen3.8-Flash-Next-ANE"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default True2456/Qwen3.8-Flash-Next-ANE

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use True2456/Qwen3.8-Flash-Next-ANE with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "True2456/Qwen3.8-Flash-Next-ANE"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "True2456/Qwen3.8-Flash-Next-ANE" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-Flash-Next-ANE

Hybrid inference package for Qwen3.8-Flash-Next on Apple Silicon, designed for use with qwen-ane.

What it is

Flash-Next is a hybrid silicon deployment:

Attention on Apple Neural Engine (ANE): 36 Gated DeltaNet (linear recurrence) layers and 12 Qwen Sparse Attention layers run on the Neural Engine via precompiled MIL graphs, drawing approximately 1.1 W.
MoE Experts on Metal GPU: The 512-expert routed MoE bank (68 GB) and output projections remain resident in Unified Memory and execute via MLX, drawing approximately 9 W.
Speculative Drafting: 4-slot MTP speculative drafting and PLE n-gram tables deliver high decode throughput.

For a 100% pure on-chip ANE model (0% GPU), see True2456/Qwen3.8-27B-ANE.

Package layout

Path	Size	Role
`ane-h17/`	~5.6 GB	Precompiled ANE MIL graphs and weight blobs for 36 GDN + 12 QSA layers
`model.safetensors`	76.7 GB	4-bit language weights (experts 4-bit gs64; embeddings/mixers 8-bit)
`ngram/`	~95 MB	BF16 PLE n-gram table shards
`mtp/`	~1.4 GB	4-bit MTP drafter weights
`model-indexer.safetensors`	42.6 MB	QSA indexer weights
tokenizer / config	—	`config.json`, `tokenizer.json`, `chat_template.jinja`, and related metadata

Requirements

Apple Silicon Mac with 96 GB or 128 GB Unified Memory (tested on M5 Max, macOS 27)
Python 3.10+
qwen-ane

Install and run

1. Install qwen-ane

git clone https://github.com/True2456/qwen-ane.git
cd qwen-ane
pip install -e ".[mlx]"

2. Download weights

Download automatically into ~/.qwenANE/models/flash-next:

qwen-ane pull flash-next

Or using the Hugging Face CLI:

hf download True2456/Qwen3.8-Flash-Next-ANE --local-dir ~/.qwenANE/models/flash-next

3. Interactive chat

qwen-ane chat -model flash-next -ctx 32768

4. OpenAI-compatible API server

qwen-ane serve -model flash-next -port 2457 -ctx 32768

Point any OpenAI-compatible client, web UI, or coding agent to http://127.0.0.1:2457/v1.

For Pi the coding agent:

pi -e extensions/flashnext-pi.ts --provider flashnext --model Qwen3.8-Flash-Next

Measured performance (Apple M5 Max)

Cold prefix (reused=0), serve context 33,792, FLASHNEXT_SPEC=4, FLASHNEXT_PREFILL_MIL_K=32, FLASHNEXT_MOE=mlxresident, FLASHNEXT_HEAD=mlx. Power measured via powermetrics:

Prompt Tokens	Completion	TTFT	Prefill Speed	Decode Speed	Active Package Power
4,096	128	49.3 s	80.6 tok/s	15.1 tok/s	~17.9 W (1.1 W ANE + 9.0 W GPU + 7.8 W CPU)
8,192	128	100.5 s	79.2 tok/s	14.8 tok/s	~17.9 W (1.1 W ANE + 9.1 W GPU + 7.8 W CPU)
16,384	128	201.7 s	79.1 tok/s	15.7 tok/s	~17.7 W (1.1 W ANE + 9.5 W GPU + 7.1 W CPU)
32,768	128	399.2 s	79.9 tok/s	15.8 tok/s	~19.4 W (1.1 W ANE + 10.5 W GPU + 7.8 W CPU)

Prefill: Sustained ~80 tok/s linearly up to 32k context.
Decode: 15–16 tok/s with 4-slot MTP speculative decoding (or 4.5 tok/s single-step).
Prefix cache: Multi-turn agent follow-ups with prefix hits evaluate in ~0.8 s.

Quality verification

Measured on M5 Max against unmodified MLX 4-bit reference on identical prompts:

Benchmark	Setting	ANE + MLX Hybrid	Reference MLX 4-bit GPU
MMLU	5-shot, likelihood of A/B/C/D (600 questions)	87.33% (524/600)	86.33% (518/600)
GSM8K	8-shot greedy, last number (200 test questions)	96.50% (193/200)	—

License

Qwen Community License 1.0 (see LICENSE). Inference engine: True2456/qwen-ane.

Downloads last month: 551

Safetensors

Model size

128B params

Tensor type

U32

BF16

I64

MLX

Hardware compatibility

Quantized

Model tree for True2456/Qwen3.8-Flash-Next-ANE

Base model

Qwen/Qwen3.8-Flash-Next

Finetuned

(48)

this model

Evaluation results

5-shot likelihood on MMLU (600 questions, 5 subjects)
test set self-reported

87.330
8-shot greedy on GSM8K (200 test questions)
test set self-reported

96.500