Instructions to use schsu/Qwen3.8-9B-MLX-MXFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use schsu/Qwen3.8-9B-MLX-MXFP4 with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("schsu/Qwen3.8-9B-MLX-MXFP4")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use schsu/Qwen3.8-9B-MLX-MXFP4 with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "schsu/Qwen3.8-9B-MLX-MXFP4"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "schsu/Qwen3.8-9B-MLX-MXFP4"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use schsu/Qwen3.8-9B-MLX-MXFP4 with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "schsu/Qwen3.8-9B-MLX-MXFP4"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "schsu/Qwen3.8-9B-MLX-MXFP4"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "schsu/Qwen3.8-9B-MLX-MXFP4",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use schsu/Qwen3.8-9B-MLX-MXFP4 with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "schsu/Qwen3.8-9B-MLX-MXFP4"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default schsu/Qwen3.8-9B-MLX-MXFP4

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use schsu/Qwen3.8-9B-MLX-MXFP4 with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "schsu/Qwen3.8-9B-MLX-MXFP4"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "schsu/Qwen3.8-9B-MLX-MXFP4" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-9B MLX MXFP4

This repository contains an MLX MXFP4 quantization of empero-ai/Qwen3.8-9B, prepared for local inference on Apple silicon.

Qwen3.8-9B is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-9B architecture. The source model was trained by Empero on approximately 70,000 curated teacher traces covering mathematics, code, general reasoning, instruction following, and tool use.

Model details

Item	Value
Source model	`empero-ai/Qwen3.8-9B`
Base architecture	Qwen3.5-9B
Parameters	9.65B
MLX quantization mode	MXFP4 (E2M1)
Bits	4
Group size	32
Average storage	4.251 bits per weight
Quantized checkpoint size	Approximately 4.5 GB
Context length	Up to 262,144 tokens (architecture limit)
License	Apache-2.0

The model uses weight-only MXFP4 quantization. Non-quantized tensors retain the dtype selected from the source configuration (bfloat16). The tokenizer, generation configuration, and chat template are included in this repository.

Requirements

An Apple silicon Mac
A recent version of macOS
Python 3.10 or newer (Python 3.12 tested)
mlx-lm 0.31.3 or newer
mlx 0.32.0 or newer

Install the runtime:

python -m pip install --upgrade mlx-lm

Quick start

Hugging Face model ID: schsu/Qwen3.8-9B-MLX-MXFP4.

Command line

mlx_lm.generate \
  --model schsu/Qwen3.8-9B-MLX-MXFP4 \
  --prompt "Explain why the sky is blue." \
  --max-tokens 2048 \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20

The model normally emits a reasoning block before its final answer. To disable the explicit thinking block for short responses:

mlx_lm.generate \
  --model schsu/Qwen3.8-9B-MLX-MXFP4 \
  --prompt "Reply with exactly: Hello" \
  --max-tokens 64 \
  --chat-template-config '{"enable_thinking": false}'

Python

from mlx_lm import generate, load
from mlx_lm.sample_utils import make_sampler

model_id = "schsu/Qwen3.8-9B-MLX-MXFP4"
model, tokenizer = load(model_id)

messages = [
    {"role": "user", "content": "Explain why the sky is blue."},
]
prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=True,
)

response = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=2048,
    sampler=make_sampler(temp=0.6, top_p=0.95, top_k=20),
)
print(response)

OpenAI-compatible server

mlx_lm.server \
  --model schsu/Qwen3.8-9B-MLX-MXFP4 \
  --host 127.0.0.1 \
  --port 8080

Example request:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "schsu/Qwen3.8-9B-MLX-MXFP4",
    "messages": [{"role": "user", "content": "Hello!"}],
    "temperature": 0.6,
    "top_p": 0.95,
    "max_tokens": 512
  }'

Function calling

The bundled Qwen chat template supports function definitions and multi-turn tool results. Applications should preserve the complete conversation and add each tool result with role: "tool" before requesting the next assistant turn.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

messages = [{
    "role": "user",
    "content": "What is the weather in Taipei? Use the tool.",
}]

prompt = tokenizer.apply_chat_template(
    messages,
    tools=tools,
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=False,
)

The model's native template renders tool calls using <tool_call> and <function=...> XML-style tags. Tool parsers must support this Qwen3.5 format. When implementing an agent loop, do not treat the end of a tool call as the end of the entire task.

Recommended generation settings

The source model recommends:

Parameter	Value
Temperature	0.6
Top-p	0.95
Top-k	20

Allow enough output tokens for reasoning tasks. Very short generation limits may stop inside the model's thinking block, while greedy decoding over long outputs can increase the chance of repetition. For interactive applications, start with 2,048 to 8,192 output tokens and adjust for the task and available memory.

The architecture supports a context length of up to 262,144 tokens, but the practical limit depends on unified memory because the KV cache grows with the active context.

Local conversion

This checkpoint was produced from the source BF16 weights with:

mlx_lm.convert \
  --hf-path empero-ai/Qwen3.8-9B \
  --mlx-path Qwen3.8-9B-MLX-MXFP4 \
  --quantize \
  --q-mode mxfp4 \
  --q-group-size 32 \
  --q-bits 4

Conversion environment:

mlx-lm==0.31.3
mlx==0.32.0
transformers==5.15.0
Python 3.12

Smoke-test results

The converted checkpoint was loaded and tested locally on an Apple silicon Mac with 24 GB of unified memory.

Test	Result
Short deterministic generation	Passed (`MXFP4 OK`)
Tool call generation	Passed (`get_weather(city="Taipei")`)
Tool-result continuation	Passed
Peak memory during short generation	Approximately 4.94 GB
Short generation throughput	Approximately 65 tokens/s

These numbers are a functional smoke test, not a comprehensive benchmark. Throughput varies with Mac model, prompt length, thermals, and generation settings.

Evaluation

No full downstream benchmark suite has been run specifically on this MXFP4 checkpoint. The following results are reported by the source model authors for the unquantized Qwen3.8-9B checkpoint and should not be interpreted as measured MXFP4 results.

Task	Metric	Qwen3.5-9B base	Source Qwen3.8-9B
GSM8K CoT	Exact match (flexible)	0.885	0.870
GSM8K CoT	Exact match (strict)	0.875	0.850
MMLU CoT, 57 subjects	Accuracy (flexible extract)	0.546	0.751
MMLU CoT, 57 subjects	Accuracy (strict match)	0.251	0.511

See the source model card for the authors' evaluation methodology and additional details.

Limitations

Quantization can change output probabilities and may reduce accuracy relative to the BF16 source model.
The model can produce incorrect, biased, or fabricated information. Verify important outputs independently.
Function-calling reliability depends on the surrounding tool parser, chat template handling, stopping criteria, context management, and agent-loop limits.
The fine-tuning data was text-only. Vision behavior is inherited from the Qwen3.5 base and was not evaluated by the source model authors or during this conversion.
Long-context behavior was not validated at the full architectural limit.
This model is not intended to provide professional medical, legal, financial, or safety-critical advice without qualified human review.

License and attribution

The weights are distributed under the Apache License 2.0, following the source model and its Qwen3.5-9B base. Users are responsible for reviewing and complying with the applicable license and policies.

Source fine-tune: empero-ai/Qwen3.8-9B
Base model: Qwen/Qwen3.5-9B
Source model developer: Empero
MLX framework: ml-explore/mlx
MLX language-model tooling: ml-explore/mlx-lm

Acknowledgements

Thanks to Empero for releasing Qwen3.8-9B, the Qwen team for Qwen3.5-9B, and Apple's MLX team for the Apple-silicon inference and quantization ecosystem.