Instructions to use ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4")
model = AutoModelForCausalLM.from_pretrained("ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4

SGLang

How to use ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4 with Docker Model Runner:
```
docker model run hf.co/ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4
```

Qwen3-Coder-30B-A3B-MXFP4

OCP MXFP4 quantization of Qwen3-Coder-30B-A3B-Instruct. 61.1 GB to 17.17 GB (3.56x), reaching a native Blackwell FP4 kernel, with the highest HumanEval score of any 4-bit build we made from this base model.

Read this first: our NVFP4 build of the same model is faster on the same hardware. On a single B200 this checkpoint runs at 0.66x to 0.74x of Qwen3-Coder-30B-A3B-NVFP4. That is a kernel-maturity gap in vLLM 0.27.1, not a property of the number format, and the section below shows exactly where it comes from. We are publishing this anyway because the comparison is the useful part, and because if your stack standardises on OCP MXFP4 you should have a properly built one to run.

Measured on one B200

vLLM 0.27.1, single B200, input 1746 tokens / output 256, unique prompt per request, --ignore-eos, temperature 0. Ratios are taken over concurrency 32 to 128, where repeated samples of the same run vary under ~2.6%; concurrency 256 is past saturation and varies 12 to 26%, so nothing is claimed from it.

concurrency	bf16	FP8	W4A16	MXFP4 (this)	NVFP4
32	4454.2	3978.8	3013.3	3357.7	5099.2
64	6448.2	6054.7	4292.4	5395.0	7608.3
128	8586.2	8130.3	5427.7	7339.8	9941.8

Against FP8 this runs at 0.84x to 0.90x. Against NVFP4, 0.66x to 0.74x. It does beat W4A16 at every rung, which is the expected ordering: MXFP4 computes on the FP4 tensor cores while W4A16 dequantizes to bf16 before every matmul.

Energy tells the same story from the other side. At concurrency 128 this checkpoint delivers 10.13 tokens per joule against FP8's 9.48 and NVFP4's 12.82 — so it is more efficient than FP8 while being slower than it, which is what you would expect from a format that moves fewer weight bytes but has not yet got the fused kernel.

Code quality

lm-eval 0.4.12, humaneval_instruct, n=164, served through vLLM.

model	pass@1	stderr
bf16 (control)	0.9207	0.0212
FP8	0.9146	0.0219
W4A16	0.9268	0.0204
MXFP4 (this)	0.9268	0.0204
NVFP4	0.9024	0.0232

This is the top score among the 4-bit builds, and it means nothing on its own. The benchmark moves in steps of one problem out of 164, which is 0.61 percentage points, and rerunning an identical configuration moves the score by exactly one problem — we measured that twice independently on H200. The whole spread across these five formats is four problems. Read the table as "no format is distinguishable from any other here," not as a ranking.

Why it is slower than NVFP4: different kernel, same silicon

Both formats reach a native SM100 FP4 path. They do not reach the same one.

	NVFP4	MXFP4 (this)
MoE kernel	`FLASHINFER_TRTLLM` (`trtllm_fp4_block_scale_moe`, fused)	`CutlassExpertsMxfp4`
Linear	TensorRT-LLM path	`FlashInferMxFp4LinearKernel`
scale block	16 elements, FP8 E4M3 scale	32 elements, E8M0 scale

NVFP4 gets TensorRT-LLM's fused block-scaled MoE kernel; MXFP4 gets a CUTLASS grouped-experts path. On a 48-layer, 128-expert MoE that difference is most of the gap. Neither number is an emulation fallback — we check the engine log for the backend selection line and for the "does not have native support for FP4" warning on every run, and this build reports mm_fp4_sm100a with no fallback.

On the published +15% claim. arXiv:2509.23202 reports MXFP4 ahead of NVFP4 by about 15%. We do not reproduce that here; we measure the opposite by a wide margin. This is one model, one engine version, one recipe, on one GPU, so it is not a claim that MXFP4 cannot be made fast — it is a claim about what vLLM 0.27.1 does with a Qwen3 MoE today. If you run this on a different engine we would like to see the numbers.

Hardware

MXFP4 needs FP4 tensor cores, so this is Blackwell (SM100 and up). On Hopper (H200, H100) there is no native FP4 path and none of the numbers above apply. First load on a fresh pod spends about four minutes autotuning FlashInfer CuTe-DSL kernels (21 profiles) before the server comes up; the result is cached under /root/.cache/flashinfer, but a new pod pays it again. That is a cold-start cost, not a steady-state one.

Serving

vllm serve ThakiCloud/Qwen3-Coder-30B-A3B-MXFP4 \
  --max-model-len 8192 --gpu-memory-utilization 0.90

vLLM selects the MXFP4 path from the checkpoint config; no --quantization flag is needed. Confirm it took the native path by looking for CutlassExpertsMxfp4 in the startup log.

Which of our 4-bit builds to use

Blackwell, want speed → NVFP4. Fastest of everything we measured, 1.22x to 1.28x over FP8.
Blackwell, standardised on OCP MXFP4 → this one.
Hopper, or smallest possible footprint → W4A16. Note that on Hopper it is a memory play too, at 0.81x to 0.84x of FP8.
Just want it fast and 4-bit is not a requirement → FP8.

Caveats

One model, one engine version, one run per cell on this checkpoint. The SM90 arms in the same campaign were run twice; this one was not.
HumanEval only. No multilingual, long-context, or agentic evaluation.
Concurrency 256 is measured but excluded from every ratio above.
Quantization recipe and calibration are in recipe.yaml in this repo.

Provenance

Quantized by ThakiCloud with llm-compressor. Base model Qwen/Qwen3-Coder-30B-A3B-Instruct is Apache-2.0 and not gated; this derivative inherits that license. Throughput, energy and quality numbers are from 2026-08-15-per-hardware-quant-matrix.json in our whitepaper ledger, built from the raw arm artifacts rather than transcribed.