EliovpAI/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit-Paiton-RDNA4

Qwen3-Coder 30B with Paiton on R9700

A ready-to-run coding release for one AMD Radeon AI PRO R9700, 32 GB. Use terminal chat or connect a coding client to the local OpenAI-compatible API.

This Hub repo hosts the compiled Paiton artifact, manifest, launcher and release information. It uses the same executable artifact as the published v1.0.0 container. The container fetches the pinned 18.1 GB quantized checkpoint from cyankiwi and caches it locally. No model weights or private compiler source are hosted here. This is a Paiton runtime package, not a Transformers checkpoint that can be loaded by passing this repository name to from_pretrained.

Version: v1.0.0 · GPU: gfx1201 · Context: 4,096 tokens including output · Concurrent sequences: 1 or 2 · Precision: INT4 G32 / BF16 activations

Download from Hugging Face and start coding

With Linux, Docker, a working AMD GPU driver and the Hugging Face CLI:

hf download EliovpAI/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit-Paiton-RDNA4 \
  --revision v1.0.0 --local-dir ./paiton-qwen3-coder
bash ./paiton-qwen3-coder/serve-docker.sh --chat

Install the CLI with python3 -m pip install huggingface_hub if needed. The launcher pulls the prebuilt image, starts the server and opens chat after it is ready. No local compiler or container build is needed. The default cache is ~/.cache/paiton; set PAITON_CACHE_DIR to choose another directory. Use /reset for a new conversation and /quit to exit chat. The API stays running; stop it with docker stop paiton-qwen3-coder.

For a download without the HF CLI, use the ready-to-run archive and run ./run.sh --chat after extracting it.

Coding clients and API

Client setting	Value
Provider	OpenAI-compatible
Base URL	http://127.0.0.1:8010/v1
Model	qwen3-coder
API key if the client requires a value	local
Context budget	4096, including output
Suggested maximum output	1024

The endpoint is bound to localhost and needs no authentication. Ordinary chat, streaming and automatic function tool calls are enabled. Both ordinary and streamed tool-call arguments were checked with the published runtime. Use focused functions, files and coding questions; whole-repository agent workloads and long-context quality are not qualified.

curl http://127.0.0.1:8010/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen3-coder","messages":[{"role":"user","content":"Write a Python binary search with tests."}],"max_tokens":1024,"temperature":0}'

Omit --chat to start only the API. --offline requires the pinned checkpoint in the cache. --download-only populates the cache without loading the GPU. Use --stock for the stock comparison; stop the existing server before changing modes. See the complete guide for port, container-name and cache options.

Hardware, performance and quality

The tested machine had an R9700, 16 GB host RAM with 4 GB swap, an i5-8400 and AMD driver 6.19.14.31400000. Reserve at least 30 GB free disk for weights, the roughly 5.1 GB image and caches. Model loading took about 46 seconds; cached startup takes roughly two minutes, with extra time for fresh compilation caches. The first weight download is separate. Sampled warm GPU usage peaked near 20.1 GiB, with the weights fully resident on the GPU.

Matched concurrency	Stock output tokens/s	Paiton output tokens/s	Gain
1	104.42	126.68	21.3%
2	101.61	172.82	70.1%

These are the already-published fixed short-workload results: two measured runs of 16 requests per engine/setting, 229–241 actual input tokens and exactly 256 output tokens. All 128 measured requests completed with matching counts. Mean request latency fell from 2.451 to 2.020 seconds at concurrency 1 and from 5.038 to 2.961 seconds at concurrency 2. This comparison uses the fastest stock configuration tested, not every possible checkpoint or runtime.

Both engines passed four executable coding checks and scored 7/8 on the small quality suite. Both failed the JSON sorting check by returning strings instead of numbers. Numerical and generated-text differences remain; the suite does not establish comprehensive quality parity. No long-context, higher-concurrency, LoRA or speculative-decoding performance is qualified. No new performance claim is introduced by publishing these files on Hugging Face.

Full measurements, latency definitions and limitations · Reproduction commands

Artifact use and provenance

The normal launcher uses the identical artifact already inside the container. For manual use of the downloaded Hub artifact, start the image with this read-only mount and artifact path:

docker run -d --name paiton-qwen3-coder \
  --device /dev/kfd --device /dev/dri --group-add video --ipc=host \
  -p 127.0.0.1:8010:8010 -v paiton-qwen3-coder-cache:/models/cache \
  -v "$PWD/paiton-qwen3-coder/overlay:/models/paiton-overlay:ro" \
  -e PAITON_QWEN3_CODER_ARTIFACT=/models/paiton-overlay/qwen3_coder_moe_w4a16_g32_gfx1201.so \
  ghcr.io/eliovp/paiton-vllm-plugin:qwen3-coder-30b-awq-rdna4-v1.0.0

Keep the adjacent .json manifest with the .so. The runtime checks the artifact hash and GPU architecture before loading it. The immutable image is:

ghcr.io/eliovp/paiton-vllm-plugin@sha256:fb47f4ab6073da943e553849985c81326e33b425e1017f97f3f79fad09586a8e

The checkpoint is pinned to cyankiwi/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit revision 4bd30395b72ea6045edd04806c4fea448d4467b3, with compressed-tensors symmetric INT4 groups of 32 and BF16 activations. It derives from the official Qwen3-Coder-30B-A3B-Instruct. The quantizer does not identify its exact upstream weight revision. The Hub quantization relationship associates this runtime package with that checkpoint; this release performs no additional weight quantization.

SHA256SUMS covers the package files and paiton-hub.json records artifact and runtime provenance. The stack uses HIP 7.14.60850, PyTorch 2.12.0+rocm7.14.0, Transformers 5.15.1, and vLLM commit 39bd959b582c85e78e7e0326d49042ce7c3c07ed.

Licenses and links

Apache-2.0, Qwen license, component notices, model provenance and retained license texts accompany this package. Original checkpoint terms continue to apply to weights fetched from the publisher.

Public runtime · Release files · Paiton