EliovpAI/Qwen3.8-27B-Quark-Qronos-INT4-W4A16-Paiton-RDNA4

Paiton Qwen3.8 27B on AMD Radeon AI PRO R9700

Run amd/Qwen3.8-27B-Quark-Qronos-INT4-W4A16 locally through Paiton's compiled AMD GPU runtime and an OpenAI-compatible API.

This repository contains compiled runtime artifacts, configuration and manifests only, about 7.21 MB. Model weights are downloaded directly from the original publisher and remain in your local cache. The .so files contain executable model graphs and GPU kernels. The private Paiton compiler is not required to use them and is not included.

Release: v1.3.0 · GPU: Radeon AI PRO R9700 (gfx1201, 32 GB) · Batch: 1 · Tensor parallelism: 1 · Context: 8,192 tokens · Text only

Quick start

You need Linux x86-64, Docker and working ROCm device access through /dev/kfd and /dev/dri. The image includes the qualified ROCm 7.14, PyTorch, vLLM and Paiton runtime. This release is qualified on the R9700; other GPUs and runtime versions are not covered by that qualification.

docker run -d \
  --name paiton-qwen38 \
  --device /dev/kfd \
  --device /dev/dri \
  --group-add video \
  --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v paiton-qwen38-cache:/models/cache \
  ghcr.io/eliovp/paiton-vllm-plugin:qwen38-qronos-rdna4-v1.3.0

The first start downloads the 19.9 GB checkpoint directly from AMD's original Hugging Face repository. The named volume caches it for later runs. With cached weights, the published guide reports roughly 10–12 minutes for model assembly on the qualified host. Wait for readiness before sending requests.

The container already bundles the same compiled artifacts published here. You do not need to download the .so separately for this quick start. The upstream repositories are public; no Hugging Face token is required. Run one example server at a time because both use port 8000.

Follow startup:

docker logs -f paiton-qwen38

Wait for Application startup complete. Then, from another terminal:

curl --fail http://127.0.0.1:8000/v1/models
docker exec -it paiton-qwen38 paiton-chat --model qwen38

Use /reset to clear the chat and /quit to leave it. Or call the API:

curl --fail http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen38","messages":[{"role":"user","content":"Explain why the sky is blue."}],"temperature":0,"max_tokens":256,"chat_template_kwargs":{"enable_thinking":false}}'

Model and execution details

The Hub quantization relationship identifies this compiled runtime package with its upstream quantized checkpoint. It does not indicate that this repository publishes a new set of quantized weights.

• Original repository: amd/Qwen3.8-27B-Quark-Qronos-INT4-W4A16.
• Pinned weight revision: 649ca9d47a7de5364c6fcccc0c1b4f6e542e15e2.
• Execution format: Qronos INT4 checkpoint; fitted INT3 MLP decode shadows and W4 LM head.
• One active request; additional requests queue. Changing server flags does not add multi-batch or multi-GPU support to this release.

This release uses the original Qronos W4 checkpoint for prefill and fitted decode-only INT3 MLP shadows during generation. The runtime prepares these shadows locally; this repository contains no fitted weight tensors. Decode quantization intentionally differs from source W4 execution, so bit-identical output to stock W4 is not promised. Thinking is disabled by default.

Performance and quality

This Hub package redistributes the existing v1.3.0 release without changing its artifacts. It does not introduce a new benchmark or claim task-quality equivalence with stock W4 execution. See the published release and runtime guide for the released configuration.

Download and use these Hub artifacts

For manual artifact use, install the Hugging Face CLI in your local Python environment, then download the release into a separate directory:

python3 -m pip install huggingface_hub
hf download EliovpAI/Qwen3.8-27B-Quark-Qronos-INT4-W4A16-Paiton-RDNA4 \
  --revision v1.3.0 --include 'overlay/*' \
  --local-dir ./paiton-qwen38-hf

Use the quick-start Docker command above, adding these two options before the image name:

  -e PAITON_OVERLAY_DIR=/models/paiton-overlay \
  -v "$PWD/paiton-qwen38-hf/overlay:/models/paiton-overlay:ro" \

The runtime validates the artifact hashes before use. Keep the overlay/ directory intact: it contains the exact files expected by the release loader. The .so is loaded by Paiton; it is not a standalone Transformers checkpoint.

For an immutable runtime reference, replace the friendly image tag with:

ghcr.io/eliovp/paiton-vllm-plugin@sha256:c56baf54aca1ad229829c1de26e8792806608e65ee9d06ee210b79cd49f70bc9

File hashes and provenance are in SHA256SUMS and paiton-hub.json. All artifacts are byte-for-byte copies of the published release. This repository contains no checkpoint weights, draft weights, calibration data or compiler sources.

Links and licenses

• Public Paiton plugin and complete model guide.
• Paiton.
• Our other RDNA4 model: Ornith-1.5.

The Paiton release is distributed under Apache-2.0, with retained third-party notices and license texts. Original model and draft terms remain those of their respective upstream repositories. This artifact package does not change those terms.