Instructions to use ukisai/Swift-Qwen3.8-27b-int4-AMD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ukisai/Swift-Qwen3.8-27b-int4-AMD with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ukisai/Swift-Qwen3.8-27b-int4-AMD")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ukisai/Swift-Qwen3.8-27b-int4-AMD")
model = AutoModelForMultimodalLM.from_pretrained("ukisai/Swift-Qwen3.8-27b-int4-AMD", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ukisai/Swift-Qwen3.8-27b-int4-AMD with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ukisai/Swift-Qwen3.8-27b-int4-AMD"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ukisai/Swift-Qwen3.8-27b-int4-AMD",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ukisai/Swift-Qwen3.8-27b-int4-AMD

SGLang

How to use ukisai/Swift-Qwen3.8-27b-int4-AMD with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ukisai/Swift-Qwen3.8-27b-int4-AMD" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ukisai/Swift-Qwen3.8-27b-int4-AMD",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ukisai/Swift-Qwen3.8-27b-int4-AMD" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ukisai/Swift-Qwen3.8-27b-int4-AMD",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ukisai/Swift-Qwen3.8-27b-int4-AMD with Docker Model Runner:
```
docker model run hf.co/ukisai/Swift-Qwen3.8-27b-int4-AMD
```

Website • Learn more • GGUF • Enterprise licensing

Swift-Qwen3.8-27b-int4-AMD

AMD Quark AWQ INT4 (W4A16) edition of Swift. The following introduction describes the base Swift results; release-specific details are below.

Swift-Qwen3.8-27B is UkisAI's reasoning-efficient derivative of Qwen3.8-27B, using 58.3% fewer thinking tokens while maintaining near-identical performance (<1% loss) and as a result getting a x1.95 speed-up on several tasks.

The prompt is a sample from LiveCodeBench v6

AMD Quark INT4 release

This is the INT4 W4A16 quantization of Swift for AMD hardware workflows, produced with AMD Quark. It uses Quark's PyTorch workflow and native Hugging Face safetensors export: signed symmetric INT4 weights, groups of 128, and BF16 activations.

The full-precision companion is Swift-Qwen3.8-27b-BF16-AMD.

Property	This checkpoint
Source	Swift-Qwen3.8-27B
Quantizer	AMD Quark AWQ
Weight / activation precision	INT4 / BF16 (W4A16)
Weight grouping	Symmetric, group size 128
Format	Native Quark safetensors, `real_quantized`, `reorder` packing
Weight files	19.513 GB; BF16 source: 55.563 GB
Calibration	128 Pile validation samples, 512 tokens each
Quantized layers	496 eligible language-model linear layers
Preserved components	BF16 vision tower, output head, embeddings, and all 15 MTP tensors

Quark supports preparing models for AMD deployment. This checkpoint was quantized and validated on an NVIDIA H100; AMD/ROCm serving and throughput have not yet been validated. Serving needs a runtime that supports this native Quark INT4 format. The Quark project and installation guide describe its supported CUDA and ROCm environments.

Checkpoint validation

Sanity check	BF16	This INT4 export
Wikitext perplexity	9.16197	9.54254
Arithmetic generation	Pass	Pass
JSON generation	Pass	Pass

Perplexity uses the same eight non-overlapping 512-token Wikitext-2 test windows. The 4.15% perplexity increase is a small sanity result, not a full accuracy benchmark. The packed checkpoint was independently reloaded, including its final configuration and index, and reproduced the evaluation NLLs exactly. All floating tensors are finite; 349 preserved vision/output-head/MTP tensors match the source exactly. Vision inference and MTP decoding were not exercised in this validation. See quantization_report.json.

The Swift benchmarks and speed demonstration below are reproduced from the base Swift model card. They do not measure this Quark export or AMD hardware.

Training approach

We built Swift by identifying reasoning-marker tokens that, in our analysis, trigger overthinking in Qwen’s reasoning rollouts. We then fine-tuned Qwen by penalizing usage of those tokens while it reasons.

Swift produces shorter reasoning traces. In our testing, we also observe fewer overthinking errors.

For maximum gains, Swift also includes a transfer component derived from BottleCap AI's ThinkingCap-Qwen3.6-27B.

Evaluation scope

All results below compare the Qwen3.8-27B BF16 base with the same base plus the Swift adapter.

Benchmarks

Benchmark	Score		Mean tokens			Median tokens
Benchmark	Base	Swift	Base	Swift	Reduction	Reduction
General reasoning
GPQA-Diamond	88.38%	88.28%	15,014	8,855	↓ 41.0%	↓ 58.3%
MMLU-Pro	85.47%	84.95%	2,980	1,603	↓ 46.2%	↓ 28.3%
C-Eval	90.00%	90.62%	1,492	804	↓ 46.1%	↓ 19.3%
IFBench	73.53%	71.80%	8,052	4,657	↓ 42.2%	↓ 50.5%
Mathematics
AIME 2026	98.67%	94.00%	22,014	16,143	↓ 26.7%	↓ 50.2%
HMMT (Nov 2025)	99.33%	96.00%	22,032	15,189	↓ 31.1%	↓ 45.9%
Multimodal
ERQA	67.45%	66.30%	4,137	2,045	↓ 50.6%	↓ 54.6%
Agentic coding
Terminal-Bench 2.1	66.74%	65.84%	37,086	27,272	↓ 26.5%	↓ 38.7%
LiveCodeBench v6	76.76%	81.55%	11,374	8,615	↓ 24.3%	↓ 45.8%

How to reproduce

Serving: BF16 · vLLM 0.27.1 · Qwen3 parser · context 262,144 · thinking xhigh.
Sampling: temperature 1.0 · top_p 0.95 · top_k 20 · min_p 0 · presence_penalty 0 · repetition_penalty 1.
Benchmarks: averages over five seeds (0–4) per model; five trials per task for Terminal-Bench.

Benchmark	Output cap
GPQA-Diamond	100,000
MMLU-Pro	100,000
C-Eval	16,384
IFBench	81,920
AIME 2026	250,000
HMMT Nov 2025	250,000
ERQA	100,000
Terminal-Bench 2.1	Agent/task limits
LiveCodeBench v6	32,768

Efficiency across and versus reasoning efforts

Qwen3.8's reasoning_effort setting lets users choose how much the model thinks. For Swift to be useful across these settings, it needs to reduce thinking while keeping accuracy close to the base. We therefore tested xhigh, medium, and low: thinking-token savings persist at every level.

Reasoning effort	Mean thinking reduction
Xhigh	↓ 41.0%
Medium	↓ 22.7%
Low	↓ 25.8%

The efficiency also holds up against the base's own lower effort settings. On GPQA-Diamond (198 questions, 5 seeds, 990 paired calls), Swift at xhigh is compared with the base at xhigh and at medium:

GPQA-Diamond	Score	Mean tokens	Median tokens
Base · xhigh	88.38%	15,014	6,642
Swift · xhigh	88.28%	8,855	2,771
Base · medium	84.14%	4,451	1,753

Swift retains the accuracy of xhigh while using about half the tokens, although it uses about double the tokens of medium.

Quantized models

Quantized deployment is the intended use for Swift: lower-memory weights paired with shorter reasoning. The INT4 evaluations below retain token savings across GPQA, IFBench, and AIME. On AIME, Swift matches or improves accuracy and reduces output-cap failures by 31–33%.

Benchmark / quantization	Base accuracy	Swift accuracy	Mean token reduction	Median token reduction
GPQA-Diamond Mixed-precision quant W4A16 · thinking tokens	88.69%	88.38%	↓ 32.1%	↓ 50.2%
IFBench Mixed-precision quant W4A16 · completion tokens	72.58%	71.25%	↓ 30.1%	↓ 38.0%
AIME 2026 Mixed-precision quant W4A16 · completion tokens	84.00%	84.00%	↓ 19.0%	↓ 37.5%
AIME 2026 AWQ INT4 · completion tokens	82.67%	84.00%	↓ 22.8%	↓ 34.8%

Quantized evaluation settings

Each row compares the same quantized base with and without the Swift adapter. GPQA and AIME use five seeds; IFBench uses four samples per prompt and strict scoring. Output caps: GPQA 100,000; IFBench 81,920; AIME 32,768. GPQA and IFBench use saved historical base runs. AIME uses template-default effort and counts truncated answers as incorrect. Its shorter cap makes it a separate comparison from the BF16 table.

How to use

PyTorch with AMD Quark

Install the GPU-specific PyTorch and Quark packages from the official installation guide. Validation used Python 3.12, PyTorch 2.11.0+cu128, Transformers 5.2.0, AMD Quark 0.12.post1+cu128.torch2.11, Accelerate 1.15.0, and Safetensors 0.8.0. For AMD, select the corresponding supported ROCm environment.

Download this repository and run the included loader:

hf download ukisai/Swift-Qwen3.8-27b-int4-AMD --local-dir Swift-Qwen3.8-27b-int4-AMD
python Swift-Qwen3.8-27b-int4-AMD/load_quark.py \
  --model Swift-Qwen3.8-27b-int4-AMD \
  --prompt "What is 17 multiplied by 23? Answer with only the number."

load_quark.py imports the packed weights through Quark's PyTorch API. The included quark_compat.py handles the public Quark 0.12 dense-Qwen reload path. The model uses the qwen3_5 Transformers architecture identifier. recipe.py records the AWQ configuration. The example disables thinking for a short deterministic smoke check.

Serving

A serving engine must support native Quark W4A16 signed INT4 with reorder packing and this Qwen architecture. As of September 14, 2026, vLLM's native Quark INT4 support PR remains open. Stock vLLM compatibility and AMD performance are not established by the PyTorch validation above. The preserved MTP head also needs compatible runtime support before speculative decoding can be used.

For standard BF16 serving instructions, see the BF16 companion. The base Swift card also documents the UkisAI API and other Swift formats; that API is separate from this downloadable Quark checkpoint.

License and access

Swift weights are distributed under the Swift Open License v1.0. Personal, research, educational, evaluation, and commercial use are free for individuals and organizations with annual recurring revenue, including affiliates, of up to US$1,000,000. Above that threshold, commercial use requires a separate Swift Enterprise License. Contact UkisAI for terms.

Citation

@misc{swift-qwen3.8-27b,
  title  = {Swift-Qwen3.8-27B},
  author = {UkisAI},
  year   = {2026},
  url    = {https://huggingface.co/ukisai/Swift-Qwen3.8-27b}
}