Swift-Qwen3.8-27B-NVFP4
NVFP4 (W4A4) quantization of Swift-Qwen3.8-27B,
UkisAI's reasoning-efficient derivative of Qwen3.8-27B. Weights and activations are 4-bit
floating point with per-group-16 scales in the compressed-tensors format, ready to serve
with vLLM on NVIDIA Blackwell GPUs. The MTP head is included, so speculative decoding works
out of the box.
What is in this checkpoint
| Format | compressed-tensors, nvfp4-pack-quantized (W4A4, group size 16) |
| Size on disk | 28.6 GB (BF16 original: 55.6 GB) |
| Quantized | all attention and MLP Linear layers of the language model |
| Kept in BF16 | lm_head, embeddings, the vision tower, the 48 Gated DeltaNet (linear attention) layers, and the MTP head |
| Calibration | 256 samples × 4096 tokens from perfectblend |
| Tool | llm-compressor QuantizationModifier(scheme="NVFP4") |
The linear-attention projections are left unquantized because their fused layout is incompatible with NVFP4; this matches the layout of NVIDIA's own Qwen3.5 NVFP4 checkpoints.
Requirements
NVFP4 needs Blackwell tensor cores (SM100 and newer: B200, B300, GB200, RTX PRO 6000 Blackwell). On Hopper and older GPUs vLLM falls back to weight-only dequantization with no speedup; use the BF16 model or the GGUF there instead.
Evaluation
Measured on one NVIDIA RTX PRO 6000 Blackwell (96 GB, SM120) with vLLM 0.29.0, this checkpoint against the BF16 original, same server flags, MTP speculative decoding with 3 draft tokens, 8 concurrent requests, template-default reasoning effort, sampling temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0, 8,192-token output cap.
| BF16 | NVFP4 (this repo) | |
|---|---|---|
| GSM8K test, first 200 questions, exact match | 98.0% | 96.5% |
| Paired outcome (both right / only BF16 / only NVFP4) | 192 / 4 / 1 | |
| Truncated at 8,192 tokens | 1 | 0 |
| Mean completion tokens | 344 | 391 |
| Median tokens/s per request (8 concurrent) | 54 | 85 |
| MTP draft-token acceptance | 62% | 61% |
| Weights on disk | 55.6 GB | 28.6 GB |
| Weights in GPU memory (vLLM) | ~54 GB | ~29 GB |
The accuracy gap is within sampling noise at this sample size (4 vs 1 discordant
questions, not significant). Outputs were checked for coherence and the FP4 GEMM path was
confirmed to run on Blackwell tensor cores (FlashInferCutlassNvFp4LinearKernel), not a
weight-only fallback. Larger benchmark runs will be added as they complete.
How to use
vLLM
vllm serve ukisai/Swift-Qwen3.8-27B-NVFP4 \
--tensor-parallel-size 1 \
--max-model-len 262144 \
--reasoning-parser qwen3 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--port 8000
Adjust tensor parallelism and context length to your GPU memory. See the base model's vLLM recipe for installation and hardware-specific settings.
Optional MTP decoding
The checkpoint includes the base model's MTP head in BF16. To enable self-speculative decoding, append to the server command above:
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
UkisAI API
Swift is also served through an OpenAI-compatible API at https://ukisai.com/api/swift/v1.
It is free for research purposes and needs no API key. The model id is swift.
License and access
Swift weights are distributed through gated access under the Swift Open License v1.0. Personal, research, educational, evaluation, and commercial use are free for individuals and organizations with annual recurring revenue, including affiliates, of up to US$1,000,000. Above that threshold, commercial use requires a separate Swift Enterprise License. Contact UkisAI for terms.
Citation
@misc{swift-qwen3.8-27b,
title = {Swift-Qwen3.8-27B},
author = {UkisAI},
year = {2026},
url = {https://huggingface.co/ukisai/Swift-Qwen3.8-27b}
}
Acknowledgements
We acknowledge the NVIDIA Innovation Lab for providing access to 8× NVIDIA H100 GPUs to train Swift.
- Downloads last month
- 26