knoopx/Swift-Qwen3.8-27B-NInfer

Swift-Qwen3.8-27B — NInfer Artifact Card

Swift-Qwen3.8-27B is a token-efficient, thinking-oriented, multimodal (text + image/video → text) 27B finetune of Qwen/Qwen3.8-27B. The upstream repo publishes BF16 safetensors (18 shards, ~55.6 GB) plus a vLLM-quantized NVFP4 partner ukisai/Swift-Qwen3.8-27B-NVFP4. This artifact is a single NVFP4 (W4A4, W8G32-generation) NInfer bundle built from the Swift source pair, intended for serving through the NInfer engine.

This build is a multi-source graft: pre-quantized NVFP4 partners are mixed by byte-repack, with zero conversion or requantization. The NVFP4 text weights (MLP and full-attention) come from the Swift NVFP4 partner; the GDN projection weights come from the w8g32 recipe's own GDN partner (the Swift source ships GDN in BF16); the vision Q4/Q5/Q6 weights and the 6 front-end resources are grafted verbatim from the ostfralla/Qwen3.8-27B-NInfer-nvfp4-w8g32-q4g64-q5g64-q6g64-bf16.ninfer artifact. The profile carries no DFlash2 companion weights.

Scope note. This is a derived inference artifact. The upstream ukisai/Swift-Qwen3.8-27B repository remains the source of truth for the model weights and the canonical (Apache-2.0-licensed) distribution. What is described here is a derived, quantized, engine-specific container produced by the graft run.

Model Architecture

Property	Value
Architecture	qwen3_5 (Qwen3_5ForConditionalGeneration)
Internal model id	qwen3.8-27b
Hidden layers	64
Attention	hybrid — 16 full-attention (interval 4: layers 3, 7, 11, …, 63) + 48 GDN (recurrent, short convolution width 4)
Hidden size	5120
Intermediate	17408
Attention heads	24 query / 4 KV (head dim 256)
Vocabulary size	248,320
Max position	262,144
Modality	Multimodal (text + image/video input); 27-layer vision encoder (hidden 1152, intermediate 4304, 16 heads, patch 16, out_hidden 5120)
MTP / draft	1 MTP head (mtp_num_hidden_layers = 1) → MTP speculative decoding
Base	finetune of Qwen/Qwen3.8-27B
License	Apache-2.0 (all source partners)

The NInfer Artifact

The single self-contained inference bundle produced from the Swift source pair:

Property	Value
Filename	Swift-Qwen3.8-27b-nvfp4-w8g32-q4g64-q5g64-q6g64-bf16.ninfer
Size	18,324,067,840 bytes (17.06 GiB)
SHA-256	c6cdeb1100dfc1ab9935ce975742490cdfe53cc0c232082e3841817b6ca20756
Objects	1,307 (tensors 1,301 + resources 6)
model_id	qwen3.8-27b
weights_id	nvfp4
Recipe	qwen3_8_27b_nvfp4_w8g32-v1

Quantization scheme

NVFP4 W4A4: packed 4-bit FP4 (E2M1) weights with per-tensor group scales and input_activations: tensor_group, so activations are also 4-bit at inference. This is a mixed-precision layout: the bulk of the MLP / full-attention / GDN weight matrices are NVFP4, while a set of control tensors stays full BF16/FP32 and the two vocabulary endpoints (token embedding, output head) are row-scaled W8G32.

Weight types present in the artifact:

Weight type	Description	Notes
nvfp4	4-bit FP4 (E2M1) weights, 4-bit activations (W4A4)	MLP, full-attention, GDN
w8g32	8-bit weights, 32-element group scale	vocabulary endpoints (embedding, head)
q4g64 / q5g64 / q6g64	4/5/6-bit weights, 64-element group	vision towers
bf16	Full precision	control weights, MTP head

Format histogram

Format	Count
NVFP4	247
W8G32_F16S	9
Q4G64_F16S	55
Q5G64_F16S	54
Q6G64_F16S	1
BF16	591
FP32	343
I32	1

(1,301 tensors total; the NVFP4 and *_F16S counts together account for the 366 quantized matrices.) Layouts: blockscale-k16-m128x4-v1 ×247, contiguous-le-v1 ×935, row-split-k128-v1 ×119; 6 resources encoded raw-bytes-v1. Zero FP8, zero DFlash2 companion objects.

Provenance / Conversion

This artifact was produced by the NInfer source converter (the NInfer converter source tree), running the W8G32-generation NVFP4 recipe as a multi-source graft (byte-repack, zero conversion / requantization).

Converter

• Module: tools.convert.qwen3_8_27b.w8g32_nvfp4.convert_nvfp4 (in the NInfer converter source tree)
• Recipe: qwen3_8_27b_nvfp4_w8g32-v1
• Mode: multi-source graft (byte-repack; no on-the-fly requantization)
• Sources:
  • Swift NVFP4 partner: ukisai/Swift-Qwen3.8-27B-NVFP4 — 152 MLP + full-attention NVFP4 matrices (with their input-scale divisors)
  • GDN partner: sakamakismile/Qwen3.8-27B-MTP-NVFP4 — 95 GDN NVFP4 matrices (in_proj_qkv / in_proj_z / out_proj packed, + 95 divisors)
  • Vision + front-end graft source: ostfralla/Qwen3.8-27B-NInfer-nvfp4-w8g32-q4g64-q5g64-q6g64-bf16.ninfer — 110 vision Q4/Q5/Q6 tensors + 6 front-end resources (byte-exact)
  • BF16 base: ukisai/Swift-Qwen3.8-27B — 591 BF16 + 9 W8 text endpoints + FP32 + I32

Notes:

• The Swift source ships the GDN projections in BF16, so the w8g32 recipe's own GDN partner supplies the 95 GDN NVFP4 objects.
• The GDN partner's separate qkv/z global scales diverge by up to 1.85× within a fused GDN group, so the 48 GDN fused input divisors are reconciled: d_w = 448*6 / max(amax over fused group).

Known Issues & Limitations

• No quality benchmark run. No KL-divergence or perplexity comparison has been run on this artifact. Quantization fidelity is not validated by a benchmark.
• Not numerically validated. No smoke/quality run against this artifact has been completed; treat quantization fidelity as unmeasured. Validation is structural only: format histogram, object/tensor/resource counts, and byte/alignment integrity.
• No DFlash2. This W8G32 profile carries zero DFlash2 companion weights; MTP (via the 1-layer MTP head) is the only supported speculative backend.

License

This NInfer artifact is distributed under the Apache-2.0 license (the front-matter license: apache-2.0 field follows the knoopx artifact-repo convention for the derived container). All source partners (Qwen/Qwen3.8-27B, the Swift finetune, and the GDN partner) are Apache-2.0. Users remain responsible for complying with the license and applicable laws.

Citation / Provenance

• Upstream repo: ukisai/Swift-Qwen3.8-27B (BF16 source)
• Swift NVFP4 partner: ukisai/Swift-Qwen3.8-27B-NVFP4
• GDN NVFP4 partner: sakamakismile/Qwen3.8-27B-MTP-NVFP4
• Base: Qwen/Qwen3.8-27B
• Vision + front-end graft source: ostfralla/Qwen3.8-27B-NInfer-nvfp4-w8g32-q4g64-q5g64-q6g64-bf16.ninfer
• Inference engine: NInfer (CUDA 13, sm_120a)
• Converter: tools.convert.qwen3_8_27b.w8g32_nvfp4.convert_nvfp4 (recipe qwen3_8_27b_nvfp4_w8g32-v1), multi-source graft