Qwen3.8-27B-QUASAR-All-NVFP4-Attention-MLP-DFlash2-W8-NInfer

This is a artifact of Qwen3.8-27B. It features a fully synchronized, end-to-end Full-NVFP4 (W4A4) quantization pipeline. While vanilla NInfer master rejects full-NVFP4 attention geometry, this artifact is compiled under a customized Windows execution layer incorporating the registered QUASAR-QAT layout engine and custom CUDA-graph state slots.

Benchmarks (Tested on RTX 5090 Desktop)

• Speculative DFlash2 Acceptance Rate:: +7.7% relative (56.0% 3-run average vs. 52.0% in legacy hybrid mix)

Quality & Evaluation Baseline (GPQA-Diamond & AIME'26)

Among all public 4-bit NVFP4 builds, the QUASAR-QAT core inside this artifact achieves the highest fidelity, scoring near-parity with the uncompressed BF16 baseline while outperforming Unsloth and Inferact.

The evaluations below were conducted under the strict Qwen3.8-27B reasoning protocol (0-shot, thinking mode ON, rule-scored, pooled $n=396$ for GPQA-D and $n=90$ for AIME):

Model Checkpoint / Profile	Size on Disk	NVFP4 Linears	GPQA-Diamond Score	AIME'26 Accuracy
QUASAR-QAT Full-NVFP4 (This Artifact)	19.8 GiB	496 / 496	90.91	100.0%
Official BF16 Original	55.6 GiB	0 / 496	91.41	100.0%
Unsloth NVFP4 (cometkim baseline)	23.4 GB	168 / 496	89.39	97.78%
Inferact NVFP4	26.4 GB	304 / 496	87.63	96.67%

Note: While standard nvfp4full implementations compromise on attention parameters to fit raw layouts, our customized Windows runtime maps all 496 linear layers seamlessly into hardware Tensor Cores, eliminating the 3% logit noise typical of row-scaled FP8 head conversions.

---

How it was Built & Component Sourcing

This monolithic .ninfer container is compiled using a multi-source extraction pipeline. The converter reads from three distinct repositories to build a single, self-contained deployment artifact:

1. Full-Decoder NVFP4 Text Backbone: Extracted and synchronized directly from the QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 repository. All 496 text linears are mapped bit-exactly into hardware Tensor Cores.
2. Multimodal Vision Tower: Extracted natively from the official Qwen/Qwen3.8-27B factory checkpoint. The converter embeds the complete 333-object vision pipeline into the image, allowing direct processing of high-resolution images, multi-image prompts, and video streams when running with the --vision flag.
3. Dual Speculative Decoding Channels:
   • Multi-Token Prediction (MTP): The official 15-tensor high-precision BF16 MTP module is extracted directly from the QUASAR checkpoint and mapped with an optimized proposal-head lookup.
   • DFlash2 Block Diffusion: Inco AI’s 2B-parameter masked block-diffusion companion (z-lab/Qwen3.8-27B-DFlash2) is embedded at the file's tail as a 66-tensor W8/BF16 suffix.

No separate model or drafter downloads are needed at runtime. The engine dynamically activates execution leaves based on your --spec choice.

---

Layer Topology & Architecture: Zero Compromises

Unlike legacy nvfp4 or nvfp4full profiles—which cautiously retain the critical Self-Attention layers in heavy FP8 or BF16 layouts—this artifact delivers the absolute limit of 4-bit precision by utilizing a synchronized Full-NVFP4 backbone.

Every single text linear transformer block has been converted bit-for-bit against the high-fidelity Quantization-Aware Training (QAT) source checkpoint, ensuring zero cross-domain mismatch or activation noise.

• Self-Attention Blocks (QKV & Out): Pure NVFP4 (4-bit). Eliminates activation scale drift and token-loop degradation.
• Transformer Backbone (MLP blocks): Pure QUASAR-QAT NVFP4 (4-bit). Retains near-BF16 reasoning depth (0.909 GPQA Diamond / 100% AIME'26).
• Gated DeltaNet (GDN QKVZ & Out): Pure NVFP4 (4-bit).
• Vocab Endpoints (Embeddings & LM_Head): Native W8G32_F16S.
• Speculative Companion: Integrated uncompressed W8 z-lab/DFlash2 engine (66 tensors embedded as a single postfix bundle). No second file needed.

---

Lineage & Compilation Provenance

Qwen/Qwen3.8-27B (BF16 Baseline)
  └─ QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 (Full-decoder 4-bit QAT, vLLM)
       └─ This Artifact (Windows native .ninfer container with DFlash2)

Compiled on Windows 11 Pro via MSVC 17.14, CUDA Toolkit 13.1, and CMake, using the official vcpkg package manager toolchain to statically resolve native production-grade bindings for FFmpeg, libcurl, and zlib.

The Windows runtime compatibility layer uses an optimized port based on the Don-Chad/ninfer-3090 sub-allocator, completely cleaned of arbitrary kernel reschedules or architecture retargets, natively optimizing execution parameters for desktop sm_120a (Blackwell) instructions.

---

License & Attribution

Distributed under the Apache License 2.0, matching the official terms of Qwen3.8, QUASAR, and pinned DFlash2 models. Users remain fully responsible for complying with the underlying source terms.

• Base architecture and weights by Qwen Team.
• QAT model distillation by QUASAR-QAT Authors (arXiv:2608.13966).
• Speculative engine by Inco AI / z-lab.
• Underlying C++/CUDA runtime framework by Neroued and contributors.