d3-NVFP4-AWQ

d3-NVFP4-AWQ is d3, the 27B multimodal foundation decision model of Decision 3.0, with its decoder's projections in NVFP4 after AWQ smoothing. Give it an input (text or JSON, optionally with images and videos) and the questions you need answered: pick one of several options, say yes or no, or rate on a scale. It answers them all in one call and returns a probability for every answer, as d3 does, in 17.2 GB of weights instead of 52.2 GB, multiplying on the FP4 tensor cores of Blackwell GPUs.

Parameters	26.09B, including the 0.46B vision encoder
Base model	vllm-sr/d3 at revision 093cffba (the same weights as v3.1.0 and the current main)
Quantization	NVFP4 W4A4 (compressed-tensors nvfp4-pack-quantized), as d3-NVFP4, after AWQ smoothing: per-channel scales searched on the calibration data and folded into the layers before them, for each full-attention input_layernorm into q/k/v_proj, each Gated DeltaNet input_layernorm into in_proj_{qkv,z,a,b}, each post_attention_layernorm into gate/up_proj and each up_proj into down_proj (no v_proj into o_proj: d3's attention output is gated). Then FP4 (E2M1) weights rounded to nearest and activations in groups of 16 with FP8 (E4M3) block scales under an FP32 global scale per tensor. 400 projections; the Gated DeltaNet in_proj_a / in_proj_b, the vision tower, the embeddings and the readout stay in BF16 (in_proj_a / in_proj_b take the smoothing scales)
Calibration data	512 text prompts (345,580 tokens) from the validation split of tasksource-jev-typed-decisions, rendered with d3's prompt template
Weights on disk	17.2 GB (d3: 52.2 GB)
Needs	FP4 tensor cores: an NVIDIA Blackwell GPU (compute capability 10.x or 12.x, such as B200, the RTX 50 series or RTX PRO 6000) and a PyTorch with FP4 tensors and block-scaled scaled_mm (tested: 2.11). Tested on NVIDIA RTX PRO 6000
Inputs	Text or JSON, plus images and videos (several per request)
Decision types	Choice · Yes / No · Score
License	Apache-2.0

Highlights

• Scores 65.02 on the Jev Decision Index 0.3 public suite against d3's 65.45 (d3-NVFP4: 64.83), giving d3's top answer on 97.07% of its 312,320 questions; every area within 0.66 points of d3.
• 2.7× faster than d3 on a Blackwell GPU, in a third of its memory: 23.2 ms against 62.6 ms per request (median) on one RTX PRO 6000 in the vLLM Semantic Router model runtime with NVFP4 support at --dtype auto and graphs replaying, in 17.2 GB of GPU memory instead of 52.2 GB.
• Reads images and videos as d3 does: the vision tower is unchanged. On the vision board's public suite it scores 73.51 against d3's 74.15 on the same machine (94.06% of d3's answers), and on Perception Test 77.26% against 77.73% (94.16%).
• Keeps 96.81% of d3's answers on the 14,031 test requests of tasksource-jev-typed-decisions, with accuracy 82.92% against d3's 83.23%.

Quickstart

pip install "transformers==5.17.0" "compressed-tensors>=0.19" "torch>=2.11" torchvision triton pillow opencv-python-headless safetensors accelerate
pip install flash-linear-attention causal-conv1d  # optional: fast GPU kernels for the linear-attention layers

import json
import sys

from huggingface_hub import hf_hub_download, snapshot_download

path = snapshot_download("vllm-sr/d3-NVFP4-AWQ")
sys.path.insert(0, path)
from d3q.load import load_d3

model = load_d3(path, device="cuda:0")

# Text
result = model.system_one(
    state="The order arrived damaged yesterday. The customer has a receipt and asks for a replacement today.",
    questions={
        "route": {
            "type": "choice",
            "instructions": "Which team should handle this request?",
            "criteria": {
                "returns": "Refunds, replacements and damaged deliveries",
                "billing": "Payments, invoices and charges",
                "technical": "Product setup and faults"
            }
        },
        "receipt": {
            "type": "noul",
            "instructions": "Does the customer have a receipt?"
        },
        "urgency": {
            "type": "score",
            "instructions": "How urgent is this request?",
            "criteria": [
                "Routine",
                "Soon",
                "Today"
            ]
        }
    },
)
print(json.dumps(result["answers"], indent=2))

# Text and an image (or several)
receipt = hf_hub_download("vllm-sr/d3", "assets/example-receipt.png")
result = model.system_one(
    state="The customer says the blender arrived cracked and attached the receipt.",
    images=[receipt],  # local paths, http(s) URLs, PIL images or base64 data URLs
    questions={
        "on_receipt": {
            "type": "noul",
            "instructions": "Does the receipt list the blender?"
        }
    },
)
print(json.dumps(result["answers"], indent=2))

# Text and a video (or several)
clip = hf_hub_download("vllm-sr/d3", "assets/example-video.mp4")
result = model.system_one(
    state="A short clip from a test camera.",
    videos=[clip],  # local paths, http(s) URLs, base64 data URLs or frame arrays
    questions={
        "color_change": {
            "type": "noul",
            "instructions": "Does the square change color?"
        }
    },
)
print(json.dumps(result["answers"], indent=2))

load_d3 is d3's own runtime (d3_runtime.py) with its backbone loader swapped for d3q/: transformers loads the checkpoint with its FP4 weights still packed (dequantize=False), and each quantized projection becomes a layer that quantizes its input to NVFP4 with one Triton kernel (16-value block scales under the calibrated global scale, as compressed-tensors computes them) and multiplies on the FP4 tensor cores in torch.nn.functional.scaled_mm. Prompting, noncausal full attention, the FP32 readout, images and videos are d3's own; images and videos are read as d3 reads them. AutoModel.from_pretrained(..., trust_remote_code=True) alone dequantizes the whole checkpoint to BF16 and needs about 52 GB of GPU memory again. On a GPU without FP4 tensor cores the layers cannot run.

Run with vLLM Semantic Router

The vLLM Semantic Router serves d3-NVFP4-AWQ in Engine mode, which runs one model behind the System One API on your own machine, as it serves d3. It verifies the package against its MODEL_MANIFEST.json, as it verifies d3, and keeps the weights in NVFP4 on the GPU, multiplying on the FP4 tensor cores with its own NVFP4 layers (#5225, #5233) rather than d3q/.

curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev --no-launch
export PATH="$HOME/.local/bin:$PATH"
vllm-sr serve -e vllm-sr/d3-NVFP4-AWQ

NVFP4 needs FP4 tensor cores, so it runs on NVIDIA Blackwell GPUs only (see Needs above), with about 18 GB of free GPU memory; any other device refuses the package before its weights load. The platform is detected automatically (--platform cuda chooses it), and --revision chooses a commit. Then ask questions about a request:

curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{
  "model": "vllm-sr/d3-NVFP4-AWQ",
  "state": "Write a Python function that merges two sorted lists.",
  "questions": {
    "kind": {"type": "choice", "instructions": "What kind of work is this?",
             "criteria": {"code": "Writing or fixing code", "math": "Mathematics", "chat": "Anything else"}},
    "reasoning": {"type": "noul", "instructions": "Does answering this need multi-step reasoning?"}
  }
}'

Images and videos go in images and videos as for d3. vllm-sr stop stops it. See Router and Engine modes for limits and options.

Serving performance

On one NVIDIA RTX PRO 6000, the model runtime with NVFP4 support answers a request in 23.2 ms against d3's 62.6 ms (median, one request at a time, graphs replaying, served at --dtype auto), in 17.2 GB of GPU memory against 52.2 GB.

GPU		d3 (BF16)	d3-NVFP4-AWQ, exact kernels	Served (--dtype auto)	Ratio (served)
NVIDIA RTX PRO 6000	Median latency, graphs (500 requests)	62.6 ms	28.5 ms	23.2 ms	0.37×
	p95 latency, graphs	192.3 ms	76.3 ms	64.2 ms	0.33×
	Mean latency, graphs	93.2 ms	40.6 ms	32.6 ms	0.35×
	Median latency, no graphs (1,000 requests)	63.1 ms	47.7 ms	37.8 ms	0.60×
	Mean latency, no graphs	90.2 ms	52.7 ms	42.4 ms	0.47×
	GPU memory after loading	52.2 GB	17.2 GB	17.2 GB	0.33×
	GPU memory, peak of the 20 longest requests	52.8 GB	17.9 GB	17.9 GB	0.34×

• The layers of d3-NVFP4: AWQ changes the weights, not the layers, so this package runs the NVFP4 layers of d3-NVFP4 at its speed; that card explains why they need graphs (#4999) and how their launches were cut. Over the 140,178 requests of the public suite the two take 41.9 and 41.5 ms at the median.
• Served (--dtype auto): the model runtime (vllm-sr serve -e above, or vllm-srun serve) runs a quantized package at --dtype auto by default, and there it also runs the fused element-wise Triton kernels the d3 family allows quantized packages (norms, residual adds and gates; not bit-identical on CUDA), as every score in Evaluation was answered. d3 (BF16) keeps exact kernels at every dtype, so its column is what it serves at.
• Merged projections (#5233): q / k / v and gate / up hold one input and one weight global scale in every layer of this package too, so the runtime runs one quantization and one FP4 product per group, with the same answers byte for byte. It took d3-NVFP4 from 23.3 to 21.5 ms with graphs; the served column above was measured before it, on #5225.
• Answers: graph replays match eager forwards byte for byte.
• Throughput under load: not measured yet.

_{Measured on 2026-10-17 (d3: 2026-10-16), one NVIDIA RTX PRO 6000 Blackwell Workstation Edition (driver 595.99.02): PyTorch 2.11.0+cu128, Triton 3.6.0, FLA 0.5.2, causal-conv1d 1.7.0. Model runtime: #5066 (89402c83) with the NVFP4 support of #5225 (03492c11; since merged into main) and the graph capture of #4999 merged on top; exact profile for the exact columns, and #5225 (95245aec) at --dtype auto for the served column (2026-10-18); this package with a MODEL_MANIFEST.json written for it. Requests: the 1,000 train rows of tasksource-jev-typed-decisions in the #5066 record, one question each, each package in its own process, the device synchronized around every request: the first 500 rows answered a third time so that their shape's graph replays, or all 1,000 a second time without graphs. Memory: allocated after loading and at the peak of the 20 longest of those rows.}

Evaluation

Text: Jev Decision Index 0.3, public suite

	d3 (BF16)	d3-NVFP4-AWQ	Top-1 agreement with d3
Public index ↑	65.45	65.02 (−0.43)	97.07%
Knowledge & Reasoning	55.26	54.90 (−0.36)	93.41%
Language Understanding	68.04	67.38 (−0.66)	98.80%
Retrieval & Classification	66.86	66.39 (−0.47)	97.75%
Tools & Automation	82.57	82.33 (−0.24)	98.68%
Arts & Human Taste	51.02	50.70 (−0.32)	94.77%
Raw index ↑	74.01	73.73 (−0.28)	–
Requests answered	140,178	140,178	–

The five areas are the index's areas (chance-corrected skill; the public index averages them with fixed weights), with differences from d3 in parentheses. Agreement compares the top answer of every question with d3's official run (choice and score: the highest-probability option; yes / no: p ≥ 0.5), 312,320 questions in all, and per area the questions of that area's benchmarks.

The complete public suite (every request of d3's official run), run with the official kit (apolinario/decision-index f8552a1) through the vLLM Semantic Router model runtime with NVFP4 support (one request per call, graphs replaying) and scored by it, against d3's official run in vllm-sr/d3-decision-index; rescoring that run with the same kit reproduces 65.45 exactly. The board's full score (66.3 for d3) also needs private tests that only the maintainers run, so it is not measured here: compare with d3's public 65.45, not 66.3.

Also on the test split of tasksource/tasksource-jev-typed-decisions:

	d3 (BF16)	d3-NVFP4-AWQ	Top-1 agreement with d3
Accuracy	83.23%	82.92 (−0.31)%	96.81% (448 of 14,031 answers changed)

Every usable row of the split at revision 28c39125 (a state of at most 6,000 characters, no repeated options): 14,031 requests of one question each, answered by the vLLM Semantic Router model runtime with NVFP4 support on one RTX PRO 6000, d3 (BF16) by the same runtime on the same machine (83.25% on one AMD Instinct MI300X in the #5066 record). Agreement compares the top answer of every question with that BF16 run (choice and score: the highest-probability option; yes / no: p ≥ 0.5). Two calibration states, an LSAT logic-game passage and a news article, are also the states of 2 of these 14,031 requests.

Images: Jev Decision Index vision board 0.3.2

	d3 (BF16)	d3-NVFP4-AWQ	Top-1 agreement with d3
Public vision index ↑	74.15	73.51 (−0.64)	94.06%

By benchmark (chance-corrected skill, %)

Benchmark	Questions	d3 (BF16)	d3-NVFP4-AWQ	Top-1 agreement with d3
CV-Bench	2,038	76.28	75.96 (−0.32)	97.40%
BLINK	961	60.05	59.40 (−0.64)	96.77%
RealWorldQA	589	73.36	72.25 (−1.11)	96.94%
CharXiv	532	79.45	80.70 (+1.25)	96.99%
InfographicVQA	244	97.81	97.81 (+0.00)	98.36%
Mind2Web	973	84.10	83.42 (−0.69)	95.38%
Winoground	800	85.75	85.50 (−0.25)	98.62%
KIE (CORD+FUNSD)	2,465	92.65	92.72 (+0.07)	99.31%
Moderation (Hateful Memes)	1,213	45.42	41.47 (−3.96)	93.57%
R-Bench-M	665	33.22	32.86 (−0.36)	78.65%
MMMU-Pro vision	1,730	45.84	44.24 (−1.59)	82.08%

All 12,210 rows of the kit's public vision suite (11 benchmarks), one image request per row, through the model runtime with NVFP4 support and scored by the official kit. d3 (BF16) answered the same rows through d3q/ on the same RTX PRO 6000 and scores 74.15, in line with the official vision board's 74.1. Agreement compares the top answer of every question with that BF16 run.

Videos: Perception Test

	d3 (BF16)	d3-NVFP4-AWQ	Top-1 agreement with d3
All 19,140 questions (validation) ↑	77.73	77.26 (−0.47)	94.16%

By area (accuracy, %)

Area	Questions	d3 (BF16)	d3-NVFP4-AWQ	Top-1 agreement with d3
Abstraction	5,607	71.55	71.57 (+0.02)	92.40%
Memory	746	80.56	80.03 (−0.54)	95.31%
Physics	6,938	73.19	72.36 (−0.84)	93.18%
Semantics	5,849	88.66	88.17 (−0.50)	96.85%

Every multiple-choice question of Perception Test validation (19,140 questions over 5,260 videos, three options each, chance 33.3): one request per video with all its questions and the state "A video.", videos read as d3 reads them, through the model runtime with NVFP4 support on one RTX PRO 6000. d3 (BF16) answered the same requests through d3q/ on the same machine and scores 77.73; d3's card reports 77.4 from its internal evaluation, whose prompt may differ.

_{Quantized with llm-compressor 0.14.0 (AWQModifier with these mappings, duo scaling, a grid of 20, then QuantizationModifier with the NVFP4 scheme) and compressed-tensors 0.19.0 (torch 2.11.0+cu128, transformers 5.17.0) on one RTX PRO 6000 in 41 minutes, calibrating the text model layer by layer with the full-attention layers under the noncausal key mask of d3_runtime.PromptEncoder, as at inference. The calibration prompts are text only and none of their states appears in the Decision Index public suite. QUANT_INFO.json records the recipe, versions and calibration data. d3q_engine.py is the Decision Index engine for the kit (--engine d3q_engine:D3QEngine --option model=<this folder>).}

License

Apache-2.0 (LICENSE), as d3. Built on vllm-sr/d3, which is built on Qwen/Qwen3.8-27B (Apache-2.0).

Citation

@misc{d3_2026,
  title        = {{d3}: A Multimodal Foundation Decision Model},
  author       = {{vLLM Semantic Router Team}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/vllm-sr/d3}}
}