d3-ultra-INT4

d3-ultra-INT4 is d3-ultra, the newest 27B multimodal decision model of Decision 3.0, with its decoder's projections in int4. Give it an input (text or JSON, optionally with images and videos) and the questions you need answered: pick one of several options, say yes or no, rate on a scale, or point to where something is in an image or a video. It reads the input once per request, answers every question on it and returns a probability for every answer, as d3-ultra does, in 17.6 GB of weights instead of 52.2 GB.

Box loses 3.4 points. Box accuracy at IoU 0.5 is 0.836 against d3-ultra's 0.870, and most of the loss is on receipts (0.793 against 0.957) and screenshots (0.961 against 0.999). Text, images and images in options stay within about 1 point. If Box answers on documents or screens matter, use d3-ultra-FP8, which keeps Box and is also faster.

Parameters	26.09B, including the 0.46B vision encoder
Base model	vllm-sr/d3-ultra, the release of 2026-10-24 (the weights its Decision Index runs used)
Quantization	INT4 weight-only W4A16 (compressed-tensors pack-quantized): asymmetric int4 weights with zero points in groups of 32 with BF16 scales, from AWQ smoothing (activation-aware scales folded into the preceding norms and up_proj) followed by GPTQ rounding; activations stay in BF16. 400 projections; the Gated DeltaNet in_proj_a / in_proj_b, the vision tower, the embeddings and the readout stay as published, except where the scales were folded
Calibration data	The 832 requests of d3-turbo-INT4, shuffled together: 512 text prompts from the first train shard of tasksource-jev-typed-decisions, 256 Box requests (32 per family) and 64 images-in-options requests from d3-turbo's training data. No image that appears in the Box or images-in-options development and test splits below was used; no video requests
Weights on disk	17.6 GB (d3-ultra: 52.2 GB)
Needs	No special matrix units. Measured on one AMD Instinct MI300X
Inputs	Text or JSON, plus images and videos (several per request); images can also sit inside questions and options
Decision types	Choice · Yes / No · Score · Box · Track, all measured against d3-ultra
License	Apache-2.0

Highlights

• 71.47 on the Jev Decision Index 0.3 public suite against d3-ultra's 71.57, every area within 0.5 points, with d3-ultra's top answer on 98.6% of 272,229 questions.
• 74.40 on the public vision suite against 75.35 for d3-ultra on the same GPU, with d3-ultra's top answer on 95.8% of 12,210 questions.
• Images in options and tracking within 2.5 points: images in questions and options 0.941 against 0.952, tracking success AUC 0.754 / 0.757 against 0.778 / 0.782. Box loses more (see the note above).
• 17.6 GB of weights instead of 52.2 GB. It saves memory, not time: 67 ms against 51 ms at the median for one text question at a time on one AMD Instinct MI300X.

Quickstart

pip install "transformers==5.17.0" "compressed-tensors>=0.17" torch torchvision triton pillow opencv-python-headless safetensors accelerate
pip install flash-linear-attention  # optional: fast GPU kernels for the linear-attention layers

import sys

from huggingface_hub import hf_hub_download, snapshot_download

path = snapshot_download("vllm-sr/d3-ultra-INT4")
sys.path.insert(0, path)
from d3q.load import load_d3

model = load_d3(path, device="cuda:0")


def asset(name):
    return hf_hub_download("vllm-sr/d3-ultra", f"assets/{name}")


# Text: several questions, one call
print(model.system_one(
    state="Ticket: the invoice total is wrong after the plan change.",
    questions={
        "route": {"type": "choice", "instructions": "Which team handles this?",
                  "criteria": {"billing": "Billing", "tech": "Technical support"}},
        "urgent": {"type": "noul", "instructions": "Is the customer asking for an urgent fix?"},
    },
))

# Box: who is in the canoe? Three questions, one read of the painting
print(model.system_one(
    state="A painting of fur traders on a river.",
    images=[asset("fur-traders.jpg")],  # local paths, http(s) URLs, PIL images or base64-encoded images
    questions={
        "cat": {"type": "box", "instructions": "The cat"},
        "canoe": {"type": "box", "instructions": "The canoe"},
        "dog": {"type": "box", "instructions": "A dog"},
    },
))

# Photo to etching: the question holds a photo, each option is an etching
print(model.system_one(
    state="An architecture quiz.",
    questions={"pick": {"type": "choice", "instructions": "Which 18th-century etching shows the same building as <building>?",
                        "criteria": {"a": "<a>", "b": "<b>", "c": "<c>", "d": "<d>"}}},
    assets={"<building>": asset("pantheon.jpg"), "<a>": asset("pantheon-a.jpg"), "<b>": asset("pantheon-b.jpg"),
            "<c>": asset("pantheon-c.jpg"), "<d>": asset("pantheon-d.jpg")},
))

load_d3 is d3-ultra's own runtime (d3_runtime.py) with its backbone loader swapped for d3q/: transformers loads the checkpoint with its weights packed, and each quantized projection becomes a layer that unpacks its int4 weight to BF16 with one Triton kernel and multiplies in BF16. Prompting, reading the input once, the FP32 readout, Box and Track answers, images and videos are d3-ultra's own; the answers are formatted as on d3-ultra's card. AutoModel.from_pretrained(..., trust_remote_code=True) alone does not run int4 weights.

Checked on one AMD Instinct MI300X with transformers 5.17.0 and compressed-tensors 0.17.0: the three requests above are answered as d3-ultra answers them through the same path. Every edge of the cat and canoe boxes is within 0.003 of d3-ultra's, the dog is not found by either (confidence 0.14 against 0.00), etching d gets 0.996 from both, and the ticket goes to billing (urgent: 0.09 against 0.10).

Run with vLLM Semantic Router

The vLLM Semantic Router model runtime loads d3-ultra-INT4 as it loads d3-ultra, verifies the package against its MODEL_MANIFEST.json, and keeps the weights packed in int4 on the GPU:

pip install -e "src/model-runtime[multimodal]" flash-linear-attention   # in a checkout of vllm-project/semantic-router
vllm-srun serve vllm-sr/d3-ultra-INT4 --revision eaf00167a8dc98cebd17a4270dbda105752baf67 --port 8899 --max-request-bytes 67108864

curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{
  "model": "vllm-sr/d3-ultra-INT4",
  "state": "Write a Python function that reverses a string.",
  "questions": {
    "kind": {"type": "choice", "instructions": "What kind of work is this?",
             "criteria": {"code": "Writing or fixing code", "math": "Mathematics", "chat": "Anything else"}},
    "reasoning": {"type": "noul", "instructions": "Does answering this need multi-step reasoning?"}
  }
}'

This package was served with the model runtime from main as of 2026-10-29. A runtime that predates d3-ultra's prompt layout and per-request temperature refuses the package, since the model identity in its manifest differs. vllm-srun takes a Hub package that is not built in only at a full 40-hex --revision; requests with videos need --max-request-bytes above the default 8 MiB. The Box, images-in-options, tracking and tasksource results below come from the model runtime run on a local copy of this repository (the same weights and loaded files).

The full speed benchmark of the d3-turbo packages has not been run on this package. One question per request on the tasksource test split below, one request at a time through the model runtime on one AMD Instinct MI300X: median 67.4 ms against 51.3 ms for d3-ultra. The runtime's fused int4 kernel runs on CUDA only, so on ROCm every int4 projection unpacks its weights to BF16 before a BF16 product, which costs time on short requests.

Evaluation

Every comparison is against d3-ultra (BF16) on the same questions. The text index compares with d3-ultra's own public run (vllm-sr/d3-ultra-decision-index); images, Box, images in options and tracking compare with d3-ultra run on the same MI300X through the same engine. d3-ultra's vision run on this MI300X gives 75.35 against 75.33 in its public run, so the differences below are the quantization's.

Text: Jev Decision Index 0.3, public suite

	d3-ultra (BF16)	d3-ultra-INT4
Decision Index (public)	71.57	71.47
Raw index	78.59	78.50
Knowledge	62.3	61.8
Language	72.4	72.5
Retrieval	71.8	71.8
Tools	87.8	87.9
Arts	63.5	63.1

The 140,178 requests of d3-ultra's public run, scored by the official 0.3 kit; the index reads the 110,087 requests (272,229 questions) of its 37 benchmarks. Re-scored by the same scorer as this package, d3-ultra's results give the 71.57 of its public run. This package's run, on another build of the same 0.3 suite, also answered 442 requests that d3-ultra's run does not have; they are left out, and scoring this package on all its own requests gives the same 71.47. This package gives d3-ultra's top answer on 98.59% of the index's questions (3,826 change); the lowest agreement is on HLE text (86.0%), ChessBench legal moves (88.5%) and GPQA Diamond (88.8%). KL(d3-ultra ‖ this package): mean 0.0042, median 0.000052, 95th percentile 0.015, 99th percentile 0.079.

Images: Jev Decision Index 0.3, public vision suite

	d3-ultra (BF16)	d3-ultra-INT4
Public vision index	75.35	74.40

By benchmark (accuracy)

Benchmark	d3-ultra (BF16)	d3-ultra-INT4
CV-Bench	0.892	0.882
BLINK	0.816	0.793
RealWorldQA	0.815	0.810
CharXiv	0.852	0.853
InfographicVQA	0.971	0.971
Mind2Web	0.892	0.887
Winoground	0.920	0.917
KIE (CORD+FUNSD)	0.954	0.953
Moderation (Hateful Memes)	0.702	0.695
R-Bench-M	0.526	0.516
MMMU-Pro vision	0.513	0.502

All 12,210 rows, both packages through the same engine on the same MI300X. This package gives d3-ultra's top answer on 95.79% of the questions (514 change), most of the changes on MMMU-Pro vision (88.1% agreement) and R-Bench-M (88.1%), where d3-ultra itself is right about half the time; every other benchmark agrees on 93.9% (BLINK) to 100%. KL: mean 0.014, median 0.00072, 99th percentile 0.22.

Images: Box and images in options

Measure	d3-ultra (BF16)	d3-ultra-INT4
Box: accuracy at IoU 0.5	0.8703	0.8358
Box: accuracy at IoU 0.75	0.7084	0.6378
Box: confidence AUROC	0.9639	0.9589
Images in questions and options: accuracy	0.9524	0.9415

The Box test set of d3-turbo (5,836 items) and its images-in-options test set (2,752 questions), scored as d3-turbo's evaluation scores them. On Box this package hits or misses as d3-ultra does on 95.7% of the items (225 hits become misses, 24 misses become hits): −3.4 points at IoU 0.5 (95% interval −4.0 to −2.9), and at IoU 0.75 7.1 points lower (mean IoU 0.732 against 0.771). Receipts lose the most (0.957 → 0.793), then screens (0.999 → 0.961) and COCO spatial (0.650 → 0.635); COCO exemplar 0.679 → 0.670, unique 0.799 → 0.792, and charts, shapes and tables stay within 0.011. On images in options it is right or wrong as d3-ultra is on 98.4% of the questions (37 right answers become wrong, 7 wrong ones right): −1.1 points (95% interval −1.6 to −0.6).

Videos: tracking

Measure	d3-ultra (BF16)	d3-ultra-INT4
Track, 1 update per second: success AUC / at-time accuracy at IoU 0.5 / presence F1	0.778 / 0.891 / 0.974	0.754 / 0.872 / 0.963
Track, 2 updates per second: success AUC / at-time accuracy at IoU 0.5 / presence F1	0.782 / 0.887 / 0.968	0.757 / 0.867 / 0.956

The 96 tracking test clips of d3-turbo of at most 60 s (2,690 moments at rate 1, 5,377 at rate 2), sent as their own JPEG frames muxed into Matroska at the clip's frame rate without re-encoding. At rate 1, the tracks of the three targets of one clip (120 moments) come back with times that do not line up with the test's moments for both packages and count as misses. This package hits or misses as d3-ultra does on 97.4% and 96.7% of the moments with a visible target, and calls the target visible or not as d3-ultra does on 97.6% and 96.8%.

Text: tasksource-jev-typed-decisions, test split

Type	Requests	Top-1 agreement with d3-ultra	Accuracy, d3-ultra (BF16)	Accuracy, d3-ultra-INT4
Choice	11,033	97.34% (293 changed)	82.66%	82.80%
Yes / No	2,057	97.67% (48 changed)	88.53%	88.24%
Score	941	95.75% (40 changed)	74.71%	75.03%

Every usable row of the test split of tasksource/tasksource-jev-typed-decisions (a state of at most 6,000 characters, no repeated options): 14,031 requests of one question each, answered by the vLLM Semantic Router model runtime on one AMD Instinct MI300X. Agreement compares the top answer of every question with d3-ultra on the same requests (Choice and Score: the highest-probability option; Yes / No: p ≥ 0.5). An answer is correct when it is the request's target (Yes / No: the target's side of 0.5). Accuracy is given per type only, since the three types count a correct answer differently. The text calibration prompts come from the train split of the same dataset, and none of their states appears in the test split, so this checks the package on the distribution it was calibrated on; the Jev Decision Index suites above are the independent check. KL(d3-ultra ‖ this package), mean / median: Choice 0.015 / 0.000062, Yes / No 0.0075 / 0.000044, Score 0.014 / 0.00052.

How the package was chosen

Four int4 builds were made with the recipe of d3-turbo-INT4: groups of 64 or 32, each with d3-turbo-INT4's calibration requests or with its Box rows replaced by rows from d3-ultra's own Box training data. They were measured on development data, not on the test splits above, against rules set before any of them was measured: lose at most 2 points of accuracy on the Box development split (480 items, 60 per family) and on the images-in-options development split (180 questions, 30 per family), and give d3-ultra's top answer on at least 97.0% of 10,000 rows of the first train shard of tasksource-jev-typed-decisions. Among the builds that pass, the best Box accuracy wins, then the best agreement.

Package	Box, dev split, IoU 0.5	Images in options, dev split	Agreement on the 10,000 train rows
d3-ultra (BF16)	0.902	0.967	100%
Groups of 64, d3-turbo-INT4's calibration	0.852	0.950	97.51%
Groups of 64, d3-ultra's Box rows	0.852	0.944	97.48%
Groups of 32, d3-turbo-INT4's calibration (this package)	0.852	0.956	97.76%
Groups of 32, d3-ultra's Box rows	0.850	0.950	97.84%

Every build failed the Box rule by about 5 points and passed the agreement rule, and all but one passed the images-in-options rule, so neither the group size nor the Box calibration rows move Box: it is lost to int4 rounding on this model. This package is the one the ranking picks, published with the Box loss stated, and was then run once on the test splits above.

_{Built on 2026-10-29 by awq.py in bench/d3-quant-eval, the recipe of vllm-sr/d3-turbo-INT4 with groups of 32, on one AMD Instinct MI300X. For each group of projections that read one input, a per-input-channel scale is searched to minimise the squared error of the decoder layer's output with those projections rounded to int4, measured on 96 of the shuffled calibration requests, and folded into the norm or up_proj before it; GPTQ then rounds each projection column by column through the inverse Hessian of its smoothed inputs over all 832 calibration requests. Evaluation runs, all on one AMD Instinct MI300X: the Decision Index text suite with the official 0.3 kit through d3q, one request at a time; the public vision suite with the same kit; Box, images in options, tracking and tasksource through the model runtime from main as of 2026-10-29 at its default settings.}

License

Apache-2.0 (LICENSE), as d3-ultra. Built on vllm-sr/d3-ultra.

Citation

@misc{d3ultra_2026,
  title        = {{d3-ultra}: A Multimodal Decision Model},
  author       = {{vLLM Semantic Router Team}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/vllm-sr/d3-ultra}}
}