Dexy-Decision-9B-V2

Dexy-Decision-9B-V2 is a fine-tuned decision model based on vllm-sr/d3-flash. Given a shared state and named questions, it returns probabilities for classification, binary judgments and ordered ratings without generating text. Inputs may include text, JSON, images and videos.

The architecture and inference interface are unchanged from d3-flash. The score validator is extended from 2–10 to 2–255 ordered levels; this uses the existing answer-code readout.

Model specifications

Property	Value
Architecture	d3-flash: Qwen3.5-9B multimodal backbone + 255-code decision readout
Parameters	8.394B, including the 0.456B vision encoder
Precision	BF16 backbone; FP32 decision readout and softmax
Backbone context configuration	262,144 positions
choice	1–255 candidates
score	2–255 ordered levels
noul	Binary judgment
Inputs	Text / JSON, images and videos
License	Apache-2.0

Full-attention layers attend bidirectionally over the input; Gated DeltaNet layers retain their causal scan. Each question has its own forward pass, with questions batched by the runtime. Keep readout.safetensors, the tokenizer, chat template and decision_config.json with the backbone weights. Loading the plain backbone alone does not implement decisions.

The context configuration is a position ceiling, not a guarantee that every input length fits GPU memory. The supplied decision configuration has no additional input-length cap (max_length: null); inputs are not silently truncated. Long inputs require additional memory.

Installation

Recommended environment: Python 3.12, Linux x86-64, and an NVIDIA RTX 5090 with CUDA 13.0 support. vLLM nightly is required: vLLM 0.32.0 does not have the d3 model adapter used here. Install the verified CUDA 13.0 nightly and all shared dependencies in one environment:

# Installs the pinned vLLM nightly as well as the Transformers runtime.
pip install --pre -r requirements.txt

requirements.txt selects the immutable nightly wheel 0.33.0rc1.dev485+g0e5b2ba11 (commit 0e5b2ba118cfb3763a3640942e02dd8d93e8ed6c), rather than a moving nightly build. Both inference paths use the same PyTorch 2.14.0, torchvision 0.29.0, Transformers 5.19.0, Accelerate 1.12.0 and Flash Linear Attention / fla-core 0.4.2. No second environment is needed. Use a CUDA 13.0-compatible NVIDIA driver.

Transformers inference

From the complete downloaded model directory:

from transformers import AutoModel

model = AutoModel.from_pretrained(
    ".", trust_remote_code=True, device_map="cuda:0"
)
result = model.system_one(
    state="The customer received a damaged item and asks for a replacement.",
    questions={
        "team": {
            "type": "choice",
            "instructions": "Which team should handle this request?",
            "criteria": {
                "returns": "Damaged goods and replacements",
                "billing": "Payments and charges",
                "technical": "Setup and troubleshooting"
            }
        },
        "replacement": {
            "type": "noul",
            "instructions": "Does the customer request a replacement?"
        },
        "urgency": {
            "type": "score",
            "instructions": "How urgent is the request?",
            "criteria": ["Routine", "Soon", "Immediate"]
        }
    }
)
print(result["answers"])

Or run a saved request:

python inference.py --model . --request examples/request.json --device cuda:0

For an example above the upstream 10-level limit, use --request examples/score_11.json.

Use images=["document.png"] or videos=["clip.mp4"] with model.system_one(...) to include media. The official runtime accepts local paths, HTTP(S) URLs and base64 data URLs; images may also be PIL objects, and videos may be frame arrays. Every question sees the same media. Images are processed at up to 1.6 MP each. Videos are sampled at 2 FPS, with at most 32 frames and up to 0.2 MP per frame. See the upstream usage guide for more examples.

vLLM deployment

From the complete downloaded model directory, start the native vLLM pooling backend:

CUDA_VISIBLE_DEVICES=0 python serve_vllm.py . \
  --served-model-name Dexy-Decision-9B-V2 \
  --runner pooling --port 19850 \
  --max-model-len 4096

serve_vllm.py calls the official vLLM CLI and only raises its d3 API score-validation limit from 10 to 255. The nightly itself loads the backbone and readout.safetensors, applies the checkpoint's attention mode, and provides /v1/systemone.

curl -fsS http://127.0.0.1:19850/health
curl -fsS http://127.0.0.1:19850/v1/systemone \
  -H 'Content-Type: application/json' --data-binary @examples/request.json
curl -fsS http://127.0.0.1:19850/v1/systemone \
  -H 'Content-Type: application/json' --data-binary @examples/score_11.json

The service also exposes GET /v1/models and accepts standard vLLM CLI options. For media requests, provide images or videos as arrays of base64 data URLs or URLs supported by vLLM's media loader. The Transformers API additionally accepts local paths.

Transformers HTTP service

For a service backed by the same Transformers runtime as the Python example:

CUDA_VISIBLE_DEVICES=0 python d3_server.py \
  --model . --name Dexy-Decision-9B-V2 --host 127.0.0.1 --port 19850

It uses the same request endpoint and extended score validator. Set DECISION_API_KEY to require bearer authentication. Its HTTP media inputs must be base64 data URLs; this server does not read client-supplied local paths or fetch remote media URLs.

Request and response format

A request contains state and a nonempty questions map. Each question includes type, instructions and, where applicable, criteria:

Type	criteria	Answer
choice	Map of candidate names to descriptions or null	choice, probabilities
noul	Omitted, or optional false / true descriptions	noul = P(true)
score	Distinct nonempty descriptions, ordered low to high	Expected zero-based score, legend, probabilities

Responses contain model, answers and usage. Choice and score answers include probabilities and confidence (one minus normalized Shannon entropy, as in d3-flash). Noul answers provide P(true) as noul; P(false) is its complement. Choice and score probabilities sum to one over the available options. For score questions, {"0": 0.2, "1": 0.5, "2": 0.3} gives an expected score of 1.1.

Evaluate a dataset

Use any System One-compatible benchmark client, or the included resumable JSONL client:

python client.py --input examples/requests.jsonl --output /tmp/predictions.jsonl \
  --url http://127.0.0.1:19850/v1/systemone --concurrency 1

Each row contains state and questions, with an optional unique id. Gold labels stay local. Optional images and videos fields are forwarded to the server. Use your benchmark's own scorer on the returned answers. vLLM schedules concurrent requests; the Transformers HTTP service serializes requests and batches their questions.

d3_engine.py is also included for the upstream Decision Index engine interface; its benchmark toolkit is installed separately. File hashes are listed in MODEL_MANIFEST.json and SHA256SUMS. See LICENSE and NOTICE for upstream attribution.