Dexy-Decision-9B

Dexy-Decision-9B is a text decision model for classification, routing, binary judgments and ordered ratings. It accepts a shared state and named questions, returning a probability distribution for each question. Use it through the Python runtime or a System One-compatible POST /v1/systemone service.

Model specifications

Property	Value
Architecture	Qwen3.5-9B + JevVerbalizerHead
Parameters / precision	8.95B / BF16; approximately 17.91 GB of weight files
Default context limit	65,536 tokens, including the rendered state, questions, options and one read token
choice	2–256 candidates
score	2–256 ordered levels
noul	Binary judgment
Input modality	Text and JSON state
License	Apache-2.0

The backbone configuration retains a 262,144-position ceiling; the supplied runtimes default to 65,536 tokens. The JevVerbalizerHead maps the backbone output to typed decision probabilities using the saved verbalizer token IDs and decision biases in decision_config.json. The backbone is loaded as Qwen3_5ForCausalLM; use the supplied runtime to apply the complete decision head. Each question is processed independently. GPU memory must accommodate the weights plus activations and caches; long inputs require additional memory. The vLLM service was verified on an NVIDIA A800 80GB.

Installation

Use Python 3.12 on Linux with an NVIDIA GPU. Both Transformers inference and vLLM serving use the same environment. The following versions are verified together:

Library	Version
PyTorch	2.13.0 (CUDA 13.0)
Transformers	5.19.0
vLLM	0.32.0
Accelerate	1.12.0
Flash Linear Attention / fla-core	0.4.2

The CUDA 13.0 setup was verified with NVIDIA driver 580.65.06. From the complete model repository, install dependencies once:

pip install -r requirements.txt

Transformers inference

python inference.py --model . --request examples/request.json --device cuda:0

Python usage, run from the downloaded repository:

from decision_runtime import DecisionModel

model = DecisionModel(".", device="cuda:0")
result = model.predict({
    "state": {"message": "My card was charged twice."},
    "questions": {
        "team": {
            "type": "choice",
            "instructions": "Which team should handle the message?",
            "criteria": {"billing": "Payments and refunds", "shipping": "Delivery issues"}
        }
    }
})
print(result["answers"]["team"]["probabilities"])

Use the supplied DecisionModel to apply the decision template and scoring configuration. Loading the backbone alone with AutoModelForCausalLM does not implement this interface.

vLLM service

Start the service in the same environment:

CUDA_VISIBLE_DEVICES=0 bash deployment/vllm/start.sh

The service uses native vLLM model execution with a System One protocol adapter.

curl -fsS http://127.0.0.1:19850/health
curl -fsS http://127.0.0.1:19850/v1/systemone \
  -H 'Content-Type: application/json' \
  --data-binary @examples/request.json

Useful startup settings:

Environment variable	Default	Purpose
HOST / PORT	127.0.0.1 / 19850	API bind address and port
MODEL_NAME	Dexy-Decision-9B	Model alias exposed by /v1/models
DECISION_MAX_MODEL_LEN	65536	Context limit
GPU_MEMORY_UTILIZATION	0.40	vLLM fraction of total GPU memory
DECISION_API_KEY	unset	Optional bearer token
LOG_DIR	/tmp/dexy-logs	Backend and API logs

Set HOST=0.0.0.0 for access from other machines. When authentication is enabled, send Authorization: Bearer <token>. Stop the startup script with Ctrl+C to stop both processes.

System One request and response

A request contains state and a nonempty questions object. The optional model field selects the service alias. examples/request.json contains all three question types.

Type	criteria	Answer fields
choice	Map of candidate names to descriptions or null; a list of unique names is also accepted	choice, probabilities
noul	Omitted, null, or a map with optional true / false descriptions	noul = P(true), probabilities keyed by false / true
score	List of level descriptions, ordered low to high	score = expected zero-based level, legend, probabilities keyed by level index

Both backends return the same answer schema under answers, plus model and token-usage information. Every answer also includes type and confidence (the largest returned candidate probability). Distributions sum to one over the provided candidates. For example, score probabilities {"0": 0.2, "1": 0.5, "2": 0.3} give score = 1.1. To choose the most likely level instead, take the argmax of probabilities. Images, video, streaming and joint multi-question reads are unsupported.

Optional "include_diagnostics": true returns prompt token counts and hashes under diagnostics. These fields are omitted by default. Backend kernels and batching can produce small probability differences.

Evaluate any dataset

Point any benchmark client that accepts the System One protocol at /v1/systemone, or use the included JSONL client:

python client.py --input examples/requests.jsonl --output /tmp/predictions.jsonl \
  --url http://127.0.0.1:19850/v1/systemone --concurrency 8

Each input row contains state and questions, with an optional unique id. Other columns, including gold labels, stay local. Output retains row IDs and full answers; apply your benchmark's own scorer. The client retries temporary failures and resumes an existing output. A small metadata sidecar prevents mixing inputs or endpoints when resuming. Use --model for services requiring an explicit alias, --api-key-env for a token environment variable, and --diagnostics for prompt traces. The client uses httpx; it is already included in requirements.txt.

Keep the tokenizer and decision_config.json with the weights. File checksums are provided in SHA256SUMS. See LICENSE and NOTICE for license and attribution.