rdaum/decisive-dory

Decisive Dory

A fine-tune of NVIDIA Dory 1.0 4B for decisions over application-defined answers. Supply context, a question, and the allowed answers. The model scores those answers without generating an explanation.

Use it for workflow routing, tool-call validation, evidence checks, rubric scoring, and action selection. Questions and answer meanings are supplied at request time, rather than fixed to a single classification task.

The model was fine-tuned with PyTorch. It has been evaluated with Eider, an inference server optimized for DGX Spark, and llama.cpp. The GGUF supports llama.cpp's native decision endpoint in builds that include Dory support.

Model and formats

Property	Value
Base model	NVIDIA Dory 1.0 4B
Adaptation	Rank-8 LoRA, merged into the supplied weights
Recurrent computation	Two internal loops per inference
Outputs	Binary probability, named-choice distribution, or ordered-score distribution
Probability calibration	Uncalibrated answer probabilities
Licence	OpenMDW-1.1

Format	Files	Use
BF16 safetensors	Eight root-level shards and index	Merged weights for runtimes with Dory support.
NVFP4 safetensors	nvfp4/	Packed 4-bit weights, configuration, and tokenizer for Eider.
Q8_0 GGUF	dory-decisions-Q8_0.gguf, 4.23 GB	8-bit weights for llama.cpp.
LoRA adapter	adapter/	Standalone adaptation for the original NVIDIA BF16 checkpoint.

All formats contain the same fine-tuned model. NVFP4 and Q8_0 are quantized from the merged BF16 weights. The standalone adapter requires the supplied Python loader; it is not in PEFT format. Apply it only to the original NVIDIA checkpoint. The BF16, NVFP4, and GGUF files already include the adaptation.

How typed decisions work

Answer type	Result	Example
Binary (noul)	Probability of the supplied true condition	Does this tool call satisfy the request?
Choice	Distribution over named options	Which queue should receive this ticket?
Ordered score	Distribution over rubric levels and their expected index	How well does this response satisfy the rubric?

The prompt contains context, a question, and answer descriptions mapped to single-token labels such as A and B. For example:

Question: Is the response supported by the supplied evidence?

Options:
A: {"option":"true","description":"Every claim is supported by the evidence."}
B: {"option":"false","description":"At least one claim lacks support."}

The runtime reads the next-token logits for those labels and applies softmax over the allowed answers. It returns the resulting distribution without generating answer tokens. The model uses its ordinary vocabulary head, with no separate classification head. Dory's two recurrent loops repeat internal computation without producing a chain of thought.

The runtime enforces the answer type and option membership. The model supplies the judgement. This method supports the answer types above, rather than arbitrary JSON schemas. The exact prompt and label mapping matter; the GGUF includes the decision template.

Training data

The following sources provide the decision questions and targets used for fine-tuning. training-data.json records source versions, licences, hashes, sampling, and split rules. The full source datasets are not redistributed here. The evaluation subset includes source attribution and licence terms.

Area	Source	Training task
General decisions	Open-Jev	Control, workflow, reasoning, and game questions with supplied target distributions.
General decisions	Jev Distill Corpus v3, yuri_v3 subset	Diverse typed questions with supplied probability targets.
General decisions	Procedural Typed Decisions	Rule-based retrieval, arithmetic, routing, state tracking, and planning.
Inference and controls	Open-Jev-v1.1	WANLI-derived inference and controlled tool, document, support, and rubric scenarios.
Tools	ToolACE	Assess reference tool calls and variants with invalid arguments.
Evidence grounding	RAGTruth	Judge response support using documents and human span annotations.
Reading	RACE	Select answers to reading-comprehension questions.
Mathematics	GSM8K	Select numeric answers from generated alternatives, without reference reasoning in the input.
Code	Devign through CodeXGLUE	Classify C-function defects.
Navigation	JAT BabyAI	Select expert actions involving navigation, doors, keys, and object placement.
Household tasks	ALFWorld through Early Experience	Select expert commands from admissible actions and observation history.
Board games	Competitive Game Dataset, Connect Four	Select solver-labelled moves, preserving ties between optimal actions.
Interactive tasks	ScienceWorld gold paths	Select among 17 operation types from the goal, room, inventory, and past results.
State tracking	SCONE, Alchemy and Tangrams	Infer final world states from instruction sequences.
Planning	Synthetic planning tasks inspired by PlanBench	Assess legality and choose optimal moves in block stacking and package delivery.
Human judgements	NVIDIA HelpSteer2	Learn correctness, helpfulness, coherence, and preference distributions from human ratings.

Planning tasks use independently generated problems and search-derived targets. Published PlanBench instances and generator code are not used. Training and evaluation keep related source groups together, such as episodes, games, documents, conversations, and equivalent planning problems. Public test examples are excluded from training.

Training method

Rank-8 LoRA with alpha 16 updates 16,220,160 parameters across 288 attention and MLP projections; the base weights remain frozen during training. Training uses two recurrent loops and soft-target conditional cross-entropy plus 0.5 times Brier error over the allowed answers. The adapter is merged into BF16 before quantization. NVFP4 uses NVIDIA Model Optimizer with local-Hessian calibration on 256 training examples. The method uses model activations to choose block scales that reduce projection-output error. All 288 attention and MLP projections use packed E2M1 weights, E4M3 block scales, and an F32 tensor scale. Embeddings, the vocabulary head, and other non-projection tensors retain BF16 precision. Activations also use BF16. Export metadata records the quantization settings and source-weight hashes.

Evaluation

NVFP4 validation

The NVFP4 model scores 78.52% accuracy, 0.2569 Brier error, and 0.9667 cross-entropy on 1,024 validation questions. Quantization uses 256 separate training examples, with no shared question IDs or source groups between calibration and validation. Validation guided quantization selection, so these results are not an independent test. Validation details identify the weights, task mix, and scoring results.

Test and hosted-model comparisons

The 512-question test set includes the exact inputs, answer options, and target distributions used in all tables below. Scoring rules and source attribution explain how to reproduce the results.

Within each base-versus-fine-tune comparison, both models use the same quantization recipe and runtime. All local models use two recurrent loops, identical typed-decision prompts, and no probability calibration.

The model was selected using a separate 1,024-question validation set before the test questions were scored.

NVFP4 — evaluated with Eider

Base Dory and the fine-tune use NVIDIA Model Optimizer with local-Hessian calibration on the same 256 training examples. Both models use the same questions, prompt tokens, answer labels, two recurrent loops, and Eider executable. No fitted probability calibration is applied.

Test task	Questions	Base Dory accuracy	Fine-tuned accuracy	Base Brier ↓	Fine-tuned Brier ↓
SCONE state tracking	192	31.25%	75.52%	0.9245	0.3981
Block-stacking and delivery planning	192	36.98%	96.35%	0.6013	0.1558
HelpSteer2 human judgements	128	48.44%	50.78%	0.3162	0.2278
Combined	512	37.70%	77.15%	0.6512	0.2646

Results and weight hashes identify the models and test configuration.

Q8_0 GGUF — evaluated with llama.cpp

The native /v1/systemone endpoint scored both models with Q8_0 weights. Base and fine-tuned Dory receive identical prompt tokens and answer labels. These results measure the Q8_0 model with the supplied decision template and allowed-answer scoring.

Test task	Questions	Base Dory accuracy	Fine-tuned accuracy	Base Brier ↓	Fine-tuned Brier ↓
SCONE state tracking	192	31.77%	76.04%	0.8954	0.3967
Block-stacking and delivery planning	192	38.02%	96.88%	0.6506	0.1405
HelpSteer2 human judgements	128	49.22%	51.56%	0.3753	0.2338
Combined	512	38.48%	77.73%	0.6736	0.2599

llama.cpp comparison results include the model hashes, runtime configuration, and per-task metrics.

Accuracy counts any answer tied for the highest target probability as correct. Brier error compares the complete predicted and target distributions. These are adapted decision tasks, not scores under the source datasets' original benchmark protocols.

The merged BF16 weights have not been measured on this 512-question test set.

Comparison with hosted decision models

The same 512 questions were submitted to four models through OpenRouter's native decision API on 21 October 2026. Each request used the same context, question, answer descriptions, and option order as the local evaluations. Reference answers were not sent. Hosted models used their providers' prompt templates and probability settings; no prompts or calibration were fitted for this comparison.

Model	State tracking	Planning	Human judgements	Combined	Brier ↓
Base Dory — Q8_0 / llama.cpp	31.77%	38.02%	49.22%	38.48%	0.6736
Base Dory — NVFP4 / Eider	31.25%	36.98%	48.44%	37.70%	0.6512
Decisive Dory — Q8_0 / llama.cpp	76.04%	96.88%	51.56%	77.73%	0.2599
Decisive Dory — NVFP4 / Eider	75.52%	96.35%	50.78%	77.15%	0.2646
TypeSafe Jev 1.14	64.58%	74.48%	55.47%	66.02%	0.3843
Kev 4B	38.54%	36.46%	57.81%	42.58%	0.5134
Liquid d1	58.33%	79.17%	58.59%	66.21%	0.4156
Perplexity Decider 1.2	72.92%	88.02%	57.03%	74.61%	0.3178

Accuracy columns show percentages; bold marks the best result in each column. Combined accuracy weights all 512 questions equally. On human judgements, Jev scores 55.47% and Liquid d1 scores 58.59%. This model scores 50.78% with NVFP4 and 51.56% with Q8_0.

This test measures new examples from task families included in this fine-tune's training. It does not establish a general ranking of decision models. The hosted models' training overlap with these sources is unknown. Small rounding errors in returned distributions were normalized before Brier scoring; this preserves their answer ranking. Comparison details include exact served model versions, per-task scores, and normalization counts. Latency is not ranked here: remote requests include network and provider overhead, while the local models ran on DGX Spark.

JevMan gameplay

Model / runtime	Games	Mean score ↑	Median score	Mean decision latency ↓
Decisive Dory — NVFP4 · Eider	6	2,338	2,280	163 ms
Decisive Dory — Q8_0 · llama.cpp	6	2,362	2,265	342 ms
Jev 1.14 — OpenRouter	12	3,704	3,190	202 ms

Games use the original fixed map, a 300-second gameplay cap, one game at a time, and no artificial latency. Dots show individual game scores. The Jev result includes all 12 games from two runs with the same protocol. Local models run on NVIDIA DGX Spark. Jev runs through OpenRouter, so its request latency includes network and provider overhead. Each local game starts a fresh server. Server startup is excluded from gameplay and decision latency. All completed replays passed the upstream verifier.

Jev scores higher in this sample. Dory has similar scores through both runtimes, with lower request latency through Eider. Unadapted Dory with the same NVFP4 recipe scored 1,097 mean and 1,145 median over six games.

These repeated real-time games do not establish performance across other maps or games. Per-game results and configuration provide the measurements behind the chart.

Limitations

Probabilities are normalized over the supplied answers and have no fitted calibration profile. They are not verified confidence estimates or real-world outcome probabilities. Targets vary by source: factual labels, expert actions, optimal actions, human votes, or supplied distributions. A distribution over optimal actions, for example, is not a probability of winning.

The evaluation does not establish general chat quality, reliable decisions in every domain, full-context quality, or complete agent-task success. ScienceWorld training predicts operation types, not complete object-specific commands. Changing the prompt, label mapping, loop count, or quantization can change the results.

Use the GGUF with llama.cpp

Use a llama.cpp build that includes Dory architecture support to serve the GGUF through /v1/systemone. The build instructions below use the Dory support PR branch. Once merged, an upstream build containing that change also works.

Download

Install the Hugging Face CLI to download the weights. Download only the format you need:

hf download rdaum/decisive-dory \
    dory-decisions-Q8_0.gguf --local-dir ./dory-decisions

For NVFP4, use --include 'nvfp4/*' instead of the GGUF filename, then point your runtime at ./dory-decisions/nvfp4. For BF16, use --include '*.json' '*.jinja' '*.safetensors' --exclude 'nvfp4/*' 'adapter/*' instead of the GGUF filename.

Build llama.cpp with Dory support

These commands require Git, CMake, and a C++ compiler. The example enables CUDA for NVIDIA GPUs and requires the CUDA toolkit. For CPU-only use, omit -DGGML_CUDA=ON. For other GPU backends, follow the llama.cpp build instructions.

git clone https://github.com/ggml-org/llama.cpp llama.cpp-dory
git -C llama.cpp-dory fetch origin pull/30306/head
git -C llama.cpp-dory checkout --detach FETCH_HEAD
cmake -S llama.cpp-dory -B llama.cpp-dory/build \
    -DCMAKE_BUILD_TYPE=Release \
    -DGGML_CUDA=ON \
    -DLLAMA_CURL=OFF \
    -DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_EXAMPLES=OFF \
    -DLLAMA_BUILD_SERVER=ON
cmake --build llama.cpp-dory/build --target llama-server -j 6

Start the server

./llama.cpp-dory/build/bin/llama-server \
    --model ./dory-decisions/dory-decisions-Q8_0.gguf \
    --alias dory-decisions \
    --host 127.0.0.1 --port 8080 \
    --ctx-size 4096 --gpu-layers all

With a GPU-enabled build, --gpu-layers all requests full GPU offload; reduce it if GPU memory is insufficient. Omit that option for CPU-only use. Adjust --ctx-size for your workload and available memory; 4,096 is only an example.

Ask a typed question

curl http://127.0.0.1:8080/v1/systemone \
    -H 'Content-Type: application/json' \
    -d '{"state":"The key is red.","questions":{"colour":{"type":"choice","instructions":"What colour is the key?","criteria":{"red":"Red","blue":"Blue"}}}}'

The endpoint returns answer probabilities with zero generated tokens. The GGUF includes the decision prompt template and answer-label mapping. It also retains the normal chat template.

Question type	Supported input
choice	2–64 named options with text or null descriptions.
noul	True/false, with optional text descriptions; defaults are Yes and No.
score	2–10 ordered text descriptions.

The template accepts text, JSON states, and plain text chat. Serialize tool-call history as state text. Image and audio inputs are unsupported.

Attribution and provenance

NVIDIA supplies the base model; the linked dataset authors supply the training sources. This is an independent derivative under OpenMDW-1.1; see NOTICE.

BF16 provenance, NVFP4 provenance, and file hashes identify the source weights and adapter.