Sieve-31B

Sieve-31B is a decision model from the Sieve family. You give it a piece of text or JSON (the state) and typed questions, and it returns a calibrated probability for every option of every question. It reads the state once, scores the options directly, and never generates text.

It is a LoRA adapter (rank 128) on google/gemma-4-31B-it (revision 842da37), read at one answer slot per question. The code is at github.com/sthanika-ai/Sieve.

Summary

Backbone	google/gemma-4-31B-it (revision 842da37)
Design	answer slots
Trained weights	LoRA r=128, α=128 on q/k/v/o/gate/up/down_proj of all 60 text layers (979M parameters)
Question types	choice, noul and score
Decision Index	Full score 63.8 on the leaderboard (edition 0.3.2, board data of 2026-10-14); public index 62.52 on the 0.3 suite (our own run)
Temperature	fixed at 8.5495, the training temperature
Token limits	32,768 tokens for each prompt row (the state plus one question), 255 options
GPU memory	about 83 GB in bf16, including PyTorch's cache: one 96 GB GPU, or two GPUs with split

How it works

• Prompt. Each question is one user turn in Gemma's own chat template, with thinking off, followed by an Answer: ( slot. Options keep the order the request gives them.
• Score. At the slot, the final hidden state is multiplied by the output-layer rows of the option label tokens (A, B, …, up to 255 options). A softmax over the question's own options at the fixed temperature T = 8.5495 gives the probabilities.
• Adapter. Applied unmerged with PEFT, exactly as in training and in the submitted run.
• Never. It never truncates, drops or reorders options, tunes prompts per benchmark, or sees gold labels.
• Declared limits. Up to 255 options per question and 32,768 tokens per prompt row (state plus one question). Larger requests are refused as Unsupported. No public request reached either limit.
• Out of memory. The forward chunk is halved and the request retried. A single row that still does not fit raises an ordinary error, so a resumed run retries it.

type	options	returns
choice	the caller's keys, optionally described (up to 255)	a probability per key
noul	yes / no	P(yes)
score	ordered levels	a probability per level

Use

pip install torch==2.8.0 --index-url https://download.pytorch.org/whl/cu128
pip install transformers==5.17.0 peft==0.21.0 accelerate==1.15.0 huggingface_hub==1.32.0 jinja2 httpx pydantic
pip install --no-deps decider-ai==1.6.0 "sieve-decisions @ git+https://github.com/sthanika-ai/Sieve"
pip install --no-deps "decision-index @ git+https://github.com/apolinario/decision-index@62d2f51"

from sieve import load_sieve, decide

m = load_sieve("sthanika-ai/Sieve-31B")
decide(m, {"subject": "Duplicate charge on invoice 4411", "body": "Billed twice. Refund today or we cancel."},
       {"department": {"type": "choice", "instructions": "Which team handles this?",
                       "criteria": {"billing": "invoices, payments, refunds", "technical": "bugs and outages"}},
        "churn_risk": {"type": "noul", "instructions": "Does the customer threaten to cancel?"}})

• Requirements. The decider-ai package (1.6.0) is needed to build the prompts; install it without its dependencies, as above.
• Server. sieve-serve --model sthanika-ai/Sieve-31B.
• Decision Index. python -m decision_index pipeline --engine sieve.decision_index_engine:SieveEngine --option model=sthanika-ai/Sieve-31B --out runs/Sieve-31B.
• Defaults. Besides model, the engine needs no options. It loads the base model at its pinned revision and the adapter from model (this repo, or a local copy of it). The temperature is fixed at 8.5495.
• Smaller GPUs. --option split='{"0": "40GiB", "1": "40GiB"}' spreads the model over two GPUs.

Results

Decision Index

Sieve-31B is on the Decision Index leaderboard, edition 0.3.2 (board data of 2026-10-14), with a Full score of 63.8. The Full score counts the public benchmarks 20%, the maintainers' private tests of the same skills 50% and their private tasks from new domains 30%. Our own run of the edition 0.3 public suite, with the official kit at 62d2f51, gives a public index of 62.52 (raw 71.08, breadth 61.55). The two numbers measure different things: the public index is the public suite alone, while the Full score is mostly private tests, so they are not expected to match.

The area table below is our own run of the public suite. The board's area scores combine public and private tests, so they differ slightly (for example Knowledge & Reasoning 45.4, Language Understanding 63.0, Retrieval & Classification 69.2, Tools & Automation 77.4, Arts & Human Taste 46.6).

• Coverage. All 140,178 scoreable requests over 43 benchmarks were answered, with no truncation, no unsupported request and no error. The index averages 37 of the benchmarks.
• Results. sthanika-ai/Sieve-31B-decision-index-results.

Area	Weight	Skill × 100
Knowledge & Reasoning	25.8%	49.9
Language Understanding	25.8%	65.6
Retrieval & Classification	20.0%	69.5
Tools & Automation	18.3%	78.7
Arts & Human Taste	10%	43.5
Public index		62.52

Latency

Measured with this engine on 1× RTX PRO 6000 Blackwell Server Edition: single process, eager, one request at a time, over 400 random public Decision Index 0.3 requests. Median 163 ms, mean 644 ms, 80th percentile 675 ms. The leaderboard measures latency separately, also on one NVIDIA RTX PRO 6000, and lists a median of 253 ms.

Training

• Method. LoRA r = 128, α = 128, dropout 0, on q/k/v/o/gate/up/down_proj of all 60 text layers (979M trainable parameters); the base stays frozen in bf16.
• Objective. Soft-label cross-entropy at T = 8.5495 + 0.5 × KL to the frozen base model's option distribution + 0.3 × multi-class Brier score.
• Optimisation. AdamW, peak learning rate 5e-5, warm-up / constant / linear decay over the last 20% to 0.1×, gradient clip 1.0, one pass of 2,742 steps (~69 questions per step).
• Data. 190,314 decision questions (76.1M tokens) over 155 tracks.
• Hardware. 2× NVIDIA B200, about 12 h.
• Data sources. The data covers:
  • classification, NLI, verification, retrieval relevance, tool selection, preference and knowledge multiple-choice tasks built from public training splits;
  • programmatic tasks (dates, policies, numeric reasoning, look-ups, long documents);
  • synthetic and model-labelled decision tasks.
• Data handling.
  • Option order was shuffled per question, and prompts were capped at 8,192 tokens. Longer prompts were dropped, never truncated.
  • Decontamination. Every training row was checked against every Decision Index suite row, using 13-gram, short-field and 8-gram near-duplicate checks. No suite row is in the training data.
• Settings. The full settings are in training_config.json.

Files

file	contents
adapter_model.safetensors, adapter_config.json	the LoRA adapter (rank 128, PEFT format)
training_config.json	the full training settings

License

The adapter is released under Apache-2.0, as is its code at github.com/sthanika-ai/Sieve. Use of the base model is governed by Gemma's terms, including Google's prohibited-use policy.