modelfaces/granite-4.1-30b-decision

Granite 4.1 30B Decision

Model summary

Granite 4.1 30B Decision is a decision model. It reads one document (the state) and a set of typed questions about it, and returns a calibrated probability distribution over the options supplied with each question. It does this in a single forward pass, without generating text. It is intended for developers who classify, route, triage or check documents and who need probabilities that can be thresholded, for example to send uncertain cases to human review.

It is a rank-16 LoRA adapter and a pointer head on IBM's Granite 4.1 30B base, trained with the open-source Kev framework. Through Kev's server it implements TypeSafe's public System One API (POST /v1/systemone), so the TypeSafe SDK works against it unchanged.

This is an independent community model. It is not an official Kev, TypeSafe or IBM release.

Model details

Developer	modelfaces
Model type	Decision model: a causal language-model backbone run prefill-only, with a pointer head over the options
Backbone	ibm-granite/granite-4.1-30b-base (revision 9bb22a92d17f4a51788c4750d32764ed068d94f2): dense decoder, 64 layers, hidden size 4,096, grouped-query attention with 8 key-value heads. The backbone is frozen; a LoRA adapter is trained on top
Adapter	LoRA rank 16, alpha 32, dropout 0.05, on every attention and MLP projection (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
Head	Pointer head: two 4,096 → 256 projections score each option's closing token against the question's final token; a softmax gives the probabilities
Parameters	28.9B in the backbone (frozen), about 143M in the adapter, 2.1M in the head
Precision	bf16 adapter (562 MB); served in bf16
Context	Training states were at most 8,192 tokens
Calibration	One temperature, T = 1.74, stored in head.pt and applied at load time
Languages	English
License	Apache-2.0 (adapter and head); the base model is Apache-2.0
Version	v1, released 2026-10-17

Input. A state (text, or a JSON object or array rendered as labelled text) and any number of named questions, each of one of three types:

Type	Options	Output
choice	1–255 named options, each with an optional description	a probability per option, the most likely option and a confidence
score	1–255 ordered levels	a probability per level and the expected level index
noul	yes / no, with optional descriptions	the probability of yes

Each question is answered as its own row that continues from the shared state, so questions cannot influence one another. The state is computed once and cached.

Files.

File	Purpose
adapter_model.safetensors	The trained LoRA weights
adapter_config.json	How the adapter attaches to the base model
head.pt	Pointer head, calibration temperature and loader settings
training_config.json	The five delimiter tokens the model was trained with. Kev checks them at load time
kev-granite.patch	Granite support for Kev (see How to use)

The tokenizer is IBM's unchanged Granite tokenizer, which Kev loads from the base model.

Intended uses

• Typed decisions over documents: classification, routing, triage, extraction choices, policy and eligibility checks, and judging a proposed answer against stated criteria.
• Tool-use decisions: whether to call a tool, ask the user for missing information, or decline.
• Workflows that act on confidence: automate the confident cases and queue the rest, with thresholds frozen on a labelled sample of the user's own workload.
• A self-hosted, drop-in replacement for a System One endpoint.

How to use

Stock Kev targets Qwen backbones. Granite needs two small changes, both in kev-granite.patch:

1. Delimiters. Kev marks the state, each question, each option and the decision point with special tokens that exist only in Qwen's vocabulary. The patch maps them to five spare Granite tokens (<|fim_prefix|>, <|fim_middle|>, <|fim_pad|>, <|unused_15|>, <|fim_suffix|>). It also refuses to load a checkpoint whose recorded delimiters differ from the tokenizer's.
2. Hidden-state scaling. Granite's backbone outputs are scaled by config.logits_scaling (16). The patch divides them back before the pointer head.

The patch applies to Kev at commit dd375f27fd61486c52204205e10a0d2e10f3b34a.

Serve it with the Kev repository on one 80 GB-class GPU or larger (H100 80 GB, H200, B200, RTX PRO 6000 96 GB). The base model takes about 58 GB in bf16, and the server about 64 GB resident.

git clone https://github.com/jaredpalmer/kev.git && cd kev
git checkout dd375f27fd61486c52204205e10a0d2e10f3b34a
hf download modelfaces/granite-4.1-30b-decision kev-granite.patch --local-dir .
git apply kev-granite.patch
uv sync --extra serve
KEV_DTYPE=bf16 uv run --extra serve python -m kev.serve --run modelfaces/granite-4.1-30b-decision --port 8008

On first use, Kev downloads the Granite base model at the pinned revision and this adapter, then attaches the adapter to the base in memory.

from typesafe_sdk import Choice, Noul, TypeSafeClient

client = TypeSafeClient(api_key="local", base_url="http://127.0.0.1:8008", model="kev-latest")
response = client.system_one(
    state="I was charged twice for order 1182. Please refund one of the charges.",
    questions={
        "team": Choice(instructions="Which team should handle this?",
                       criteria={"billing": "Charges and refunds", "shipping": "Deliveries", "returns": "Exchanges"}),
        "urgent": Noul(instructions="Does this need a reply today?"),
    },
)
print(response.choices["team"].choice, response.nouls["urgent"].noul)

The calibrated temperature is applied by default; KEV_TEMPERATURE=1.0 returns the raw probabilities. The server is open unless KEV_API_KEY is set.

To load the model directly in Python:

from kev.checkpoint import Checkpoint, LoadOptions

ck = Checkpoint("modelfaces/granite-4.1-30b-decision")
tok, model = ck.load("cuda", LoadOptions.from_env())
print(ck.meta.temperature)   # 1.74

Training data

One epoch over 29,115 records from Kev's public training suites, with states of at most 8,192 tokens.

Component	Records	Content and labels
decision-v7 (replay)	12,576	Ten public classification and comprehension datasets (banking77, BoolQ, AG News, MultiNLI, SST-5, Yelp reviews, TREC, DBpedia-14, Amazon reviews, IMDB) with their native labels, plus generated policy and rule records and compositional records with labels computed by code
hard-v1	6,000	Generated multi-step records in seven families (long policies, trade-offs, probability, multi-hop, temporal and numeric, judging, ambiguity); labels computed by code. Training uses surface templates 0–3 only
documents-v1	5,219	Decisions over US CFPB consumer-complaint narratives
devtools-v1	5,320	Developer-tooling decisions from CodeReviewer, CommitPackFT, Aegis 2.0 and FlakeFlagger

Sources and labels. The public datasets are listed in this card's metadata. CodeReviewer and FlakeFlagger come from Zenodo. The consumer-complaint labels come from Kev's documents-v1 suite: produced by open-weight models, then filtered by judge models and adjudication. No output of Jev (TypeSafe's hosted decision model) was used.

Licences. CodeReviewer, Aegis 2.0 and the FlakeFlagger data are CC-BY-4.0 (the FlakeFlagger repository is BSD-3-Clause, and its test code is under each project's licence). CommitPackFT is MIT, with a per-row repository licence. The CFPB complaint narratives are US government works (public domain). The classification datasets remain under their own licences. Users must follow the attribution terms of each source.

Training procedure

1. LoRA fine-tuning. One epoch from ibm-granite/granite-4.1-30b-base on one NVIDIA RTX PRO 6000 Blackwell (96 GB). AdamW, learning rate 5e-5, weight decay 0.01, gradient norm clipped at 1.0; 8 records per optimizer step (2 per batch, 4 accumulation steps), 3,640 steps; length-sorted batches; gradient checkpointing; bf16 backbone and adapter weights; seed 0. The loss is cross-entropy over each question's options. Option order is shuffled, and "none of the above" options and distractors are inserted at random. A quarter of choice records with states of at most 8,192 tokens also yield a minimal pair: the question with a "none of the above" option, once with the correct option present and once with it removed. Each state is run once and its questions branch from it.
2. Calibration. A single temperature, T = 1.74, minimising negative log-likelihood on 648 questions from held-out public datasets: 448 from the calibration split of Kev's transfer-r3 suite (QNLI, SciQ, TweetEval-offensive, PAWS, MMLU and Emotion, plus two held-out families of generated policy records) and 200 MMLU-Pro questions. None of these sources is in the training data.

Bias, risks and ethical considerations

• Calibrated probabilities can create unwarranted trust. The temperature was fitted on public held-out datasets and does not transfer to every workload. Measure accuracy and calibration on a labelled sample of your own data before setting thresholds.
• Accuracy and calibration shift under domain change. Monitor production error rates.
• Do not use it for consequential automated decisions about people without human review. Biases of the base model and of the training data (including labels produced by other models) are not measured.
• States may contain personal or confidential data. Self-hosting keeps inputs on your own hardware. The server is open unless KEV_API_KEY is set, so apply your own access control and data-handling policy.

Compute

• Fine-tuning: 1 × NVIDIA RTX PRO 6000 Blackwell (96 GB) for 5.0 h of training time, excluding evaluation.
• Calibration: a few minutes on the same GPU.

Checksums

File	sha256
adapter_model.safetensors	0d75b7230d87e5294953e91903ad2edfb0c3630d8dd3ed6b18dfc6dbe8406b1b
head.pt	62a393898c3b637a0d3e2e7b1dbf24846947996d840fce79957717c56bb4151e
kev-granite.patch	a48134a456299781f355c360c730e62bd1a40e9c82815476d7d570302e1097a6

Acknowledgements

• IBM Granite for the base model (Apache-2.0).
• Kev by Jared Palmer for the decision-model framework, pointer head, training suites and calibration recipe (Apache-2.0).
• The authors of the datasets listed above.

Citation

@misc{modelfaces2026granite30bdecision,
  title        = {Granite 4.1 30B Decision: a calibrated decision model on Granite 4.1 30B},
  author       = {modelfaces},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/modelfaces/granite-4.1-30b-decision}},
  note         = {Version 1, released 2026-10-17}
}

Contact

Questions and issues: the Community tab of this repository.