liquamen

liquamen is a fine-tune of google/gemma-4-31B-it that answers multiple-choice decision questions with a probability for each option. It includes a calibrated last-layer head (vbll_head.safetensors, vbll_head.json) whose output is the answer. For ordinary multiple-choice questions that output is the softmax of the option letters' logits at the answer position, divided by one fitted temperature (T = 1.085); no posterior is involved. For yes / no questions, a Bayesian last-layer residual on the model's hidden state (a Laplace posterior, integrated over) adjusts the two logits.

Inference engines. The repository includes these Decision Index engines; all use the same weights and head:

• lenavg_bayes_balanced_k5_engine:LenAvgBayesBalancedK5Engine (lenavg_bayes_balanced_k5_engine.py, with lenavg_bayes_l6k_engine.py, lenavg_bayes_engine.py, lenavg_engine.py and oavg_engine.py): length-adaptive option-order averaging with cyclic Latin-square shifts for K >= 5 option sets. For a request whose raw length L_raw is at most 939 or that has a single question with rendered text length L_rendered at most 6,000, every question is scored across 4 option orders (2 orders for yes / no) and the results are averaged; longer requests use the single pass. For choice questions with 5 <= K <= 52 options, the 4 option orders are evenly spaced cyclic shifts of the base order so every option rotates across early, middle, and late letter positions. Adds a binary Bayesian correction on averaged yes / no predictions and retains the existing binary posterior on unaveraged yes / no.
• lenavg_bayes_l6k_engine:LenAvgBayesL6KEngine: the same extended length gate (L_raw <= 939 or single-question L_rendered <= 6,000) with the original random re-letterings (oavg_engine.random_orders).
• lenavg_bayes_engine:LenAvgBayesEngine: the L_raw <= 939 averaging gate with the binary Bayesian correction on averaged yes / no.
• lenavg_engine:LenAvgEngine: the L_raw <= 939 averaging gate without the correction on averaged yes / no questions.
• dikit_engine:DikitEngine: the single pass (one forward pass per question).

config.json's jev_requires_head block holds training-time labels (for example "readout": "posterior_predictive"); the engines only check that the block is present, and the head's behaviour is as described above.

Two parts of the model are not optional:

• Attention. In the full-attention layers, each prompt's tokens attend to every token of the same prompt; the sliding-window layers stay causal (window 1,024). transformers' stock attention would run the full-attention layers causally and give different answers, so the model is served with the jev_mem attention function in fullft_mem.py. It registers with transformers' attention interface and runs on PyTorch SDPA (memory-efficient and flash kernels). The engine refuses any other attention.
• The head. The letter logits on their own are not the model's answer.

The weights (model-*.safetensors, vbll_head.safetensors) are hosted here on the Hugging Face Hub; the GitHub mirror, treewaterer/liquamen, holds the code and this card only.

Usage

The engines are lenavg_bayes_balanced_k5_engine.py, lenavg_bayes_l6k_engine.py, lenavg_bayes_engine.py, lenavg_engine.py and oavg_engine.py on top of the single-pass engine dikit_engine.py, with garas_vbll_engine.py, vbll_core.py, fullft_mem.py, garas_engine.py and garas_core.py. They need the Decision Index kit:

pip install "git+https://github.com/apolinario/decision-index@9eb2dbe2a358004c8782c66e40a83ac07b953fec"
REPO=$(python -c "from huggingface_hub import snapshot_download; print(snapshot_download('plantwaterer/liquamen'))")
PYTHONPATH="$REPO" python -m decision_index run --engine lenavg_bayes_balanced_k5_engine:LenAvgBayesBalancedK5Engine --option model="$REPO" --compact --out runs/liquamen

For the single pass, use --engine dikit_engine:DikitEngine instead.

In Python:

import sys
from huggingface_hub import snapshot_download

repo = snapshot_download("plantwaterer/liquamen")
sys.path.insert(0, repo)
from lenavg_bayes_balanced_k5_engine import LenAvgBayesBalancedK5Engine  # needs the decision_index kit installed

engine = LenAvgBayesBalancedK5Engine(model=repo)     # bf16 on "cuda"; the head runs on the CPU
response, _ = engine(
    "The forecast gives a 70% chance of heavy rain this afternoon. The match is outdoors, on grass.",
    {"plan": {"type": "choice", "instructions": "What should the organisers do?",
               "criteria": {"play": "Play as planned", "indoors": "Move it indoors", "postpone": "Postpone it"}},
     "rain": {"type": "noul", "instructions": "Is heavy rain likely?"}})
print(response["answers"])

The inference procedure is as follows:

• Prompts: one prompt per question, with the model's chat template and the options lettered A-Z a-z. A choice question's options are shown in one fixed non-identity order derived from a hash of the request. Yes / no questions use the options Yes and No; when the request gives their definitions (its criteria), they are shown as Yes: <definition> / No: <definition>.
• Read-out: one forward pass per prompt. The final hidden state and the option letters' logits (fp32, with the model's logit soft-cap) go through the head, on the CPU in float64, which gives the probabilities.
• Order averaging (lenavg_bayes_balanced_k5_engine:LenAvgBayesBalancedK5Engine):
  • Which requests: L_raw = request_length(state, questions) and L_rendered = request_length(render_state(state, "text"), questions), where request_length is the sum over the request's questions of [len(state) + len(instructions) + the sum over the question's criteria of (len(str(key)) + len(str(value)))], with Python's len(). A request is averaged if and only if L_raw <= 939 or (len(questions) == 1 and L_rendered <= 6,000); otherwise every question takes the single pass described above.
  • Choice questions with up to 52 options: scored in 4 option orders. For K <= 4 options, the 4 orders are the engine's own order and 3 re-letterings numpy.random.default_rng(20261021 + 1000 * j + K).permutation(K) for j = 1, 2, 3. For 5 <= K <= 52 options, the 4 orders are cyclic Latin-square shifts [(i + s) % K for i in range(K)] with evenly spaced steps s = round(j * K / 4) % K for j = 0, 1, 2, 3 (deduplicated modulo K). For each order the head's output (the softmax of the letters' logits divided by T = 1.085) is mapped back to the options; the 4 vectors are averaged with equal weight, then raised to the power 1 / 0.95438 and renormalised. (0.95438 here and 0.93738 below are fixed constants of the engine, fitted when the averaging was built for allec and used unchanged here.)
  • Yes / no: the native letter softmax of the base order and of the swapped order (No first), averaged, then raised to the power 1 / 0.93738 and renormalised (p). Then the binary Bayesian correction: the head's yes / no readout is computed from the base prompt's hidden state and the log of the averaged native probabilities, once with its fitted residual and posterior variance and once with both switched off (beta = 0, infl = 0); the difference of the two log-odds is added to the log-odds of p. No extra temperature.
  • More than 52 options: the single pass.
  • The re-lettered prompts of a request are packed into the same forward passes as its other prompts.
• Batching: the prompts of one request run in packed rows of up to 32,768 tokens, each prompt with its own positions, so a prompt only attends to itself.
• More than 52 options: handled in two rounds.
• No truncation: a request that does not fit 32,768 tokens is reported as unsupported.

Requirements: one GPU with at least 75 GB (the bf16 weights are 62 GB). Tested on 1 x H100 80GB and 1 x RTX PRO 6000 (96 GB) with torch 2.14.1 (CUDA 13.0), transformers 5.12.1, huggingface_hub, safetensors and numpy. bf16 results differ slightly between GPU types.

Training

liquamen is the uniform average (equal weights, 0.50 + 0.50, in fp32) of two multi-task post-training branches of google/gemma-4-31B-it built on top of our earlier fine-tunes (allec and oenogarum). Each branch was trained for 400 updates (25,600 decision questions of at most 4,096 tokens) over a 164-family mixture combining the base decision pool, procedural state-transition and exact-Bayes tasks, and decontaminated training-split records from ACOS, ANLI, ContractNLI, Amazon ESCI, HoVer, iSarcasmEval, New Yorker Caption Contest, NLI4CT, RAGTruth, SuperGLUE MultiRC, VAST, When2Call and WinoGrande. The second branch applies Prentice-Pyke (1979) case-control intercept calibration (cc_logit_offset) on oversampled sparse binary tasks (ACOS and MultiRC) and 4-class balanced sampling on Amazon ESCI. The attention, renderer and optimiser settings are those of allec. The averaged model's head (vbll_head.safetensors, T = 1.085) was then fitted anew on its own hidden states, on the same head fit and calibration data as allec's head.

Evaluation

Decision Index 0.3, public panel (37 benchmarks), with balanced K>=5 order averaging (lenavg_bayes_balanced_k5_engine:LenAvgBayesBalancedK5Engine). The run used this repository's weights, head and engines, with the kit's runner and scorer at commit 9eb2dbe2, in bf16. The full public suite ran as 16 row shards on single H100 80GB GPUs (with the 119,734 K <= 4 or unaveraged requests shared with lenavg_bayes_l6k_engine:LenAvgBayesL6KEngine, where the two engines are identical, and all 20,886 averaged 5 <= K <= 52 requests evaluated with lenavg_bayes_balanced_k5_engine:LenAvgBayesBalancedK5Engine). This is our own run of the public kit, not a leaderboard entry. The run files are at plantwaterer/liquamen-decision-index. 120,036 of the 140,620 requests (85.4%) were averaged.

	public index	knowledge	language	retrieval	tools	arts
board rule (the test item listed below counted as wrong)	65.79	53.86	72.38	65.08	81.44	52.40
as scored	65.79	53.86	72.38	65.08	81.44	52.40

140,620 requests were run and 140,178 scored after the official exclusions; all of them were answered (0 unsupported, 0 errors). Nothing was truncated. The listed trained-on test item was answered correctly; counting it as wrong leaves the rounded index at 65.79 (65.7916 as scored vs 65.7912 with the 1 item counted as wrong).

Comparisons on the full Decision Index 0.3 public suite:

• against oenogarum (62.80): +2.99 points;
• with lenavg_bayes_l6k_engine:LenAvgBayesL6KEngine (before the cyclic Latin-square shift for 5 <= K <= 52), these weights and head score 65.71 (raw 73.99%, breadth 65.00), so the balanced K >= 5 order schedule adds +0.08 points (+0.17 in Knowledge, +0.33 in Arts).

Latency (our measurement; the board measures on its own private sample): the kit runner's per-request wall time (model_request_wall_ms) for lenavg_bayes_balanced_k5_engine:LenAvgBayesBalancedK5Engine with these weights on 1 x NVIDIA H100 80GB HBM3, one request at a time, across the 1,000-request stratified sample of the public suite (100 drawn from each tenth of the length L) and across all 140,620 requests of the public suite:

sample	median	mean	80th percentile	95th percentile	slowest
1,000 requests (stratified by L, 1 x H100 80GB)	407 ms	585 ms	702 ms	1,426 ms	19.7 s
full 140,620 requests (1 x H100 80GB per shard)	399 ms	606 ms	725 ms	1,439 ms	96.3 s

These are sample statistics, not a per-request bound; the board measures on its own private sample on 1 x RTX PRO 6000 (where oenogarum, with the same 31B architecture and size under L <= 939, measured median 227 ms, mean 609 ms, p80 566 ms, p95 2,094 ms).

Overlap with evaluation data

For Decision Index benchmarks, only decontaminated train splits were used (CLINC150, GSM8K, Amazon ESCI, BANKING77 and WinoGrande in the head's fit and calibration data, plus decontaminated train splits of ACOS, ANLI, ContractNLI, Amazon ESCI, GSM8K, HoVer, iSarcasmEval, New Yorker Caption Contest, NLI4CT, RAGTruth, SuperGLUE MultiRC, VAST, When2Call and WinoGrande in the post-training mixtures; the training data also include Lichess puzzles, a different dataset from ChessBench's). Some non-Decision-Index sources in the general training pool (such as BBQ, PROST, Breaking NLI, Robust-NLI, IBM ArgQ, Scruples and PubMedQA pqa_labeled) are evaluation-only or pooled splits.

An audit of the training text, the head's data included, against the Decision Index public suite found 1 test item that is an exact or near-exact duplicate of a CLINC150 train-split utterance in the head's fit data: 5:CLINC150+OOS:CLINC150+OOS:test:1591. It is counted as wrong in the board-rule number above, as the board does for train-split duplicates of test items (see the Contamination section of the Decision Index methodology).

Licence

The weights and the head are released under Apache-2.0, and the base model, google/gemma-4-31B-it, is Apache-2.0. The training data come from the sources below, under the licences and terms listed. Some of these sources carry use conditions that apply to the data itself; they are repeated here: US Census PUMS (never use the data to identify any person or household), NVD (not endorsed or certified), California DMHC (modified; not official government data), MITRE CVE / CWE (copyright notices below), ANLI (CC BY-NC 4.0, non-commercial) and XNLI (CC BY-NC 4.0, non-commercial; its translations are part of AmericasNLI).

Data attribution

liquamen was trained on decision questions built from the public data sources below, plus synthetic questions generated by the author (no third-party data). Thanks to everyone who builds and maintains them. Records were reformatted, and many were modified (fields selected, text clipped, identifiers masked), so none of the material should be read as an official copy of its source.

Open data and public records

• GDELT Project: event data. Data from The GDELT Project, https://www.gdeltproject.org/.
• Wikipedia: article text from the English Wikipedia dump of 2023-11-01, via wikimedia/wikipedia 20231101.en, by Wikipedia contributors. Licensed CC BY-SA 3.0/4.0 and GFDL. https://en.wikipedia.org/
• Wikipedia Articles for Deletion: deletion debates by English Wikipedia contributors (CC BY-SA 3.0/4.0, GFDL), via the AfD corpus of Mayfield & Black (2019), https://github.com/emayfield/AFD_Decision_Corpus (the repository's code is GPL-3.0; the corpus files are distributed separately).
• Wikidata: structured data (CC0 1.0), and Requests-for-deletion discussions by the Wikidata contributors (CC BY-SA 4.0). https://www.wikidata.org/
• Swiss Judgment Prediction XL: rcds/swiss_judgment_prediction_xl (Rasiah et al., 2023), CC BY-SA 4.0. Decisions of the Swiss Federal Supreme Court, anonymised by the court.
• NHTSA Office of Defects Investigation investigations and complaints (public domain), https://static.nhtsa.gov/odi/ffdd/
• OSHA Severe Injury Reports and enforcement data, US Department of Labor (public domain), https://www.osha.gov/severe-injury-reports
• NTSB aviation accident database (public domain), https://data.ntsb.gov/avdata
• openFDA enforcement reports and device recalls, CC0 1.0, https://open.fda.gov/
• ClinicalTrials.gov (US National Library of Medicine), via the ClinicalTrials.gov API v2; data processed by ClinicalTrials.gov on 2026-10-01. Modified: selected registration fields only, free text clipped, trial ids masked, study-status sentences removed. Some ClinicalTrials.gov data may be subject to third-party copyright outside the United States.
• ClinVar (NCBI): Landrum et al., Nucleic Acids Research 2018, PMID 29165669.
• Orphanet inheritance data, CC BY 4.0, https://www.orpha.net/
• CIViC (civicdb.org), CC0 1.0: Griffith et al., Nature Genetics 2017, PMID 28138153.
• California Department of Managed Health Care, "DMHC IMR Data, 2001 - Current", CC BY, https://data.ca.gov/dataset/independent-medical-review-imr-determinations-trend. Modified; not official government data.
• NOAA / NWS Storm Prediction Center text products (public domain), via the Iowa Environmental Mesonet, Iowa State University.
• LIGO / Virgo / KAGRA public alerts and the GWOSC event catalogue, CC BY 4.0, by the LIGO Scientific, Virgo and KAGRA Collaborations. This model uses data obtained from the Gravitational Wave Open Science Center (gwosc.org), a service of the LIGO Scientific Collaboration, the Virgo Collaboration and KAGRA.
• USGS PAGER and ComCat (public domain). Population exposure is from ORNL LandScan (CC BY 4.0).
• National Vulnerability Database (NIST, public domain) and CVE records (CVE Terms of Use; Copyright The MITRE Corporation). CWE: Copyright 2006-2026, The MITRE Corporation; CWE is a trademark of The MITRE Corporation. This product uses the NVD API but is not endorsed or certified by the NVD.
• American Community Survey PUMS, US Census Bureau (CC0 1.0), with task definitions from folktables (Ding et al., 2021; MIT). Never to be used to identify any person or household.
• OpenReview reviews and scores, CC BY 4.0 (titles and abstracts: CC0 1.0), https://openreview.net/
• Lichess open database, CC0 1.0, https://database.lichess.org/
• Amazon ESCI shopping queries, amazon-science/esci-data (Reddy et al., 2022), Apache-2.0. Copyright Amazon.com, Inc. or its affiliates.
• SWE-rebench: nebius/SWE-rebench-openhands-trajectories (Trofimova et al., 2025), CC BY 4.0.
• Open-Jev: ZefanCai/Open-Jev, CC0 1.0.
• LLM Bayesian Teaching Data: Qiu, L. (2025), Zenodo, doi:10.5281/zenodo.17677329, CC BY 4.0.

Research datasets

dataset	reference	licence
ACOS (Restaurant-ACOS, Laptop-ACOS)	Cai et al., 2021	MIT
ANLI	Nie et al., 2020	CC BY-NC 4.0
BANKING77	Casanueva et al., 2020	CC BY 4.0
CLINC150	Larson et al., 2019	CC BY 3.0
ContractNLI	Koreeda & Manning, 2021	CC BY 4.0
GoEmotions	Demszky et al., 2020	Apache-2.0
GSM8K	Cobbe et al., 2021	MIT
HelpSteer2	Wang et al., 2024	CC BY 4.0
HoVer	Jiang et al., 2020	CC BY-SA 4.0
iSarcasmEval	Abu Farha et al., 2022	MIT
Measuring Hate Speech	Kennedy et al., 2020	CC BY 4.0
IBM Debater ArgQ-Rank-30kArgs	Gretz et al., 2019	CC BY 3.0
BBQ	Parrish et al., 2022	CC BY 4.0
CaSiNo	Chawla et al., 2021	CC BY 4.0
Circa	Louis et al., 2020	CC BY 4.0 (dataset card; its README links CC BY-SA 4.0)
CodeContests	Li et al., 2022	CC BY 4.0
CosmosQA	Huang et al., 2019	CC BY 4.0
CommonsenseQA	Talmor et al., 2019	MIT
e-CARE	Du et al., 2022	MIT
MultiRC (SuperGLUE)	Khashabi et al., 2018; Wang et al., 2019	CC BY 4.0 / MIT
New Yorker Caption Contest	Hessel et al., 2023	CC BY 4.0
NLI4CT	Jullien et al., 2023	CC BY 4.0
RAGTruth	Niu et al., 2024	MIT
Social IQa	Sap et al., 2019	CC BY 4.0
VAST	Allaway & McKeown, 2020	Apache-2.0
When2Call	Ross et al., 2025	Apache-2.0
WinoGrande	Sakaguchi et al., 2019	CC BY (code Apache-2.0)
MedMCQA	Pal et al., 2022	Apache-2.0
PubMedQA	Jin et al., 2019	MIT
Stanford Politeness Corpus, via ConvoKit	Danescu-Niculescu-Mizil et al., 2013	CC BY 4.0
Scruples	Lourie et al., 2021	Apache-2.0
TabFact	Chen et al., 2020	CC BY 4.0
QASPER	Dasigi et al., 2021	CC BY 4.0
CUAD	Hendrycks et al., 2021	CC BY 4.0
TyDi QA	Clark et al., 2020	Apache-2.0 (the Wikipedia text: CC BY-SA)

Through tasksource (Sileo, 2024), each dataset under its own licence as recorded on its dataset page:

dataset (Hugging Face)	licence
hope_edi/english	CC BY 4.0
AndyChiang/cloth	MIT
bogdanminko/wildguardmix-cleaned	ODC-BY
cambridgeltl/xcopa	CC BY 4.0
copenlu/scientific-exaggeration-detection	GPL-3.0
corypaik/prost	Apache-2.0
declare-lab/cicero	MIT
dynabench/dynasent	CC BY 4.0
fancyzhx/amazon_polarity	Apache-2.0
flaitenberger/wnut_17	CC BY 4.0
GGLab/PARADISE	MIT
google-research-datasets/poem_sentiment	CC BY 4.0
google/civil_comments	CC0 1.0
jana4/turingbench-humanized	Apache-2.0
joey234/nan-nli	CC BY-SA 4.0
juletxara/xstory_cloze	CC BY-SA 4.0 (translations of the StoryCloze test)
lasha-nlp/CONDAQA	Apache-2.0
mteb/MassiveIntentClassification	CC BY 4.0
mteb/NusaX-senti	CC BY-SA 4.0
mteb/tweet_sentiment_multilingual	CC BY 3.0
nala-cub/americas_nli	CC BY-SA 4.0 (a translation of a subset of XNLI, whose licence is CC BY-NC 4.0)
nedjmaou/MLMA_hate_speech	MIT
nvidia/HelpSteer	CC BY 4.0
nvidia/HelpSteer3	CC BY 4.0
oneonlee/cleansed_emocontext	MPL-2.0
pietrolesci/breaking_nli	CC BY-SA 4.0
pietrolesci/gen_debiased_nli	CC BY-SA 4.0, OANC
pietrolesci/robust_nli	CC BY-SA 4.0, MIT, OANC
pietrolesci/robust_nli_is_sd	OANC
pietrolesci/robust_nli_li_ts	OANC
pkavumba/balanced-copa	CC BY 4.0, BSD-2-Clause
pushpdeep/UltraFeedback-paired	MIT
qbao775/PARARULE-Plus	MIT
SemEvalWorkshop/sem_eval_2010_task_8	CC0 1.0
sileod/discovery	Apache-2.0
sileod/mindgames	Apache-2.0
sileod/probability_words_nli	Apache-2.0, CC BY-SA 4.0
stanfordnlp/snli	CC BY-SA 4.0
tasksource/arct	Apache-2.0
tasksource/autotnli	Apache-2.0
tasksource/babi_nli	BSD
tasksource/clcd-english	Apache-2.0
tasksource/conceptrules_v2	MIT, custom (see the dataset page)
tasksource/corr2cause	MIT
tasksource/defeasible-nli	Apache-2.0, MIT
tasksource/dynahate	GPL
tasksource/ethos	AGPL-3.0
tasksource/google_wellformed_query	CC BY-SA 4.0
tasksource/hate_speech18	CC BY-SA 3.0
tasksource/help-nli	Creative Commons (version not stated), CC BY-SA 4.0
tasksource/I2D2	Apache-2.0
tasksource/implicatures	GPL
tasksource/logical-entailment	Apache-2.0
tasksource/lonli	MIT
tasksource/monotonicity-entailment	Apache-2.0
tasksource/multilingual-sentiments	Apache-2.0
tasksource/recast	CC BY-SA 4.0
tasksource/resnli	CC BY 4.0
tasksource/ruletaker	Apache-2.0
tasksource/scone	CC0 1.0
tasksource/SDOH-NLI	CC BY 4.0
tasksource/sherliic	CC BY 4.0
tasksource/silicone	CC BY-SA 4.0, CC BY 4.0
tasksource/SpaceNLI	MIT
tasksource/spartqa-mchoice	MIT
tasksource/spartqa-yn	Apache-2.0
tasksource/starcon	CC0 1.0
tasksource/stepgame	MIT
tasksource/subjectivity	MIT
tasksource/syntactic-augmentation-nli	MIT
tasksource/temporal-nli	Apache-2.0
tasksource/tomi-nli	GPL-3.0
tasksource/universal-joy	GPL
tweets-hate-speech-detection/tweets_hate_speech_detection	GPL-3.0
unigram/FOL-nli	Apache-2.0
webis/args_me	CC BY 4.0
zeroshot/twitter-financial-news-sentiment	MIT
Zhengping/UNLI	MIT

Community synthetic datasets

• tasksource/procedural-typed-decisions (tasksource, Sileo, 2024), Apache-2.0
• frontier-infra/jebadiah-synth-v2, Apache-2.0
• kaivoss/system-one-270m-data, Apache-2.0
• vagmi/jevlite_dataset, CC BY-SA 4.0
• mghafiri/decision-model-scenarios, MIT