decisionmodels/reax-1.2

REAX 1.2

REAX 1.2 is a 27B decision model by REAX (decisionmodels.io). It reads a state and one or more questions (choice, yes/no, score) in one forward pass and returns probabilities for every option without generating text. It is a fine-tune of REAX 1 (decisionmodels/reax-1, revision ca2a0cc5), itself a fine-tune of pplx-decider-v1.2 (Perplexity AI, Apache-2.0), derived from Qwen/Qwen3.8-27B (Apache-2.0). This model is not affiliated with or endorsed by Perplexity AI or the Qwen team.

What is new: REAX 1.2 abstains far more readily on hard knowledge questions. If a choice question offers an "I don't know" / abstain option, REAX 1.2 picks it much more often than REAX 1 when the answer is not well known, instead of guessing. The answers it does give are more often right (about 66 % of answered questions vs 55 % for REAX 1 on our AA-style knowledge questions), but it also abstains on questions it could have answered. Without such an option the protocol is that of REAX 1; on benchmarks without an abstain option we measured no loss (see Evaluation).

REAX 1.2 descends from REAX 1, not from REAX 1.1 (the round-2 model trained for the SN92 competition prompt).

Intended use

Routing, triage, moderation, grading, retrieval/tool reranking and other structured judgements where the answer is one of a closed set, in particular knowledge questions where a wrong answer costs more than an abstention. Intended also as a base for further fine-tuning. Not for sole decision-making in high-stakes settings (medical, legal, credit, employment) without human review.

Protocol

Identical to REAX 1 and pplx-decider-v1.2 (joint protocol v3, template joint-question-inline-readout-v1):

{"state": str|dict|list, "questions": {"<id>": {"type": "choice"|"noul"|"score", "instructions": str|None, "criteria": ...}}, "images": [...]}

Full attention is noncausal: every question attends to the state and to all other questions. One readout per question at the <|fim_suffix|> marker via the separate readout head ([255, 5120]); probabilities use softmax over option codes A, B, ... with temperature 1.9754 (decision_config.json, unchanged from REAX 1). To allow abstention, add an option such as "IDK": "I don't know" to a choice question; its wording, key and position are free.

Loading uses the pplx_decider package bundled in source/src/ (code by Perplexity AI, Apache-2.0, unmodified):

import sys; from pathlib import Path
from huggingface_hub import snapshot_download
ckpt = Path(snapshot_download("decisionmodels/reax-1.2"))
sys.path.insert(0, str(ckpt / "source" / "src"))
from pplx_decider import Decider
decider = Decider.from_pretrained(ckpt, device="cuda")   # ~52 GiB BF16
print(decider.decide({"state": "...", "questions": {"q": {"type": "choice", "instructions": "...",
      "criteria": {"A": "...", "B": "...", "C": "...", "D": "...", "IDK": "I don't know"}}}}))

What changed vs REAX 1

One LoRA fine-tune (r16, KL-to-base 0.5, 1 epoch, native format only), merged at weight 1.0 into REAX 1:

• 8,930 closed-book knowledge questions built from Wikidata (CC0) in six domains (health, science, finance, humanities and social sciences, law, software), 4 answer options plus an abstain option, incl. 1,301 deliberately unanswerable variants. Gold labels are model-relative: a question is relabelled to the abstain option where REAX 1 itself answered wrongly or put less than 0.5 probability on the correct answer (44 % of the training items end up with gold = abstain; 5 % of the most popular entities, 53 % of the rarest). The abstain option's wording (11 phrasings in English, German, French, Spanish), key and position are randomised so the model learns the decision, not a string.
• 4,000 replayed generated decision items from our round-1 training pool (the hard-mined generated sets used for REAX 1 and generated hard items from our data-difficulty work; 948 of them are rows of decisionmodels/sn92-specialists, CC-BY-4.0), to keep its other skills. The generated items imitate the task types of public decision benchmarks but contain no benchmark items or splits; they were written and cross-checked by OpenAI, DeepSeek, GLM, Qwen and MiMo models, and the providers' output terms may apply to that text.
• No exact or normalized question-text overlap between the training data and the HF Decision Index 0.3 suite rows, the AA-style replica items or our When2Call evaluation set. See MODIFICATIONS.md for the full change list.

Evaluation (our measurements, internal, not official board results)

All three models on the same harness (the pplx_decider helper, native protocol, one question per request where the index does so). Differences are paired on the same items, ± bootstrap standard error.

Measurement	pplx-decider-v1.2	REAX 1	REAX 1.2
AA-Decision-Index-style replica (5 components, public items), our index	54.51	56.35	60.11
- paired difference vs REAX 1			+3.76 (SE 0.54)
AA-Omniscience public questions (407): accuracy / non-hallucination / abstain rate	51.6 / 7.6 / 3.7 %	47.4 / 26.2 / 13.8 %	26.3 / 81.3 / 60.0 %
HLE text multiple choice (513) accuracy	15.4	16.4	15.4
HF Decision Index 0.3 public suite, subset stratified over the suite's benchmarks (9,710 requests)	67.03	66.36	66.91
- paired difference vs REAX 1			+0.55 (SE 0.33)
Own composite validator (frozen eval v4, 6,842 items, fitted temperature)	79.76	81.39	82.00
- paired difference vs REAX 1			+0.62 (SE 0.31)

Notes:

• On the HF public subset REAX 1.2 and pplx-decider-v1.2 are statistically level (66.91 vs 67.03).
• The abstention behaviour holds when the abstain option is worded, keyed and placed differently from training (two out-of-format variants: non-hallucination 84.6 and 83.0, abstain rate 64 % and 62 %).
• Non-hallucination = abstentions / (wrong answers + abstentions). On hard knowledge questions REAX 1.2 trades accuracy for far fewer wrong answers: it abstains on 60 % of these questions, and answers correctly on 26 % (REAX 1: 47 %). If a task needs a forced answer, do not offer an abstain option.
• Our replica of the AA Decision Index uses public items: AA-Omniscience public questions with distractors written by us, the real HLE text MC set, When2Call MCQ, and Aegis 2.0 as a stand-in for the gated WildGuardTest. The official indices use private items; our numbers are not a prediction of an official score or rank.
• The HF Decision Index 0.4 scores only private tests; the public-suite subset above is a sanity check, not an index score.
• Not measured for REAX 1.2: the SN92 competition prompt path (REAX 1.1 remains the model tuned for it).

Limitations

• Same protocol as the base; our fine-tuning used requests of up to ~8k tokens.
• Abstains much more often than REAX 1 when an abstain option is offered, including on some questions it would have answered correctly; the abstention training data covers six knowledge domains in four languages.
• Safety classification: on our Aegis 2.0 units REAX 1.2 is 1.9 points below REAX 1 (still at pplx-decider-v1.2 level).
• Small regressions vs REAX 1: our Aegis 2.0 unit is 1.9 lower, the HF-suite arts area 0.7 lower (single benchmarks such as RAGTruth -4.5 and NLI4CT -3.0 points), and one private-style unit of our validator ("education grading") regresses. On the HF public subset REAX 1.2 is 0.12 below pplx-decider-v1.2 (within noise).
• REAX 1.2 was chosen among four trained/merged variants and a zero-training merge using the AA-style replica and our own validator, so the paired gains (+3.76, +0.62) are mildly optimistic; the validator and the replica partly use the same kind of items. The wrong options of the replica's Omniscience questions were written by one LLM; real AA items may behave differently, and on harder items the abstain rate will be higher still. Offering an abstain option on easy routing tasks will also raise abstentions there.
• Single training run (no seed average); evaluation sets are public-item replicas plus our own validator.

Licence and attribution

Apache-2.0. Based on REAX 1, pplx-decider-v1.2 by Perplexity AI and Qwen3.8-27B by the Qwen team; abstention training questions built from Wikidata (CC0). See NOTICE and MODIFICATIONS.md. Not affiliated with, endorsed by, or sponsored by Perplexity AI or the Qwen team.