Arbiter-8B

Arbiter-8B is a Jev-inspired local decision model built on Alibaba's Qwen3-8B, for wealth and asset management. It reads verbatim news articles and SEC Form 10-K passages and answers typed questions about a named company, such as event type, implied outlook, how settled an event is, risk category, or filing flags. It returns a calibrated probability for every option in a single forward pass, with no generated text.

Model details

• Model type: decision model (single-pass scoring of a fixed set of options), built on the Qwen3 causal language model (Qwen3ForCausalLM, 8B parameters)
• Base model: Qwen/Qwen3-8B, fine-tuned with LoRA (rank 32) and a decision head
• Language: English
• Domain: financial news and SEC Form 10-K filings, for wealth and asset management
• License: Apache-2.0, the same as Qwen3-8B
• Precision: fp16 (the fine-tuning is merged into the weights)
• Method: follows "Releasing the Inner Jev" (Geuter et al., 2026)

How it works

You give Arbiter a text, a question and a set of options, each with a short definition. It scores every option in one forward pass and returns the probabilities, calibrated per question type with the temperatures in calibration.json. It never generates text.

Versions in this repository

• Full model (default): the weights at the top level load with transformers, with no separate download of the base model.
• 4-bit adapter (adapter/): the unmerged LoRA for Qwen/Qwen3-8B, as trained with 4-bit QLoRA. It needs about 6.5 GB of GPU memory.

The full model was built by merging the LoRA into the dequantized 4-bit base weights it was trained against, saved in fp16. It reproduces the trained model: it gives the same answer on 730 of 731 evaluation decisions, with probabilities within 0.004.

Quick start

pip install torch transformers accelerate safetensors huggingface_hub     # + peft bitsandbytes for the 4-bit adapter
hf download ternary/Arbiter-8B --local-dir Arbiter-8B
cd Arbiter-8B

from arbiter import Arbiter
from questions import OUTLOOK

arbiter = Arbiter.from_pretrained(".")                    # full model (fp16)
# arbiter = Arbiter.from_pretrained(".", adapter=True)    # LoRA on Qwen/Qwen3-8B in 4-bit, as trained

passage = ("Revenue increased 15% to $8,449 million, reflecting higher shipments and higher "
           "revenue per ton. Cost of sales increased 14%.")
answer = arbiter.decide(state=passage, question=OUTLOOK, focus="Constellium")

print(answer["choice"])          # e.g. "improved"
print(answer["probabilities"])   # {"improved": ..., "worsened": ..., "unchanged": ..., "unclear": ...}

decide() returns a typed answer: {"type": "choice", "choice": ..., "probabilities": {...}} for choice questions, {"type": "noul", "noul": P(true), ...} for yes/no questions, and {"type": "score", "score": level, "probabilities": {...}} for ordered scales. If a state is too long, it is shortened from the end (never the options), and the answer reports state_truncated: true. arbiter.fallback_counts counts every fallback taken.

Requirements

• Python 3.10 or later, with torch, transformers, accelerate, safetensors and huggingface_hub.
• Full model (default): about 17 GB of GPU memory in fp16 or bf16, for example one 24 GB GPU. device_map="auto" also spreads the model over several smaller GPUs. If part of it is offloaded to CPU memory, it still runs, only more slowly.
• 4-bit adapter (adapter=True): about 6.5 GB of GPU memory, so an 8 GB GPU is enough. Also needs peft and bitsandbytes (CUDA).
• Compute: one forward pass per decision, no text generation.

Load the model through arbiter.py as shown above. The backbone also loads with plain AutoModelForCausalLM.from_pretrained, but typed decisions need the decision head, which arbiter.py applies. The Hub's automatic code snippet labels this model text-classification; it doesn't apply.

Questions it was trained on

Import the exact definitions from questions.py; Arbiter is most reliable with them.

Constant	Type	Options
EVENT_TYPE	choice	corporate_action, financial_results, company_guidance, product_operations, analyst_action, analyst_opinion, market_move, legal_safety, industry_macro, insufficient_evidence
OUTLOOK	choice	improved, worsened, unchanged, unclear (conflicting signals → the stronger, more material signal; unclear = no outlook signal at all)
CERTAINTY	score	0 rumoured … 3 realised
MAIN_SUBJECT	yes/no	is the company the main subject of the event?
ROLE	choice	acquirer, target, regulator, mentioned_only
FILING_RISK	choice	ten risk categories for an Item 1A passage
FILING_LEGAL	score	0 possible … 3 resolved
GOING_CONCERN, CUSTOMER_CONCENTRATION, CYBER_INCIDENT	yes/no	10-K flags

You can pass your own question in the same shape ({"type", "instructions", "criteria"}, at most 16 options). The base model's general decision ability carries over, but quality on questions outside this list has not been measured.

Evaluation

All results are on companies held out from training. Labels and limits are described under each table.

2,000 news decisions (stratified estimate). Labels come from blind adjudication by Claude Opus 5.5, which saw no system's answer, of all 323 disagreements between Arbiter v0.2 and Jev 1.14 plus 201 randomly sampled agreements. The judge applied the project's labelling conventions; on the 207 human-labelled decisions it agreed with the human label 84.5% of the time (90.3% on news).

	Accuracy (95% CI)	Test companies	Dev companies	Calibration (100 = perfect)
Arbiter v0.4 (this full model)	85.8% ± 3.3	89.4%	82.8%	98.2
Arbiter v0.4 (4-bit adapter)	85.7% ± 3.3	89.3%	82.8%	98.2
Qwen3.8-27B (teacher)	85.6% ± 3.4	89.2%	82.6%	95.1
Arbiter v0.3	83.2% ± 3.5	85.3%	81.4%	98.0
Jev 1.14 (TypeSafe API)	79.3% ± 3.5	79.9%	78.8%	93.8

Calibration is 100 × (1 − expected calibration error), 10 bins.

Per question on the same decisions: event type 87.0% (Jev 72.8%), outlook 84.8% (85.6%), certainty 74.4% (79.3%), main subject 92.1% (87.7%), role 95.2% (85.5%; few decisions).

Part of the lead over Jev comes from the labelling conventions, such as "earnings previews are analyst_opinion", which Jev never sees. When event-type disagreements were re-judged on the option definitions alone, Arbiter v0.2's lead over Jev fell from 4.0 to 1.4 points (not significant). This ablation was not repeated for v0.4.

Human-labelled decisions (single human adjudicator):

	Dev, 151 (news + 10-K)	Test, 56 (10-K)
Arbiter v0.4 (full model and adapter alike)	85.4%	83.9%
Jev 1.14	84.8%	82.1%
Qwen3-8B zero-shot	74.2%	76.8%

These sets are small: a difference of one or two points is within noise. Calibration temperatures were fitted on the dev set.

Training

• Data: about 120k decisions. 100,000 news decisions, 11,752 SEC 10-K decisions and 8,000 news-outlook decisions were labelled by a local Qwen3.8-27B (one-pass letter readout, soft probability targets). The rest are 106 human-reviewed 10-K decisions. No human-labelled news rows; no output of Jev or of any Claude model was used for training. The training data is not released: the news text is licensed and not redistributable.
• Recipe: 4-bit QLoRA (NF4, double quantization), rank 32, alpha 64, on all attention and MLP projections, trained jointly with the head. 2,000 steps × 16 decisions, sampled per question so that weaker questions (certainty, outlook) get more draws. Options are shuffled on every presentation. Cross-entropy against the teacher's soft targets. Details in metadata.json.

Limitations

• English, US companies, financial news and 10-K text only. Not investment advice; use it as decision support with human review.
• Weakest on certainty (how settled an event is): there it trails both its teacher and Jev, and the teacher is the ceiling.
• Labels follow this project's definitions and conventions. A desk with other conventions should re-evaluate on its own labelled examples before relying on the probabilities.
• The model sees only the supplied text; it has no knowledge of later events.

Files

File	Purpose
model-*.safetensors, config.json, tokenizer files, chat_template.jinja	full model: Qwen3-8B with the LoRA merged in, fp16
adapter/	the unmerged LoRA for Qwen/Qwen3-8B (4-bit use), with its own calibration.json
decision_head.safetensors, head_config.json	decision head weights and the letter token ids it expects
calibration.json	per-type temperatures for the full model, fitted on the dev split
arbiter.py, model.py	loading and inference code
questions.py	the exact question definitions used in training
metadata.json	training settings and counts