Arbiter-8B-GGUF

GGUF quantizations of ternary/Arbiter-8B (Arbiter v0.4) for llama.cpp, Ollama, LM Studio and other GGUF runtimes.

Arbiter-8B is a Jev-inspired decision model for wealth and asset management, fine-tuned from Qwen/Qwen3-8B. It keeps the Qwen3 architecture, tokenizer and chat template, so any runtime that supports Qwen3 GGUF files runs it. It reads news articles and SEC Form 10-K passages and answers typed questions about a named company, such as event type, implied outlook, how settled an event is, risk category, or filing flags. It answers with the letter of the chosen option.

Files

File	Quant	Size	Notes
Arbiter-8B-Q4_K_M.gguf	Q4_K_M	5.0 GB	recommended for most users
Arbiter-8B-Q5_K_M.gguf	Q5_K_M	5.9 GB
Arbiter-8B-Q6_K.gguf	Q6_K	6.7 GB
Arbiter-8B-Q8_0.gguf	Q8_0	8.7 GB	near-lossless
Arbiter-8B-F16.gguf	F16	16.4 GB	full precision

Usage

llama.cpp

llama-server -hf ternary/Arbiter-8B-GGUF:Q4_K_M -c 4096
# or with a downloaded file
llama-server -m Arbiter-8B-Q4_K_M.gguf -c 4096 -ngl 99

Ollama

ollama run hf.co/ternary/Arbiter-8B-GGUF:Q4_K_M

Prompt format

Arbiter was trained on one fixed prompt. Use it exactly, with thinking disabled, and generate a single token.

• System: Apply the supplied criterion to the supplied evidence. Choose exactly one listed option. Respond with only its uppercase letter, with no explanation or reasoning.
• User: a JSON object with evidence (the verbatim text), criterion (the question, followed by Focus entity: <company>.) and options, each with a letter and a description of the form <option id>: <definition>.

Example against llama-server's OpenAI-compatible endpoint:

import json, math, requests

SYSTEM = ("Apply the supplied criterion to the supplied evidence. Choose exactly one listed option. "
          "Respond with only its uppercase letter, with no explanation or reasoning.")
options = {
    "improved": "The article's signals point to a better or more favorable outlook.",
    "worsened": "The article's signals point to a worse or less favorable outlook.",
    "unchanged": "The article's signals point to an outlook that has not changed, such as reaffirmed guidance or a maintained rating or target.",
    "unclear": "The article gives no signal about the outlook.",
}
user = json.dumps({
    "evidence": "Revenue increased 15% to $8,449 million, reflecting higher shipments and higher revenue per ton.",
    "criterion": "What does the article imply about the focus entity's business outlook? Focus entity: Constellium.",
    "options": [{"letter": "ABCD"[i], "description": f"{k}: {v}"} for i, k in enumerate(options)],
})
reply = requests.post("http://127.0.0.1:8080/v1/chat/completions", json={
    "messages": [{"role": "system", "content": SYSTEM}, {"role": "user", "content": user}],
    "max_tokens": 1, "temperature": 0, "logprobs": True, "top_logprobs": 20,
    "chat_template_kwargs": {"enable_thinking": False},
}).json()["choices"][0]

letter = reply["message"]["content"].strip()                      # e.g. "A" -> improved
probs = {t["token"]: math.exp(t["logprob"]) for t in reply["logprobs"]["content"][0]["top_logprobs"]
         if t["token"] in "ABCD"}                                   # confidence per option letter

The question definitions Arbiter was trained on (event type, outlook, certainty, main subject, role, 10-K risk, legal status and filing flags) are in questions.py in the main repository. It works best with those exact definitions and with at most 16 options.

Accuracy

On 151 human-labelled decisions about companies held out from training (news and 10-K):

	Accuracy
Arbiter, Q4_K_M GGUF (letter answer)	82.1%
Arbiter, Q8_0 GGUF (letter answer)	82.8%
Arbiter with its decision head (main repository)	85.4%
Qwen3-8B, not fine-tuned	74.2%

The GGUF files contain the fine-tuned language model. The main repository also ships a small decision head and per-question calibration, which add about 3 points of accuracy and well-calibrated probabilities. Use it with transformers when you need the best decisions or calibrated confidence; use these GGUF files for lightweight local inference.

License

Apache-2.0, like the base model Qwen3-8B.