Horopter 1B (horopter-1b-20261020)

An encoder-only decision model built on jhu-clsp/ettin-encoder-1b. Give it a state (a document, a record or a JSON object) and typed questions about it (yes/no, choose one, score). It answers them in one pass, and returns three things with each answer:

• evidence spans: whole-word spans in the state that support or refute each option;
• relation: whether the state bears on the question at all, and with what stance;
• abstention: a calibrated probability that the answer is wrong, with a reason.

The state is encoded once, without seeing any question, and cached. Questions never change each other's answers, and many questions about one state cost about one encoding.

Code, wire format and training recipe: https://github.com/jmccardle/horopter. Smaller sibling: horopter-0.4B.

Use

import horopter
m = horopter.load("jmccardle/horopter-1B")
out = m.answer("Refunds within 30 days. The lamp was bought 12 days ago.",
               {"q": {"type": "noul", "instructions": "Is a refund allowed?"}}, evidence=True)

Server: python -m horopter.server --model jmccardle/horopter-1B --port 8420. Model id horopter is the router, which picks a family adapter per question. horopter-<adapter> (e.g. horopter-base, horopter-check2) forces one adapter. --prefix horopter-1b renames them.

Files

• model.safetensors: the trunk with its answer, relation and evidence heads.
• adapters/:
  • router;
  • family LoRAs: typed, nli, relation, classification, knowledge, clause;
  • evidence (deciding-rule evidence);
  • check2 (single-condition checks).
• abstain_head.json: the calibrated abstention head.
• quantized/model-w8.safetensors and quantized/model-w4g32.safetensors: Q8 and Q4 trunks (see Quantized weights).
• config.json and the tokenizer (from Ettin).

Quantized weights

Every attention and MLP weight of the trunk is stored as integers with fp32 scales (symmetric absmax, round to nearest, no calibration): Q8 is int8 with one scale per output row; Q4 is int4 with one scale per 32 inputs. Embeddings, norms, heads, adapters and the abstention head are unchanged.

m = horopter.load("jmccardle/horopter-1B", quant="w8")   # downloads only the Q8 weights file

Server: add --quant w8 (or --quant w4g32). The weights are dequantized at load time, so this saves download and disk, not GPU memory or time. Answers are exactly those of the evaluated quantized grid.

Scores of the full routed stack (router + adapters + abstention head):

weights	file	JevBench hard	JF100	LLM-AggreFact	E0 relation F1	ProofWriter OWA true/false	ShARC irrelevant recall
full precision (model.safetensors, bf16)	2.07 GB	40.5	49	75.7	78.6	62.7	36.2
Q8 (quantized/model-w8.safetensors)	1.13 GB	40.5	48	75.7	78.7	62.6	37.0
Q4 (quantized/model-w4g32.safetensors)	0.78 GB	43.2	49	75.7	78.2	65.6	9.4

Q8 changes 0–4% of answers, all near-ties; no paired test is significant. Q4 changes 2–13% of answers, with gains and losses balancing on the headline sets, but it gives up most "this rule does not apply" answers (ShARC irrelevant recall 36 → 9) and predicts "unknown" less often. Use Q4 where size matters more than abstaining on inapplicable rules.

Results

Horopter rows use the router + adapters, in one pass with no generation. The reference decoders were scored on the same items. mbnli-3 beta is our internal predecessor: the same 1B base, without this release's rule-application data.

	Horopter 1B	Horopter 0.4B	mbnli-3 beta 1B	clef-flash (9B)	decider-2b	openjev v5 (4B)
LLM-AggreFact, balanced accuracy (macro, 11 datasets)	75.7	74.8	75.7	77.2	72.6	74.2
4-way relation (support / refute / undetermined / unrelated), macro-F1	78.6	76.6	78.6	75.3	72.3	71.0
irrelevant-state detection, AUROC	0.956	0.948	0.947	0.961	0.935	0.941
JevBench original (72 decisions)	80.6	66.7	81.9	97.2	88.9	98.6
JevBench hard (111 multi-step policy decisions)	40.5	36.0	44.1	64.0	57.7	62.2
JF100 (100 paired counterfactual and formal decisions)	49.0	52.0	47.0	71.0	63.0	60.0

• Grounding. Horopter is level with 4–9B decoders at telling whether a document supports a claim, and whether a text bears on a question at all.
• Multi-step decisions. It trails them by 20 or more points. The differences between Horopter and the beta on JevBench and JF100 are within item-level noise (paired tests, p > 0.4).

Real rule text (dev sets, accuracy). Qwen3-30B-A3B answers each question directly, with thinking off.

set	Horopter 1B	Horopter 1B + check2	Horopter 0.4B	Qwen3-30B-A3B
ProofWriter OWA depth 0–5, true/false items	62.7	67.0	58.4	75.5
ShARC, 4-way (yes / no / irrelevant / more info)	54.3	54.4	48.1	64.8
ShARC, yes/no items	74.9	74.5	71.2	72.2
ConditionalQA, yes/no (95)	73.7	75.8	71.6	78.9

Generated policies. These are 1,868 held-out rule sets and cases at depth 1–5. One domain (robot SOPs) was held out of training entirely.

Horopter 1B	one-shot, JSON policy	one-shot, prose policy	interpreter (compiled per-rule checks via check2)
all	78.0	97.7	99.6
held-out domain	61.2	91.5	99.4

The evidence span lands on the deciding rule 98% of the time.

Training data

• A committee-distilled typed-decision mixture (NLI, relation, classification, knowledge and clause questions, with evidence spans).
• This release adds:
  • 12,000 generated rule sets with cases (169k decisions), labelled by executing the rules;
  • 1,620 single-condition check sets;
  • 392 counterfactual minimal pairs;
  • 3,353 deciding-rule evidence targets.
• Labels come from human annotations, a committee of open decoder models (Gemma-family models among them), and TypeSafe's Jev API.
• Some sources are licensed non-commercial or research-only. The per-source list, with its permissions column, is in the recipe.
• LLM-AggreFact and the other benchmark sets were not trained on.

Limits

• Multi-step policy decisions trail 2B+ decoders by about 20 points in one pass. The interpreter path closes the gap on generated policies, but only partly on real policy text.
• On real rule text it rarely answers "more information needed" or "this rule does not apply". All of this release's rule-application data has a determined answer.
  • ShARC recall is 36% for "irrelevant" and under 1% for "more info", against 75% and 16% for the beta.
  • Expect a yes/no where a careful reader would say it can't tell.
  • The relation output and abstain.p are unaffected on our relation benchmark.
• English only. States up to 2,048 tokens.
• Not for safety-critical use without a human in the loop.

Licence

• Weights: CC BY-NC-SA 4.0. Research and non-commercial use only; share adaptations under the same terms.
• Code: MIT.
• Base model: Ettin is MIT, © its authors.

A permissively licensed variant, trained only on commercially usable sources, is planned.

The training data includes outputs of TypeSafe's Jev API. TypeSafe's terms do not permit using them to imitate their model or to build a competing service.

Versions

• horopter-1b-20261020: first release.