Horos

The boundary stone for personal data. A 268 MB on-device router for Topos: free text in → the data scopes it touches, an explicit none, or a hand-off to an LLM when it isn't sure.

v2 (2026-08-16) — macro-F1 0.613, up from v1's 0.512. Trained against a gap v1's own failures exposed: it handled band phrasings ("how has my sleep been") and fell silent on artifact ones ("what's my resting heart rate trend"). Still not promoted — see Limitations.

ὅρος — the stones Athenians planted at property lines: "whose land begins here." Same job, for your data. Horos runs before the permission gate, so the raw question never leaves the device.

Router, not authorizer. Every scope it emits is still permission-checked downstream. A node without Horos escalates more often; it does not become less safe.

Use

The scope role in a Topos model pack. Load through topos.query.scope_head.load_head — it validates the label set against the live scope registry and the training manifest against the licence policy before touching a weight. Not intended for any task besides Topos scope routing, and not intended to run without the escalation path.

Pin a revision. The thresholds in head.json are selected per-artifact and the label set can change; main is not a stable contract. Topos pins the SHA it was gated against:

snapshot_download("Dialogues/horos", revision="1873a1b667dc2d1bd630835f1f4f47025f3864d9")

The ladder. Four branches, because "no scopes" is two different states — the model deciding there is no personal data here, and the model having no idea:

labels ≥ tau_high, none low, nothing in band → ACT on the scope set (1..N scopes)
high none, nothing else                      → abstain: no personal data
any label in [tau_low, tau_high)             → escalate to the pack's LLM (ambiguity)
nothing anywhere, none included              → escalate to the pack's LLM (ignorance)

Report card

Every release publishes all five blocks below, in full, including the numbers that got worse — this is a standing contract, not a summary of a good run. Every row is versioned and every cross-arm claim carries an interval. Each block earns its slot by having caught something the others hid:

• 1 — the gate. disjoint sat at 0.206–0.265 across three training rounds while macro-F1 climbed 20%. An average would have called that pure progress.
• 2 — the split. Artifact-concrete swallowing was 26% while the overall dead rate looked healthy at 0.171. The total hid a defect the split names.
• 3 — unseen data. wrong rose 11% → 17% in the same round routing improved 14 points. Nothing else here can see a model converting silence into error.
• 4 — composed. Horos never runs alone; a head-only number is not a product number. It is also where an unqualified 0.500-vs-0.495 claim went out on this card and turned out to be inside the noise (see block 4).
• 5 — per-scope. Every other block is an average or a count, and both let a gain on one scope pay for a regression on another without saying so. Only this view names the scope that got worse.

swallowed and escalated stay separate throughout: they have opposite fixes, and collapsing them into "not answered" hides which one you have.

Blocks 1–3 and 5 regenerate from one command in topos-eval (scripts/scope_head_report_card.py --head A --label v1 --head B --label v2). That repo is private, so the numbers here are auditable by us and take-it-or-leave-it for you — the benchmark itself is what we'd have to open to change that.

1. role_classify_8 — the promotion gate

1,071 cases, template-disjoint from training, multi-label gold, 24% adversarial negatives. Built to be hard: don't anchor against CLINC-style intent scores. The right column is the same benchmark's LLM baseline.

metric	v2	v1	mistral:7b (4.4 GB)	gate
macro-F1	0.613	0.512	0.495	≥ incumbent ✅
exact set match	0.558	0.496	0.243	—
negatives abstained	0.941	0.984	0.126	≥0.85 ✅
single / multi-gold recall	0.502 / 0.596	0.388 / 0.425	—	gap ≤0.05 ✅
dead rate	0.149	0.171	—	<0.20 ✅
disjoint rate	0.206	0.217	—	≤0.03 ❌
per-scope recall ≥ 0.60	6 / 14	2 / 14	9 / 14	14/14 ❌

Which six, and which scope went backwards: block 5.

2. Real language — a gap finder, not a score

53 hand-annotated natural phrasings neither version trained on. The labels are one annotator's judgement, so the totals measure agreement-with-the-annotator; the split is the signal worth acting on.

metric	v2	v1	gate
correct	0.774	0.585	—
swallowed	0.132	0.189	—
escalated	0.094	0.189	—
swallowed — artifact-concrete	0.176	0.265	≤0.10 ❌
swallowed — abstract / band	0.053	0.053	—

3. Unseen phrasings — did the authoring generalise?

2,745 rows from the schema-grounded generator's heldout half, which the training corpus never draws from. Provably unseen by both versions and absent from the benchmark. This block separates learning a register from memorising the rows we wrote.

metric	v2	v1	ratchet
routed	0.565	0.421	≥ incumbent ✅
swallowed	0.172	0.214	≤ incumbent ✅
wrong scope	0.169	0.113	≤ incumbent ❌
escalated	0.094	0.252	—

Read the wrong row, not just the first one. v2 routes 14 points more of this traffic and is confidently wrong 6 points more often. It got better at recognising that a question concerns personal data, and no better at discriminating which data — the same finding as the flat disjoint rate. Those are separate abilities; one improved.

4. Composed with its LLM escalation

Horos is not deployed alone, so this is the number that describes the product. Each case is routed by the ladder above: Horos answers when it acts or abstains, the LLM sees only what it escalates.

arm	macro-F1	exact	neg-abstain	wrong-scope	LLM sees
mistral:7b (4.4 GB) alone	0.495	0.243	0.126	0.237	100%
Horos v1 + mistral:7b	0.550	0.524	0.972	0.143	16.4%
Horos v2 + mistral:7b	0.620	0.578	0.909	0.163	15.3%
llama3.2 (2 GB) alone	0.376	0.261	0.510	0.204	100%
Horos v1 + llama3.2	0.500	0.508	0.976	0.146	16.4%
Horos v2 + llama3.2	0.583	0.565	0.917	0.146	15.3%

A 2 GB machine running the hybrid beats a 4.4 GB machine running LLM-only — 0.583 vs 0.495, +0.088, 95% CI [+0.052, +0.123] (paired bootstrap, 2,000 resamples of the same 1,071 cases). Horos is identical on every hardware tier, so only the escalated ~1/6th of traffic degrades with weaker hardware.

That claim did not survive this test at v1, where the same comparison was +0.005, CI [−0.036, +0.042] — indistinguishable from zero. It was stated on this card as fact for one release. Cross-arm claims here now carry an interval or they don't get made.

Note the direction of the v1 → v2 trade in this table: composed accuracy up ~7 points, composed negatives-abstained down ~6. The escalation path was covering for the head's false-positives, and v2 hands it less to cover.

5. Per-scope — where it moved, and where it didn't

Blocks 1 and 3 report counts ("6 / 14 above the floor", "routed 0.565"). A count cannot be audited: it says how many scopes cleared the bar and never which, so a large gain on one scope silently pays for a regression on another. Both slices, sorted by movement.

Unseen phrasings — did the gains reach language nobody wrote down?

scope	n	v1	v2	Δ
places	217	30%	68%	+37
schedule	65	31%	65%	+34
work_context	420	22%	51%	+29
relationship_context	174	16%	42%	+26
public_bio	229	67%	85%	+17
messages	253	49%	66%	+17
attention	145	35%	46%	+10
complexity	178	39%	47%	+8
activity	308	61%	68%	+7
resources	151	28%	34%	+6
availability	116	21%	25%	+4
contacts	119	76%	73%	−3
ai_conversations	111	42%	39%	−4
health	259	61%	56%	−5

Gate benchmark — this is what "6 / 14 above the floor" expands to.

scope	n	v1	v2	Δ	≥0.60
contacts	58	69%	83%	+14	✅
public_bio	54	67%	81%	+15	✅
attention	73	25%	71%	+47	✅
schedule	59	44%	69%	+25	✅
activity	78	58%	64%	+6	✅
messages	61	51%	64%	+13	✅
health	116	47%	54%	+8	❌
places	61	26%	49%	+23	❌
availability	62	35%	45%	+10	❌
resources	70	27%	37%	+10	❌
complexity	82	35%	37%	+1	❌
ai_conversations	54	43%	35%	−7	❌
relationship_context	77	17%	34%	+17	❌
work_context	73	25%	30%	+5	❌

Three things only this view shows:

• ai_conversations is down on both slices (−7, −4) — the one unambiguous regression, not a slice artifact. v2's training targeted artifact-concrete phrasings, and questions about your own past AI conversations are the scope least like an artifact.
• health and contacts flip sign between slices. health gains 8 on the benchmark and loses 5 on unseen phrasings; contacts gains 14 and loses 3. The two instruments measure genuinely different things, and a card publishing only one of them would report either as a clean win.
• The four biggest unseen gains are exactly the four scopes v2's authoring targeted (places, schedule, work_context, relationship_context, +26 to +37). That is the evidence the authoring generalised rather than being memorised — the gains land on phrasings of those scopes that nobody wrote down.

Floors are still unmet: on the gate slice, 8 of 14 scopes sit under the 0.60 recall bar. work_context (30%), complexity (37%) and resources (37%) are the weakest, and relationship_context at 34% remains the hardest scope in the taxonomy.

Limitations

• Confident-none swallowing, now concentrated rather than general. Overall dead rate is 0.149, but on artifact-concrete phrasings ("what's my bank balance", "what's in my review queue") it is 17.6% against 5.3% on abstract ones. v2 closed most of v1's 26% gap here and gave part of it back for the safety property below — one threshold currently trades them against each other, which is the next fix. Treat a none on a plausibly-personal question with suspicion.
• Disjoint rate 0.206 — the blocking defect, and unsolved. One in five acted-on turns names a scope set sharing nothing with the truth. It is threshold-invariant (0.256–0.270 across τ 0.4–0.9 on a sibling run), so it lives in the weights, not the operating point. Three training rounds have not moved it.
• v2 gained recognition, not discrimination. It routes far more personal-data questions than v1 and is confidently wrong on a larger share of them (block 3: wrong 0.113 → 0.169, against a flat disjoint). Knowing a question is about your data and knowing which of your data are separate abilities; this round only advanced the first. The escalation path absorbs less of that than it used to — composed negatives-abstained fell 0.972 → 0.909.
• 8 of 14 scopes are under the 0.60 recall floor (v1: 12), weakest work_context 30%, relationship_context 34%, complexity and resources 37%. This artifact has not cleared its promotion gate; it fronts an LLM in shadow/advisory postures only.
• ai_conversations regressed on both slices (−7 gate, −4 unseen) — the one scope v2 made unambiguously worse. Questions about your own past AI conversations are the scope least like the artifact-concrete register v2 was trained to fix.
• All numbers are synthetic-benchmark. Real-traffic behaviour is being measured in shadow mode.
• English only. No user data, ever — the loader refuses artifacts whose manifest says otherwise.

Training data

source	licence	rows
schema-grounded synthetic + compounds + boundary negatives (Topos)	internal	7,369
AmazonScience/massive	CC BY 4.0	5,200
CLINC150 OOS	CC BY 3.0	1,200

v2 added ~95 artifact-concrete realizations and restored the concept-negative ratio (0.054 → 0.070). The second half is the interesting one: without a "what is a curriculum vitae?" negative beside "what's on my CV?", an earlier v2 candidate learned the artifact noun as sufficient evidence of ownership and fired on 43 near-miss definition questions, dropping negatives-abstained to 0.739. Teaching a positive without its matching negative was the whole regression.

This card is the CC BY attribution notice. No share-alike or non-commercial data.

Architecture

DistilBERT-base-uncased, 15 sigmoid outputs (14 scopes + none), BCEWithLogitsLoss with per-label pos_weight, max_length 64. Artifact = head.json (labels, thresholds, manifest, metrics) + model/ (HF-standard).