Dialogues/topos-facts-gate-v0.1

Topos Facts v0.1 · gate

Topos Facts reads one message and writes structured facts about its author, such as a goal, a move or a new job, across 27 fact packs. It uses two small encoders and no generative model, so every value it writes is either an option from a fixed list or text copied from the message.

This repo holds the gate: a DeBERTa-v3-large NLI cross-encoder that decides which facts a message states and picks the options for choice fields. Its partner, the reader, copies free-text values out of the message: Dialogues/topos-facts-reader-v0.1.

Try it: Dialogues/topos-facts runs the whole pipeline on any message you type, in your browser. Nothing you type leaves your device.

How it works

1. Gate. Every predicate of every pack is a hypothesis, The author mentions their <predicate> (<pack title>). The gate scores entailment for each (message, hypothesis) pair, and a predicate fires above its tier's threshold.
2. Options. For each fired predicate the gate scores <hypothesis> <Field>: <option>. for every option of every choice field, and keeps the best one above the option threshold. The same pass scores whether the message is about its author.
3. Reader. Each free-text field is asked as a question, such as What goal of the author's goal does the message state?, and the answer is a span copied from the message.

The hypotheses, options, questions and thresholds are in topos_facts_spec.json. topos_facts.py is the reference implementation, and it is plain transformers code that needs no trust_remote_code.

Threshold	Value
Gate, special predicates (health, beliefs and similar)	0.985
Gate, personal predicates	0.99
Choice-field option	0.01
Reader span margin over "no answer"	-6.0

The gate thresholds are high on purpose: v0.1 was tuned for precision first, so it writes fewer facts and fewer wrong ones.

Usage

pip install torch transformers huggingface_hub

from huggingface_hub import hf_hub_download
import importlib.util, sys

path = hf_hub_download("Dialogues/topos-facts-gate-v0.1", "topos_facts.py")
spec = importlib.util.spec_from_file_location("topos_facts", path)
topos_facts = importlib.util.module_from_spec(spec); spec.loader.exec_module(topos_facts)

tf = topos_facts.ToposFacts()           # downloads both models; GPU or Apple MPS if present, else CPU
tf.extract("Booked the flights: we move to Toronto in August for my new job.")
# [{'pack': 'life.events', 'predicate': ..., 'gate': 0.99..., 'value': {...}, 'empty': [...]}, ...]
tf.extract("...", packs=["aspirations.goals", "life.events"])   # limit the packs

You can also use the gate directly as an NLI model. Label 0 is entailment and label 1 is not_entailment. Pass the message as the premise and a hypothesis from the spec as the second sequence.

Evaluation

Synthetic gold set (40 hand-written messages × 27 packs, 1,080 pairs, predicate-level):

Model	Predicate F1	Precision	Recall	Wrong predicates
GPT-5.4 mini, prompted	0.681	0.52	1.00	~74
Qwen 3.5 9B, prompted	0.645	0.53	0.83	~50
Topos Facts v0.1	0.577	0.81	0.45	5
Llama 3.3 70B, prompted	0.439	0.29	0.90	~250

Real messages (private). On a private set of real messages, a blind LLM labeler working from written rules judged 73% of a 240-fact sample right (95% interval 0.67–0.78). The same labeler judged 75% of the facts a prompted 9B LLM had written on the same data right. Precision varies by pack: life events 0.90, goals 0.67, commitments 0.57, self-narrative 0.55. The labeler is a stand-in for human judgement and is probably kinder to this model than a person would be.

Limitations

• English only, and trained on informal messages, so other writing styles may transfer less well.
• Recall is low by design: most of what an LLM would write, v0.1 does not.
• Free-text fields are often left empty when the reader finds no clear span.
• obligations.commitments and narrative.self are the weakest packs. Consider leaving them off.
• The "is this about the author" score is reported, but at v0.1's thresholds it filters nothing.
• A fact is only as good as the message it came from. Sarcasm, quotes and hypotheticals can fire the gate.

In the browser

onnx/model_fp16.onnx is an fp16 ONNX export for transformers.js, which is what the Space runs:

import { AutoModelForSequenceClassification, AutoTokenizer } from "@huggingface/transformers";
const model = await AutoModelForSequenceClassification.from_pretrained("Dialogues/topos-facts-gate-v0.1", { dtype: "fp16", device: "webgpu" });

With both models in fp16, the pipeline matches PyTorch on the gold set (1,079 of 1,080 pairs identical). There is no int8 export: quantizing to int8 moves the gate's scores far enough to break its 0.985–0.99 thresholds.

Training

• Base: MoritzLaurer/deberta-v3-large-zeroshot-v2.0-c (MIT), the variant trained only on commercially usable NLI data.
• Data: private and not released. Labels came from an LLM teacher and were then checked by an LLM labeler against written rules. No example in this card or the demo comes from the training data.
• Fine-tuning: full fine-tune on entailment pairs for predicates and options. AdamW at learning rate 1e-5 with weight decay 0.01, batch size 16, 10% warm-up, 512 tokens, seed 13. Up to 4 epochs, with epoch 3 kept on a held-out dev set.

Licence

MIT, as the base model. The reader is CC-BY-4.0 because its base is.

Made by Dialogues for Topos.