Cite

Which paragraphs answer the question.

A small encoder that, given a question and ordered paragraph texts, returns a calibrated probability per paragraph and two thresholds (needed, contributing).

Task	Paragraph citation / needed-blocks
Size	~42 MB weights (trimmed vocab, int8 embeddings, fp16 rest); a larger 68M build is under 68m/
Runtime	CPU (PyTorch); browser (ONNX Runtime WASM)
Weights	v0.3.0 · web/ for WASM
Demo	localangle/cite-demo (in-browser)

Cite does not write an answer. It ranks the paragraphs you already have and flags which ones are needed to answer the question.

Question: Who voted against the library renovation? → Cite highlights the paragraph naming Paulsen and Chen.

Try it

• Live demo: localangle/cite-demo — paste an article, ask a question, see which paragraphs Cite cites.
• Local playground: from the cite repo, uv run python deploy/cite_playground.py.

Install

pip install "cite @ git+https://github.com/localangle/cite.git"
# or, from a checkout:
uv sync

Python ≥ 3.12. Needs torch, transformers, safetensors, huggingface_hub.

Usage

from cite import CitePackage

cite = CitePackage.from_pretrained("localangle/cite")

paragraphs = [
    "The council approved a $4.2 million library renovation.",
    "Paulsen and Chen voted against the measure.",
    "Construction starts in March.",
]
scores = cite.predict("Who voted against the renovation?", paragraphs)

for score in scores:
    if score.needed:
        print(score.index, score.probability, paragraphs[score.index])

score.needed uses the packaged threshold (0.76). score.contributing is a looser relatedness cut (0.17). Logits are Platt-scaled to probabilities at load time; you do not fit thresholds yourself for the published package.

Load a local folder the same way:

cite = CitePackage.load("path/to/package")  # or from_pretrained("./path")

Files

File	Contents
cite.json	Format, Platt scaling, thresholds, source metadata
weights.safetensors	Trimmed encoder + head (int8 token embeddings, fp16 elsewhere)
encoder/	ModernBERT/Ettin config for the trimmed vocabulary
tokenizer/	Tokenizer with [MASK] as the block marker
web/	Browser package: model.onnx (~36 MB int8), vocab_map.bin, cite-web.json, tokenizer/
68m/	The 68M build as a full Python package (same layout as the repo root; ~107 MB weights)
web-68m/	Browser package for the 68M build (model.onnx ~88 MB int8)

PyTorch weights expand to fp32 at load; only storage is compressed. The web/ ONNX graph is what the static demo runs in WASM (soft cap 2048 tokens).

Accuracy

On a held-out real-news needed-blocks eval (v59, 1,800 questions over local news outlets; threshold chosen on a separate calibration split):

Metric	Cite-32M package
Coverage	0.692
Exact set match	0.725
Empty-set accuracy	0.897
Any-extra rate	0.066

The browser ONNX build on the same eval: exact 0.722, coverage 0.687, empty 0.902.

Broad and abstract questions

The eval above uses specific fact-lookup questions. Later versions add training so that broad questions ("How will the library change as a result of the renovations?") score the paragraph that answers them instead of washing out.

• v0.2.0 fine-tuned on LLM-written broad rewrites of real training questions.
• v0.3.0 adds synthetic short articles where nearly every paragraph shares the topic word, with hard negatives, then averages the weights 0.75/0.25 with v0.2.0 to keep the specific-question operating point.

On a 117-item hand-written probe (specific question vs. an abstract rewrite of it; 92 items after removing 25 where several paragraphs plausibly answer the rewrite), abstract-question results:

Abstract question	v0.1.0	v0.2.0	v0.3.0
Top-1 (all 117)	0.650	0.744	0.752
Needed recall (all 117)	0.402	0.752	0.761
Top-1 (92 audited)	0.826	0.848	0.859
Top-3 (92 audited)	n/a	0.957	0.978

Packaged-model numbers for v0.2.0 and v0.3.0; v0.1.0 from the fp32 student. Specific-question probe top-1 is 0.949 (v0.2.0) and 0.966 (v0.3.0). The probe is small and written by one author; treat it as a regression check, not a benchmark.

On a short, topic-saturated example article, v0.3.0 improves but does not solve the hardest case: for "What specific improvements will be made...?" the improvements paragraph moves from rank 4 (18%) to rank 2 (60%), still behind a funding paragraph that repeats the word "renovation". See experiments/107-cite-abstract-probe, experiments/108-cite-broad-consistency and experiments/109-cite-hard-negatives.

Cite 68M

A larger build, distilled from the same teacher with the same data and training steps, on Ettin-68M. It is more accurate, especially on broad and abstract questions, and about 3x slower. The 32M build above stays the default.

cite = CitePackage.from_pretrained("localangle/cite", subfolder="68m")

The demo has a model switch; the 68M build downloads about 88 MB on first use and is cached after that. Packaged thresholds: needed 0.79, contributing 0.17.

	Cite 32M (v0.3.0)	Cite 68M
Weights (package)	41.8 MB	107.0 MB
Browser model.onnx	36.3 MB	88.3 MB
v59 eval: coverage / exact / empty-set / any-extra rate	0.692 / 0.725 / 0.897 / 0.066	0.729 / 0.763 / 0.923 / 0.042
Abstract probe, 92 audited items: top-1 / needed recall	0.859 / 0.859	0.902 / 0.891
Abstract probe, all 117: top-1	0.752	0.786
Browser (WASM, 1 thread): 7 paragraphs / 44 paragraphs	185 ms / 1.4 s	547 ms / 4.2 s

On the short example article, for "What specific improvements will be made to the library as a result of the renovation?", the improvements paragraph scores 93.5% on 68M (60% on 32M, behind a funding paragraph). The same caveats apply: one run per size, a small hand-written probe, and browser timings from a single machine. See experiments/110-cite-68m.

Latency

Batch-1, CPU, AMD EPYC (HF cpu-upgrade, 8 vCPU) for the related ONNX int8 form: about 44 ms p50 at 5–12 paragraphs and 268 ms at 33–60. The portable PyTorch package this repo ships is the accuracy-matched serving artifact; long documents are dominated by attention cost.

How it works

Cite is a fine-tuned Ettin encoder (ModernBERT family). The question and each paragraph are packed into one sequence with a [MASK] marker before every paragraph; the head reads the marker states. A 400M teacher labeled real news; the 32M student was distilled then fine-tuned on LLM-written broad rewrites of its training questions (consistency distillation) and on synthetic hard-negative articles, and compressed (trimmed vocabulary, int8 embeddings, fp16 weights).

Limitations

• Cite ranks paragraphs; it does not generate answers or quote spans inside a paragraph.
• Lexical overlap still matters: "big 10" and "Big Ten" can score differently. Scores for a paragraph also drift down in very long articles.
• Eval is English news-style text. Other domains are unmeasured.
• Empty-set accuracy drops a few points from calibration to eval; treat the packaged threshold as a starting point for a new domain.
• Documents longer than max_tokens (7999) drop trailing paragraphs.
• Training used private teacher labels and news text that is not redistributed here.

Built on

• Ettin encoder 32M (MIT) by JHU CLSP. Fine-tuned and compressed for Cite; see THIRD_PARTY_NOTICES.md.

License

MIT for the Cite code and this package's weights, subject to the Ettin MIT notice above.

Citation

@software{cite_2026,
  title  = {Cite: Which paragraphs answer the question},
  author = {Local Angle},
  year   = {2026},
  url    = {https://huggingface.co/localangle/cite},
}