solvi extract-base

A general field-by-description extractor for solvi: give it a document and a plain description of a field ("the notice period required to terminate the agreement") and it returns the exact span in the text, with a confidence, or says the field is absent. Long documents are read in overlapping 1024-token windows.

It is meant as a starting point: good out of the box on common fields (dates, totals, names), and a better initialization than plain ModernBERT when you fine-tune on your own labeled documents. It is not a reliable zero-shot extractor for arbitrary fields yet — see the numbers below.

Use

from solvi import Answer, Catalog, Question, System
from solvi.extract_long import LongSpanExtractor

ex = LongSpanExtractor.load("solvi-ai/extract-base")          # pip install "solvi[model]"
contract_text = open("contract.txt").read()
cat = Catalog()
cat.extract(ex.field("governing_law", "the clause that says which state's or country's law governs the contract"))

@cat.rule("law")
def law(governing_law):
    return next((s for s in ["Delaware", "New York", "California"] if s in governing_law), "other")

res = System(cat, [Question("law", "Which law governs?", Answer.choice(["Delaware", "New York", "California", "other"]))]).ask({"doc": contract_text})
print(res["law"].answer, res.computed_state)                 # the answer and the quoted clause with its offsets

Fine-tune on your documents: ex.fit([(text, description, (start, end) or None), ...], epochs=3), then ex.tune_threshold(name, held_out) and ex.save(dir).

Architecture

ModernBERT-large encoder + a linear start/end pointer head. Input: [CLS] field description [SEP] document window [SEP]. "No answer" is position 0. The best span over all windows is returned if its score is above the field's threshold (a default threshold of 0.3 was tuned on held-out data of many fields; tune your own per field with a few labeled documents). Files: model.safetensors (encoder, bf16), span_head.pt, solvi_extract.json (window settings, thresholds).

Training data

Two epochs over a mix of ~58 000 windows:

• SQuAD 2.0 — 40 000 questions, including unanswerable ones (CC BY-SA 4.0);
• CUAD v1 — 368 training contracts, 33 of the 41 clause types, with CUAD's own clause explanations as descriptions (CC BY 4.0). Held out entirely: Governing Law, Agreement Date, Non-Compete, Exclusivity, Cap On Liability, Effective Date, Uncapped Liability, Competitive Restriction Exception; the 102 official test contracts were never seen;
• CORD v2 — 800 training receipts, 20 field types with written descriptions (CC BY 4.0); tax and change held out.

Evaluation (held-out fields and datasets, one seed)

No labeled examples of the task:

test	result
CORD receipts, fields never trained (tax, change), value match	46.5% / 67.9% (a model trained on 7 fields only: 14% / 66%)
CUAD contracts, 5 typed questions over 5 never-trained clause types	73.1% (a fine-tuned answerer model trained on 408 contracts: 77.5%)
Kleister-NDA (dataset never seen): effective date / jurisdiction / party / term	43% / 30% / 54% / 59%
SROIE receipts (dataset never seen): company / date / address / total	71% / 95% / 2% / 73%

With a few labeled documents:

setup	result
CUAD, per-field thresholds and calibration from 40 labeled contracts	85.5% (vs 77.5% for the answerer trained on 408)
SROIE, fine-tuned on 25 / 50 / 100 receipts: field match, this model vs plain ModernBERT-large	85.8 / 86.1 / 89.2% vs 82.6 / 85.5 / 88.0%
SROIE, 100 receipts, typed questions answered by solvi rules	96.1% vs 94.1%

Weak spots: addresses (the zero-shot span boundaries do not match), jurisdiction phrased only as a state name, and fields whose wording is far from anything in the training mix. For production, label ~25–100 documents and fine-tune.

Limitations

English only. Trained on receipts, contracts and Wikipedia-style QA; other document types are untested. Scores are confidences of a span model, not calibrated probabilities — use System.calibrate on held-out documents.

ONNX (browser and CPU)

onnx/model_fp16.onnx — the encoder and span head in one graph (inputs input_ids, attention_mask int64 [batch, length] → logits float [batch, length, 2]: start and end scores), fp16 weights with float32 inputs and outputs. On 175 test fields it returns exactly the same spans as the PyTorch model. Runs with onnxruntime (CPU) and onnxruntime-web (WebGPU); windowing and span decoding follow LongSpanExtractor.predict. Export your own fine-tuned model with tools/export_onnx.py in the solvi repo. (Dynamic int8 quantization is not provided: it changed half of the spans.)