superhq-ai/tern-extract-v0.1

Tern Extract v0.1

Tern Extract is a 70.7M parameter model that extracts typed fields from English text using an explicit JSON Schema. It runs locally and includes an OpenAI SDK compatible adapter for structured extraction requests.

Give it a short document, field descriptions, and a schema. It predicts source spans and whether each field is present, missing, or conflicting, then assembles and validates the JSON. Strings come from the source; explicit numbers and booleans are converted to their JSON types.

This is an experimental release. It is intended for evaluating local extraction from short records, intake notes, and document text in workflows where the result can be checked.

Quick start

Download the repository while signed in to Hugging Face:

hf download superhq-ai/tern-extract-v0.1 --local-dir tern-extract
cd tern-extract
python3.12 -m venv .venv
.venv/bin/pip install -r requirements.txt
.venv/bin/python -m tern_extraction.server --checkpoint checkpoint --port 8001

In another terminal:

curl http://127.0.0.1:8001/v1/responses \
  -H 'Content-Type: application/json' \
  --data-binary @examples/example-01.json

The example extracts an invoice into:

{"invoice_code": "INV-4112", "total_amount": 296.5, "vendor": "Riverbend 61001"}

Use the same request through the OpenAI Python SDK:

import json
from pathlib import Path
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8001/v1", api_key="local")
request = json.loads(Path("examples/example-01.json").read_text())
response = client.responses.create(**request)
print(response.output_text)

For direct inference:

from tern_extraction import Tern

model = Tern("checkpoint")
result = model.predict(request["instructions"], request["input"], request["text"]["format"]["schema"])
print(result["value"])
print(result["evidence"])

The server supports stateless, nonstreaming /v1/responses and /v1/chat/completions extraction requests, including tested Pydantic parsing. It accepts exactly one user source message and an explicit json_schema format. Tools, conversation state, images, audio, and streaming are unsupported.

Supported schema

Use descriptive field names and descriptions. The schema can contain up to eight scalar fields, including required nested objects up to depth eight. Supported leaf types are strings, numbers, integers, booleans, and string enums, with nullable or optional scalar fields. Objects must set additionalProperties: false. Nonrecursive local $defs references are supported.

Each field's instruction, query, and source must fit within 512 tokens. Longer inputs are rejected without truncation. The model accepts extracted text; PDF parsing and OCR are separate preprocessing steps. Arrays, nullable or optional objects, arithmetic, unit conversion, and unrestricted text normalization are outside this version's scope.

Evaluation

Development task	Correct requests
Authored workflow documents	71/75
Same documents with renamed keys	70/75
Same documents with rewritten descriptions	68/75
Same documents with both changes	67/75
Current paired-role challenge	140/140
Earlier paired-role challenge	133/140
Older prose examples	8/12
Total	557/592

Field values were correct on 1,174/1,216 queries. Of the 75 authored documents, 65 were correct under all four schema views. These are synthetic development sets used during model development. Repeated schema views and paired queries share source documents, so the totals are not independent examples or a held-out generalization estimate. Broader generalization and superiority over other extraction models have not been established.

On an Apple M1 CPU with two PyTorch threads, the three-field invoice example took 105 ms median and 110 ms p95 across 30 requests after five warmups. This measures direct inference, including tokenization and assembly, and excludes model loading and server/network overhead. See evaluation/cpu-timing.json for the environment and workload.

Complete saved errors are in evaluation/failures.json. Representative limitations include:

Source fact	Expected	Observed
Current budget is missing; 1167.21 belongs to last year's work	null	1167.21
Corrected reimbursement amount is 152.43 USD	152.43 and "USD"	Both fields null
Assigned engineer is Carmen Okafor; sender is T. Reyes	"Carmen Okafor"	"T. Reyes"

Schema validation checks types and structure, not factual correctness. Role confusion, historical values, false nulls, and sensitivity to field wording remain known weaknesses. Status probabilities are uncalibrated and should not be treated as confidence in an extracted value.

Model and training

Tern uses a shared DeBERTa-v3-xsmall encoder with span and three-way status heads, totaling 70,684,037 parameters. It evaluates each schema field against the source and selects type-valid spans. It does not generate JSON tokens autoregressively or call an external model at inference.

The QA initialization comes from nbroad/deberta-v3-xsmall-squad2, built on Microsoft DeBERTa-v3-xsmall. Adaptation uses synthetic English workflow data with span and evidence-status supervision, type/span validation, and cross-teacher review of generated prose. The data has not received an independent human annotation audit. Additional provenance is in provenance/weights.json.

License and attribution

Weights are released under CC BY 4.0, following the QA initialization's model-card license. Runtime code is MIT licensed. See ATTRIBUTION.md and LICENSE-CODE for attribution and license details.