Access Noah 2.5 (standard export)

Free for non-commercial use under CC BY-NC 4.0. Access is granted automatically. For commercial use, contact Beinex (beinex.ai).

Noah by Beinex AI

Noah 2.5, standard export

Beinex AI · 2026-10-11

A plain ModernBertForSequenceClassification (no custom code) distilled from Noah 2.5, Beinex's research model (not public). It takes (premise, hypothesis) and returns P(supported) in label 0. Use it where only standard Transformers models are allowed (leaderboard scorers, inference servers). It has no number audit, no evidence head and no chunking; for those, use the full model.

Open weights, non-commercial (CC BY-NC 4.0). Free for non-commercial use; for commercial use, contact Beinex (beinex.ai). The full Noah 2.5, with the number audit, evidence and certified escalation, is available from Beinex.

Nutrient grounding leaderboard: 6th of 15; first independent entry; first among models not built by Nutrient or a cloud AI vendor; open weights under CC BY-NC 4.0.

Usage

from transformers import pipeline

clf = pipeline("text-classification", model="beinexai/noah-v2.5-research-std", top_k=None)   # gated: accept the terms on the model page once, then log in (huggingface-cli login)
premise = "Item | FY2023 | FY2022\nNet sales | $4,213M | $3,987M"
out = clf(dict(text=premise, text_pair="Net sales were $4.2 billion in FY2023."))
p_supported = next(d["score"] for d in out if d["label"] == "supported")

Batch scoring, exactly as the Nutrient benchmark scorer does it:

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tok = AutoTokenizer.from_pretrained("beinexai/noah-v2.5-research-std")
model = AutoModelForSequenceClassification.from_pretrained("beinexai/noah-v2.5-research-std").eval()
premises = ["Item | FY2023 | FY2022\nNet sales | $4,213M | $3,987M"]
hypotheses = ["Net sales were $4.2 billion in FY2023."]
enc = tok(premises, hypotheses, truncation=True, padding=True, max_length=1024, return_tensors="pt")
with torch.no_grad():
    p_supported = model(**enc).logits.softmax(-1)[:, 0]

Results

Nutrient grounding benchmark (en, 5,000 rows), official unmodified score.py: 0.8145 overall / 0.7041 number. Independently reproduced: the leaderboard maintainers reran the scorer on their hardware before merging (12 Oct 2026) and obtained the same numbers. Everything else in this table is self-reported.

	Standard export	Full Noah 2.5 (own pipeline)
Nutrient (en) ROC-AUC overall, official `score.py`	0.8145	0.800 (Noah-native scoring)
Nutrient number AUC	0.7041	0.755
Sources not in our training mix (3,551 rows), overall	0.819 (DeBERTa-v3-large zero-shot: 0.840)	—
Nutrient table / prose / date / string AUC (official)	0.803 / 0.908 / 0.965 / 0.846	0.787 / 0.922 / 0.935 / 0.754
Agreement with the full model (held-out training items, decision at 0.5)	0.966	—
Correlation with the full model's calibrated score	0.979	—
Dev ECE	0.022	—

How it was made. The encoder is copied from Noah 2.5, then the whole model is fine-tuned on 60,000 of Noah 2.5's own training items (filtered against the benchmark by exact hash and 20-word shingles), in the official input format, to match Noah 2.5's calibrated probability (soft targets) plus the gold label (weight 0.3). One epoch, lr 1e-05. The number channel cannot be carried into a standard model, so its gains are only partly retained.

Training data. The same mix as Noah 2.5: 460,352 items after filtering, no ANLI. The 60,000 distillation items were drawn from it. The item counts below sum to 480,486; the artifact filter then removed 20,134, giving the 460,352 trained on.

Source	What it teaches	Licence	Items
Synthetic table claims (Beinex)	cell lookups and numeric hard negatives (adjacent cell, wrong period, ×10³ scale, sign, transposed digits, offsets) with format-equivalent positives	Beinex	130,933
TabFact (train)	table fact verification	MIT (code); Wikipedia tables CC BY-SA	91,675
FinQA (train)	financial-report QA recast as claims + numeric negatives	MIT	18,508
TAT-QA (train)	hybrid table + text financial QA recast as claims	CC BY 4.0	21,851
MultiNLI	general NLI	see dataset card (mostly OANC)	80,000
e-SNLI	NLI with evidence highlights	SNLI CC BY-SA 4.0; explanations MIT	40,000
WANLI	harder NLI	CC BY 4.0 (LLM-generated; licence review pending)	60,000
MiniCheck C2D/D2C	grounding of LLM-style claims	MIT (LLM-generated; the generating model's terms are under review)	14,395
ContractNLI (train)	contract clauses, evidence spans	CC BY 4.0	23,124

Data hygiene: contamination guard against the Nutrient benchmark (1,184 items removed by exact hash and 20-word shingles); AFLite-style artifact filter (claim-only probe AUC 0.648 → 0.543; 20,134 items removed). Two LLM-generated sources — WANLI (CC BY 4.0) and MiniCheck (MIT) — are under provenance review. The base model, answerdotai/ModernBERT-large, is Apache-2.0.

Licence: CC BY-NC 4.0: free for non-commercial use with attribution. Commercial licences: beinex.ai.