AIDAM_VNEWS_435M_V1.0.0

A news-grounding verifier. Given a source document and a claim about it, the model judges whether the document entails, is neutral to, or contradicts the claim — a 3-way natural language inference (NLI) call used to check whether a summary or generated claim is actually grounded in its source text.

This is part of AIDAM, a project built around a simple rule: the factual verdict on a claim never comes from a large language model. It comes from a small, specialised NLI encoder like this one, plus deterministic aggregation code. The LLM in the pipeline drafts and explains; it does not judge.

Model details

Task	3-class NLI (entailment / neutral / contradiction)
Architecture	DeBERTa-v3-large
Parameters	435,064,835
Base checkpoint	MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli
Formats provided	ONNX (5 MB graph + 1.7 GB external weights) and SafeTensors (~830 MB)
Inference window	2048 tokens per chunk, 1536-token stride (25% overlap) for documents that exceed 2048 tokens
License	Apache 2.0
Language	English only — see "What this is not"

Benchmark: LLM-AggreFact

Measured on LLM-AggreFact, 11 subsets, 29,320 grounding pairs pulled from summarization and RAG-style factual-consistency datasets. Metric: balanced accuracy per subset, macro-averaged over the eleven subsets.

Subset	Balanced accuracy
ExpertQA	59.5
FactCheck-GPT	69.7
AggreFact-XSum	72.4
Wice	73.6
TofuEval-MediaS	74.8
AggreFact-CNN	75.3
TofuEval-MeetB	76.5
ClaimVerify	79.5
Reveal	82.3
RAGTruth	86.5
Lfqa	88.2
Macro (headline number)	76.2
Pooled (all pairs, unweighted by subset)	84.7

This model was measured against a pre-registered gate before training began:

• Macro balanced accuracy > 72.51 (the gate threshold): scored 76.2 — pass
• No individual subset regresses beyond measurement noise against the previous-generation verifier (macro 71.45): pass
• Parameter budget < 500,000,000: 435,064,835 — pass

At its own weight class (under 500M parameters), the current published leader on this benchmark is FactCG-DeBERTa-v3-Large at 75.6 macro balanced accuracy. This model scores 76.2 — 0.6 points, about 1.1 standard errors, above that leader. A commonly cited 77.4 figure on this benchmark belongs to a 7-billion-parameter model, a different and much larger weight class, and is not a comparable number here.

Training

• Base checkpoint: MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli, itself fine-tuned on MNLI, FEVER, ANLI, LiNG and WANLI
• Training corpus: 38,007 rows — FactCG v4 (26,615 rows) plus a capped slice of LLM-AggreFact's own dev split (11,392 rows: all 7,392 contradiction rows, and 4,000 of 23,002 available entailment rows)
• Learning rate 5e-5, 3 epochs, effective batch size 32 (micro-batch 1, gradient accumulation 32), max sequence length 2048 with gradient checkpointing, OneCycleLR schedule with 6% warmup, 8-bit AdamW, bf16
• Best checkpoint: step 3,200 of 3,561, internal dev balanced accuracy 89.23
• Hardware: a single 12 GB consumer GPU, ~2.3 hours training time
• This was the 17th training attempt for this specialisation; the first 16 either used a smaller architecture or a shorter context window and did not clear the gate.

What this is not

• This score is specific to news-grounding and summarization-style factual consistency. It is not a general reasoning or knowledge benchmark, and it says nothing about the model's behavior on encyclopedic claims, scientific claims, or viral/real-world claims — those are judged by sibling models in the same family, each measured on its own register.
• English only. Both training and evaluation used English-language text; no Spanish-language measurement exists for this checkpoint.
• Weakest subset is ExpertQA (59.5 balanced accuracy), a noisier minority-class subset than the rest of LLM-AggreFact.
• Per-class recall leans toward "not supported": 81.4 recall on the supported class versus 88.1 on not-supported.
• This model was the first of an 11-model family to be promoted in its current training era; a stronger same-size-class model on this exact benchmark could exist and has not been surveyed by this project.

Usage

The model is a standard 3-class sequence-pair classifier (entailment / neutral / contradiction), usable directly through transformers or through the ONNX export. Example with transformers:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("DeliVali/AIDAM_VNEWS_435M_V1.0.0")
model = AutoModelForSequenceClassification.from_pretrained("DeliVali/AIDAM_VNEWS_435M_V1.0.0")

evidence = "The company reported revenue of $4.2 billion."
claim = "Revenue exceeded four billion dollars."

inputs = tokenizer(evidence, claim, return_tensors="pt", truncation=True, max_length=2048)
with torch.no_grad():
    logits = model(**inputs).logits
label = logits.argmax(-1).item()

For documents longer than the 2048-token window, chunk with an ~25% overlap (1536-token stride) and aggregate chunk-level predictions.