atomizer-gliner-large

A single-pass span atomizer with a critic. It splits a text into short, self-contained claims ("atoms") and scores each one:

• verifiable: is this a factual claim that can be checked?
• checkworthy: is it worth fact-checking?

Atomization is the step any claim-level fact-checking tool depends on: search, evidence judgment and the verdict all work on the claims it produces, so a claim that is missed, merged or wrongly resolved here cannot be checked correctly later. The figure shows fact-assessor, our own opinionated fact-assessment tool: atomizer is the "trained atomizer" option of its Atomization step, and its verifiable / checkworthy scores drive Claim filtering. The integration is in progress (see Intended use).

The model does not generate text. It selects words from the input, replaces references ("She" → "Marie Curie") with earlier spans or keeps them as written when the referent is not in the text, and adds closed-class glue words ("is", "the", "'s"). An atom cannot contain a name or number that isn't in the input.

Backbone: gliner-community/gliner_large-v2.5 (DeBERTa-v3-large, 1024 → 768 projection), 456M parameters. It is the large sibling of atomizer-gliner-small (163M), with the same architecture, data, splits and recipe as small v0.5.

Status: research prototype (v0.1). Large splits chat and agent text better than small v0.5 and is tied elsewhere, but puts a wrong name in place of a reference more often. For general use, small v0.5 is the recommended model (a third of the size, nearly the same quality); use large when splitting quality matters most and a GPU is available.

Intended use

atomizer turns free text into checkable claims: short, self-contained statements, each scored for whether it is a factual claim (verifiable) and worth checking (checkworthy). It is meant as the first stage of:

• fact-checking and claim verification: split an LLM answer, article or post into claims, keep the checkable ones, and verify each against evidence;
• hallucination detection and factual-precision scores for LLM output (the share of claims that are supported);
• check-worthiness triage: rank claims for human fact-checkers;
• retrieval units: propositions as the indexed unit for retrieval.

It does not decide whether a claim is true: verifiable means "can be checked", not "correct".

Our own pipeline: fact-assessor, an opinionated fact-assessment tool (figure at the top) that checks any text against evidence and returns a verdict per claim, a fact score and a knowledge graph. atomizer works with any claim-level verifier; fact-assessor is where we use it.

There, atomizer is the "trained atomizer" option of the Atomization step: one local forward pass gives the claims and their verifiable / checkworthy scores, and Claim filtering keeps the claims whose scores pass a cut-off, so neither step needs an LLM or API call. That means no API cost, milliseconds instead of seconds, and no name or number in a claim that is not in the input. A wrong claim can still come from a wrong reference (see Limitations). The fact-assessor integration is not done yet; it and an end-to-end comparison with the LLM atomizer are in progress.

Usage

The loading code is the atomizer package at NISH1001/atomizer (private for now, access on request).

from atomizer import Atomizer

atomizer = Atomizer.load("nishparadox/atomizer-gliner-large", revision="v0.1")
for atom in atomizer.atomize("I think the height of mount everest is 8846m."):
    print(atom.text, round(atom.verifiable, 2), round(atom.checkworthy, 2))

Training

One run from the backbone (not from a small checkpoint) with small v0.5's recipe (scripts/slurm/recipe.sh), every training example in every epoch:

data per epoch	Propositionizer 42,856 + critic 40,402 (ClaimBuster + CheckThat! 2024) + chat templates 5,999, once each; synthetic 1,496, Decontextualization 1,212 and Factcheck-Bench 62, three times each = 97,567
epochs / steps	3 / 9,147; released: the last step
batch	32 (2 × 16, data-parallel)
learning rates (encoder / GLiNER layers / heads)	3e-5 / 1e-4 / 5e-4 (small's, not tuned for large)
hardware, time	2× A100 80GB, 2 h 15 min
W&B run	large-v0.1-rc2

Training texts that also appear in a validation or test split are dropped. Validation exact F1 levels off at 0.41–0.44 from step 5,000 on while training exact F1 keeps rising (0.68 at the end), so the second half of training mostly memorises.

Evaluation

The test suite of the small card, same splits. "sig." = paired bootstrap over texts (2,000 resamples; 1,000 for critic sets), 95% CI excluding 0.

set	metric	small v0.5	large v0.1-rc1	large v0.1	v0.1 vs small v0.5
synthetic (323)	soft F1	0.905	0.907	0.919	+0.013 sig.
	exact F1	0.554	0.584	0.619	+0.064 sig.
	wrong references ↓	0.012	0.012	0.020	+0.008 sig. (worse)
chat held-out (500)	soft F1	0.989	0.991	0.993
Factcheck-Bench (30)	soft / exact F1	0.753 / 0.146	0.761 / 0.201	0.751 / 0.181	tie / tie
Decontextualization valid (195)	soft F1	0.551	0.558	0.578	+0.027 sig.
	references resolved	0.703	0.595	0.665	tie
	wrong references ↓	0.073	0.088	0.106	tie
Propositionizer (500)	soft F1	0.742	0.739	0.740	tie
PropSegmEnt (500)	soft F1	0.757	0.758	0.754	tie
CheckThat! 2024 test (318)	checkworthy F1 / AUC	0.865 / 0.978	0.836 / 0.970	0.882 / 0.985	tie / tie
ClaimBuster valid (2,353)	checkworthy F1 / AUC	0.767 / 0.938	0.771 / 0.939	0.774 / 0.939	tie / tie
fact-assessor claims (19)	verifiable F1	1.00	1.00	1.00

Against v0.1-rc1 (the staged recipe), v0.1 has 4 significant wins (synthetic soft and exact, CheckThat! F1 and AUC) and 1 loss (CheckThat! dev F1). Against the best-validation checkpoint of the same run (v0.1-rc2, step 5,000) it wins 4 and loses 4; across all pairs it has the best record (11 wins, 6 losses). Factcheck-Bench's atom-level verifiable AUC is left out: its test set has only 6 non-verifiable atoms.

Versions

Tag	Status	Training
v0.1	release (main)	one run from the backbone, last step 9,147 (= v0.1-rc3)
v0.1-rc3	candidate	same as v0.1
v0.1-rc2	candidate	same run, best validation exact F1 (step 5,000): best CheckThat! F1 (0.915), weaker on Factcheck-Bench (0.725)
v0.1-rc1	candidate	small v0.4's staged recipe, stage 5 step 3,000

Limitations

Those of small v0.5, plus:

• Wrong references: about 1 in 10 substituted references in Decontextualization is wrong (small v0.5: 1 in 14, small v0.4: 1 in 22).
• Size: 2.8× the parameters of small (model.pt 1.8 GB, fp32) for gains of about 0.01 soft F1. Half the gold atoms in the largest training set (Propositionizer) are rewrites a span model cannot build, so a larger encoder does not lift that ceiling.
• Thresholds (atoms 0.35, critic 0.5) are not tuned for large.