freelawproject/caselaw-block-tagger

Case-law block tagger

A token-classification model that labels the structural components of a U.S. case-law opinion: given the OCR'd text of an opinion as lightly-marked-up HTML, it tags the caption and headmatter elements and the opinion skeleton as character spans.

Classes (12, BIO): party, separator, docketnumber, court, attorneys, judges, datefiled, otherdate, history, disposition, author, heading. Everything else (body prose) is O.

Built by the Free Law Project for structuring scanned case-law reporters; also applicable to born-digital opinion text serialized to the same input format.

Model details

• Architecture: ModernBERT-large (0.4B), fine-tuned from CaseLawModernBERT-large (continued-pretrained on 13B words of U.S. court opinions), 8,192-token context.
• Input format: one court case per sequence, as minimal HTML — <p> per text block, plus <blockquote>, <em>, <sup>; footnote content omitted. One \n between blocks. The model never generates text; it emits per-token BIO labels (12 classes × begin/inside + outside = 25 label ids).
• Training data: ~4,400 pages of scanned U.S. reporter volumes (16 volumes across 8 reporter series), OCR'd through a multi-engine pipeline; labels human-annotated (a ~680-page golden set) and model-seeded + human-reviewed (a ~3,900-page extension). Augmentations: random block-boundary drops (OCR segmentation robustness) and styling dropout (<em>/<sup>-stripped copies).
• Training: warm-started from a golden-set fine-tune; lr 3e-5, effective batch 16, bf16, best epoch by span-F1 on a mixed validation set, early stopping.

Evaluation

Span-level F1: exact (start, end, label) sets, gold and predictions normalized identically (whitespace/markup edge anchoring; the "normalized" column additionally ignores edge punctuation). No prediction post-processing. All evaluation volumes are unseen in training.

eval set	span-F1 strict	normalized
validation (3 volumes, human-reviewed gold)	0.929	0.935
test (4 volumes, human-reviewed gold)	0.937	0.950

Generalization detail: on the validation volume from a reporter series entirely absent from training (Pacific 3d), span-F1 is 0.934. On the test set, the in-sample-reporter volume scores 0.979; the fully out-of-sample reporter series (South Eastern 2d, Bankruptcy Reporter) score 0.872–0.903.

Per-class on validation (strict): party 0.990, attorneys 0.983, datefiled 0.981, author 0.968, court 0.963, separator 0.994, docketnumber 0.949, heading 0.938, judges 0.865, otherdate 0.842, history 0.815, disposition 0.801.

Caption classes (the six fields composing a case caption — party, separator, docketnumber, court, datefiled, otherdate — typically the primary extraction targets), aggregated as micro-F1 over the six:

eval set	strict	normalized	gold spans
validation	0.972	0.973	627
test	0.979	0.986	2,005

Per class (strict, val / test): party 0.990 / 0.977 · separator 0.994 / 0.967 · docketnumber 0.949 / 0.968 · court 0.963 / 0.997 · datefiled 0.981 / 0.998 · otherdate 0.842 / 0.750 (rare secondary argued/decided dates; 12–28 gold spans per set).

Usage

from transformers import (AutoModelForTokenClassification,
                          AutoTokenizer)

repo = "freelawproject/caselaw-block-tagger"  # e.g. freelawproject/caselaw-block-tagger
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForTokenClassification.from_pretrained(repo)

text = (
    "<p>Jane ROE, Appellant,</p>\n"
    "<p>v.</p>\n"
    "<p>STATE of Example, Appellee.</p>\n"
    "<p>No. 24-123</p>\n"
    "<p>Court of Appeals of Example.</p>\n"
    "<p>January 1, 2026</p>\n"
    "<p>Affirmed.</p>"
)
enc = tok(text, return_tensors="pt", return_offsets_mapping=True)
logits = model(**{k: v for k, v in enc.items()
                  if k != "offset_mapping"}).logits
labels = [model.config.id2label[i]
          for i in logits.argmax(-1)[0].tolist()]

Decode BIO runs over the offset mapping to recover character spans. For best results segment the input at case boundaries (one case per sequence, ≤8k tokens) and strip footnote content — the model was trained that way.

Limitations

• Trained on U.S. reporter typography; other layout traditions (official state reporters, foreign case law) are untested.
• disposition is the hardest class (two-tier convention: a final short ruling plus in-text holdings; 0.77 on test), and judges degrades on unfamiliar concur-line formats (0.58 on out-of-sample reporters); otherdate has thin support.
• Citations are deliberately NOT a class — they are handled by a separate model.
• Input must follow the canonical serialization above; markup tokens are label-masked during training, so unexpected markup degrades output.