ERCDiDip/charter-diplomatic-segmentation

segmenter-tiny — diplomatics structure segmenter (distilled)

A compact, knowledge-distilled token-classification model that segments the diplomatic (structural) parts of medieval Latin / Middle High German charters: INVOCATIO, INTITULATIO, PUBLICATIO, NARRATIO, DISPOSITIO, DATATIO, SANCTIO, CORROBORATIO, ARENGA, APPRECATIO, SUBSCRIPTIO.

It is a BertForTokenClassification head over Multilingual-MiniLM-L12-H384 (12 layers, hidden 384, ~250k vocab ≈ 117M params), distilled from a larger XLM-R-base teacher. At roughly 40% of the teacher's parameters it reaches comparable quantitative quality, making it cheaper for weakly-supervised corpus-scale charter processing.

Model description

• Teacher: XLM-R-base token-classifier trained weakly-supervised on the didip charter corpus (macro_segmenter_best; ~278M params) — the supervising pseudo-labels come from a larger macro_segmenter_best XLM-R labeler, hence weakly-supervised (no gold truth is used to supervise the teacher).
• Student: microsoft/Multilingual-MiniLM-L12-H384 (117M params) + linear token-classification head with the same 22-label BIO scheme (no O class).
• Distillation: token-level knowledge distillation with a T² (T=2.0) scaled KL against the teacher's soft logits, plus a hard pseudo-CE against the teacher argmax labels, plus a class-weighted gold-CE over didip gold-reconstruction spans. Class weighting (inverse-sqrt of token frequency) compensates the extreme class imbalance (e.g. B-NARRATIO ≈ 293 tokens vs I-DISPOSITIO ≈ 329k tokens ≈ 1100×).

Architecture check (config): the checkpoint ships as a standard BertForTokenClassification (model_type: bert, max position 512), so it loads with plain AutoModelForTokenClassification — no custom/remote code required.

Label scheme

22 labels = 11 sections × {B-, I-}, no O/outside class (every token is a part of some section):

B/I-APPRECATIO   B/I-ARENGA       B/I-CORROBORATIO  B/I-DATATIO
B/I-DISPOSITIO   B/I-INTITULATIO  B/I-INVOCATIO     B/I-NARRATIO
B/I-PUBLICATIO   B/I-SANCTIO      B/I-SUBSCRIPTIO

The config id2label/label2id is alphabetical, so index order differs from the diplomatics order shown above. Always read labels from the checkpoint config — never hard-code the ordering. The B-/I- prefixes are stripped at decode time and the BIO ordering is used only for phrase-continuation.

Uses

Charter-structure reconstruction for early- and high-medieval documentary sources, e.g. surveying which diplomatics section a phrase belongs to, aligning NARRATIO/DISPOSITIO/DATATIO boundaries across a corpus, or as a feature source for higher-level weakly-supervised models. Primary intended input: a single charter text (Latin or German), pre-tokenized at rough word level.

Example (windowed inference + label decode)

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

MODEL = "ERCDiDip/charter-diplomatic-segmentation"
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForTokenClassification.from_pretrained(MODEL)

def predict_window(words):
    enc = tok(words, is_split_into_words=True, return_tensors="pt")
    with torch.no_grad():
        logits = model(**enc).logits[0]
    ids = enc.word_ids()
    labels, conf = [], []
    for w_ix in range(len(words)):
        idxs = [i for i, wid in enumerate(ids) if wid == w_ix]
        if not idxs:
            continue                       # special tokens skipped
        ps = logits[idxs].softmax(-1)
        votes = ps.sum(0)
        labels.append(int(votes.argmax()))
        conf.append(float(votes.max()))
    return labels, conf

# usage on one ~<500-token charter window; for longer texts slide a 512 window
# and stitch, then map indices -> section names via the stripped label.

At decode time: aggregate a predicted label per word (from its subwords, weighted by softmax confidence), strip the B-/I- prefix into section names, collapse consecutive runs into spans, and merge spans shorter than 5 words into the previous span (removes fragment noise, mirrors the reference inference_macro.py decode). The pipeline strips no vocabulary — it expects word-tokenized input and handles subword boundaries internally.

Bias, Risks, and Limitations

Read this before using the model on real charter material.

• German NARRATIO — documented weak point. On Middle High German charters the model tends to merge NARRATIO into DISPOSITIO. This is not a quantifiable regression versus the teacher (see Evaluation — the distilled model matches or beats the teacher numerically); it is an intrinsic capacity/signal limit observed across every distillation recipe we tried: the teacher's own token-level NARRATIO signal is weak and alternating in that region, and the 117M student cannot reproduce it. If precise German NARRATIO boundaries matter, prefer the teacher or a larger student.
• Weak supervision. Labels inherit the teacher's segmentation; errors in the teacher propagate. No gold truth is available for medieval diplomatics, so reported metrics are against a proxy reconstruction — treat absolute numbers as indicative, not authoritative.
• Annotation-convention conflict. The didip and Lambach gold reconstructions disagree on NARRATIO (Lambach annotates NARRATIO+DISPOSITIO as a single span). This checkpoint deliberately follows the didip convention (keeps NARRATIO distinct); models fine-tuned against Lambach gold score higher on a Lambach-split benchmark precisely because they learn the collapsed convention.
• Language scope. Trained on medieval Latin and German; behavior on other languages is untested and likely poor.
• Length. Context window 512 tokens. Longer charters must be windowed and stitched — boundary effects are possible at window seams.
• License: CC-BY-NC-4.0 (non-commercial). The checkpoint is derived from teacher + gold reconstruction assets inside a private research repo (didip/ structural_annotation, diplomatic_ssl). It is released under a non-commercial Creative Commons license (cc-by-nc-4.0); commercial use and redistribution are not permitted.

Training Details

• Teacher: XLM-R-base, token-classification, trained on the didip corpus (20 epochs), weakly-supervised from a larger macro_segmenter_best labeler.
• Distillation recipe (v4):
  • Student: microsoft/Multilingual-MiniLM-L12-H384 (117M) + linear head.
  • KD: T²·KL(softmax(teacher_logits/T) ∥ softmax(student_logits/T)), T=2.0, weighted per-token by the inverse-sqrt frequency of the teacher's argmax class (--kd-class-weight-sqrt).
  • Hard CE vs teacher argmax pseudo-labels.
  • Gold CE over didip reconstruction spans, class-weighted (--ce-class-weight).
  • Lambach gold excluded from the gold-CE (--gold-lambach off by default) so the student keeps the didip NARRATIO/DISPOSITIO distinction instead of learning Lambach's merged-NARRATIO convention.
  • Optimizer: AdamW (lr 5e-5) with a OneCycleLR schedule (6% warmup phase), fp32 training on a single RTX A5000 24G.
• Lineage / prior versions: v1 collapsed to a single I-DISPOSITIO blob; v2/v3 (and later experiment v5–v7) focus on different class-weighting/loss-balance/kd-scale choices. v4 (this checkpoint) is the chosen balance: best of the improved runs and keeps Latin structuring clean.

Evaluation

Weakly-supervised benchmark on a Lambach charter hold-out split (94 docs). Two models, identical decode pipeline (word-level aggregation + confidence weighting + <5-word span smoothing):

Model	Params	token_acc	token_macro-F1	span IoU@0.5 F1
XLM-R-base teacher (didip macro_segmenter_best)	278M	0.8097	0.3529	0.7547
segmenter-tiny v4 (this checkpoint)	117M	0.8157	0.3606	0.7682

The distilled student matches or slightly beats the teacher on all three metrics at ~41% of the parameter count, on this proxy benchmark. (Note: the Lambach-split metric rewards the collapsed-NARRATIO convention; see Limitations. The didip-aligned raw reconstruction quality in Latin is qualitatively clean — fully structured INVOCATIO → INTITULATIO → PUBLICATIO → NARRATIO → DISPOSITIO → … → DATATIO.)

Confusion matrix

The table below is the token-level confusion matrix for this checkpoint on a full-scheme benchmark. Rows are gold sections, columns predicted sections; each cell is the number of tokens whose gold section was the row's and whose prediction was the column's. B-/I- prefixes are stripped. Computed on 107 charters, didip annotation convention, all 11 sections present, in-window tokens only, 512-subword truncation tail excluded.

Why not the 94-doc Lambach split used for the headline numbers above? That gold collapses NARRATIO into DISPOSITIO and annotates only five sections, so it cannot show the section-pair confusions below. The headline token_acc/macro-F1/span-IoU table above is unchanged from the Lambach benchmark.

gold \ pred	INVOCATIO	INTITULATIO	ARENGA	PUBLICATIO	NARRATIO	DISPOSITIO	CORROBORATIO	SUBSCRIPTIO	DATATIO	APPRECATIO	SANCTIO	OTHER	recall
INVOCATIO	183	0	0	0	0	0	0	0	0	0	0	0	1.000
INTITULATIO	0	1,599	0	14	2	3	0	0	0	0	0	0	0.988
ARENGA	0	0	561	9	0	3	0	0	0	0	0	0	0.979
PUBLICATIO	0	14	0	1,599	0	1	0	0	0	0	0	0	0.991
NARRATIO	0	2	9	3	2,531	681	2	3	15	0	0	0	0.780
DISPOSITIO	0	0	17	24	564	13,258	144	0	36	0	0	0	0.944
CORROBORATIO	0	0	0	0	0	0	1,533	0	0	0	0	0	1.000
SUBSCRIPTIO	0	0	0	0	0	0	0	0	0	0	0	0	0.000
DATATIO	0	0	0	0	0	0	12	0	1,308	0	0	0	0.991
APPRECATIO	0	0	0	0	0	0	0	0	0	42	0	0	1.000
SANCTIO	0	0	9	0	0	0	11	0	0	6	0	0	0.000
OTHER	0	0	0	0	0	0	0	0	0	0	0	0	0.000

Per-section precision / recall / F1 and gold token counts:

section	precision	recall	F1	gold tokens
INVOCATIO	1.000	1.000	1.000	183
INTITULATIO	0.990	0.988	0.989	1,618
ARENGA	0.941	0.979	0.960	573
PUBLICATIO	0.970	0.991	0.980	1,614
NARRATIO	0.817	0.780	0.798	3,246
DISPOSITIO	0.951	0.944	0.947	14,043
CORROBORATIO	0.901	1.000	0.948	1,533
SUBSCRIPTIO	0.000	0.000	0.000	0
DATATIO	0.963	0.991	0.977	1,320
APPRECATIO	0.875	1.000	0.933	42
SANCTIO	0.000	0.000	0.000	26
OTHER	0.000	0.000	0.000	0

How to read it. The distilled student keeps the teacher's NARRATIO↔DISPOSITIO weakness but slightly attenuates it on full-scheme gold (DISPOSITIO → NARRATIO 564 vs the teacher's 1,357; NARRATIO F1 0.798 vs 0.660) — at 41% of the teacher's parameters. The residual DISPOSITIO → NARRATIO (564) and NARRATIO → DISPOSITIO (681) cells are the documented NARRATIO weak point, and on the Lambach benchmark these collapses are rewarded because Lambach merges the two sections. SANCTIO is essentially not recovered by the student (0 recall, 26 gold tokens); SUBSCRIPTIO/OTHER rows are empty because the gold has none.

Provenance

• model.safetensors sha256: 5ba28bb846da0ed8a173cc3790677db839b60ea8e85ff761a69bab077fc2fecb
• Teacher: didip/structural_annotation/macro_segmenter_best (private repo).
• Distillation/eval code: distill_tiny_segmenter.py, eval_mixed_segmenter.py, inference_tiny_test.py (private repo).
• Tokenizer: microsoft/Multilingual-MiniLM-L12-H384 SentencePiece tokenizer (vocab 250,037) shipped with this checkpoint.

Model Card Contact

Contact the repo owner for provenance/licensing questions.

Citation

@misc{charter-diplomatic-segmentation,
  title={Distilled Weakly-Supervised Multi-Head Segmentation of Medieval Charters},
  author={Kovács, Tamás; Nicolaou, Anguelos; Atzenhofer-Baumgartner, Florian; Renet, Nicolas; Consolo, Giuseppe, Tscherne Niklas; Decker, Franziska; and Vogeler, Georg},
  year         = { 2026 },
  url          = { https://huggingface.co/ERCDiDip/charter-diplomatic-segmentation },
  doi          = { 10.57967/hf/10291 },
  publisher    = { Hugging Face }
}