Nerthus-Project/MATILDA_UD_parser_of_Old_English

MATILDA — Old English Universal Dependencies pipeline

A spaCy pipeline for Old English (Anglo-Saxon, c. 650–1150 CE) covering part-of-speech tagging, morphological analysis, lemmatisation and Universal Dependencies parsing.

On a matched evaluation set it improves on the published state of the art (Martín Arista et al., 2025) by +5.57 LAS and +3.42 UAS.

Usage

pip install https://huggingface.co/<org>/en_matilda_oe_ud/resolve/main/en_matilda_oe_ud-1.0.0-py3-none-any.whl

import spacy

nlp = spacy.load("en_matilda_oe_ud")
doc = nlp("Hwæt we Gardena in geardagum þeodcyninga þrym gefrunon .")

for token in doc:
    print(token.text, token.lemma_, token.pos_, token.dep_, token.head.text)

The opening of Beowulf — "Lo, we have heard of the glory of the Spear-Danes in days of old" — produces:

TEXT	LEMMA	UPOS	DEP	HEAD
Hwæt	hwæt	ADV	advmod	gefrunon
we	we	PRON	nsubj	gefrunon
Gardena	Gardenā	ADJ	nsubj	gefrunon
in	in	ADP	case	geardagum
geardagum	geardeg	NOUN	obl:tmod	gefrunon
þeodcyninga	þeodcyning	NOUN	nmod	þrym
þrym	þrym	NOUN	obj	gefrunon
gefrunon	gefrunon	VERB	ROOT	gefrunon
.	.	PUNCT	punct	gefrunon

This is verse and therefore out of domain, since the treebank is prose. The analysis is nonetheless mostly right: gefrunon is correctly the root, we its subject, þrym its object, and þeodcyninga a genitive modifier of þrym. Gardena is the visible error, a genitive plural attached as a second subject and tagged ADJ.

Note also that the input uses thorn ⟨þ⟩ while the training treebank is normalised to eth ⟨ð⟩. The floret subword vectors absorb the difference, which is one of the reasons they were chosen.

Performance

Held-out test set, 561 sentences, decontaminated against the silver training data. All figures are percentages.

Metric	Score
TOK	99.80
XPOS (TAG)	94.73
UPOS (POS)	95.10
FEATS (MORPH)	85.42
LEMMA	86.94
UAS	86.35
LAS	79.43
Sentence F	87.44

Against the published baseline

The reference point is the best configuration of Martín Arista et al. (2025), measured on a matched 285-sentence evaluation set with identical tokenisation:

Metric	BDCC 2025	This model	Δ
XPOS	93.20	94.91	+1.71
UPOS	92.96	95.02	+2.06
FEATS	84.21	87.77	+3.56
UAS	83.24	86.66	+3.42
LAS	74.23	79.80	+5.57
SENT-F	71.38	84.18	+12.80

For wider context, the best cross-lingual result reported for Old English — Brigada Villa and Giarda (2023), Old English paired with Icelandic — reached 68.44 UAS and 58.70 LAS.

Lemma accuracy is excluded from this comparison. The 2025 figure of 79.83 was measured against lemma cells that still carried glosses and word-class markers inherited from the source spreadsheets, so the two references differ and the numbers are not comparable. The normalisation that removed those markers is described below and analysed in the accompanying paper.

Training data

Source	Size	Role
MATILDA gold UD treebank	87,993 tokens	supervised training
MATILDA silver treebanks	212,471 tokens	supervised training, filtered
Dictionary of Old English Corpus	~3.4 M tokens	floret vectors, tok2vec pretraining
Synthetic Old English	1.73 M words	floret vectors, tok2vec pretraining

The gold treebank derives from ParCorOEv3 and from doctoral work in the same project, covering Ælfric's Catholic Homilies, the Anglo-Saxon Chronicle, Anglo-Saxon laws, St Mark's Gospel, the Old English Orosius, Bede, charters, martyrologies and Boethius.

Its LEMMA and XPOS columns were normalised before training. 31.7 % of lemma cells carried a gloss or word-class marker (and 'and' (CONJ) rather than and), and the tag column ran to 71 strings for roughly 25 categories. Because the reference contradicted itself, a model that always guessed the most frequent lemma for each (form, UPOS) pair could not have exceeded 79.83 % accuracy; after normalisation that ceiling is 97.81 %. The same normalisation has been applied to the treebank itself, so the published corpus and this model follow one set of conventions.

The silver portion is machine-annotated and was filtered before use: four rule-completed treebanks and one heavily underspecified treebank were excluded entirely, sentences carrying the dep placeholder were dropped, and 338 sentences overlapping the evaluation sets were removed by 4-gram matching. What makes the remainder valuable is its manually revised morphology; its trees came from an earlier, weaker parser.

The synthetic portion is machine-translated Latin from works attested in pre-1100 Anglo-Saxon libraries. It is used only in the unlabelled channels — vectors and pretraining — and never as supervised annotation.

Architecture

tokenizer → tok2vec → tagger → morphologizer → trainable_lemmatizer → parser

• Vectors: floret subword embeddings, 50,000 rows × 300 dimensions, chosen for a zero out-of-vocabulary rate on a morphologically rich language with unstable spelling. A perfect word-level table built on the entire 3.4 M-token corpus still leaves 28 % of the types in the gold treebank unseen.
• tok2vec: MultiHashEmbed with MaxoutWindowEncoder, pretrained on the Dictionary of Old English Corpus plus the synthetic text, with orthography harmonised to eth before pretraining.
• Parser: TransitionBasedParser (arc-eager).

There is no transformer. The 2025 baseline study found MobileBERT trained from scratch reached only 60.17 UAS and 45.51 LAS on this data, at a parameter-to-token ratio of roughly 1250:1, and transfer from modern English checkpoints is blocked by the graphemes ⟨æ⟩, ⟨ð⟩, ⟨þ⟩, ⟨ȝ⟩ and ⟨ƿ⟩.

Limitations

• The tokenizer is English. spaCy provides none for Old English, so the pipeline's lang is en and tokenisation uses English rules, as in the baseline study. Every trained component is Old English only. This is why the package is named en_matilda_oe_ud while the repository is tagged ang.
• Non-projective structures fail. Crossing dependencies arising from Old English word order cannot be represented by a transition-based parser. This affects relative clauses, clausal modifiers and some coordination.
• Long-distance relations are weak. Local and morphologically marked relations parse well (det:poss 94.06 F1, case 93.38, cc 90.52); clausal complements and oblique variants much less so.
• Genre skew. The treebank is prose — homiletic, legal, historical and biblical. Poetry is under-represented.
• Part of the training data is machine-annotated. If you need a parser whose syntax derives solely from human annotation, use the gold-parser-silver-morphology variant, which costs 1.4 LAS.
• The parser sometimes emits dep. In Universal Dependencies dep means "unclassified", so it carries no information. It appears for two reasons. 490 tokens of the treebank still carry it as an unresolved placeholder, almost all of them sentence-initial annal years and chapter numbers. And spaCy's arc-eager parser declares dep a builtin label and adds it as a backoff whatever the training data contains, so a model trained on data with no dep at all still predicts it. Treat dep in the output as "no relation assigned" rather than as a relation. Roughly 0.5 % of tokens are affected.

Citation

Please cite the baseline study, which this model extends:

@article{martinarista2025parsing,
  title   = {Parsing Old English with Universal Dependencies---The Impacts
             of Model Architectures and Dataset Sizes},
  author  = {Mart{\'i}n Arista, Javier and Ojanguren L{\'o}pez, Ana Elvira
             and Dom{\'i}nguez Barrag{\'a}n, Sara},
  journal = {Big Data and Cognitive Computing},
  volume  = {9},
  number  = {8},
  pages   = {199},
  year    = {2025},
  issn    = {2504-2289},
  doi     = {10.3390/bdcc9080199}
}

Licence and funding

CC BY 4.0. Funded by AEI /10.13039/501100011033, grant PID2023-149762NB-100 MCIN. Nerthus Project, Universidad de La Rioja.