eu-acquis-digital-policy

Does an EU legislative provision constitute digital policy? (digital / not_digital).

Fine-tuned ModernBERT-large encoder from the paper Mapping Europe's Digital Acquis (Seidl & Kosti, 2026). The model classifies individual provisions of EU regulations, directives, and decisions — recitals and (sub-)articles as extracted with eurlex-builder — and was trained on labels produced by a five-model LLM ensemble whose coding instructions were validated against two human coders.

Labels

not_digital, digital. Applies to: all provisions (recitals and sub-articles).

Training

• Base model: answerdotai/ModernBERT-large (ModernBertForSequenceClassification); max length at training 1024 tokens.
• Training data: 17,315 provisions labelled by the LLM ensemble (majority vote of Qwen3-235B, Llama-3.3-70B, gpt-oss-120b, GPT-5.2, DeepSeek-V4 Pro); soft labels (per-class vote fractions); negative:positive ratio 4:1.
• Hyperparameters: learning rate 2e-05, weight decay 0.1, 5 epochs, batch size 4 × 8 accumulation.
• Saved 2026-06-08.

Evaluation

• Held-out distillation test (agreement with the ensemble's labels): F1 = 0.954.
• Against the human-coded validation set (n = 500, training items excluded by id): agreement 0.962, Krippendorff's α = 0.902.
• Same, additionally excluding validation items whose exact text recurs in the training pool (n = 461): α = 0.899.
• For reference: the LLM ensemble reaches α = 0.899 against the same coder; the two human coders agree at α = 0.910.

Full validation design, error rates at natural prevalence, temporal stability, prompt stability, and interpretability checks are in the paper's online appendix.

Intended use and limits

• Input: one provision of EU secondary legislation in English (the text field as produced by eurlex-builder). The model was trained on provisions of 50 or more characters; very long provisions were truncated at 1,024 tokens during training.
• Scope: all provisions (recitals and sub-articles). Applying it to other document types (communications, national law, case law) is untested.
• Classification error is documented but not zero; for aggregate analyses, see the paper's discussion of error rates and their correction.

Usage

from transformers import pipeline
clf = pipeline("text-classification", model="tseidl/eu-acquis-digital-policy", truncation=True, max_length=1024)
clf("Member States shall ensure that all consumers have access to an adequate broadband internet access service at an affordable price.")

Citation

Seidl, T. & Kosti, N. (2026). Mapping Europe's Digital Acquis: A Granular History of EU Digital Policymaking. Preprint.

Corpus construction: eurlex-builder (https://github.com/tseidl/eurlex-builder; archived at https://doi.org/10.5281/zenodo.21496963).

Model card generated 2026-09-16 from the training metadata and the paper's validation results file.