contradictor-team/modernbert-framer

ModernBERT Framer

ModernBertForSequenceClassification fine-tuned from answerdotai/ModernBERT-base for 15-class single-label classification of argument text into interpretative frames. Trained on jmLuis/MediaFrameCorpus-PhilippineFrameCorpus-Combined (English).

Labels

Label ids 0–14, mapped positionally during training to MediaFrameCorpus code_frames 1–15. The mapping is stored in model.config.id2label.

Id	Frame
0	Economic
1	Capacity and Resources
2	Morality
3	Fairness and Equality
4	Constitutionality and Legality
5	Policy Prescription and Evaluation
6	Crime and Justice
7	Security and Defense
8	Health and Safety
9	Quality of Life
10	Cultural Identity
11	Public Opinion
12	Political
13	External Regulation and Reputation
14	Other

Quick start

import torch
from transformers import AutoTokenizer, ModernBertForSequenceClassification

model_id = "contradictor-team/modernbert-framer"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ModernBertForSequenceClassification.from_pretrained(model_id)

argument = (
    "The government should build more shelters for the homeless "
    "and teach the public about the real causes of homelessness."
)

inputs = tokenizer(
    argument,
    return_tensors="pt",
    truncation=True,
    max_length=8192,
)
with torch.no_grad():
    logits = model(**inputs).logits
    probs = torch.nn.functional.softmax(logits, dim=-1)[0]

predicted_id = probs.argmax().item()
frame = model.config.id2label[str(predicted_id)]
print(f"Primary frame: {frame}  ({probs[predicted_id]:.3f})")
for frame_name, prob in zip(model.config.id2label.values(), probs.tolist()):
    print(f"  {frame_name}: {prob:.3f}")

Evaluation

Metrics computed on the held-out test split and published as evaluation_metrics.json on the Hugging Face Hub alongside the model weights and training_config.json.

• Dataset: jmLuis/MediaFrameCorpus-PhilippineFrameCorpus-Combined
• Splits: train 10,000 / validation 5,783 / test 5,783
• Base model: answerdotai/ModernBERT-base
• Training: extended-lora-and-head (LoRA r=4, α=8 on Wqkv), 3 epochs, batch size 2, gradient accumulation 8 (effective batch 16), learning rate 2e-5, bfloat16 on GPU, max sequence length 8,192 tokens; LoRA adapters merged into base weights before saving
• Metrics: accuracy, micro-F1, macro-F1, macro precision, macro recall, per-class precision/recall/F1/support, confusion matrix

Test-set results

Metric	Value
Accuracy	0.6706
Micro-F1	0.6706
Macro-F1	0.5912
Macro precision	0.6202
Macro recall	0.5744

Per-class results (support = number of test examples):

Frame	Support	Precision	Recall	F1
Economic	368	0.7000	0.7038	0.7019
Capacity and Resources	115	0.4940	0.3565	0.4141
Morality	172	0.7368	0.5698	0.6426
Fairness and Equality	134	0.5758	0.2836	0.3800
Constitutionality and Legality	983	0.7509	0.8342	0.7904
Policy Prescription and Evaluation	446	0.5306	0.5448	0.5376
Crime and Justice	699	0.6971	0.7639	0.7290
Security and Defense	177	0.5909	0.4407	0.5049
Health and Safety	368	0.7158	0.7120	0.7139
Quality of Life	259	0.5294	0.4517	0.4875
Cultural Identity	288	0.5491	0.5243	0.5364
Public Opinion	286	0.6007	0.5629	0.5812
Political	1136	0.7367	0.7782	0.7568
External Regulation and Reputation	177	0.6802	0.6610	0.6705
Other	175	0.4144	0.4286	0.4213

Full confusion matrix in evaluation_metrics.json on the Hub.