nusaiba19/xlmr-xfact-binary

XLM-R Binary - Multilingual Misinformation Detection

Fine-tuned xlm-roberta-base on X-Fact for binary veracity classification (true-ish vs. false-ish).

Model

• Base: xlm-roberta-base (278M params)
• Task: Binary classification
• Languages: 13 (multilingual)

Training Data

X-Fact all_languages - 19,079 train / 2,535 dev / 3,826 test

Training Setup

• LR: 2e-5, Batch: 16, Max len: 128, Epochs: 5, fp16
• Optimizer: AdamW, Warmup: 0.1

Results (macro-F1 / accuracy)

Test split

Language	Macro-F1	Accuracy
Spanish	0.6718	0.7441
Hindi	0.4654	0.8706
Portuguese	0.6636	0.6637

OOD split

Language	Macro-F1	Accuracy
Hindi	0.3327	0.4987
Portuguese	0.6203	0.6504

Per-class F1 (test)

Language	true-ish	false-ish
Spanish	0.8258	0.5179
Hindi	0.0000	0.9308
Portuguese	0.6582	0.6690

Key Finding

Class imbalance in low-resource languages masks model failure. Hindi achieves 87.7% accuracy but true-ish F1 = 0.000 - the model defaults entirely to majority class. Macro-F1 and per-class F1 are essential for multilingual disinformation detection.

Usage

Install transformers and load the model:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained('nusaiba19/xlmr-xfact-binary')
model = AutoModelForSequenceClassification.from_pretrained('nusaiba19/xlmr-xfact-binary')

text = 'Your claim here'
inputs = tokenizer(text, return_tensors='pt', padding=True, truncation=True)
pred = model(**inputs).logits.argmax(-1).item()
print(['true-ish', 'false-ish'][pred])

References

• X-Fact: Gupta & Srikumar (2021), arXiv:2010.01262
• XLM-R: Conneau et al. (2020), arXiv:1911.02116