xlm-roberta-xstance

A multilingual stance detection model fine-tuned from FacebookAI/xlm-roberta-base on the ZurichNLP/x_stance dataset.

The model predicts whether a political comment expresses a FAVOR or AGAINST stance toward a given political question. It supports multilingual inference and demonstrates strong cross-lingual transfer across Swiss national languages.

---

Highlights

• 🌍 Multilingual stance detection (🇩🇪 German (75%), 🇫🇷 French (25%), and 🇮🇹 Italian (only a few to test zero-shot cross-lingual transfer))
• ⚡ Built on XLM-RoBERTa
• 🎯 Binary stance classification (FAVOR / AGAINST)
• 🔄 Cross-lingual transfer capabilities

---

Performance

Evaluation on the official validation split:

Metric	Score
Loss	0.5225
Accuracy	76.87%
Macro F1	76.87%

---

Quick Start

Installation

pip install transformers torch

Run inference

Using the pipeline API (Recommended)

from transformers import pipeline

classifier = pipeline(
    task="text-classification",
    model="MatteoFasulo/xlm-roberta-xstance"
)

question = "Soll der Bundesrat ein Freihandelsabkommen mit den USA anstreben?"

comment = "Nicht unter einem Präsidenten, welcher die Rechte anderer mit Füssen tritt und Respektlos gegenüber ändern ist."

result = classifier(
    {
        "text": question,
        "text_pair": comment,
    }
)

print(result)

Example output:

[{'label': 'AGAINST', 'score': 0.9823}]

For sequence-pair classification tasks such as stance detection, the text-classification pipeline accepts a dictionary with "text" and "text_pair" keys.

---

Using AutoModelForSequenceClassification

import torch
from transformers import (
    AutoTokenizer,
    AutoModelForSequenceClassification,
)

model_name = "MatteoFasulo/xlm-roberta-xstance"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

question = "Soll der Bundesrat ein Freihandelsabkommen mit den USA anstreben?"

comment = "Nicht unter einem Präsidenten, welcher die Rechte anderer mit Füssen tritt und Respektlos gegenüber ändern ist."

inputs = tokenizer(
    question,
    comment,
    return_tensors="pt",
    truncation=True,
)

with torch.no_grad():
    outputs = model(**inputs)

probabilities = torch.softmax(outputs.logits, dim=-1)

prediction = probabilities.argmax(dim=-1).item()

id2label = model.config.id2label

print("Prediction:", id2label[prediction])
print("Confidence:", probabilities[0, prediction].item())

Example output:

Prediction: AGAINST
Confidence: 0.9823

---

Input Format

The model expects two text sequences:

1. Target political question
2. Candidate comment

Example:

Question:
Should Switzerland increase renewable energy subsidies?

Comment:
Investing in renewable energy will reduce emissions and improve energy independence.

The tokenizer automatically formats these as sentence pairs for XLM-RoBERTa.

---

Output Labels

The classifier predicts one of two classes.

Label	Description
FAVOR	The comment supports the target question.
AGAINST	The comment opposes the target question.

The model outputs logits for both classes.

---

Model Description

This model is a fine-tuned version of FacebookAI/xlm-roberta-base trained for multilingual, multi-target stance detection.

Unlike sentiment analysis, stance detection predicts whether a text supports or opposes a specific target question.

Because the underlying encoder is multilingual, the model can transfer knowledge across languages and perform inference on languages that were only partially represented during training.

---

Intended Uses

The model is suitable for:

• Political stance detection
• Cross-lingual stance classification
• Research on multilingual NLP
• Opinion mining
• Benchmarking stance detection methods

---

Out-of-Scope Uses

This model is not intended for:

• Fact checking
• Political affiliation prediction
• Hate speech detection
• Toxicity classification
• General sentiment analysis
• Automated political decision-making

---

Training Dataset

Training was performed using the ZurichNLP/x_stance dataset.

Dataset characteristics:

• 150+ political questions
• 67,000 candidate comments
• Swiss political debates
• Multilingual annotations

Languages:

• German (majority)
• French
• Italian

Each sample consists of:

(Target Question, Candidate Comment)
→ FAVOR / AGAINST

---

Training Procedure

Hyperparameters

Parameter	Value
Base model	FacebookAI/xlm-roberta-base
Learning rate	2e-5
Batch size	16
Epochs	3
Optimizer	AdamW (Torch Fused)
Scheduler	Linear
Warmup steps	850
Mixed precision	Native AMP
Seed	42

---

Training Results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Macro F1
0.5537	1	2853	0.5749	0.7175	0.7175
0.4712	2	5706	0.4957	0.7588	0.7587
0.3804	3	8559	0.5225	0.7687	0.7687

---

Limitations

Although the model performs well on multilingual political stance detection, several limitations should be considered.

• Trained primarily on Swiss political debates.
• Binary labels only (no Neutral class).
• Performance outside politics has not been evaluated.
• Implicit or sarcastic opinions remain challenging.
• Domain shift may reduce performance on social media or informal discussions.

---

Ethical Considerations

This model predicts stance, not factual correctness.

Predictions should not be interpreted as:

• political affiliation
• truthfulness
• misinformation detection
• ideological profiling

Human oversight is recommended for any downstream application.

---

Framework Versions

• Transformers 5.12.1
• PyTorch 2.8.0
• Datasets 5.0.0
• Tokenizers 0.22.2

---

Citation

If you use this model, please cite the original X-Stance dataset.

@inproceedings{vamvas2020xstance,
    author    = "Vamvas, Jannis and Sennrich, Rico",
    title     = "{X-Stance}: A Multilingual Multi-Target Dataset for Stance Detection",
    booktitle = "Proceedings of the 5th Swiss Text Analytics Conference (SwissText) \& 16th Conference on Natural Language Processing (KONVENS)",
    address   = "Zurich, Switzerland",
    year      = "2020",
    month     = "jun",
    url       = "http://ceur-ws.org/Vol-2624/paper9.pdf"
}

---

License

This model is released under the MIT License.

Please also respect the licenses of:

• FacebookAI/xlm-roberta-base
• ZurichNLP/x_stance

---

Acknowledgements

This model builds upon:

• Facebook AI Research for XLM-RoBERTa
• Zurich NLP Group for the X-Stance dataset
• Hugging Face Transformers