LuisaLuligo/sexism-binary-xlm-roberta

Sexism Detection Models for EXIST2025

This repository provides two fine-tuned transformer models for the EXIST2025 sexism detection challenge:

• Binary Sexism Detection (sexism-binary-xlm-roberta)

Both models are based on xlm-roberta-large and are ready for direct use with Hugging Face Transformers.

---

How to Use

1. Binary Sexism Detection

Task: Classify a tweet as sexist (YES) or not sexist (NO).

Load and use:

from transformers import AutoTokenizer
from model_class import SoftBinaryClassifier  # See below for class definition
import torch

model_path = "LuisaLuligo/sexism-binary-xlm-roberta"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = SoftBinaryClassifier(model_name=model_path)
model.eval()

text = "Example tweet here"
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding="max_length", max_length=128)
with torch.no_grad():
    outputs = model(**inputs)
    prob = torch.sigmoid(outputs.logits).item()
    label = "YES" if prob > 0.5 else "NO"
print(f"Prediction: {label} (prob: {prob:.3f})")

---

Model Classes

You must use the following classes for inference:

import torch
import torch.nn as nn
from transformers import AutoModel, AutoConfig
from transformers.modeling_outputs import SequenceClassifierOutput

class SoftBinaryClassifier(nn.Module):
    def __init__(self, model_name="xlm-roberta-large", layers_to_freeze=4, dropout=0.3):
        super().__init__()
        self.config = AutoConfig.from_pretrained(model_name)
        self.encoder = AutoModel.from_pretrained(model_name)
        for name, param in self.encoder.named_parameters():
            should_freeze = False
            if 'embeddings' in name:
                should_freeze = True
            elif 'encoder.layer.' in name:
                try:
                    layer_num = int(name.split('encoder.layer.')[1].split('.')[0])
                    if layer_num < layers_to_freeze:
                        should_freeze = True
                except Exception:
                    pass
            param.requires_grad = not should_freeze
        self.dropout = nn.Dropout(dropout)
        self.classifier = nn.Linear(self.config.hidden_size, 1)
        self.to(torch.device("cuda" if torch.cuda.is_available() else "cpu"))

    def forward(self, input_ids, attention_mask=None, **kwargs):
        output = self.encoder(input_ids=input_ids, attention_mask=attention_mask)
        pooled = self.dropout(output.last_hidden_state[:, 0])
        logits = self.classifier(pooled).squeeze(-1)
        return SequenceClassifierOutput(loss=None, logits=logits)

---

Model Card: sexism-binary-xlm-roberta

Architecture: xlm-roberta-large
Task: Binary classification (sexist / not sexist)
Languages: Spanish and English
Training Data: Tweets annotated by 6 experts, soft labels used
Loss: BCEWithLogitsLoss with soft labels
Frozen Layers: Embeddings + first 4 encoder layers
Metrics: F1, accuracy, ROC-AUC, correlation with soft labels
Intended Use: Detects sexist content in tweets
Limitations: May not generalize to other domains or languages; sensitive to annotation bias.

---

Citation

If you use these models, please cite the EXIST2025 challenge