mmendoza/robertuito-cl2-multigroup

RoBERTuito + CL2 — multi-label hate speech and target-group detection (Chilean Spanish)

A multi-label classifier for Chilean Spanish social-media text. For each message it predicts, independently, five binary targets:

Target	Meaning
hate	the message is hate speech
women	it references women
lgbtq	it references the LGBTQ+ community
immigrants	it references immigrant communities
indigenous	it references Indigenous (Native American) peoples

So it detects both whether a message is hateful and which protected group(s) it targets or mentions. It is RoBERTuito fine-tuned on the CL2 corpus, using both its hate/non-hate label and its target-group annotations.

This is a companion to mmendoza/robertuito-cl2-hate-speech (binary hate/non-hate). Use the binary model if you only need hate detection; use this one if you also need the targeted group. The group layer is a secondary annotation in CL2 and is not part of the main results in the paper.

Usage

The model requires the same normalisation used at training (mentions → @usuario, links → url, lowercase). Outputs are independent sigmoids (multi-label), not a softmax.

import re, torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

_user = re.compile(r"@\w+"); _url = re.compile(r"https?://\S+|www\.\S+")
def preprocess(t):
    return _url.sub("url", _user.sub("@usuario", str(t))).lower().strip()

name = "mmendoza/robertuito-cl2-multigroup"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name).eval()
labels = [model.config.id2label[i] for i in range(model.config.num_labels)]

text = "estos inmigrantes de mierda que se vayan de mi pais"
enc = tok(preprocess(text), return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
    probs = torch.sigmoid(model(**enc).logits)[0]
for lab, p in zip(labels, probs):
    flag = "  <-- active" if p >= 0.5 else ""
    print(f"{lab:11s} {p:.3f}{flag}")

Label order: hate, women, lgbtq, immigrants, indigenous. Each is an independent probability; threshold at 0.5 (tune per target if needed).

Training data

CL2 — 4,547 Chilean Spanish tweets, 45.6 % hate, annotated by three independent annotators (majority vote) for a hate/non-hate label and for four target groups. Group prevalence (majority vote): women 17.2 %, immigrants 14.5 %, Indigenous 12.8 %, LGBTQ+ 10.8 %. A tweet can reference more than one group. Corpus: https://zenodo.org/records/14619078

Training procedure

Fine-tuned with a multi-label head (problem_type="multi_label_classification", BCEWithLogitsLoss) on the full CL2 corpus:

Hyperparameter	Value
Base model	pysentimiento/robertuito-base-uncased
Epochs	5
Batch size	32
Learning rate	5e-5
Max sequence length	128
Weight decay	0.01
Warmup ratio	0.1
Precision	fp16
Seed	42

Evaluation

Held-out 20 % of CL2 (stratified on the hate label, n = 910), threshold 0.5:

Target	Precision	Recall	F1	AUC
hate	0.846	0.884	0.865	0.935
women	0.741	0.818	0.778	0.960
lgbtq	0.739	0.791	0.764	0.970
immigrants	0.878	0.915	0.896	0.994
indigenous	0.946	0.911	0.928	0.991
macro-F1			0.846
micro-F1			0.853

Group detection is even stronger than hate detection (group references are more lexical/topical), and adding the group targets does not degrade the hate output (F1 0.865 vs 0.854 for the binary-only model).

(The released weights are trained on the full CL2 corpus for deployment; the figures above come from a held-out split.)

Limitations and biases

• Scope is Chilean Spanish and the four groups annotated in CL2 (women, immigrants, Indigenous peoples, LGBTQ+). Other targets and varieties are out of scope.
• The group layer is a secondary annotation; per-group performance depends on each group's prevalence and is lower for rarer groups.
• Trained on keyword/hashtag/account-sampled data, which over-represents explicit hate; the model can err on colloquial profanity and on non-hateful mentions of a group. Use human review for consequential decisions.
• The model detects the group referenced in a message, which is not always the group being attacked; read the hate and group outputs together.

Citation

@article{benoit_hate_chilean_spanish,
  title   = {Hate speech detection in Chilean Spanish and its cross-lingual transferability},
  author  = {Benoit, Domingo and {\~N}anculef, Ricardo and Mendoza, Marcelo},
  journal = {International Journal of Data Science and Analytics (under review)},
  year    = {2026}
}

Please also cite the base model (Pérez et al., RoBERTuito, LREC 2022) and the CL2 corpus (Zenodo 14619078).