mmendoza/robertuito-cl2-hate-speech

RoBERTuito fine-tuned on CL2 — Hate speech detection in Chilean Spanish

This is a binary hate-speech detector for Chilean Spanish social-media text. It is RoBERTuito (a RoBERTa model pre-trained on 500M Spanish tweets) fine-tuned on CL2, a balanced Chilean Spanish hate-speech corpus. The model outputs hate or non-hate.

It is the deployment model recommended in the paper Hate speech detection in Chilean Spanish and its cross-lingual transferability, where CL2 is shown to be the most effective training resource for Chilean Spanish and RoBERTuito the most competitive model on the Chilean target.

Intended use

• Detecting hate speech directed at protected groups (women, immigrants, Indigenous peoples, and the LGBTQ+ community) in Chilean Spanish tweets and short social-media messages.
• Best suited to balanced or pre-filtered inputs (e.g. moderation queues). On a raw, low-prevalence stream, tune the decision threshold and see the paper for the trade-offs.

Usage

The model requires the same text normalisation used at training time (mentions → @usuario, links → url, lowercase). Skipping it degrades results.

import re, torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

_user = re.compile(r"@\w+"); _url = re.compile(r"https?://\S+|www\.\S+")
def preprocess(t):
    return _url.sub("url", _user.sub("@usuario", str(t))).lower().strip()

name = "mmendoza/robertuito-cl2-hate-speech"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name).eval()

text = "no los soporto, que se vayan de mi pais"
enc = tok(preprocess(text), return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
    prob = torch.softmax(model(**enc).logits, -1)[0]
print(model.config.id2label[int(prob.argmax())], f"p(hate)={prob[1]:.3f}")

Labels: {0: "non-hate", 1: "hate"}.

Training data

CL2 — a Chilean Spanish hate-speech corpus of 4,547 tweets collected from the Chilean X (formerly Twitter) network (Jan 2020–Jun 2022), 45.6 % labelled as hate, annotated by three independent annotators (majority vote) within the full conversation thread. It deliberately covers four protected groups: women, immigrants, Native Americans (Indigenous peoples), and the LGBTQ+ community. Available on Zenodo: https://zenodo.org/records/14619078

Training procedure

Fine-tuned on the full CL2 corpus with:

Hyperparameter	Value
Base model	pysentimiento/robertuito-base-uncased
Epochs	5
Batch size	32
Learning rate	5e-5
Max sequence length	128
Weight decay	0.01
Warmup ratio	0.1
Precision	fp16
Seed	42

Evaluation

In-domain (held-out 20 % of CL2, stratified, n = 910):

Accuracy	Precision	Recall	F1	AUC
0.864	0.833	0.877	0.854	0.939

(These figures come from a held-out split; the released weights are trained on the full CL2 corpus for deployment.)

Functional tests (Multilingual HateCheck, Spanish, 27 functionalities): the CL2-trained model reaches the best average accuracy among the compared corpora (0.74), matching the XLM-T reference and surpassing or tying it on 14 of the 27 functionalities. It exhibits low unintended bias across seven demographic groups. See the paper for full transferability, functional, and bias results.

Limitations and biases

• Scope is Chilean Spanish. The model encodes Chilean localisms and slang; it is not intended as a general or pan-Hispanic hate-speech detector, and transfers less well to other varieties and to English.
• Trained on keyword/hashtag/account-sampled data, which over-represents explicit hate; implicit or subtly framed hate is harder to detect.
• The corpus reflects the annotators' local judgement and the four targeted groups; other targets are out of scope.
• Predictions can err on profanity-heavy but non-hateful colloquial speech and on non-hateful references to protected groups. Use with human review for consequential decisions.

Citation

@article{benoit_hate_chilean_spanish,
  title   = {Hate speech detection in Chilean Spanish and its cross-lingual transferability},
  author  = {Benoit, Domingo and {\~N}anculef, Ricardo and Mendoza, Marcelo},
  journal = {International Journal of Data Science and Analytics (under review)},
  year    = {2026}
}

Please also cite the base model (Pérez et al., RoBERTuito, LREC 2022) and the CL2 corpus (Zenodo 14619078).