colombian-spanish-cyberbullying-detector

This model is a fine-tuned version of PlanTL-GOB-ES/roberta-base-bne on a dataset created by manually gathering posts from the social network Twitter to detect cyberbullying in Colombian Spanish.

Training and evaluation data

The dataset used consisted of 2566 tweets and maintained a balanced distribution between cyberbullying and not cyberbullying. For every keyword or phrase, there is an annotated tweet labeled as cyberbullying that contains that word or phrase.

The not cyberbullying category predominantly includes tweets that do not contain obscene words and are sourced from popular and varied discussions involving colombian users, reflecting a wide range of topics and conversations.

The distribution of cyberbullying tweets and non-cyberbullying tweets was the same. The keywords and phrases used in the creation of the dataset were selected based on the categories provided in the article Guidelines for the Fine-Grained Analysis of Cyberbullying authored by Cynthia Van Hee, Ben Verhoeven, Els Lefever, Guy De Pauw, Walter Daelemans, and Véronique Hoste. Four categories were included: insult, threat, curse, and defamation. The insult category involves the use of offensive words intended to verbally hurt another person, while threat aims to harm the victim's integrity. Curse includes words that wish harm or misfortune upon a person, and defamation seeks to damage the victim’s reputation. These categories were chosen to capture a broad representation of the forms in which cyberbullying can manifest. The tweets were labeled by an occupational therapist associated with the project.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
weight_decay=0.01
warmup_steps=500
num_epochs: 3

Training results

Epoch	ROC-AUC	Validation Loss	Training Loss
1.0	0.9849	0.2066	---
2.0	0.9946	0.1461	0.3015
3.0	0.9956	0.1453	0.3015

Model in action 🚀

Fast usage with pipelines:

!pip install -q transformers
from transformers import pipeline

model_path = "FelipeGuerra/colombian-spanish-cyberbullying-detector"
bullying_analysis = pipeline("text-classification", model=model_path, tokenizer=model_path)

bullying_analysis(
    "Como dice mi mamá: va caer palo de agua"
    )

# Output:
[{'label': 'Not_bullying', 'score': 0.9986886382102966}]

bullying_analysis(
    "en estos días te caemos y te vamos a dar una golpiza sapo"
    )
# Output:
[{'label': 'Bullying', 'score': 0.9995431900024414}]

Framework versions

Transformers 4.34.0
Pytorch 2.0.1+cu118
Pandas 1.5.3
scikit-learn 1.2.2

Created by Felipe Guerra Sáenz| LinkedIn