Hate Speech Detection (DistilBERT, fine-tuned)
DistilBERT fine-tuned for 3-class hate speech / offensive language detection, trained on the Davidson et al. (2017) hate speech and offensive language dataset (24,783 tweets).
Labels
- 0: Hate Speech
- 1: Offensive Language
- 2: Neither
Performance (held-out test set, n=3718)
- Macro-F1: 0.7663
- Accuracy: 0.8994
Best-performing model out of a 6-model ladder (TF-IDF+LogReg/SVM, BiLSTM, CNN-BiLSTM, BiLSTM+Attention, DistilBERT) -- full comparison and methodology in the source repository.
Bias audit
Tested against a documented failure mode in this dataset family (Sap et al., 2019): over-flagging African-American Vernacular English (AAVE) as offensive. On 20 hand-constructed AAVE/Standard-English sentence pairs with equivalent (benign) meaning, this model showed a 0% false-positive-rate gap between dialects, versus a 20-point gap in a TF-IDF+LogReg baseline tested the same way (see repo for full methodology, including statistical tests).
Known limitations
Qualitative error analysis (see repo, backed by LIME explanations) found the model's decisions are driven heavily by the presence of specific slur tokens as near-decisive lexical triggers, rather than deeper contextual understanding of intent or target. Personal insults using slurs can be escalated to "Hate Speech" when annotators labeled them "Offensive Language," and coded/indirect offensive language without explicit slur vocabulary is sometimes missed entirely.
Intended use
Content-moderation assistance / research and portfolio demonstration. NOT intended as a sole or automatic basis for real moderation decisions against real people. Predictions are probabilistic and can be wrong -- see the full disclaimer and bias-audit results in the source repository before any other use.
- Downloads last month
- -