wagesj45/toxic-comment-classifier

Toxic Comment Classifier

This is a binary text-classification model for identifying whether an English comment is toxic or not_toxic. It is a fine-tuned version of distilbert/distilbert-base-multilingual-cased.

Intended use

This model is intended as a research baseline or as one signal in a broader content-moderation workflow. Predictions should be reviewed in context and should not be used as the sole basis for high-impact decisions about people.

Usage

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="wagesj45/toxic-comment-classifier",
)

print(classifier("Thanks for sharing this helpful explanation."))

The output label is either not_toxic or toxic, together with a confidence score. The score is not a calibrated probability; applications should choose and validate their own decision threshold.

Training

The model was trained with transformers on the Heliosoph/Jigsaw-Toxic-Comments dataset. The six source labels (toxic, severe_toxic, obscene, threat, insult, and identity_hate) were combined into one binary label: a comment was assigned toxic when any source label was positive.

Training used a deterministic, stratified 90/10 train/validation split with seed 42, three epochs, a learning rate of 2e-5, batch sizes of 16/32, weight decay of 0.01, and a maximum tokenized length of 256. The exported model is the best validation checkpoint selected by F1, from training step 17,952.

Evaluation

On the held-out validation split:

Metric	Score
Accuracy	0.9672
F1	0.8332

These results come from the training split and evaluation procedure described above. They should not be interpreted as performance on arbitrary production comments or on languages outside the evaluation data.

Limitations and risks

• The training data contains noisy, subjective toxicity annotations and may reflect historical or cultural biases.
• The model can produce false positives for profanity, reclaimed language, discussions of abuse, identity terms, or strongly worded criticism.
• It can miss implicit, coded, contextual, or adversarially written abuse.
• The model was trained and evaluated on English comments; multilingual base model support does not make this fine-tune multilingual.
• Thresholds, appeals, human review, and monitoring are recommended for real moderation systems.

Licensing and provenance

The fine-tuned model artifacts in this repository are released under the Apache License 2.0. The base model is also listed under Apache 2.0. The training dataset is listed as CC0-1.0 by its Hugging Face repository. The dataset card credits Jigsaw and Wikipedia as the underlying sources; see the dataset card for details.