wagesj45/multilabel-toxic-comment-classifier

Multilabel Toxic Comment Classifier

This model produces seven independent moderation scores for English comments: toxicity, severe_toxicity, obscene, threat, insult, identity_attack, and sexual_explicit. It is a fine-tune of datalama/mmBERT-small, a multilingual ModernBERT/mmBERT base model.

This is a multi-label classifier, not a mutually exclusive class classifier. Every output is a sigmoid score and a comment can score highly on multiple dimensions. toxicity is trained as its own target; it is not computed as an OR of the other labels.

Intended use

Use the scores as one input to a broader content-moderation workflow. Choose and validate a separate decision threshold for every label and review decisions in context. Do not use this model as the sole basis for high-impact decisions about people.

Usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "wagesj45/multilabel-toxic-comment-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).eval()

inputs = tokenizer(
    "Your text to score goes here.",
    return_tensors="pt",
    truncation=True,
    max_length=256,
)
with torch.no_grad():
    scores = torch.sigmoid(model(**inputs).logits[0]).tolist()

result = {
    model.config.id2label.get(index, model.config.id2label.get(str(index))): score
    for index, score in enumerate(scores)
}
print(result)

The model's saved label order is:

1. toxicity
2. severe_toxicity
3. obscene
4. threat
5. insult
6. identity_attack
7. sexual_explicit

Scores are model outputs, not calibrated probabilities. Thresholds should be chosen for the target application and monitored after deployment.

Training

The model was fine-tuned with transformers on google/civil_comments and Heliosoph/Jigsaw-Toxic-Comments. Civil Comments supplies all seven continuous annotation-fraction targets. Jigsaw supplies binary targets for the first six dimensions; its unavailable sexual_explicit label is masked from loss rather than treated as negative.

The combined corpus was shuffled and split deterministically 90/10 with seed 42. Training used three epochs, a learning rate of 2e-5, batch sizes 16/32, weight decay 0.01, and a 256-token input limit. The exported weights are the best checkpoint, selected by validation macro ROC-AUC at epoch 2.

Evaluation

Evaluation used the held-out split described above. Metrics threshold soft annotation targets at 0.5 for F1; ROC-AUC uses the continuous model scores.

Label	ROC-AUC	F1 at 0.5
toxicity	0.9744	0.6992
severe_toxicity	0.9991	0.4085
obscene	0.9936	0.7141
threat	0.9894	0.4847
insult	0.9802	0.6959
identity_attack	0.9873	0.3902
sexual_explicit	0.9967	0.5075
Macro average	0.9887	0.5571

These results are not a measure of performance on arbitrary production comments or languages outside the evaluation data.

Limitations and risks

• The training and evaluation comments are English. The multilingual base model does not establish validated performance for non-English text.
• Toxicity labels are subjective and noisy, and data may carry historical or cultural biases.
• Profanity, reclaimed language, identity terms, quotations, discussions of abuse, and strongly worded criticism can cause false positives.
• The model can miss implicit, coded, contextual, or adversarially written abuse.
• Use application-specific thresholds, appeals, human review, and monitoring for real moderation systems.

Licensing and provenance

The fine-tuned model artifacts in this repository are released under the Apache License 2.0. The listed base model and both listed Hugging Face training datasets use Apache-2.0 and CC0-1.0 licensing, respectively; refer to their dataset cards for source attribution and terms. This repository does not redistribute training rows. Any future hard-negative data must be documented with its source and license before use in a released model.