metadata
license: bigscience-bloom-rail-1.0
datasets:
- jigsaw_toxicity_pred
language:
- en
metrics:
- accuracy
- f1
library_name: transformers
pipeline_tag: text-classification
Model description
This model is a fine-tuned version of the bert-base-uncased model to classify toxic comments.
The BERT model is finetuned using adversarial training to boost robustness against textual adversarial attacks.
How to use
You can use the model with the following code.
from transformers import BertForSequenceClassification, BertTokenizer, TextClassificationPipeline
model_path = "JiaqiLee/robust-bert-jigsaw"
tokenizer = BertTokenizer.from_pretrained(model_path)
model = BertForSequenceClassification.from_pretrained(model_path, num_labels=2)
pipeline = TextClassificationPipeline(model=model, tokenizer=tokenizer)
print(pipeline("You're a fucking nerd."))
Training data
The training data comes from this Kaggle competition. We use 90% of the train.csv
data to train the model.
We augment original training data with adversarial examples generated by PWWS, TextBugger and TextFooler.
Evaluation results
The model achieves 0.95 AUC in a 1500 rows held-out test set.