Edit model card

toxicity-score-multi-classification

This model is a fine-tuned version of line-corporation/line-distilbert-base-japanese on a Japanese toxicity dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2649
  • Roc Auc: 0.7992

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 8.133692392125703e-06
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30

Training results

Training Loss Epoch Step Validation Loss Roc Auc
No log 1.0 20 0.6213 0.5148
No log 2.0 40 0.4762 0.4616
No log 3.0 60 0.3754 0.5830
No log 4.0 80 0.3314 0.5706
No log 5.0 100 0.3140 0.5740
No log 6.0 120 0.3067 0.6238
No log 7.0 140 0.3010 0.6645
No log 8.0 160 0.2975 0.7177
No log 9.0 180 0.2949 0.7392
No log 10.0 200 0.2892 0.7731
No log 11.0 220 0.2828 0.7954
No log 12.0 240 0.2827 0.7932
No log 13.0 260 0.2756 0.7984
No log 14.0 280 0.2715 0.8052
No log 15.0 300 0.2733 0.8100
No log 16.0 320 0.2754 0.8142
No log 17.0 340 0.2668 0.8130
No log 18.0 360 0.2642 0.8138
No log 19.0 380 0.2639 0.8117
No log 20.0 400 0.2659 0.8052
No log 21.0 420 0.2646 0.8082
No log 22.0 440 0.2643 0.8039
No log 23.0 460 0.2646 0.8022
No log 24.0 480 0.2644 0.8044
0.2305 25.0 500 0.2639 0.8035
0.2305 26.0 520 0.2639 0.8027
0.2305 27.0 540 0.2647 0.8001
0.2305 28.0 560 0.2643 0.8005
0.2305 29.0 580 0.2649 0.8001
0.2305 30.0 600 0.2649 0.7992

Framework versions

  • Transformers 4.34.0
  • Pytorch 2.0.1
  • Datasets 2.14.5
  • Tokenizers 0.14.0
Downloads last month
50
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for Alnusjaponica/toxicity-score-multi-classification

Finetuned
(19)
this model