GItaf's picture
update model card README.md
6592ca3
|
raw
history blame
3.58 kB
metadata
tags:
  - generated_from_trainer
model-index:
  - name: gpt2-gpt2-mc-weight0.25-epoch15-new
    results: []

gpt2-gpt2-mc-weight0.25-epoch15-new

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 4.7276
  • Cls loss: 3.0579
  • Lm loss: 3.9626
  • Cls Accuracy: 0.6110
  • Cls F1: 0.6054
  • Cls Precision: 0.6054
  • Cls Recall: 0.6110
  • Perplexity: 52.59

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 15

Training results

Training Loss Epoch Step Validation Loss Cls loss Lm loss Cls Accuracy Cls F1 Cls Precision Cls Recall Perplexity
4.674 1.0 3470 4.4372 1.5961 4.0380 0.5487 0.5279 0.5643 0.5487 56.71
4.3809 2.0 6940 4.3629 1.4483 4.0006 0.6023 0.5950 0.6174 0.6023 54.63
4.2522 3.0 10410 4.3721 1.5476 3.9849 0.6012 0.5981 0.6186 0.6012 53.78
4.1478 4.0 13880 4.3892 1.6429 3.9782 0.6081 0.6019 0.6128 0.6081 53.42
4.0491 5.0 17350 4.4182 1.8093 3.9656 0.6156 0.6091 0.6163 0.6156 52.75
3.9624 6.0 20820 4.4757 2.0348 3.9666 0.6121 0.6048 0.6189 0.6121 52.81
3.8954 7.0 24290 4.4969 2.1327 3.9634 0.6092 0.6028 0.6087 0.6092 52.64
3.846 8.0 27760 4.5632 2.4063 3.9613 0.6017 0.5972 0.6014 0.6017 52.52
3.8036 9.0 31230 4.6068 2.5888 3.9592 0.6052 0.5988 0.6026 0.6052 52.41
3.7724 10.0 34700 4.6175 2.6197 3.9621 0.6052 0.6006 0.6009 0.6052 52.57
3.7484 11.0 38170 4.6745 2.8470 3.9622 0.6046 0.5996 0.6034 0.6046 52.57
3.7291 12.0 41640 4.6854 2.8950 3.9611 0.6110 0.6056 0.6049 0.6110 52.52
3.7148 13.0 45110 4.7103 2.9919 3.9618 0.6063 0.6002 0.6029 0.6063 52.55
3.703 14.0 48580 4.7226 3.0417 3.9616 0.6081 0.6027 0.6021 0.6081 52.54
3.6968 15.0 52050 4.7276 3.0579 3.9626 0.6110 0.6054 0.6054 0.6110 52.59

Framework versions

  • Transformers 4.21.2
  • Pytorch 1.12.1
  • Datasets 2.4.0
  • Tokenizers 0.12.1