Edit model card

distilgpt_new2_0060

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 2.5626
  • Validation Loss: 2.4481
  • Epoch: 59

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
  • training_precision: float32

Training results

Train Loss Validation Loss Epoch
2.6241 2.5129 0
2.6228 2.5112 1
2.6216 2.5105 2
2.6204 2.5101 3
2.6191 2.5088 4
2.6180 2.5064 5
2.6166 2.5045 6
2.6155 2.5038 7
2.6143 2.5024 8
2.6132 2.5009 9
2.6120 2.5014 10
2.6108 2.4984 11
2.6097 2.4983 12
2.6085 2.4976 13
2.6073 2.4948 14
2.6064 2.4945 15
2.6052 2.4939 16
2.6039 2.4925 17
2.6030 2.4912 18
2.6019 2.4890 19
2.6007 2.4889 20
2.5998 2.4872 21
2.5987 2.4865 22
2.5977 2.4859 23
2.5965 2.4844 24
2.5956 2.4834 25
2.5944 2.4821 26
2.5934 2.4805 27
2.5925 2.4790 28
2.5914 2.4798 29
2.5904 2.4777 30
2.5893 2.4781 31
2.5883 2.4755 32
2.5872 2.4763 33
2.5862 2.4743 34
2.5851 2.4736 35
2.5841 2.4732 36
2.5831 2.4726 37
2.5822 2.4691 38
2.5812 2.4689 39
2.5803 2.4670 40
2.5792 2.4675 41
2.5783 2.4650 42
2.5773 2.4650 43
2.5763 2.4625 44
2.5754 2.4615 45
2.5743 2.4619 46
2.5735 2.4597 47
2.5725 2.4594 48
2.5715 2.4582 49
2.5707 2.4568 50
2.5699 2.4562 51
2.5689 2.4565 52
2.5680 2.4550 53
2.5671 2.4536 54
2.5661 2.4529 55
2.5653 2.4513 56
2.5644 2.4515 57
2.5635 2.4510 58
2.5626 2.4481 59

Framework versions

  • Transformers 4.20.1
  • TensorFlow 2.8.2
  • Datasets 2.3.2
  • Tokenizers 0.12.1
Downloads last month
6