Edit model card

distilgpt_new2_0040

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 2.5812
  • Validation Loss: 2.4689
  • Epoch: 39

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
  • training_precision: float32

Training results

Train Loss Validation Loss Epoch
2.6241 2.5129 0
2.6228 2.5112 1
2.6216 2.5105 2
2.6204 2.5101 3
2.6191 2.5088 4
2.6180 2.5064 5
2.6166 2.5045 6
2.6155 2.5038 7
2.6143 2.5024 8
2.6132 2.5009 9
2.6120 2.5014 10
2.6108 2.4984 11
2.6097 2.4983 12
2.6085 2.4976 13
2.6073 2.4948 14
2.6064 2.4945 15
2.6052 2.4939 16
2.6039 2.4925 17
2.6030 2.4912 18
2.6019 2.4890 19
2.6007 2.4889 20
2.5998 2.4872 21
2.5987 2.4865 22
2.5977 2.4859 23
2.5965 2.4844 24
2.5956 2.4834 25
2.5944 2.4821 26
2.5934 2.4805 27
2.5925 2.4790 28
2.5914 2.4798 29
2.5904 2.4777 30
2.5893 2.4781 31
2.5883 2.4755 32
2.5872 2.4763 33
2.5862 2.4743 34
2.5851 2.4736 35
2.5841 2.4732 36
2.5831 2.4726 37
2.5822 2.4691 38
2.5812 2.4689 39

Framework versions

  • Transformers 4.20.1
  • TensorFlow 2.8.2
  • Datasets 2.3.2
  • Tokenizers 0.12.1
Downloads last month
6