Edit model card

distilgpt_new3_0080

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 2.4883
  • Validation Loss: 2.3693
  • Epoch: 79

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
  • training_precision: float32

Training results

Train Loss Validation Loss Epoch
2.5407 2.4254 0
2.5399 2.4247 1
2.5391 2.4238 2
2.5383 2.4232 3
2.5375 2.4210 4
2.5368 2.4210 5
2.5361 2.4197 6
2.5353 2.4193 7
2.5345 2.4191 8
2.5339 2.4177 9
2.5332 2.4188 10
2.5324 2.4160 11
2.5317 2.4164 12
2.5309 2.4145 13
2.5302 2.4153 14
2.5295 2.4139 15
2.5288 2.4134 16
2.5282 2.4123 17
2.5274 2.4116 18
2.5267 2.4110 19
2.5259 2.4106 20
2.5251 2.4097 21
2.5244 2.4074 22
2.5238 2.4078 23
2.5232 2.4072 24
2.5223 2.4062 25
2.5217 2.4054 26
2.5211 2.4057 27
2.5204 2.4044 28
2.5197 2.4026 29
2.5189 2.4017 30
2.5182 2.4026 31
2.5176 2.4012 32
2.5168 2.4013 33
2.5161 2.3990 34
2.5154 2.3999 35
2.5149 2.3978 36
2.5142 2.3981 37
2.5135 2.3981 38
2.5130 2.3972 39
2.5123 2.3957 40
2.5116 2.3940 41
2.5108 2.3933 42
2.5103 2.3927 43
2.5095 2.3923 44
2.5090 2.3918 45
2.5083 2.3914 46
2.5078 2.3905 47
2.5070 2.3888 48
2.5062 2.3894 49
2.5058 2.3898 50
2.5051 2.3868 51
2.5045 2.3873 52
2.5041 2.3872 53
2.5035 2.3859 54
2.5027 2.3850 55
2.5020 2.3851 56
2.5016 2.3833 57
2.5009 2.3816 58
2.5002 2.3821 59
2.4995 2.3813 60
2.4990 2.3803 61
2.4984 2.3794 62
2.4977 2.3798 63
2.4971 2.3779 64
2.4964 2.3778 65
2.4959 2.3778 66
2.4954 2.3787 67
2.4947 2.3758 68
2.4942 2.3751 69
2.4935 2.3739 70
2.4929 2.3754 71
2.4923 2.3750 72
2.4918 2.3730 73
2.4912 2.3729 74
2.4906 2.3712 75
2.4901 2.3714 76
2.4894 2.3704 77
2.4888 2.3699 78
2.4883 2.3693 79

Framework versions

  • Transformers 4.20.1
  • TensorFlow 2.8.2
  • Datasets 2.3.2
  • Tokenizers 0.12.1
Downloads last month
3
Hosted inference API
Text Generation
Examples
Examples
This model can be loaded on the Inference API on-demand.