README.md

metadata

tags:
  - generated_from_keras_callback
model-index:
  - name: distilgpt_new3_0035
    results: []

distilgpt_new3_0035

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 2.5161
• Validation Loss: 2.3990
• Epoch: 34

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
2.5407	2.4254	0
2.5399	2.4247	1
2.5391	2.4238	2
2.5383	2.4232	3
2.5375	2.4210	4
2.5368	2.4210	5
2.5361	2.4197	6
2.5353	2.4193	7
2.5345	2.4191	8
2.5339	2.4177	9
2.5332	2.4188	10
2.5324	2.4160	11
2.5317	2.4164	12
2.5309	2.4145	13
2.5302	2.4153	14
2.5295	2.4139	15
2.5288	2.4134	16
2.5282	2.4123	17
2.5274	2.4116	18
2.5267	2.4110	19
2.5259	2.4106	20
2.5251	2.4097	21
2.5244	2.4074	22
2.5238	2.4078	23
2.5232	2.4072	24
2.5223	2.4062	25
2.5217	2.4054	26
2.5211	2.4057	27
2.5204	2.4044	28
2.5197	2.4026	29
2.5189	2.4017	30
2.5182	2.4026	31
2.5176	2.4012	32
2.5168	2.4013	33
2.5161	2.3990	34

Framework versions

• Transformers 4.20.1
• TensorFlow 2.8.2
• Datasets 2.3.2
• Tokenizers 0.12.1