madatnlp/ke-t5-scratch

This model is a fine-tuned version of madatnlp/ke-t5-math-py on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.4760
• Validation Loss: 0.7360
• Epoch: 36

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
4.2751	2.1074	0
2.2716	1.7945	1
1.8889	1.5726	2
1.6760	1.3722	3
1.5021	1.3280	4
1.4369	1.2523	5
1.3352	1.0619	6
1.2749	1.1156	7
1.2170	1.0452	8
1.1713	1.0596	9
1.1410	1.0080	10
1.0884	1.0213	11
1.0508	0.9223	12
0.9933	0.9353	13
0.9871	0.8749	14
0.9251	0.9173	15
0.9282	0.8620	16
0.8849	0.8093	17
0.8613	0.7823	18
0.8322	0.8016	19
0.8070	0.8844	20
0.7737	0.7635	21
0.7465	0.8440	22
0.7178	0.7958	23
0.7036	0.7739	24
0.6813	0.7347	25
0.6597	0.7545	26
0.6427	0.7394	27
0.6154	0.7212	28
0.5892	0.7653	29
0.5696	0.7073	30
0.5644	0.6977	31
0.5307	0.6977	32
0.5159	0.7736	33
0.5131	0.8138	34
0.4812	0.7623	35
0.4760	0.7360	36

Framework versions

• Transformers 4.18.0
• TensorFlow 2.8.0
• Datasets 2.1.0
• Tokenizers 0.12.1