madatnlp/prefix-ket5-scratch

This model is a fine-tuned version of madatnlp/ke-t5-math-py on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.7214
• Validation Loss: 0.8747
• Epoch: 98

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
8.0101	5.1280	0
4.8040	3.6005	1
3.7550	2.8108	2
3.2740	2.6402	3
2.9682	2.3173	4
2.6871	2.1585	5
2.4782	2.0828	6
2.3507	1.9557	7
2.2131	1.8513	8
2.1235	1.6324	9
2.0157	1.6270	10
1.9722	1.6217	11
1.8733	1.5436	12
1.8680	1.5872	13
1.8365	1.6040	14
1.7528	1.5049	15
1.7411	1.4754	16
1.6733	1.4409	17
1.6544	1.4230	18
1.6271	1.4556	19
1.5658	1.3797	20
1.5774	1.3269	21
1.5150	1.3108	22
1.5057	1.3785	23
1.4605	1.3114	24
1.4702	1.2618	25
1.4220	1.2164	26
1.4194	1.2409	27
1.3942	1.2603	28
1.3921	1.3010	29
1.3645	1.1850	30
1.3336	1.1273	31
1.3499	1.1533	32
1.3022	1.1683	33
1.2990	1.1403	34
1.2876	1.1241	35
1.2479	1.0957	36
1.2441	1.1989	37
1.2464	1.1416	38
1.2353	1.0636	39
1.2152	1.1136	40
1.2212	1.0635	41
1.1892	1.0818	42
1.1959	1.1041	43
1.1957	1.0912	44
1.1542	1.0949	45
1.1403	1.1272	46
1.1396	1.1169	47
1.1149	1.0606	48
1.1238	1.0610	49
1.1246	1.0234	50
1.0971	0.9865	51
1.0883	1.0568	52
1.0774	1.0099	53
1.0581	1.0023	54
1.0680	1.0197	55
1.0682	0.9835	56
1.0390	0.9789	57
1.0480	1.0217	58
1.0273	0.9622	59
1.0062	1.0174	60
1.0088	0.9612	61
0.9909	0.9998	62
0.9821	1.0115	63
0.9752	0.9712	64
0.9816	0.9677	65
0.9569	0.9503	66
0.9521	1.0052	67
0.9384	0.9752	68
0.9468	0.9767	69
0.9241	1.0076	70
0.9211	0.9414	71
0.9166	1.0294	72
0.9044	0.9772	73
0.9025	0.9273	74
0.8909	1.0077	75
0.8831	0.9292	76
0.8702	0.9320	77
0.8644	0.9879	78
0.8599	0.9027	79
0.8434	0.9197	80
0.8561	0.9447	81
0.8330	0.9730	82
0.8328	0.9137	83
0.8221	0.9232	84
0.8166	0.9115	85
0.8025	0.9530	86
0.8070	0.9270	87
0.7968	0.8474	88
0.7880	0.9171	89
0.7834	0.8668	90
0.7786	0.9049	91
0.7595	0.9348	92
0.7573	0.8826	93
0.7505	0.8765	94
0.7474	0.9312	95
0.7386	0.9211	96
0.7490	0.9223	97
0.7214	0.8747	98

Framework versions

• Transformers 4.18.0
• TensorFlow 2.8.0
• Datasets 2.2.1
• Tokenizers 0.12.1