codet5-kormath

This model is a fine-tuned version of Salesforce/codet5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.1215
• Validation Loss: 0.5475
• Epoch: 43

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'learning_rate': 0.001, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: mixed_bfloat16

Training results

Train Loss	Validation Loss	Epoch
6.8077	3.6916	0
3.3252	2.3874	1
2.2389	1.7377	2
1.6852	1.3997	3
1.3617	1.1785	4
1.1875	1.0796	5
1.0887	1.0150	6
1.0234	0.9669	7
0.9709	0.9350	8
0.9292	0.9016	9
0.8982	0.8763	10
0.8653	0.8565	11
0.8389	0.8378	12
0.8094	0.8241	13
0.7828	0.8078	14
0.7631	0.7945	15
0.7386	0.7856	16
0.7209	0.7749	17
0.7039	0.7703	18
0.6858	0.7708	19
0.6718	0.7705	20
0.6544	0.7710	21
0.6477	0.7728	22
0.6289	0.7590	23
0.6102	0.7616	24
0.5966	0.7644	25
0.5773	0.7559	26
0.5597	0.7609	27
0.5384	0.7823	28
0.5256	0.7263	29
0.4780	0.6671	30
0.4328	0.6591	31
0.3844	0.5918	32
0.3366	0.5852	33
0.3036	0.5633	34
0.2746	0.5566	35
0.2486	0.5644	36
0.2286	0.5293	37
0.2023	0.5256	38
0.1870	0.5276	39
0.1638	0.5272	40
0.1479	0.5642	41
0.1386	0.5482	42
0.1215	0.5475	43

Framework versions

• Transformers 4.19.2
• TensorFlow 2.8.0
• Datasets 2.2.1
• Tokenizers 0.12.1