pakawadeep/mt5-small-finetuned-ctfl-augmented_2

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.6759
• Validation Loss: 0.9239
• Train Rouge1: 7.9562
• Train Rouge2: 1.3861
• Train Rougel: 7.9562
• Train Rougelsum: 7.9915
• Train Gen Len: 11.9653
• Epoch: 29

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Rouge1	Train Rouge2	Train Rougel	Train Rougelsum	Train Gen Len	Epoch
8.2342	2.0102	1.5963	0.0	1.6101	1.6005	16.5050	0
3.7896	1.7810	4.8515	0.7426	4.8845	4.8680	11.8614	1
2.7018	1.7212	6.8812	1.4851	6.8812	6.8812	11.9554	2
2.1382	1.5885	7.8996	2.0792	7.9066	7.9066	11.9010	3
1.7718	1.4799	7.7086	2.0792	7.7086	7.7086	11.8911	4
1.5137	1.3902	7.7086	2.0792	7.7086	7.7086	11.9653	5
1.3341	1.3439	8.6987	2.0792	8.6987	8.6987	11.9307	6
1.2253	1.2605	8.6987	2.0792	8.6987	8.6987	11.9356	7
1.1425	1.2215	8.9816	2.3762	8.9816	8.9816	11.9356	8
1.0872	1.1772	8.9816	2.3762	8.9816	8.9816	11.9554	9
1.0400	1.1338	8.6987	1.8812	8.6987	8.7341	11.9604	10
0.9997	1.0986	8.6987	1.8812	8.6987	8.7341	11.9554	11
0.9732	1.0846	8.4512	1.3861	8.4158	8.4866	11.9653	12
0.9388	1.0718	8.4512	1.3861	8.4158	8.4866	11.9752	13
0.9140	1.0483	8.4512	1.3861	8.4158	8.4866	11.9505	14
0.8902	1.0285	8.4512	1.3861	8.4158	8.4866	11.9554	15
0.8704	1.0147	8.4512	1.3861	8.4158	8.4866	11.9505	16
0.8490	1.0094	8.4512	1.3861	8.4158	8.4866	11.9505	17
0.8338	0.9880	7.9562	1.3861	7.9562	7.9915	11.9406	18
0.8139	0.9921	7.9562	1.3861	7.9562	7.9915	11.9455	19
0.7992	0.9765	7.9562	1.3861	7.9562	7.9915	11.9604	20
0.7806	0.9704	7.9562	1.3861	7.9562	7.9915	11.9604	21
0.7651	0.9523	7.9562	1.3861	7.9562	7.9915	11.9554	22
0.7560	0.9615	7.9562	1.3861	7.9562	7.9915	11.9653	23
0.7370	0.9489	7.9562	1.3861	7.9562	7.9915	11.9752	24
0.7263	0.9350	7.9562	1.3861	7.9562	7.9915	11.9604	25
0.7111	0.9425	7.9562	1.3861	7.9562	7.9915	11.9802	26
0.7005	0.9348	7.9562	1.3861	7.9562	7.9915	11.9752	27
0.6912	0.9213	7.9562	1.3861	7.9562	7.9915	11.9752	28
0.6759	0.9239	7.9562	1.3861	7.9562	7.9915	11.9653	29

Framework versions

• Transformers 4.41.2
• TensorFlow 2.15.0
• Datasets 2.20.0
• Tokenizers 0.19.1