pakawadeep/mt5-base-finetuned-ctfl

This model is a fine-tuned version of pakawadeep/mt5-base-finetuned-ctfl on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.3091
• Validation Loss: 1.1147
• Train Rouge1: 8.9816
• Train Rouge2: 1.1881
• Train Rougel: 8.8048
• Train Rougelsum: 8.7871
• Train Gen Len: 11.9604
• Epoch: 27

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Rouge1	Train Rouge2	Train Rougel	Train Rougelsum	Train Gen Len	Epoch
1.1067	1.0353	7.4965	1.6832	7.4257	7.3904	11.8762	0
0.9573	1.0010	7.9915	1.6832	7.9208	7.7793	11.9109	1
0.8858	1.0002	8.4866	2.1782	8.2744	8.2744	11.9158	2
0.8402	0.9827	8.4866	2.1782	8.2744	8.2744	11.9554	3
0.7900	0.9961	8.4866	2.1782	8.2744	8.2744	11.9158	4
0.7646	0.9898	8.4866	2.1782	8.2744	8.2744	11.9505	5
0.7190	0.9805	8.4866	2.1782	8.2744	8.2744	11.9208	6
0.7021	0.9683	8.4866	2.1782	8.2744	8.2744	11.9455	7
0.6613	0.9732	8.9816	2.1782	8.7694	8.8755	11.9703	8
0.6416	0.9807	8.4866	2.1782	8.2744	8.2744	11.9505	9
0.6139	0.9771	8.4866	2.1782	8.2744	8.2744	11.9307	10
0.5864	0.9723	8.4866	2.1782	8.2744	8.2744	11.9505	11
0.5844	0.9919	8.4866	2.1782	8.2744	8.2744	11.9653	12
0.5679	1.0097	8.4866	2.1782	8.2744	8.2744	11.9307	13
0.5329	0.9947	7.9915	1.1881	7.8501	7.7793	11.9554	14
0.5173	0.9877	8.2037	1.6832	8.0622	8.0269	11.9505	15
0.4823	0.9955	7.7793	1.1881	7.5318	7.5318	11.9109	16
0.4626	1.0106	7.9915	1.1881	7.8501	7.7793	11.9703	17
0.4497	1.0056	7.7793	1.1881	7.5318	7.5318	11.9109	18
0.4276	1.0341	7.7793	1.1881	7.5318	7.5318	11.8911	19
0.4056	1.0482	7.7793	1.1881	7.5318	7.5318	11.8960	20
0.4003	1.0365	8.2390	1.1881	7.9915	8.1683	11.9356	21
0.3774	1.0646	8.2390	1.1881	7.9915	8.1683	11.9158	22
0.3668	1.0713	8.2390	1.1881	7.9915	8.1683	11.9158	23
0.3539	1.0748	8.2390	1.1881	7.9915	8.1683	11.9257	24
0.3355	1.0859	8.0387	0.8911	7.9208	7.9208	11.8663	25
0.3247	1.0929	8.7694	1.1881	8.5573	8.5573	11.9356	26
0.3091	1.1147	8.9816	1.1881	8.8048	8.7871	11.9604	27

Framework versions

• Transformers 4.38.2
• TensorFlow 2.15.0
• Datasets 2.18.0
• Tokenizers 0.15.2