nmt-mpst-id-en-lr_0.001-ep_30-seq_128_bs-16

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.3591
• Bleu: 0.2073
• Meteor: 0.3779

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Meteor
No log	1.0	404	2.0642	0.1068	0.2561
2.5607	2.0	808	1.7482	0.1392	0.299
1.7768	3.0	1212	1.6392	0.1614	0.325
1.4132	4.0	1616	1.6131	0.1728	0.3418
1.205	5.0	2020	1.5724	0.1854	0.3543
1.205	6.0	2424	1.5988	0.1897	0.3592
1.0069	7.0	2828	1.5839	0.1922	0.3618
0.8711	8.0	3232	1.6187	0.196	0.3678
0.7759	9.0	3636	1.6453	0.1968	0.3672
0.6838	10.0	4040	1.6837	0.1981	0.3685
0.6838	11.0	4444	1.7401	0.1976	0.3698
0.5903	12.0	4848	1.7686	0.2016	0.3712
0.5207	13.0	5252	1.8075	0.2026	0.3733
0.4712	14.0	5656	1.8665	0.2028	0.3743
0.4154	15.0	6060	1.9114	0.204	0.3746
0.4154	16.0	6464	1.9556	0.2036	0.376
0.3726	17.0	6868	1.9961	0.2011	0.374
0.326	18.0	7272	2.0437	0.2027	0.3739
0.2936	19.0	7676	2.0946	0.2038	0.3754
0.2671	20.0	8080	2.1319	0.2041	0.374
0.2671	21.0	8484	2.1717	0.2044	0.3756
0.2407	22.0	8888	2.2025	0.2045	0.3756
0.2143	23.0	9292	2.2375	0.2031	0.3734
0.1974	24.0	9696	2.2544	0.2057	0.3765
0.182	25.0	10100	2.2875	0.2057	0.3767
0.1686	26.0	10504	2.3153	0.2048	0.3762
0.1686	27.0	10908	2.3395	0.2063	0.3786
0.1548	28.0	11312	2.3493	0.2071	0.3783
0.145	29.0	11716	2.3569	0.2072	0.3781
0.1412	30.0	12120	2.3591	0.2073	0.3779

Framework versions

• Transformers 4.24.0
• Pytorch 1.12.1+cu113
• Datasets 2.7.0
• Tokenizers 0.13.2