t5-small-finetuned-thai-informal-to-formal

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3091
• Bleu: 20.5964
• Gen Len: 19.9981

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 300
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
2.2862	1.0	1011	2.2028	31.6678	20.0
2.1228	2.0	2022	2.0339	32.3643	20.0
2.0581	3.0	3033	1.9386	32.3784	20.0
1.9714	4.0	4044	1.8899	31.9728	20.0
1.9169	5.0	5055	1.8318	32.1064	20.0
1.8969	6.0	6066	1.8005	31.4324	20.0
1.8486	7.0	7077	1.7813	31.7758	20.0
1.802	8.0	8088	1.7464	31.9055	20.0
1.7654	9.0	9099	1.7352	31.9598	20.0
1.7439	10.0	10110	1.7009	32.1696	20.0
1.7603	11.0	11121	1.6873	31.8118	20.0
1.7288	12.0	12132	1.6678	31.5711	20.0
1.7004	13.0	13143	1.6482	31.4575	20.0
1.6851	14.0	14154	1.6374	31.9579	20.0
1.6497	15.0	15165	1.6290	31.4299	20.0
1.656	16.0	16176	1.6130	31.2145	20.0
1.6423	17.0	17187	1.5931	31.365	20.0
1.6024	18.0	18198	1.5797	31.2247	20.0
1.6064	19.0	19209	1.5736	31.1535	20.0
1.5974	20.0	20220	1.5609	31.431	20.0
1.5961	21.0	21231	1.5578	30.9905	20.0
1.5621	22.0	22242	1.5466	30.8979	20.0
1.5307	23.0	23253	1.5285	31.277	20.0
1.5359	24.0	24264	1.5370	31.4321	20.0
1.5558	25.0	25275	1.5215	31.2769	20.0
1.513	26.0	26286	1.5173	30.9782	19.9997
1.5241	27.0	27297	1.5105	30.6717	20.0
1.5133	28.0	28308	1.4973	30.3152	20.0
1.4713	29.0	29319	1.4927	30.276	19.9997
1.478	30.0	30330	1.4887	30.1004	19.9989
1.4572	31.0	31341	1.4845	29.8939	19.9983
1.4485	32.0	32352	1.4653	30.0169	19.9986
1.4404	33.0	33363	1.4648	28.9061	19.9989
1.4408	34.0	34374	1.4586	29.598	19.9994
1.4296	35.0	35385	1.4585	28.9821	19.9981
1.408	36.0	36396	1.4517	29.6025	19.9986
1.4004	37.0	37407	1.4456	27.8564	19.9992
1.3991	38.0	38418	1.4411	28.8947	19.9994
1.401	39.0	39429	1.4309	27.6809	19.9994
1.391	40.0	40440	1.4278	29.1687	19.9994
1.3709	41.0	41451	1.4217	28.2947	19.9989
1.3726	42.0	42462	1.4247	27.2108	19.9983
1.3702	43.0	43473	1.4144	25.9973	19.9981
1.3636	44.0	44484	1.4163	26.0146	19.9953
1.3673	45.0	45495	1.4118	25.8126	19.9978
1.3539	46.0	46506	1.4076	25.5185	19.9981
1.3434	47.0	47517	1.4023	26.2123	19.9947
1.3428	48.0	48528	1.4008	25.8932	19.9955
1.3325	49.0	49539	1.4003	25.7762	19.9969
1.3258	50.0	50550	1.3896	24.8206	19.9961
1.3151	51.0	51561	1.3852	24.4683	19.9978
1.3035	52.0	52572	1.3843	24.9821	19.9992
1.2931	53.0	53583	1.3847	24.715	19.9989
1.2707	54.0	54594	1.3776	24.4374	19.9986
1.2792	55.0	55605	1.3801	23.7683	19.9967
1.284	56.0	56616	1.3781	23.6961	19.9975
1.2664	57.0	57627	1.3680	23.6677	19.9975
1.2783	58.0	58638	1.3695	23.3193	19.9986
1.2762	59.0	59649	1.3741	22.613	19.9972
1.2759	60.0	60660	1.3629	23.9067	19.9964
1.2618	61.0	61671	1.3687	23.7587	19.9967
1.2614	62.0	62682	1.3613	23.2615	19.9975
1.2455	63.0	63693	1.3623	23.8722	19.9986
1.1977	64.0	64704	1.3528	23.1421	19.9981
1.2199	65.0	65715	1.3520	22.6977	19.9975
1.2368	66.0	66726	1.3552	23.2495	19.9989
1.2087	67.0	67737	1.3404	22.6422	19.9989
1.214	68.0	68748	1.3499	21.979	19.9972
1.2322	69.0	69759	1.3453	22.1766	19.9978
1.2028	70.0	70770	1.3402	21.8311	19.9975
1.2163	71.0	71781	1.3399	22.1417	19.9989
1.1769	72.0	72792	1.3446	22.253	19.9972
1.221	73.0	73803	1.3413	22.1546	19.9986
1.1768	74.0	74814	1.3335	21.8914	19.9972
1.1829	75.0	75825	1.3323	21.7763	19.9947
1.1687	76.0	76836	1.3344	21.4495	19.9964
1.1873	77.0	77847	1.3337	21.7655	19.9964
1.1807	78.0	78858	1.3308	21.4564	19.9967
1.1735	79.0	79869	1.3282	21.233	19.9967
1.1693	80.0	80880	1.3240	21.0794	19.9955
1.1714	81.0	81891	1.3262	21.1856	19.9969
1.154	82.0	82902	1.3282	20.5583	19.9964
1.1572	83.0	83913	1.3229	20.9262	19.995
1.1473	84.0	84924	1.3233	20.5432	19.995
1.1315	85.0	85935	1.3227	20.4939	19.9942
1.1567	86.0	86946	1.3203	21.3354	19.9964
1.1485	87.0	87957	1.3211	20.9952	19.9939
1.1313	88.0	88968	1.3202	20.1199	19.9961
1.1428	89.0	89979	1.3188	20.414	19.9925
1.1374	90.0	90990	1.3220	20.003	19.993
1.1274	91.0	92001	1.3153	20.7172	19.9953
1.1174	92.0	93012	1.3126	20.5997	19.9953
1.1155	93.0	94023	1.3131	20.0402	19.993
1.1167	94.0	95034	1.3140	20.219	19.9905
1.1301	95.0	96045	1.3142	19.8332	19.9922
1.0975	96.0	97056	1.3096	19.6051	19.9942
1.1025	97.0	98067	1.3148	20.4323	19.993
1.0932	98.0	99078	1.3134	20.0839	19.9942
1.0871	99.0	100089	1.3071	20.0202	19.9939
1.102	100.0	101100	1.3091	20.0454	19.9947
1.0969	101.0	102111	1.3090	19.4474	19.9947
1.0988	102.0	103122	1.3117	20.1905	19.9922
1.0816	103.0	104133	1.3048	20.3346	19.9928
1.0809	104.0	105144	1.3058	20.323	19.9953
1.0861	105.0	106155	1.3052	20.6984	19.9944
1.0907	106.0	107166	1.3076	20.3413	19.9947
1.0747	107.0	108177	1.3050	20.3362	19.9955
1.0839	108.0	109188	1.3060	20.5379	19.9936
1.0755	109.0	110199	1.3071	20.3886	19.9939
1.0463	110.0	111210	1.3058	19.9524	19.9953
1.0644	111.0	112221	1.3033	19.7226	19.9972
1.0771	112.0	113232	1.3089	19.9861	19.9958
1.0819	113.0	114243	1.3031	20.5527	19.9942
1.0483	114.0	115254	1.3063	20.0048	19.9978
1.04	115.0	116265	1.3020	20.2327	19.9969
1.0574	116.0	117276	1.3025	19.6818	19.995
1.0356	117.0	118287	1.3077	20.1054	19.9967
1.0525	118.0	119298	1.3022	20.14	19.9967
1.0409	119.0	120309	1.2983	19.7657	19.9972
1.0431	120.0	121320	1.2945	20.1315	19.9975
1.0419	121.0	122331	1.3035	19.8364	19.9972
1.0411	122.0	123342	1.2951	20.204	19.9981
1.0396	123.0	124353	1.3019	20.6711	19.9955
1.0424	124.0	125364	1.2950	20.6527	19.9969
1.0203	125.0	126375	1.3008	20.4314	19.9972
1.0351	126.0	127386	1.3008	20.0237	19.9978
1.0424	127.0	128397	1.2993	20.3024	19.9983
1.0165	128.0	129408	1.2960	20.1769	19.9978
1.0216	129.0	130419	1.2977	19.8483	19.9972
1.0207	130.0	131430	1.2939	20.0639	19.9969
1.0119	131.0	132441	1.2985	19.731	19.9972
0.9965	132.0	133452	1.3006	19.5983	19.9969
1.0034	133.0	134463	1.2974	19.6943	19.9989
1.0241	134.0	135474	1.3015	20.0083	19.9981
1.0181	135.0	136485	1.2982	19.6057	19.9989
1.0112	136.0	137496	1.2931	19.3408	19.9986
0.9927	137.0	138507	1.2999	19.5222	19.9983
1.0134	138.0	139518	1.2909	19.42	19.9989
0.9921	139.0	140529	1.2951	19.8604	19.9989
0.9891	140.0	141540	1.2916	20.0752	19.9989
0.9896	141.0	142551	1.2910	19.7536	19.9992
1.0034	142.0	143562	1.2934	20.0064	19.9986
0.9718	143.0	144573	1.2973	19.9304	19.9989
1.0141	144.0	145584	1.2940	20.5053	19.9986
0.99	145.0	146595	1.2980	20.0913	19.9975
0.9729	146.0	147606	1.2927	19.7229	19.9978
0.9732	147.0	148617	1.2920	20.2104	19.9975
0.9778	148.0	149628	1.2947	20.1365	19.9981
0.987	149.0	150639	1.3007	20.3436	19.9972
0.987	150.0	151650	1.3003	20.2827	19.9983
0.9788	151.0	152661	1.2953	20.2941	19.9972
0.9899	152.0	153672	1.2951	20.5454	19.9978
0.978	153.0	154683	1.2946	20.7448	19.9969
0.9614	154.0	155694	1.2975	20.5359	19.9969
0.9759	155.0	156705	1.2925	20.3661	19.9975
0.9627	156.0	157716	1.2954	20.5535	19.9969
0.9692	157.0	158727	1.2930	20.1919	19.9969
0.9737	158.0	159738	1.2922	20.484	19.9972
0.9642	159.0	160749	1.2952	20.5444	19.9975
0.9679	160.0	161760	1.2930	20.3731	19.9983
0.9571	161.0	162771	1.2933	20.4158	19.9978
0.9542	162.0	163782	1.2937	20.4823	19.9978
0.9537	163.0	164793	1.2997	20.6457	19.9964
0.951	164.0	165804	1.2982	20.0897	19.9986
0.9556	165.0	166815	1.2944	20.45	19.9986
0.9534	166.0	167826	1.2961	20.2743	19.9967
0.9381	167.0	168837	1.2922	19.8311	19.9969
0.9347	168.0	169848	1.2938	19.9427	19.9978
0.9514	169.0	170859	1.2968	20.2039	19.9983
0.9439	170.0	171870	1.3014	19.9784	19.9961
0.9379	171.0	172881	1.3000	20.1213	19.9964
0.9326	172.0	173892	1.2930	20.0882	19.9969
0.9178	173.0	174903	1.2942	20.1997	19.9972
0.9511	174.0	175914	1.2931	20.6471	19.9969
0.9438	175.0	176925	1.2945	20.7321	19.9983
0.929	176.0	177936	1.2967	20.5813	19.9964
0.9343	177.0	178947	1.2940	20.2307	19.9978
0.9344	178.0	179958	1.2949	20.2401	19.9969
0.9319	179.0	180969	1.2974	19.9881	19.9972
0.9286	180.0	181980	1.2974	20.2666	19.9961
0.9074	181.0	182991	1.2939	20.2549	19.9969
0.93	182.0	184002	1.2990	20.0121	19.9969
0.9303	183.0	185013	1.2944	20.056	19.9978
0.9259	184.0	186024	1.3003	19.9021	19.9953
0.9014	185.0	187035	1.2962	20.0381	19.9958
0.9288	186.0	188046	1.2976	20.1909	19.9947
0.9086	187.0	189057	1.2969	20.2923	19.9969
0.9183	188.0	190068	1.2941	20.1649	19.9967
0.9141	189.0	191079	1.3028	20.0891	19.9958
0.9264	190.0	192090	1.2935	20.0164	19.9958
0.9307	191.0	193101	1.2956	19.8606	19.9964
0.9179	192.0	194112	1.2933	19.9815	19.9961
0.9123	193.0	195123	1.2977	20.1232	19.9953
0.9221	194.0	196134	1.3014	20.0674	19.995
0.9195	195.0	197145	1.3031	19.9839	19.9944
0.9139	196.0	198156	1.2947	20.0344	19.9953
0.9074	197.0	199167	1.2956	20.1076	19.9961
0.9149	198.0	200178	1.2963	20.0898	19.9955
0.9219	199.0	201189	1.2990	20.171	19.9964
0.8989	200.0	202200	1.2983	20.1548	19.9961
0.9004	201.0	203211	1.2977	20.2135	19.9955
0.9043	202.0	204222	1.3023	20.3024	19.9964
0.917	203.0	205233	1.3014	20.5967	19.9967
0.9012	204.0	206244	1.3001	20.5489	19.9961
0.9136	205.0	207255	1.2963	20.5013	19.9969
0.897	206.0	208266	1.3016	20.3285	19.9969
0.9036	207.0	209277	1.2981	20.3278	19.9967
0.9225	208.0	210288	1.3055	20.4756	19.9967
0.8959	209.0	211299	1.2987	20.3112	19.9972
0.903	210.0	212310	1.2977	20.5512	19.9961
0.9012	211.0	213321	1.3026	20.4304	19.9964
0.8906	212.0	214332	1.2998	20.4206	19.9967
0.8906	213.0	215343	1.3031	20.4499	19.9964
0.9049	214.0	216354	1.3029	20.6908	19.9958
0.9034	215.0	217365	1.2980	20.3614	19.9969
0.8971	216.0	218376	1.2985	20.6196	19.9972
0.885	217.0	219387	1.3019	20.584	19.9972
0.8799	218.0	220398	1.3041	20.5843	19.9967
0.8805	219.0	221409	1.3035	20.5123	19.9972
0.8896	220.0	222420	1.3006	20.7331	19.9975
0.8851	221.0	223431	1.2973	20.6914	19.9975
0.893	222.0	224442	1.3004	20.7484	19.9978
0.8903	223.0	225453	1.3001	20.5207	19.9981
0.8924	224.0	226464	1.3026	20.6635	19.9972
0.8839	225.0	227475	1.3056	20.6999	19.9978
0.8631	226.0	228486	1.3042	20.9581	19.9967
0.8677	227.0	229497	1.3037	20.8283	19.9964
0.867	228.0	230508	1.3042	20.8781	19.9978
0.8878	229.0	231519	1.3035	20.6884	19.9981
0.8805	230.0	232530	1.3092	20.716	19.9975
0.8769	231.0	233541	1.2988	20.6323	19.9975
0.8833	232.0	234552	1.3039	20.5529	19.9978
0.8798	233.0	235563	1.3028	20.5848	19.9981
0.8694	234.0	236574	1.3037	20.4147	19.9983
0.8888	235.0	237585	1.3022	20.5179	19.9983
0.8724	236.0	238596	1.3027	20.4379	19.9978
0.8864	237.0	239607	1.3024	20.3993	19.9972
0.8684	238.0	240618	1.3043	20.5063	19.9969
0.8753	239.0	241629	1.3072	20.4079	19.9969
0.8734	240.0	242640	1.3026	20.5173	19.9967
0.867	241.0	243651	1.3044	20.6249	19.9972
0.8671	242.0	244662	1.3094	20.6827	19.9972
0.8721	243.0	245673	1.3045	20.5017	19.9978
0.8726	244.0	246684	1.3065	20.5748	19.9967
0.8741	245.0	247695	1.3063	20.5345	19.9972
0.8634	246.0	248706	1.3036	20.6084	19.9972
0.8527	247.0	249717	1.3045	20.535	19.9972
0.8662	248.0	250728	1.3089	20.5306	19.9972
0.8681	249.0	251739	1.3081	20.6414	19.9967
0.8711	250.0	252750	1.3061	20.6039	19.9975
0.8653	251.0	253761	1.3018	20.5632	19.9975
0.8697	252.0	254772	1.3090	20.5056	19.9978
0.8655	253.0	255783	1.3082	20.5235	19.9978
0.8636	254.0	256794	1.3067	20.5607	19.9972
0.8667	255.0	257805	1.3066	20.6694	19.9964
0.8596	256.0	258816	1.3073	20.617	19.9967
0.8507	257.0	259827	1.3083	20.6035	19.9964
0.8677	258.0	260838	1.3077	20.6196	19.9975
0.8614	259.0	261849	1.3094	20.6928	19.9969
0.8677	260.0	262860	1.3098	20.7181	19.9969
0.8628	261.0	263871	1.3065	20.679	19.9975
0.8636	262.0	264882	1.3055	20.7476	19.9975
0.8624	263.0	265893	1.3065	20.7045	19.9972
0.8594	264.0	266904	1.3093	20.5442	19.9964
0.8658	265.0	267915	1.3105	20.7153	19.9972
0.8476	266.0	268926	1.3076	20.677	19.9972
0.858	267.0	269937	1.3091	20.6701	19.9969
0.8707	268.0	270948	1.3111	20.5985	19.9975
0.8613	269.0	271959	1.3092	20.6108	19.9975
0.8497	270.0	272970	1.3070	20.5836	19.9964
0.8654	271.0	273981	1.3082	20.5806	19.9983
0.8621	272.0	274992	1.3088	20.6817	19.9975
0.8619	273.0	276003	1.3090	20.5567	19.9975
0.8638	274.0	277014	1.3087	20.6233	19.9975
0.8642	275.0	278025	1.3092	20.667	19.9967
0.8498	276.0	279036	1.3069	20.6295	19.9969
0.8572	277.0	280047	1.3107	20.6376	19.9969
0.8543	278.0	281058	1.3114	20.6473	19.9964
0.8453	279.0	282069	1.3105	20.6931	19.9967
0.8575	280.0	283080	1.3077	20.691	19.9972
0.8492	281.0	284091	1.3101	20.7528	19.9969
0.8519	282.0	285102	1.3094	20.6812	19.9981
0.8431	283.0	286113	1.3114	20.6608	19.9969
0.8546	284.0	287124	1.3093	20.6336	19.9981
0.86	285.0	288135	1.3108	20.6077	19.9967
0.8674	286.0	289146	1.3096	20.6742	19.9978
0.8493	287.0	290157	1.3106	20.6674	19.9981
0.8521	288.0	291168	1.3099	20.5915	19.9981
0.856	289.0	292179	1.3102	20.6448	19.9978
0.8614	290.0	293190	1.3096	20.6515	19.9981
0.8628	291.0	294201	1.3108	20.6679	19.9978
0.8498	292.0	295212	1.3104	20.6623	19.9978
0.8617	293.0	296223	1.3097	20.6591	19.9978
0.8563	294.0	297234	1.3098	20.6266	19.9978
0.856	295.0	298245	1.3095	20.6536	19.9978
0.8493	296.0	299256	1.3095	20.6273	19.9978
0.8498	297.0	300267	1.3092	20.5942	19.9978
0.8539	298.0	301278	1.3092	20.5942	19.9978
0.8608	299.0	302289	1.3091	20.5915	19.9981
0.8437	300.0	303300	1.3091	20.5964	19.9981

Framework versions

• Transformers 4.21.0
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1