Code_Corrector_Model

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.9156
• Bleu: 0.0
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
No log	1.0	8	6.6371	0.0	19.0
6.8242	2.0	16	5.2780	0.0	19.0
6.8242	3.0	24	4.4851	0.0	19.0
5.345	4.0	32	3.7656	0.0	19.0
5.345	5.0	40	3.0462	0.0	19.0
4.0321	6.0	48	2.4729	0.0	19.0
3.2425	7.0	56	2.1585	0.0	11.7931
3.2425	8.0	64	2.0606	0.0	0.0
2.8344	9.0	72	2.0090	0.0	0.0
2.8344	10.0	80	1.9443	0.0	0.0
2.6721	11.0	88	1.8702	0.0	0.0
2.6721	12.0	96	1.8071	0.0	0.0
2.5019	13.0	104	1.7541	0.0	0.0
2.3339	14.0	112	1.7014	0.0	0.0
2.3339	15.0	120	1.6502	0.0	0.0
2.2227	16.0	128	1.6094	0.0	0.0
2.2227	17.0	136	1.5746	0.0	0.0
2.1738	18.0	144	1.5353	0.0	0.0
2.1738	19.0	152	1.5066	0.0	0.0
2.054	20.0	160	1.4870	0.0	0.0
1.9707	21.0	168	1.4581	0.0	0.0
1.9707	22.0	176	1.4359	0.0	0.0
1.96	23.0	184	1.4032	0.0	0.0
1.96	24.0	192	1.3737	0.0	0.0
1.7402	25.0	200	1.3482	0.0	0.0
1.7402	26.0	208	1.3257	0.0	0.0
1.7044	27.0	216	1.3047	0.0	0.0
1.751	28.0	224	1.2861	0.0	0.0
1.751	29.0	232	1.2644	0.0	0.0
1.6414	30.0	240	1.2353	0.0	0.0
1.6414	31.0	248	1.2160	0.0	0.0
1.6418	32.0	256	1.1991	0.0	0.0
1.6418	33.0	264	1.1937	0.0	0.0
1.6258	34.0	272	1.1762	0.0	0.0
1.6102	35.0	280	1.1632	0.0	0.0
1.6102	36.0	288	1.1498	0.0	0.0
1.5266	37.0	296	1.1361	0.0	0.0
1.5266	38.0	304	1.1205	0.0	10.4828
1.5756	39.0	312	1.1108	0.0	10.4828
1.5756	40.0	320	1.1028	0.0	10.4828
1.5136	41.0	328	1.0937	0.0	10.4828
1.529	42.0	336	1.0837	0.0	10.4828
1.529	43.0	344	1.0714	0.0	11.7931
1.4738	44.0	352	1.0599	0.0	13.1034
1.4738	45.0	360	1.0514	0.0	13.1034
1.4521	46.0	368	1.0467	0.0	13.1034
1.4521	47.0	376	1.0438	0.0	13.1034
1.4758	48.0	384	1.0358	0.0	13.1034
1.4698	49.0	392	1.0264	0.0	13.1034
1.4698	50.0	400	1.0205	0.0	17.6897
1.3355	51.0	408	1.0159	0.0	18.3448
1.3355	52.0	416	1.0087	0.0	19.0
1.36	53.0	424	1.0040	0.0	19.0
1.36	54.0	432	1.0005	0.0	19.0
1.3025	55.0	440	0.9955	0.0	19.0
1.2773	56.0	448	0.9910	0.0	19.0
1.2773	57.0	456	0.9873	0.0	19.0
1.3006	58.0	464	0.9840	0.0	19.0
1.3006	59.0	472	0.9826	0.0	19.0
1.3037	60.0	480	0.9813	0.0	19.0
1.3037	61.0	488	0.9765	0.0	19.0
1.3133	62.0	496	0.9717	0.0	19.0
1.2601	63.0	504	0.9671	0.0	19.0
1.2601	64.0	512	0.9637	0.0	19.0
1.2442	65.0	520	0.9610	0.0	19.0
1.2442	66.0	528	0.9585	0.0	19.0
1.2394	67.0	536	0.9568	0.0	19.0
1.2394	68.0	544	0.9546	0.0	19.0
1.2746	69.0	552	0.9509	0.0	19.0
1.233	70.0	560	0.9478	0.0	19.0
1.233	71.0	568	0.9452	0.0	19.0
1.2382	72.0	576	0.9424	0.0	19.0
1.2382	73.0	584	0.9400	0.0	19.0
1.2603	74.0	592	0.9379	0.0	19.0
1.2603	75.0	600	0.9357	0.0	19.0
1.2028	76.0	608	0.9338	0.0	19.0
1.2755	77.0	616	0.9330	0.0	19.0
1.2755	78.0	624	0.9316	0.0	19.0
1.244	79.0	632	0.9303	0.0	19.0
1.244	80.0	640	0.9291	0.0	19.0
1.115	81.0	648	0.9281	0.0	19.0
1.115	82.0	656	0.9272	0.0	19.0
1.2373	83.0	664	0.9258	0.0	19.0
1.2035	84.0	672	0.9243	0.0	19.0
1.2035	85.0	680	0.9231	0.0	19.0
1.1881	86.0	688	0.9216	0.0	19.0
1.1881	87.0	696	0.9205	0.0	19.0
1.1713	88.0	704	0.9200	0.0	19.0
1.1713	89.0	712	0.9191	0.0	19.0
1.1984	90.0	720	0.9184	0.0	19.0
1.2879	91.0	728	0.9177	0.0	19.0
1.2879	92.0	736	0.9174	0.0	19.0
1.1823	93.0	744	0.9171	0.0	19.0
1.1823	94.0	752	0.9170	0.0	19.0
1.2293	95.0	760	0.9166	0.0	19.0
1.2293	96.0	768	0.9162	0.0	19.0
1.2154	97.0	776	0.9160	0.0	19.0
1.1625	98.0	784	0.9158	0.0	19.0
1.1625	99.0	792	0.9156	0.0	19.0
1.1679	100.0	800	0.9156	0.0	19.0

Framework versions

• Transformers 4.33.0
• Pytorch 2.0.0
• Datasets 2.1.0
• Tokenizers 0.13.3