alexantonov/ru-chv-marian-bt-3m-768d

model

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.0449
• Bleu: 19.7852
• Chrf: 52.1691
• Gen Len: 33.5979

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 256
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: inverse_sqrt
• lr_scheduler_warmup_steps: 8000
• num_epochs: 12
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Chrf	Gen Len
15.7108	0.1170	2000	7.7167	0.1227	9.1436	127.0
11.9208	0.2340	4000	5.9151	3.411	26.2609	127.0
9.5886	0.3510	6000	4.8183	7.1743	36.1064	115.4311
8.8915	0.4681	8000	4.4545	9.021	38.9109	76.0174
8.3764	0.5851	10000	4.1853	10.0952	40.9698	61.2619
8.0093	0.7021	12000	3.9794	10.5973	42.881	51.1286
7.7510	0.8191	14000	3.8382	12.373	43.9795	48.0614
7.5581	0.9361	16000	3.7683	12.5697	44.4739	48.9131
7.3176	1.0531	18000	3.6800	13.0398	45.5483	60.6640
7.2305	1.1701	20000	3.6033	14.0428	46.0606	35.8610
7.1481	1.2872	22000	3.5377	14.878	46.6609	38.3824
7.0607	1.4042	24000	3.5070	15.1898	47.4569	51.3198
7.0017	1.5212	26000	3.4737	16.0498	48.2444	34.2294
6.9453	1.6382	28000	3.4205	16.2782	48.3454	38.7173
6.8799	1.7552	30000	3.4045	15.3028	48.7879	35.3395
6.8324	1.8722	32000	3.3622	16.3933	49.0446	35.0811
6.7993	1.9892	34000	3.3549	16.327	48.8867	33.7462
6.6331	2.1062	36000	3.3195	16.2476	48.9527	35.2294
6.6308	2.2233	38000	3.3113	16.0936	49.0612	35.2294
6.6033	2.3403	40000	3.3108	16.0058	48.9797	35.5261
6.5890	2.4573	42000	3.2807	17.1487	49.695	33.7845
6.5677	2.5743	44000	3.2556	18.0844	50.1868	32.8946
6.5463	2.6913	46000	3.2500	17.7437	50.3211	34.6315
6.5322	2.8083	48000	3.2305	17.1847	50.0703	34.3395
6.5154	2.9253	50000	3.2057	17.2239	49.7552	36.0811
6.3802	3.0424	52000	3.2188	17.3479	50.321	34.0429
6.3541	3.1594	54000	3.2132	17.6497	50.4238	33.7462
6.3544	3.2764	56000	3.1990	17.8669	50.3505	33.9664
6.3517	3.3934	58000	3.1796	18.2486	50.8451	33.7462
6.3523	3.5104	60000	3.1908	17.6748	50.4264	36.0811
6.3430	3.6274	62000	3.1701	18.344	51.3013	34.8227
6.3392	3.7444	64000	3.1662	19.001	51.2261	34.8946
6.3319	3.8615	66000	3.1477	17.9256	50.8211	33.8181
6.3206	3.9785	68000	3.1540	18.1694	50.7788	33.7080
6.1742	4.0955	70000	3.1480	18.3839	51.2883	35.6362
6.1884	4.2125	72000	3.1480	18.098	50.4005	34.3013
6.1911	4.3295	74000	3.1502	17.6987	50.2971	35.1912
6.1996	4.4465	76000	3.1366	17.9486	50.7817	34.3013
6.1994	4.5635	78000	3.1193	17.8277	51.1316	35.0429
6.1929	4.6806	80000	3.1103	17.6871	50.8913	34.8899
6.2017	4.7976	82000	3.1208	17.8896	50.4156	34.1147
6.1970	4.9146	84000	3.1206	18.3889	51.2227	33.9282
6.1101	5.0316	86000	3.1182	18.1708	51.1254	35.8946
6.0711	5.1486	88000	3.1072	18.9565	51.7926	34.1912
6.0807	5.2656	90000	3.1035	18.7878	51.35	33.1147
6.0750	5.3826	92000	3.1039	18.4917	51.4391	34.0429
6.0887	5.4996	94000	3.1028	18.7455	51.3372	34.0046
6.1003	5.6167	96000	3.0964	18.9129	51.5941	34.1530
6.0964	5.7337	98000	3.0880	19.1538	51.6121	34.3395
6.1024	5.8507	100000	3.0821	19.0157	51.8337	34.5979
6.0991	5.9677	102000	3.0834	19.127	51.8449	34.0046
5.9529	6.0847	104000	3.0927	18.9951	51.8414	34.0765
5.9803	6.2017	106000	3.0881	19.0848	51.6448	32.5597
5.9968	6.3187	108000	3.0823	19.1365	52.0502	34.4114
5.9981	6.4358	110000	3.0814	18.9723	51.9731	33.4114
6.0142	6.5528	112000	3.0726	18.9398	51.9883	36.0811
6.0171	6.6698	114000	3.0849	18.8193	51.3184	33.7080
6.0197	6.7868	116000	3.0854	18.48	51.0727	33.0382
6.0269	6.9038	118000	3.0658	18.7856	52.0755	33.8563
5.9685	7.0208	120000	3.0732	19.426	51.891	33.8181
5.8872	7.1378	122000	3.0690	19.5588	51.8144	34.3013
5.9167	7.2549	124000	3.0656	19.4649	51.8155	34.7462
5.9220	7.3719	126000	3.0745	19.1167	51.6037	33.7798
5.9333	7.4889	128000	3.0643	19.0742	51.5488	35.0429
5.9365	7.6059	130000	3.0663	19.1219	51.4578	34.3395
5.9513	7.7229	132000	3.0636	18.6926	51.6251	35.2630
5.9437	7.8399	134000	3.0500	19.2458	51.9176	33.1147
5.9538	7.9569	136000	3.0555	18.6213	51.616	33.6315
5.8115	8.0740	138000	3.0731	18.1509	51.3331	35.4739
5.8357	8.1910	140000	3.0618	18.2878	51.361	35.0765
5.8532	8.3080	142000	3.0648	18.6191	51.4876	35.3731
5.8726	8.4250	144000	3.0573	18.8563	51.5889	34.4832
5.8783	8.5420	146000	3.0560	19.4867	52.0309	35.0046
5.8840	8.6590	148000	3.0558	18.917	51.8036	34.1530
5.8898	8.7760	150000	3.0498	19.4045	51.9541	34.3395
5.9013	8.8930	152000	3.0449	19.7852	52.1691	33.5979
5.8745	9.0101	154000	3.0494	18.6923	51.6402	49.1866
5.7689	9.1271	156000	3.0498	19.008	51.7807	35.9861
5.7923	9.2441	158000	3.0531	19.3957	51.8088	35.5411
5.8074	9.3611	160000	3.0464	19.2631	51.8252	35.4403
5.8202	9.4781	162000	3.0508	18.9163	51.8393	36.1344
5.8340	9.5951	164000	3.0412	18.8592	51.9976	34.3013
5.8403	9.7121	166000	3.0363	19.2968	51.9118	36.3163
5.8421	9.8292	168000	3.0288	19.6602	51.959	34.4496
5.8428	9.9462	170000	3.0337	19.1082	52.1028	36.3163
5.7135	10.0632	172000	3.0497	18.5526	51.5155	36.3499
5.7345	10.1802	174000	3.0585	18.7664	51.3669	35.1298
5.7551	10.2972	176000	3.0570	18.9602	51.4472	33.5214
5.7685	10.4142	178000	3.0476	19.1937	51.8817	37.2538
5.7830	10.5312	180000	3.0528	18.9631	51.4576	33.1866
5.7846	10.6483	182000	3.0340	18.6598	51.7472	34.1912
5.7908	10.7653	184000	3.0349	19.782	52.1106	36.4357
7.0077	10.8823	186000	3.2087	18.1469	50.2508	35.4878
8.2229	10.9993	188000	3.5513	13.0171	44.2126	37.3059
11.2715	11.1163	190000	6.1653	0.0819	13.5598	127.0
11.7661	11.2333	192000	6.7296	0.0285	4.2246	6.0
11.5937	11.3503	194000	6.8796	0.006	3.0901	5.0
11.6312	11.4674	196000	6.6285	0.006	3.1418	5.0
11.3841	11.5844	198000	7.0674	0.014	3.0825	4.0
11.3345	11.7014	200000	7.0541	0.0	0.9631	4.0
11.2887	11.8184	202000	7.0348	0.0062	3.9599	7.0
11.2479	11.9354	204000	7.6489	0.0	1.9604	2.0
11.2350	12.0	205104	7.1126	0.0063	3.1944	5.0

Framework versions

• Transformers 5.6.2
• Pytorch 2.5.1+cu121
• Datasets 4.8.5
• Tokenizers 0.22.2