bert-30M-uncased-classification-CMC-fqa-new-100e

This model is a fine-tuned version of vietgpt/bert-30M-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0430
• Accuracy: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	20	3.4282	0.0323
No log	2.0	40	3.4133	0.0323
No log	3.0	60	3.4014	0.0645
No log	4.0	80	3.3872	0.1613
No log	5.0	100	3.3703	0.2581
No log	6.0	120	3.3503	0.2903
No log	7.0	140	3.3193	0.5161
No log	8.0	160	3.2707	0.5484
No log	9.0	180	3.1889	0.5484
No log	10.0	200	3.0520	0.4839
No log	11.0	220	2.8649	0.3548
No log	12.0	240	2.6818	0.4839
No log	13.0	260	2.5264	0.5161
No log	14.0	280	2.3816	0.5806
No log	15.0	300	2.2296	0.7419
No log	16.0	320	2.0829	0.7097
No log	17.0	340	1.9155	0.8387
No log	18.0	360	1.7785	0.8065
No log	19.0	380	1.6237	0.8710
No log	20.0	400	1.4944	0.9355
No log	21.0	420	1.3666	0.9355
No log	22.0	440	1.2578	0.9355
No log	23.0	460	1.1628	0.9677
No log	24.0	480	1.0483	0.9355
2.5506	25.0	500	0.9724	0.9677
2.5506	26.0	520	0.9024	0.9355
2.5506	27.0	540	0.8159	0.9677
2.5506	28.0	560	0.7614	0.9355
2.5506	29.0	580	0.6883	0.9677
2.5506	30.0	600	0.6365	0.9677
2.5506	31.0	620	0.5864	0.9677
2.5506	32.0	640	0.5423	1.0
2.5506	33.0	660	0.4985	1.0
2.5506	34.0	680	0.4666	1.0
2.5506	35.0	700	0.4262	1.0
2.5506	36.0	720	0.3942	1.0
2.5506	37.0	740	0.3629	1.0
2.5506	38.0	760	0.3336	1.0
2.5506	39.0	780	0.3079	1.0
2.5506	40.0	800	0.2836	1.0
2.5506	41.0	820	0.2579	1.0
2.5506	42.0	840	0.2381	1.0
2.5506	43.0	860	0.2167	1.0
2.5506	44.0	880	0.2053	1.0
2.5506	45.0	900	0.1846	1.0
2.5506	46.0	920	0.1720	1.0
2.5506	47.0	940	0.1579	1.0
2.5506	48.0	960	0.1452	1.0
2.5506	49.0	980	0.1366	1.0
0.4668	50.0	1000	0.1262	1.0
0.4668	51.0	1020	0.1165	1.0
0.4668	52.0	1040	0.1098	1.0
0.4668	53.0	1060	0.1042	1.0
0.4668	54.0	1080	0.1022	1.0
0.4668	55.0	1100	0.0956	1.0
0.4668	56.0	1120	0.0916	1.0
0.4668	57.0	1140	0.0875	1.0
0.4668	58.0	1160	0.0819	1.0
0.4668	59.0	1180	0.0785	1.0
0.4668	60.0	1200	0.0761	1.0
0.4668	61.0	1220	0.0740	1.0
0.4668	62.0	1240	0.0709	1.0
0.4668	63.0	1260	0.0692	1.0
0.4668	64.0	1280	0.0672	1.0
0.4668	65.0	1300	0.0654	1.0
0.4668	66.0	1320	0.0653	1.0
0.4668	67.0	1340	0.0628	1.0
0.4668	68.0	1360	0.0618	1.0
0.4668	69.0	1380	0.0608	1.0
0.4668	70.0	1400	0.0593	1.0
0.4668	71.0	1420	0.0582	1.0
0.4668	72.0	1440	0.0579	1.0
0.4668	73.0	1460	0.0558	1.0
0.4668	74.0	1480	0.0540	1.0
0.0861	75.0	1500	0.0528	1.0
0.0861	76.0	1520	0.0522	1.0
0.0861	77.0	1540	0.0516	1.0
0.0861	78.0	1560	0.0502	1.0
0.0861	79.0	1580	0.0494	1.0
0.0861	80.0	1600	0.0490	1.0
0.0861	81.0	1620	0.0484	1.0
0.0861	82.0	1640	0.0480	1.0
0.0861	83.0	1660	0.0479	1.0
0.0861	84.0	1680	0.0476	1.0
0.0861	85.0	1700	0.0470	1.0
0.0861	86.0	1720	0.0465	1.0
0.0861	87.0	1740	0.0455	1.0
0.0861	88.0	1760	0.0449	1.0
0.0861	89.0	1780	0.0444	1.0
0.0861	90.0	1800	0.0444	1.0
0.0861	91.0	1820	0.0445	1.0
0.0861	92.0	1840	0.0441	1.0
0.0861	93.0	1860	0.0439	1.0
0.0861	94.0	1880	0.0438	1.0
0.0861	95.0	1900	0.0436	1.0
0.0861	96.0	1920	0.0434	1.0
0.0861	97.0	1940	0.0433	1.0
0.0861	98.0	1960	0.0431	1.0
0.0861	99.0	1980	0.0430	1.0
0.0493	100.0	2000	0.0430	1.0

Framework versions

• Transformers 4.37.1
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1