PhoBertLexical-finetuned_70KURL_daydu

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2260
• Accuracy: 0.9670
• F1: 0.9671

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.2326	200	0.1641	0.9425	0.9417
No log	0.4651	400	0.1297	0.9576	0.9580
No log	0.6977	600	0.1202	0.9590	0.9595
No log	0.9302	800	0.1118	0.9645	0.9647
0.1684	1.1628	1000	0.1167	0.9611	0.9615
0.1684	1.3953	1200	0.1192	0.9615	0.9619
0.1684	1.6279	1400	0.1101	0.9662	0.9665
0.1684	1.8605	1600	0.1054	0.9677	0.9678
0.1023	2.0930	1800	0.1181	0.9650	0.9651
0.1023	2.3256	2000	0.1011	0.9703	0.9704
0.1023	2.5581	2200	0.1091	0.9687	0.9687
0.1023	2.7907	2400	0.1048	0.9678	0.9678
0.0844	3.0233	2600	0.1147	0.9681	0.9681
0.0844	3.2558	2800	0.1254	0.9659	0.9661
0.0844	3.4884	3000	0.1007	0.9701	0.9702
0.0844	3.7209	3200	0.1102	0.9654	0.9657
0.0844	3.9535	3400	0.1134	0.9687	0.9689
0.0696	4.1860	3600	0.1093	0.9666	0.9668
0.0696	4.4186	3800	0.1081	0.9672	0.9674
0.0696	4.6512	4000	0.1141	0.9690	0.9691
0.0696	4.8837	4200	0.1263	0.9647	0.9650
0.0591	5.1163	4400	0.1361	0.9651	0.9653
0.0591	5.3488	4600	0.1291	0.9669	0.9669
0.0591	5.5814	4800	0.1378	0.9635	0.9634
0.0591	5.8140	5000	0.1402	0.9628	0.9632
0.0468	6.0465	5200	0.1280	0.9707	0.9707
0.0468	6.2791	5400	0.1357	0.9690	0.9690
0.0468	6.5116	5600	0.1311	0.9696	0.9697
0.0468	6.7442	5800	0.1419	0.9677	0.9678
0.0468	6.9767	6000	0.1252	0.9702	0.9704
0.0413	7.2093	6200	0.1386	0.9656	0.9658
0.0413	7.4419	6400	0.1673	0.9642	0.9645
0.0413	7.6744	6600	0.1526	0.9646	0.9646
0.0413	7.9070	6800	0.1427	0.9701	0.9702
0.0341	8.1395	7000	0.1487	0.9687	0.9688
0.0341	8.3721	7200	0.1524	0.9663	0.9664
0.0341	8.6047	7400	0.1489	0.9680	0.9683
0.0341	8.8372	7600	0.1419	0.9698	0.9699
0.0286	9.0698	7800	0.1817	0.9652	0.9654
0.0286	9.3023	8000	0.1744	0.9696	0.9696
0.0286	9.5349	8200	0.1645	0.9692	0.9693
0.0286	9.7674	8400	0.1624	0.9695	0.9697
0.0235	10.0	8600	0.1638	0.9659	0.9660
0.0235	10.2326	8800	0.1889	0.9652	0.9652
0.0235	10.4651	9000	0.1929	0.9645	0.9645
0.0235	10.6977	9200	0.1802	0.9674	0.9676
0.0235	10.9302	9400	0.1879	0.9647	0.9649
0.0199	11.1628	9600	0.1800	0.9682	0.9682
0.0199	11.3953	9800	0.2004	0.9652	0.9652
0.0199	11.6279	10000	0.1883	0.9671	0.9672
0.0199	11.8605	10200	0.1840	0.9692	0.9694
0.0178	12.0930	10400	0.1986	0.9660	0.9660
0.0178	12.3256	10600	0.1970	0.9656	0.9658
0.0178	12.5581	10800	0.1952	0.9682	0.9683
0.0178	12.7907	11000	0.1931	0.9661	0.9660
0.0152	13.0233	11200	0.1962	0.9655	0.9656
0.0152	13.2558	11400	0.2098	0.9662	0.9662
0.0152	13.4884	11600	0.2088	0.9655	0.9654
0.0152	13.7209	11800	0.2031	0.9658	0.9658
0.0152	13.9535	12000	0.2044	0.9666	0.9666
0.0118	14.1860	12200	0.2025	0.9674	0.9676
0.0118	14.4186	12400	0.2159	0.9656	0.9656
0.0118	14.6512	12600	0.2098	0.9665	0.9666
0.0118	14.8837	12800	0.1995	0.9676	0.9677
0.0105	15.1163	13000	0.2108	0.9673	0.9674
0.0105	15.3488	13200	0.2233	0.9652	0.9652
0.0105	15.5814	13400	0.2219	0.9654	0.9654
0.0105	15.8140	13600	0.2251	0.9665	0.9665
0.0094	16.0465	13800	0.2199	0.9660	0.9661
0.0094	16.2791	14000	0.2344	0.9652	0.9652
0.0094	16.5116	14200	0.2260	0.9669	0.9670
0.0094	16.7442	14400	0.2123	0.9674	0.9675
0.0094	16.9767	14600	0.2129	0.9678	0.9679
0.0072	17.2093	14800	0.2134	0.9676	0.9677
0.0072	17.4419	15000	0.2235	0.9663	0.9663
0.0072	17.6744	15200	0.2199	0.9672	0.9672
0.0072	17.9070	15400	0.2280	0.9654	0.9655
0.0069	18.1395	15600	0.2242	0.9670	0.9670
0.0069	18.3721	15800	0.2188	0.9679	0.9680
0.0069	18.6047	16000	0.2202	0.9676	0.9676
0.0069	18.8372	16200	0.2261	0.9664	0.9664
0.0062	19.0698	16400	0.2264	0.9667	0.9668
0.0062	19.3023	16600	0.2238	0.9666	0.9667
0.0062	19.5349	16800	0.2243	0.9669	0.9670
0.0062	19.7674	17000	0.2262	0.9671	0.9671
0.0055	20.0	17200	0.2260	0.9670	0.9671

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.19.1