Revision_PhoBert_Lexical_Dataset_46k

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4809
• Accuracy: 0.8827
• F1: 0.8737

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.2770	200	0.3825	0.8319	0.8251
No log	0.5540	400	0.3179	0.8645	0.8554
No log	0.8310	600	0.2937	0.8715	0.8602
0.3815	1.1080	800	0.2846	0.8788	0.8693
0.3815	1.3850	1000	0.2777	0.8832	0.8744
0.3815	1.6620	1200	0.2644	0.8854	0.8774
0.3815	1.9391	1400	0.2586	0.8885	0.8792
0.2836	2.2161	1600	0.2886	0.8860	0.8748
0.2836	2.4931	1800	0.2598	0.8898	0.8820
0.2836	2.7701	2000	0.2503	0.8909	0.8825
0.2380	3.0471	2200	0.2906	0.8753	0.8692
0.2380	3.3241	2400	0.2642	0.8912	0.8826
0.2380	3.6011	2600	0.2667	0.8864	0.8795
0.2380	3.8781	2800	0.2545	0.8943	0.8867
0.2043	4.1551	3000	0.2719	0.8878	0.8788
0.2043	4.4321	3200	0.2644	0.8897	0.8816
0.2043	4.7091	3400	0.2596	0.8953	0.8866
0.2043	4.9861	3600	0.2805	0.8883	0.8801
0.1775	5.2632	3800	0.3051	0.8826	0.8748
0.1775	5.5402	4000	0.2976	0.8888	0.8808
0.1775	5.8172	4200	0.2938	0.8890	0.8803
0.1521	6.0942	4400	0.3383	0.8742	0.8673
0.1521	6.3712	4600	0.3245	0.8862	0.8762
0.1521	6.6482	4800	0.3070	0.8867	0.8780
0.1521	6.9252	5000	0.3315	0.8904	0.8825
0.1304	7.2022	5200	0.3334	0.8878	0.8790
0.1304	7.4792	5400	0.3491	0.8769	0.8698
0.1304	7.7562	5600	0.3576	0.8854	0.8752
0.1118	8.0332	5800	0.3673	0.8856	0.8775
0.1118	8.3102	6000	0.3810	0.8842	0.8764
0.1118	8.5873	6200	0.3614	0.8884	0.8792
0.1118	8.8643	6400	0.3830	0.8876	0.8780
0.0975	9.1413	6600	0.4031	0.8880	0.8791
0.0975	9.4183	6800	0.3994	0.8880	0.8781
0.0975	9.6953	7000	0.4077	0.8866	0.8785
0.0975	9.9723	7200	0.3929	0.8834	0.8754
0.0813	10.2493	7400	0.4131	0.8850	0.8761
0.0813	10.5263	7600	0.4223	0.8803	0.8722
0.0813	10.8033	7800	0.4177	0.8796	0.8716
0.0744	11.0803	8000	0.4272	0.8831	0.8742
0.0744	11.3573	8200	0.4248	0.8852	0.8763
0.0744	11.6343	8400	0.4415	0.8828	0.8741
0.0744	11.9114	8600	0.4364	0.8840	0.8750
0.0637	12.1884	8800	0.4380	0.8833	0.8741
0.0637	12.4654	9000	0.4627	0.8808	0.8719
0.0637	12.7424	9200	0.4487	0.8838	0.8746
0.0588	13.0194	9400	0.4635	0.8833	0.8741
0.0588	13.2964	9600	0.4712	0.8807	0.8724
0.0588	13.5734	9800	0.4858	0.8799	0.8713
0.0588	13.8504	10000	0.4723	0.8823	0.8733
0.0489	14.1274	10200	0.4840	0.8802	0.8713
0.0489	14.4044	10400	0.4806	0.8821	0.8732
0.0489	14.6814	10600	0.4771	0.8833	0.8738
0.0489	14.9584	10800	0.4809	0.8827	0.8737

Framework versions

• Transformers 5.3.0
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.2