phobert-base-v2-70k-khduoi

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6366
• Accuracy: 0.9152
• F1: 0.9155

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.2909	500	0.2524	0.8975	0.8969
No log	0.5817	1000	0.2447	0.9009	0.8999
No log	0.8726	1500	0.2319	0.9018	0.9015
0.263	1.1635	2000	0.2539	0.9060	0.9063
0.263	1.4543	2500	0.2433	0.9017	0.9029
0.263	1.7452	3000	0.2358	0.9100	0.9099
0.2084	2.0361	3500	0.2755	0.9044	0.9059
0.2084	2.3269	4000	0.2547	0.9102	0.9100
0.2084	2.6178	4500	0.2223	0.9109	0.9125
0.2084	2.9087	5000	0.2189	0.9150	0.9150
0.1729	3.1995	5500	0.2825	0.9101	0.9112
0.1729	3.4904	6000	0.2663	0.9110	0.9121
0.1729	3.7813	6500	0.2367	0.9157	0.9165
0.1448	4.0721	7000	0.2891	0.9118	0.9121
0.1448	4.3630	7500	0.3180	0.9042	0.9060
0.1448	4.6539	8000	0.2441	0.9117	0.9126
0.1448	4.9447	8500	0.2638	0.9142	0.9145
0.1234	5.2356	9000	0.3499	0.9130	0.9141
0.1234	5.5265	9500	0.3086	0.9123	0.9135
0.1234	5.8173	10000	0.3203	0.9141	0.9140
0.1033	6.1082	10500	0.3234	0.9170	0.9173
0.1033	6.3991	11000	0.3367	0.9095	0.9105
0.1033	6.6899	11500	0.3402	0.9157	0.9159
0.1033	6.9808	12000	0.3843	0.9107	0.9111
0.0904	7.2717	12500	0.3559	0.9182	0.9182
0.0904	7.5625	13000	0.3646	0.9079	0.9096
0.0904	7.8534	13500	0.3392	0.9130	0.9137
0.0785	8.1443	14000	0.4064	0.9155	0.9164
0.0785	8.4351	14500	0.4013	0.9126	0.9135
0.0785	8.7260	15000	0.4351	0.9124	0.9135
0.0701	9.0169	15500	0.4190	0.9158	0.9161
0.0701	9.3077	16000	0.4567	0.9116	0.9126
0.0701	9.5986	16500	0.4230	0.9147	0.9147
0.0701	9.8895	17000	0.3956	0.9148	0.9150
0.0599	10.1803	17500	0.4854	0.9133	0.9135
0.0599	10.4712	18000	0.4958	0.9156	0.9158
0.0599	10.7621	18500	0.4552	0.9148	0.9146
0.0536	11.0529	19000	0.4678	0.9160	0.9163
0.0536	11.3438	19500	0.4802	0.9142	0.9135
0.0536	11.6347	20000	0.5360	0.9130	0.9133
0.0536	11.9255	20500	0.5305	0.9133	0.9137
0.0464	12.2164	21000	0.5413	0.9115	0.9122
0.0464	12.5073	21500	0.4867	0.9150	0.9155
0.0464	12.7981	22000	0.5100	0.9147	0.9153
0.0446	13.0890	22500	0.5750	0.9161	0.9157
0.0446	13.3799	23000	0.5742	0.9174	0.9172
0.0446	13.6707	23500	0.5790	0.9142	0.9146
0.0446	13.9616	24000	0.5476	0.9151	0.9150
0.0374	14.2525	24500	0.5621	0.9160	0.9163
0.0374	14.5433	25000	0.5633	0.9140	0.9146
0.0374	14.8342	25500	0.5496	0.9148	0.9152
0.0341	15.1251	26000	0.5869	0.9138	0.9142
0.0341	15.4159	26500	0.5901	0.9142	0.9141
0.0341	15.7068	27000	0.5548	0.9154	0.9158
0.0303	15.9977	27500	0.5832	0.9141	0.9136
0.0303	16.2885	28000	0.6070	0.9148	0.9157
0.0303	16.5794	28500	0.6208	0.9159	0.9162
0.0303	16.8703	29000	0.6134	0.9137	0.9143
0.0273	17.1611	29500	0.6021	0.9166	0.9168
0.0273	17.4520	30000	0.6063	0.9150	0.9153
0.0273	17.7429	30500	0.5942	0.9135	0.9142
0.0254	18.0337	31000	0.6073	0.9150	0.9155
0.0254	18.3246	31500	0.6304	0.9165	0.9167
0.0254	18.6155	32000	0.6121	0.9155	0.9157
0.0254	18.9063	32500	0.6087	0.9153	0.9156
0.0221	19.1972	33000	0.6234	0.9147	0.9151
0.0221	19.4881	33500	0.6312	0.9145	0.9149
0.0221	19.7789	34000	0.6366	0.9152	0.9155

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1