metadata-cls-gemini

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0111
• Accuracy: 0.8183
• F1: 0.7722

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.9411	1.6949	200	0.5887	0.8002	0.6394
0.4312	3.3898	400	0.5675	0.8097	0.7790
0.2745	5.0847	600	0.5685	0.8164	0.7767
0.2199	6.7797	800	0.6602	0.8116	0.7998
0.1536	8.4746	1000	0.8254	0.8069	0.7707
0.1021	10.1695	1200	0.8674	0.8097	0.7889
0.0928	11.8644	1400	0.8696	0.8145	0.7789
0.0692	13.5593	1600	0.9211	0.8259	0.7926
0.049	15.2542	1800	0.9573	0.8221	0.7899
0.0391	16.9492	2000	1.0388	0.8154	0.7713
0.0372	18.6441	2200	1.0111	0.8183	0.7722

Framework versions

• Transformers 4.42.4
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.19.1