Hugging Face's logo

gechim
/
PhoBert_70KURL

Text Classification
Transformers
Safetensors
roberta
Generated from Trainer
Inference Endpoints
Model card Files Community
PhoBert_70KURL / README.md
gechim's picture
gechim
End of training
5a9a7fa verified
preview code
|
raw
history blame
4.55 kB
metadata
base_model: vinai/phobert-base-v2
tags:
  - generated_from_trainer
metrics:
  - accuracy
  - f1
model-index:
  - name: PhoBert_70KURL
    results: []

PhoBert_70KURL

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3781
  • Accuracy: 0.9371
  • F1: 0.9417

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 64
  • eval_batch_size: 64
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 128
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 2150
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss Accuracy F1
No log 0.4651 200 0.3374 0.9302 0.9233
No log 0.9302 400 0.1954 0.9353 0.9383
No log 1.3953 600 0.1273 0.9598 0.9609
No log 1.8605 800 0.1601 0.9423 0.9461
0.3263 2.3256 1000 0.1059 0.9666 0.9675
0.3263 2.7907 1200 0.0832 0.9742 0.9746
0.3263 3.2558 1400 0.1073 0.9621 0.9637
0.3263 3.7209 1600 0.1053 0.9644 0.9658
0.1315 4.1860 1800 0.0994 0.9653 0.9663
0.1315 4.6512 2000 0.1109 0.9617 0.9633
0.1315 5.1163 2200 0.1350 0.9491 0.9523
0.1315 5.5814 2400 0.1204 0.9543 0.9567
0.0959 6.0465 2600 0.2663 0.8906 0.9034
0.0959 6.5116 2800 0.1309 0.9510 0.9538
0.0959 6.9767 3000 0.2027 0.9279 0.9340
0.0959 7.4419 3200 0.1277 0.9509 0.9535
0.0959 7.9070 3400 0.1892 0.9473 0.9507
0.0689 8.3721 3600 0.2090 0.9350 0.9396
0.0689 8.8372 3800 0.1247 0.9579 0.9596
0.0689 9.3023 4000 0.2984 0.9132 0.9216
0.0689 9.7674 4200 0.2277 0.9352 0.9401
0.0483 10.2326 4400 0.2595 0.9266 0.9328
0.0483 10.6977 4600 0.2725 0.9249 0.9313
0.0483 11.1628 4800 0.2483 0.9341 0.9391
0.0483 11.6279 5000 0.2195 0.9442 0.9478
0.0379 12.0930 5200 0.7268 0.8450 0.8672
0.0379 12.5581 5400 0.2236 0.9503 0.9530
0.0379 13.0233 5600 0.2610 0.9438 0.9474
0.0379 13.4884 5800 0.3155 0.9366 0.9412
0.0379 13.9535 6000 0.3225 0.9374 0.9418
0.0284 14.4186 6200 0.4604 0.9158 0.9238
0.0284 14.8837 6400 0.3269 0.9372 0.9417
0.0284 15.3488 6600 0.3419 0.9401 0.9442
0.0284 15.8140 6800 0.2869 0.9465 0.9497
0.0213 16.2791 7000 0.3623 0.9364 0.9410
0.0213 16.7442 7200 0.2880 0.9433 0.9466
0.0213 17.2093 7400 0.3010 0.9454 0.9484
0.0213 17.6744 7600 0.3719 0.9344 0.9391
0.0165 18.1395 7800 0.3160 0.9448 0.9481
0.0165 18.6047 8000 0.3767 0.9388 0.9431
0.0165 19.0698 8200 0.3702 0.9383 0.9426
0.0165 19.5349 8400 0.3998 0.9343 0.9393
0.0121 20.0 8600 0.3781 0.9371 0.9417

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.1.2
  • Datasets 2.19.2
  • Tokenizers 0.19.1