Justice0893's picture
End of training
3bf8640 verified
metadata
license: apache-2.0
base_model: google-bert/bert-base-multilingual-uncased
tags:
  - generated_from_trainer
metrics:
  - precision
  - recall
  - f1
  - accuracy
model-index:
  - name: bert-multi-base-uncased-finetuned-pos-ky
    results: []

bert-multi-base-uncased-finetuned-pos-ky

This model is a fine-tuned version of google-bert/bert-base-multilingual-uncased on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0007
  • Precision: 0.8230
  • Recall: 0.8280
  • F1: 0.8255
  • Accuracy: 0.8850

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 50

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
No log 1.0 40 0.6248 0.6903 0.6708 0.6804 0.8350
No log 2.0 80 0.4938 0.7555 0.7555 0.7555 0.8626
No log 3.0 120 0.4931 0.7939 0.7948 0.7944 0.8764
No log 4.0 160 0.4948 0.7776 0.8034 0.7903 0.8735
No log 5.0 200 0.4744 0.8102 0.8231 0.8166 0.8850
No log 6.0 240 0.5698 0.8042 0.8071 0.8056 0.8787
No log 7.0 280 0.5787 0.7878 0.8120 0.7998 0.8758
No log 8.0 320 0.6357 0.7841 0.8120 0.7978 0.8718
No log 9.0 360 0.6359 0.8265 0.8366 0.8315 0.8879
No log 10.0 400 0.6735 0.8048 0.8305 0.8174 0.8827
No log 11.0 440 0.7243 0.8087 0.8206 0.8146 0.8804
No log 12.0 480 0.7430 0.8133 0.8292 0.8212 0.8827
0.244 13.0 520 0.7097 0.8058 0.8206 0.8131 0.8810
0.244 14.0 560 0.7885 0.8152 0.8182 0.8167 0.8787
0.244 15.0 600 0.7925 0.8082 0.8231 0.8156 0.8827
0.244 16.0 640 0.7850 0.8270 0.8280 0.8275 0.8879
0.244 17.0 680 0.7881 0.8162 0.8292 0.8227 0.8850
0.244 18.0 720 0.8490 0.8168 0.8219 0.8194 0.8810
0.244 19.0 760 0.8470 0.8163 0.8243 0.8203 0.8815
0.244 20.0 800 0.8792 0.8007 0.8194 0.8100 0.8752
0.244 21.0 840 0.9056 0.8084 0.8243 0.8163 0.8769
0.244 22.0 880 0.9099 0.8152 0.8292 0.8222 0.8827
0.244 23.0 920 0.8455 0.8166 0.8317 0.8241 0.8844
0.244 24.0 960 0.9336 0.8140 0.8170 0.8155 0.8775
0.0193 25.0 1000 0.9462 0.8145 0.8145 0.8145 0.8787
0.0193 26.0 1040 0.9457 0.8200 0.8170 0.8185 0.8792
0.0193 27.0 1080 0.9312 0.8177 0.8268 0.8222 0.8798
0.0193 28.0 1120 0.9553 0.8235 0.8194 0.8214 0.8833
0.0193 29.0 1160 0.9450 0.8207 0.8268 0.8237 0.8821
0.0193 30.0 1200 0.9337 0.8335 0.8366 0.8351 0.8896
0.0193 31.0 1240 0.9476 0.8203 0.8354 0.8278 0.8861
0.0193 32.0 1280 0.9443 0.8182 0.8292 0.8237 0.8838
0.0193 33.0 1320 0.9713 0.8197 0.8268 0.8232 0.8844
0.0193 34.0 1360 0.9751 0.8210 0.8280 0.8245 0.8821
0.0193 35.0 1400 0.9850 0.8129 0.8219 0.8173 0.8815
0.0193 36.0 1440 0.9546 0.8182 0.8292 0.8237 0.8821
0.0193 37.0 1480 0.9713 0.8216 0.8317 0.8266 0.8844
0.0049 38.0 1520 0.9696 0.8234 0.8305 0.8269 0.8850
0.0049 39.0 1560 0.9722 0.8222 0.8354 0.8288 0.8861
0.0049 40.0 1600 0.9705 0.8273 0.8354 0.8313 0.8879
0.0049 41.0 1640 0.9777 0.8190 0.8280 0.8235 0.8838
0.0049 42.0 1680 0.9841 0.8167 0.8268 0.8217 0.8850
0.0049 43.0 1720 0.9799 0.8234 0.8305 0.8269 0.8873
0.0049 44.0 1760 0.9785 0.8248 0.8329 0.8289 0.8873
0.0049 45.0 1800 0.9863 0.8205 0.8256 0.8230 0.8856
0.0049 46.0 1840 0.9860 0.8278 0.8329 0.8304 0.8879
0.0049 47.0 1880 0.9870 0.8278 0.8329 0.8304 0.8879
0.0049 48.0 1920 0.9896 0.8278 0.8329 0.8304 0.8879
0.0049 49.0 1960 0.9997 0.8230 0.8280 0.8255 0.8850
0.0022 50.0 2000 1.0007 0.8230 0.8280 0.8255 0.8850

Framework versions

  • Transformers 4.34.1
  • Pytorch 2.2.1+cu118
  • Datasets 2.14.6
  • Tokenizers 0.14.1