Hugging Face's logo

LazzeKappa
/
BERT_B01

Token Classification
Transformers
PyTorch
distilbert
Generated from Trainer
Inference Endpoints
Model card Files Community
BERT_B01 / README.md
LazzeKappa's picture
LazzeKappa
End of training
52c2173
preview code
|
raw
history blame contribute delete
4.22 kB
metadata
license: apache-2.0
base_model: distilbert-base-multilingual-cased
tags:
  - generated_from_trainer
metrics:
  - precision
  - recall
  - f1
  - accuracy
model-index:
  - name: BERT_B01
    results: []

BERT_B01

This model is a fine-tuned version of distilbert-base-multilingual-cased on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6902
  • Precision: 0.6636
  • Recall: 0.6946
  • F1: 0.6788
  • Accuracy: 0.8776

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
1.0749 1.0 47 0.9390 0.4203 0.3480 0.3807 0.7831
0.6411 2.0 94 0.6110 0.5948 0.5392 0.5657 0.8452
0.4786 3.0 141 0.5279 0.6784 0.6121 0.6435 0.8630
0.3573 4.0 188 0.4972 0.6462 0.6382 0.6422 0.8691
0.2824 5.0 235 0.4868 0.6339 0.6479 0.6408 0.8689
0.2434 6.0 282 0.4970 0.6490 0.6561 0.6525 0.8715
0.1854 7.0 329 0.5004 0.6578 0.6795 0.6685 0.8721
0.1336 8.0 376 0.5091 0.6508 0.6768 0.6635 0.8736
0.1186 9.0 423 0.5437 0.6340 0.6768 0.6547 0.8739
0.103 10.0 470 0.5482 0.6570 0.6823 0.6694 0.8771
0.0799 11.0 517 0.5620 0.6444 0.6781 0.6609 0.8752
0.1045 12.0 564 0.5812 0.6557 0.6864 0.6707 0.8760
0.0562 13.0 611 0.6009 0.6667 0.6850 0.6757 0.8780
0.0637 14.0 658 0.5937 0.6707 0.6946 0.6824 0.8780
0.0657 15.0 705 0.6017 0.6788 0.6946 0.6866 0.8789
0.0371 16.0 752 0.6227 0.6858 0.6905 0.6881 0.8776
0.0389 17.0 799 0.6476 0.6499 0.6919 0.6702 0.8767
0.0461 18.0 846 0.6667 0.6556 0.7043 0.6790 0.8786
0.0377 19.0 893 0.6515 0.6788 0.6919 0.6853 0.8793
0.0364 20.0 940 0.6480 0.6791 0.7015 0.6901 0.8784
0.0383 21.0 987 0.6646 0.6719 0.7070 0.6890 0.8802
0.0173 22.0 1034 0.6724 0.6750 0.7029 0.6887 0.8793
0.0613 23.0 1081 0.6779 0.6580 0.6988 0.6778 0.8778
0.0578 24.0 1128 0.6847 0.6592 0.6864 0.6725 0.8767
0.0201 25.0 1175 0.6714 0.6706 0.7001 0.6851 0.8791
0.022 26.0 1222 0.6874 0.6667 0.6878 0.6770 0.8782
0.0298 27.0 1269 0.6926 0.6675 0.6960 0.6815 0.8789
0.03 28.0 1316 0.6895 0.6662 0.6974 0.6815 0.8784
0.0216 29.0 1363 0.6888 0.6636 0.6946 0.6788 0.8780
0.0236 30.0 1410 0.6902 0.6636 0.6946 0.6788 0.8776

Framework versions

  • Transformers 4.33.3
  • Pytorch 2.0.1+cu117
  • Datasets 2.14.4
  • Tokenizers 0.13.3