Hugging Face's logo

raulgdp
/
roberta-large-ner-qlorafinetune-runs

PEFT
TensorBoard
Safetensors
Generated from Trainer
Model card Files Metrics Community
Edit model card

roberta-large-ner-qlorafinetune-runs

This model is a fine-tuned version of FacebookAI/xlm-roberta-large on the conll2002 dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0853
  • Precision: 0.8767
  • Recall: 0.8819
  • F1: 0.8793
  • Accuracy: 0.9814

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0004
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • training_steps: 1820
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
0.9933 0.0766 20 0.4461 0.1132 0.0565 0.0754 0.8691
0.2822 0.1533 40 0.2098 0.5212 0.6337 0.5720 0.9451
0.146 0.2299 60 0.1251 0.7357 0.7652 0.7502 0.9657
0.0986 0.3065 80 0.1222 0.7928 0.8134 0.8030 0.9720
0.1087 0.3831 100 0.1080 0.7738 0.7953 0.7844 0.9714
0.0874 0.4598 120 0.0974 0.8118 0.8205 0.8161 0.9742
0.0876 0.5364 140 0.0791 0.8374 0.8532 0.8452 0.9791
0.0909 0.6130 160 0.0831 0.8333 0.8486 0.8408 0.9779
0.0835 0.6897 180 0.0802 0.8452 0.8479 0.8465 0.9773
0.0706 0.7663 200 0.0879 0.8234 0.8359 0.8296 0.9757
0.0634 0.8429 220 0.0812 0.8368 0.8504 0.8435 0.9782
0.0548 0.9195 240 0.0918 0.8067 0.8235 0.8150 0.9741
0.0602 0.9962 260 0.0903 0.8328 0.8458 0.8393 0.9762
0.0528 1.0728 280 0.0878 0.8125 0.8456 0.8287 0.9760
0.0513 1.1494 300 0.0991 0.8269 0.8332 0.8300 0.9754
0.0565 1.2261 320 0.0805 0.8575 0.8683 0.8629 0.9788
0.0514 1.3027 340 0.0864 0.8104 0.8359 0.8230 0.9751
0.0499 1.3793 360 0.1011 0.8012 0.8194 0.8102 0.9740
0.0585 1.4559 380 0.0945 0.8179 0.8258 0.8219 0.9739
0.0489 1.5326 400 0.0887 0.8177 0.8389 0.8282 0.9755
0.0534 1.6092 420 0.0747 0.8283 0.8523 0.8401 0.9784
0.0612 1.6858 440 0.0743 0.8336 0.8532 0.8433 0.9775
0.0531 1.7625 460 0.0722 0.8632 0.8699 0.8666 0.9802
0.0453 1.8391 480 0.0752 0.8346 0.8502 0.8423 0.9776
0.0504 1.9157 500 0.0754 0.8373 0.8562 0.8466 0.9783
0.0468 1.9923 520 0.0734 0.8534 0.8612 0.8573 0.9794
0.0573 2.0690 540 0.0728 0.8412 0.8653 0.8531 0.9785
0.0344 2.1456 560 0.0741 0.8549 0.8690 0.8619 0.9794
0.0303 2.2222 580 0.0767 0.8428 0.8658 0.8541 0.9783
0.0421 2.2989 600 0.0753 0.8370 0.8587 0.8477 0.9786
0.0335 2.3755 620 0.0762 0.8523 0.8594 0.8558 0.9788
0.0376 2.4521 640 0.0735 0.8511 0.8732 0.8620 0.9797
0.0366 2.5287 660 0.0764 0.8647 0.8619 0.8633 0.9792
0.0351 2.6054 680 0.0771 0.8711 0.8713 0.8712 0.9795
0.0389 2.6820 700 0.0734 0.8563 0.8640 0.8601 0.9788
0.037 2.7586 720 0.0708 0.8475 0.8660 0.8567 0.9804
0.0434 2.8352 740 0.0719 0.8441 0.8587 0.8513 0.9783
0.0399 2.9119 760 0.0681 0.8572 0.8773 0.8671 0.9807
0.0381 2.9885 780 0.0679 0.8615 0.8759 0.8686 0.9809
0.0343 3.0651 800 0.0680 0.8593 0.8844 0.8717 0.9816
0.0283 3.1418 820 0.0708 0.8760 0.8780 0.8770 0.9812
0.0286 3.2184 840 0.0757 0.8688 0.8796 0.8742 0.9802
0.0279 3.2950 860 0.0721 0.875 0.8798 0.8774 0.9810
0.0274 3.3716 880 0.0717 0.8651 0.8766 0.8708 0.9796
0.0296 3.4483 900 0.0697 0.8558 0.8782 0.8669 0.9796
0.0355 3.5249 920 0.0624 0.8741 0.8856 0.8798 0.9820
0.0223 3.6015 940 0.0751 0.8693 0.8803 0.8748 0.9814
0.0265 3.6782 960 0.0785 0.8694 0.875 0.8722 0.9798
0.0261 3.7548 980 0.0701 0.8768 0.8830 0.8799 0.9808
0.0326 3.8314 1000 0.0693 0.8708 0.8842 0.8774 0.9820
0.028 3.9080 1020 0.0719 0.8579 0.8752 0.8665 0.9795
0.032 3.9847 1040 0.0726 0.8766 0.8801 0.8783 0.9809
0.0176 4.0613 1060 0.0766 0.8753 0.8821 0.8787 0.9809
0.0218 4.1379 1080 0.0808 0.8681 0.8805 0.8743 0.9810
0.0163 4.2146 1100 0.0833 0.8682 0.875 0.8716 0.9803
0.0201 4.2912 1120 0.0882 0.8702 0.8752 0.8727 0.9802
0.0193 4.3678 1140 0.0838 0.8678 0.8778 0.8727 0.9801
0.023 4.4444 1160 0.0855 0.8648 0.8761 0.8704 0.9802
0.0241 4.5211 1180 0.0793 0.8585 0.8768 0.8676 0.9795
0.0224 4.5977 1200 0.0806 0.8792 0.8863 0.8827 0.9814
0.0263 4.6743 1220 0.0737 0.8674 0.8775 0.8724 0.9816
0.0266 4.7510 1240 0.0824 0.8501 0.8706 0.8603 0.9794
0.0212 4.8276 1260 0.0770 0.8657 0.8709 0.8683 0.9811
0.0228 4.9042 1280 0.0724 0.8673 0.8771 0.8722 0.9810
0.0216 4.9808 1300 0.0703 0.8747 0.8842 0.8794 0.9817
0.0155 5.0575 1320 0.0782 0.8799 0.8904 0.8851 0.9824
0.015 5.1341 1340 0.0792 0.8822 0.8842 0.8832 0.9822
0.0159 5.2107 1360 0.0787 0.8771 0.8872 0.8821 0.9818
0.0182 5.2874 1380 0.0766 0.8767 0.8821 0.8794 0.9816
0.0147 5.3640 1400 0.0756 0.8753 0.8778 0.8765 0.9814
0.0182 5.4406 1420 0.0813 0.8755 0.8794 0.8775 0.9809
0.0163 5.5172 1440 0.0822 0.8823 0.8835 0.8829 0.9815
0.0172 5.5939 1460 0.0819 0.8767 0.8810 0.8789 0.9813
0.015 5.6705 1480 0.0777 0.8714 0.8796 0.8755 0.9811
0.0108 5.7471 1500 0.0801 0.8801 0.8835 0.8818 0.9814
0.0169 5.8238 1520 0.0798 0.8793 0.8853 0.8823 0.9818
0.016 5.9004 1540 0.0832 0.8723 0.8771 0.8747 0.9810
0.0128 5.9770 1560 0.0829 0.8730 0.8780 0.8755 0.9809
0.0101 6.0536 1580 0.0834 0.8711 0.8773 0.8742 0.9808
0.0124 6.1303 1600 0.0831 0.8710 0.8796 0.8753 0.9808
0.012 6.2069 1620 0.0855 0.8702 0.8761 0.8731 0.9805
0.0124 6.2835 1640 0.0822 0.8719 0.8805 0.8762 0.9811
0.0126 6.3602 1660 0.0823 0.8762 0.8830 0.8796 0.9812
0.0123 6.4368 1680 0.0814 0.8769 0.8853 0.8811 0.9812
0.0098 6.5134 1700 0.0827 0.8732 0.8828 0.8780 0.9813
0.0126 6.5900 1720 0.0820 0.8741 0.8810 0.8775 0.9810
0.0108 6.6667 1740 0.0840 0.8778 0.8830 0.8804 0.9815
0.0118 6.7433 1760 0.0852 0.8757 0.8824 0.8790 0.9814
0.0103 6.8199 1780 0.0854 0.8773 0.8824 0.8798 0.9814
0.0103 6.8966 1800 0.0854 0.8782 0.8828 0.8805 0.9815
0.0081 6.9732 1820 0.0853 0.8767 0.8819 0.8793 0.9814

Framework versions

  • PEFT 0.13.2
  • Transformers 4.46.3
  • Pytorch 2.5.1
  • Datasets 3.1.0
  • Tokenizers 0.20.3
Downloads last month
13
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for raulgdp/roberta-large-ner-qlorafinetune-runs

Base model

FacebookAI/xlm-roberta-large
Adapter
(7)
this model

Dataset used to train raulgdp/roberta-large-ner-qlorafinetune-runs

Evaluation results

Metadata error: specify a dataset to view leaderboard