Hugging Face's logo

raulgdp
/
roberta-large-ner-qlorafinetune-runs-colab

PEFT
TensorBoard
Safetensors
Generated from Trainer
Model card Files Metrics Community
raulgdp commited on
Commit
35840d0
•
1 Parent(s): fdb85ab

End of training

Browse files
Files changed (1) hide show
  1. README.md +161 -0
README.md ADDED
@@ -0,0 +1,161 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ license: mit
4
+ base_model: FacebookAI/xlm-roberta-large
5
+ tags:
6
+ - generated_from_trainer
7
+ datasets:
8
+ - conll2002
9
+ metrics:
10
+ - precision
11
+ - recall
12
+ - f1
13
+ - accuracy
14
+ model-index:
15
+ - name: roberta-large-ner-qlorafinetune-runs-colab
16
+ results: []
17
+ ---
18
+
19
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
20
+ should probably proofread and complete it, then remove this comment. -->
21
+
22
+ # roberta-large-ner-qlorafinetune-runs-colab
23
+
24
+ This model is a fine-tuned version of [FacebookAI/xlm-roberta-large](https://huggingface.co/FacebookAI/xlm-roberta-large) on the conll2002 dataset.
25
+ It achieves the following results on the evaluation set:
26
+ - Loss: 0.0818
27
+ - Precision: 0.8756
28
+ - Recall: 0.8830
29
+ - F1: 0.8793
30
+ - Accuracy: 0.9821
31
+
32
+ ## Model description
33
+
34
+ More information needed
35
+
36
+ ## Intended uses & limitations
37
+
38
+ More information needed
39
+
40
+ ## Training and evaluation data
41
+
42
+ More information needed
43
+
44
+ ## Training procedure
45
+
46
+ ### Training hyperparameters
47
+
48
+ The following hyperparameters were used during training:
49
+ - learning_rate: 0.0004
50
+ - train_batch_size: 32
51
+ - eval_batch_size: 32
52
+ - seed: 42
53
+ - optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
54
+ - lr_scheduler_type: linear
55
+ - training_steps: 1820
56
+ - mixed_precision_training: Native AMP
57
+
58
+ ### Training results
59
+
60
+ | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
61
+ |:-------------:|:------:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
62
+ | 1.1988 | 0.0766 | 20 | 0.5183 | 0.0 | 0.0 | 0.0 | 0.8570 |
63
+ | 0.346 | 0.1533 | 40 | 0.3115 | 0.3631 | 0.4614 | 0.4064 | 0.9159 |
64
+ | 0.2248 | 0.2299 | 60 | 0.1851 | 0.6607 | 0.6994 | 0.6795 | 0.9504 |
65
+ | 0.1411 | 0.3065 | 80 | 0.1653 | 0.6161 | 0.6687 | 0.6413 | 0.9568 |
66
+ | 0.1164 | 0.3831 | 100 | 0.1017 | 0.7897 | 0.8196 | 0.8044 | 0.9744 |
67
+ | 0.0937 | 0.4598 | 120 | 0.1144 | 0.7908 | 0.8224 | 0.8063 | 0.9732 |
68
+ | 0.0846 | 0.5364 | 140 | 0.0809 | 0.8454 | 0.8667 | 0.8559 | 0.9804 |
69
+ | 0.0852 | 0.6130 | 160 | 0.1207 | 0.7424 | 0.7727 | 0.7573 | 0.9702 |
70
+ | 0.0798 | 0.6897 | 180 | 0.0878 | 0.8264 | 0.8564 | 0.8411 | 0.9775 |
71
+ | 0.0668 | 0.7663 | 200 | 0.0832 | 0.8362 | 0.8539 | 0.8449 | 0.9781 |
72
+ | 0.062 | 0.8429 | 220 | 0.0766 | 0.8411 | 0.8564 | 0.8487 | 0.9796 |
73
+ | 0.0624 | 0.9195 | 240 | 0.0750 | 0.8478 | 0.8624 | 0.8550 | 0.9792 |
74
+ | 0.0577 | 0.9962 | 260 | 0.1152 | 0.7684 | 0.7934 | 0.7807 | 0.9707 |
75
+ | 0.0519 | 1.0728 | 280 | 0.0889 | 0.8155 | 0.8440 | 0.8295 | 0.9766 |
76
+ | 0.0516 | 1.1494 | 300 | 0.0855 | 0.8339 | 0.8614 | 0.8474 | 0.9780 |
77
+ | 0.0584 | 1.2261 | 320 | 0.0794 | 0.8418 | 0.8633 | 0.8524 | 0.9789 |
78
+ | 0.0518 | 1.3027 | 340 | 0.0879 | 0.8240 | 0.8458 | 0.8348 | 0.9766 |
79
+ | 0.06 | 1.3793 | 360 | 0.0922 | 0.8100 | 0.8277 | 0.8187 | 0.9756 |
80
+ | 0.0556 | 1.4559 | 380 | 0.1141 | 0.7741 | 0.7927 | 0.7833 | 0.9704 |
81
+ | 0.0524 | 1.5326 | 400 | 0.0863 | 0.8247 | 0.8412 | 0.8329 | 0.9765 |
82
+ | 0.0526 | 1.6092 | 420 | 0.0779 | 0.8391 | 0.8518 | 0.8454 | 0.9781 |
83
+ | 0.0699 | 1.6858 | 440 | 0.0903 | 0.8099 | 0.8290 | 0.8193 | 0.9750 |
84
+ | 0.0626 | 1.7625 | 460 | 0.0746 | 0.8569 | 0.8571 | 0.8570 | 0.9794 |
85
+ | 0.05 | 1.8391 | 480 | 0.0725 | 0.8443 | 0.8709 | 0.8574 | 0.9798 |
86
+ | 0.0521 | 1.9157 | 500 | 0.0694 | 0.8558 | 0.8771 | 0.8663 | 0.9813 |
87
+ | 0.0463 | 1.9923 | 520 | 0.0798 | 0.8521 | 0.8672 | 0.8596 | 0.9796 |
88
+ | 0.0748 | 2.0690 | 540 | 0.1192 | 0.7639 | 0.7868 | 0.7752 | 0.9717 |
89
+ | 0.0419 | 2.1456 | 560 | 0.0905 | 0.7932 | 0.8327 | 0.8125 | 0.9758 |
90
+ | 0.0329 | 2.2222 | 580 | 0.0771 | 0.8443 | 0.8663 | 0.8552 | 0.9801 |
91
+ | 0.0434 | 2.2989 | 600 | 0.0838 | 0.8204 | 0.8472 | 0.8336 | 0.9772 |
92
+ | 0.0378 | 2.3755 | 620 | 0.0770 | 0.8220 | 0.8382 | 0.8300 | 0.9773 |
93
+ | 0.0378 | 2.4521 | 640 | 0.0784 | 0.8288 | 0.8575 | 0.8429 | 0.9796 |
94
+ | 0.0417 | 2.5287 | 660 | 0.0863 | 0.8510 | 0.8571 | 0.8540 | 0.9794 |
95
+ | 0.0423 | 2.6054 | 680 | 0.0912 | 0.8394 | 0.8504 | 0.8449 | 0.9777 |
96
+ | 0.0484 | 2.6820 | 700 | 0.0837 | 0.8215 | 0.8472 | 0.8342 | 0.9759 |
97
+ | 0.0394 | 2.7586 | 720 | 0.0777 | 0.8522 | 0.8585 | 0.8553 | 0.9800 |
98
+ | 0.0383 | 2.8352 | 740 | 0.0724 | 0.8663 | 0.8713 | 0.8688 | 0.9814 |
99
+ | 0.0446 | 2.9119 | 760 | 0.0774 | 0.8636 | 0.8828 | 0.8731 | 0.9804 |
100
+ | 0.0424 | 2.9885 | 780 | 0.0750 | 0.8641 | 0.875 | 0.8695 | 0.9815 |
101
+ | 0.0518 | 3.0651 | 800 | 0.0784 | 0.8359 | 0.8683 | 0.8518 | 0.9799 |
102
+ | 0.0282 | 3.1418 | 820 | 0.0752 | 0.8649 | 0.8720 | 0.8684 | 0.9808 |
103
+ | 0.0292 | 3.2184 | 840 | 0.0820 | 0.8562 | 0.8690 | 0.8626 | 0.9795 |
104
+ | 0.0304 | 3.2950 | 860 | 0.0847 | 0.8528 | 0.8679 | 0.8603 | 0.9796 |
105
+ | 0.0288 | 3.3716 | 880 | 0.0784 | 0.8583 | 0.8683 | 0.8633 | 0.9800 |
106
+ | 0.0281 | 3.4483 | 900 | 0.0753 | 0.8546 | 0.8619 | 0.8583 | 0.9795 |
107
+ | 0.0338 | 3.5249 | 920 | 0.0710 | 0.8589 | 0.8686 | 0.8637 | 0.9807 |
108
+ | 0.0263 | 3.6015 | 940 | 0.0752 | 0.8635 | 0.8693 | 0.8664 | 0.9806 |
109
+ | 0.0317 | 3.6782 | 960 | 0.0732 | 0.8649 | 0.8722 | 0.8686 | 0.9798 |
110
+ | 0.0264 | 3.7548 | 980 | 0.0711 | 0.8650 | 0.8851 | 0.8750 | 0.9808 |
111
+ | 0.0342 | 3.8314 | 1000 | 0.0694 | 0.8729 | 0.8821 | 0.8775 | 0.9821 |
112
+ | 0.0294 | 3.9080 | 1020 | 0.0726 | 0.8662 | 0.8794 | 0.8727 | 0.9802 |
113
+ | 0.0338 | 3.9847 | 1040 | 0.0749 | 0.8747 | 0.8787 | 0.8767 | 0.9812 |
114
+ | 0.0203 | 4.0613 | 1060 | 0.0777 | 0.8711 | 0.8761 | 0.8736 | 0.9803 |
115
+ | 0.0221 | 4.1379 | 1080 | 0.0836 | 0.8629 | 0.8736 | 0.8682 | 0.9801 |
116
+ | 0.0186 | 4.2146 | 1100 | 0.0800 | 0.8644 | 0.8805 | 0.8724 | 0.9806 |
117
+ | 0.02 | 4.2912 | 1120 | 0.0844 | 0.8683 | 0.8817 | 0.8749 | 0.9811 |
118
+ | 0.0172 | 4.3678 | 1140 | 0.0797 | 0.8701 | 0.8851 | 0.8775 | 0.9810 |
119
+ | 0.0227 | 4.4444 | 1160 | 0.0806 | 0.8755 | 0.8824 | 0.8789 | 0.9810 |
120
+ | 0.0198 | 4.5211 | 1180 | 0.0809 | 0.8658 | 0.8778 | 0.8717 | 0.9803 |
121
+ | 0.022 | 4.5977 | 1200 | 0.0826 | 0.8748 | 0.8798 | 0.8773 | 0.9812 |
122
+ | 0.0226 | 4.6743 | 1220 | 0.0765 | 0.8668 | 0.8849 | 0.8757 | 0.9815 |
123
+ | 0.0248 | 4.7510 | 1240 | 0.0799 | 0.8598 | 0.8722 | 0.8660 | 0.9803 |
124
+ | 0.0229 | 4.8276 | 1260 | 0.0803 | 0.8646 | 0.8727 | 0.8686 | 0.9809 |
125
+ | 0.0229 | 4.9042 | 1280 | 0.0773 | 0.8639 | 0.875 | 0.8694 | 0.9811 |
126
+ | 0.0202 | 4.9808 | 1300 | 0.0756 | 0.8765 | 0.8824 | 0.8794 | 0.9822 |
127
+ | 0.0156 | 5.0575 | 1320 | 0.0764 | 0.8646 | 0.8835 | 0.8740 | 0.9817 |
128
+ | 0.0139 | 5.1341 | 1340 | 0.0818 | 0.8673 | 0.8801 | 0.8736 | 0.9813 |
129
+ | 0.0181 | 5.2107 | 1360 | 0.0792 | 0.8732 | 0.8844 | 0.8788 | 0.9812 |
130
+ | 0.018 | 5.2874 | 1380 | 0.0778 | 0.8750 | 0.8801 | 0.8775 | 0.9819 |
131
+ | 0.0139 | 5.3640 | 1400 | 0.0762 | 0.8704 | 0.8824 | 0.8763 | 0.9820 |
132
+ | 0.0154 | 5.4406 | 1420 | 0.0791 | 0.8753 | 0.8791 | 0.8772 | 0.9819 |
133
+ | 0.0151 | 5.5172 | 1440 | 0.0816 | 0.8779 | 0.8803 | 0.8791 | 0.9818 |
134
+ | 0.0177 | 5.5939 | 1460 | 0.0807 | 0.8721 | 0.8803 | 0.8762 | 0.9816 |
135
+ | 0.0164 | 5.6705 | 1480 | 0.0762 | 0.8701 | 0.8837 | 0.8769 | 0.9817 |
136
+ | 0.0131 | 5.7471 | 1500 | 0.0783 | 0.8775 | 0.8872 | 0.8823 | 0.9822 |
137
+ | 0.0174 | 5.8238 | 1520 | 0.0774 | 0.8759 | 0.8824 | 0.8791 | 0.9816 |
138
+ | 0.0168 | 5.9004 | 1540 | 0.0822 | 0.8619 | 0.8718 | 0.8668 | 0.9808 |
139
+ | 0.0131 | 5.9770 | 1560 | 0.0822 | 0.8696 | 0.8778 | 0.8736 | 0.9815 |
140
+ | 0.009 | 6.0536 | 1580 | 0.0848 | 0.8704 | 0.8794 | 0.8748 | 0.9816 |
141
+ | 0.0108 | 6.1303 | 1600 | 0.0824 | 0.8773 | 0.8821 | 0.8797 | 0.9820 |
142
+ | 0.0132 | 6.2069 | 1620 | 0.0842 | 0.8762 | 0.8801 | 0.8781 | 0.9816 |
143
+ | 0.0136 | 6.2835 | 1640 | 0.0814 | 0.8764 | 0.8867 | 0.8816 | 0.9824 |
144
+ | 0.0144 | 6.3602 | 1660 | 0.0798 | 0.8764 | 0.8863 | 0.8813 | 0.9825 |
145
+ | 0.0113 | 6.4368 | 1680 | 0.0812 | 0.8790 | 0.8849 | 0.8819 | 0.9823 |
146
+ | 0.0102 | 6.5134 | 1700 | 0.0826 | 0.8742 | 0.8828 | 0.8785 | 0.9821 |
147
+ | 0.014 | 6.5900 | 1720 | 0.0792 | 0.8754 | 0.8847 | 0.88 | 0.9825 |
148
+ | 0.0111 | 6.6667 | 1740 | 0.0815 | 0.8734 | 0.8817 | 0.8775 | 0.9821 |
149
+ | 0.0098 | 6.7433 | 1760 | 0.0818 | 0.8740 | 0.8828 | 0.8784 | 0.9821 |
150
+ | 0.012 | 6.8199 | 1780 | 0.0822 | 0.8749 | 0.8840 | 0.8794 | 0.9820 |
151
+ | 0.01 | 6.8966 | 1800 | 0.0819 | 0.8742 | 0.8833 | 0.8787 | 0.9820 |
152
+ | 0.0093 | 6.9732 | 1820 | 0.0818 | 0.8756 | 0.8830 | 0.8793 | 0.9821 |
153
+
154
+
155
+ ### Framework versions
156
+
157
+ - PEFT 0.13.2
158
+ - Transformers 4.46.2
159
+ - Pytorch 2.5.1+cu121
160
+ - Datasets 3.1.0
161
+ - Tokenizers 0.20.3