Hugging Face's logo

WillHeld
/
roberta-base-sst2

Text Classification
Transformers
PyTorch
English
roberta
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
2
WillHeld commited on
Commit
ba2a19f
•
1 Parent(s): 5021433

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +158 -0
README.md ADDED
@@ -0,0 +1,158 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - generated_from_trainer
5
+ datasets:
6
+ - glue
7
+ metrics:
8
+ - accuracy
9
+ model-index:
10
+ - name: roberta-base-sst2
11
+ results:
12
+ - task:
13
+ name: Text Classification
14
+ type: text-classification
15
+ dataset:
16
+ name: glue
17
+ type: glue
18
+ config: sst2
19
+ split: train
20
+ args: sst2
21
+ metrics:
22
+ - name: Accuracy
23
+ type: accuracy
24
+ value: 0.9415137614678899
25
+ ---
26
+
27
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
28
+ should probably proofread and complete it, then remove this comment. -->
29
+
30
+ # roberta-base-sst2
31
+
32
+ This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the glue dataset.
33
+ It achieves the following results on the evaluation set:
34
+ - Loss: 0.4270
35
+ - Accuracy: 0.9415
36
+
37
+ ## Model description
38
+
39
+ More information needed
40
+
41
+ ## Intended uses & limitations
42
+
43
+ More information needed
44
+
45
+ ## Training and evaluation data
46
+
47
+ More information needed
48
+
49
+ ## Training procedure
50
+
51
+ ### Training hyperparameters
52
+
53
+ The following hyperparameters were used during training:
54
+ - learning_rate: 2e-05
55
+ - train_batch_size: 16
56
+ - eval_batch_size: 8
57
+ - seed: 42
58
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
59
+ - lr_scheduler_type: linear
60
+ - lr_scheduler_warmup_ratio: 0.06
61
+ - num_epochs: 10.0
62
+
63
+ ### Training results
64
+
65
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
66
+ |:-------------:|:-----:|:-----:|:---------------:|:--------:|
67
+ | 0.575 | 0.12 | 500 | 0.2665 | 0.9071 |
68
+ | 0.2989 | 0.24 | 1000 | 0.2088 | 0.9220 |
69
+ | 0.2725 | 0.36 | 1500 | 0.2560 | 0.9243 |
70
+ | 0.2814 | 0.48 | 2000 | 0.2016 | 0.9266 |
71
+ | 0.2586 | 0.59 | 2500 | 0.2293 | 0.9174 |
72
+ | 0.2536 | 0.71 | 3000 | 0.2340 | 0.9323 |
73
+ | 0.2494 | 0.83 | 3500 | 0.1952 | 0.9323 |
74
+ | 0.2396 | 0.95 | 4000 | 0.2494 | 0.9323 |
75
+ | 0.2123 | 1.07 | 4500 | 0.2187 | 0.9381 |
76
+ | 0.2042 | 1.19 | 5000 | 0.2812 | 0.9151 |
77
+ | 0.2083 | 1.31 | 5500 | 0.2739 | 0.9346 |
78
+ | 0.2041 | 1.43 | 6000 | 0.2087 | 0.9381 |
79
+ | 0.1969 | 1.54 | 6500 | 0.2590 | 0.9255 |
80
+ | 0.1982 | 1.66 | 7000 | 0.2445 | 0.9300 |
81
+ | 0.1943 | 1.78 | 7500 | 0.2798 | 0.9266 |
82
+ | 0.1848 | 1.9 | 8000 | 0.2844 | 0.9312 |
83
+ | 0.1788 | 2.02 | 8500 | 0.2998 | 0.9255 |
84
+ | 0.1623 | 2.14 | 9000 | 0.2696 | 0.9392 |
85
+ | 0.1499 | 2.26 | 9500 | 0.2533 | 0.9278 |
86
+ | 0.1426 | 2.38 | 10000 | 0.2971 | 0.9300 |
87
+ | 0.1479 | 2.49 | 10500 | 0.2596 | 0.9358 |
88
+ | 0.1405 | 2.61 | 11000 | 0.2945 | 0.9255 |
89
+ | 0.1577 | 2.73 | 11500 | 0.4061 | 0.9002 |
90
+ | 0.1521 | 2.85 | 12000 | 0.2724 | 0.9335 |
91
+ | 0.1426 | 2.97 | 12500 | 0.2712 | 0.9427 |
92
+ | 0.1206 | 3.09 | 13000 | 0.2954 | 0.9358 |
93
+ | 0.1074 | 3.21 | 13500 | 0.2653 | 0.9392 |
94
+ | 0.112 | 3.33 | 14000 | 0.2778 | 0.9346 |
95
+ | 0.1147 | 3.44 | 14500 | 0.3705 | 0.9312 |
96
+ | 0.1196 | 3.56 | 15000 | 0.2890 | 0.9346 |
97
+ | 0.1159 | 3.68 | 15500 | 0.3449 | 0.9266 |
98
+ | 0.119 | 3.8 | 16000 | 0.3207 | 0.9335 |
99
+ | 0.1268 | 3.92 | 16500 | 0.3235 | 0.9312 |
100
+ | 0.1074 | 4.04 | 17000 | 0.3650 | 0.9335 |
101
+ | 0.0805 | 4.16 | 17500 | 0.3338 | 0.9381 |
102
+ | 0.0838 | 4.28 | 18000 | 0.4302 | 0.9209 |
103
+ | 0.0848 | 4.39 | 18500 | 0.4096 | 0.9323 |
104
+ | 0.0922 | 4.51 | 19000 | 0.3332 | 0.9369 |
105
+ | 0.091 | 4.63 | 19500 | 0.3024 | 0.9438 |
106
+ | 0.0977 | 4.75 | 20000 | 0.2674 | 0.9495 |
107
+ | 0.0897 | 4.87 | 20500 | 0.3993 | 0.9300 |
108
+ | 0.1013 | 4.99 | 21000 | 0.3227 | 0.9289 |
109
+ | 0.0671 | 5.11 | 21500 | 0.3374 | 0.9427 |
110
+ | 0.0671 | 5.23 | 22000 | 0.4108 | 0.9278 |
111
+ | 0.0652 | 5.34 | 22500 | 0.3550 | 0.9381 |
112
+ | 0.0664 | 5.46 | 23000 | 0.3398 | 0.9358 |
113
+ | 0.0742 | 5.58 | 23500 | 0.3286 | 0.9381 |
114
+ | 0.0758 | 5.7 | 24000 | 0.3276 | 0.9312 |
115
+ | 0.075 | 5.82 | 24500 | 0.3202 | 0.9369 |
116
+ | 0.0686 | 5.94 | 25000 | 0.3481 | 0.9415 |
117
+ | 0.0729 | 6.06 | 25500 | 0.3816 | 0.9335 |
118
+ | 0.0568 | 6.18 | 26000 | 0.3132 | 0.9381 |
119
+ | 0.0529 | 6.29 | 26500 | 0.3757 | 0.9300 |
120
+ | 0.0506 | 6.41 | 27000 | 0.3396 | 0.9381 |
121
+ | 0.0476 | 6.53 | 27500 | 0.3642 | 0.9404 |
122
+ | 0.0555 | 6.65 | 28000 | 0.3430 | 0.9404 |
123
+ | 0.0574 | 6.77 | 28500 | 0.3401 | 0.9392 |
124
+ | 0.0524 | 6.89 | 29000 | 0.3378 | 0.9346 |
125
+ | 0.0492 | 7.01 | 29500 | 0.3833 | 0.9381 |
126
+ | 0.039 | 7.13 | 30000 | 0.3347 | 0.9346 |
127
+ | 0.0411 | 7.24 | 30500 | 0.4404 | 0.9335 |
128
+ | 0.0412 | 7.36 | 31000 | 0.3618 | 0.9381 |
129
+ | 0.0477 | 7.48 | 31500 | 0.3806 | 0.9381 |
130
+ | 0.0435 | 7.6 | 32000 | 0.3912 | 0.9335 |
131
+ | 0.0443 | 7.72 | 32500 | 0.3900 | 0.9392 |
132
+ | 0.0421 | 7.84 | 33000 | 0.4152 | 0.9369 |
133
+ | 0.0495 | 7.96 | 33500 | 0.3832 | 0.9289 |
134
+ | 0.0293 | 8.08 | 34000 | 0.4427 | 0.9346 |
135
+ | 0.0253 | 8.19 | 34500 | 0.4425 | 0.9381 |
136
+ | 0.0407 | 8.31 | 35000 | 0.4102 | 0.9358 |
137
+ | 0.0311 | 8.43 | 35500 | 0.4447 | 0.9369 |
138
+ | 0.0291 | 8.55 | 36000 | 0.4612 | 0.9346 |
139
+ | 0.035 | 8.67 | 36500 | 0.4241 | 0.9346 |
140
+ | 0.0381 | 8.79 | 37000 | 0.4198 | 0.9312 |
141
+ | 0.0234 | 8.91 | 37500 | 0.4345 | 0.9369 |
142
+ | 0.0311 | 9.03 | 38000 | 0.4558 | 0.9312 |
143
+ | 0.028 | 9.14 | 38500 | 0.4245 | 0.9381 |
144
+ | 0.0213 | 9.26 | 39000 | 0.4462 | 0.9381 |
145
+ | 0.0276 | 9.38 | 39500 | 0.4210 | 0.9381 |
146
+ | 0.0183 | 9.5 | 40000 | 0.4310 | 0.9404 |
147
+ | 0.0184 | 9.62 | 40500 | 0.4437 | 0.9404 |
148
+ | 0.0296 | 9.74 | 41000 | 0.4311 | 0.9392 |
149
+ | 0.019 | 9.86 | 41500 | 0.4244 | 0.9415 |
150
+ | 0.0245 | 9.98 | 42000 | 0.4270 | 0.9415 |
151
+
152
+
153
+ ### Framework versions
154
+
155
+ - Transformers 4.21.3
156
+ - Pytorch 1.7.1
157
+ - Datasets 1.18.3
158
+ - Tokenizers 0.11.6