Hugging Face's logo

gngpostalsrvc
/
BERiT_2000_custom_architecture_relu_40_epochs

Fill-Mask
Transformers
PyTorch
TensorBoard
roberta
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
1
gngpostalsrvc commited on
Commit
fff30fe
·
1 Parent(s): 5b60956

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +262 -0
README.md ADDED
@@ -0,0 +1,262 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - generated_from_trainer
5
+ model-index:
6
+ - name: BERiT_2000_custom_architecture_relu_40_epochs
7
+ results: []
8
+ ---
9
+
10
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
11
+ should probably proofread and complete it, then remove this comment. -->
12
+
13
+ # BERiT_2000_custom_architecture_relu_40_epochs
14
+
15
+ This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
16
+ It achieves the following results on the evaluation set:
17
+ - Loss: 6.3968
18
+
19
+ ## Model description
20
+
21
+ More information needed
22
+
23
+ ## Intended uses & limitations
24
+
25
+ More information needed
26
+
27
+ ## Training and evaluation data
28
+
29
+ More information needed
30
+
31
+ ## Training procedure
32
+
33
+ ### Training hyperparameters
34
+
35
+ The following hyperparameters were used during training:
36
+ - learning_rate: 0.0005
37
+ - train_batch_size: 8
38
+ - eval_batch_size: 8
39
+ - seed: 42
40
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
41
+ - lr_scheduler_type: linear
42
+ - num_epochs: 40
43
+ - label_smoothing_factor: 0.2
44
+
45
+ ### Training results
46
+
47
+ | Training Loss | Epoch | Step | Validation Loss |
48
+ |:-------------:|:-----:|:------:|:---------------:|
49
+ | 14.8756 | 0.19 | 500 | 8.3404 |
50
+ | 7.8098 | 0.39 | 1000 | 7.3110 |
51
+ | 7.2696 | 0.58 | 1500 | 7.1646 |
52
+ | 7.1277 | 0.77 | 2000 | 7.0953 |
53
+ | 7.0939 | 0.97 | 2500 | 7.0701 |
54
+ | 7.0621 | 1.16 | 3000 | 7.0003 |
55
+ | 7.0236 | 1.36 | 3500 | 6.9189 |
56
+ | 6.9898 | 1.55 | 4000 | 6.8536 |
57
+ | 6.9625 | 1.74 | 4500 | 6.8450 |
58
+ | 6.9125 | 1.94 | 5000 | 6.7799 |
59
+ | 6.9115 | 2.13 | 5500 | 6.8028 |
60
+ | 6.8954 | 2.32 | 6000 | 6.7288 |
61
+ | 6.8289 | 2.52 | 6500 | 6.7664 |
62
+ | 6.855 | 2.71 | 7000 | 6.7064 |
63
+ | 6.8134 | 2.9 | 7500 | 6.7155 |
64
+ | 6.7907 | 3.1 | 8000 | 6.7050 |
65
+ | 6.791 | 3.29 | 8500 | 6.6695 |
66
+ | 6.7659 | 3.49 | 9000 | 6.6815 |
67
+ | 6.775 | 3.68 | 9500 | 6.6449 |
68
+ | 6.7508 | 3.87 | 10000 | 6.6684 |
69
+ | 6.7627 | 4.07 | 10500 | 6.6397 |
70
+ | 6.7229 | 4.26 | 11000 | 6.6417 |
71
+ | 6.7336 | 4.45 | 11500 | 6.6824 |
72
+ | 6.7138 | 4.65 | 12000 | 6.6252 |
73
+ | 6.7123 | 4.84 | 12500 | 6.6374 |
74
+ | 6.703 | 5.03 | 13000 | 6.6400 |
75
+ | 6.7054 | 5.23 | 13500 | 6.6264 |
76
+ | 6.6978 | 5.42 | 14000 | 6.6094 |
77
+ | 6.6944 | 5.62 | 14500 | 6.6627 |
78
+ | 6.6857 | 5.81 | 15000 | 6.6363 |
79
+ | 6.694 | 6.0 | 15500 | 6.6026 |
80
+ | 6.6882 | 6.2 | 16000 | 6.6123 |
81
+ | 6.6694 | 6.39 | 16500 | 6.5918 |
82
+ | 6.6781 | 6.58 | 17000 | 6.6201 |
83
+ | 6.6605 | 6.78 | 17500 | 6.6316 |
84
+ | 6.6435 | 6.97 | 18000 | 6.5789 |
85
+ | 6.6658 | 7.16 | 18500 | 6.5999 |
86
+ | 6.6551 | 7.36 | 19000 | 6.5425 |
87
+ | 6.6603 | 7.55 | 19500 | 6.5790 |
88
+ | 6.6589 | 7.75 | 20000 | 6.5767 |
89
+ | 6.675 | 7.94 | 20500 | 6.6005 |
90
+ | 6.6362 | 8.13 | 21000 | 6.5962 |
91
+ | 6.6391 | 8.33 | 21500 | 6.5716 |
92
+ | 6.6379 | 8.52 | 22000 | 6.5830 |
93
+ | 6.6164 | 8.71 | 22500 | 6.6137 |
94
+ | 6.638 | 8.91 | 23000 | 6.5877 |
95
+ | 6.6255 | 9.1 | 23500 | 6.6197 |
96
+ | 6.6284 | 9.3 | 24000 | 6.5573 |
97
+ | 6.6198 | 9.49 | 24500 | 6.5717 |
98
+ | 6.6025 | 9.68 | 25000 | 6.5627 |
99
+ | 6.6334 | 9.88 | 25500 | 6.5902 |
100
+ | 6.6305 | 10.07 | 26000 | 6.5628 |
101
+ | 6.5797 | 10.26 | 26500 | 6.5625 |
102
+ | 6.5906 | 10.46 | 27000 | 6.5808 |
103
+ | 6.5904 | 10.65 | 27500 | 6.5690 |
104
+ | 6.5935 | 10.84 | 28000 | 6.5845 |
105
+ | 6.6231 | 11.04 | 28500 | 6.5282 |
106
+ | 6.5923 | 11.23 | 29000 | 6.6107 |
107
+ | 6.6136 | 11.43 | 29500 | 6.5475 |
108
+ | 6.5954 | 11.62 | 30000 | 6.5823 |
109
+ | 6.5821 | 11.81 | 30500 | 6.5721 |
110
+ | 6.5993 | 12.01 | 31000 | 6.5492 |
111
+ | 6.5584 | 12.2 | 31500 | 6.4938 |
112
+ | 6.5886 | 12.39 | 32000 | 6.6026 |
113
+ | 6.5625 | 12.59 | 32500 | 6.5902 |
114
+ | 6.572 | 12.78 | 33000 | 6.5436 |
115
+ | 6.5807 | 12.97 | 33500 | 6.5588 |
116
+ | 6.5853 | 13.17 | 34000 | 6.5555 |
117
+ | 6.5727 | 13.36 | 34500 | 6.5606 |
118
+ | 6.5456 | 13.56 | 35000 | 6.5386 |
119
+ | 6.5538 | 13.75 | 35500 | 6.5712 |
120
+ | 6.5456 | 13.94 | 36000 | 6.5582 |
121
+ | 6.5734 | 14.14 | 36500 | 6.4951 |
122
+ | 6.5639 | 14.33 | 37000 | 6.5323 |
123
+ | 6.5712 | 14.52 | 37500 | 6.5049 |
124
+ | 6.5739 | 14.72 | 38000 | 6.5523 |
125
+ | 6.5534 | 14.91 | 38500 | 6.5188 |
126
+ | 6.5401 | 15.1 | 39000 | 6.5968 |
127
+ | 6.5456 | 15.3 | 39500 | 6.5413 |
128
+ | 6.5555 | 15.49 | 40000 | 6.5347 |
129
+ | 6.538 | 15.69 | 40500 | 6.5180 |
130
+ | 6.537 | 15.88 | 41000 | 6.5372 |
131
+ | 6.537 | 16.07 | 41500 | 6.5514 |
132
+ | 6.5445 | 16.27 | 42000 | 6.5242 |
133
+ | 6.5285 | 16.46 | 42500 | 6.5071 |
134
+ | 6.5046 | 16.65 | 43000 | 6.5342 |
135
+ | 6.5609 | 16.85 | 43500 | 6.5329 |
136
+ | 6.527 | 17.04 | 44000 | 6.5569 |
137
+ | 6.5199 | 17.23 | 44500 | 6.5438 |
138
+ | 6.5328 | 17.43 | 45000 | 6.5380 |
139
+ | 6.5183 | 17.62 | 45500 | 6.5273 |
140
+ | 6.5349 | 17.82 | 46000 | 6.5209 |
141
+ | 6.5283 | 18.01 | 46500 | 6.4884 |
142
+ | 6.5111 | 18.2 | 47000 | 6.5036 |
143
+ | 6.4895 | 18.4 | 47500 | 6.5675 |
144
+ | 6.5308 | 18.59 | 48000 | 6.5378 |
145
+ | 6.5159 | 18.78 | 48500 | 6.4792 |
146
+ | 6.4875 | 18.98 | 49000 | 6.4846 |
147
+ | 6.5076 | 19.17 | 49500 | 6.5203 |
148
+ | 6.4991 | 19.36 | 50000 | 6.5007 |
149
+ | 6.5269 | 19.56 | 50500 | 6.4796 |
150
+ | 6.4887 | 19.75 | 51000 | 6.5197 |
151
+ | 6.4995 | 19.95 | 51500 | 6.5009 |
152
+ | 6.4762 | 20.14 | 52000 | 6.5049 |
153
+ | 6.4872 | 20.33 | 52500 | 6.4880 |
154
+ | 6.5117 | 20.53 | 53000 | 6.4917 |
155
+ | 6.5035 | 20.72 | 53500 | 6.4791 |
156
+ | 6.4784 | 20.91 | 54000 | 6.4771 |
157
+ | 6.4749 | 21.11 | 54500 | 6.5230 |
158
+ | 6.4867 | 21.3 | 55000 | 6.4954 |
159
+ | 6.4921 | 21.49 | 55500 | 6.5079 |
160
+ | 6.4587 | 21.69 | 56000 | 6.5309 |
161
+ | 6.4839 | 21.88 | 56500 | 6.4476 |
162
+ | 6.5011 | 22.08 | 57000 | 6.5025 |
163
+ | 6.471 | 22.27 | 57500 | 6.5122 |
164
+ | 6.4689 | 22.46 | 58000 | 6.4689 |
165
+ | 6.4764 | 22.66 | 58500 | 6.5073 |
166
+ | 6.4764 | 22.85 | 59000 | 6.4741 |
167
+ | 6.4751 | 23.04 | 59500 | 6.4978 |
168
+ | 6.4823 | 23.24 | 60000 | 6.4857 |
169
+ | 6.4594 | 23.43 | 60500 | 6.4817 |
170
+ | 6.4795 | 23.63 | 61000 | 6.5292 |
171
+ | 6.4565 | 23.82 | 61500 | 6.4684 |
172
+ | 6.4627 | 24.01 | 62000 | 6.4900 |
173
+ | 6.4542 | 24.21 | 62500 | 6.4373 |
174
+ | 6.4692 | 24.4 | 63000 | 6.4787 |
175
+ | 6.4772 | 24.59 | 63500 | 6.4553 |
176
+ | 6.4613 | 24.79 | 64000 | 6.4695 |
177
+ | 6.4673 | 24.98 | 64500 | 6.5077 |
178
+ | 6.466 | 25.17 | 65000 | 6.4919 |
179
+ | 6.4595 | 25.37 | 65500 | 6.4451 |
180
+ | 6.444 | 25.56 | 66000 | 6.4750 |
181
+ | 6.438 | 25.76 | 66500 | 6.4672 |
182
+ | 6.4499 | 25.95 | 67000 | 6.4358 |
183
+ | 6.4578 | 26.14 | 67500 | 6.4762 |
184
+ | 6.4701 | 26.34 | 68000 | 6.4462 |
185
+ | 6.4296 | 26.53 | 68500 | 6.4879 |
186
+ | 6.4305 | 26.72 | 69000 | 6.4519 |
187
+ | 6.443 | 26.92 | 69500 | 6.4530 |
188
+ | 6.4571 | 27.11 | 70000 | 6.4564 |
189
+ | 6.4477 | 27.3 | 70500 | 6.4557 |
190
+ | 6.443 | 27.5 | 71000 | 6.4862 |
191
+ | 6.4429 | 27.69 | 71500 | 6.4498 |
192
+ | 6.4374 | 27.89 | 72000 | 6.4225 |
193
+ | 6.4363 | 28.08 | 72500 | 6.4723 |
194
+ | 6.4127 | 28.27 | 73000 | 6.4733 |
195
+ | 6.4116 | 28.47 | 73500 | 6.4499 |
196
+ | 6.4312 | 28.66 | 74000 | 6.4600 |
197
+ | 6.4251 | 28.85 | 74500 | 6.4451 |
198
+ | 6.4318 | 29.05 | 75000 | 6.4337 |
199
+ | 6.4432 | 29.24 | 75500 | 6.4713 |
200
+ | 6.4183 | 29.43 | 76000 | 6.4699 |
201
+ | 6.4109 | 29.63 | 76500 | 6.4591 |
202
+ | 6.3939 | 29.82 | 77000 | 6.4768 |
203
+ | 6.4194 | 30.02 | 77500 | 6.4786 |
204
+ | 6.4262 | 30.21 | 78000 | 6.4407 |
205
+ | 6.4392 | 30.4 | 78500 | 6.4202 |
206
+ | 6.4311 | 30.6 | 79000 | 6.4361 |
207
+ | 6.3963 | 30.79 | 79500 | 6.4346 |
208
+ | 6.3872 | 30.98 | 80000 | 6.3810 |
209
+ | 6.4277 | 31.18 | 80500 | 6.4451 |
210
+ | 6.4112 | 31.37 | 81000 | 6.4243 |
211
+ | 6.4202 | 31.56 | 81500 | 6.4502 |
212
+ | 6.444 | 31.76 | 82000 | 6.4572 |
213
+ | 6.4066 | 31.95 | 82500 | 6.4033 |
214
+ | 6.4101 | 32.15 | 83000 | 6.4154 |
215
+ | 6.3985 | 32.34 | 83500 | 6.4377 |
216
+ | 6.4294 | 32.53 | 84000 | 6.4392 |
217
+ | 6.397 | 32.73 | 84500 | 6.4387 |
218
+ | 6.4217 | 32.92 | 85000 | 6.4305 |
219
+ | 6.4061 | 33.11 | 85500 | 6.4541 |
220
+ | 6.4014 | 33.31 | 86000 | 6.4173 |
221
+ | 6.4223 | 33.5 | 86500 | 6.4403 |
222
+ | 6.3953 | 33.69 | 87000 | 6.4333 |
223
+ | 6.4135 | 33.89 | 87500 | 6.4183 |
224
+ | 6.3955 | 34.08 | 88000 | 6.3958 |
225
+ | 6.4064 | 34.28 | 88500 | 6.3913 |
226
+ | 6.3997 | 34.47 | 89000 | 6.4330 |
227
+ | 6.4212 | 34.66 | 89500 | 6.3955 |
228
+ | 6.3957 | 34.86 | 90000 | 6.4438 |
229
+ | 6.3936 | 35.05 | 90500 | 6.4382 |
230
+ | 6.3927 | 35.24 | 91000 | 6.4055 |
231
+ | 6.3972 | 35.44 | 91500 | 6.4006 |
232
+ | 6.4137 | 35.63 | 92000 | 6.4245 |
233
+ | 6.3947 | 35.82 | 92500 | 6.4057 |
234
+ | 6.3798 | 36.02 | 93000 | 6.4006 |
235
+ | 6.4011 | 36.21 | 93500 | 6.3943 |
236
+ | 6.4012 | 36.41 | 94000 | 6.3766 |
237
+ | 6.3961 | 36.6 | 94500 | 6.4260 |
238
+ | 6.3819 | 36.79 | 95000 | 6.3801 |
239
+ | 6.3795 | 36.99 | 95500 | 6.4019 |
240
+ | 6.3954 | 37.18 | 96000 | 6.4387 |
241
+ | 6.3874 | 37.37 | 96500 | 6.4477 |
242
+ | 6.3844 | 37.57 | 97000 | 6.4177 |
243
+ | 6.3898 | 37.76 | 97500 | 6.4213 |
244
+ | 6.3855 | 37.96 | 98000 | 6.3838 |
245
+ | 6.3825 | 38.15 | 98500 | 6.4048 |
246
+ | 6.3615 | 38.34 | 99000 | 6.4636 |
247
+ | 6.392 | 38.54 | 99500 | 6.4197 |
248
+ | 6.3773 | 38.73 | 100000 | 6.4505 |
249
+ | 6.3834 | 38.92 | 100500 | 6.3889 |
250
+ | 6.3846 | 39.12 | 101000 | 6.4394 |
251
+ | 6.376 | 39.31 | 101500 | 6.3923 |
252
+ | 6.3699 | 39.5 | 102000 | 6.4025 |
253
+ | 6.3826 | 39.7 | 102500 | 6.3951 |
254
+ | 6.373 | 39.89 | 103000 | 6.3968 |
255
+
256
+
257
+ ### Framework versions
258
+
259
+ - Transformers 4.24.0
260
+ - Pytorch 1.12.1+cu113
261
+ - Datasets 2.7.1
262
+ - Tokenizers 0.13.2