DouglasPontes commited on
Commit
5f0397f
1 Parent(s): 14d7a61

Model save

Browse files
Files changed (2) hide show
  1. README.md +305 -304
  2. pytorch_model.bin +1 -1
README.md CHANGED
@@ -4,18 +4,18 @@ base_model: cardiffnlp/twitter-roberta-base-2019-90m
4
  tags:
5
  - generated_from_trainer
6
  model-index:
7
- - name: 2020-Q1-50p-filtered
8
  results: []
9
  ---
10
 
11
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
  should probably proofread and complete it, then remove this comment. -->
13
 
14
- # 2020-Q1-50p-filtered
15
 
16
  This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-2019-90m](https://huggingface.co/cardiffnlp/twitter-roberta-base-2019-90m) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 2.4514
19
 
20
  ## Model description
21
 
@@ -34,318 +34,319 @@ More information needed
34
  ### Training hyperparameters
35
 
36
  The following hyperparameters were used during training:
37
- - learning_rate: 4.1e-07
38
  - train_batch_size: 16
39
  - eval_batch_size: 16
40
  - seed: 42
41
  - optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
42
  - lr_scheduler_type: linear
 
43
  - training_steps: 2400000
44
 
45
  ### Training results
46
 
47
  | Training Loss | Epoch | Step | Validation Loss |
48
  |:-------------:|:-----:|:-------:|:---------------:|
49
- | No log | 0.03 | 8000 | 2.8937 |
50
- | 3.073 | 0.07 | 16000 | 2.7660 |
51
- | 3.073 | 0.1 | 24000 | 2.7233 |
52
- | 2.8244 | 0.13 | 32000 | 2.6878 |
53
- | 2.8244 | 0.16 | 40000 | 2.6520 |
54
- | 2.7542 | 0.2 | 48000 | 2.6300 |
55
- | 2.7542 | 0.23 | 56000 | 2.6135 |
56
- | 2.7083 | 0.26 | 64000 | 2.6068 |
57
- | 2.7083 | 0.3 | 72000 | 2.5854 |
58
- | 2.6752 | 0.33 | 80000 | 2.5755 |
59
- | 2.6752 | 0.36 | 88000 | 2.5721 |
60
- | 2.6657 | 0.39 | 96000 | 2.5709 |
61
- | 2.6657 | 0.43 | 104000 | 2.5656 |
62
- | 2.6534 | 0.46 | 112000 | 2.5558 |
63
- | 2.6534 | 0.49 | 120000 | 2.5496 |
64
- | 2.646 | 0.52 | 128000 | 2.5471 |
65
- | 2.646 | 0.56 | 136000 | 2.5408 |
66
- | 2.625 | 0.59 | 144000 | 2.5315 |
67
- | 2.625 | 0.62 | 152000 | 2.5365 |
68
- | 2.6222 | 0.66 | 160000 | 2.5372 |
69
- | 2.6222 | 0.69 | 168000 | 2.5342 |
70
- | 2.6256 | 0.72 | 176000 | 2.5308 |
71
- | 2.6256 | 0.75 | 184000 | 2.5312 |
72
- | 2.6074 | 0.79 | 192000 | 2.5228 |
73
- | 2.6074 | 0.82 | 200000 | 2.5292 |
74
- | 2.6071 | 0.85 | 208000 | 2.5295 |
75
- | 2.6071 | 0.89 | 216000 | 2.5235 |
76
- | 2.5955 | 0.92 | 224000 | 2.5219 |
77
- | 2.5955 | 0.95 | 232000 | 2.5191 |
78
- | 2.6036 | 0.98 | 240000 | 2.5171 |
79
- | 2.6036 | 1.02 | 248000 | 2.5102 |
80
- | 2.6046 | 1.05 | 256000 | 2.5070 |
81
- | 2.6046 | 1.08 | 264000 | 2.5109 |
82
- | 2.5892 | 1.11 | 272000 | 2.5105 |
83
- | 2.5892 | 1.15 | 280000 | 2.5087 |
84
- | 2.5929 | 1.18 | 288000 | 2.5094 |
85
- | 2.5929 | 1.21 | 296000 | 2.5086 |
86
- | 2.5857 | 1.25 | 304000 | 2.4991 |
87
- | 2.5857 | 1.28 | 312000 | 2.5089 |
88
- | 2.5828 | 1.31 | 320000 | 2.5017 |
89
- | 2.5828 | 1.34 | 328000 | 2.5039 |
90
- | 2.5812 | 1.38 | 336000 | 2.5065 |
91
- | 2.5812 | 1.41 | 344000 | 2.5083 |
92
- | 2.5775 | 1.44 | 352000 | 2.5099 |
93
- | 2.5775 | 1.48 | 360000 | 2.5079 |
94
- | 2.5711 | 1.51 | 368000 | 2.4922 |
95
- | 2.5711 | 1.54 | 376000 | 2.5012 |
96
- | 2.5797 | 1.57 | 384000 | 2.4999 |
97
- | 2.5797 | 1.61 | 392000 | 2.4881 |
98
- | 2.5718 | 1.64 | 400000 | 2.4960 |
99
- | 2.5718 | 1.67 | 408000 | 2.4908 |
100
- | 2.5627 | 1.7 | 416000 | 2.4971 |
101
- | 2.5627 | 1.74 | 424000 | 2.4916 |
102
- | 2.5641 | 1.77 | 432000 | 2.4971 |
103
- | 2.5641 | 1.8 | 440000 | 2.4954 |
104
- | 2.5633 | 1.84 | 448000 | 2.4860 |
105
- | 2.5633 | 1.87 | 456000 | 2.4894 |
106
- | 2.5676 | 1.9 | 464000 | 2.4893 |
107
- | 2.5676 | 1.93 | 472000 | 2.4884 |
108
- | 2.5687 | 1.97 | 480000 | 2.4921 |
109
- | 2.5687 | 2.0 | 488000 | 2.4873 |
110
- | 2.5633 | 2.03 | 496000 | 2.4919 |
111
- | 2.5633 | 2.07 | 504000 | 2.4821 |
112
- | 2.5547 | 2.1 | 512000 | 2.4909 |
113
- | 2.5547 | 2.13 | 520000 | 2.4818 |
114
- | 2.5617 | 2.16 | 528000 | 2.4855 |
115
- | 2.5617 | 2.2 | 536000 | 2.4850 |
116
- | 2.5569 | 2.23 | 544000 | 2.4803 |
117
- | 2.5569 | 2.26 | 552000 | 2.4776 |
118
- | 2.5535 | 2.29 | 560000 | 2.4824 |
119
- | 2.5535 | 2.33 | 568000 | 2.4822 |
120
- | 2.5534 | 2.36 | 576000 | 2.4763 |
121
- | 2.5534 | 2.39 | 584000 | 2.4797 |
122
- | 2.5583 | 2.43 | 592000 | 2.4872 |
123
- | 2.5583 | 2.46 | 600000 | 2.4812 |
124
- | 2.5545 | 2.49 | 608000 | 2.4748 |
125
- | 2.5545 | 2.52 | 616000 | 2.4736 |
126
- | 2.5561 | 2.56 | 624000 | 2.4714 |
127
- | 2.5561 | 2.59 | 632000 | 2.4858 |
128
- | 2.5384 | 2.62 | 640000 | 2.4829 |
129
- | 2.5384 | 2.66 | 648000 | 2.4766 |
130
- | 2.541 | 2.69 | 656000 | 2.4836 |
131
- | 2.541 | 2.72 | 664000 | 2.4651 |
132
- | 2.5439 | 2.75 | 672000 | 2.4797 |
133
- | 2.5439 | 2.79 | 680000 | 2.4702 |
134
- | 2.5597 | 2.82 | 688000 | 2.4751 |
135
- | 2.5597 | 2.85 | 696000 | 2.4744 |
136
- | 2.5491 | 2.88 | 704000 | 2.4756 |
137
- | 2.5491 | 2.92 | 712000 | 2.4731 |
138
- | 2.5505 | 2.95 | 720000 | 2.4756 |
139
- | 2.5505 | 2.98 | 728000 | 2.4704 |
140
- | 2.5432 | 3.02 | 736000 | 2.4763 |
141
- | 2.5432 | 3.05 | 744000 | 2.4743 |
142
- | 2.5485 | 3.08 | 752000 | 2.4627 |
143
- | 2.5485 | 3.11 | 760000 | 2.4714 |
144
- | 2.5482 | 3.15 | 768000 | 2.4685 |
145
- | 2.5482 | 3.18 | 776000 | 2.4673 |
146
- | 2.5411 | 3.21 | 784000 | 2.4726 |
147
- | 2.5411 | 3.25 | 792000 | 2.4761 |
148
- | 2.5407 | 3.28 | 800000 | 2.4612 |
149
- | 2.5407 | 3.31 | 808000 | 2.4743 |
150
- | 2.5307 | 3.34 | 816000 | 2.4699 |
151
- | 2.5307 | 3.38 | 824000 | 2.4721 |
152
- | 2.5391 | 3.41 | 832000 | 2.4614 |
153
- | 2.5391 | 3.44 | 840000 | 2.4641 |
154
- | 2.5378 | 3.47 | 848000 | 2.4652 |
155
- | 2.5378 | 3.51 | 856000 | 2.4641 |
156
- | 2.5399 | 3.54 | 864000 | 2.4691 |
157
- | 2.5399 | 3.57 | 872000 | 2.4612 |
158
- | 2.5412 | 3.61 | 880000 | 2.4696 |
159
- | 2.5412 | 3.64 | 888000 | 2.4638 |
160
- | 2.5389 | 3.67 | 896000 | 2.4658 |
161
- | 2.5389 | 3.7 | 904000 | 2.4725 |
162
- | 2.5325 | 3.74 | 912000 | 2.4642 |
163
- | 2.5325 | 3.77 | 920000 | 2.4599 |
164
- | 2.5351 | 3.8 | 928000 | 2.4617 |
165
- | 2.5351 | 3.84 | 936000 | 2.4646 |
166
- | 2.522 | 3.87 | 944000 | 2.4665 |
167
- | 2.522 | 3.9 | 952000 | 2.4762 |
168
- | 2.5331 | 3.93 | 960000 | 2.4669 |
169
- | 2.5331 | 3.97 | 968000 | 2.4550 |
170
- | 2.5276 | 4.0 | 976000 | 2.4662 |
171
- | 2.5276 | 4.03 | 984000 | 2.4645 |
172
- | 2.5206 | 4.06 | 992000 | 2.4587 |
173
- | 2.5206 | 4.1 | 1000000 | 2.4725 |
174
- | 2.5294 | 4.13 | 1008000 | 2.4588 |
175
- | 2.5294 | 4.16 | 1016000 | 2.4591 |
176
- | 2.5312 | 4.2 | 1024000 | 2.4681 |
177
- | 2.5312 | 4.23 | 1032000 | 2.4625 |
178
- | 2.525 | 4.26 | 1040000 | 2.4659 |
179
- | 2.525 | 4.29 | 1048000 | 2.4609 |
180
- | 2.5318 | 4.33 | 1056000 | 2.4571 |
181
- | 2.5318 | 4.36 | 1064000 | 2.4582 |
182
- | 2.5332 | 4.39 | 1072000 | 2.4566 |
183
- | 2.5332 | 4.43 | 1080000 | 2.4588 |
184
- | 2.5168 | 4.46 | 1088000 | 2.4606 |
185
- | 2.5168 | 4.49 | 1096000 | 2.4598 |
186
- | 2.5181 | 4.52 | 1104000 | 2.4543 |
187
- | 2.5181 | 4.56 | 1112000 | 2.4620 |
188
- | 2.5246 | 4.59 | 1120000 | 2.4639 |
189
- | 2.5246 | 4.62 | 1128000 | 2.4556 |
190
- | 2.5318 | 4.65 | 1136000 | 2.4571 |
191
- | 2.5318 | 4.69 | 1144000 | 2.4636 |
192
- | 2.512 | 4.72 | 1152000 | 2.4568 |
193
- | 2.512 | 4.75 | 1160000 | 2.4644 |
194
- | 2.5174 | 4.79 | 1168000 | 2.4529 |
195
- | 2.5174 | 4.82 | 1176000 | 2.4614 |
196
- | 2.5196 | 4.85 | 1184000 | 2.4638 |
197
- | 2.5196 | 4.88 | 1192000 | 2.4534 |
198
- | 2.5248 | 4.92 | 1200000 | 2.4553 |
199
- | 2.5248 | 4.95 | 1208000 | 2.4537 |
200
- | 2.5201 | 4.98 | 1216000 | 2.4579 |
201
- | 2.5201 | 5.02 | 1224000 | 2.4525 |
202
- | 2.5164 | 5.05 | 1232000 | 2.4645 |
203
- | 2.5164 | 5.08 | 1240000 | 2.4480 |
204
- | 2.5186 | 5.11 | 1248000 | 2.4606 |
205
- | 2.5186 | 5.15 | 1256000 | 2.4623 |
206
- | 2.5123 | 5.18 | 1264000 | 2.4566 |
207
- | 2.5123 | 5.21 | 1272000 | 2.4644 |
208
- | 2.5233 | 5.24 | 1280000 | 2.4576 |
209
- | 2.5233 | 5.28 | 1288000 | 2.4519 |
210
- | 2.513 | 5.31 | 1296000 | 2.4570 |
211
- | 2.513 | 5.34 | 1304000 | 2.4627 |
212
- | 2.5226 | 5.38 | 1312000 | 2.4500 |
213
- | 2.5226 | 5.41 | 1320000 | 2.4563 |
214
- | 2.5222 | 5.44 | 1328000 | 2.4521 |
215
- | 2.5222 | 5.47 | 1336000 | 2.4591 |
216
- | 2.5191 | 5.51 | 1344000 | 2.4509 |
217
- | 2.5191 | 5.54 | 1352000 | 2.4559 |
218
- | 2.5243 | 5.57 | 1360000 | 2.4502 |
219
- | 2.5243 | 5.61 | 1368000 | 2.4515 |
220
- | 2.5157 | 5.64 | 1376000 | 2.4563 |
221
- | 2.5157 | 5.67 | 1384000 | 2.4526 |
222
- | 2.5162 | 5.7 | 1392000 | 2.4586 |
223
- | 2.5162 | 5.74 | 1400000 | 2.4584 |
224
- | 2.5169 | 5.77 | 1408000 | 2.4542 |
225
- | 2.5169 | 5.8 | 1416000 | 2.4602 |
226
- | 2.5127 | 5.84 | 1424000 | 2.4587 |
227
- | 2.5127 | 5.87 | 1432000 | 2.4529 |
228
- | 2.5144 | 5.9 | 1440000 | 2.4620 |
229
- | 2.5144 | 5.93 | 1448000 | 2.4509 |
230
- | 2.5175 | 5.97 | 1456000 | 2.4503 |
231
- | 2.5175 | 6.0 | 1464000 | 2.4545 |
232
- | 2.5147 | 6.03 | 1472000 | 2.4440 |
233
- | 2.5147 | 6.06 | 1480000 | 2.4577 |
234
- | 2.5128 | 6.1 | 1488000 | 2.4566 |
235
- | 2.5128 | 6.13 | 1496000 | 2.4499 |
236
- | 2.5168 | 6.16 | 1504000 | 2.4480 |
237
- | 2.5168 | 6.2 | 1512000 | 2.4436 |
238
- | 2.5225 | 6.23 | 1520000 | 2.4467 |
239
- | 2.5225 | 6.26 | 1528000 | 2.4520 |
240
- | 2.5135 | 6.29 | 1536000 | 2.4535 |
241
- | 2.5135 | 6.33 | 1544000 | 2.4463 |
242
- | 2.5161 | 6.36 | 1552000 | 2.4556 |
243
- | 2.5161 | 6.39 | 1560000 | 2.4605 |
244
- | 2.5144 | 6.43 | 1568000 | 2.4516 |
245
- | 2.5144 | 6.46 | 1576000 | 2.4488 |
246
- | 2.5209 | 6.49 | 1584000 | 2.4525 |
247
- | 2.5209 | 6.52 | 1592000 | 2.4502 |
248
- | 2.5102 | 6.56 | 1600000 | 2.4538 |
249
- | 2.5102 | 6.59 | 1608000 | 2.4491 |
250
- | 2.5176 | 6.62 | 1616000 | 2.4528 |
251
- | 2.5176 | 6.65 | 1624000 | 2.4460 |
252
- | 2.5208 | 6.69 | 1632000 | 2.4485 |
253
- | 2.5208 | 6.72 | 1640000 | 2.4513 |
254
- | 2.5064 | 6.75 | 1648000 | 2.4519 |
255
- | 2.5064 | 6.79 | 1656000 | 2.4493 |
256
- | 2.5111 | 6.82 | 1664000 | 2.4505 |
257
- | 2.5111 | 6.85 | 1672000 | 2.4502 |
258
- | 2.5141 | 6.88 | 1680000 | 2.4560 |
259
- | 2.5141 | 6.92 | 1688000 | 2.4500 |
260
- | 2.5089 | 6.95 | 1696000 | 2.4513 |
261
- | 2.5089 | 6.98 | 1704000 | 2.4418 |
262
- | 2.5174 | 7.02 | 1712000 | 2.4477 |
263
- | 2.5174 | 7.05 | 1720000 | 2.4508 |
264
- | 2.5198 | 7.08 | 1728000 | 2.4486 |
265
- | 2.5198 | 7.11 | 1736000 | 2.4577 |
266
- | 2.4974 | 7.15 | 1744000 | 2.4416 |
267
- | 2.4974 | 7.18 | 1752000 | 2.4549 |
268
- | 2.5016 | 7.21 | 1760000 | 2.4557 |
269
- | 2.5016 | 7.24 | 1768000 | 2.4532 |
270
- | 2.5112 | 7.28 | 1776000 | 2.4451 |
271
- | 2.5112 | 7.31 | 1784000 | 2.4607 |
272
- | 2.5172 | 7.34 | 1792000 | 2.4452 |
273
- | 2.5172 | 7.38 | 1800000 | 2.4427 |
274
- | 2.5089 | 7.41 | 1808000 | 2.4511 |
275
- | 2.5089 | 7.44 | 1816000 | 2.4441 |
276
- | 2.5136 | 7.47 | 1824000 | 2.4492 |
277
- | 2.5136 | 7.51 | 1832000 | 2.4524 |
278
- | 2.509 | 7.54 | 1840000 | 2.4512 |
279
- | 2.509 | 7.57 | 1848000 | 2.4528 |
280
- | 2.5157 | 7.61 | 1856000 | 2.4440 |
281
- | 2.5157 | 7.64 | 1864000 | 2.4402 |
282
- | 2.5181 | 7.67 | 1872000 | 2.4538 |
283
- | 2.5181 | 7.7 | 1880000 | 2.4481 |
284
- | 2.5145 | 7.74 | 1888000 | 2.4417 |
285
- | 2.5145 | 7.77 | 1896000 | 2.4512 |
286
- | 2.5013 | 7.8 | 1904000 | 2.4560 |
287
- | 2.5013 | 7.83 | 1912000 | 2.4509 |
288
- | 2.5064 | 7.87 | 1920000 | 2.4473 |
289
- | 2.5064 | 7.9 | 1928000 | 2.4576 |
290
- | 2.5068 | 7.93 | 1936000 | 2.4461 |
291
- | 2.5068 | 7.97 | 1944000 | 2.4451 |
292
- | 2.5152 | 8.0 | 1952000 | 2.4421 |
293
- | 2.5152 | 8.03 | 1960000 | 2.4458 |
294
- | 2.5025 | 8.06 | 1968000 | 2.4532 |
295
- | 2.5025 | 8.1 | 1976000 | 2.4541 |
296
- | 2.5151 | 8.13 | 1984000 | 2.4499 |
297
- | 2.5151 | 8.16 | 1992000 | 2.4501 |
298
- | 2.5138 | 8.2 | 2000000 | 2.4448 |
299
- | 2.5138 | 8.23 | 2008000 | 2.4562 |
300
- | 2.5039 | 8.26 | 2016000 | 2.4613 |
301
- | 2.5039 | 8.29 | 2024000 | 2.4471 |
302
- | 2.5055 | 8.33 | 2032000 | 2.4450 |
303
- | 2.5055 | 8.36 | 2040000 | 2.4493 |
304
- | 2.5085 | 8.39 | 2048000 | 2.4482 |
305
- | 2.5085 | 8.42 | 2056000 | 2.4572 |
306
- | 2.5114 | 8.46 | 2064000 | 2.4443 |
307
- | 2.5114 | 8.49 | 2072000 | 2.4456 |
308
- | 2.5132 | 8.52 | 2080000 | 2.4528 |
309
- | 2.5132 | 8.56 | 2088000 | 2.4497 |
310
- | 2.5072 | 8.59 | 2096000 | 2.4548 |
311
- | 2.5072 | 8.62 | 2104000 | 2.4548 |
312
- | 2.504 | 8.65 | 2112000 | 2.4443 |
313
- | 2.504 | 8.69 | 2120000 | 2.4452 |
314
- | 2.5128 | 8.72 | 2128000 | 2.4510 |
315
- | 2.5128 | 8.75 | 2136000 | 2.4480 |
316
- | 2.5133 | 8.79 | 2144000 | 2.4470 |
317
- | 2.5133 | 8.82 | 2152000 | 2.4437 |
318
- | 2.5067 | 8.85 | 2160000 | 2.4447 |
319
- | 2.5067 | 8.88 | 2168000 | 2.4531 |
320
- | 2.4996 | 8.92 | 2176000 | 2.4475 |
321
- | 2.4996 | 8.95 | 2184000 | 2.4438 |
322
- | 2.5123 | 8.98 | 2192000 | 2.4552 |
323
- | 2.5123 | 9.01 | 2200000 | 2.4441 |
324
- | 2.5044 | 9.05 | 2208000 | 2.4438 |
325
- | 2.5044 | 9.08 | 2216000 | 2.4534 |
326
- | 2.5068 | 9.11 | 2224000 | 2.4497 |
327
- | 2.5068 | 9.15 | 2232000 | 2.4440 |
328
- | 2.5165 | 9.18 | 2240000 | 2.4577 |
329
- | 2.5165 | 9.21 | 2248000 | 2.4507 |
330
- | 2.5087 | 9.24 | 2256000 | 2.4494 |
331
- | 2.5087 | 9.28 | 2264000 | 2.4393 |
332
- | 2.5036 | 9.31 | 2272000 | 2.4487 |
333
- | 2.5036 | 9.34 | 2280000 | 2.4423 |
334
- | 2.5086 | 9.38 | 2288000 | 2.4456 |
335
- | 2.5086 | 9.41 | 2296000 | 2.4496 |
336
- | 2.5034 | 9.44 | 2304000 | 2.4499 |
337
- | 2.5034 | 9.47 | 2312000 | 2.4433 |
338
- | 2.5099 | 9.51 | 2320000 | 2.4534 |
339
- | 2.5099 | 9.54 | 2328000 | 2.4495 |
340
- | 2.5065 | 9.57 | 2336000 | 2.4510 |
341
- | 2.5065 | 9.6 | 2344000 | 2.4513 |
342
- | 2.502 | 9.64 | 2352000 | 2.4512 |
343
- | 2.502 | 9.67 | 2360000 | 2.4469 |
344
- | 2.5043 | 9.7 | 2368000 | 2.4544 |
345
- | 2.5043 | 9.74 | 2376000 | 2.4493 |
346
- | 2.5068 | 9.77 | 2384000 | 2.4537 |
347
- | 2.5068 | 9.8 | 2392000 | 2.4387 |
348
- | 2.5118 | 9.83 | 2400000 | 2.4494 |
349
 
350
 
351
  ### Framework versions
 
4
  tags:
5
  - generated_from_trainer
6
  model-index:
7
+ - name: 2020-Q2-50p-filtered
8
  results: []
9
  ---
10
 
11
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
  should probably proofread and complete it, then remove this comment. -->
13
 
14
+ # 2020-Q2-50p-filtered
15
 
16
  This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-2019-90m](https://huggingface.co/cardiffnlp/twitter-roberta-base-2019-90m) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 2.8566
19
 
20
  ## Model description
21
 
 
34
  ### Training hyperparameters
35
 
36
  The following hyperparameters were used during training:
37
+ - learning_rate: 1e-05
38
  - train_batch_size: 16
39
  - eval_batch_size: 16
40
  - seed: 42
41
  - optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
42
  - lr_scheduler_type: linear
43
+ - lr_scheduler_warmup_steps: 1400
44
  - training_steps: 2400000
45
 
46
  ### Training results
47
 
48
  | Training Loss | Epoch | Step | Validation Loss |
49
  |:-------------:|:-----:|:-------:|:---------------:|
50
+ | No log | 0.03 | 8000 | 2.6645 |
51
+ | 2.8656 | 0.07 | 16000 | 2.6465 |
52
+ | 2.8656 | 0.1 | 24000 | 2.6186 |
53
+ | 2.7946 | 0.13 | 32000 | 2.6235 |
54
+ | 2.7946 | 0.17 | 40000 | 2.6151 |
55
+ | 2.7911 | 0.2 | 48000 | 2.6128 |
56
+ | 2.7911 | 0.24 | 56000 | 2.6010 |
57
+ | 2.7898 | 0.27 | 64000 | 2.6144 |
58
+ | 2.7898 | 0.3 | 72000 | 2.5976 |
59
+ | 2.7791 | 0.34 | 80000 | 2.6006 |
60
+ | 2.7791 | 0.37 | 88000 | 2.5889 |
61
+ | 2.7776 | 0.4 | 96000 | 2.5888 |
62
+ | 2.7776 | 0.44 | 104000 | 2.5842 |
63
+ | 2.7702 | 0.47 | 112000 | 2.5760 |
64
+ | 2.7702 | 0.51 | 120000 | 2.5720 |
65
+ | 2.7661 | 0.54 | 128000 | 2.5710 |
66
+ | 2.7661 | 0.57 | 136000 | 2.5673 |
67
+ | 2.7609 | 0.61 | 144000 | 2.5693 |
68
+ | 2.7609 | 0.64 | 152000 | 2.5623 |
69
+ | 2.7557 | 0.67 | 160000 | 2.5559 |
70
+ | 2.7557 | 0.71 | 168000 | 2.5650 |
71
+ | 2.7584 | 0.74 | 176000 | 2.5584 |
72
+ | 2.7584 | 0.77 | 184000 | 2.5591 |
73
+ | 2.7619 | 0.81 | 192000 | 2.5597 |
74
+ | 2.7619 | 0.84 | 200000 | 2.5650 |
75
+ | 2.7678 | 0.88 | 208000 | 2.5728 |
76
+ | 2.7678 | 0.91 | 216000 | 2.5712 |
77
+ | 2.7735 | 0.94 | 224000 | 2.5729 |
78
+ | 2.7735 | 0.98 | 232000 | 2.5755 |
79
+ | 2.777 | 1.01 | 240000 | 2.5715 |
80
+ | 2.777 | 1.04 | 248000 | 2.5747 |
81
+ | 2.7692 | 1.08 | 256000 | 2.5782 |
82
+ | 2.7692 | 1.11 | 264000 | 2.5841 |
83
+ | 2.7826 | 1.15 | 272000 | 2.5731 |
84
+ | 2.7826 | 1.18 | 280000 | 2.5836 |
85
+ | 2.7845 | 1.21 | 288000 | 2.5841 |
86
+ | 2.7845 | 1.25 | 296000 | 2.5811 |
87
+ | 2.7909 | 1.28 | 304000 | 2.5928 |
88
+ | 2.7909 | 1.31 | 312000 | 2.5977 |
89
+ | 2.7993 | 1.35 | 320000 | 2.6025 |
90
+ | 2.7993 | 1.38 | 328000 | 2.6072 |
91
+ | 2.8107 | 1.41 | 336000 | 2.6110 |
92
+ | 2.8107 | 1.45 | 344000 | 2.6020 |
93
+ | 2.8102 | 1.48 | 352000 | 2.6065 |
94
+ | 2.8102 | 1.52 | 360000 | 2.6207 |
95
+ | 2.8247 | 1.55 | 368000 | 2.6192 |
96
+ | 2.8247 | 1.58 | 376000 | 2.6224 |
97
+ | 2.8271 | 1.62 | 384000 | 2.6205 |
98
+ | 2.8271 | 1.65 | 392000 | 2.6292 |
99
+ | 2.8415 | 1.68 | 400000 | 2.6348 |
100
+ | 2.8415 | 1.72 | 408000 | 2.6518 |
101
+ | 2.842 | 1.75 | 416000 | 2.6465 |
102
+ | 2.842 | 1.79 | 424000 | 2.6434 |
103
+ | 2.8431 | 1.82 | 432000 | 2.6414 |
104
+ | 2.8431 | 1.85 | 440000 | 2.6532 |
105
+ | 2.8599 | 1.89 | 448000 | 2.6645 |
106
+ | 2.8599 | 1.92 | 456000 | 2.6651 |
107
+ | 2.8567 | 1.95 | 464000 | 2.6694 |
108
+ | 2.8567 | 1.99 | 472000 | 2.6610 |
109
+ | 2.8682 | 2.02 | 480000 | 2.6877 |
110
+ | 2.8682 | 2.05 | 488000 | 2.6724 |
111
+ | 2.8693 | 2.09 | 496000 | 2.6839 |
112
+ | 2.8693 | 2.12 | 504000 | 2.6923 |
113
+ | 2.8881 | 2.16 | 512000 | 2.6964 |
114
+ | 2.8881 | 2.19 | 520000 | 2.6982 |
115
+ | 2.8874 | 2.22 | 528000 | 2.6961 |
116
+ | 2.8874 | 2.26 | 536000 | 2.6884 |
117
+ | 2.8899 | 2.29 | 544000 | 2.7055 |
118
+ | 2.8899 | 2.32 | 552000 | 2.6988 |
119
+ | 2.8966 | 2.36 | 560000 | 2.7103 |
120
+ | 2.8966 | 2.39 | 568000 | 2.7100 |
121
+ | 2.9 | 2.43 | 576000 | 2.7169 |
122
+ | 2.9 | 2.46 | 584000 | 2.7180 |
123
+ | 2.9237 | 2.49 | 592000 | 2.7270 |
124
+ | 2.9237 | 2.53 | 600000 | 2.7265 |
125
+ | 2.9236 | 2.56 | 608000 | 2.7323 |
126
+ | 2.9236 | 2.59 | 616000 | 2.7350 |
127
+ | 2.9276 | 2.63 | 624000 | 2.7333 |
128
+ | 2.9276 | 2.66 | 632000 | 2.7345 |
129
+ | 2.9252 | 2.69 | 640000 | 2.7497 |
130
+ | 2.9252 | 2.73 | 648000 | 2.7428 |
131
+ | 2.9364 | 2.76 | 656000 | 2.7392 |
132
+ | 2.9364 | 2.8 | 664000 | 2.7505 |
133
+ | 2.9366 | 2.83 | 672000 | 2.7393 |
134
+ | 2.9366 | 2.86 | 680000 | 2.7372 |
135
+ | 2.9437 | 2.9 | 688000 | 2.7451 |
136
+ | 2.9437 | 2.93 | 696000 | 2.7488 |
137
+ | 2.9483 | 2.96 | 704000 | 2.7586 |
138
+ | 2.9483 | 3.0 | 712000 | 2.7613 |
139
+ | 2.9588 | 3.03 | 720000 | 2.7619 |
140
+ | 2.9588 | 3.07 | 728000 | 2.7680 |
141
+ | 2.9422 | 3.1 | 736000 | 2.7546 |
142
+ | 2.9422 | 3.13 | 744000 | 2.7629 |
143
+ | 2.965 | 3.17 | 752000 | 2.7595 |
144
+ | 2.965 | 3.2 | 760000 | 2.7763 |
145
+ | 2.959 | 3.23 | 768000 | 2.7739 |
146
+ | 2.959 | 3.27 | 776000 | 2.7839 |
147
+ | 2.9604 | 3.3 | 784000 | 2.7681 |
148
+ | 2.9604 | 3.33 | 792000 | 2.7816 |
149
+ | 2.9638 | 3.37 | 800000 | 2.7812 |
150
+ | 2.9638 | 3.4 | 808000 | 2.7846 |
151
+ | 2.9704 | 3.44 | 816000 | 2.7766 |
152
+ | 2.9704 | 3.47 | 824000 | 2.7869 |
153
+ | 2.9684 | 3.5 | 832000 | 2.7741 |
154
+ | 2.9684 | 3.54 | 840000 | 2.7735 |
155
+ | 2.9723 | 3.57 | 848000 | 2.7701 |
156
+ | 2.9723 | 3.6 | 856000 | 2.7780 |
157
+ | 2.9734 | 3.64 | 864000 | 2.7833 |
158
+ | 2.9734 | 3.67 | 872000 | 2.7910 |
159
+ | 2.9806 | 3.71 | 880000 | 2.7941 |
160
+ | 2.9806 | 3.74 | 888000 | 2.7997 |
161
+ | 2.9808 | 3.77 | 896000 | 2.8027 |
162
+ | 2.9808 | 3.81 | 904000 | 2.7972 |
163
+ | 3.0008 | 3.84 | 912000 | 2.8026 |
164
+ | 3.0008 | 3.87 | 920000 | 2.7975 |
165
+ | 2.9934 | 3.91 | 928000 | 2.7971 |
166
+ | 2.9934 | 3.94 | 936000 | 2.8030 |
167
+ | 2.9927 | 3.97 | 944000 | 2.8082 |
168
+ | 2.9927 | 4.01 | 952000 | 2.8208 |
169
+ | 3.0013 | 4.04 | 960000 | 2.8129 |
170
+ | 3.0013 | 4.08 | 968000 | 2.8236 |
171
+ | 2.9996 | 4.11 | 976000 | 2.8226 |
172
+ | 2.9996 | 4.14 | 984000 | 2.8273 |
173
+ | 3.0125 | 4.18 | 992000 | 2.8161 |
174
+ | 3.0125 | 4.21 | 1000000 | 2.8249 |
175
+ | 3.0086 | 4.24 | 1008000 | 2.8320 |
176
+ | 3.0086 | 4.28 | 1016000 | 2.8313 |
177
+ | 3.0077 | 4.31 | 1024000 | 2.8321 |
178
+ | 3.0077 | 4.35 | 1032000 | 2.8332 |
179
+ | 3.0186 | 4.38 | 1040000 | 2.8288 |
180
+ | 3.0186 | 4.41 | 1048000 | 2.8392 |
181
+ | 3.0311 | 4.45 | 1056000 | 2.8243 |
182
+ | 3.0311 | 4.48 | 1064000 | 2.8524 |
183
+ | 3.0199 | 4.51 | 1072000 | 2.8347 |
184
+ | 3.0199 | 4.55 | 1080000 | 2.8438 |
185
+ | 3.0198 | 4.58 | 1088000 | 2.8415 |
186
+ | 3.0198 | 4.61 | 1096000 | 2.8460 |
187
+ | 3.0279 | 4.65 | 1104000 | 2.8551 |
188
+ | 3.0279 | 4.68 | 1112000 | 2.8528 |
189
+ | 3.0319 | 4.72 | 1120000 | 2.8601 |
190
+ | 3.0319 | 4.75 | 1128000 | 2.8544 |
191
+ | 3.0371 | 4.78 | 1136000 | 2.8553 |
192
+ | 3.0371 | 4.82 | 1144000 | 2.8597 |
193
+ | 3.038 | 4.85 | 1152000 | 2.8653 |
194
+ | 3.038 | 4.88 | 1160000 | 2.8560 |
195
+ | 3.0318 | 4.92 | 1168000 | 2.8602 |
196
+ | 3.0318 | 4.95 | 1176000 | 2.8484 |
197
+ | 3.0449 | 4.99 | 1184000 | 2.8612 |
198
+ | 3.0449 | 5.02 | 1192000 | 2.8598 |
199
+ | 3.0384 | 5.05 | 1200000 | 2.8581 |
200
+ | 3.0384 | 5.09 | 1208000 | 2.8481 |
201
+ | 3.0243 | 5.12 | 1216000 | 2.8458 |
202
+ | 3.0243 | 5.15 | 1224000 | 2.8494 |
203
+ | 3.0345 | 5.19 | 1232000 | 2.8544 |
204
+ | 3.0345 | 5.22 | 1240000 | 2.8488 |
205
+ | 3.0251 | 5.25 | 1248000 | 2.8453 |
206
+ | 3.0251 | 5.29 | 1256000 | 2.8464 |
207
+ | 3.0234 | 5.32 | 1264000 | 2.8486 |
208
+ | 3.0234 | 5.36 | 1272000 | 2.8436 |
209
+ | 3.0205 | 5.39 | 1280000 | 2.8476 |
210
+ | 3.0205 | 5.42 | 1288000 | 2.8327 |
211
+ | 3.0228 | 5.46 | 1296000 | 2.8452 |
212
+ | 3.0228 | 5.49 | 1304000 | 2.8372 |
213
+ | 3.0063 | 5.52 | 1312000 | 2.8306 |
214
+ | 3.0063 | 5.56 | 1320000 | 2.8411 |
215
+ | 3.0068 | 5.59 | 1328000 | 2.8273 |
216
+ | 3.0068 | 5.63 | 1336000 | 2.8343 |
217
+ | 3.0109 | 5.66 | 1344000 | 2.8328 |
218
+ | 3.0109 | 5.69 | 1352000 | 2.8431 |
219
+ | 3.0068 | 5.73 | 1360000 | 2.8332 |
220
+ | 3.0068 | 5.76 | 1368000 | 2.8275 |
221
+ | 3.002 | 5.79 | 1376000 | 2.8314 |
222
+ | 3.002 | 5.83 | 1384000 | 2.8324 |
223
+ | 3.0037 | 5.86 | 1392000 | 2.8394 |
224
+ | 3.0037 | 5.89 | 1400000 | 2.8338 |
225
+ | 3.0086 | 5.93 | 1408000 | 2.8448 |
226
+ | 3.0086 | 5.96 | 1416000 | 2.8326 |
227
+ | 2.9977 | 6.0 | 1424000 | 2.8311 |
228
+ | 2.9977 | 6.03 | 1432000 | 2.8410 |
229
+ | 2.9984 | 6.06 | 1440000 | 2.8359 |
230
+ | 2.9984 | 6.1 | 1448000 | 2.8393 |
231
+ | 3.0095 | 6.13 | 1456000 | 2.8388 |
232
+ | 3.0095 | 6.16 | 1464000 | 2.8448 |
233
+ | 3.0051 | 6.2 | 1472000 | 2.8472 |
234
+ | 3.0051 | 6.23 | 1480000 | 2.8421 |
235
+ | 3.0142 | 6.27 | 1488000 | 2.8424 |
236
+ | 3.0142 | 6.3 | 1496000 | 2.8477 |
237
+ | 3.0149 | 6.33 | 1504000 | 2.8428 |
238
+ | 3.0149 | 6.37 | 1512000 | 2.8529 |
239
+ | 3.0147 | 6.4 | 1520000 | 2.8541 |
240
+ | 3.0147 | 6.43 | 1528000 | 2.8519 |
241
+ | 3.0205 | 6.47 | 1536000 | 2.8527 |
242
+ | 3.0205 | 6.5 | 1544000 | 2.8471 |
243
+ | 3.029 | 6.53 | 1552000 | 2.8583 |
244
+ | 3.029 | 6.57 | 1560000 | 2.8497 |
245
+ | 3.024 | 6.6 | 1568000 | 2.8653 |
246
+ | 3.024 | 6.64 | 1576000 | 2.8553 |
247
+ | 3.0371 | 6.67 | 1584000 | 2.8653 |
248
+ | 3.0371 | 6.7 | 1592000 | 2.8604 |
249
+ | 3.0319 | 6.74 | 1600000 | 2.8624 |
250
+ | 3.0319 | 6.77 | 1608000 | 2.8657 |
251
+ | 3.0369 | 6.8 | 1616000 | 2.8616 |
252
+ | 3.0369 | 6.84 | 1624000 | 2.8667 |
253
+ | 3.0357 | 6.87 | 1632000 | 2.8660 |
254
+ | 3.0357 | 6.91 | 1640000 | 2.8682 |
255
+ | 3.0342 | 6.94 | 1648000 | 2.8676 |
256
+ | 3.0342 | 6.97 | 1656000 | 2.8815 |
257
+ | 3.0375 | 7.01 | 1664000 | 2.8667 |
258
+ | 3.0375 | 7.04 | 1672000 | 2.8735 |
259
+ | 3.0419 | 7.07 | 1680000 | 2.8788 |
260
+ | 3.0419 | 7.11 | 1688000 | 2.8767 |
261
+ | 3.0403 | 7.14 | 1696000 | 2.8812 |
262
+ | 3.0403 | 7.17 | 1704000 | 2.8795 |
263
+ | 3.0482 | 7.21 | 1712000 | 2.8805 |
264
+ | 3.0482 | 7.24 | 1720000 | 2.8794 |
265
+ | 3.0533 | 7.28 | 1728000 | 2.8788 |
266
+ | 3.0533 | 7.31 | 1736000 | 2.8844 |
267
+ | 3.0453 | 7.34 | 1744000 | 2.8709 |
268
+ | 3.0453 | 7.38 | 1752000 | 2.8835 |
269
+ | 3.0562 | 7.41 | 1760000 | 2.8891 |
270
+ | 3.0562 | 7.44 | 1768000 | 2.8903 |
271
+ | 3.0617 | 7.48 | 1776000 | 2.8849 |
272
+ | 3.0617 | 7.51 | 1784000 | 2.8766 |
273
+ | 3.0539 | 7.55 | 1792000 | 2.8872 |
274
+ | 3.0539 | 7.58 | 1800000 | 2.8981 |
275
+ | 3.0561 | 7.61 | 1808000 | 2.8862 |
276
+ | 3.0561 | 7.65 | 1816000 | 2.8940 |
277
+ | 3.0529 | 7.68 | 1824000 | 2.8874 |
278
+ | 3.0529 | 7.71 | 1832000 | 2.8839 |
279
+ | 3.0484 | 7.75 | 1840000 | 2.8838 |
280
+ | 3.0484 | 7.78 | 1848000 | 2.8856 |
281
+ | 3.0562 | 7.81 | 1856000 | 2.8984 |
282
+ | 3.0562 | 7.85 | 1864000 | 2.8844 |
283
+ | 3.0578 | 7.88 | 1872000 | 2.8874 |
284
+ | 3.0578 | 7.92 | 1880000 | 2.8887 |
285
+ | 3.0553 | 7.95 | 1888000 | 2.8798 |
286
+ | 3.0553 | 7.98 | 1896000 | 2.8789 |
287
+ | 3.0623 | 8.02 | 1904000 | 2.8968 |
288
+ | 3.0623 | 8.05 | 1912000 | 2.8834 |
289
+ | 3.0652 | 8.08 | 1920000 | 2.8902 |
290
+ | 3.0652 | 8.12 | 1928000 | 2.8822 |
291
+ | 3.0487 | 8.15 | 1936000 | 2.8844 |
292
+ | 3.0487 | 8.19 | 1944000 | 2.8909 |
293
+ | 3.0546 | 8.22 | 1952000 | 2.8915 |
294
+ | 3.0546 | 8.25 | 1960000 | 2.8870 |
295
+ | 3.0524 | 8.29 | 1968000 | 2.8828 |
296
+ | 3.0524 | 8.32 | 1976000 | 2.8781 |
297
+ | 3.0491 | 8.35 | 1984000 | 2.8948 |
298
+ | 3.0491 | 8.39 | 1992000 | 2.8904 |
299
+ | 3.0534 | 8.42 | 2000000 | 2.8839 |
300
+ | 3.0534 | 8.45 | 2008000 | 2.8918 |
301
+ | 3.0547 | 8.49 | 2016000 | 2.8739 |
302
+ | 3.0547 | 8.52 | 2024000 | 2.8684 |
303
+ | 3.0544 | 8.56 | 2032000 | 2.8740 |
304
+ | 3.0544 | 8.59 | 2040000 | 2.8784 |
305
+ | 3.0448 | 8.62 | 2048000 | 2.8758 |
306
+ | 3.0448 | 8.66 | 2056000 | 2.8801 |
307
+ | 3.0499 | 8.69 | 2064000 | 2.8793 |
308
+ | 3.0499 | 8.72 | 2072000 | 2.8707 |
309
+ | 3.0368 | 8.76 | 2080000 | 2.8722 |
310
+ | 3.0368 | 8.79 | 2088000 | 2.8752 |
311
+ | 3.0548 | 8.83 | 2096000 | 2.8880 |
312
+ | 3.0548 | 8.86 | 2104000 | 2.8781 |
313
+ | 3.0457 | 8.89 | 2112000 | 2.8825 |
314
+ | 3.0457 | 8.93 | 2120000 | 2.8827 |
315
+ | 3.0377 | 8.96 | 2128000 | 2.8810 |
316
+ | 3.0377 | 8.99 | 2136000 | 2.8727 |
317
+ | 3.0341 | 9.03 | 2144000 | 2.8750 |
318
+ | 3.0341 | 9.06 | 2152000 | 2.8638 |
319
+ | 3.0275 | 9.09 | 2160000 | 2.8690 |
320
+ | 3.0275 | 9.13 | 2168000 | 2.8660 |
321
+ | 3.0413 | 9.16 | 2176000 | 2.8578 |
322
+ | 3.0413 | 9.2 | 2184000 | 2.8692 |
323
+ | 3.0272 | 9.23 | 2192000 | 2.8702 |
324
+ | 3.0272 | 9.26 | 2200000 | 2.8707 |
325
+ | 3.034 | 9.3 | 2208000 | 2.8666 |
326
+ | 3.034 | 9.33 | 2216000 | 2.8734 |
327
+ | 3.0346 | 9.36 | 2224000 | 2.8685 |
328
+ | 3.0346 | 9.4 | 2232000 | 2.8675 |
329
+ | 3.0234 | 9.43 | 2240000 | 2.8662 |
330
+ | 3.0234 | 9.47 | 2248000 | 2.8670 |
331
+ | 3.0256 | 9.5 | 2256000 | 2.8764 |
332
+ | 3.0256 | 9.53 | 2264000 | 2.8664 |
333
+ | 3.0232 | 9.57 | 2272000 | 2.8625 |
334
+ | 3.0232 | 9.6 | 2280000 | 2.8647 |
335
+ | 3.0309 | 9.63 | 2288000 | 2.8561 |
336
+ | 3.0309 | 9.67 | 2296000 | 2.8657 |
337
+ | 3.0254 | 9.7 | 2304000 | 2.8667 |
338
+ | 3.0254 | 9.73 | 2312000 | 2.8618 |
339
+ | 3.0198 | 9.77 | 2320000 | 2.8650 |
340
+ | 3.0198 | 9.8 | 2328000 | 2.8630 |
341
+ | 3.0109 | 9.84 | 2336000 | 2.8533 |
342
+ | 3.0109 | 9.87 | 2344000 | 2.8656 |
343
+ | 3.0316 | 9.9 | 2352000 | 2.8607 |
344
+ | 3.0316 | 9.94 | 2360000 | 2.8572 |
345
+ | 3.0225 | 9.97 | 2368000 | 2.8617 |
346
+ | 3.0225 | 10.0 | 2376000 | 2.8604 |
347
+ | 3.0132 | 10.04 | 2384000 | 2.8577 |
348
+ | 3.0132 | 10.07 | 2392000 | 2.8535 |
349
+ | 3.0202 | 10.11 | 2400000 | 2.8566 |
350
 
351
 
352
  ### Framework versions
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4f246e58f656cf3ea7eb5c9f06b705a9ee0bf6f8f61d017ad0cca9e173fe0ba9
3
  size 498859189
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cac39220d7bc7f4de8ceecc068c6892221f707297da8650ac0bd093ef12de89a
3
  size 498859189