File size: 23,304 Bytes
cffc241
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- common_voice
model-index:
- name: ''
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 2.3347
- Wer: 1.0286

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 7.5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 20
- num_epochs: 5.0
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 10.7838       | 0.01  | 5    | 14.5035         | 1.0    |
| 13.0582       | 0.03  | 10   | 13.6658         | 1.0    |
| 7.3034        | 0.04  | 15   | 9.7898          | 1.0    |
| 6.1847        | 0.05  | 20   | 6.9148          | 1.0    |
| 5.3371        | 0.07  | 25   | 5.3661          | 1.0    |
| 4.4274        | 0.08  | 30   | 4.6945          | 1.0    |
| 4.0918        | 0.1   | 35   | 4.3172          | 1.0    |
| 4.1734        | 0.11  | 40   | 4.0759          | 1.0    |
| 3.7332        | 0.12  | 45   | 3.9039          | 1.0    |
| 3.6871        | 0.14  | 50   | 3.7777          | 1.0    |
| 3.4428        | 0.15  | 55   | 3.6718          | 1.0    |
| 3.5514        | 0.16  | 60   | 3.5947          | 1.0    |
| 3.4307        | 0.18  | 65   | 3.5144          | 1.0    |
| 3.4102        | 0.19  | 70   | 3.4432          | 1.0    |
| 3.4964        | 0.21  | 75   | 3.3890          | 1.0    |
| 3.3936        | 0.22  | 80   | 3.3467          | 1.0    |
| 3.3051        | 0.23  | 85   | 3.3102          | 1.0    |
| 3.278         | 0.25  | 90   | 3.2801          | 1.0    |
| 3.2223        | 0.26  | 95   | 3.2440          | 1.0    |
| 3.1888        | 0.27  | 100  | 3.2900          | 1.0    |
| 3.218         | 0.29  | 105  | 3.2627          | 1.0    |
| 3.1308        | 0.3   | 110  | 3.2152          | 1.0    |
| 3.109         | 0.31  | 115  | 3.1686          | 1.0    |
| 3.1188        | 0.33  | 120  | 3.1734          | 1.0    |
| 3.1132        | 0.34  | 125  | 3.1431          | 1.0    |
| 3.0667        | 0.36  | 130  | 3.1686          | 1.0    |
| 3.1167        | 0.37  | 135  | 3.1885          | 1.0    |
| 3.0592        | 0.38  | 140  | 3.1100          | 1.0    |
| 3.0531        | 0.4   | 145  | 3.1149          | 1.0    |
| 3.1224        | 0.41  | 150  | 3.1205          | 1.0    |
| 3.0651        | 0.42  | 155  | 3.1101          | 1.0    |
| 3.0077        | 0.44  | 160  | 3.0980          | 1.0    |
| 3.0027        | 0.45  | 165  | 3.1132          | 1.0    |
| 3.0423        | 0.47  | 170  | 3.0886          | 1.0    |
| 3.0462        | 0.48  | 175  | 3.0865          | 1.0    |
| 3.0701        | 0.49  | 180  | 3.0863          | 1.0    |
| 3.0871        | 0.51  | 185  | 3.0825          | 1.0    |
| 3.0585        | 0.52  | 190  | 3.0720          | 1.0    |
| 3.0274        | 0.53  | 195  | 3.0736          | 1.0    |
| 3.0983        | 0.55  | 200  | 3.0658          | 1.0    |
| 3.0538        | 0.56  | 205  | 3.1241          | 1.0    |
| 3.0862        | 0.57  | 210  | 3.0573          | 1.0    |
| 3.0041        | 0.59  | 215  | 3.0608          | 1.0    |
| 3.027         | 0.6   | 220  | 3.0614          | 1.0    |
| 2.9916        | 0.62  | 225  | 3.0527          | 1.0    |
| 3.0157        | 0.63  | 230  | 3.0514          | 1.0    |
| 3.0429        | 0.64  | 235  | 3.0391          | 1.0    |
| 2.999         | 0.66  | 240  | 3.0462          | 1.0    |
| 3.0053        | 0.67  | 245  | 3.0438          | 1.0    |
| 2.9812        | 0.68  | 250  | 3.0447          | 1.0    |
| 3.0062        | 0.7   | 255  | 3.0660          | 1.0    |
| 3.0045        | 0.71  | 260  | 3.0103          | 1.0    |
| 2.9684        | 0.73  | 265  | 3.0106          | 1.0    |
| 2.9885        | 0.74  | 270  | 3.0014          | 1.0    |
| 3.0062        | 0.75  | 275  | 2.9885          | 1.0    |
| 2.9736        | 0.77  | 280  | 3.0330          | 1.0    |
| 2.9766        | 0.78  | 285  | 2.9910          | 1.0    |
| 2.9545        | 0.79  | 290  | 2.9972          | 1.0    |
| 2.9936        | 0.81  | 295  | 2.9872          | 1.0    |
| 3.0832        | 0.82  | 300  | 2.9978          | 1.0    |
| 2.974         | 0.83  | 305  | 2.9978          | 1.0    |
| 2.9846        | 0.85  | 310  | 2.9849          | 1.0    |
| 2.9554        | 0.86  | 315  | 2.9810          | 1.0    |
| 2.9524        | 0.88  | 320  | 2.9731          | 1.0    |
| 2.9426        | 0.89  | 325  | 2.9824          | 1.0    |
| 2.9416        | 0.9   | 330  | 2.9731          | 1.0    |
| 2.9705        | 0.92  | 335  | 2.9830          | 1.0    |
| 2.9502        | 0.93  | 340  | 2.9713          | 1.0    |
| 2.9393        | 0.94  | 345  | 2.9790          | 1.0    |
| 2.9336        | 0.96  | 350  | 2.9684          | 1.0    |
| 2.9542        | 0.97  | 355  | 2.9689          | 1.0    |
| 2.9408        | 0.98  | 360  | 2.9556          | 1.0    |
| 2.9544        | 1.0   | 365  | 2.9563          | 1.0    |
| 2.9187        | 1.01  | 370  | 2.9624          | 1.0    |
| 2.9935        | 1.03  | 375  | 2.9500          | 1.0    |
| 2.9803        | 1.04  | 380  | 2.9558          | 1.0    |
| 2.9867        | 1.05  | 385  | 2.9473          | 1.0    |
| 2.8925        | 1.07  | 390  | 2.9444          | 1.0    |
| 2.9633        | 1.08  | 395  | 2.9490          | 1.0    |
| 2.9191        | 1.1   | 400  | 2.9362          | 1.0    |
| 2.9081        | 1.11  | 405  | 2.9394          | 1.0    |
| 2.9381        | 1.12  | 410  | 2.9846          | 1.0    |
| 2.9271        | 1.14  | 415  | 2.9638          | 1.0    |
| 2.959         | 1.15  | 420  | 2.9835          | 1.0    |
| 2.9486        | 1.16  | 425  | 2.9361          | 1.0    |
| 2.9246        | 1.18  | 430  | 2.9615          | 1.0    |
| 2.923         | 1.19  | 435  | 2.9313          | 1.0    |
| 2.8908        | 1.21  | 440  | 2.9362          | 1.0    |
| 2.8976        | 1.22  | 445  | 2.9224          | 1.0    |
| 2.9278        | 1.23  | 450  | 2.9276          | 1.0    |
| 2.8429        | 1.25  | 455  | 2.9299          | 1.0    |
| 2.867         | 1.26  | 460  | 2.9258          | 1.0    |
| 2.9734        | 1.27  | 465  | 2.9281          | 1.0000 |
| 2.934         | 1.29  | 470  | 2.9229          | 1.0    |
| 2.9521        | 1.3   | 475  | 2.9134          | 1.0    |
| 2.9098        | 1.31  | 480  | 2.9051          | 0.9993 |
| 2.9112        | 1.33  | 485  | 2.9028          | 0.9999 |
| 2.8799        | 1.34  | 490  | 2.9101          | 0.9986 |
| 2.857         | 1.36  | 495  | 2.9005          | 0.9992 |
| 2.8525        | 1.37  | 500  | 2.8937          | 1.0    |
| 2.8682        | 1.38  | 505  | 2.8904          | 1.0000 |
| 2.8899        | 1.4   | 510  | 2.8914          | 0.9964 |
| 2.7475        | 1.41  | 515  | 2.8842          | 0.9950 |
| 2.9263        | 1.42  | 520  | 2.8852          | 0.9972 |
| 2.8603        | 1.44  | 525  | 2.8762          | 0.9966 |
| 2.864         | 1.45  | 530  | 2.8680          | 0.9978 |
| 2.8632        | 1.47  | 535  | 2.8602          | 0.9964 |
| 2.9289        | 1.48  | 540  | 2.8584          | 0.9952 |
| 2.8689        | 1.49  | 545  | 2.8587          | 0.9956 |
| 2.8304        | 1.51  | 550  | 2.8511          | 0.9993 |
| 2.8024        | 1.52  | 555  | 2.8460          | 1.0    |
| 2.7649        | 1.53  | 560  | 2.8460          | 1.0000 |
| 2.8756        | 1.55  | 565  | 2.8348          | 0.9987 |
| 2.8808        | 1.56  | 570  | 2.8539          | 0.9993 |
| 2.9027        | 1.57  | 575  | 2.8282          | 0.9975 |
| 2.8586        | 1.59  | 580  | 2.8288          | 0.9976 |
| 2.8193        | 1.6   | 585  | 2.8101          | 1.0051 |
| 2.811         | 1.62  | 590  | 2.7965          | 1.0014 |
| 2.7332        | 1.63  | 595  | 2.7884          | 1.0026 |
| 2.7717        | 1.64  | 600  | 2.7883          | 1.0060 |
| 2.6901        | 1.66  | 605  | 2.7801          | 0.9974 |
| 2.6905        | 1.67  | 610  | 2.8113          | 0.9968 |
| 2.7442        | 1.68  | 615  | 2.8113          | 1.0007 |
| 2.8431        | 1.7   | 620  | 2.8152          | 1.0343 |
| 2.8028        | 1.71  | 625  | 2.7790          | 1.0250 |
| 2.7151        | 1.73  | 630  | 2.7653          | 1.0287 |
| 2.7405        | 1.74  | 635  | 2.7714          | 1.1303 |
| 2.7566        | 1.75  | 640  | 2.7488          | 1.0312 |
| 2.7337        | 1.77  | 645  | 2.7498          | 1.0176 |
| 2.7486        | 1.78  | 650  | 2.7496          | 1.0760 |
| 2.6918        | 1.79  | 655  | 2.7391          | 1.0353 |
| 2.7142        | 1.81  | 660  | 2.7500          | 1.0283 |
| 2.7057        | 1.82  | 665  | 2.7612          | 1.0127 |
| 2.8348        | 1.83  | 670  | 2.7441          | 1.0056 |
| 2.705         | 1.85  | 675  | 2.7473          | 1.0519 |
| 2.7547        | 1.86  | 680  | 2.7216          | 1.0218 |
| 2.7045        | 1.88  | 685  | 2.7261          | 1.1414 |
| 2.7121        | 1.89  | 690  | 2.7223          | 1.0287 |
| 2.6877        | 1.9   | 695  | 2.7283          | 1.0274 |
| 2.6879        | 1.92  | 700  | 2.7451          | 1.1322 |
| 2.6958        | 1.93  | 705  | 2.7166          | 1.0364 |
| 2.6692        | 1.94  | 710  | 2.7148          | 1.0074 |
| 2.5786        | 1.96  | 715  | 2.7101          | 1.0504 |
| 2.6919        | 1.97  | 720  | 2.6963          | 1.0454 |
| 2.7256        | 1.98  | 725  | 2.7201          | 1.0349 |
| 2.6507        | 2.0   | 730  | 2.7099          | 1.1339 |
| 2.7833        | 2.01  | 735  | 2.7111          | 1.0124 |
| 2.7521        | 2.03  | 740  | 2.7024          | 1.0275 |
| 2.6732        | 2.04  | 745  | 2.7058          | 1.0647 |
| 2.719         | 2.05  | 750  | 2.7200          | 1.0211 |
| 2.701         | 2.07  | 755  | 2.7024          | 1.0808 |
| 2.6444        | 2.08  | 760  | 2.6813          | 1.0582 |
| 2.5592        | 2.1   | 765  | 2.6783          | 1.1010 |
| 2.6444        | 2.11  | 770  | 2.6707          | 1.0946 |
| 2.6944        | 2.12  | 775  | 2.7012          | 1.1315 |
| 2.6733        | 2.14  | 780  | 2.7072          | 1.1144 |
| 2.6998        | 2.15  | 785  | 2.7132          | 1.0206 |
| 2.796         | 2.16  | 790  | 2.7076          | 1.1262 |
| 2.6881        | 2.18  | 795  | 2.6953          | 1.0841 |
| 2.7382        | 2.19  | 800  | 2.6605          | 1.1234 |
| 2.5814        | 2.21  | 805  | 2.6814          | 1.1865 |
| 2.6695        | 2.22  | 810  | 2.6531          | 1.0985 |
| 2.6415        | 2.23  | 815  | 2.6590          | 1.0804 |
| 2.646         | 2.25  | 820  | 2.6514          | 1.0853 |
| 2.6028        | 2.26  | 825  | 2.6723          | 1.1411 |
| 2.6429        | 2.27  | 830  | 2.6729          | 1.0395 |
| 2.6736        | 2.29  | 835  | 2.7039          | 1.0355 |
| 2.6959        | 2.3   | 840  | 2.6510          | 1.0414 |
| 2.6426        | 2.31  | 845  | 2.6660          | 1.1591 |
| 2.7152        | 2.33  | 850  | 2.6361          | 1.0276 |
| 2.7148        | 2.34  | 855  | 2.6723          | 1.2461 |
| 2.6336        | 2.36  | 860  | 2.6332          | 1.0310 |
| 2.665         | 2.37  | 865  | 2.6365          | 1.1312 |
| 2.5607        | 2.38  | 870  | 2.6344          | 1.1301 |
| 2.5614        | 2.4   | 875  | 2.6437          | 1.1513 |
| 2.4899        | 2.41  | 880  | 2.6418          | 1.1532 |
| 2.6794        | 2.42  | 885  | 2.6403          | 1.0272 |
| 2.6814        | 2.44  | 890  | 2.6420          | 1.1323 |
| 2.6614        | 2.45  | 895  | 2.6183          | 1.0525 |
| 2.6629        | 2.47  | 900  | 2.6414          | 1.1569 |
| 2.6166        | 2.48  | 905  | 2.6167          | 1.0265 |
| 2.6374        | 2.49  | 910  | 2.6299          | 1.1720 |
| 2.6035        | 2.51  | 915  | 2.6139          | 1.1565 |
| 2.595         | 2.52  | 920  | 2.6126          | 1.0557 |
| 2.6416        | 2.53  | 925  | 2.6190          | 1.0414 |
| 2.6785        | 2.55  | 930  | 2.6352          | 1.0289 |
| 2.6986        | 2.56  | 935  | 2.6268          | 1.0077 |
| 2.6145        | 2.57  | 940  | 2.6166          | 1.0445 |
| 2.6961        | 2.59  | 945  | 2.6142          | 1.0185 |
| 2.6852        | 2.6   | 950  | 2.6072          | 1.0122 |
| 2.5792        | 2.62  | 955  | 2.6078          | 1.1165 |
| 2.6118        | 2.63  | 960  | 2.6177          | 1.1210 |
| 2.5472        | 2.64  | 965  | 2.6126          | 1.0044 |
| 2.577         | 2.66  | 970  | 2.6051          | 1.0881 |
| 2.5602        | 2.67  | 975  | 2.5992          | 1.0178 |
| 2.695         | 2.68  | 980  | 2.6023          | 1.0248 |
| 2.7017        | 2.7   | 985  | 2.6190          | 1.0041 |
| 2.6327        | 2.71  | 990  | 2.6024          | 1.0142 |
| 2.6193        | 2.73  | 995  | 2.5897          | 1.0148 |
| 2.5939        | 2.74  | 1000 | 2.5900          | 1.0329 |
| 2.5477        | 2.75  | 1005 | 2.5971          | 1.0338 |
| 2.6089        | 2.77  | 1010 | 2.5969          | 1.0064 |
| 2.5625        | 2.78  | 1015 | 2.5899          | 1.0648 |
| 2.5745        | 2.79  | 1020 | 2.5861          | 1.0627 |
| 2.5702        | 2.81  | 1025 | 2.5923          | 1.0526 |
| 2.645         | 2.82  | 1030 | 2.6053          | 1.0199 |
| 2.6869        | 2.83  | 1035 | 2.6227          | 1.0011 |
| 2.6678        | 2.85  | 1040 | 2.6094          | 1.0179 |
| 2.6787        | 2.86  | 1045 | 2.5978          | 1.0028 |
| 2.6246        | 2.88  | 1050 | 2.5965          | 1.0093 |
| 2.5676        | 2.89  | 1055 | 2.5927          | 1.0627 |
| 2.6773        | 2.9   | 1060 | 2.5907          | 1.0817 |
| 2.6114        | 2.92  | 1065 | 2.5932          | 1.1013 |
| 2.6227        | 2.93  | 1070 | 2.5840          | 1.0402 |
| 2.594         | 2.94  | 1075 | 2.5997          | 1.1371 |
| 2.751         | 2.96  | 1080 | 2.5909          | 1.0972 |
| 2.6366        | 2.97  | 1085 | 2.6081          | 1.0598 |
| 2.577         | 2.98  | 1090 | 2.5915          | 1.0410 |
| 2.579         | 3.0   | 1095 | 2.5953          | 1.1433 |
| 2.6706        | 3.01  | 1100 | 2.5913          | 1.0456 |
| 2.6161        | 3.03  | 1105 | 2.6079          | 1.1009 |
| 2.6397        | 3.04  | 1110 | 2.5951          | 1.1771 |
| 2.6246        | 3.05  | 1115 | 2.5730          | 1.0299 |
| 2.5637        | 3.07  | 1120 | 2.5622          | 1.0848 |
| 2.5692        | 3.08  | 1125 | 2.5561          | 1.1472 |
| 2.5948        | 3.1   | 1130 | 2.5568          | 1.0802 |
| 2.5372        | 3.11  | 1135 | 2.5638          | 1.1261 |
| 2.4995        | 3.12  | 1140 | 2.5727          | 1.1395 |
| 2.6304        | 3.14  | 1145 | 2.5671          | 1.0259 |
| 2.6395        | 3.15  | 1150 | 2.5778          | 1.0212 |
| 2.6127        | 3.16  | 1155 | 2.5609          | 1.0457 |
| 2.5919        | 3.18  | 1160 | 2.5604          | 1.0902 |
| 2.6111        | 3.19  | 1165 | 2.5463          | 1.0014 |
| 2.5971        | 3.21  | 1170 | 2.5429          | 1.0022 |
| 2.5887        | 3.22  | 1175 | 2.5394          | 1.0412 |
| 2.5644        | 3.23  | 1180 | 2.5342          | 1.0469 |
| 2.4805        | 3.25  | 1185 | 2.6066          | 1.2668 |
| 2.5324        | 3.26  | 1190 | 2.5395          | 1.0234 |
| 2.5491        | 3.27  | 1195 | 2.5431          | 1.0644 |
| 2.6302        | 3.29  | 1200 | 2.5558          | 1.0680 |
| 2.6139        | 3.3   | 1205 | 2.5711          | 1.0565 |
| 2.5607        | 3.31  | 1210 | 2.5635          | 1.0415 |
| 2.6535        | 3.33  | 1215 | 2.5505          | 1.0613 |
| 2.6129        | 3.34  | 1220 | 2.5403          | 1.0724 |
| 2.5157        | 3.36  | 1225 | 2.5294          | 1.0585 |
| 2.551         | 3.37  | 1230 | 2.5242          | 1.1599 |
| 2.5527        | 3.38  | 1235 | 2.5474          | 1.2327 |
| 2.4964        | 3.4   | 1240 | 2.5244          | 1.0857 |
| 2.5781        | 3.41  | 1245 | 2.5299          | 1.0470 |
| 2.6143        | 3.42  | 1250 | 2.5313          | 1.0019 |
| 2.6566        | 3.44  | 1255 | 2.5431          | 1.0488 |
| 2.5373        | 3.45  | 1260 | 2.5281          | 1.0901 |
| 2.6597        | 3.47  | 1265 | 2.5300          | 1.0610 |
| 2.5457        | 3.48  | 1270 | 2.5130          | 1.0420 |
| 2.5632        | 3.49  | 1275 | 2.5306          | 1.1418 |
| 2.5267        | 3.51  | 1280 | 2.5021          | 1.0293 |
| 2.507         | 3.52  | 1285 | 2.5013          | 1.0196 |
| 2.5713        | 3.53  | 1290 | 2.4978          | 1.0664 |
| 2.4783        | 3.55  | 1295 | 2.4958          | 1.0530 |
| 2.5874        | 3.56  | 1300 | 2.4968          | 1.0059 |
| 2.5744        | 3.57  | 1305 | 2.5078          | 1.0287 |
| 2.5701        | 3.59  | 1310 | 2.4971          | 1.0366 |
| 2.5366        | 3.6   | 1315 | 2.4897          | 1.0191 |
| 2.5679        | 3.62  | 1320 | 2.4830          | 1.0223 |
| 2.5239        | 3.63  | 1325 | 2.4833          | 1.0784 |
| 2.5411        | 3.64  | 1330 | 2.4851          | 1.1522 |
| 2.5037        | 3.66  | 1335 | 2.4792          | 1.0928 |
| 2.5907        | 3.67  | 1340 | 2.4750          | 1.0187 |
| 2.5107        | 3.68  | 1345 | 2.4805          | 1.0873 |
| 2.5908        | 3.7   | 1350 | 2.4753          | 1.0098 |
| 2.6274        | 3.71  | 1355 | 2.4765          | 1.0045 |
| 2.5708        | 3.73  | 1360 | 2.4597          | 1.0456 |
| 2.6039        | 3.74  | 1365 | 2.4503          | 1.0485 |
| 2.5305        | 3.75  | 1370 | 2.4439          | 1.0126 |
| 2.4878        | 3.77  | 1375 | 2.4407          | 1.0162 |
| 2.5055        | 3.78  | 1380 | 2.4421          | 1.0605 |
| 2.5249        | 3.79  | 1385 | 2.4499          | 1.1163 |
| 2.5508        | 3.81  | 1390 | 2.4654          | 1.1472 |
| 2.5827        | 3.82  | 1395 | 2.4510          | 1.0561 |
| 2.6148        | 3.83  | 1400 | 2.4496          | 0.9998 |
| 2.5763        | 3.85  | 1405 | 2.4417          | 1.0067 |
| 2.6077        | 3.86  | 1410 | 2.4458          | 1.0682 |
| 2.5388        | 3.88  | 1415 | 2.4352          | 1.0820 |
| 2.5235        | 3.89  | 1420 | 2.4277          | 1.0784 |
| 2.4996        | 3.9   | 1425 | 2.4245          | 1.0671 |
| 2.5601        | 3.92  | 1430 | 2.4202          | 1.0650 |
| 2.5805        | 3.93  | 1435 | 2.4199          | 1.0530 |
| 2.5841        | 3.94  | 1440 | 2.4228          | 1.0797 |
| 2.4877        | 3.96  | 1445 | 2.4284          | 1.1159 |
| 2.5542        | 3.97  | 1450 | 2.4190          | 1.0575 |
| 2.5961        | 3.98  | 1455 | 2.4162          | 1.0676 |
| 2.495         | 4.0   | 1460 | 2.4165          | 1.0821 |
| 2.6157        | 4.01  | 1465 | 2.4119          | 1.0117 |
| 2.5415        | 4.03  | 1470 | 2.4089          | 1.0110 |
| 2.4916        | 4.04  | 1475 | 2.4032          | 1.0498 |
| 2.5445        | 4.05  | 1480 | 2.3997          | 1.0429 |
| 2.4941        | 4.07  | 1485 | 2.4008          | 1.0141 |
| 2.5113        | 4.08  | 1490 | 2.3975          | 1.0357 |
| 2.4707        | 4.1   | 1495 | 2.3938          | 1.0288 |
| 2.4952        | 4.11  | 1500 | 2.3910          | 1.0300 |
| 2.5017        | 4.12  | 1505 | 2.3861          | 1.0813 |
| 2.5566        | 4.14  | 1510 | 2.3919          | 1.1082 |
| 2.5754        | 4.15  | 1515 | 2.3947          | 1.0074 |
| 2.6138        | 4.16  | 1520 | 2.4040          | 0.9989 |
| 2.5024        | 4.18  | 1525 | 2.3949          | 1.0039 |
| 2.5136        | 4.19  | 1530 | 2.3993          | 1.0496 |
| 2.5646        | 4.21  | 1535 | 2.3981          | 1.0729 |
| 2.4556        | 4.22  | 1540 | 2.3952          | 1.0494 |
| 2.5774        | 4.23  | 1545 | 2.3924          | 1.0345 |
| 2.5126        | 4.25  | 1550 | 2.3888          | 1.0306 |
| 2.4596        | 4.26  | 1555 | 2.3960          | 1.0775 |
| 2.521         | 4.27  | 1560 | 2.3978          | 1.1025 |
| 2.6304        | 4.29  | 1565 | 2.3885          | 1.0433 |
| 2.543         | 4.3   | 1570 | 2.3849          | 1.0072 |
| 2.5601        | 4.31  | 1575 | 2.3855          | 1.0110 |
| 2.6304        | 4.33  | 1580 | 2.3878          | 1.0369 |
| 2.4121        | 4.34  | 1585 | 2.3783          | 1.0366 |
| 2.4261        | 4.36  | 1590 | 2.3746          | 1.0307 |
| 2.5038        | 4.37  | 1595 | 2.3789          | 1.0611 |
| 2.5391        | 4.38  | 1600 | 2.3849          | 1.0738 |
| 2.4341        | 4.4   | 1605 | 2.3779          | 1.0573 |
| 2.5306        | 4.41  | 1610 | 2.3751          | 1.0460 |
| 2.5818        | 4.42  | 1615 | 2.3743          | 1.0251 |
| 2.5531        | 4.44  | 1620 | 2.3723          | 1.0209 |
| 2.51          | 4.45  | 1625 | 2.3755          | 1.0316 |
| 2.5788        | 4.47  | 1630 | 2.3725          | 1.0396 |
| 2.5701        | 4.48  | 1635 | 2.3663          | 1.0292 |
| 2.4194        | 4.49  | 1640 | 2.3641          | 1.0261 |
| 2.5439        | 4.51  | 1645 | 2.3629          | 1.0376 |
| 2.4527        | 4.52  | 1650 | 2.3629          | 1.0563 |
| 2.5705        | 4.53  | 1655 | 2.3654          | 1.0766 |
| 2.4552        | 4.55  | 1660 | 2.3708          | 1.0802 |
| 2.5657        | 4.56  | 1665 | 2.3638          | 1.0248 |
| 2.5371        | 4.57  | 1670 | 2.3639          | 1.0053 |
| 2.5365        | 4.59  | 1675 | 2.3626          | 1.0072 |
| 2.5383        | 4.6   | 1680 | 2.3584          | 1.0170 |
| 2.546         | 4.62  | 1685 | 2.3574          | 1.0469 |
| 2.6006        | 4.63  | 1690 | 2.3517          | 1.0509 |
| 2.4894        | 4.64  | 1695 | 2.3489          | 1.0452 |
| 2.4732        | 4.66  | 1700 | 2.3489          | 1.0586 |
| 2.4933        | 4.67  | 1705 | 2.3501          | 1.0694 |
| 2.4784        | 4.68  | 1710 | 2.3472          | 1.0647 |
| 2.5349        | 4.7   | 1715 | 2.3419          | 1.0299 |
| 2.553         | 4.71  | 1720 | 2.3420          | 1.0115 |
| 2.5035        | 4.73  | 1725 | 2.3415          | 1.0117 |
| 2.561         | 4.74  | 1730 | 2.3418          | 1.0242 |
| 2.4773        | 4.75  | 1735 | 2.3420          | 1.0325 |
| 2.4691        | 4.77  | 1740 | 2.3422          | 1.0394 |
| 2.4959        | 4.78  | 1745 | 2.3405          | 1.0418 |
| 2.4928        | 4.79  | 1750 | 2.3394          | 1.0449 |
| 2.5058        | 4.81  | 1755 | 2.3392          | 1.0489 |
| 2.5193        | 4.82  | 1760 | 2.3390          | 1.0506 |
| 2.5369        | 4.83  | 1765 | 2.3392          | 1.0384 |
| 2.4843        | 4.85  | 1770 | 2.3398          | 1.0236 |
| 2.5074        | 4.86  | 1775 | 2.3400          | 1.0150 |
| 2.4941        | 4.88  | 1780 | 2.3386          | 1.0150 |
| 2.4352        | 4.89  | 1785 | 2.3370          | 1.0172 |
| 2.4372        | 4.9   | 1790 | 2.3362          | 1.0208 |
| 2.4855        | 4.92  | 1795 | 2.3358          | 1.0238 |
| 2.4516        | 4.93  | 1800 | 2.3355          | 1.0276 |
| 2.5281        | 4.94  | 1805 | 2.3356          | 1.0312 |
| 2.5519        | 4.96  | 1810 | 2.3352          | 1.0318 |
| 2.4641        | 4.97  | 1815 | 2.3349          | 1.0294 |
| 2.4515        | 4.98  | 1820 | 2.3348          | 1.0284 |
| 2.553         | 5.0   | 1825 | 2.3347          | 1.0286 |


### Framework versions

- Transformers 4.17.0.dev0
- Pytorch 1.10.2+cu102
- Datasets 1.18.2.dev0
- Tokenizers 0.11.0