File size: 19,847 Bytes
a811c65
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
f42ec53
 
 
 
 
a811c65
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
f42ec53
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
a811c65
 
 
 
f42ec53
a811c65
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
---
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: beto-finetuned-token-reqadjzar
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# beto-finetuned-token-reqadjzar

This model is a fine-tuned version of [dccuchile/bert-base-spanish-wwm-cased](https://huggingface.co/dccuchile/bert-base-spanish-wwm-cased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1061
- Precision: 0.2533
- Recall: 0.3333
- F1: 0.2879
- Accuracy: 0.8498

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 200

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.7331        | 1.0   | 24   | 0.5920          | 0.0       | 0.0    | 0.0    | 0.7532   |
| 0.4759        | 2.0   | 48   | 0.3954          | 0.0085    | 0.0175 | 0.0115 | 0.8321   |
| 0.3186        | 3.0   | 72   | 0.5127          | 0.0188    | 0.0702 | 0.0296 | 0.8159   |
| 0.1906        | 4.0   | 96   | 0.4865          | 0.1190    | 0.2632 | 0.1639 | 0.8509   |
| 0.145         | 5.0   | 120  | 0.4650          | 0.1597    | 0.3333 | 0.2159 | 0.8760   |
| 0.1107        | 6.0   | 144  | 0.5465          | 0.1062    | 0.2105 | 0.1412 | 0.8514   |
| 0.0903        | 7.0   | 168  | 0.5441          | 0.1359    | 0.2456 | 0.175  | 0.8796   |
| 0.0698        | 8.0   | 192  | 0.4353          | 0.1204    | 0.2281 | 0.1576 | 0.8842   |
| 0.0505        | 9.0   | 216  | 0.7170          | 0.19      | 0.3333 | 0.2420 | 0.8432   |
| 0.0687        | 10.0  | 240  | 0.5893          | 0.1963    | 0.3684 | 0.2561 | 0.8860   |
| 0.039         | 11.0  | 264  | 0.5877          | 0.1951    | 0.4211 | 0.2667 | 0.8780   |
| 0.0278        | 12.0  | 288  | 0.5715          | 0.2237    | 0.2982 | 0.2556 | 0.8577   |
| 0.0354        | 13.0  | 312  | 0.9535          | 0.2283    | 0.3684 | 0.2819 | 0.8532   |
| 0.024         | 14.0  | 336  | 0.6500          | 0.2169    | 0.3158 | 0.2571 | 0.8674   |
| 0.0223        | 15.0  | 360  | 0.7513          | 0.1855    | 0.4035 | 0.2541 | 0.8722   |
| 0.0156        | 16.0  | 384  | 0.6566          | 0.3       | 0.4737 | 0.3673 | 0.9012   |
| 0.0156        | 17.0  | 408  | 0.8436          | 0.2292    | 0.3860 | 0.2876 | 0.8696   |
| 0.0189        | 18.0  | 432  | 0.8043          | 0.1711    | 0.2281 | 0.1955 | 0.8181   |
| 0.0128        | 19.0  | 456  | 0.6518          | 0.1619    | 0.2982 | 0.2099 | 0.8814   |
| 0.0122        | 20.0  | 480  | 0.8418          | 0.2347    | 0.4035 | 0.2968 | 0.8793   |
| 0.0242        | 21.0  | 504  | 0.7948          | 0.2292    | 0.3860 | 0.2876 | 0.8814   |
| 0.0124        | 22.0  | 528  | 0.8059          | 0.2037    | 0.3860 | 0.2667 | 0.8842   |
| 0.0098        | 23.0  | 552  | 0.9458          | 0.1765    | 0.2632 | 0.2113 | 0.8584   |
| 0.0287        | 24.0  | 576  | 0.7110          | 0.1488    | 0.3158 | 0.2022 | 0.8825   |
| 0.0253        | 25.0  | 600  | 0.6823          | 0.2021    | 0.3333 | 0.2517 | 0.8781   |
| 0.0151        | 26.0  | 624  | 0.7382          | 0.2022    | 0.3158 | 0.2466 | 0.8791   |
| 0.0118        | 27.0  | 648  | 0.6036          | 0.2360    | 0.3684 | 0.2877 | 0.8965   |
| 0.0102        | 28.0  | 672  | 0.9152          | 0.1765    | 0.3158 | 0.2264 | 0.8446   |
| 0.0229        | 29.0  | 696  | 0.6878          | 0.2584    | 0.4035 | 0.3151 | 0.8982   |
| 0.0168        | 30.0  | 720  | 0.7333          | 0.2784    | 0.4737 | 0.3506 | 0.8937   |
| 0.0145        | 31.0  | 744  | 0.6051          | 0.1864    | 0.3860 | 0.2514 | 0.9      |
| 0.0207        | 32.0  | 768  | 0.9083          | 0.3279    | 0.3509 | 0.3390 | 0.8894   |
| 0.0191        | 33.0  | 792  | 0.6983          | 0.2222    | 0.3509 | 0.2721 | 0.8884   |
| 0.0103        | 34.0  | 816  | 0.7287          | 0.2449    | 0.4211 | 0.3097 | 0.8840   |
| 0.0091        | 35.0  | 840  | 0.5929          | 0.2184    | 0.3333 | 0.2639 | 0.8851   |
| 0.0059        | 36.0  | 864  | 0.7604          | 0.2421    | 0.4035 | 0.3026 | 0.8810   |
| 0.0035        | 37.0  | 888  | 0.9380          | 0.2143    | 0.3684 | 0.2710 | 0.8622   |
| 0.0025        | 38.0  | 912  | 0.9824          | 0.2       | 0.3509 | 0.2548 | 0.8704   |
| 0.0059        | 39.0  | 936  | 1.0658          | 0.2796    | 0.4561 | 0.3467 | 0.8669   |
| 0.0199        | 40.0  | 960  | 0.9755          | 0.1705    | 0.3860 | 0.2366 | 0.8449   |
| 0.0034        | 41.0  | 984  | 0.9697          | 0.2619    | 0.3860 | 0.3121 | 0.8656   |
| 0.0035        | 42.0  | 1008 | 1.0582          | 0.1959    | 0.3333 | 0.2468 | 0.8461   |
| 0.0088        | 43.0  | 1032 | 0.8500          | 0.1849    | 0.3860 | 0.25   | 0.8515   |
| 0.0263        | 44.0  | 1056 | 1.2832          | 0.2       | 0.3509 | 0.2548 | 0.8255   |
| 0.0088        | 45.0  | 1080 | 0.9282          | 0.2308    | 0.4211 | 0.2981 | 0.8534   |
| 0.0343        | 46.0  | 1104 | 0.7165          | 0.2222    | 0.3158 | 0.2609 | 0.8594   |
| 0.0024        | 47.0  | 1128 | 0.7355          | 0.2308    | 0.4737 | 0.3103 | 0.8782   |
| 0.0019        | 48.0  | 1152 | 0.6493          | 0.2165    | 0.3684 | 0.2727 | 0.8779   |
| 0.0009        | 49.0  | 1176 | 0.6999          | 0.1964    | 0.3860 | 0.2604 | 0.8766   |
| 0.0008        | 50.0  | 1200 | 0.7496          | 0.2062    | 0.3509 | 0.2597 | 0.8709   |
| 0.0009        | 51.0  | 1224 | 0.7670          | 0.2019    | 0.3684 | 0.2609 | 0.8750   |
| 0.0006        | 52.0  | 1248 | 0.7549          | 0.24      | 0.4211 | 0.3057 | 0.8832   |
| 0.0007        | 53.0  | 1272 | 0.7556          | 0.2706    | 0.4035 | 0.3239 | 0.8870   |
| 0.0007        | 54.0  | 1296 | 0.7188          | 0.1695    | 0.3509 | 0.2286 | 0.8833   |
| 0.0005        | 55.0  | 1320 | 0.7120          | 0.1927    | 0.3684 | 0.2530 | 0.8822   |
| 0.0009        | 56.0  | 1344 | 0.7377          | 0.2245    | 0.3860 | 0.2839 | 0.8819   |
| 0.0008        | 57.0  | 1368 | 0.7295          | 0.2277    | 0.4035 | 0.2911 | 0.8859   |
| 0.0009        | 58.0  | 1392 | 0.7158          | 0.2340    | 0.3860 | 0.2914 | 0.8900   |
| 0.0013        | 59.0  | 1416 | 0.6715          | 0.1897    | 0.3860 | 0.2543 | 0.8941   |
| 0.0006        | 60.0  | 1440 | 0.6787          | 0.21      | 0.3684 | 0.2675 | 0.8861   |
| 0.0007        | 61.0  | 1464 | 0.6794          | 0.2584    | 0.4035 | 0.3151 | 0.8940   |
| 0.0012        | 62.0  | 1488 | 0.6823          | 0.2273    | 0.3509 | 0.2759 | 0.8778   |
| 0.0008        | 63.0  | 1512 | 0.7189          | 0.2588    | 0.3860 | 0.3099 | 0.8791   |
| 0.0008        | 64.0  | 1536 | 0.7077          | 0.2371    | 0.4035 | 0.2987 | 0.8905   |
| 0.0007        | 65.0  | 1560 | 0.7201          | 0.2738    | 0.4035 | 0.3262 | 0.8860   |
| 0.0005        | 66.0  | 1584 | 0.7339          | 0.2584    | 0.4035 | 0.3151 | 0.8894   |
| 0.0005        | 67.0  | 1608 | 0.7490          | 0.2157    | 0.3860 | 0.2767 | 0.8845   |
| 0.0006        | 68.0  | 1632 | 0.7342          | 0.2162    | 0.4211 | 0.2857 | 0.8833   |
| 0.0012        | 69.0  | 1656 | 0.7287          | 0.3108    | 0.4035 | 0.3511 | 0.8895   |
| 0.0012        | 70.0  | 1680 | 0.8877          | 0.2079    | 0.3684 | 0.2658 | 0.8615   |
| 0.0007        | 71.0  | 1704 | 0.9370          | 0.2095    | 0.3860 | 0.2716 | 0.8644   |
| 0.002         | 72.0  | 1728 | 0.7715          | 0.2391    | 0.3860 | 0.2953 | 0.8677   |
| 0.0007        | 73.0  | 1752 | 0.8765          | 0.22      | 0.3860 | 0.2803 | 0.8628   |
| 0.0006        | 74.0  | 1776 | 0.8515          | 0.2371    | 0.4035 | 0.2987 | 0.8639   |
| 0.0007        | 75.0  | 1800 | 0.8448          | 0.2286    | 0.4211 | 0.2963 | 0.8633   |
| 0.0009        | 76.0  | 1824 | 0.8501          | 0.2232    | 0.4386 | 0.2959 | 0.8650   |
| 0.0007        | 77.0  | 1848 | 0.8550          | 0.2198    | 0.3509 | 0.2703 | 0.8657   |
| 0.0005        | 78.0  | 1872 | 0.7445          | 0.25      | 0.4035 | 0.3087 | 0.8780   |
| 0.0007        | 79.0  | 1896 | 0.8889          | 0.26      | 0.4561 | 0.3312 | 0.8630   |
| 0.0005        | 80.0  | 1920 | 0.8930          | 0.2812    | 0.4737 | 0.3529 | 0.8650   |
| 0.0004        | 81.0  | 1944 | 0.8678          | 0.26      | 0.4561 | 0.3312 | 0.8745   |
| 0.0005        | 82.0  | 1968 | 0.8747          | 0.2784    | 0.4737 | 0.3506 | 0.8746   |
| 0.0005        | 83.0  | 1992 | 0.8726          | 0.2872    | 0.4737 | 0.3576 | 0.8687   |
| 0.001         | 84.0  | 2016 | 0.8887          | 0.2857    | 0.4211 | 0.3404 | 0.8693   |
| 0.0006        | 85.0  | 2040 | 0.7915          | 0.2963    | 0.4211 | 0.3478 | 0.8821   |
| 0.0007        | 86.0  | 2064 | 1.0194          | 0.2857    | 0.4211 | 0.3404 | 0.8606   |
| 0.0009        | 87.0  | 2088 | 0.7594          | 0.2366    | 0.3860 | 0.2933 | 0.8777   |
| 0.0021        | 88.0  | 2112 | 0.9788          | 0.25      | 0.3333 | 0.2857 | 0.8539   |
| 0.0012        | 89.0  | 2136 | 0.8719          | 0.2093    | 0.3158 | 0.2517 | 0.8697   |
| 0.0019        | 90.0  | 2160 | 1.1859          | 0.1810    | 0.3684 | 0.2428 | 0.8111   |
| 0.001         | 91.0  | 2184 | 0.9690          | 0.2118    | 0.3158 | 0.2535 | 0.8421   |
| 0.0007        | 92.0  | 2208 | 0.9863          | 0.1880    | 0.3860 | 0.2529 | 0.8495   |
| 0.0006        | 93.0  | 2232 | 0.9942          | 0.1868    | 0.2982 | 0.2297 | 0.8641   |
| 0.0007        | 94.0  | 2256 | 1.0118          | 0.2159    | 0.3333 | 0.2621 | 0.8637   |
| 0.0007        | 95.0  | 2280 | 1.0435          | 0.2754    | 0.3333 | 0.3016 | 0.8615   |
| 0.0008        | 96.0  | 2304 | 0.9795          | 0.2471    | 0.3684 | 0.2958 | 0.8657   |
| 0.0007        | 97.0  | 2328 | 0.9189          | 0.2020    | 0.3509 | 0.2564 | 0.8807   |
| 0.0009        | 98.0  | 2352 | 0.9240          | 0.2273    | 0.3509 | 0.2759 | 0.8762   |
| 0.0005        | 99.0  | 2376 | 0.8891          | 0.2561    | 0.3684 | 0.3022 | 0.8821   |
| 0.0004        | 100.0 | 2400 | 0.9028          | 0.2469    | 0.3509 | 0.2899 | 0.8818   |
| 0.0004        | 101.0 | 2424 | 0.9228          | 0.2410    | 0.3509 | 0.2857 | 0.8830   |
| 0.0004        | 102.0 | 2448 | 0.9409          | 0.2278    | 0.3158 | 0.2647 | 0.8795   |
| 0.0006        | 103.0 | 2472 | 0.9777          | 0.24      | 0.3158 | 0.2727 | 0.8796   |
| 0.0005        | 104.0 | 2496 | 0.9872          | 0.2432    | 0.3158 | 0.2748 | 0.8791   |
| 0.0006        | 105.0 | 2520 | 0.9820          | 0.2329    | 0.2982 | 0.2615 | 0.8746   |
| 0.0006        | 106.0 | 2544 | 1.0301          | 0.2879    | 0.3333 | 0.3089 | 0.8702   |
| 0.0006        | 107.0 | 2568 | 1.0468          | 0.3226    | 0.3509 | 0.3361 | 0.8637   |
| 0.0004        | 108.0 | 2592 | 1.0155          | 0.2941    | 0.3509 | 0.3200 | 0.8683   |
| 0.0005        | 109.0 | 2616 | 0.9970          | 0.2821    | 0.3860 | 0.3259 | 0.8678   |
| 0.0004        | 110.0 | 2640 | 1.0453          | 0.28      | 0.3684 | 0.3182 | 0.8687   |
| 0.0009        | 111.0 | 2664 | 0.9247          | 0.2278    | 0.3158 | 0.2647 | 0.8747   |
| 0.0006        | 112.0 | 2688 | 0.8811          | 0.2785    | 0.3860 | 0.3235 | 0.8921   |
| 0.0005        | 113.0 | 2712 | 0.9462          | 0.1905    | 0.2807 | 0.2270 | 0.8817   |
| 0.0005        | 114.0 | 2736 | 0.9685          | 0.2078    | 0.2807 | 0.2388 | 0.8792   |
| 0.0006        | 115.0 | 2760 | 1.0339          | 0.2712    | 0.2807 | 0.2759 | 0.8672   |
| 0.0004        | 116.0 | 2784 | 1.0155          | 0.2571    | 0.3158 | 0.2835 | 0.8687   |
| 0.0005        | 117.0 | 2808 | 0.9998          | 0.25      | 0.3509 | 0.2920 | 0.8768   |
| 0.0006        | 118.0 | 2832 | 0.9849          | 0.2473    | 0.4035 | 0.3067 | 0.8715   |
| 0.0033        | 119.0 | 2856 | 0.7929          | 0.2376    | 0.4211 | 0.3038 | 0.8832   |
| 0.0485        | 120.0 | 2880 | 0.9585          | 0.2       | 0.2807 | 0.2336 | 0.8585   |
| 0.0114        | 121.0 | 2904 | 0.7619          | 0.2472    | 0.3860 | 0.3014 | 0.8831   |
| 0.0177        | 122.0 | 2928 | 0.7737          | 0.2881    | 0.2982 | 0.2931 | 0.8688   |
| 0.02          | 123.0 | 2952 | 1.1362          | 0.1959    | 0.3333 | 0.2468 | 0.8214   |
| 0.0056        | 124.0 | 2976 | 1.2073          | 0.3659    | 0.2632 | 0.3061 | 0.8277   |
| 0.0208        | 125.0 | 3000 | 0.8549          | 0.2162    | 0.2807 | 0.2443 | 0.8430   |
| 0.0066        | 126.0 | 3024 | 0.9482          | 0.2667    | 0.2807 | 0.2735 | 0.8383   |
| 0.0155        | 127.0 | 3048 | 0.7532          | 0.2289    | 0.3333 | 0.2714 | 0.8629   |
| 0.0091        | 128.0 | 3072 | 0.7973          | 0.2368    | 0.3158 | 0.2707 | 0.8524   |
| 0.0029        | 129.0 | 3096 | 0.8988          | 0.25      | 0.3684 | 0.2979 | 0.8621   |
| 0.0054        | 130.0 | 3120 | 0.9882          | 0.2299    | 0.3509 | 0.2778 | 0.8362   |
| 0.0037        | 131.0 | 3144 | 1.0792          | 0.2093    | 0.3158 | 0.2517 | 0.8468   |
| 0.0012        | 132.0 | 3168 | 0.9729          | 0.2632    | 0.3509 | 0.3008 | 0.8427   |
| 0.0009        | 133.0 | 3192 | 0.9521          | 0.2043    | 0.3333 | 0.2533 | 0.8416   |
| 0.0011        | 134.0 | 3216 | 0.9539          | 0.1978    | 0.3158 | 0.2432 | 0.8401   |
| 0.0006        | 135.0 | 3240 | 0.9692          | 0.2754    | 0.3333 | 0.3016 | 0.8504   |
| 0.0007        | 136.0 | 3264 | 0.9811          | 0.2603    | 0.3333 | 0.2923 | 0.8526   |
| 0.0007        | 137.0 | 3288 | 0.9732          | 0.25      | 0.3333 | 0.2857 | 0.8444   |
| 0.0004        | 138.0 | 3312 | 0.9955          | 0.2278    | 0.3158 | 0.2647 | 0.8373   |
| 0.0005        | 139.0 | 3336 | 0.9939          | 0.2466    | 0.3158 | 0.2769 | 0.8389   |
| 0.001         | 140.0 | 3360 | 1.0081          | 0.2432    | 0.3158 | 0.2748 | 0.8377   |
| 0.0006        | 141.0 | 3384 | 1.0216          | 0.2308    | 0.3158 | 0.2667 | 0.8404   |
| 0.0005        | 142.0 | 3408 | 1.0364          | 0.25      | 0.3158 | 0.2791 | 0.8332   |
| 0.0004        | 143.0 | 3432 | 1.0185          | 0.2571    | 0.3158 | 0.2835 | 0.8426   |
| 0.0006        | 144.0 | 3456 | 1.0168          | 0.2603    | 0.3333 | 0.2923 | 0.8458   |
| 0.0005        | 145.0 | 3480 | 1.0079          | 0.2754    | 0.3333 | 0.3016 | 0.8476   |
| 0.0006        | 146.0 | 3504 | 1.0080          | 0.25      | 0.3333 | 0.2857 | 0.8438   |
| 0.0004        | 147.0 | 3528 | 1.0194          | 0.2346    | 0.3333 | 0.2754 | 0.8396   |
| 0.0004        | 148.0 | 3552 | 1.0299          | 0.2262    | 0.3333 | 0.2695 | 0.8373   |
| 0.0005        | 149.0 | 3576 | 1.0331          | 0.2289    | 0.3333 | 0.2714 | 0.8387   |
| 0.0004        | 150.0 | 3600 | 1.0294          | 0.2436    | 0.3333 | 0.2815 | 0.8412   |
| 0.0004        | 151.0 | 3624 | 1.0366          | 0.2405    | 0.3333 | 0.2794 | 0.8410   |
| 0.0004        | 152.0 | 3648 | 1.0533          | 0.2468    | 0.3333 | 0.2836 | 0.8448   |
| 0.0005        | 153.0 | 3672 | 1.0379          | 0.2879    | 0.3333 | 0.3089 | 0.8458   |
| 0.0005        | 154.0 | 3696 | 1.0395          | 0.2836    | 0.3333 | 0.3065 | 0.8454   |
| 0.0004        | 155.0 | 3720 | 1.0438          | 0.2836    | 0.3333 | 0.3065 | 0.8453   |
| 0.0004        | 156.0 | 3744 | 1.0475          | 0.2879    | 0.3333 | 0.3089 | 0.8453   |
| 0.0004        | 157.0 | 3768 | 1.0558          | 0.2794    | 0.3333 | 0.304  | 0.8450   |
| 0.0004        | 158.0 | 3792 | 1.0596          | 0.2754    | 0.3333 | 0.3016 | 0.8444   |
| 0.0004        | 159.0 | 3816 | 1.0633          | 0.2836    | 0.3333 | 0.3065 | 0.8445   |
| 0.0004        | 160.0 | 3840 | 1.0653          | 0.2836    | 0.3333 | 0.3065 | 0.8445   |
| 0.0004        | 161.0 | 3864 | 1.0687          | 0.2754    | 0.3333 | 0.3016 | 0.8446   |
| 0.0004        | 162.0 | 3888 | 1.0732          | 0.2714    | 0.3333 | 0.2992 | 0.8448   |
| 0.0005        | 163.0 | 3912 | 1.0729          | 0.2568    | 0.3333 | 0.2901 | 0.8444   |
| 0.0004        | 164.0 | 3936 | 1.0764          | 0.2533    | 0.3333 | 0.2879 | 0.8436   |
| 0.0005        | 165.0 | 3960 | 1.0737          | 0.2794    | 0.3333 | 0.304  | 0.8465   |
| 0.0005        | 166.0 | 3984 | 1.0700          | 0.2754    | 0.3333 | 0.3016 | 0.8482   |
| 0.0004        | 167.0 | 4008 | 1.0679          | 0.2794    | 0.3333 | 0.304  | 0.8496   |
| 0.0005        | 168.0 | 4032 | 1.0695          | 0.2676    | 0.3333 | 0.2969 | 0.8498   |
| 0.0004        | 169.0 | 4056 | 1.0704          | 0.2714    | 0.3333 | 0.2992 | 0.8498   |
| 0.0005        | 170.0 | 4080 | 1.0716          | 0.2794    | 0.3333 | 0.304  | 0.8495   |
| 0.0004        | 171.0 | 4104 | 1.0702          | 0.2639    | 0.3333 | 0.2946 | 0.8498   |
| 0.0005        | 172.0 | 4128 | 1.0713          | 0.25      | 0.3333 | 0.2857 | 0.8491   |
| 0.0004        | 173.0 | 4152 | 1.0736          | 0.2436    | 0.3333 | 0.2815 | 0.8491   |
| 0.0005        | 174.0 | 4176 | 1.0808          | 0.2568    | 0.3333 | 0.2901 | 0.8486   |
| 0.0004        | 175.0 | 4200 | 1.0867          | 0.2639    | 0.3333 | 0.2946 | 0.8486   |
| 0.0004        | 176.0 | 4224 | 1.0899          | 0.2754    | 0.3333 | 0.3016 | 0.8486   |
| 0.0004        | 177.0 | 4248 | 1.0900          | 0.2603    | 0.3333 | 0.2923 | 0.8486   |
| 0.0005        | 178.0 | 4272 | 1.0871          | 0.2754    | 0.3333 | 0.3016 | 0.8489   |
| 0.0004        | 179.0 | 4296 | 1.0863          | 0.2794    | 0.3333 | 0.304  | 0.8492   |
| 0.0004        | 180.0 | 4320 | 1.0892          | 0.2754    | 0.3333 | 0.3016 | 0.8493   |
| 0.0004        | 181.0 | 4344 | 1.0919          | 0.2639    | 0.3333 | 0.2946 | 0.8489   |
| 0.0004        | 182.0 | 4368 | 1.0933          | 0.2639    | 0.3333 | 0.2946 | 0.8490   |
| 0.0004        | 183.0 | 4392 | 1.0949          | 0.2639    | 0.3333 | 0.2946 | 0.8489   |
| 0.0004        | 184.0 | 4416 | 1.0953          | 0.2639    | 0.3333 | 0.2946 | 0.8489   |
| 0.0004        | 185.0 | 4440 | 1.1031          | 0.2714    | 0.3333 | 0.2992 | 0.8496   |
| 0.0004        | 186.0 | 4464 | 1.1049          | 0.2639    | 0.3333 | 0.2946 | 0.8494   |
| 0.0004        | 187.0 | 4488 | 1.1082          | 0.2676    | 0.3333 | 0.2969 | 0.8495   |
| 0.0004        | 188.0 | 4512 | 1.1091          | 0.2639    | 0.3333 | 0.2946 | 0.8494   |
| 0.0004        | 189.0 | 4536 | 1.1109          | 0.2639    | 0.3333 | 0.2946 | 0.8494   |
| 0.0004        | 190.0 | 4560 | 1.1119          | 0.2639    | 0.3333 | 0.2946 | 0.8494   |
| 0.0004        | 191.0 | 4584 | 1.1129          | 0.2603    | 0.3333 | 0.2923 | 0.8494   |
| 0.0004        | 192.0 | 4608 | 1.1139          | 0.2639    | 0.3333 | 0.2946 | 0.8494   |
| 0.0005        | 193.0 | 4632 | 1.1051          | 0.2676    | 0.3333 | 0.2969 | 0.8497   |
| 0.0004        | 194.0 | 4656 | 1.1037          | 0.2639    | 0.3333 | 0.2946 | 0.8495   |
| 0.0004        | 195.0 | 4680 | 1.1045          | 0.2568    | 0.3333 | 0.2901 | 0.8496   |
| 0.0004        | 196.0 | 4704 | 1.1052          | 0.2568    | 0.3333 | 0.2901 | 0.8496   |
| 0.0004        | 197.0 | 4728 | 1.1057          | 0.2568    | 0.3333 | 0.2901 | 0.8496   |
| 0.0004        | 198.0 | 4752 | 1.1057          | 0.2533    | 0.3333 | 0.2879 | 0.8497   |
| 0.0004        | 199.0 | 4776 | 1.1061          | 0.2533    | 0.3333 | 0.2879 | 0.8497   |
| 0.0004        | 200.0 | 4800 | 1.1061          | 0.2533    | 0.3333 | 0.2879 | 0.8498   |


### Framework versions

- Transformers 4.31.0.dev0
- Pytorch 2.0.0
- Datasets 2.1.0
- Tokenizers 0.13.3