File size: 30,542 Bytes
f5aa332 773f81f f5aa332 773f81f f5aa332 3555dec f5aa332 ae401d6 f5aa332 ae401d6 f5aa332 e734735 f5aa332 e734735 ae401d6 f5aa332 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 |
---
tags:
- generated_from_trainer
datasets:
- yawnick/QMSum
metrics:
- rouge
base_model: google/pegasus-xsum
model-index:
- name: pegasus-qmsum-meeting-summarization
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# pegasus-qmsum-meeting-summarization
This model is a fine-tuned version of [google/pegasus-xsum](https://huggingface.co/google/pegasus-xsum) on the QMSum dataset.
It achieves the following results on the evaluation set:
- Loss: 4.2331
- Rouge1: 32.7156
- Rouge2: 10.5699
- Rougel: 23.2759
- Rougelsum: 29.7903
- Gen Len: 61.65
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-06
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 300
- label_smoothing_factor: 0.1
### Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|:-------------:|:------:|:-----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| 5.5746 | 1.09 | 100 | 5.1739 | 9.4941 | 1.7868 | 7.2455 | 8.4302 | 29.825 |
| 5.5784 | 2.17 | 200 | 5.0939 | 9.113 | 1.7887 | 6.9741 | 8.0457 | 26.85 |
| 5.3777 | 3.26 | 300 | 4.9723 | 9.6387 | 1.9301 | 7.349 | 8.7941 | 25.325 |
| 5.1884 | 4.35 | 400 | 4.8423 | 10.6045 | 2.4008 | 7.8423 | 9.4593 | 22.625 |
| 5.0795 | 5.43 | 500 | 4.7313 | 13.7621 | 3.1231 | 9.6944 | 12.2204 | 32.175 |
| 4.9369 | 6.52 | 600 | 4.6555 | 19.5696 | 4.9121 | 14.2603 | 16.9622 | 46.45 |
| 4.8926 | 7.61 | 700 | 4.6038 | 22.8411 | 5.9791 | 17.2227 | 20.1173 | 51.825 |
| 4.7502 | 8.7 | 800 | 4.5659 | 24.0555 | 6.1971 | 18.967 | 20.9143 | 54.25 |
| 4.6876 | 9.78 | 900 | 4.5379 | 24.7066 | 6.0317 | 19.542 | 21.5774 | 57.575 |
| 4.6266 | 10.87 | 1000 | 4.5160 | 26.128 | 6.5089 | 20.5573 | 22.5338 | 58.0 |
| 4.6303 | 11.96 | 1100 | 4.4983 | 26.6639 | 7.1208 | 20.5222 | 23.5783 | 57.925 |
| 4.6263 | 13.04 | 1200 | 4.4815 | 26.8262 | 7.1029 | 20.5172 | 23.6216 | 57.575 |
| 4.577 | 14.13 | 1300 | 4.4667 | 27.7952 | 7.8331 | 21.1111 | 24.6086 | 56.95 |
| 4.5797 | 15.22 | 1400 | 4.4559 | 27.728 | 7.8144 | 21.1519 | 24.4858 | 56.6 |
| 4.4923 | 16.3 | 1500 | 4.4448 | 28.0998 | 8.1346 | 21.4004 | 25.3769 | 55.975 |
| 4.4583 | 17.39 | 1600 | 4.4335 | 28.9003 | 8.6135 | 22.0139 | 26.0409 | 56.55 |
| 4.5036 | 18.48 | 1700 | 4.4246 | 29.2187 | 8.8301 | 22.3569 | 26.1964 | 58.125 |
| 4.4383 | 19.57 | 1800 | 4.4144 | 28.8424 | 8.9131 | 22.0398 | 25.9214 | 56.75 |
| 4.4797 | 20.65 | 1900 | 4.4054 | 28.9285 | 8.9298 | 22.222 | 26.0316 | 56.225 |
| 4.4264 | 21.74 | 2000 | 4.3989 | 29.7184 | 9.0477 | 22.2885 | 26.7439 | 56.225 |
| 4.3615 | 22.83 | 2100 | 4.3902 | 29.1538 | 8.9529 | 22.0076 | 26.4925 | 57.175 |
| 4.329 | 23.91 | 2200 | 4.3839 | 29.5186 | 9.2777 | 21.9025 | 26.3141 | 55.5 |
| 4.3578 | 25.0 | 2300 | 4.3766 | 28.4309 | 8.9423 | 21.0945 | 25.8191 | 53.975 |
| 4.3748 | 26.09 | 2400 | 4.3707 | 28.3 | 9.0625 | 21.4946 | 25.1966 | 53.0 |
| 4.3233 | 27.17 | 2500 | 4.3639 | 28.2325 | 8.9889 | 21.6226 | 25.3677 | 54.6 |
| 4.339 | 28.26 | 2600 | 4.3578 | 28.0744 | 8.774 | 21.2509 | 25.2901 | 54.1 |
| 4.2798 | 29.35 | 2700 | 4.3532 | 27.772 | 8.7096 | 21.1687 | 25.3345 | 54.025 |
| 4.2964 | 30.43 | 2800 | 4.3465 | 27.7827 | 8.1597 | 20.8139 | 25.0152 | 54.45 |
| 4.3365 | 31.52 | 2900 | 4.3423 | 28.2039 | 8.4661 | 21.3546 | 25.6381 | 55.5 |
| 4.2385 | 32.61 | 3000 | 4.3380 | 28.1098 | 8.6483 | 21.5279 | 25.2009 | 53.95 |
| 4.2451 | 33.7 | 3100 | 4.3331 | 28.2745 | 8.5024 | 21.4456 | 25.3363 | 52.6 |
| 4.2393 | 34.78 | 3200 | 4.3289 | 28.7597 | 9.0881 | 21.6532 | 25.8954 | 52.65 |
| 4.2116 | 35.87 | 3300 | 4.3252 | 29.0463 | 9.1218 | 21.8026 | 26.2037 | 53.65 |
| 4.2175 | 36.96 | 3400 | 4.3210 | 28.8009 | 9.0188 | 21.8368 | 25.8678 | 52.85 |
| 4.2071 | 38.04 | 3500 | 4.3169 | 28.9313 | 8.9787 | 21.3554 | 26.0628 | 54.325 |
| 4.1775 | 39.13 | 3600 | 4.3132 | 28.837 | 8.9621 | 21.6342 | 26.0569 | 54.025 |
| 4.1962 | 40.22 | 3700 | 4.3086 | 28.9265 | 9.0701 | 21.588 | 26.0702 | 53.075 |
| 4.1452 | 41.3 | 3800 | 4.3060 | 29.7968 | 9.366 | 22.1712 | 26.8461 | 54.925 |
| 4.1912 | 42.39 | 3900 | 4.3018 | 29.1488 | 9.1631 | 21.6566 | 26.1476 | 54.25 |
| 4.1356 | 43.48 | 4000 | 4.2984 | 30.0138 | 9.2456 | 22.2547 | 27.2714 | 54.85 |
| 4.1272 | 44.57 | 4100 | 4.2949 | 29.8858 | 9.1498 | 22.1221 | 27.0798 | 55.65 |
| 4.1174 | 45.65 | 4200 | 4.2895 | 30.0427 | 9.2297 | 22.2602 | 27.4219 | 56.175 |
| 4.1029 | 46.74 | 4300 | 4.2885 | 29.9443 | 9.4293 | 22.1229 | 27.3496 | 56.45 |
| 4.157 | 47.83 | 4400 | 4.2851 | 30.3693 | 9.406 | 22.471 | 27.7511 | 56.775 |
| 4.1105 | 48.91 | 4500 | 4.2827 | 30.6193 | 9.7082 | 22.6169 | 27.8044 | 57.225 |
| 4.083 | 50.0 | 4600 | 4.2796 | 30.8083 | 9.9211 | 22.5228 | 28.1236 | 57.575 |
| 4.0891 | 51.09 | 4700 | 4.2764 | 30.4201 | 9.6192 | 22.4747 | 27.7514 | 57.475 |
| 4.0603 | 52.17 | 4800 | 4.2741 | 30.7777 | 9.7432 | 22.6705 | 27.5956 | 57.1 |
| 4.0472 | 53.26 | 4900 | 4.2731 | 30.8093 | 9.7916 | 22.5533 | 27.7858 | 56.15 |
| 4.0712 | 54.35 | 5000 | 4.2703 | 29.9667 | 9.5645 | 22.113 | 26.647 | 56.525 |
| 4.0658 | 55.43 | 5100 | 4.2674 | 29.5415 | 9.4291 | 21.6862 | 26.7816 | 56.55 |
| 4.059 | 56.52 | 5200 | 4.2659 | 30.2032 | 9.8875 | 22.2539 | 27.1801 | 56.925 |
| 4.0257 | 57.61 | 5300 | 4.2629 | 30.3181 | 9.8187 | 22.4266 | 27.4318 | 56.925 |
| 4.0002 | 58.7 | 5400 | 4.2608 | 29.6641 | 9.9252 | 22.1725 | 27.0764 | 56.6 |
| 4.0978 | 59.78 | 5500 | 4.2591 | 30.653 | 10.087 | 22.6956 | 27.7481 | 56.25 |
| 3.9978 | 60.87 | 5600 | 4.2568 | 29.5473 | 9.5653 | 21.6367 | 26.391 | 55.825 |
| 3.9832 | 61.96 | 5700 | 4.2552 | 30.6368 | 10.1624 | 22.7204 | 27.5866 | 57.425 |
| 3.9841 | 63.04 | 5800 | 4.2525 | 30.3045 | 9.7966 | 22.2939 | 27.0978 | 57.725 |
| 4.002 | 64.13 | 5900 | 4.2507 | 30.4468 | 9.9323 | 22.6572 | 27.0761 | 57.5 |
| 3.9705 | 65.22 | 6000 | 4.2491 | 30.1218 | 9.6921 | 22.465 | 26.3835 | 57.55 |
| 3.9863 | 66.3 | 6100 | 4.2477 | 31.3982 | 9.9901 | 22.8762 | 27.6169 | 58.975 |
| 3.9308 | 67.39 | 6200 | 4.2454 | 30.2673 | 9.5804 | 22.4474 | 26.6111 | 59.2 |
| 3.9794 | 68.48 | 6300 | 4.2449 | 30.8612 | 9.8254 | 22.8444 | 27.4979 | 58.075 |
| 3.9499 | 69.57 | 6400 | 4.2412 | 30.8366 | 9.7 | 22.4469 | 27.1621 | 59.025 |
| 3.9722 | 70.65 | 6500 | 4.2414 | 30.9625 | 9.8251 | 22.4089 | 27.4342 | 59.1 |
| 3.9125 | 71.74 | 6600 | 4.2394 | 30.5777 | 9.5514 | 22.1581 | 26.8665 | 58.75 |
| 3.9184 | 72.83 | 6700 | 4.2396 | 30.8306 | 9.5469 | 22.6571 | 27.4302 | 59.725 |
| 3.9337 | 73.91 | 6800 | 4.2377 | 30.8688 | 9.6733 | 22.3073 | 27.2943 | 58.975 |
| 3.9145 | 75.0 | 6900 | 4.2358 | 30.467 | 9.6393 | 22.225 | 27.0127 | 58.45 |
| 3.9038 | 76.09 | 7000 | 4.2353 | 30.6344 | 9.3676 | 22.1945 | 27.1871 | 59.275 |
| 3.893 | 77.17 | 7100 | 4.2335 | 31.4486 | 9.8839 | 22.735 | 27.7854 | 59.025 |
| 3.885 | 78.26 | 7200 | 4.2318 | 30.7118 | 9.8568 | 22.2546 | 27.3983 | 58.5 |
| 3.9266 | 79.35 | 7300 | 4.2304 | 31.6171 | 9.8817 | 22.6145 | 27.6888 | 59.25 |
| 3.8826 | 80.43 | 7400 | 4.2299 | 31.0976 | 9.4662 | 22.2285 | 27.817 | 58.95 |
| 3.8775 | 81.52 | 7500 | 4.2286 | 31.1379 | 10.0975 | 22.5686 | 27.883 | 59.8 |
| 3.8455 | 82.61 | 7600 | 4.2292 | 32.076 | 10.0214 | 22.8866 | 28.3828 | 59.15 |
| 3.8838 | 83.7 | 7700 | 4.2269 | 31.5696 | 9.7812 | 22.7619 | 28.2236 | 58.6 |
| 3.8425 | 84.78 | 7800 | 4.2266 | 31.1731 | 9.97 | 22.4203 | 27.4956 | 59.1 |
| 3.8766 | 85.87 | 7900 | 4.2260 | 32.3221 | 10.6243 | 23.079 | 28.9008 | 58.45 |
| 3.8217 | 86.96 | 8000 | 4.2258 | 31.9956 | 10.4201 | 23.083 | 28.4945 | 58.5 |
| 3.8319 | 88.04 | 8100 | 4.2245 | 32.0272 | 10.4673 | 23.3471 | 28.9845 | 58.35 |
| 3.8283 | 89.13 | 8200 | 4.2231 | 32.2943 | 10.2594 | 23.1819 | 29.1345 | 60.5 |
| 3.8394 | 90.22 | 8300 | 4.2221 | 31.3976 | 10.3085 | 22.6581 | 28.2494 | 59.25 |
| 3.8258 | 91.3 | 8400 | 4.2203 | 31.4433 | 10.1184 | 22.672 | 28.1236 | 58.85 |
| 3.7981 | 92.39 | 8500 | 4.2205 | 31.1313 | 10.0056 | 22.677 | 27.7409 | 59.075 |
| 3.8349 | 93.48 | 8600 | 4.2215 | 31.5779 | 10.0303 | 22.6155 | 28.0566 | 59.2 |
| 3.8225 | 94.57 | 8700 | 4.2201 | 31.9646 | 10.0643 | 22.7808 | 28.67 | 58.925 |
| 3.8145 | 95.65 | 8800 | 4.2193 | 32.0347 | 10.5103 | 23.095 | 28.6056 | 57.225 |
| 3.7771 | 96.74 | 8900 | 4.2180 | 30.8138 | 9.602 | 22.2649 | 27.7948 | 57.875 |
| 3.823 | 97.83 | 9000 | 4.2168 | 31.3785 | 9.7046 | 22.3877 | 28.2578 | 58.675 |
| 3.7701 | 98.91 | 9100 | 4.2169 | 31.4511 | 9.9183 | 22.6645 | 28.1932 | 59.0 |
| 3.773 | 100.0 | 9200 | 4.2169 | 31.7392 | 9.9669 | 22.5894 | 28.218 | 58.15 |
| 3.7661 | 101.09 | 9300 | 4.2161 | 31.5507 | 9.8992 | 22.4602 | 28.3357 | 58.375 |
| 3.7875 | 102.17 | 9400 | 4.2163 | 31.5145 | 9.5173 | 22.321 | 27.8613 | 58.375 |
| 3.7659 | 103.26 | 9500 | 4.2152 | 31.2967 | 9.8797 | 22.6247 | 28.0317 | 57.925 |
| 3.7576 | 104.35 | 9600 | 4.2139 | 31.5739 | 9.8376 | 22.7561 | 28.2318 | 58.4 |
| 3.7784 | 105.43 | 9700 | 4.2144 | 32.2269 | 10.2299 | 22.6582 | 28.6249 | 58.425 |
| 3.7356 | 106.52 | 9800 | 4.2139 | 32.3031 | 10.1505 | 22.7079 | 28.9052 | 58.475 |
| 3.7799 | 107.61 | 9900 | 4.2124 | 31.1334 | 9.1481 | 22.1297 | 27.5951 | 58.6 |
| 3.7269 | 108.7 | 10000 | 4.2122 | 31.6957 | 9.2874 | 22.4867 | 28.225 | 58.4 |
| 3.719 | 109.78 | 10100 | 4.2108 | 31.477 | 10.0245 | 22.4703 | 28.1316 | 58.075 |
| 3.7411 | 110.87 | 10200 | 4.2112 | 31.4165 | 9.9791 | 22.4396 | 28.3068 | 58.275 |
| 3.7135 | 111.96 | 10300 | 4.2122 | 31.4924 | 9.9864 | 22.496 | 28.2414 | 57.8 |
| 3.7317 | 113.04 | 10400 | 4.2120 | 31.6599 | 10.1605 | 22.5322 | 28.3045 | 59.075 |
| 3.7113 | 114.13 | 10500 | 4.2127 | 31.6814 | 10.106 | 22.4311 | 28.5808 | 59.5 |
| 3.7063 | 115.22 | 10600 | 4.2132 | 31.2448 | 10.0006 | 22.5549 | 28.4686 | 57.775 |
| 3.681 | 116.3 | 10700 | 4.2123 | 31.1739 | 10.0533 | 22.2954 | 28.0822 | 58.35 |
| 3.7369 | 117.39 | 10800 | 4.2118 | 31.8541 | 10.1452 | 22.7607 | 28.9501 | 58.8 |
| 3.6645 | 118.48 | 10900 | 4.2122 | 31.7128 | 9.8554 | 22.4464 | 28.5888 | 58.375 |
| 3.6766 | 119.57 | 11000 | 4.2118 | 31.1492 | 9.8058 | 22.0978 | 28.1827 | 58.725 |
| 3.6915 | 120.65 | 11100 | 4.2110 | 31.1679 | 9.5755 | 22.1391 | 28.0886 | 58.375 |
| 3.6702 | 121.74 | 11200 | 4.2129 | 31.0682 | 9.7375 | 22.0118 | 28.2189 | 59.15 |
| 3.6946 | 122.83 | 11300 | 4.2118 | 31.6134 | 9.5918 | 22.2506 | 28.5343 | 59.175 |
| 3.6713 | 123.91 | 11400 | 4.2110 | 31.3585 | 9.4211 | 22.1884 | 27.8744 | 59.05 |
| 3.6694 | 125.0 | 11500 | 4.2126 | 32.0058 | 9.6453 | 22.3911 | 28.6928 | 59.55 |
| 3.6585 | 126.09 | 11600 | 4.2123 | 31.7679 | 9.7101 | 22.2378 | 28.4985 | 59.2 |
| 3.6857 | 127.17 | 11700 | 4.2118 | 31.7766 | 10.0375 | 22.5097 | 28.8104 | 59.6 |
| 3.6338 | 128.26 | 11800 | 4.2126 | 32.2508 | 10.2617 | 22.6745 | 29.0714 | 59.075 |
| 3.6412 | 129.35 | 11900 | 4.2135 | 32.0515 | 10.0905 | 22.7015 | 29.0028 | 58.9 |
| 3.6594 | 130.43 | 12000 | 4.2122 | 32.7784 | 10.351 | 23.0969 | 29.6672 | 59.525 |
| 3.6571 | 131.52 | 12100 | 4.2120 | 32.3165 | 10.329 | 22.8445 | 29.2886 | 59.5 |
| 3.6002 | 132.61 | 12200 | 4.2120 | 32.5553 | 10.0875 | 22.6064 | 29.1046 | 59.425 |
| 3.6621 | 133.7 | 12300 | 4.2126 | 31.7637 | 9.9785 | 22.5716 | 28.7173 | 59.275 |
| 3.6651 | 134.78 | 12400 | 4.2122 | 31.7568 | 9.7503 | 22.3876 | 28.6015 | 59.6 |
| 3.6127 | 135.87 | 12500 | 4.2123 | 31.5708 | 9.5203 | 21.9951 | 28.2082 | 58.75 |
| 3.6544 | 136.96 | 12600 | 4.2124 | 32.0767 | 9.8955 | 22.2724 | 28.4755 | 59.5 |
| 3.5994 | 138.04 | 12700 | 4.2125 | 31.8523 | 9.9159 | 22.2978 | 28.8159 | 59.175 |
| 3.6174 | 139.13 | 12800 | 4.2114 | 32.2165 | 9.784 | 22.4377 | 28.5603 | 59.1 |
| 3.6122 | 140.22 | 12900 | 4.2115 | 32.0247 | 9.6881 | 22.3116 | 28.61 | 58.9 |
| 3.6174 | 141.3 | 13000 | 4.2116 | 31.9549 | 9.5924 | 22.3997 | 28.9145 | 59.15 |
| 3.5965 | 142.39 | 13100 | 4.2113 | 32.6173 | 10.4241 | 22.8644 | 29.3928 | 60.9 |
| 3.6076 | 143.48 | 13200 | 4.2112 | 33.0058 | 10.6417 | 23.0297 | 29.8375 | 61.0 |
| 3.6013 | 144.57 | 13300 | 4.2105 | 33.005 | 10.5398 | 22.9758 | 29.7266 | 60.325 |
| 3.6181 | 145.65 | 13400 | 4.2117 | 31.0558 | 9.4714 | 21.9025 | 27.9627 | 60.025 |
| 3.6288 | 146.74 | 13500 | 4.2107 | 32.7196 | 10.4991 | 22.9182 | 29.6586 | 60.25 |
| 3.5879 | 147.83 | 13600 | 4.2091 | 32.6755 | 10.3936 | 22.9559 | 29.5314 | 60.425 |
| 3.591 | 148.91 | 13700 | 4.2101 | 33.2956 | 10.6616 | 22.8509 | 29.5237 | 60.4 |
| 3.5658 | 150.0 | 13800 | 4.2116 | 33.4712 | 10.3725 | 23.1449 | 30.0987 | 60.2 |
| 3.574 | 151.09 | 13900 | 4.2115 | 33.5427 | 10.5852 | 22.9671 | 29.8456 | 60.175 |
| 3.5795 | 152.17 | 14000 | 4.2115 | 33.4387 | 10.5744 | 23.4785 | 30.0494 | 60.15 |
| 3.5728 | 153.26 | 14100 | 4.2119 | 33.1244 | 10.0308 | 22.8377 | 29.7725 | 60.775 |
| 3.5441 | 154.35 | 14200 | 4.2121 | 32.9226 | 9.9625 | 22.9013 | 29.6004 | 59.7 |
| 3.5236 | 155.43 | 14300 | 4.2114 | 32.3717 | 9.9122 | 22.78 | 28.8305 | 59.725 |
| 3.5679 | 156.52 | 14400 | 4.2120 | 33.6347 | 10.7457 | 23.5191 | 30.1966 | 60.65 |
| 3.5574 | 157.61 | 14500 | 4.2119 | 33.4821 | 10.986 | 23.3567 | 30.1972 | 60.1 |
| 3.5935 | 158.7 | 14600 | 4.2115 | 32.7255 | 10.2639 | 23.1617 | 29.8065 | 60.35 |
| 3.5316 | 159.78 | 14700 | 4.2118 | 32.8033 | 10.0216 | 22.7099 | 29.3968 | 60.525 |
| 3.5618 | 160.87 | 14800 | 4.2118 | 32.6244 | 10.7228 | 22.8601 | 29.3613 | 60.8 |
| 3.545 | 161.96 | 14900 | 4.2132 | 32.6231 | 10.0711 | 22.4686 | 29.5341 | 59.675 |
| 3.5466 | 163.04 | 15000 | 4.2129 | 32.7601 | 10.3376 | 22.2373 | 29.3588 | 59.4 |
| 3.5594 | 164.13 | 15100 | 4.2127 | 32.4645 | 10.5106 | 22.6804 | 29.6229 | 60.375 |
| 3.4839 | 165.22 | 15200 | 4.2130 | 32.1799 | 10.0462 | 22.5474 | 29.1419 | 59.75 |
| 3.5492 | 166.3 | 15300 | 4.2133 | 32.6831 | 10.5307 | 22.8539 | 29.6406 | 59.875 |
| 3.5053 | 167.39 | 15400 | 4.2133 | 32.8614 | 10.0344 | 23.0577 | 29.5848 | 60.975 |
| 3.5427 | 168.48 | 15500 | 4.2140 | 32.7897 | 10.178 | 22.6287 | 29.4839 | 60.1 |
| 3.5495 | 169.57 | 15600 | 4.2126 | 33.1428 | 10.2866 | 22.9377 | 29.6883 | 60.525 |
| 3.5245 | 170.65 | 15700 | 4.2116 | 32.9892 | 10.1082 | 23.1528 | 29.576 | 60.675 |
| 3.5121 | 171.74 | 15800 | 4.2131 | 33.2677 | 10.5916 | 23.3002 | 29.8222 | 59.975 |
| 3.5559 | 172.83 | 15900 | 4.2126 | 32.5155 | 9.9557 | 22.6846 | 29.1171 | 60.85 |
| 3.4758 | 173.91 | 16000 | 4.2133 | 32.374 | 9.9127 | 22.4816 | 29.2839 | 60.9 |
| 3.5148 | 175.0 | 16100 | 4.2125 | 32.5611 | 9.8266 | 22.5993 | 28.9821 | 61.1 |
| 3.5093 | 176.09 | 16200 | 4.2132 | 32.1092 | 9.6761 | 22.3612 | 28.7771 | 60.05 |
| 3.5248 | 177.17 | 16300 | 4.2143 | 32.2696 | 9.6471 | 22.2791 | 28.9759 | 60.925 |
| 3.4807 | 178.26 | 16400 | 4.2139 | 31.9593 | 9.3878 | 22.0643 | 28.5392 | 61.3 |
| 3.5138 | 179.35 | 16500 | 4.2144 | 32.0284 | 9.8303 | 22.5724 | 29.0168 | 59.95 |
| 3.4834 | 180.43 | 16600 | 4.2153 | 32.3203 | 9.5741 | 22.4998 | 28.8014 | 60.5 |
| 3.4701 | 181.52 | 16700 | 4.2156 | 31.7243 | 9.544 | 22.1355 | 28.2238 | 61.275 |
| 3.5501 | 182.61 | 16800 | 4.2152 | 32.519 | 9.9372 | 22.3881 | 28.8347 | 61.45 |
| 3.4789 | 183.7 | 16900 | 4.2148 | 32.3324 | 9.7556 | 22.2474 | 28.7559 | 61.575 |
| 3.5172 | 184.78 | 17000 | 4.2156 | 32.161 | 9.4847 | 22.2358 | 28.8895 | 60.95 |
| 3.4681 | 185.87 | 17100 | 4.2167 | 32.6524 | 9.7116 | 22.8415 | 29.0798 | 60.575 |
| 3.4936 | 186.96 | 17200 | 4.2173 | 32.533 | 9.9478 | 22.7379 | 29.1301 | 61.575 |
| 3.4664 | 188.04 | 17300 | 4.2165 | 32.4549 | 10.1094 | 22.7097 | 28.7992 | 61.4 |
| 3.4599 | 189.13 | 17400 | 4.2164 | 32.6665 | 10.3463 | 22.7678 | 29.308 | 61.575 |
| 3.4724 | 190.22 | 17500 | 4.2175 | 32.4146 | 10.1782 | 22.7414 | 29.3546 | 60.75 |
| 3.4923 | 191.3 | 17600 | 4.2163 | 32.3624 | 9.8306 | 22.7311 | 28.7497 | 59.825 |
| 3.4771 | 192.39 | 17700 | 4.2161 | 33.1427 | 10.429 | 23.462 | 29.6967 | 60.35 |
| 3.4737 | 193.48 | 17800 | 4.2168 | 31.6894 | 9.7073 | 22.527 | 28.3711 | 60.65 |
| 3.4307 | 194.57 | 17900 | 4.2182 | 32.4769 | 10.1673 | 22.8356 | 29.4565 | 60.75 |
| 3.4843 | 195.65 | 18000 | 4.2168 | 32.5461 | 10.2855 | 22.8587 | 29.1242 | 60.825 |
| 3.4479 | 196.74 | 18100 | 4.2170 | 32.9284 | 10.2293 | 23.2679 | 29.8067 | 61.075 |
| 3.489 | 197.83 | 18200 | 4.2180 | 32.9561 | 10.481 | 23.2807 | 29.5499 | 61.25 |
| 3.4596 | 198.91 | 18300 | 4.2179 | 33.1418 | 10.2768 | 22.8762 | 30.0241 | 61.2 |
| 3.4552 | 200.0 | 18400 | 4.2171 | 33.5524 | 10.5969 | 23.5734 | 30.1587 | 61.525 |
| 3.4699 | 201.09 | 18500 | 4.2176 | 33.1941 | 10.3296 | 23.1962 | 30.1624 | 61.45 |
| 3.4281 | 202.17 | 18600 | 4.2187 | 33.3715 | 10.1919 | 23.1843 | 30.3192 | 61.55 |
| 3.4561 | 203.26 | 18700 | 4.2186 | 32.5288 | 9.9299 | 22.6515 | 29.2853 | 61.575 |
| 3.446 | 204.35 | 18800 | 4.2188 | 33.4268 | 10.7152 | 23.6525 | 30.4668 | 61.575 |
| 3.4259 | 205.43 | 18900 | 4.2189 | 33.1715 | 10.198 | 22.9264 | 29.8387 | 61.25 |
| 3.4497 | 206.52 | 19000 | 4.2192 | 33.3472 | 10.5372 | 23.0833 | 30.2925 | 61.25 |
| 3.4674 | 207.61 | 19100 | 4.2192 | 32.7581 | 10.2502 | 23.0554 | 29.6639 | 61.175 |
| 3.4521 | 208.7 | 19200 | 4.2186 | 33.7883 | 10.8639 | 23.4038 | 30.6114 | 61.475 |
| 3.443 | 209.78 | 19300 | 4.2194 | 33.029 | 10.6622 | 22.9009 | 29.9762 | 61.675 |
| 3.4356 | 210.87 | 19400 | 4.2199 | 32.7229 | 9.9204 | 22.5445 | 29.5517 | 61.3 |
| 3.4198 | 211.96 | 19500 | 4.2208 | 33.5216 | 10.3836 | 22.9423 | 29.9006 | 61.625 |
| 3.4417 | 213.04 | 19600 | 4.2210 | 32.7772 | 10.3206 | 22.9031 | 29.3774 | 61.625 |
| 3.4348 | 214.13 | 19700 | 4.2214 | 31.9959 | 10.0821 | 22.2012 | 28.6722 | 61.375 |
| 3.4528 | 215.22 | 19800 | 4.2213 | 32.5434 | 10.2807 | 22.6512 | 29.1705 | 61.65 |
| 3.3955 | 216.3 | 19900 | 4.2220 | 32.9148 | 10.5869 | 22.8107 | 29.4975 | 61.675 |
| 3.4437 | 217.39 | 20000 | 4.2227 | 32.8879 | 10.4334 | 22.6863 | 29.6794 | 61.125 |
| 3.4374 | 218.48 | 20100 | 4.2225 | 32.1453 | 9.9115 | 22.2936 | 28.9428 | 61.1 |
| 3.429 | 219.57 | 20200 | 4.2230 | 33.0805 | 10.5792 | 22.9417 | 29.9572 | 61.55 |
| 3.4089 | 220.65 | 20300 | 4.2239 | 32.0499 | 10.1613 | 22.6264 | 28.9217 | 61.65 |
| 3.418 | 221.74 | 20400 | 4.2237 | 32.6069 | 10.5032 | 22.8024 | 29.5804 | 61.275 |
| 3.4274 | 222.83 | 20500 | 4.2235 | 31.8624 | 10.2513 | 22.2816 | 28.8234 | 61.2 |
| 3.4156 | 223.91 | 20600 | 4.2242 | 32.2666 | 10.4604 | 22.5607 | 29.0666 | 61.025 |
| 3.4135 | 225.0 | 20700 | 4.2247 | 31.3445 | 10.0898 | 22.0664 | 28.5988 | 60.5 |
| 3.4283 | 226.09 | 20800 | 4.2245 | 31.47 | 10.0171 | 21.9423 | 28.4329 | 61.175 |
| 3.4048 | 227.17 | 20900 | 4.2242 | 31.93 | 10.4874 | 22.5287 | 29.1292 | 60.7 |
| 3.3925 | 228.26 | 21000 | 4.2243 | 32.3618 | 10.0902 | 22.6176 | 29.2689 | 60.775 |
| 3.4371 | 229.35 | 21100 | 4.2245 | 32.174 | 10.0424 | 22.516 | 28.9855 | 60.775 |
| 3.3789 | 230.43 | 21200 | 4.2239 | 33.0237 | 10.8644 | 23.3016 | 29.916 | 61.275 |
| 3.4109 | 231.52 | 21300 | 4.2248 | 32.88 | 10.6969 | 22.8426 | 30.0468 | 60.8 |
| 3.4128 | 232.61 | 21400 | 4.2257 | 32.6551 | 10.6032 | 22.6787 | 29.5307 | 60.725 |
| 3.3941 | 233.7 | 21500 | 4.2266 | 31.9296 | 10.0718 | 22.5 | 28.9451 | 60.75 |
| 3.3734 | 234.78 | 21600 | 4.2266 | 32.4862 | 10.0754 | 22.9705 | 29.2087 | 61.225 |
| 3.4144 | 235.87 | 21700 | 4.2269 | 32.1757 | 10.1225 | 22.6842 | 29.1731 | 60.75 |
| 3.3986 | 236.96 | 21800 | 4.2273 | 32.3403 | 10.481 | 22.7186 | 29.3236 | 60.725 |
| 3.3898 | 238.04 | 21900 | 4.2275 | 32.4957 | 10.4595 | 22.8682 | 29.6414 | 60.8 |
| 3.4031 | 239.13 | 22000 | 4.2275 | 32.4625 | 10.3807 | 22.7121 | 29.5187 | 60.725 |
| 3.3836 | 240.22 | 22100 | 4.2274 | 31.8107 | 10.2075 | 22.4437 | 28.9719 | 60.725 |
| 3.4084 | 241.3 | 22200 | 4.2272 | 32.3374 | 10.1027 | 22.5784 | 29.2192 | 61.2 |
| 3.3805 | 242.39 | 22300 | 4.2276 | 32.2783 | 10.375 | 22.7825 | 29.3762 | 61.2 |
| 3.3815 | 243.48 | 22400 | 4.2277 | 32.3337 | 10.3561 | 22.8489 | 29.4485 | 61.15 |
| 3.418 | 244.57 | 22500 | 4.2273 | 32.333 | 10.2841 | 22.8481 | 29.403 | 61.125 |
| 3.369 | 245.65 | 22600 | 4.2277 | 32.038 | 10.3555 | 22.6939 | 29.242 | 60.7 |
| 3.4305 | 246.74 | 22700 | 4.2276 | 32.7594 | 10.6867 | 23.0632 | 29.5852 | 61.575 |
| 3.3928 | 247.83 | 22800 | 4.2282 | 32.4979 | 10.5013 | 22.7875 | 29.4793 | 61.55 |
| 3.3676 | 248.91 | 22900 | 4.2286 | 32.6014 | 10.5697 | 22.8526 | 29.7876 | 61.6 |
| 3.3918 | 250.0 | 23000 | 4.2288 | 32.4746 | 10.6321 | 22.586 | 29.6323 | 60.675 |
| 3.395 | 251.09 | 23100 | 4.2294 | 32.4704 | 10.5456 | 22.6785 | 29.5769 | 60.725 |
| 3.363 | 252.17 | 23200 | 4.2296 | 32.2721 | 10.2554 | 22.5303 | 29.4554 | 60.725 |
| 3.3884 | 253.26 | 23300 | 4.2298 | 32.2746 | 10.434 | 22.6686 | 29.4486 | 60.725 |
| 3.3891 | 254.35 | 23400 | 4.2296 | 32.5382 | 10.5112 | 23.0243 | 29.8106 | 61.125 |
| 3.3679 | 255.43 | 23500 | 4.2296 | 32.4656 | 10.5631 | 22.9952 | 29.6832 | 61.125 |
| 3.4078 | 256.52 | 23600 | 4.2297 | 32.3377 | 10.4791 | 22.8362 | 29.6212 | 60.7 |
| 3.3642 | 257.61 | 23700 | 4.2302 | 32.2519 | 10.5551 | 22.6957 | 29.3763 | 61.075 |
| 3.3745 | 258.7 | 23800 | 4.2300 | 31.9413 | 10.4752 | 22.7447 | 29.1 | 61.175 |
| 3.3844 | 259.78 | 23900 | 4.2305 | 32.237 | 10.5492 | 23.0342 | 29.4079 | 61.65 |
| 3.3501 | 260.87 | 24000 | 4.2302 | 31.9797 | 10.4631 | 22.9089 | 29.332 | 61.65 |
| 3.4259 | 261.96 | 24100 | 4.2304 | 31.7515 | 10.3564 | 22.5923 | 29.1275 | 61.175 |
| 3.3578 | 263.04 | 24200 | 4.2309 | 32.0462 | 10.3883 | 22.9083 | 29.3591 | 61.65 |
| 3.39 | 264.13 | 24300 | 4.2308 | 31.9307 | 10.3057 | 22.8501 | 29.2547 | 61.65 |
| 3.3805 | 265.22 | 24400 | 4.2312 | 32.1836 | 10.3577 | 23.1293 | 29.4325 | 61.65 |
| 3.3667 | 266.3 | 24500 | 4.2309 | 32.1545 | 10.301 | 23.0613 | 29.343 | 61.65 |
| 3.3977 | 267.39 | 24600 | 4.2313 | 31.9549 | 10.2824 | 23.0397 | 29.2684 | 61.65 |
| 3.3434 | 268.48 | 24700 | 4.2314 | 31.9432 | 10.167 | 23.098 | 29.2669 | 61.65 |
| 3.3577 | 269.57 | 24800 | 4.2316 | 31.9679 | 10.3075 | 23.0715 | 29.3077 | 61.65 |
| 3.3781 | 270.65 | 24900 | 4.2317 | 32.2292 | 10.2988 | 23.0879 | 29.4171 | 61.65 |
| 3.3514 | 271.74 | 25000 | 4.2321 | 32.1653 | 10.4198 | 23.0554 | 29.3574 | 61.65 |
| 3.3935 | 272.83 | 25100 | 4.2320 | 32.134 | 10.2884 | 22.9444 | 29.2272 | 61.65 |
| 3.3447 | 273.91 | 25200 | 4.2324 | 32.3498 | 10.4505 | 23.0734 | 29.4438 | 61.65 |
| 3.3872 | 275.0 | 25300 | 4.2323 | 32.1743 | 10.4152 | 22.9462 | 29.3187 | 61.65 |
| 3.3755 | 276.09 | 25400 | 4.2324 | 32.2311 | 10.372 | 22.9563 | 29.3285 | 61.65 |
| 3.3832 | 277.17 | 25500 | 4.2323 | 32.0289 | 10.2105 | 22.9636 | 29.1449 | 61.65 |
| 3.3367 | 278.26 | 25600 | 4.2321 | 32.3053 | 10.2512 | 23.0834 | 29.4111 | 61.65 |
| 3.3767 | 279.35 | 25700 | 4.2323 | 32.4099 | 10.2793 | 23.0137 | 29.4049 | 61.65 |
| 3.3989 | 280.43 | 25800 | 4.2324 | 32.3471 | 10.4356 | 23.0179 | 29.4453 | 61.65 |
| 3.3625 | 281.52 | 25900 | 4.2325 | 32.2213 | 10.4363 | 22.9573 | 29.2886 | 61.65 |
| 3.3352 | 282.61 | 26000 | 4.2328 | 32.713 | 10.7489 | 23.2367 | 29.8725 | 61.65 |
| 3.3899 | 283.7 | 26100 | 4.2328 | 32.2145 | 10.2347 | 22.7896 | 29.2107 | 61.65 |
| 3.359 | 284.78 | 26200 | 4.2327 | 32.2466 | 10.4236 | 22.916 | 29.4227 | 61.65 |
| 3.3866 | 285.87 | 26300 | 4.2327 | 32.2466 | 10.4236 | 22.916 | 29.4227 | 61.65 |
| 3.3845 | 286.96 | 26400 | 4.2328 | 32.2466 | 10.4236 | 22.916 | 29.4227 | 61.65 |
| 3.3486 | 288.04 | 26500 | 4.2328 | 32.595 | 10.5041 | 23.1214 | 29.69 | 61.65 |
| 3.3807 | 289.13 | 26600 | 4.2328 | 32.759 | 10.566 | 23.3108 | 29.8555 | 61.65 |
| 3.3676 | 290.22 | 26700 | 4.2330 | 32.759 | 10.566 | 23.3108 | 29.8555 | 61.65 |
| 3.3361 | 291.3 | 26800 | 4.2332 | 32.759 | 10.566 | 23.3108 | 29.8555 | 61.65 |
| 3.3897 | 292.39 | 26900 | 4.2331 | 32.7251 | 10.566 | 23.3108 | 29.7958 | 61.65 |
| 3.3579 | 293.48 | 27000 | 4.2331 | 32.759 | 10.566 | 23.3108 | 29.8555 | 61.65 |
| 3.3809 | 294.57 | 27100 | 4.2331 | 32.759 | 10.566 | 23.3108 | 29.8555 | 61.65 |
| 3.3885 | 295.65 | 27200 | 4.2331 | 32.759 | 10.566 | 23.3108 | 29.8555 | 61.65 |
| 3.3173 | 296.74 | 27300 | 4.2331 | 32.7156 | 10.5699 | 23.2759 | 29.7903 | 61.65 |
| 3.3648 | 297.83 | 27400 | 4.2331 | 32.7156 | 10.5699 | 23.2759 | 29.7903 | 61.65 |
| 3.3793 | 298.91 | 27500 | 4.2331 | 32.7156 | 10.5699 | 23.2759 | 29.7903 | 61.65 |
| 3.3604 | 300.0 | 27600 | 4.2331 | 32.7156 | 10.5699 | 23.2759 | 29.7903 | 61.65 |
### Framework versions
- Transformers 4.18.0
- Pytorch 1.11.0+cu113
- Datasets 2.1.0
- Tokenizers 0.12.1
|