metadata
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: finetuned-marktextepoch-n800
results: []
finetuned-marktextepoch-n800
This model is a fine-tuned version of leokai/finetuned-marktextepoch-n600 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 2.8433
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 200
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 0.287 | 1.0 | 1606 | 2.8473 |
| 0.2913 | 2.0 | 3212 | 2.8147 |
| 0.2865 | 3.0 | 4818 | 2.8809 |
| 0.2947 | 4.0 | 6424 | 2.8510 |
| 0.2988 | 5.0 | 8030 | 2.8883 |
| 0.3109 | 6.0 | 9636 | 2.9016 |
| 0.309 | 7.0 | 11242 | 2.8869 |
| 0.301 | 8.0 | 12848 | 2.9201 |
| 0.303 | 9.0 | 14454 | 2.8902 |
| 0.3156 | 10.0 | 16060 | 2.8888 |
| 0.3132 | 11.0 | 17666 | 2.8777 |
| 0.3089 | 12.0 | 19272 | 2.9429 |
| 0.3146 | 13.0 | 20878 | 2.9131 |
| 0.3297 | 14.0 | 22484 | 2.8983 |
| 0.3214 | 15.0 | 24090 | 2.9321 |
| 0.3095 | 16.0 | 25696 | 2.9436 |
| 0.3171 | 17.0 | 27302 | 2.9163 |
| 0.308 | 18.0 | 28908 | 2.9545 |
| 0.3174 | 19.0 | 30514 | 2.9161 |
| 0.3163 | 20.0 | 32120 | 2.9081 |
| 0.3191 | 21.0 | 33726 | 2.9465 |
| 0.3254 | 22.0 | 35332 | 2.9404 |
| 0.3168 | 23.0 | 36938 | 2.9054 |
| 0.33 | 24.0 | 38544 | 2.9274 |
| 0.3115 | 25.0 | 40150 | 2.9277 |
| 0.3125 | 26.0 | 41756 | 2.9627 |
| 0.3246 | 27.0 | 43362 | 2.9583 |
| 0.3133 | 28.0 | 44968 | 2.9433 |
| 0.3221 | 29.0 | 46574 | 2.9747 |
| 0.3185 | 30.0 | 48180 | 2.9793 |
| 0.3123 | 31.0 | 49786 | 2.9170 |
| 0.3169 | 32.0 | 51392 | 2.9711 |
| 0.3175 | 33.0 | 52998 | 2.9457 |
| 0.3253 | 34.0 | 54604 | 2.9518 |
| 0.3163 | 35.0 | 56210 | 2.9218 |
| 0.3113 | 36.0 | 57816 | 2.9524 |
| 0.3208 | 37.0 | 59422 | 2.9570 |
| 0.3197 | 38.0 | 61028 | 2.9439 |
| 0.3213 | 39.0 | 62634 | 2.9416 |
| 0.3259 | 40.0 | 64240 | 2.9884 |
| 0.3216 | 41.0 | 65846 | 2.9641 |
| 0.3154 | 42.0 | 67452 | 2.9797 |
| 0.3258 | 43.0 | 69058 | 2.9813 |
| 0.3236 | 44.0 | 70664 | 2.9700 |
| 0.3134 | 45.0 | 72270 | 2.9881 |
| 0.3219 | 46.0 | 73876 | 2.9982 |
| 0.3243 | 47.0 | 75482 | 2.9702 |
| 0.3246 | 48.0 | 77088 | 2.9706 |
| 0.3245 | 49.0 | 78694 | 2.9965 |
| 0.3124 | 50.0 | 80300 | 2.9893 |
| 0.3172 | 51.0 | 81906 | 2.9859 |
| 0.3118 | 52.0 | 83512 | 2.9707 |
| 0.3187 | 53.0 | 85118 | 2.9771 |
| 0.3256 | 54.0 | 86724 | 2.9827 |
| 0.3222 | 55.0 | 88330 | 2.9776 |
| 0.3212 | 56.0 | 89936 | 2.9607 |
| 0.3215 | 57.0 | 91542 | 2.9664 |
| 0.3266 | 58.0 | 93148 | 2.9638 |
| 0.3209 | 59.0 | 94754 | 2.9842 |
| 0.333 | 60.0 | 96360 | 3.0053 |
| 0.3202 | 61.0 | 97966 | 2.9833 |
| 0.3155 | 62.0 | 99572 | 2.9952 |
| 0.32 | 63.0 | 101178 | 2.9737 |
| 0.3291 | 64.0 | 102784 | 2.9804 |
| 0.3259 | 65.0 | 104390 | 2.9767 |
| 0.32 | 66.0 | 105996 | 2.9610 |
| 0.3208 | 67.0 | 107602 | 3.0111 |
| 0.3277 | 68.0 | 109208 | 2.9588 |
| 0.337 | 69.0 | 110814 | 2.9920 |
| 0.3296 | 70.0 | 112420 | 2.9466 |
| 0.3197 | 71.0 | 114026 | 2.9619 |
| 0.323 | 72.0 | 115632 | 2.9733 |
| 0.3247 | 73.0 | 117238 | 2.9787 |
| 0.3246 | 74.0 | 118844 | 2.9383 |
| 0.3203 | 75.0 | 120450 | 3.0123 |
| 0.3272 | 76.0 | 122056 | 3.0284 |
| 0.3407 | 77.0 | 123662 | 3.0047 |
| 0.3312 | 78.0 | 125268 | 2.9465 |
| 0.3262 | 79.0 | 126874 | 2.9805 |
| 0.3221 | 80.0 | 128480 | 2.9713 |
| 0.3246 | 81.0 | 130086 | 2.9869 |
| 0.3208 | 82.0 | 131692 | 2.9970 |
| 0.3196 | 83.0 | 133298 | 2.9864 |
| 0.3311 | 84.0 | 134904 | 3.0080 |
| 0.3235 | 85.0 | 136510 | 2.9739 |
| 0.3251 | 86.0 | 138116 | 2.9749 |
| 0.3248 | 87.0 | 139722 | 2.9588 |
| 0.3342 | 88.0 | 141328 | 2.9509 |
| 0.3456 | 89.0 | 142934 | 2.9713 |
| 0.3337 | 90.0 | 144540 | 2.9968 |
| 0.323 | 91.0 | 146146 | 2.9790 |
| 0.3202 | 92.0 | 147752 | 2.9919 |
| 0.3308 | 93.0 | 149358 | 3.0100 |
| 0.3232 | 94.0 | 150964 | 2.9873 |
| 0.3356 | 95.0 | 152570 | 2.9786 |
| 0.3282 | 96.0 | 154176 | 2.9965 |
| 0.3404 | 97.0 | 155782 | 3.0198 |
| 0.3212 | 98.0 | 157388 | 2.9713 |
| 0.3307 | 99.0 | 158994 | 2.9979 |
| 0.337 | 100.0 | 160600 | 2.9805 |
| 0.3354 | 101.0 | 162206 | 2.9759 |
| 0.3252 | 102.0 | 163812 | 2.9810 |
| 0.3324 | 103.0 | 165418 | 2.9433 |
| 0.3278 | 104.0 | 167024 | 3.0079 |
| 0.3419 | 105.0 | 168630 | 2.9576 |
| 0.343 | 106.0 | 170236 | 2.9610 |
| 0.3294 | 107.0 | 171842 | 2.9147 |
| 0.3271 | 108.0 | 173448 | 2.9740 |
| 0.3315 | 109.0 | 175054 | 2.9736 |
| 0.3413 | 110.0 | 176660 | 2.9819 |
| 0.3344 | 111.0 | 178266 | 2.9783 |
| 0.3399 | 112.0 | 179872 | 2.9836 |
| 0.3314 | 113.0 | 181478 | 2.9605 |
| 0.3344 | 114.0 | 183084 | 2.9629 |
| 0.3346 | 115.0 | 184690 | 2.9535 |
| 0.3324 | 116.0 | 186296 | 2.9139 |
| 0.3493 | 117.0 | 187902 | 2.9383 |
| 0.341 | 118.0 | 189508 | 2.9547 |
| 0.3414 | 119.0 | 191114 | 2.9592 |
| 0.335 | 120.0 | 192720 | 2.9822 |
| 0.3423 | 121.0 | 194326 | 2.9498 |
| 0.3415 | 122.0 | 195932 | 2.9371 |
| 0.3557 | 123.0 | 197538 | 2.9625 |
| 0.3544 | 124.0 | 199144 | 2.9637 |
| 0.3528 | 125.0 | 200750 | 2.9881 |
| 0.3567 | 126.0 | 202356 | 2.9576 |
| 0.3336 | 127.0 | 203962 | 2.9427 |
| 0.3282 | 128.0 | 205568 | 2.9659 |
| 0.3605 | 129.0 | 207174 | 2.9555 |
| 0.3436 | 130.0 | 208780 | 2.9590 |
| 0.3489 | 131.0 | 210386 | 2.9250 |
| 0.3604 | 132.0 | 211992 | 2.9411 |
| 0.347 | 133.0 | 213598 | 2.9093 |
| 0.3623 | 134.0 | 215204 | 2.9324 |
| 0.3449 | 135.0 | 216810 | 2.9564 |
| 0.3459 | 136.0 | 218416 | 2.9254 |
| 0.3519 | 137.0 | 220022 | 2.9512 |
| 0.3499 | 138.0 | 221628 | 2.9411 |
| 0.3588 | 139.0 | 223234 | 2.8994 |
| 0.3657 | 140.0 | 224840 | 2.9372 |
| 0.3564 | 141.0 | 226446 | 2.9237 |
| 0.3445 | 142.0 | 228052 | 2.9380 |
| 0.359 | 143.0 | 229658 | 2.9547 |
| 0.3495 | 144.0 | 231264 | 2.9238 |
| 0.3545 | 145.0 | 232870 | 2.9436 |
| 0.3523 | 146.0 | 234476 | 2.9390 |
| 0.3785 | 147.0 | 236082 | 2.8861 |
| 0.356 | 148.0 | 237688 | 2.9239 |
| 0.3624 | 149.0 | 239294 | 2.8960 |
| 0.3619 | 150.0 | 240900 | 2.9224 |
| 0.3607 | 151.0 | 242506 | 2.9155 |
| 0.3585 | 152.0 | 244112 | 2.9144 |
| 0.3735 | 153.0 | 245718 | 2.8805 |
| 0.3534 | 154.0 | 247324 | 2.9095 |
| 0.3667 | 155.0 | 248930 | 2.8888 |
| 0.3705 | 156.0 | 250536 | 2.9049 |
| 0.3711 | 157.0 | 252142 | 2.8801 |
| 0.3633 | 158.0 | 253748 | 2.8874 |
| 0.36 | 159.0 | 255354 | 2.8984 |
| 0.3752 | 160.0 | 256960 | 2.9004 |
| 0.3717 | 161.0 | 258566 | 2.8577 |
| 0.3742 | 162.0 | 260172 | 2.8772 |
| 0.3815 | 163.0 | 261778 | 2.9183 |
| 0.3695 | 164.0 | 263384 | 2.9144 |
| 0.3809 | 165.0 | 264990 | 2.8968 |
| 0.3813 | 166.0 | 266596 | 2.8690 |
| 0.3803 | 167.0 | 268202 | 2.8748 |
| 0.3813 | 168.0 | 269808 | 2.8676 |
| 0.3782 | 169.0 | 271414 | 2.8473 |
| 0.3848 | 170.0 | 273020 | 2.8816 |
| 0.371 | 171.0 | 274626 | 2.8929 |
| 0.3843 | 172.0 | 276232 | 2.8858 |
| 0.381 | 173.0 | 277838 | 2.8590 |
| 0.3889 | 174.0 | 279444 | 2.8484 |
| 0.3814 | 175.0 | 281050 | 2.8634 |
| 0.3865 | 176.0 | 282656 | 2.8713 |
| 0.3968 | 177.0 | 284262 | 2.8490 |
| 0.4007 | 178.0 | 285868 | 2.8497 |
| 0.3805 | 179.0 | 287474 | 2.8435 |
| 0.3903 | 180.0 | 289080 | 2.8582 |
| 0.392 | 181.0 | 290686 | 2.8473 |
| 0.3926 | 182.0 | 292292 | 2.8584 |
| 0.3921 | 183.0 | 293898 | 2.8850 |
| 0.3958 | 184.0 | 295504 | 2.8532 |
| 0.3858 | 185.0 | 297110 | 2.8568 |
| 0.4002 | 186.0 | 298716 | 2.7939 |
| 0.3999 | 187.0 | 300322 | 2.8548 |
| 0.3932 | 188.0 | 301928 | 2.8598 |
| 0.4005 | 189.0 | 303534 | 2.8390 |
| 0.4048 | 190.0 | 305140 | 2.8336 |
| 0.3983 | 191.0 | 306746 | 2.8286 |
| 0.394 | 192.0 | 308352 | 2.8437 |
| 0.3989 | 193.0 | 309958 | 2.8594 |
| 0.3966 | 194.0 | 311564 | 2.8541 |
| 0.397 | 195.0 | 313170 | 2.8697 |
| 0.4007 | 196.0 | 314776 | 2.8549 |
| 0.3978 | 197.0 | 316382 | 2.8815 |
| 0.4005 | 198.0 | 317988 | 2.8565 |
| 0.4025 | 199.0 | 319594 | 2.8451 |
| 0.4078 | 200.0 | 321200 | 2.8433 |
Framework versions
- Transformers 4.21.1
- Pytorch 1.12.1+cu113
- Datasets 2.4.0
- Tokenizers 0.12.1