File size: 16,101 Bytes
49052b6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 |
---
license: mit
tags:
- generated_from_trainer
model-index:
- name: label-transfer
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# label-transfer
This model is a fine-tuned version of [saattrupdan/verdict-classifier](https://huggingface.co/saattrupdan/verdict-classifier) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0452
- F1 Macro: 0.9872
- F1 Misinformation: 0.9918
- F1 Factual: 0.9979
- F1 Other: 0.9720
- Prec Macro: 0.9842
- Prec Misinformation: 0.9958
- Prec Factual: 0.9979
- Prec Other: 0.9588
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- gradient_accumulation_steps: 32
- total_train_batch_size: 2048
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1423
- num_epochs: 1000
### Training results
| Training Loss | Epoch | Step | Validation Loss | F1 Macro | F1 Misinformation | F1 Factual | F1 Other | Prec Macro | Prec Misinformation | Prec Factual | Prec Other |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:-----------------:|:----------:|:--------:|:----------:|:-------------------:|:------------:|:----------:|
| 0.4236 | 0.9 | 5 | 0.4070 | 0.8866 | 0.9477 | 0.9658 | 0.7463 | 0.9306 | 0.9075 | 0.9766 | 0.9077 |
| 0.4175 | 1.9 | 10 | 0.4001 | 0.8872 | 0.9480 | 0.9658 | 0.7477 | 0.9308 | 0.9079 | 0.9766 | 0.9080 |
| 0.4115 | 2.9 | 15 | 0.3884 | 0.8896 | 0.9487 | 0.9668 | 0.7534 | 0.9323 | 0.9093 | 0.9787 | 0.9090 |
| 0.3932 | 3.9 | 20 | 0.3719 | 0.8943 | 0.9509 | 0.9668 | 0.7652 | 0.9343 | 0.9133 | 0.9787 | 0.9110 |
| 0.3785 | 4.9 | 25 | 0.3505 | 0.8973 | 0.9522 | 0.9668 | 0.7730 | 0.9353 | 0.9160 | 0.9787 | 0.9112 |
| 0.3653 | 5.9 | 30 | 0.3266 | 0.9009 | 0.9535 | 0.9683 | 0.7809 | 0.9369 | 0.9186 | 0.9818 | 0.9104 |
| 0.3337 | 6.9 | 35 | 0.3028 | 0.9143 | 0.9599 | 0.9694 | 0.8137 | 0.9425 | 0.9310 | 0.9818 | 0.9148 |
| 0.3181 | 7.9 | 40 | 0.2796 | 0.9181 | 0.9624 | 0.9673 | 0.8245 | 0.9431 | 0.9361 | 0.9807 | 0.9125 |
| 0.2976 | 8.9 | 45 | 0.2570 | 0.9199 | 0.9633 | 0.9673 | 0.8291 | 0.9434 | 0.9383 | 0.9807 | 0.9113 |
| 0.2845 | 9.9 | 50 | 0.2349 | 0.9242 | 0.9658 | 0.9668 | 0.8401 | 0.9453 | 0.9430 | 0.9797 | 0.9131 |
| 0.2649 | 10.9 | 55 | 0.2134 | 0.9270 | 0.9673 | 0.9668 | 0.8470 | 0.9451 | 0.9472 | 0.9797 | 0.9086 |
| 0.2399 | 11.9 | 60 | 0.1929 | 0.9330 | 0.9704 | 0.9668 | 0.8619 | 0.9467 | 0.9547 | 0.9797 | 0.9057 |
| 0.224 | 12.9 | 65 | 0.1735 | 0.9369 | 0.9724 | 0.9673 | 0.8710 | 0.9467 | 0.9608 | 0.9797 | 0.8996 |
| 0.1992 | 13.9 | 70 | 0.1564 | 0.9496 | 0.9783 | 0.9711 | 0.8995 | 0.9531 | 0.9744 | 0.9809 | 0.9039 |
| 0.1908 | 14.9 | 75 | 0.1427 | 0.9501 | 0.9784 | 0.9711 | 0.9006 | 0.9519 | 0.9765 | 0.9799 | 0.8993 |
| 0.1785 | 15.9 | 80 | 0.1309 | 0.9542 | 0.9790 | 0.9765 | 0.9072 | 0.9549 | 0.9782 | 0.9791 | 0.9076 |
| 0.1637 | 16.9 | 85 | 0.1215 | 0.9531 | 0.9791 | 0.9745 | 0.9056 | 0.9536 | 0.9784 | 0.9750 | 0.9073 |
| 0.151 | 17.9 | 90 | 0.1131 | 0.9540 | 0.9787 | 0.9771 | 0.9064 | 0.9549 | 0.9776 | 0.9771 | 0.9099 |
| 0.1395 | 18.9 | 95 | 0.1049 | 0.9555 | 0.9790 | 0.9787 | 0.9088 | 0.9558 | 0.9784 | 0.9772 | 0.9119 |
| 0.1285 | 19.9 | 100 | 0.0963 | 0.9600 | 0.9799 | 0.9833 | 0.9169 | 0.9602 | 0.9798 | 0.9843 | 0.9164 |
| 0.1228 | 20.9 | 105 | 0.0887 | 0.9654 | 0.9829 | 0.9844 | 0.9289 | 0.9639 | 0.9850 | 0.9854 | 0.9215 |
| 0.1163 | 21.9 | 110 | 0.0832 | 0.9672 | 0.9839 | 0.9849 | 0.9329 | 0.9655 | 0.9864 | 0.9864 | 0.9237 |
| 0.1045 | 22.9 | 115 | 0.0792 | 0.9690 | 0.9849 | 0.9849 | 0.9374 | 0.9666 | 0.9883 | 0.9864 | 0.9251 |
| 0.0975 | 23.9 | 120 | 0.0758 | 0.9701 | 0.9854 | 0.9854 | 0.9396 | 0.9682 | 0.9880 | 0.9864 | 0.9303 |
| 0.0957 | 24.9 | 125 | 0.0731 | 0.9710 | 0.9856 | 0.9864 | 0.9411 | 0.9691 | 0.9883 | 0.9885 | 0.9305 |
| 0.0911 | 25.9 | 130 | 0.0702 | 0.9743 | 0.9862 | 0.9901 | 0.9467 | 0.9722 | 0.9891 | 0.9896 | 0.9377 |
| 0.0884 | 26.9 | 135 | 0.0676 | 0.9759 | 0.9875 | 0.9901 | 0.9502 | 0.9728 | 0.9916 | 0.9886 | 0.9381 |
| 0.087 | 27.9 | 140 | 0.0652 | 0.9770 | 0.9878 | 0.9912 | 0.9521 | 0.9739 | 0.9919 | 0.9906 | 0.9392 |
| 0.0813 | 28.9 | 145 | 0.0631 | 0.9791 | 0.9880 | 0.9938 | 0.9555 | 0.9758 | 0.9925 | 0.9938 | 0.9412 |
| 0.0758 | 29.9 | 150 | 0.0612 | 0.9805 | 0.9887 | 0.9943 | 0.9584 | 0.9767 | 0.9938 | 0.9938 | 0.9424 |
| 0.0734 | 30.9 | 155 | 0.0598 | 0.9796 | 0.9882 | 0.9943 | 0.9564 | 0.9762 | 0.9927 | 0.9938 | 0.9422 |
| 0.0713 | 31.9 | 160 | 0.0586 | 0.9798 | 0.9883 | 0.9943 | 0.9569 | 0.9765 | 0.9927 | 0.9938 | 0.9430 |
| 0.0662 | 32.9 | 165 | 0.0568 | 0.9805 | 0.9887 | 0.9943 | 0.9584 | 0.9768 | 0.9936 | 0.9938 | 0.9432 |
| 0.063 | 33.9 | 170 | 0.0552 | 0.9813 | 0.9893 | 0.9943 | 0.9602 | 0.9778 | 0.9938 | 0.9938 | 0.9459 |
| 0.0623 | 34.9 | 175 | 0.0538 | 0.9819 | 0.9897 | 0.9943 | 0.9616 | 0.9785 | 0.9941 | 0.9938 | 0.9477 |
| 0.0601 | 35.9 | 180 | 0.0531 | 0.9828 | 0.9901 | 0.9948 | 0.9635 | 0.9793 | 0.9947 | 0.9938 | 0.9496 |
| 0.0549 | 36.9 | 185 | 0.0521 | 0.9826 | 0.9900 | 0.9948 | 0.9631 | 0.9790 | 0.9947 | 0.9938 | 0.9487 |
| 0.0539 | 37.9 | 190 | 0.0512 | 0.9824 | 0.9898 | 0.9948 | 0.9626 | 0.9789 | 0.9944 | 0.9938 | 0.9486 |
| 0.0525 | 38.9 | 195 | 0.0503 | 0.9827 | 0.9898 | 0.9953 | 0.9630 | 0.9792 | 0.9944 | 0.9938 | 0.9495 |
| 0.0494 | 39.9 | 200 | 0.0498 | 0.9831 | 0.9898 | 0.9958 | 0.9635 | 0.9796 | 0.9944 | 0.9948 | 0.9496 |
| 0.0502 | 40.9 | 205 | 0.0489 | 0.9838 | 0.9901 | 0.9964 | 0.9650 | 0.9804 | 0.9947 | 0.9958 | 0.9506 |
| 0.0499 | 41.9 | 210 | 0.0483 | 0.9845 | 0.9904 | 0.9969 | 0.9663 | 0.9813 | 0.9947 | 0.9958 | 0.9532 |
| 0.0484 | 42.9 | 215 | 0.0480 | 0.9847 | 0.9905 | 0.9969 | 0.9668 | 0.9814 | 0.9950 | 0.9958 | 0.9533 |
| 0.0465 | 43.9 | 220 | 0.0477 | 0.9852 | 0.9908 | 0.9969 | 0.9678 | 0.9816 | 0.9955 | 0.9958 | 0.9534 |
| 0.0453 | 44.9 | 225 | 0.0474 | 0.9856 | 0.9911 | 0.9969 | 0.9687 | 0.9822 | 0.9955 | 0.9958 | 0.9551 |
| 0.0452 | 45.9 | 230 | 0.0471 | 0.9856 | 0.9911 | 0.9969 | 0.9687 | 0.9822 | 0.9955 | 0.9958 | 0.9551 |
| 0.0453 | 46.9 | 235 | 0.0469 | 0.9854 | 0.9910 | 0.9969 | 0.9682 | 0.9821 | 0.9953 | 0.9958 | 0.9551 |
| 0.043 | 47.9 | 240 | 0.0468 | 0.9858 | 0.9912 | 0.9969 | 0.9692 | 0.9825 | 0.9955 | 0.9958 | 0.9560 |
| 0.0428 | 48.9 | 245 | 0.0465 | 0.9856 | 0.9911 | 0.9969 | 0.9687 | 0.9824 | 0.9953 | 0.9958 | 0.9560 |
| 0.0414 | 49.9 | 250 | 0.0465 | 0.9852 | 0.9911 | 0.9964 | 0.9682 | 0.9820 | 0.9953 | 0.9948 | 0.9560 |
| 0.0388 | 50.9 | 255 | 0.0462 | 0.9852 | 0.9911 | 0.9964 | 0.9682 | 0.9820 | 0.9953 | 0.9948 | 0.9560 |
| 0.0404 | 51.9 | 260 | 0.0458 | 0.9852 | 0.9911 | 0.9964 | 0.9682 | 0.9820 | 0.9953 | 0.9948 | 0.9560 |
| 0.0382 | 52.9 | 265 | 0.0454 | 0.9856 | 0.9911 | 0.9969 | 0.9687 | 0.9824 | 0.9953 | 0.9958 | 0.9560 |
| 0.042 | 53.9 | 270 | 0.0443 | 0.9862 | 0.9911 | 0.9979 | 0.9697 | 0.9831 | 0.9953 | 0.9979 | 0.9561 |
| 0.0369 | 54.9 | 275 | 0.0438 | 0.9862 | 0.9911 | 0.9979 | 0.9697 | 0.9831 | 0.9953 | 0.9979 | 0.9561 |
| 0.0383 | 55.9 | 280 | 0.0437 | 0.9862 | 0.9911 | 0.9979 | 0.9697 | 0.9831 | 0.9953 | 0.9979 | 0.9561 |
| 0.0373 | 56.9 | 285 | 0.0438 | 0.9862 | 0.9911 | 0.9979 | 0.9696 | 0.9833 | 0.9950 | 0.9979 | 0.9569 |
| 0.0402 | 57.9 | 290 | 0.0440 | 0.9862 | 0.9911 | 0.9979 | 0.9696 | 0.9833 | 0.9950 | 0.9979 | 0.9569 |
| 0.0389 | 58.9 | 295 | 0.0443 | 0.9858 | 0.9908 | 0.9979 | 0.9687 | 0.9831 | 0.9944 | 0.9979 | 0.9568 |
| 0.0361 | 59.9 | 300 | 0.0443 | 0.9860 | 0.9910 | 0.9979 | 0.9692 | 0.9832 | 0.9947 | 0.9979 | 0.9569 |
| 0.0369 | 60.9 | 305 | 0.0442 | 0.9860 | 0.9910 | 0.9979 | 0.9692 | 0.9832 | 0.9947 | 0.9979 | 0.9569 |
| 0.0353 | 61.9 | 310 | 0.0442 | 0.9862 | 0.9911 | 0.9979 | 0.9696 | 0.9833 | 0.9950 | 0.9979 | 0.9569 |
| 0.035 | 62.9 | 315 | 0.0446 | 0.9860 | 0.9910 | 0.9979 | 0.9692 | 0.9832 | 0.9947 | 0.9979 | 0.9569 |
| 0.0352 | 63.9 | 320 | 0.0449 | 0.9864 | 0.9912 | 0.9979 | 0.9701 | 0.9834 | 0.9953 | 0.9979 | 0.9570 |
| 0.0336 | 64.9 | 325 | 0.0451 | 0.9860 | 0.9910 | 0.9979 | 0.9692 | 0.9832 | 0.9947 | 0.9979 | 0.9569 |
| 0.0317 | 65.9 | 330 | 0.0448 | 0.9860 | 0.9910 | 0.9979 | 0.9692 | 0.9832 | 0.9947 | 0.9979 | 0.9569 |
| 0.0334 | 66.9 | 335 | 0.0447 | 0.9866 | 0.9914 | 0.9979 | 0.9705 | 0.9843 | 0.9944 | 0.9979 | 0.9605 |
| 0.0316 | 67.9 | 340 | 0.0447 | 0.9860 | 0.9910 | 0.9979 | 0.9691 | 0.9834 | 0.9944 | 0.9979 | 0.9577 |
| 0.0329 | 68.9 | 345 | 0.0451 | 0.9866 | 0.9914 | 0.9979 | 0.9706 | 0.9835 | 0.9955 | 0.9979 | 0.9570 |
| 0.0326 | 69.9 | 350 | 0.0454 | 0.9866 | 0.9914 | 0.9979 | 0.9706 | 0.9835 | 0.9955 | 0.9979 | 0.9570 |
| 0.032 | 70.9 | 355 | 0.0453 | 0.9868 | 0.9915 | 0.9979 | 0.9711 | 0.9838 | 0.9955 | 0.9979 | 0.9579 |
| 0.0325 | 71.9 | 360 | 0.0450 | 0.9864 | 0.9912 | 0.9979 | 0.9701 | 0.9836 | 0.9950 | 0.9979 | 0.9578 |
| 0.0319 | 72.9 | 365 | 0.0446 | 0.9868 | 0.9915 | 0.9979 | 0.9711 | 0.9838 | 0.9955 | 0.9979 | 0.9579 |
| 0.0326 | 73.9 | 370 | 0.0444 | 0.9868 | 0.9915 | 0.9979 | 0.9711 | 0.9838 | 0.9955 | 0.9979 | 0.9579 |
| 0.0315 | 74.9 | 375 | 0.0442 | 0.9873 | 0.9918 | 0.9979 | 0.9721 | 0.9840 | 0.9961 | 0.9979 | 0.9580 |
| 0.0304 | 75.9 | 380 | 0.0442 | 0.9866 | 0.9914 | 0.9979 | 0.9706 | 0.9837 | 0.9953 | 0.9979 | 0.9579 |
| 0.03 | 76.9 | 385 | 0.0444 | 0.9864 | 0.9912 | 0.9979 | 0.9702 | 0.9832 | 0.9955 | 0.9979 | 0.9561 |
| 0.0296 | 77.9 | 390 | 0.0448 | 0.9862 | 0.9911 | 0.9979 | 0.9697 | 0.9831 | 0.9953 | 0.9979 | 0.9561 |
| 0.0307 | 78.9 | 395 | 0.0452 | 0.9866 | 0.9914 | 0.9979 | 0.9706 | 0.9837 | 0.9953 | 0.9979 | 0.9579 |
| 0.0296 | 79.9 | 400 | 0.0453 | 0.9862 | 0.9911 | 0.9979 | 0.9697 | 0.9831 | 0.9953 | 0.9979 | 0.9561 |
| 0.0292 | 80.9 | 405 | 0.0454 | 0.9862 | 0.9911 | 0.9979 | 0.9697 | 0.9831 | 0.9953 | 0.9979 | 0.9561 |
| 0.0293 | 81.9 | 410 | 0.0452 | 0.9862 | 0.9911 | 0.9979 | 0.9697 | 0.9829 | 0.9955 | 0.9979 | 0.9552 |
| 0.0292 | 82.9 | 415 | 0.0454 | 0.9862 | 0.9911 | 0.9979 | 0.9697 | 0.9829 | 0.9955 | 0.9979 | 0.9552 |
| 0.0281 | 83.9 | 420 | 0.0454 | 0.9866 | 0.9914 | 0.9979 | 0.9706 | 0.9833 | 0.9958 | 0.9979 | 0.9562 |
| 0.0298 | 84.9 | 425 | 0.0452 | 0.9872 | 0.9918 | 0.9979 | 0.9720 | 0.9842 | 0.9958 | 0.9979 | 0.9588 |
### Framework versions
- Transformers 4.23.1
- Pytorch 1.12.1+cu113
- Datasets 2.6.1
- Tokenizers 0.13.1
|