distilbert-base-uncased-cohl
This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:
- Loss: 5.8197
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 200
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 7.6714 | 1.0 | 157 | 6.5491 |
| 6.4508 | 2.0 | 314 | 6.3591 |
| 6.3245 | 3.0 | 471 | 6.2702 |
| 6.2262 | 4.0 | 628 | 6.1747 |
| 6.1619 | 5.0 | 785 | 6.1199 |
| 6.1333 | 6.0 | 942 | 6.0925 |
| 6.1038 | 7.0 | 1099 | 6.0610 |
| 6.0825 | 8.0 | 1256 | 6.0783 |
| 6.0712 | 9.0 | 1413 | 6.0782 |
| 6.0594 | 10.0 | 1570 | 6.0546 |
| 6.0407 | 11.0 | 1727 | 6.0402 |
| 6.036 | 12.0 | 1884 | 6.0381 |
| 6.0332 | 13.0 | 2041 | 6.0056 |
| 6.0243 | 14.0 | 2198 | 6.0319 |
| 6.0156 | 15.0 | 2355 | 6.0127 |
| 6.0234 | 16.0 | 2512 | 6.0173 |
| 6.0071 | 17.0 | 2669 | 5.9917 |
| 6.0029 | 18.0 | 2826 | 5.9979 |
| 6.0012 | 19.0 | 2983 | 5.9878 |
| 5.9949 | 20.0 | 3140 | 5.9695 |
| 5.9894 | 21.0 | 3297 | 5.9852 |
| 5.9846 | 22.0 | 3454 | 5.9776 |
| 5.9766 | 23.0 | 3611 | 5.9655 |
| 5.9787 | 24.0 | 3768 | 5.9602 |
| 5.9717 | 25.0 | 3925 | 5.9889 |
| 5.9733 | 26.0 | 4082 | 5.9699 |
| 5.9655 | 27.0 | 4239 | 5.9611 |
| 5.9737 | 28.0 | 4396 | 5.9804 |
| 5.9605 | 29.0 | 4553 | 5.9618 |
| 5.9623 | 30.0 | 4710 | 5.9489 |
| 5.9588 | 31.0 | 4867 | 5.9630 |
| 5.9537 | 32.0 | 5024 | 5.9625 |
| 5.9536 | 33.0 | 5181 | 5.9692 |
| 5.9489 | 34.0 | 5338 | 5.9739 |
| 5.9424 | 35.0 | 5495 | 5.9553 |
| 5.945 | 36.0 | 5652 | 5.9464 |
| 5.9402 | 37.0 | 5809 | 5.9514 |
| 5.9376 | 38.0 | 5966 | 5.9398 |
| 5.9389 | 39.0 | 6123 | 5.9321 |
| 5.9274 | 40.0 | 6280 | 5.9638 |
| 5.9324 | 41.0 | 6437 | 5.9382 |
| 5.9275 | 42.0 | 6594 | 5.9396 |
| 5.9222 | 43.0 | 6751 | 5.9417 |
| 5.9282 | 44.0 | 6908 | 5.9344 |
| 5.9247 | 45.0 | 7065 | 5.9181 |
| 5.9167 | 46.0 | 7222 | 5.9462 |
| 5.9099 | 47.0 | 7379 | 5.9378 |
| 5.9126 | 48.0 | 7536 | 5.9052 |
| 5.9119 | 49.0 | 7693 | 5.9241 |
| 5.9116 | 50.0 | 7850 | 5.8920 |
| 5.9003 | 51.0 | 8007 | 5.9172 |
| 5.8978 | 52.0 | 8164 | 5.9379 |
| 5.8994 | 53.0 | 8321 | 5.9163 |
| 5.8973 | 54.0 | 8478 | 5.9284 |
| 5.8954 | 55.0 | 8635 | 5.9162 |
| 5.8959 | 56.0 | 8792 | 5.8985 |
| 5.8983 | 57.0 | 8949 | 5.9143 |
| 5.8878 | 58.0 | 9106 | 5.9355 |
| 5.8909 | 59.0 | 9263 | 5.9024 |
| 5.885 | 60.0 | 9420 | 5.9066 |
| 5.8861 | 61.0 | 9577 | 5.8989 |
| 5.8779 | 62.0 | 9734 | 5.9037 |
| 5.8849 | 63.0 | 9891 | 5.8944 |
| 5.8819 | 64.0 | 10048 | 5.9009 |
| 5.885 | 65.0 | 10205 | 5.9051 |
| 5.8747 | 66.0 | 10362 | 5.9144 |
| 5.8746 | 67.0 | 10519 | 5.9108 |
| 5.8682 | 68.0 | 10676 | 5.8830 |
| 5.8763 | 69.0 | 10833 | 5.9133 |
| 5.8664 | 70.0 | 10990 | 5.8987 |
| 5.8683 | 71.0 | 11147 | 5.8863 |
| 5.8675 | 72.0 | 11304 | 5.9088 |
| 5.8713 | 73.0 | 11461 | 5.8645 |
| 5.8584 | 74.0 | 11618 | 5.9043 |
| 5.8657 | 75.0 | 11775 | 5.8824 |
| 5.8648 | 76.0 | 11932 | 5.9092 |
| 5.8634 | 77.0 | 12089 | 5.9003 |
| 5.86 | 78.0 | 12246 | 5.8910 |
| 5.8629 | 79.0 | 12403 | 5.8885 |
| 5.8505 | 80.0 | 12560 | 5.8681 |
| 5.8608 | 81.0 | 12717 | 5.8960 |
| 5.8481 | 82.0 | 12874 | 5.9000 |
| 5.8495 | 83.0 | 13031 | 5.8935 |
| 5.8436 | 84.0 | 13188 | 5.8784 |
| 5.8493 | 85.0 | 13345 | 5.8821 |
| 5.8507 | 86.0 | 13502 | 5.8831 |
| 5.8472 | 87.0 | 13659 | 5.8779 |
| 5.8422 | 88.0 | 13816 | 5.8784 |
| 5.8412 | 89.0 | 13973 | 5.8630 |
| 5.8416 | 90.0 | 14130 | 5.8723 |
| 5.842 | 91.0 | 14287 | 5.8794 |
| 5.8375 | 92.0 | 14444 | 5.8611 |
| 5.8404 | 93.0 | 14601 | 5.8705 |
| 5.8451 | 94.0 | 14758 | 5.8883 |
| 5.8364 | 95.0 | 14915 | 5.8747 |
| 5.8365 | 96.0 | 15072 | 5.8885 |
| 5.8277 | 97.0 | 15229 | 5.8667 |
| 5.8255 | 98.0 | 15386 | 5.8603 |
| 5.8336 | 99.0 | 15543 | 5.8644 |
| 5.826 | 100.0 | 15700 | 5.8725 |
| 5.8223 | 101.0 | 15857 | 5.8714 |
| 5.8415 | 102.0 | 16014 | 5.8773 |
| 5.8286 | 103.0 | 16171 | 5.8704 |
| 5.8281 | 104.0 | 16328 | 5.8732 |
| 5.8246 | 105.0 | 16485 | 5.8582 |
| 5.8267 | 106.0 | 16642 | 5.8603 |
| 5.8176 | 107.0 | 16799 | 5.8751 |
| 5.8214 | 108.0 | 16956 | 5.8774 |
| 5.8115 | 109.0 | 17113 | 5.8826 |
| 5.8205 | 110.0 | 17270 | 5.8516 |
| 5.8136 | 111.0 | 17427 | 5.8743 |
| 5.8166 | 112.0 | 17584 | 5.8555 |
| 5.8171 | 113.0 | 17741 | 5.8695 |
| 5.8176 | 114.0 | 17898 | 5.8531 |
| 5.8108 | 115.0 | 18055 | 5.8570 |
| 5.808 | 116.0 | 18212 | 5.8552 |
| 5.8094 | 117.0 | 18369 | 5.8619 |
| 5.8108 | 118.0 | 18526 | 5.8665 |
| 5.8064 | 119.0 | 18683 | 5.8851 |
| 5.8099 | 120.0 | 18840 | 5.8507 |
| 5.8073 | 121.0 | 18997 | 5.8676 |
| 5.814 | 122.0 | 19154 | 5.8492 |
| 5.8093 | 123.0 | 19311 | 5.8506 |
| 5.8135 | 124.0 | 19468 | 5.8668 |
| 5.8031 | 125.0 | 19625 | 5.8617 |
| 5.801 | 126.0 | 19782 | 5.8626 |
| 5.8019 | 127.0 | 19939 | 5.8472 |
| 5.8106 | 128.0 | 20096 | 5.8429 |
| 5.8013 | 129.0 | 20253 | 5.8668 |
| 5.809 | 130.0 | 20410 | 5.8824 |
| 5.8 | 131.0 | 20567 | 5.8498 |
| 5.8006 | 132.0 | 20724 | 5.8757 |
| 5.8008 | 133.0 | 20881 | 5.8397 |
| 5.7908 | 134.0 | 21038 | 5.8569 |
| 5.7967 | 135.0 | 21195 | 5.8304 |
| 5.7908 | 136.0 | 21352 | 5.8265 |
| 5.7931 | 137.0 | 21509 | 5.8416 |
| 5.7896 | 138.0 | 21666 | 5.8368 |
| 5.7904 | 139.0 | 21823 | 5.8608 |
| 5.791 | 140.0 | 21980 | 5.8369 |
| 5.7887 | 141.0 | 22137 | 5.8705 |
| 5.7817 | 142.0 | 22294 | 5.8713 |
| 5.787 | 143.0 | 22451 | 5.8488 |
| 5.7913 | 144.0 | 22608 | 5.8516 |
| 5.7877 | 145.0 | 22765 | 5.8438 |
| 5.7905 | 146.0 | 22922 | 5.8595 |
| 5.7901 | 147.0 | 23079 | 5.8488 |
| 5.7906 | 148.0 | 23236 | 5.8460 |
| 5.7806 | 149.0 | 23393 | 5.8294 |
| 5.7912 | 150.0 | 23550 | 5.8776 |
| 5.7803 | 151.0 | 23707 | 5.8262 |
| 5.7821 | 152.0 | 23864 | 5.8729 |
| 5.7889 | 153.0 | 24021 | 5.8541 |
| 5.783 | 154.0 | 24178 | 5.8542 |
| 5.7901 | 155.0 | 24335 | 5.8449 |
| 5.7821 | 156.0 | 24492 | 5.8524 |
| 5.7868 | 157.0 | 24649 | 5.8675 |
| 5.7812 | 158.0 | 24806 | 5.8742 |
| 5.7821 | 159.0 | 24963 | 5.8496 |
| 5.7851 | 160.0 | 25120 | 5.8463 |
| 5.7787 | 161.0 | 25277 | 5.8573 |
| 5.7836 | 162.0 | 25434 | 5.8212 |
| 5.7786 | 163.0 | 25591 | 5.8683 |
| 5.7901 | 164.0 | 25748 | 5.8445 |
| 5.7764 | 165.0 | 25905 | 5.8253 |
| 5.7793 | 166.0 | 26062 | 5.8443 |
| 5.7709 | 167.0 | 26219 | 5.8254 |
| 5.7823 | 168.0 | 26376 | 5.8591 |
| 5.7753 | 169.0 | 26533 | 5.8154 |
| 5.7778 | 170.0 | 26690 | 5.8338 |
| 5.7785 | 171.0 | 26847 | 5.8596 |
| 5.7658 | 172.0 | 27004 | 5.8644 |
| 5.7719 | 173.0 | 27161 | 5.8282 |
| 5.781 | 174.0 | 27318 | 5.8451 |
| 5.7806 | 175.0 | 27475 | 5.8407 |
| 5.7798 | 176.0 | 27632 | 5.8622 |
| 5.7772 | 177.0 | 27789 | 5.8445 |
| 5.7686 | 178.0 | 27946 | 5.8529 |
| 5.7738 | 179.0 | 28103 | 5.8474 |
| 5.776 | 180.0 | 28260 | 5.8565 |
| 5.7685 | 181.0 | 28417 | 5.8253 |
| 5.7659 | 182.0 | 28574 | 5.8449 |
| 5.7684 | 183.0 | 28731 | 5.8497 |
| 5.7709 | 184.0 | 28888 | 5.8385 |
| 5.7631 | 185.0 | 29045 | 5.8131 |
| 5.7733 | 186.0 | 29202 | 5.8428 |
| 5.7736 | 187.0 | 29359 | 5.8388 |
| 5.7704 | 188.0 | 29516 | 5.8519 |
| 5.7719 | 189.0 | 29673 | 5.8454 |
| 5.7737 | 190.0 | 29830 | 5.8209 |
| 5.7667 | 191.0 | 29987 | 5.8681 |
| 5.7686 | 192.0 | 30144 | 5.8417 |
| 5.7754 | 193.0 | 30301 | 5.8566 |
| 5.7743 | 194.0 | 30458 | 5.8510 |
| 5.7739 | 195.0 | 30615 | 5.8308 |
| 5.7755 | 196.0 | 30772 | 5.8390 |
| 5.7702 | 197.0 | 30929 | 5.8320 |
| 5.767 | 198.0 | 31086 | 5.8447 |
| 5.7691 | 199.0 | 31243 | 5.8465 |
| 5.7753 | 200.0 | 31400 | 5.8197 |
Framework versions
- Transformers 4.30.2
- Pytorch 2.0.1+cu118
- Datasets 2.13.0
- Tokenizers 0.13.3
- Downloads last month
- 2