--- license: apache-2.0 tags: - generated_from_trainer model-index: - name: distilbert-base-uncased-cohl results: [] --- # distilbert-base-uncased-cohl This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the None dataset. It achieves the following results on the evaluation set: - Loss: 5.5505 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 128 - eval_batch_size: 128 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 200 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:-----:|:---------------:| | 7.4096 | 1.0 | 157 | 6.2368 | | 6.1384 | 2.0 | 314 | 6.0235 | | 5.9964 | 3.0 | 471 | 5.9395 | | 5.9145 | 4.0 | 628 | 5.8933 | | 5.8601 | 5.0 | 785 | 5.8380 | | 5.8461 | 6.0 | 942 | 5.7921 | | 5.82 | 7.0 | 1099 | 5.7787 | | 5.8076 | 8.0 | 1256 | 5.7794 | | 5.7927 | 9.0 | 1413 | 5.7946 | | 5.782 | 10.0 | 1570 | 5.7553 | | 5.7691 | 11.0 | 1727 | 5.7753 | | 5.7671 | 12.0 | 1884 | 5.7607 | | 5.7594 | 13.0 | 2041 | 5.7564 | | 5.7443 | 14.0 | 2198 | 5.7553 | | 5.7354 | 15.0 | 2355 | 5.7421 | | 5.7428 | 16.0 | 2512 | 5.7304 | | 5.7319 | 17.0 | 2669 | 5.7053 | | 5.7187 | 18.0 | 2826 | 5.7095 | | 5.7273 | 19.0 | 2983 | 5.7034 | | 5.7121 | 20.0 | 3140 | 5.6822 | | 5.7139 | 21.0 | 3297 | 5.7028 | | 5.7072 | 22.0 | 3454 | 5.7020 | | 5.695 | 23.0 | 3611 | 5.7085 | | 5.6921 | 24.0 | 3768 | 5.6935 | | 5.6964 | 25.0 | 3925 | 5.7071 | | 5.6771 | 26.0 | 4082 | 5.7016 | | 5.6911 | 27.0 | 4239 | 5.6765 | | 5.6874 | 28.0 | 4396 | 5.6937 | | 5.6788 | 29.0 | 4553 | 5.6744 | | 5.6709 | 30.0 | 4710 | 5.6593 | | 5.6743 | 31.0 | 4867 | 5.6719 | | 5.6623 | 32.0 | 5024 | 5.6422 | | 5.662 | 33.0 | 5181 | 5.6660 | | 5.6577 | 34.0 | 5338 | 5.6790 | | 5.6603 | 35.0 | 5495 | 5.6556 | | 5.6607 | 36.0 | 5652 | 5.6476 | | 5.6538 | 37.0 | 5809 | 5.6643 | | 5.6481 | 38.0 | 5966 | 5.6489 | | 5.6512 | 39.0 | 6123 | 5.6108 | | 5.642 | 40.0 | 6280 | 5.6647 | | 5.6475 | 41.0 | 6437 | 5.6633 | | 5.6419 | 42.0 | 6594 | 5.6256 | | 5.6364 | 43.0 | 6751 | 5.6524 | | 5.6391 | 44.0 | 6908 | 5.6424 | | 5.6307 | 45.0 | 7065 | 5.6384 | | 5.6249 | 46.0 | 7222 | 5.6451 | | 5.6242 | 47.0 | 7379 | 5.6413 | | 5.6259 | 48.0 | 7536 | 5.6230 | | 5.6223 | 49.0 | 7693 | 5.6285 | | 5.6245 | 50.0 | 7850 | 5.6107 | | 5.621 | 51.0 | 8007 | 5.6253 | | 5.6203 | 52.0 | 8164 | 5.6457 | | 5.6131 | 53.0 | 8321 | 5.6211 | | 5.6026 | 54.0 | 8478 | 5.6360 | | 5.6115 | 55.0 | 8635 | 5.6276 | | 5.6079 | 56.0 | 8792 | 5.6274 | | 5.6106 | 57.0 | 8949 | 5.6289 | | 5.6053 | 58.0 | 9106 | 5.6438 | | 5.6113 | 59.0 | 9263 | 5.6258 | | 5.5983 | 60.0 | 9420 | 5.6453 | | 5.6 | 61.0 | 9577 | 5.6351 | | 5.6007 | 62.0 | 9734 | 5.6327 | | 5.5989 | 63.0 | 9891 | 5.6102 | | 5.5974 | 64.0 | 10048 | 5.6280 | | 5.5987 | 65.0 | 10205 | 5.6299 | | 5.5903 | 66.0 | 10362 | 5.6106 | | 5.5915 | 67.0 | 10519 | 5.6149 | | 5.5928 | 68.0 | 10676 | 5.6048 | | 5.5876 | 69.0 | 10833 | 5.6279 | | 5.5886 | 70.0 | 10990 | 5.6073 | | 5.5859 | 71.0 | 11147 | 5.5987 | | 5.5881 | 72.0 | 11304 | 5.6208 | | 5.5805 | 73.0 | 11461 | 5.5869 | | 5.5808 | 74.0 | 11618 | 5.6169 | | 5.5813 | 75.0 | 11775 | 5.6019 | | 5.5881 | 76.0 | 11932 | 5.6213 | | 5.5823 | 77.0 | 12089 | 5.5931 | | 5.5735 | 78.0 | 12246 | 5.5948 | | 5.5788 | 79.0 | 12403 | 5.5878 | | 5.5735 | 80.0 | 12560 | 5.5784 | | 5.5701 | 81.0 | 12717 | 5.6084 | | 5.5757 | 82.0 | 12874 | 5.5957 | | 5.5697 | 83.0 | 13031 | 5.5931 | | 5.573 | 84.0 | 13188 | 5.5862 | | 5.5652 | 85.0 | 13345 | 5.6049 | | 5.5635 | 86.0 | 13502 | 5.5959 | | 5.5634 | 87.0 | 13659 | 5.5865 | | 5.5644 | 88.0 | 13816 | 5.6000 | | 5.5662 | 89.0 | 13973 | 5.5971 | | 5.5563 | 90.0 | 14130 | 5.5711 | | 5.5612 | 91.0 | 14287 | 5.6007 | | 5.5626 | 92.0 | 14444 | 5.5824 | | 5.5543 | 93.0 | 14601 | 5.5966 | | 5.5627 | 94.0 | 14758 | 5.5828 | | 5.5633 | 95.0 | 14915 | 5.6066 | | 5.5526 | 96.0 | 15072 | 5.5979 | | 5.5529 | 97.0 | 15229 | 5.5756 | | 5.5527 | 98.0 | 15386 | 5.5633 | | 5.5568 | 99.0 | 15543 | 5.5775 | | 5.5419 | 100.0 | 15700 | 5.5899 | | 5.5436 | 101.0 | 15857 | 5.5657 | | 5.5509 | 102.0 | 16014 | 5.5824 | | 5.5468 | 103.0 | 16171 | 5.5936 | | 5.5447 | 104.0 | 16328 | 5.5666 | | 5.5469 | 105.0 | 16485 | 5.5747 | | 5.5436 | 106.0 | 16642 | 5.5658 | | 5.537 | 107.0 | 16799 | 5.5873 | | 5.5356 | 108.0 | 16956 | 5.5981 | | 5.5355 | 109.0 | 17113 | 5.5884 | | 5.539 | 110.0 | 17270 | 5.5713 | | 5.5413 | 111.0 | 17427 | 5.5951 | | 5.5353 | 112.0 | 17584 | 5.5817 | | 5.5275 | 113.0 | 17741 | 5.5981 | | 5.5422 | 114.0 | 17898 | 5.5744 | | 5.5298 | 115.0 | 18055 | 5.5637 | | 5.5335 | 116.0 | 18212 | 5.5918 | | 5.5305 | 117.0 | 18369 | 5.5717 | | 5.5257 | 118.0 | 18526 | 5.5681 | | 5.5313 | 119.0 | 18683 | 5.5984 | | 5.5286 | 120.0 | 18840 | 5.5799 | | 5.5217 | 121.0 | 18997 | 5.5746 | | 5.5309 | 122.0 | 19154 | 5.5429 | | 5.5288 | 123.0 | 19311 | 5.5787 | | 5.5258 | 124.0 | 19468 | 5.5942 | | 5.5185 | 125.0 | 19625 | 5.5922 | | 5.5232 | 126.0 | 19782 | 5.5587 | | 5.5227 | 127.0 | 19939 | 5.5575 | | 5.5356 | 128.0 | 20096 | 5.5800 | | 5.5226 | 129.0 | 20253 | 5.5780 | | 5.5243 | 130.0 | 20410 | 5.5717 | | 5.5154 | 131.0 | 20567 | 5.5644 | | 5.5216 | 132.0 | 20724 | 5.5741 | | 5.5212 | 133.0 | 20881 | 5.5778 | | 5.5154 | 134.0 | 21038 | 5.5588 | | 5.5124 | 135.0 | 21195 | 5.5647 | | 5.5164 | 136.0 | 21352 | 5.5449 | | 5.5176 | 137.0 | 21509 | 5.5625 | | 5.5078 | 138.0 | 21666 | 5.5803 | | 5.5137 | 139.0 | 21823 | 5.5805 | | 5.5154 | 140.0 | 21980 | 5.5494 | | 5.5188 | 141.0 | 22137 | 5.5791 | | 5.5032 | 142.0 | 22294 | 5.5724 | | 5.509 | 143.0 | 22451 | 5.5921 | | 5.5112 | 144.0 | 22608 | 5.5688 | | 5.5041 | 145.0 | 22765 | 5.5619 | | 5.5103 | 146.0 | 22922 | 5.5735 | | 5.5112 | 147.0 | 23079 | 5.5763 | | 5.5085 | 148.0 | 23236 | 5.5748 | | 5.506 | 149.0 | 23393 | 5.5738 | | 5.5118 | 150.0 | 23550 | 5.5718 | | 5.5014 | 151.0 | 23707 | 5.5619 | | 5.5087 | 152.0 | 23864 | 5.5810 | | 5.51 | 153.0 | 24021 | 5.5804 | | 5.5028 | 154.0 | 24178 | 5.5870 | | 5.5157 | 155.0 | 24335 | 5.5536 | | 5.5043 | 156.0 | 24492 | 5.5856 | | 5.5083 | 157.0 | 24649 | 5.5663 | | 5.5014 | 158.0 | 24806 | 5.5883 | | 5.4994 | 159.0 | 24963 | 5.5754 | | 5.5025 | 160.0 | 25120 | 5.5567 | | 5.4998 | 161.0 | 25277 | 5.5729 | | 5.5009 | 162.0 | 25434 | 5.5422 | | 5.5063 | 163.0 | 25591 | 5.5731 | | 5.5093 | 164.0 | 25748 | 5.5734 | | 5.5011 | 165.0 | 25905 | 5.5617 | | 5.5011 | 166.0 | 26062 | 5.5586 | | 5.5017 | 167.0 | 26219 | 5.5483 | | 5.5001 | 168.0 | 26376 | 5.5617 | | 5.4964 | 169.0 | 26533 | 5.5477 | | 5.5014 | 170.0 | 26690 | 5.5646 | | 5.4981 | 171.0 | 26847 | 5.5723 | | 5.4902 | 172.0 | 27004 | 5.5530 | | 5.4957 | 173.0 | 27161 | 5.5614 | | 5.4988 | 174.0 | 27318 | 5.5699 | | 5.5005 | 175.0 | 27475 | 5.5637 | | 5.5005 | 176.0 | 27632 | 5.5769 | | 5.4973 | 177.0 | 27789 | 5.5624 | | 5.4927 | 178.0 | 27946 | 5.5736 | | 5.4962 | 179.0 | 28103 | 5.5639 | | 5.4908 | 180.0 | 28260 | 5.5541 | | 5.4909 | 181.0 | 28417 | 5.5598 | | 5.4885 | 182.0 | 28574 | 5.5642 | | 5.4902 | 183.0 | 28731 | 5.5590 | | 5.4949 | 184.0 | 28888 | 5.5707 | | 5.4935 | 185.0 | 29045 | 5.5597 | | 5.4914 | 186.0 | 29202 | 5.5823 | | 5.4914 | 187.0 | 29359 | 5.5597 | | 5.4874 | 188.0 | 29516 | 5.5595 | | 5.4934 | 189.0 | 29673 | 5.5685 | | 5.4956 | 190.0 | 29830 | 5.5578 | | 5.4902 | 191.0 | 29987 | 5.5762 | | 5.4881 | 192.0 | 30144 | 5.5697 | | 5.4934 | 193.0 | 30301 | 5.5631 | | 5.4974 | 194.0 | 30458 | 5.5730 | | 5.4939 | 195.0 | 30615 | 5.5614 | | 5.4952 | 196.0 | 30772 | 5.5492 | | 5.4892 | 197.0 | 30929 | 5.5613 | | 5.49 | 198.0 | 31086 | 5.5737 | | 5.4914 | 199.0 | 31243 | 5.5806 | | 5.4954 | 200.0 | 31400 | 5.5505 | ### Framework versions - Transformers 4.30.2 - Pytorch 2.0.1+cu118 - Datasets 2.13.0 - Tokenizers 0.13.3