Edit model card

chemical-bert-uncased-finetuned-cust-c1-cust

This model is a fine-tuned version of shafin/chemical-bert-uncased-finetuned-cust on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5420

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 64
  • eval_batch_size: 64
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 200
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
1.96 1.0 63 1.6719
1.7095 2.0 126 1.5305
1.5634 3.0 189 1.2972
1.4785 4.0 252 1.3354
1.3991 5.0 315 1.2542
1.3482 6.0 378 1.1870
1.2984 7.0 441 1.1844
1.2589 8.0 504 1.1262
1.1762 9.0 567 1.1176
1.1724 10.0 630 1.0312
1.1222 11.0 693 1.0113
1.1021 12.0 756 1.0518
1.0646 13.0 819 1.0433
1.0273 14.0 882 0.9634
1.0187 15.0 945 0.9299
0.9854 16.0 1008 0.9458
0.9799 17.0 1071 0.9733
0.95 18.0 1134 0.9169
0.934 19.0 1197 0.9246
0.907 20.0 1260 0.8939
0.8974 21.0 1323 0.8575
0.8749 22.0 1386 0.8513
0.8526 23.0 1449 0.8089
0.8359 24.0 1512 0.8600
0.8292 25.0 1575 0.8517
0.8263 26.0 1638 0.8293
0.8033 27.0 1701 0.7747
0.7999 28.0 1764 0.8169
0.7778 29.0 1827 0.7981
0.7574 30.0 1890 0.7457
0.7581 31.0 1953 0.7504
0.7404 32.0 2016 0.7637
0.7332 33.0 2079 0.7902
0.7314 34.0 2142 0.7185
0.7209 35.0 2205 0.7534
0.6902 36.0 2268 0.7334
0.6973 37.0 2331 0.7069
0.687 38.0 2394 0.6820
0.6658 39.0 2457 0.7155
0.6697 40.0 2520 0.7149
0.6584 41.0 2583 0.7413
0.6638 42.0 2646 0.7245
0.6282 43.0 2709 0.7177
0.6418 44.0 2772 0.6653
0.6323 45.0 2835 0.7715
0.6256 46.0 2898 0.7269
0.6109 47.0 2961 0.6744
0.6133 48.0 3024 0.6816
0.595 49.0 3087 0.6969
0.6058 50.0 3150 0.6965
0.5961 51.0 3213 0.6988
0.587 52.0 3276 0.6727
0.5861 53.0 3339 0.6327
0.5758 54.0 3402 0.6538
0.5692 55.0 3465 0.6612
0.567 56.0 3528 0.5989
0.5514 57.0 3591 0.6776
0.5526 58.0 3654 0.6440
0.556 59.0 3717 0.6682
0.5476 60.0 3780 0.6254
0.536 61.0 3843 0.6239
0.526 62.0 3906 0.6606
0.532 63.0 3969 0.6565
0.5189 64.0 4032 0.6586
0.5075 65.0 4095 0.6286
0.5131 66.0 4158 0.6646
0.498 67.0 4221 0.6486
0.4979 68.0 4284 0.6313
0.4885 69.0 4347 0.6419
0.4875 70.0 4410 0.6313
0.4904 71.0 4473 0.6602
0.4712 72.0 4536 0.6200
0.4798 73.0 4599 0.5912
0.4802 74.0 4662 0.6001
0.4704 75.0 4725 0.6303
0.4709 76.0 4788 0.5871
0.465 77.0 4851 0.6344
0.4651 78.0 4914 0.6030
0.4501 79.0 4977 0.5998
0.4584 80.0 5040 0.5926
0.4651 81.0 5103 0.6134
0.438 82.0 5166 0.6254
0.448 83.0 5229 0.6260
0.4295 84.0 5292 0.5866
0.434 85.0 5355 0.5740
0.4261 86.0 5418 0.5691
0.4312 87.0 5481 0.6243
0.4289 88.0 5544 0.5781
0.4255 89.0 5607 0.6226
0.4254 90.0 5670 0.5538
0.4231 91.0 5733 0.5874
0.4107 92.0 5796 0.6054
0.4082 93.0 5859 0.5898
0.4144 94.0 5922 0.5826
0.4225 95.0 5985 0.5501
0.3964 96.0 6048 0.5886
0.3972 97.0 6111 0.5831
0.4165 98.0 6174 0.5164
0.4024 99.0 6237 0.5714
0.4013 100.0 6300 0.5734
0.3933 101.0 6363 0.5727
0.3821 102.0 6426 0.5985
0.3904 103.0 6489 0.5571
0.3965 104.0 6552 0.5837
0.3789 105.0 6615 0.5989
0.3733 106.0 6678 0.5405
0.3907 107.0 6741 0.6059
0.3794 108.0 6804 0.5602
0.3689 109.0 6867 0.5590
0.3603 110.0 6930 0.5886
0.3747 111.0 6993 0.5294
0.3667 112.0 7056 0.5759
0.3754 113.0 7119 0.5821
0.3676 114.0 7182 0.5653
0.3524 115.0 7245 0.5537
0.3624 116.0 7308 0.5523
0.3527 117.0 7371 0.5799
0.3588 118.0 7434 0.6346
0.3539 119.0 7497 0.5116
0.3553 120.0 7560 0.5716
0.3483 121.0 7623 0.5721
0.3625 122.0 7686 0.5393
0.3354 123.0 7749 0.5800
0.3392 124.0 7812 0.5389
0.344 125.0 7875 0.5455
0.3451 126.0 7938 0.5428
0.3374 127.0 8001 0.5580
0.3428 128.0 8064 0.5339
0.3386 129.0 8127 0.5447
0.3318 130.0 8190 0.5738
0.3388 131.0 8253 0.5667
0.3335 132.0 8316 0.5407
0.3383 133.0 8379 0.5679
0.3299 134.0 8442 0.5846
0.327 135.0 8505 0.5511
0.3354 136.0 8568 0.5649
0.32 137.0 8631 0.5358
0.3265 138.0 8694 0.5528
0.319 139.0 8757 0.5926
0.3304 140.0 8820 0.5531
0.3191 141.0 8883 0.5379
0.3298 142.0 8946 0.5468
0.3134 143.0 9009 0.5623
0.3186 144.0 9072 0.5162
0.3179 145.0 9135 0.5570
0.3175 146.0 9198 0.5379
0.3051 147.0 9261 0.5437
0.312 148.0 9324 0.5301
0.3093 149.0 9387 0.5393
0.3227 150.0 9450 0.5531
0.3125 151.0 9513 0.5794
0.3162 152.0 9576 0.5677
0.3006 153.0 9639 0.5668
0.3011 154.0 9702 0.5797
0.3208 155.0 9765 0.5450
0.3048 156.0 9828 0.5465
0.3092 157.0 9891 0.5358
0.3125 158.0 9954 0.5043
0.3083 159.0 10017 0.5321
0.3 160.0 10080 0.5526
0.2968 161.0 10143 0.5324
0.3068 162.0 10206 0.5471
0.3129 163.0 10269 0.5575
0.3061 164.0 10332 0.5796
0.2943 165.0 10395 0.5544
0.2967 166.0 10458 0.5422
0.2959 167.0 10521 0.5149
0.2987 168.0 10584 0.5685
0.3045 169.0 10647 0.5176
0.2975 170.0 10710 0.5044
0.2948 171.0 10773 0.5264
0.3 172.0 10836 0.5174
0.2967 173.0 10899 0.5658
0.2873 174.0 10962 0.4988
0.2939 175.0 11025 0.5512
0.2954 176.0 11088 0.5139
0.301 177.0 11151 0.6007
0.2948 178.0 11214 0.5167
0.2898 179.0 11277 0.5443
0.2869 180.0 11340 0.5544
0.2973 181.0 11403 0.5644
0.2985 182.0 11466 0.5153
0.2904 183.0 11529 0.5561
0.2872 184.0 11592 0.5610
0.2894 185.0 11655 0.5511
0.297 186.0 11718 0.5408
0.2904 187.0 11781 0.5574
0.2818 188.0 11844 0.5182
0.2873 189.0 11907 0.5425
0.2973 190.0 11970 0.5198
0.2913 191.0 12033 0.5119
0.2931 192.0 12096 0.5585
0.2859 193.0 12159 0.5368
0.2853 194.0 12222 0.5274
0.294 195.0 12285 0.5685
0.2885 196.0 12348 0.5581
0.295 197.0 12411 0.4987
0.2807 198.0 12474 0.5168
0.289 199.0 12537 0.5284
0.2893 200.0 12600 0.5420

Framework versions

  • Transformers 4.24.0
  • Pytorch 1.12.1+cu113
  • Datasets 2.6.1
  • Tokenizers 0.13.2
Downloads last month
2