Edit model card

bert_sentence_classifier

This model is a fine-tuned version of bert-large-cased on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 3.0040
  • F1: 0.6123
  • Precision: 0.6123
  • Recall: 0.6123

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 8

Training results

Training Loss Epoch Step Validation Loss F1 Precision Recall
2.0049 0.04 500 1.5854 0.5693 0.5693 0.5693
1.552 0.07 1000 1.4428 0.6131 0.6131 0.6131
1.502 0.11 1500 1.3977 0.6213 0.6213 0.6213
1.4515 0.14 2000 1.3926 0.6200 0.6200 0.6200
1.43 0.18 2500 1.3553 0.6350 0.6350 0.6350
1.413 0.21 3000 1.3461 0.6346 0.6346 0.6346
1.4109 0.25 3500 1.3199 0.6496 0.6496 0.6496
1.3853 0.28 4000 1.3338 0.6406 0.6406 0.6406
1.3788 0.32 4500 1.3306 0.6471 0.6471 0.6471
1.3585 0.35 5000 1.3295 0.6410 0.6410 0.6410
1.356 0.39 5500 1.3025 0.6441 0.6441 0.6441
1.3534 0.42 6000 1.3197 0.6406 0.6406 0.6406
1.3324 0.46 6500 1.2932 0.6436 0.6436 0.6436
1.3563 0.49 7000 1.3202 0.6488 0.6488 0.6488
1.3121 0.53 7500 1.3024 0.6428 0.6428 0.6428
1.3092 0.56 8000 1.3142 0.6419 0.6419 0.6419
1.3769 0.6 8500 1.2974 0.6441 0.6441 0.6441
1.3487 0.63 9000 1.2882 0.6556 0.6556 0.6556
1.3475 0.67 9500 1.2928 0.6441 0.6441 0.6441
1.3038 0.7 10000 1.2846 0.6488 0.6488 0.6488
1.3371 0.74 10500 1.2894 0.6591 0.6591 0.6591
1.3222 0.77 11000 1.2745 0.6535 0.6535 0.6535
1.2983 0.81 11500 1.2832 0.6526 0.6526 0.6526
1.3505 0.84 12000 1.2812 0.6531 0.6531 0.6531
1.2752 0.88 12500 1.2629 0.6578 0.6578 0.6578
1.3115 0.91 13000 1.2787 0.6453 0.6453 0.6453
1.3353 0.95 13500 1.2707 0.6539 0.6539 0.6539
1.2982 0.98 14000 1.2618 0.6569 0.6569 0.6569
1.1885 1.02 14500 1.2999 0.6544 0.6544 0.6544
1.1339 1.05 15000 1.3086 0.6458 0.6458 0.6458
1.0661 1.09 15500 1.2871 0.6582 0.6582 0.6582
1.109 1.12 16000 1.2800 0.6608 0.6608 0.6608
1.0305 1.16 16500 1.3098 0.6604 0.6604 0.6604
1.0855 1.19 17000 1.2968 0.6587 0.6587 0.6587
1.0933 1.23 17500 1.3075 0.6509 0.6509 0.6509
1.1229 1.26 18000 1.3018 0.6496 0.6496 0.6496
1.1043 1.3 18500 1.2832 0.6565 0.6565 0.6565
1.1344 1.33 19000 1.2825 0.6591 0.6591 0.6591
1.1467 1.37 19500 1.2797 0.6642 0.6642 0.6642
1.0596 1.4 20000 1.2841 0.6522 0.6522 0.6522
1.1286 1.44 20500 1.2912 0.6544 0.6544 0.6544
1.1219 1.47 21000 1.3143 0.6509 0.6509 0.6509
1.1339 1.51 21500 1.3021 0.6539 0.6539 0.6539
1.1091 1.54 22000 1.2738 0.6625 0.6625 0.6625
1.1403 1.58 22500 1.2822 0.6548 0.6548 0.6548
1.146 1.61 23000 1.2724 0.6587 0.6587 0.6587
1.1237 1.65 23500 1.2757 0.6569 0.6569 0.6569
1.1453 1.68 24000 1.2985 0.6535 0.6535 0.6535
1.1309 1.72 24500 1.2876 0.6578 0.6578 0.6578
1.1494 1.75 25000 1.2892 0.6552 0.6552 0.6552
1.1571 1.79 25500 1.2806 0.6548 0.6548 0.6548
1.0766 1.82 26000 1.2889 0.6509 0.6509 0.6509
1.1416 1.86 26500 1.2673 0.6599 0.6599 0.6599
1.1179 1.89 27000 1.2919 0.6501 0.6501 0.6501
1.0838 1.93 27500 1.3198 0.6488 0.6488 0.6488
1.1426 1.96 28000 1.2766 0.6561 0.6561 0.6561
1.1559 2.0 28500 1.2839 0.6561 0.6561 0.6561
0.8783 2.03 29000 1.3377 0.6509 0.6509 0.6509
0.8822 2.07 29500 1.3813 0.6501 0.6501 0.6501
0.8823 2.1 30000 1.3738 0.6514 0.6514 0.6514
0.9094 2.14 30500 1.3667 0.6522 0.6522 0.6522
0.8828 2.17 31000 1.3654 0.6582 0.6582 0.6582
0.8489 2.21 31500 1.3404 0.6556 0.6556 0.6556
0.8719 2.24 32000 1.4173 0.6393 0.6393 0.6393
0.8926 2.28 32500 1.4026 0.6535 0.6535 0.6535
0.871 2.31 33000 1.4133 0.6428 0.6428 0.6428
0.9047 2.35 33500 1.3915 0.6449 0.6449 0.6449
0.8621 2.38 34000 1.4109 0.6483 0.6483 0.6483
0.8978 2.42 34500 1.3675 0.6471 0.6471 0.6471
0.8808 2.45 35000 1.3826 0.6522 0.6522 0.6522
0.9299 2.49 35500 1.3673 0.6535 0.6535 0.6535
0.8546 2.52 36000 1.4034 0.6518 0.6518 0.6518
0.8855 2.56 36500 1.3763 0.6458 0.6458 0.6458
0.8996 2.59 37000 1.3930 0.6539 0.6539 0.6539
0.8889 2.63 37500 1.3966 0.6471 0.6471 0.6471
0.8811 2.66 38000 1.4131 0.6475 0.6475 0.6475
0.9129 2.7 38500 1.3816 0.6445 0.6445 0.6445
0.8708 2.73 39000 1.4354 0.6492 0.6492 0.6492
0.8667 2.77 39500 1.4076 0.6380 0.6380 0.6380
0.9139 2.8 40000 1.4200 0.6423 0.6423 0.6423
0.9035 2.84 40500 1.3913 0.6462 0.6462 0.6462
0.9312 2.87 41000 1.3806 0.6449 0.6449 0.6449
0.9382 2.91 41500 1.4064 0.6522 0.6522 0.6522
0.8765 2.95 42000 1.4146 0.6380 0.6380 0.6380
0.8801 2.98 42500 1.3898 0.6445 0.6445 0.6445
0.7988 3.02 43000 1.4740 0.6436 0.6436 0.6436
0.6752 3.05 43500 1.5622 0.6372 0.6372 0.6372
0.649 3.09 44000 1.6055 0.6359 0.6359 0.6359
0.669 3.12 44500 1.5736 0.6380 0.6380 0.6380
0.7189 3.16 45000 1.5832 0.6346 0.6346 0.6346
0.6724 3.19 45500 1.6194 0.6260 0.6260 0.6260
0.7139 3.23 46000 1.5966 0.6359 0.6359 0.6359
0.6985 3.26 46500 1.5803 0.6342 0.6342 0.6342
0.6503 3.3 47000 1.6485 0.6376 0.6376 0.6376
0.6879 3.33 47500 1.5959 0.6325 0.6325 0.6325
0.7342 3.37 48000 1.5534 0.6389 0.6389 0.6389
0.6838 3.4 48500 1.5807 0.6337 0.6337 0.6337
0.7295 3.44 49000 1.6192 0.6372 0.6372 0.6372
0.7044 3.47 49500 1.6618 0.6346 0.6346 0.6346
0.7071 3.51 50000 1.6255 0.6342 0.6342 0.6342
0.7055 3.54 50500 1.5584 0.6363 0.6363 0.6363
0.6781 3.58 51000 1.5948 0.6376 0.6376 0.6376
0.7004 3.61 51500 1.6311 0.6320 0.6320 0.6320
0.715 3.65 52000 1.5972 0.6423 0.6423 0.6423
0.7399 3.68 52500 1.6402 0.6325 0.6325 0.6325
0.6972 3.72 53000 1.6186 0.6406 0.6406 0.6406
0.7219 3.75 53500 1.5945 0.6359 0.6359 0.6359
0.763 3.79 54000 1.5900 0.6380 0.6380 0.6380
0.7196 3.82 54500 1.6218 0.6320 0.6320 0.6320
0.7682 3.86 55000 1.5538 0.6372 0.6372 0.6372
0.6949 3.89 55500 1.6209 0.6295 0.6295 0.6295
0.7461 3.93 56000 1.6237 0.6316 0.6316 0.6316
0.7295 3.96 56500 1.6011 0.6333 0.6333 0.6333
0.6846 4.0 57000 1.6899 0.6312 0.6312 0.6312
0.556 4.03 57500 1.7783 0.6303 0.6303 0.6303
0.5276 4.07 58000 1.8985 0.6260 0.6260 0.6260
0.5576 4.1 58500 1.8263 0.6264 0.6264 0.6264
0.5303 4.14 59000 1.8411 0.6316 0.6316 0.6316
0.5574 4.17 59500 1.8353 0.6286 0.6286 0.6286
0.5468 4.21 60000 1.9252 0.6286 0.6286 0.6286
0.532 4.24 60500 1.8903 0.6295 0.6295 0.6295
0.5329 4.28 61000 1.9416 0.6252 0.6252 0.6252
0.5539 4.31 61500 1.9149 0.6260 0.6260 0.6260
0.5661 4.35 62000 1.9074 0.6286 0.6286 0.6286
0.5502 4.38 62500 2.0259 0.6316 0.6316 0.6316
0.5658 4.42 63000 1.9049 0.6256 0.6256 0.6256
0.5958 4.45 63500 1.9252 0.6166 0.6166 0.6166
0.5972 4.49 64000 1.8518 0.6286 0.6286 0.6286
0.5964 4.52 64500 1.8793 0.6234 0.6234 0.6234
0.5506 4.56 65000 1.9218 0.6346 0.6346 0.6346
0.5516 4.59 65500 1.8957 0.6389 0.6389 0.6389
0.5777 4.63 66000 1.9603 0.6295 0.6295 0.6295
0.5953 4.66 66500 1.8605 0.6252 0.6252 0.6252
0.5797 4.7 67000 1.8797 0.6320 0.6320 0.6320
0.5836 4.73 67500 1.9320 0.6260 0.6260 0.6260
0.6019 4.77 68000 1.8465 0.6239 0.6239 0.6239
0.6099 4.8 68500 1.9481 0.6299 0.6299 0.6299
0.6064 4.84 69000 1.9033 0.6307 0.6307 0.6307
0.5836 4.87 69500 1.8878 0.6234 0.6234 0.6234
0.5766 4.91 70000 1.8860 0.6277 0.6277 0.6277
0.623 4.94 70500 1.8033 0.6303 0.6303 0.6303
0.596 4.98 71000 1.9038 0.6333 0.6333 0.6333
0.537 5.01 71500 2.0795 0.6234 0.6234 0.6234
0.4663 5.05 72000 2.0325 0.6217 0.6217 0.6217
0.4173 5.08 72500 2.2377 0.6273 0.6273 0.6273
0.4521 5.12 73000 2.1218 0.6217 0.6217 0.6217
0.4243 5.15 73500 2.2731 0.6204 0.6204 0.6204
0.4672 5.19 74000 2.2111 0.6247 0.6247 0.6247
0.4884 5.22 74500 2.1027 0.6226 0.6226 0.6226
0.4314 5.26 75000 2.2218 0.6230 0.6230 0.6230
0.4581 5.29 75500 2.2036 0.6264 0.6264 0.6264
0.4245 5.33 76000 2.2419 0.6200 0.6200 0.6200
0.4391 5.36 76500 2.1762 0.6187 0.6187 0.6187
0.4672 5.4 77000 2.2779 0.6179 0.6179 0.6179
0.4821 5.43 77500 2.2881 0.6187 0.6187 0.6187
0.4872 5.47 78000 2.2406 0.6119 0.6119 0.6119
0.4584 5.5 78500 2.3521 0.6209 0.6209 0.6209
0.4774 5.54 79000 2.2522 0.6174 0.6174 0.6174
0.5151 5.57 79500 2.2233 0.6140 0.6140 0.6140
0.493 5.61 80000 2.2333 0.6256 0.6256 0.6256
0.4846 5.64 80500 2.1891 0.6200 0.6200 0.6200
0.478 5.68 81000 2.3159 0.6196 0.6196 0.6196
0.4851 5.71 81500 2.2356 0.6234 0.6234 0.6234
0.4902 5.75 82000 2.3525 0.6222 0.6222 0.6222
0.4992 5.79 82500 2.2111 0.6067 0.6067 0.6067
0.4799 5.82 83000 2.2650 0.6131 0.6131 0.6131
0.4849 5.86 83500 2.2628 0.6204 0.6204 0.6204
0.4772 5.89 84000 2.2711 0.6174 0.6174 0.6174
0.5465 5.93 84500 2.2793 0.6144 0.6144 0.6144
0.4466 5.96 85000 2.2369 0.6166 0.6166 0.6166
0.4885 6.0 85500 2.1963 0.6217 0.6217 0.6217
0.3862 6.03 86000 2.4233 0.6174 0.6174 0.6174
0.3738 6.07 86500 2.4405 0.6191 0.6191 0.6191
0.349 6.1 87000 2.4512 0.6161 0.6161 0.6161
0.3659 6.14 87500 2.5251 0.6226 0.6226 0.6226
0.3365 6.17 88000 2.5326 0.6217 0.6217 0.6217
0.3336 6.21 88500 2.4413 0.6179 0.6179 0.6179
0.3632 6.24 89000 2.6415 0.6114 0.6114 0.6114
0.3584 6.28 89500 2.5388 0.6179 0.6179 0.6179
0.3891 6.31 90000 2.6418 0.6123 0.6123 0.6123
0.3805 6.35 90500 2.6223 0.6127 0.6127 0.6127
0.363 6.38 91000 2.5399 0.6131 0.6131 0.6131
0.3723 6.42 91500 2.6033 0.6187 0.6187 0.6187
0.3808 6.45 92000 2.5281 0.6243 0.6243 0.6243
0.3921 6.49 92500 2.5814 0.6007 0.6007 0.6007
0.3763 6.52 93000 2.6656 0.6058 0.6058 0.6058
0.3921 6.56 93500 2.4935 0.6084 0.6084 0.6084
0.3737 6.59 94000 2.7270 0.6166 0.6166 0.6166
0.3766 6.63 94500 2.5289 0.6217 0.6217 0.6217
0.4439 6.66 95000 2.6161 0.6222 0.6222 0.6222
0.4166 6.7 95500 2.5298 0.6123 0.6123 0.6123
0.4064 6.73 96000 2.5952 0.6183 0.6183 0.6183
0.4253 6.77 96500 2.4567 0.6127 0.6127 0.6127
0.3754 6.8 97000 2.5473 0.6131 0.6131 0.6131
0.3993 6.84 97500 2.5563 0.6161 0.6161 0.6161
0.3802 6.87 98000 2.6585 0.6076 0.6076 0.6076
0.4504 6.91 98500 2.5700 0.6127 0.6127 0.6127
0.3832 6.94 99000 2.5983 0.6174 0.6174 0.6174
0.4212 6.98 99500 2.6137 0.6110 0.6110 0.6110
0.3253 7.01 100000 2.8467 0.6024 0.6024 0.6024
0.2553 7.05 100500 2.7412 0.6063 0.6063 0.6063
0.2771 7.08 101000 2.8670 0.6101 0.6101 0.6101
0.2733 7.12 101500 2.8536 0.6166 0.6166 0.6166
0.2972 7.15 102000 2.8254 0.6161 0.6161 0.6161
0.2893 7.19 102500 3.0228 0.6058 0.6058 0.6058
0.3104 7.22 103000 2.8617 0.6011 0.6011 0.6011
0.3019 7.26 103500 3.0106 0.6131 0.6131 0.6131
0.3143 7.29 104000 3.0189 0.6088 0.6088 0.6088
0.3054 7.33 104500 3.0291 0.6063 0.6063 0.6063
0.3145 7.36 105000 3.0166 0.6106 0.6106 0.6106
0.2913 7.4 105500 3.0480 0.6174 0.6174 0.6174
0.3159 7.43 106000 2.9714 0.6084 0.6084 0.6084
0.3216 7.47 106500 2.9359 0.6187 0.6187 0.6187
0.2982 7.5 107000 3.0509 0.6084 0.6084 0.6084
0.2952 7.54 107500 2.9428 0.6076 0.6076 0.6076
0.304 7.57 108000 3.0155 0.6071 0.6071 0.6071
0.2896 7.61 108500 3.0276 0.6196 0.6196 0.6196
0.3226 7.64 109000 2.9331 0.6097 0.6097 0.6097
0.299 7.68 109500 2.9671 0.6050 0.6050 0.6050
0.3079 7.71 110000 2.9394 0.6093 0.6093 0.6093
0.3064 7.75 110500 2.8690 0.6110 0.6110 0.6110
0.3423 7.78 111000 2.9095 0.6183 0.6183 0.6183
0.3085 7.82 111500 2.9967 0.6260 0.6260 0.6260
0.3071 7.85 112000 2.9429 0.6127 0.6127 0.6127
0.3197 7.89 112500 3.0123 0.6157 0.6157 0.6157
0.3361 7.92 113000 2.9832 0.6170 0.6170 0.6170
0.3252 7.96 113500 3.0174 0.6071 0.6071 0.6071
0.2802 7.99 114000 3.0040 0.6123 0.6123 0.6123

Framework versions

  • Transformers 4.19.2
  • Pytorch 1.11.0+cu113
  • Datasets 2.2.2
  • Tokenizers 0.12.1
Downloads last month
4