--- license: apache-2.0 base_model: distilbert-base-uncased tags: - generated_from_trainer metrics: - precision - recall - f1 - accuracy model-index: - name: distilbert-base-uncased-finetuned-pos-kk-3080 results: [] --- # distilbert-base-uncased-finetuned-pos-kk-3080 This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the None dataset. It achieves the following results on the evaluation set: - Loss: 2.6344 - Precision: 0.6506 - Recall: 0.6342 - F1: 0.6423 - Accuracy: 0.7199 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 100 ### Training results | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy | |:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:| | No log | 1.0 | 59 | 2.3411 | 0.6246 | 0.6067 | 0.6155 | 0.7026 | | No log | 2.0 | 118 | 2.1636 | 0.6382 | 0.625 | 0.6315 | 0.7093 | | No log | 3.0 | 177 | 2.2138 | 0.6121 | 0.6044 | 0.6082 | 0.6920 | | No log | 4.0 | 236 | 2.3509 | 0.6271 | 0.6307 | 0.6289 | 0.7045 | | No log | 5.0 | 295 | 2.2692 | 0.6408 | 0.6239 | 0.6322 | 0.7113 | | No log | 6.0 | 354 | 2.1764 | 0.6408 | 0.6261 | 0.6334 | 0.7103 | | No log | 7.0 | 413 | 2.2618 | 0.6404 | 0.6147 | 0.6273 | 0.7093 | | No log | 8.0 | 472 | 2.3162 | 0.6423 | 0.6239 | 0.6329 | 0.7122 | | 0.0115 | 9.0 | 531 | 2.3318 | 0.6338 | 0.6193 | 0.6265 | 0.7093 | | 0.0115 | 10.0 | 590 | 2.3152 | 0.6316 | 0.6273 | 0.6295 | 0.7036 | | 0.0115 | 11.0 | 649 | 2.2334 | 0.6307 | 0.6170 | 0.6238 | 0.7026 | | 0.0115 | 12.0 | 708 | 2.2705 | 0.6446 | 0.6261 | 0.6353 | 0.7161 | | 0.0115 | 13.0 | 767 | 2.3049 | 0.6313 | 0.6204 | 0.6258 | 0.7055 | | 0.0115 | 14.0 | 826 | 2.2713 | 0.6391 | 0.6193 | 0.6290 | 0.7151 | | 0.0115 | 15.0 | 885 | 2.2914 | 0.6350 | 0.6204 | 0.6276 | 0.7122 | | 0.0115 | 16.0 | 944 | 2.1958 | 0.6286 | 0.6193 | 0.6239 | 0.7055 | | 0.0078 | 17.0 | 1003 | 2.3721 | 0.6200 | 0.6193 | 0.6196 | 0.7026 | | 0.0078 | 18.0 | 1062 | 2.2756 | 0.6544 | 0.6319 | 0.6429 | 0.7238 | | 0.0078 | 19.0 | 1121 | 2.3066 | 0.6585 | 0.6433 | 0.6508 | 0.7276 | | 0.0078 | 20.0 | 1180 | 2.3065 | 0.6489 | 0.6273 | 0.6379 | 0.7141 | | 0.0078 | 21.0 | 1239 | 2.3865 | 0.6314 | 0.6365 | 0.6339 | 0.7122 | | 0.0078 | 22.0 | 1298 | 2.4231 | 0.6263 | 0.6284 | 0.6274 | 0.7084 | | 0.0078 | 23.0 | 1357 | 2.3871 | 0.6247 | 0.6319 | 0.6283 | 0.7084 | | 0.0078 | 24.0 | 1416 | 2.4611 | 0.6390 | 0.6273 | 0.6331 | 0.7170 | | 0.0078 | 25.0 | 1475 | 2.3563 | 0.6345 | 0.6193 | 0.6268 | 0.7045 | | 0.0043 | 26.0 | 1534 | 2.4208 | 0.6424 | 0.6261 | 0.6341 | 0.7190 | | 0.0043 | 27.0 | 1593 | 2.3468 | 0.6343 | 0.6227 | 0.6285 | 0.7103 | | 0.0043 | 28.0 | 1652 | 2.4458 | 0.6397 | 0.625 | 0.6323 | 0.7161 | | 0.0043 | 29.0 | 1711 | 2.4779 | 0.6394 | 0.6284 | 0.6339 | 0.7113 | | 0.0043 | 30.0 | 1770 | 2.3498 | 0.6466 | 0.6273 | 0.6368 | 0.7161 | | 0.0043 | 31.0 | 1829 | 2.4026 | 0.6454 | 0.6388 | 0.6421 | 0.7141 | | 0.0043 | 32.0 | 1888 | 2.4380 | 0.6394 | 0.6284 | 0.6339 | 0.7132 | | 0.0043 | 33.0 | 1947 | 2.4099 | 0.6360 | 0.6193 | 0.6275 | 0.7084 | | 0.0027 | 34.0 | 2006 | 2.4331 | 0.6370 | 0.6158 | 0.6262 | 0.7093 | | 0.0027 | 35.0 | 2065 | 2.4166 | 0.6519 | 0.6399 | 0.6458 | 0.7199 | | 0.0027 | 36.0 | 2124 | 2.5268 | 0.6303 | 0.6353 | 0.6328 | 0.7055 | | 0.0027 | 37.0 | 2183 | 2.4152 | 0.6377 | 0.6319 | 0.6348 | 0.7074 | | 0.0027 | 38.0 | 2242 | 2.5392 | 0.6293 | 0.6307 | 0.6300 | 0.7045 | | 0.0027 | 39.0 | 2301 | 2.5672 | 0.6324 | 0.6353 | 0.6339 | 0.7093 | | 0.0027 | 40.0 | 2360 | 2.5116 | 0.6323 | 0.6330 | 0.6327 | 0.7093 | | 0.0027 | 41.0 | 2419 | 2.5884 | 0.6362 | 0.6376 | 0.6369 | 0.7113 | | 0.0027 | 42.0 | 2478 | 2.5252 | 0.6512 | 0.6273 | 0.6390 | 0.7180 | | 0.0021 | 43.0 | 2537 | 2.4763 | 0.6480 | 0.6353 | 0.6416 | 0.7151 | | 0.0021 | 44.0 | 2596 | 2.4957 | 0.6455 | 0.6307 | 0.6381 | 0.7132 | | 0.0021 | 45.0 | 2655 | 2.5187 | 0.6441 | 0.6330 | 0.6385 | 0.7064 | | 0.0021 | 46.0 | 2714 | 2.4969 | 0.6524 | 0.6307 | 0.6414 | 0.7161 | | 0.0021 | 47.0 | 2773 | 2.5839 | 0.6567 | 0.6296 | 0.6429 | 0.7180 | | 0.0021 | 48.0 | 2832 | 2.4747 | 0.6647 | 0.6342 | 0.6491 | 0.7267 | | 0.0021 | 49.0 | 2891 | 2.5119 | 0.6492 | 0.6411 | 0.6451 | 0.7218 | | 0.0021 | 50.0 | 2950 | 2.5855 | 0.6382 | 0.6330 | 0.6356 | 0.7132 | | 0.0016 | 51.0 | 3009 | 2.5679 | 0.6549 | 0.6376 | 0.6461 | 0.7190 | | 0.0016 | 52.0 | 3068 | 2.4618 | 0.6631 | 0.6388 | 0.6507 | 0.7295 | | 0.0016 | 53.0 | 3127 | 2.5270 | 0.6529 | 0.6170 | 0.6344 | 0.7141 | | 0.0016 | 54.0 | 3186 | 2.5133 | 0.6485 | 0.6284 | 0.6383 | 0.7141 | | 0.0016 | 55.0 | 3245 | 2.4895 | 0.6560 | 0.6342 | 0.6449 | 0.7180 | | 0.0016 | 56.0 | 3304 | 2.5001 | 0.6650 | 0.6261 | 0.6450 | 0.7267 | | 0.0016 | 57.0 | 3363 | 2.5202 | 0.6516 | 0.6284 | 0.6398 | 0.7180 | | 0.0016 | 58.0 | 3422 | 2.4701 | 0.6715 | 0.6330 | 0.6517 | 0.7305 | | 0.0016 | 59.0 | 3481 | 2.4988 | 0.6598 | 0.625 | 0.6419 | 0.7238 | | 0.0015 | 60.0 | 3540 | 2.5555 | 0.6499 | 0.6216 | 0.6354 | 0.7161 | | 0.0015 | 61.0 | 3599 | 2.5242 | 0.6487 | 0.6353 | 0.6419 | 0.7228 | | 0.0015 | 62.0 | 3658 | 2.5146 | 0.6618 | 0.6284 | 0.6447 | 0.7190 | | 0.0015 | 63.0 | 3717 | 2.5632 | 0.6496 | 0.625 | 0.6371 | 0.7170 | | 0.0015 | 64.0 | 3776 | 2.5966 | 0.6486 | 0.6307 | 0.6395 | 0.7209 | | 0.0015 | 65.0 | 3835 | 2.6079 | 0.6386 | 0.6261 | 0.6323 | 0.7170 | | 0.0015 | 66.0 | 3894 | 2.5620 | 0.6355 | 0.6239 | 0.6296 | 0.7084 | | 0.0015 | 67.0 | 3953 | 2.5748 | 0.6566 | 0.625 | 0.6404 | 0.7218 | | 0.0009 | 68.0 | 4012 | 2.5582 | 0.6548 | 0.6353 | 0.6449 | 0.7209 | | 0.0009 | 69.0 | 4071 | 2.5776 | 0.6549 | 0.6181 | 0.6360 | 0.7209 | | 0.0009 | 70.0 | 4130 | 2.5435 | 0.6619 | 0.6330 | 0.6471 | 0.7267 | | 0.0009 | 71.0 | 4189 | 2.5359 | 0.6489 | 0.6296 | 0.6391 | 0.7199 | | 0.0009 | 72.0 | 4248 | 2.6138 | 0.6394 | 0.6284 | 0.6339 | 0.7151 | | 0.0009 | 73.0 | 4307 | 2.6431 | 0.6385 | 0.6158 | 0.6270 | 0.7093 | | 0.0009 | 74.0 | 4366 | 2.6701 | 0.6412 | 0.6353 | 0.6382 | 0.7151 | | 0.0009 | 75.0 | 4425 | 2.6492 | 0.6511 | 0.6376 | 0.6443 | 0.7199 | | 0.0009 | 76.0 | 4484 | 2.6477 | 0.6564 | 0.6376 | 0.6469 | 0.7218 | | 0.0007 | 77.0 | 4543 | 2.6216 | 0.6422 | 0.6216 | 0.6317 | 0.7122 | | 0.0007 | 78.0 | 4602 | 2.6166 | 0.6446 | 0.6261 | 0.6353 | 0.7132 | | 0.0007 | 79.0 | 4661 | 2.7084 | 0.6382 | 0.6353 | 0.6368 | 0.7180 | | 0.0007 | 80.0 | 4720 | 2.6783 | 0.6482 | 0.6296 | 0.6387 | 0.7199 | | 0.0007 | 81.0 | 4779 | 2.7061 | 0.6472 | 0.6353 | 0.6412 | 0.7170 | | 0.0007 | 82.0 | 4838 | 2.6468 | 0.6503 | 0.6376 | 0.6439 | 0.7190 | | 0.0007 | 83.0 | 4897 | 2.6437 | 0.6404 | 0.6330 | 0.6367 | 0.7141 | | 0.0007 | 84.0 | 4956 | 2.5965 | 0.6474 | 0.6296 | 0.6384 | 0.7170 | | 0.0009 | 85.0 | 5015 | 2.6175 | 0.6524 | 0.6307 | 0.6414 | 0.7199 | | 0.0009 | 86.0 | 5074 | 2.6304 | 0.6471 | 0.6307 | 0.6388 | 0.7161 | | 0.0009 | 87.0 | 5133 | 2.6389 | 0.6524 | 0.6284 | 0.6402 | 0.7199 | | 0.0009 | 88.0 | 5192 | 2.6132 | 0.6544 | 0.6296 | 0.6417 | 0.7218 | | 0.0009 | 89.0 | 5251 | 2.5972 | 0.6475 | 0.6319 | 0.6396 | 0.7180 | | 0.0009 | 90.0 | 5310 | 2.6066 | 0.6580 | 0.6376 | 0.6476 | 0.7257 | | 0.0009 | 91.0 | 5369 | 2.6175 | 0.6611 | 0.6330 | 0.6467 | 0.7238 | | 0.0009 | 92.0 | 5428 | 2.6420 | 0.6506 | 0.6365 | 0.6435 | 0.7238 | | 0.0009 | 93.0 | 5487 | 2.6679 | 0.6480 | 0.6376 | 0.6428 | 0.7209 | | 0.0007 | 94.0 | 5546 | 2.6318 | 0.6486 | 0.6330 | 0.6407 | 0.7199 | | 0.0007 | 95.0 | 5605 | 2.6225 | 0.6553 | 0.6365 | 0.6457 | 0.7228 | | 0.0007 | 96.0 | 5664 | 2.6299 | 0.6502 | 0.6330 | 0.6415 | 0.7199 | | 0.0007 | 97.0 | 5723 | 2.6313 | 0.6514 | 0.6342 | 0.6426 | 0.7209 | | 0.0007 | 98.0 | 5782 | 2.6338 | 0.6518 | 0.6353 | 0.6434 | 0.7209 | | 0.0007 | 99.0 | 5841 | 2.6334 | 0.6518 | 0.6353 | 0.6434 | 0.7209 | | 0.0007 | 100.0 | 5900 | 2.6344 | 0.6506 | 0.6342 | 0.6423 | 0.7199 | ### Framework versions - Transformers 4.34.1 - Pytorch 2.2.1+cu118 - Datasets 2.14.6 - Tokenizers 0.14.1