bsc-bio-ehr-es-finetuned-clinais-augmented2
This model is a fine-tuned version of joheras/bsc-bio-ehr-es-finetuned-clinais on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.3136
- Precision: 0.4988
- Recall: 0.6452
- F1: 0.5626
- Accuracy: 0.8593
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
Training results
Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
---|---|---|---|---|---|---|---|
No log | 1.0 | 114 | 0.7550 | 0.1166 | 0.1605 | 0.1351 | 0.7937 |
No log | 2.0 | 228 | 0.6577 | 0.2096 | 0.2904 | 0.2435 | 0.8123 |
No log | 3.0 | 342 | 0.5928 | 0.2167 | 0.3157 | 0.2570 | 0.8197 |
No log | 4.0 | 456 | 0.5837 | 0.2532 | 0.3791 | 0.3036 | 0.8374 |
0.5045 | 5.0 | 570 | 0.5758 | 0.3309 | 0.5206 | 0.4046 | 0.8479 |
0.5045 | 6.0 | 684 | 0.6266 | 0.3506 | 0.5216 | 0.4194 | 0.8384 |
0.5045 | 7.0 | 798 | 0.6329 | 0.3503 | 0.5586 | 0.4306 | 0.8486 |
0.5045 | 8.0 | 912 | 0.6251 | 0.3769 | 0.5639 | 0.4518 | 0.8565 |
0.1043 | 9.0 | 1026 | 0.6509 | 0.3999 | 0.5945 | 0.4781 | 0.8582 |
0.1043 | 10.0 | 1140 | 0.7431 | 0.3755 | 0.5734 | 0.4538 | 0.8453 |
0.1043 | 11.0 | 1254 | 0.7000 | 0.4181 | 0.6061 | 0.4948 | 0.8598 |
0.1043 | 12.0 | 1368 | 0.7232 | 0.4375 | 0.6315 | 0.5169 | 0.8574 |
0.1043 | 13.0 | 1482 | 0.7741 | 0.4568 | 0.6199 | 0.5260 | 0.8596 |
0.0344 | 14.0 | 1596 | 0.8108 | 0.4661 | 0.6251 | 0.5341 | 0.8528 |
0.0344 | 15.0 | 1710 | 0.7922 | 0.4516 | 0.6209 | 0.5229 | 0.8613 |
0.0344 | 16.0 | 1824 | 0.8318 | 0.4727 | 0.6315 | 0.5407 | 0.8566 |
0.0344 | 17.0 | 1938 | 0.8579 | 0.4887 | 0.6378 | 0.5534 | 0.8558 |
0.0158 | 18.0 | 2052 | 0.8745 | 0.4626 | 0.6209 | 0.5302 | 0.8539 |
0.0158 | 19.0 | 2166 | 0.8959 | 0.4461 | 0.6167 | 0.5177 | 0.8530 |
0.0158 | 20.0 | 2280 | 0.9600 | 0.4442 | 0.6135 | 0.5153 | 0.8490 |
0.0158 | 21.0 | 2394 | 0.8769 | 0.4578 | 0.6294 | 0.5300 | 0.8601 |
0.0095 | 22.0 | 2508 | 0.9427 | 0.4503 | 0.6125 | 0.5190 | 0.8516 |
0.0095 | 23.0 | 2622 | 0.9188 | 0.4793 | 0.6367 | 0.5469 | 0.8582 |
0.0095 | 24.0 | 2736 | 0.9538 | 0.4790 | 0.6367 | 0.5467 | 0.8556 |
0.0095 | 25.0 | 2850 | 0.9785 | 0.4775 | 0.6262 | 0.5418 | 0.8495 |
0.0095 | 26.0 | 2964 | 0.9745 | 0.4785 | 0.6463 | 0.5499 | 0.8514 |
0.0066 | 27.0 | 3078 | 1.0168 | 0.4909 | 0.6262 | 0.5503 | 0.8541 |
0.0066 | 28.0 | 3192 | 0.9852 | 0.4796 | 0.6452 | 0.5502 | 0.8515 |
0.0066 | 29.0 | 3306 | 1.0372 | 0.4711 | 0.6367 | 0.5415 | 0.8526 |
0.0066 | 30.0 | 3420 | 1.0234 | 0.5012 | 0.6399 | 0.5622 | 0.8530 |
0.0045 | 31.0 | 3534 | 1.0058 | 0.4944 | 0.6484 | 0.5610 | 0.8582 |
0.0045 | 32.0 | 3648 | 1.0212 | 0.4776 | 0.6420 | 0.5477 | 0.8559 |
0.0045 | 33.0 | 3762 | 1.0301 | 0.4643 | 0.6378 | 0.5374 | 0.8533 |
0.0045 | 34.0 | 3876 | 1.0545 | 0.4740 | 0.6346 | 0.5427 | 0.8529 |
0.0045 | 35.0 | 3990 | 1.0632 | 0.4703 | 0.6272 | 0.5376 | 0.8524 |
0.0059 | 36.0 | 4104 | 1.0351 | 0.4923 | 0.6389 | 0.5561 | 0.8568 |
0.0059 | 37.0 | 4218 | 1.0584 | 0.4665 | 0.6177 | 0.5316 | 0.8545 |
0.0059 | 38.0 | 4332 | 1.0513 | 0.4695 | 0.6251 | 0.5362 | 0.8566 |
0.0059 | 39.0 | 4446 | 1.0482 | 0.4740 | 0.6357 | 0.5431 | 0.8555 |
0.0028 | 40.0 | 4560 | 1.0777 | 0.4899 | 0.6378 | 0.5541 | 0.8590 |
0.0028 | 41.0 | 4674 | 1.1045 | 0.5066 | 0.6505 | 0.5696 | 0.8589 |
0.0028 | 42.0 | 4788 | 1.0910 | 0.5 | 0.6441 | 0.5630 | 0.8556 |
0.0028 | 43.0 | 4902 | 1.1249 | 0.4809 | 0.6389 | 0.5488 | 0.8546 |
0.0021 | 44.0 | 5016 | 1.0668 | 0.4806 | 0.6283 | 0.5446 | 0.8591 |
0.0021 | 45.0 | 5130 | 1.0699 | 0.5033 | 0.6536 | 0.5687 | 0.8586 |
0.0021 | 46.0 | 5244 | 1.1457 | 0.5004 | 0.6410 | 0.5620 | 0.8526 |
0.0021 | 47.0 | 5358 | 1.1226 | 0.4858 | 0.6336 | 0.5500 | 0.8501 |
0.0021 | 48.0 | 5472 | 1.1459 | 0.4963 | 0.6399 | 0.5590 | 0.8532 |
0.0045 | 49.0 | 5586 | 1.1515 | 0.4919 | 0.6441 | 0.5578 | 0.8508 |
0.0045 | 50.0 | 5700 | 1.1952 | 0.4887 | 0.6420 | 0.5550 | 0.8561 |
0.0045 | 51.0 | 5814 | 1.1382 | 0.5042 | 0.6294 | 0.5599 | 0.8601 |
0.0045 | 52.0 | 5928 | 1.1547 | 0.4757 | 0.6315 | 0.5426 | 0.8585 |
0.0017 | 53.0 | 6042 | 1.2089 | 0.4996 | 0.6188 | 0.5528 | 0.8511 |
0.0017 | 54.0 | 6156 | 1.1838 | 0.4910 | 0.6357 | 0.5541 | 0.8555 |
0.0017 | 55.0 | 6270 | 1.2072 | 0.4907 | 0.6378 | 0.5546 | 0.8572 |
0.0017 | 56.0 | 6384 | 1.1894 | 0.5012 | 0.6505 | 0.5662 | 0.8605 |
0.0017 | 57.0 | 6498 | 1.2195 | 0.4992 | 0.6441 | 0.5625 | 0.8543 |
0.002 | 58.0 | 6612 | 1.1838 | 0.5008 | 0.6484 | 0.5651 | 0.8579 |
0.002 | 59.0 | 6726 | 1.2107 | 0.5029 | 0.6484 | 0.5664 | 0.8561 |
0.002 | 60.0 | 6840 | 1.1850 | 0.5 | 0.6505 | 0.5654 | 0.8581 |
0.002 | 61.0 | 6954 | 1.1926 | 0.5004 | 0.6431 | 0.5628 | 0.8571 |
0.002 | 62.0 | 7068 | 1.2789 | 0.5062 | 0.6431 | 0.5665 | 0.8524 |
0.002 | 63.0 | 7182 | 1.2268 | 0.4939 | 0.6452 | 0.5595 | 0.8562 |
0.002 | 64.0 | 7296 | 1.2729 | 0.5103 | 0.6515 | 0.5724 | 0.8531 |
0.002 | 65.0 | 7410 | 1.2667 | 0.5127 | 0.6410 | 0.5697 | 0.8530 |
0.0016 | 66.0 | 7524 | 1.2580 | 0.5142 | 0.6484 | 0.5736 | 0.8528 |
0.0016 | 67.0 | 7638 | 1.2323 | 0.4839 | 0.6494 | 0.5546 | 0.8567 |
0.0016 | 68.0 | 7752 | 1.2647 | 0.4951 | 0.6399 | 0.5583 | 0.8514 |
0.0016 | 69.0 | 7866 | 1.2604 | 0.5178 | 0.6463 | 0.5749 | 0.8563 |
0.0016 | 70.0 | 7980 | 1.2154 | 0.4996 | 0.6410 | 0.5615 | 0.8596 |
0.0005 | 71.0 | 8094 | 1.2452 | 0.5066 | 0.6505 | 0.5696 | 0.8592 |
0.0005 | 72.0 | 8208 | 1.2100 | 0.4882 | 0.6357 | 0.5523 | 0.8609 |
0.0005 | 73.0 | 8322 | 1.2549 | 0.4936 | 0.6515 | 0.5617 | 0.8590 |
0.0005 | 74.0 | 8436 | 1.2600 | 0.5134 | 0.6463 | 0.5722 | 0.8617 |
0.0004 | 75.0 | 8550 | 1.3043 | 0.4939 | 0.6431 | 0.5587 | 0.8546 |
0.0004 | 76.0 | 8664 | 1.2772 | 0.4907 | 0.6389 | 0.5550 | 0.8591 |
0.0004 | 77.0 | 8778 | 1.3091 | 0.4955 | 0.6389 | 0.5581 | 0.8518 |
0.0004 | 78.0 | 8892 | 1.2821 | 0.4988 | 0.6452 | 0.5626 | 0.8565 |
0.0004 | 79.0 | 9006 | 1.2668 | 0.4984 | 0.6463 | 0.5628 | 0.8591 |
0.0004 | 80.0 | 9120 | 1.2872 | 0.5104 | 0.6463 | 0.5704 | 0.8567 |
0.0004 | 81.0 | 9234 | 1.2877 | 0.4919 | 0.6389 | 0.5558 | 0.8553 |
0.0004 | 82.0 | 9348 | 1.2701 | 0.5079 | 0.6441 | 0.5680 | 0.8587 |
0.0004 | 83.0 | 9462 | 1.3579 | 0.4972 | 0.6484 | 0.5628 | 0.8500 |
0.0004 | 84.0 | 9576 | 1.3292 | 0.5138 | 0.6494 | 0.5737 | 0.8542 |
0.0004 | 85.0 | 9690 | 1.3437 | 0.4963 | 0.6389 | 0.5586 | 0.8546 |
0.0004 | 86.0 | 9804 | 1.3193 | 0.5155 | 0.6505 | 0.5752 | 0.8577 |
0.0004 | 87.0 | 9918 | 1.3579 | 0.4919 | 0.6420 | 0.5570 | 0.8520 |
0.0003 | 88.0 | 10032 | 1.3109 | 0.5181 | 0.6494 | 0.5764 | 0.8585 |
0.0003 | 89.0 | 10146 | 1.2943 | 0.4876 | 0.6441 | 0.5551 | 0.8615 |
0.0003 | 90.0 | 10260 | 1.3132 | 0.4826 | 0.6463 | 0.5526 | 0.8574 |
0.0003 | 91.0 | 10374 | 1.3176 | 0.5058 | 0.6452 | 0.5671 | 0.8563 |
0.0003 | 92.0 | 10488 | 1.3130 | 0.4964 | 0.6463 | 0.5615 | 0.8583 |
0.0002 | 93.0 | 10602 | 1.3220 | 0.4992 | 0.6463 | 0.5633 | 0.8563 |
0.0002 | 94.0 | 10716 | 1.3160 | 0.5 | 0.6463 | 0.5638 | 0.8603 |
0.0002 | 95.0 | 10830 | 1.3155 | 0.4857 | 0.6473 | 0.5550 | 0.8615 |
0.0002 | 96.0 | 10944 | 1.3223 | 0.4912 | 0.6484 | 0.5589 | 0.8577 |
0.0002 | 97.0 | 11058 | 1.3138 | 0.5087 | 0.6484 | 0.5701 | 0.8593 |
0.0002 | 98.0 | 11172 | 1.3152 | 0.5037 | 0.6463 | 0.5661 | 0.8594 |
0.0002 | 99.0 | 11286 | 1.3137 | 0.4996 | 0.6452 | 0.5631 | 0.8593 |
0.0002 | 100.0 | 11400 | 1.3136 | 0.4988 | 0.6452 | 0.5626 | 0.8593 |
Framework versions
- Transformers 4.25.1
- Pytorch 1.13.0
- Datasets 2.8.0
- Tokenizers 0.12.1
- Downloads last month
- 1
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.