Edit model card

bsc-bio-ehr-es-finetuned-clinais-augmented2

This model is a fine-tuned version of joheras/bsc-bio-ehr-es-finetuned-clinais on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.3136
  • Precision: 0.4988
  • Recall: 0.6452
  • F1: 0.5626
  • Accuracy: 0.8593

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 100

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
No log 1.0 114 0.7550 0.1166 0.1605 0.1351 0.7937
No log 2.0 228 0.6577 0.2096 0.2904 0.2435 0.8123
No log 3.0 342 0.5928 0.2167 0.3157 0.2570 0.8197
No log 4.0 456 0.5837 0.2532 0.3791 0.3036 0.8374
0.5045 5.0 570 0.5758 0.3309 0.5206 0.4046 0.8479
0.5045 6.0 684 0.6266 0.3506 0.5216 0.4194 0.8384
0.5045 7.0 798 0.6329 0.3503 0.5586 0.4306 0.8486
0.5045 8.0 912 0.6251 0.3769 0.5639 0.4518 0.8565
0.1043 9.0 1026 0.6509 0.3999 0.5945 0.4781 0.8582
0.1043 10.0 1140 0.7431 0.3755 0.5734 0.4538 0.8453
0.1043 11.0 1254 0.7000 0.4181 0.6061 0.4948 0.8598
0.1043 12.0 1368 0.7232 0.4375 0.6315 0.5169 0.8574
0.1043 13.0 1482 0.7741 0.4568 0.6199 0.5260 0.8596
0.0344 14.0 1596 0.8108 0.4661 0.6251 0.5341 0.8528
0.0344 15.0 1710 0.7922 0.4516 0.6209 0.5229 0.8613
0.0344 16.0 1824 0.8318 0.4727 0.6315 0.5407 0.8566
0.0344 17.0 1938 0.8579 0.4887 0.6378 0.5534 0.8558
0.0158 18.0 2052 0.8745 0.4626 0.6209 0.5302 0.8539
0.0158 19.0 2166 0.8959 0.4461 0.6167 0.5177 0.8530
0.0158 20.0 2280 0.9600 0.4442 0.6135 0.5153 0.8490
0.0158 21.0 2394 0.8769 0.4578 0.6294 0.5300 0.8601
0.0095 22.0 2508 0.9427 0.4503 0.6125 0.5190 0.8516
0.0095 23.0 2622 0.9188 0.4793 0.6367 0.5469 0.8582
0.0095 24.0 2736 0.9538 0.4790 0.6367 0.5467 0.8556
0.0095 25.0 2850 0.9785 0.4775 0.6262 0.5418 0.8495
0.0095 26.0 2964 0.9745 0.4785 0.6463 0.5499 0.8514
0.0066 27.0 3078 1.0168 0.4909 0.6262 0.5503 0.8541
0.0066 28.0 3192 0.9852 0.4796 0.6452 0.5502 0.8515
0.0066 29.0 3306 1.0372 0.4711 0.6367 0.5415 0.8526
0.0066 30.0 3420 1.0234 0.5012 0.6399 0.5622 0.8530
0.0045 31.0 3534 1.0058 0.4944 0.6484 0.5610 0.8582
0.0045 32.0 3648 1.0212 0.4776 0.6420 0.5477 0.8559
0.0045 33.0 3762 1.0301 0.4643 0.6378 0.5374 0.8533
0.0045 34.0 3876 1.0545 0.4740 0.6346 0.5427 0.8529
0.0045 35.0 3990 1.0632 0.4703 0.6272 0.5376 0.8524
0.0059 36.0 4104 1.0351 0.4923 0.6389 0.5561 0.8568
0.0059 37.0 4218 1.0584 0.4665 0.6177 0.5316 0.8545
0.0059 38.0 4332 1.0513 0.4695 0.6251 0.5362 0.8566
0.0059 39.0 4446 1.0482 0.4740 0.6357 0.5431 0.8555
0.0028 40.0 4560 1.0777 0.4899 0.6378 0.5541 0.8590
0.0028 41.0 4674 1.1045 0.5066 0.6505 0.5696 0.8589
0.0028 42.0 4788 1.0910 0.5 0.6441 0.5630 0.8556
0.0028 43.0 4902 1.1249 0.4809 0.6389 0.5488 0.8546
0.0021 44.0 5016 1.0668 0.4806 0.6283 0.5446 0.8591
0.0021 45.0 5130 1.0699 0.5033 0.6536 0.5687 0.8586
0.0021 46.0 5244 1.1457 0.5004 0.6410 0.5620 0.8526
0.0021 47.0 5358 1.1226 0.4858 0.6336 0.5500 0.8501
0.0021 48.0 5472 1.1459 0.4963 0.6399 0.5590 0.8532
0.0045 49.0 5586 1.1515 0.4919 0.6441 0.5578 0.8508
0.0045 50.0 5700 1.1952 0.4887 0.6420 0.5550 0.8561
0.0045 51.0 5814 1.1382 0.5042 0.6294 0.5599 0.8601
0.0045 52.0 5928 1.1547 0.4757 0.6315 0.5426 0.8585
0.0017 53.0 6042 1.2089 0.4996 0.6188 0.5528 0.8511
0.0017 54.0 6156 1.1838 0.4910 0.6357 0.5541 0.8555
0.0017 55.0 6270 1.2072 0.4907 0.6378 0.5546 0.8572
0.0017 56.0 6384 1.1894 0.5012 0.6505 0.5662 0.8605
0.0017 57.0 6498 1.2195 0.4992 0.6441 0.5625 0.8543
0.002 58.0 6612 1.1838 0.5008 0.6484 0.5651 0.8579
0.002 59.0 6726 1.2107 0.5029 0.6484 0.5664 0.8561
0.002 60.0 6840 1.1850 0.5 0.6505 0.5654 0.8581
0.002 61.0 6954 1.1926 0.5004 0.6431 0.5628 0.8571
0.002 62.0 7068 1.2789 0.5062 0.6431 0.5665 0.8524
0.002 63.0 7182 1.2268 0.4939 0.6452 0.5595 0.8562
0.002 64.0 7296 1.2729 0.5103 0.6515 0.5724 0.8531
0.002 65.0 7410 1.2667 0.5127 0.6410 0.5697 0.8530
0.0016 66.0 7524 1.2580 0.5142 0.6484 0.5736 0.8528
0.0016 67.0 7638 1.2323 0.4839 0.6494 0.5546 0.8567
0.0016 68.0 7752 1.2647 0.4951 0.6399 0.5583 0.8514
0.0016 69.0 7866 1.2604 0.5178 0.6463 0.5749 0.8563
0.0016 70.0 7980 1.2154 0.4996 0.6410 0.5615 0.8596
0.0005 71.0 8094 1.2452 0.5066 0.6505 0.5696 0.8592
0.0005 72.0 8208 1.2100 0.4882 0.6357 0.5523 0.8609
0.0005 73.0 8322 1.2549 0.4936 0.6515 0.5617 0.8590
0.0005 74.0 8436 1.2600 0.5134 0.6463 0.5722 0.8617
0.0004 75.0 8550 1.3043 0.4939 0.6431 0.5587 0.8546
0.0004 76.0 8664 1.2772 0.4907 0.6389 0.5550 0.8591
0.0004 77.0 8778 1.3091 0.4955 0.6389 0.5581 0.8518
0.0004 78.0 8892 1.2821 0.4988 0.6452 0.5626 0.8565
0.0004 79.0 9006 1.2668 0.4984 0.6463 0.5628 0.8591
0.0004 80.0 9120 1.2872 0.5104 0.6463 0.5704 0.8567
0.0004 81.0 9234 1.2877 0.4919 0.6389 0.5558 0.8553
0.0004 82.0 9348 1.2701 0.5079 0.6441 0.5680 0.8587
0.0004 83.0 9462 1.3579 0.4972 0.6484 0.5628 0.8500
0.0004 84.0 9576 1.3292 0.5138 0.6494 0.5737 0.8542
0.0004 85.0 9690 1.3437 0.4963 0.6389 0.5586 0.8546
0.0004 86.0 9804 1.3193 0.5155 0.6505 0.5752 0.8577
0.0004 87.0 9918 1.3579 0.4919 0.6420 0.5570 0.8520
0.0003 88.0 10032 1.3109 0.5181 0.6494 0.5764 0.8585
0.0003 89.0 10146 1.2943 0.4876 0.6441 0.5551 0.8615
0.0003 90.0 10260 1.3132 0.4826 0.6463 0.5526 0.8574
0.0003 91.0 10374 1.3176 0.5058 0.6452 0.5671 0.8563
0.0003 92.0 10488 1.3130 0.4964 0.6463 0.5615 0.8583
0.0002 93.0 10602 1.3220 0.4992 0.6463 0.5633 0.8563
0.0002 94.0 10716 1.3160 0.5 0.6463 0.5638 0.8603
0.0002 95.0 10830 1.3155 0.4857 0.6473 0.5550 0.8615
0.0002 96.0 10944 1.3223 0.4912 0.6484 0.5589 0.8577
0.0002 97.0 11058 1.3138 0.5087 0.6484 0.5701 0.8593
0.0002 98.0 11172 1.3152 0.5037 0.6463 0.5661 0.8594
0.0002 99.0 11286 1.3137 0.4996 0.6452 0.5631 0.8593
0.0002 100.0 11400 1.3136 0.4988 0.6452 0.5626 0.8593

Framework versions

  • Transformers 4.25.1
  • Pytorch 1.13.0
  • Datasets 2.8.0
  • Tokenizers 0.12.1
Downloads last month
1
Inference API
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.