metadata
license: apache-2.0
base_model: GerMedBERT/medbert-512
model-index:
- name: GerMedBERT_NER_V01_BRONCO_CARDIO
results: []
datasets:
- bigbio/bronco
- bigbio/cardiode
language:
- de
metrics:
- f1
- precision
- recall
GerMedBERT_NER_V01_BRONCO_CARDIO
This model is a fine-tuned version of GerMedBERT/medbert-512 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.0306
- Diag: {'precision': 0.7065217391304348, 'recall': 0.6345885634588564, 'f1': 0.6686260102865541, 'number': 717}
- Med: {'precision': 0.8060029282576867, 'recall': 0.7315614617940199, 'f1': 0.7669801462904912, 'number': 1505}
- Treat: {'precision': 0.8133640552995391, 'recall': 0.7431578947368421, 'f1': 0.7766776677667767, 'number': 475}
- Overall Precision: 0.7811
- Overall Recall: 0.7078
- Overall F1: 0.7427
- Overall Accuracy: 0.9903
- Num Input Tokens Seen: 11575975
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4
Training results
| Training Loss | Epoch | Step | Validation Loss | Diag | Med | Treat | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | Input Tokens Seen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.0611 | 0.2496 | 303 | 0.0509 | {'precision': 0.6265060240963856, 'recall': 0.2900976290097629, 'f1': 0.3965681601525262, 'number': 717} | {'precision': 0.7679127725856698, 'recall': 0.3275747508305648, 'f1': 0.45924545877969264, 'number': 1505} | {'precision': 0.8493150684931506, 'recall': 0.5221052631578947, 'f1': 0.6466753585397653, 'number': 475} | 0.7496 | 0.3519 | 0.4789 | 0.9841 | 725328 |
| 0.0532 | 0.4992 | 606 | 0.0430 | {'precision': 0.7558139534883721, 'recall': 0.36262203626220363, 'f1': 0.4901036757775683, 'number': 717} | {'precision': 0.8224076281287247, 'recall': 0.4584717607973422, 'f1': 0.5887372013651877, 'number': 1505} | {'precision': 0.7891566265060241, 'recall': 0.5515789473684211, 'f1': 0.6493184634448574, 'number': 475} | 0.8 | 0.4494 | 0.5755 | 0.9860 | 1436640 |
| 0.0488 | 0.7488 | 909 | 0.0394 | {'precision': 0.6588486140724946, 'recall': 0.4309623430962343, 'f1': 0.521079258010118, 'number': 717} | {'precision': 0.803639846743295, 'recall': 0.5574750830564784, 'f1': 0.6582973715182425, 'number': 1505} | {'precision': 0.8328445747800587, 'recall': 0.5978947368421053, 'f1': 0.696078431372549, 'number': 475} | 0.7724 | 0.5310 | 0.6293 | 0.9872 | 2157328 |
| 0.0342 | 0.9984 | 1212 | 0.0361 | {'precision': 0.6908713692946058, 'recall': 0.46443514644351463, 'f1': 0.5554628857381151, 'number': 717} | {'precision': 0.76010101010101, 'recall': 0.6, 'f1': 0.6706275529149647, 'number': 1505} | {'precision': 0.8910256410256411, 'recall': 0.5852631578947368, 'f1': 0.7064803049555274, 'number': 475} | 0.7639 | 0.5614 | 0.6471 | 0.9873 | 2891248 |
| 0.0347 | 1.2479 | 1515 | 0.0368 | {'precision': 0.6760828625235404, 'recall': 0.500697350069735, 'f1': 0.5753205128205129, 'number': 717} | {'precision': 0.7350936967632027, 'recall': 0.573421926910299, 'f1': 0.6442702500933185, 'number': 1505} | {'precision': 0.7641277641277642, 'recall': 0.6547368421052632, 'f1': 0.7052154195011338, 'number': 475} | 0.7259 | 0.5684 | 0.6376 | 0.9871 | 3607825 |
| 0.0283 | 1.4975 | 1818 | 0.0351 | {'precision': 0.6774193548387096, 'recall': 0.5564853556485355, 'f1': 0.6110260336906584, 'number': 717} | {'precision': 0.7513134851138353, 'recall': 0.5700996677740864, 'f1': 0.6482810729127314, 'number': 1505} | {'precision': 0.8045685279187818, 'recall': 0.6673684210526316, 'f1': 0.7295742232451093, 'number': 475} | 0.7407 | 0.5836 | 0.6528 | 0.9872 | 4320401 |
| 0.0319 | 1.7471 | 2121 | 0.0329 | {'precision': 0.6723809523809524, 'recall': 0.49232914923291493, 'f1': 0.5684380032206119, 'number': 717} | {'precision': 0.7881619937694704, 'recall': 0.6724252491694352, 'f1': 0.7257081391179634, 'number': 1505} | {'precision': 0.8387978142076503, 'recall': 0.6463157894736842, 'f1': 0.7300832342449465, 'number': 475} | 0.7687 | 0.6199 | 0.6864 | 0.9885 | 5050561 |
| 0.0269 | 1.9967 | 2424 | 0.0311 | {'precision': 0.720353982300885, 'recall': 0.5676429567642957, 'f1': 0.6349453978159126, 'number': 717} | {'precision': 0.7833850931677019, 'recall': 0.6704318936877076, 'f1': 0.7225205871822412, 'number': 1505} | {'precision': 0.8696883852691218, 'recall': 0.6463157894736842, 'f1': 0.7415458937198067, 'number': 475} | 0.7811 | 0.6389 | 0.7028 | 0.9891 | 5776705 |
| 0.0268 | 2.2463 | 2727 | 0.0309 | {'precision': 0.6769706336939721, 'recall': 0.6108786610878661, 'f1': 0.6422287390029325, 'number': 717} | {'precision': 0.7624466571834992, 'recall': 0.7122923588039867, 'f1': 0.7365166609412571, 'number': 1505} | {'precision': 0.8233830845771144, 'recall': 0.6968421052631579, 'f1': 0.7548460661345495, 'number': 475} | 0.7499 | 0.6826 | 0.7147 | 0.9891 | 6493709 |
| 0.0265 | 2.4959 | 3030 | 0.0319 | {'precision': 0.7138103161397671, 'recall': 0.5983263598326359, 'f1': 0.6509863429438543, 'number': 717} | {'precision': 0.7537202380952381, 'recall': 0.6730897009966778, 'f1': 0.7111267111267112, 'number': 1505} | {'precision': 0.8165829145728644, 'recall': 0.6842105263157895, 'f1': 0.7445589919816724, 'number': 475} | 0.7542 | 0.6552 | 0.7012 | 0.9888 | 7214269 |
| 0.0255 | 2.7455 | 3333 | 0.0314 | {'precision': 0.6806853582554517, 'recall': 0.6094839609483961, 'f1': 0.643119941133186, 'number': 717} | {'precision': 0.7615062761506276, 'recall': 0.7255813953488373, 'f1': 0.7431099013269821, 'number': 1505} | {'precision': 0.7866666666666666, 'recall': 0.7452631578947368, 'f1': 0.7654054054054054, 'number': 475} | 0.7454 | 0.6982 | 0.7210 | 0.9892 | 7947645 |
| 0.0221 | 2.9951 | 3636 | 0.0295 | {'precision': 0.723916532905297, 'recall': 0.6290097629009763, 'f1': 0.673134328358209, 'number': 717} | {'precision': 0.8135464231354642, 'recall': 0.7102990033222591, 'f1': 0.7584249733948208, 'number': 1505} | {'precision': 0.85, 'recall': 0.7157894736842105, 'f1': 0.7771428571428571, 'number': 475} | 0.7959 | 0.6897 | 0.7390 | 0.9903 | 8667437 |
| 0.018 | 3.2446 | 3939 | 0.0307 | {'precision': 0.7097288676236044, 'recall': 0.6206415620641562, 'f1': 0.6622023809523809, 'number': 717} | {'precision': 0.7909156452775775, 'recall': 0.7289036544850498, 'f1': 0.7586445366528355, 'number': 1505} | {'precision': 0.8165137614678899, 'recall': 0.7494736842105263, 'f1': 0.7815587266739846, 'number': 475} | 0.7747 | 0.7037 | 0.7375 | 0.9901 | 9388513 |
| 0.0238 | 3.4942 | 4242 | 0.0312 | {'precision': 0.7024922118380063, 'recall': 0.6290097629009763, 'f1': 0.6637233259749816, 'number': 717} | {'precision': 0.781895937277263, 'recall': 0.7289036544850498, 'f1': 0.7544704264099036, 'number': 1505} | {'precision': 0.8235294117647058, 'recall': 0.7368421052631579, 'f1': 0.7777777777777778, 'number': 475} | 0.7684 | 0.7037 | 0.7347 | 0.9898 | 10103889 |
| 0.0196 | 3.7438 | 4545 | 0.0303 | {'precision': 0.7142857142857143, 'recall': 0.6276150627615062, 'f1': 0.6681514476614699, 'number': 717} | {'precision': 0.7932761087267525, 'recall': 0.7368770764119601, 'f1': 0.7640372028935583, 'number': 1505} | {'precision': 0.8273381294964028, 'recall': 0.7263157894736842, 'f1': 0.773542600896861, 'number': 475} | 0.7787 | 0.7060 | 0.7406 | 0.9902 | 10831905 |
| 0.0184 | 3.9934 | 4848 | 0.0306 | {'precision': 0.7065217391304348, 'recall': 0.6345885634588564, 'f1': 0.6686260102865541, 'number': 717} | {'precision': 0.8054133138258961, 'recall': 0.7315614617940199, 'f1': 0.7667130919220054, 'number': 1505} | {'precision': 0.8133640552995391, 'recall': 0.7431578947368421, 'f1': 0.7766776677667767, 'number': 475} | 0.7808 | 0.7078 | 0.7425 | 0.9903 | 11559985 |
Framework versions
- Transformers 4.40.1
- Pytorch 2.3.0+cu121
- Datasets 2.19.0
- Tokenizers 0.19.1