File size: 12,868 Bytes
03802b4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6378eeb
 
 
 
 
 
 
 
03802b4
 
 
6378eeb
 
 
03802b4
 
 
 
 
 
6378eeb
 
 
03802b4
 
 
6378eeb
 
 
03802b4
 
 
 
 
 
6378eeb
 
 
03802b4
 
 
6378eeb
 
 
03802b4
 
 
 
 
 
6378eeb
 
03802b4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6378eeb
 
 
 
 
 
 
 
 
 
03802b4
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
---
license: apache-2.0
base_model: distilbert-base-uncased
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: distilbert-base-uncased-finetuned-ner-cadec-active
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-base-uncased-finetuned-ner-cadec-active

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3506
- Precision: 0.5141
- Recall: 0.5609
- F1: 0.5365
- Accuracy: 0.9010
- Adr Precision: 0.4552
- Adr Recall: 0.5683
- Adr F1: 0.5055
- Disease Precision: 0.0
- Disease Recall: 0.0
- Disease F1: 0.0
- Drug Precision: 0.7659
- Drug Recall: 0.8351
- Drug F1: 0.7990
- Finding Precision: 0.0
- Finding Recall: 0.0
- Finding F1: 0.0
- Symptom Precision: 0.0
- Symptom Recall: 0.0
- Symptom F1: 0.0
- B-adr Precision: 0.6573
- B-adr Recall: 0.7339
- B-adr F1: 0.6935
- B-disease Precision: 0.0
- B-disease Recall: 0.0
- B-disease F1: 0.0
- B-drug Precision: 0.9318
- B-drug Recall: 0.8723
- B-drug F1: 0.9011
- B-finding Precision: 0.0
- B-finding Recall: 0.0
- B-finding F1: 0.0
- B-symptom Precision: 0.0
- B-symptom Recall: 0.0
- B-symptom F1: 0.0
- I-adr Precision: 0.4521
- I-adr Recall: 0.5422
- I-adr F1: 0.4931
- I-disease Precision: 0.0
- I-disease Recall: 0.0
- I-disease F1: 0.0
- I-drug Precision: 0.7960
- I-drug Recall: 0.8556
- I-drug F1: 0.8247
- I-finding Precision: 0.0
- I-finding Recall: 0.0
- I-finding F1: 0.0
- I-symptom Precision: 0.0
- I-symptom Recall: 0.0
- I-symptom F1: 0.0
- Macro Avg F1: 0.2912
- Weighted Avg F1: 0.5966

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy | Adr Precision | Adr Recall | Adr F1 | Disease Precision | Disease Recall | Disease F1 | Drug Precision | Drug Recall | Drug F1 | Finding Precision | Finding Recall | Finding F1 | Symptom Precision | Symptom Recall | Symptom F1 | B-adr Precision | B-adr Recall | B-adr F1 | B-disease Precision | B-disease Recall | B-disease F1 | B-drug Precision | B-drug Recall | B-drug F1 | B-finding Precision | B-finding Recall | B-finding F1 | B-symptom Precision | B-symptom Recall | B-symptom F1 | I-adr Precision | I-adr Recall | I-adr F1 | I-disease Precision | I-disease Recall | I-disease F1 | I-drug Precision | I-drug Recall | I-drug F1 | I-finding Precision | I-finding Recall | I-finding F1 | I-symptom Precision | I-symptom Recall | I-symptom F1 | Macro Avg F1 | Weighted Avg F1 |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|:-------------:|:----------:|:------:|:-----------------:|:--------------:|:----------:|:--------------:|:-----------:|:-------:|:-----------------:|:--------------:|:----------:|:-----------------:|:--------------:|:----------:|:---------------:|:------------:|:--------:|:-------------------:|:----------------:|:------------:|:----------------:|:-------------:|:---------:|:-------------------:|:----------------:|:------------:|:-------------------:|:----------------:|:------------:|:---------------:|:------------:|:--------:|:-------------------:|:----------------:|:------------:|:----------------:|:-------------:|:---------:|:-------------------:|:----------------:|:------------:|:-------------------:|:----------------:|:------------:|:------------:|:---------------:|
| No log        | 1.0   | 26   | 0.6925          | 0.0       | 0.0    | 0.0    | 0.7877   | 0.0           | 0.0        | 0.0    | 0.0               | 0.0            | 0.0        | 0.0            | 0.0         | 0.0     | 0.0               | 0.0            | 0.0        | 0.0               | 0.0            | 0.0        | 0.0             | 0.0          | 0.0      | 0.0                 | 0.0              | 0.0          | 0.0              | 0.0           | 0.0       | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.0             | 0.0          | 0.0      | 0.0                 | 0.0              | 0.0          | 0.0              | 0.0           | 0.0       | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.0          | 0.0             |
| No log        | 2.0   | 52   | 0.4818          | 0.3775    | 0.2508 | 0.3014 | 0.8527   | 0.2356        | 0.1788     | 0.2033 | 0.0               | 0.0            | 0.0        | 0.9606         | 0.6489      | 0.7746  | 0.0               | 0.0            | 0.0        | 0.0               | 0.0            | 0.0        | 0.0             | 0.0          | 0.0      | 0.0                 | 0.0              | 0.0          | 0.9841           | 0.6596        | 0.7898    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.0632          | 0.0592       | 0.0612   | 0.0                 | 0.0              | 0.0          | 0.9685           | 0.6578        | 0.7834    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.1634       | 0.1890          |
| No log        | 3.0   | 78   | 0.4090          | 0.3740    | 0.3234 | 0.3469 | 0.8761   | 0.2779        | 0.2747     | 0.2763 | 0.0               | 0.0            | 0.0        | 0.7697         | 0.6755      | 0.7195  | 0.0               | 0.0            | 0.0        | 0.0               | 0.0            | 0.0        | 0.6431          | 0.2583       | 0.3685   | 0.0                 | 0.0              | 0.0          | 0.9524           | 0.7447        | 0.8358    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.1645          | 0.1813       | 0.1725   | 0.0                 | 0.0              | 0.0          | 0.8375           | 0.7166        | 0.7723    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.2149       | 0.3628          |
| No log        | 4.0   | 104  | 0.3834          | 0.4128    | 0.3562 | 0.3824 | 0.8813   | 0.3096        | 0.3038     | 0.3067 | 0.0               | 0.0            | 0.0        | 0.8274         | 0.7394      | 0.7809  | 0.0               | 0.0            | 0.0        | 0.0               | 0.0            | 0.0        | 0.6346          | 0.4047       | 0.4942   | 0.0                 | 0.0              | 0.0          | 0.9664           | 0.7660        | 0.8546    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.2382          | 0.2442       | 0.2411   | 0.0                 | 0.0              | 0.0          | 0.8735           | 0.7754        | 0.8215    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.2411       | 0.4379          |
| No log        | 5.0   | 130  | 0.3587          | 0.4437    | 0.4882 | 0.4649 | 0.8960   | 0.3705        | 0.4782     | 0.4175 | 0.0               | 0.0            | 0.0        | 0.7914         | 0.7872      | 0.7893  | 0.0               | 0.0            | 0.0        | 0.0               | 0.0            | 0.0        | 0.6234          | 0.6803       | 0.6506   | 0.0                 | 0.0              | 0.0          | 0.9682           | 0.8085        | 0.8812    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.3689          | 0.4345       | 0.3990   | 0.0                 | 0.0              | 0.0          | 0.8065           | 0.8021        | 0.8043    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.2735       | 0.5465          |
| No log        | 6.0   | 156  | 0.3549          | 0.4600    | 0.5005 | 0.4794 | 0.8960   | 0.3876        | 0.4913     | 0.4333 | 0.0               | 0.0            | 0.0        | 0.7906         | 0.8032      | 0.7968  | 0.0               | 0.0            | 0.0        | 0.0               | 0.0            | 0.0        | 0.6281          | 0.7102       | 0.6667   | 0.0                 | 0.0              | 0.0          | 0.9625           | 0.8191        | 0.8851    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.3861          | 0.4596       | 0.4197   | 0.0                 | 0.0              | 0.0          | 0.8211           | 0.8342        | 0.8276    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.2799       | 0.5619          |
| No log        | 7.0   | 182  | 0.3583          | 0.4842    | 0.5333 | 0.5075 | 0.9013   | 0.4205        | 0.5305     | 0.4692 | 0.0               | 0.0            | 0.0        | 0.75           | 0.8298      | 0.7879  | 0.0               | 0.0            | 0.0        | 0.0               | 0.0            | 0.0        | 0.6412          | 0.7260       | 0.6809   | 0.0                 | 0.0              | 0.0          | 0.9310           | 0.8617        | 0.8950    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.4238          | 0.4794       | 0.4499   | 0.0                 | 0.0              | 0.0          | 0.7892           | 0.8610        | 0.8235    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.2849       | 0.5774          |
| No log        | 8.0   | 208  | 0.3442          | 0.4954    | 0.5455 | 0.5192 | 0.8986   | 0.4320        | 0.5451     | 0.4820 | 0.0               | 0.0            | 0.0        | 0.7670         | 0.8404      | 0.8020  | 0.0               | 0.0            | 0.0        | 0.0               | 0.0            | 0.0        | 0.6561          | 0.7150       | 0.6843   | 0.0                 | 0.0              | 0.0          | 0.9326           | 0.8830        | 0.9071    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.4279          | 0.5224       | 0.4705   | 0.0                 | 0.0              | 0.0          | 0.8010           | 0.8610        | 0.8299    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.2892       | 0.5872          |
| No log        | 9.0   | 234  | 0.3499          | 0.5075    | 0.5517 | 0.5287 | 0.9003   | 0.4480        | 0.5567     | 0.4964 | 0.0               | 0.0            | 0.0        | 0.7610         | 0.8298      | 0.7939  | 0.0               | 0.0            | 0.0        | 0.0               | 0.0            | 0.0        | 0.6585          | 0.7228       | 0.6892   | 0.0                 | 0.0              | 0.0          | 0.9314           | 0.8670        | 0.8981    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.4466          | 0.5404       | 0.4890   | 0.0                 | 0.0              | 0.0          | 0.7960           | 0.8556        | 0.8247    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.2901       | 0.5934          |
| No log        | 10.0  | 260  | 0.3506          | 0.5141    | 0.5609 | 0.5365 | 0.9010   | 0.4552        | 0.5683     | 0.5055 | 0.0               | 0.0            | 0.0        | 0.7659         | 0.8351      | 0.7990  | 0.0               | 0.0            | 0.0        | 0.0               | 0.0            | 0.0        | 0.6573          | 0.7339       | 0.6935   | 0.0                 | 0.0              | 0.0          | 0.9318           | 0.8723        | 0.9011    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.4521          | 0.5422       | 0.4931   | 0.0                 | 0.0              | 0.0          | 0.7960           | 0.8556        | 0.8247    | 0.0                 | 0.0              | 0.0          | 0.0                 | 0.0              | 0.0          | 0.2912       | 0.5966          |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.15.0
- Tokenizers 0.15.0