prajwal967 commited on
Commit
abdcb27
1 Parent(s): a70ff1c

updated readme

Browse files
Files changed (1) hide show
  1. README.md +63 -4
README.md CHANGED
@@ -1,10 +1,12 @@
1
  ---
2
  language:
3
- - en
4
  thumbnail: "https://www.onebraveidea.org/wp-content/uploads/2019/07/OBI-Logo-Website.png"
5
  tags:
6
  - deidentification
7
  - medical notes
 
 
8
  datasets:
9
  - I2B2
10
  metrics:
@@ -15,10 +17,67 @@ widget:
15
  - text: "Physician Discharge Summary Admit date: 10/12/1982 Discharge date: 10/22/1982 Patient Information Jack Reacher, 54 y.o. male (DOB = 1/21/1928)."
16
  - text: "Home Address: 123 Park Drive, San Diego, CA, 03245. Home Phone: 202-555-0199 (home)."
17
  - text: "Hospital Care Team Service: Orthopedics Inpatient Attending: Roger C Kelly, MD Attending phys phone: (634)743-5135 Discharge Unit: HCS843 Primary Care Physician: Hassan V Kim, MD 512-832-5025."
 
18
  ---
19
 
20
 
21
- # BERT-based deidentification model
22
 
23
- This repo contains model weights for clinical note de-deidentification trained on the I2B2 dataset. Note that the hosted inference API uses a different tokenizer than what we developed for this task.
24
- Please see [OBI EHR deidentification](https://github.com/obi-ds/ehr_deidentification) for more details and how to get started.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  language:
3
+ - english
4
  thumbnail: "https://www.onebraveidea.org/wp-content/uploads/2019/07/OBI-Logo-Website.png"
5
  tags:
6
  - deidentification
7
  - medical notes
8
+ - ehr
9
+ - phi
10
  datasets:
11
  - I2B2
12
  metrics:
 
17
  - text: "Physician Discharge Summary Admit date: 10/12/1982 Discharge date: 10/22/1982 Patient Information Jack Reacher, 54 y.o. male (DOB = 1/21/1928)."
18
  - text: "Home Address: 123 Park Drive, San Diego, CA, 03245. Home Phone: 202-555-0199 (home)."
19
  - text: "Hospital Care Team Service: Orthopedics Inpatient Attending: Roger C Kelly, MD Attending phys phone: (634)743-5135 Discharge Unit: HCS843 Primary Care Physician: Hassan V Kim, MD 512-832-5025."
20
+ license: mit
21
  ---
22
 
23
 
24
+ # Model Description
25
 
26
+ * A ClinicalBERT [Alsentzer et al., 2019](https://arxiv.org/pdf/1904.03323.pdf) model fine-tuned for de-identification of medical notes.
27
+ * Sequence Labeling (token classification): The model was trained to predict protected health information (PHI/PII) entities (spans). A list of protected health information categories is given by [HIPAA](https://www.hhs.gov/hipaa/for-professionals/privacy/laws-regulations/index.html).
28
+ * A token can either be classified as non-PHI or as one of the 11 PHI types. Token predictions can be aggregated to span (e.g., making use of BILOU tagging).
29
+ * The PHI labels that were used for training and other details can be found here: [Annotation Guidelines](https://github.com/obi-ml-public/ehr_deidentification/blob/master/AnnotationGuidelines.md)
30
+ * More details on how to use this model, the format of data and other useful information is present in the GitHub repo: [Robust DeID](https://github.com/obi-ml-public/ehr_deidentification).
31
+
32
+
33
+ # How to use
34
+
35
+ * A demo on how the model works (using model predictions to de-identify a medical note) is on this space: [Medical-Note-Deidentification](https://huggingface.co/spaces/obi/Medical-Note-Deidentification).
36
+ * Steps on how this model can be used to run a forward pass can be found here: [Forward Pass](https://github.com/obi-ml-public/ehr_deidentification/tree/master/steps/forward_pass)
37
+ * In brief, the steps are:
38
+ * Sentencize (the model aggregates the sentences back to the note level) and tokenize the dataset.
39
+ * Use the predict function of this model to gather the predictions (i.e., predictions for each token).
40
+ * Additionally, the model predictions can be used to remove PHI from the original note/text.
41
+
42
+
43
+ # Dataset
44
+
45
+ * The I2B2 2014 [Stubbs and Uzuner, 2015](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4978170/) dataset was used to train this model.
46
+
47
+ | | I2B2 | | I2B2 | |
48
+ | --------- | --------------------- | ---------- | -------------------- | ---------- |
49
+ | | TRAIN SET - 790 NOTES | | TEST SET - 514 NOTES | |
50
+ | PHI LABEL | COUNT | PERCENTAGE | COUNT | PERCENTAGE |
51
+ | DATE | 7502 | 43.69 | 4980 | 44.14 |
52
+ | STAFF | 3149 | 18.34 | 2004 | 17.76 |
53
+ | HOSP | 1437 | 8.37 | 875 | 7.76 |
54
+ | AGE | 1233 | 7.18 | 764 | 6.77 |
55
+ | LOC | 1206 | 7.02 | 856 | 7.59 |
56
+ | PATIENT | 1316 | 7.66 | 879 | 7.79 |
57
+ | PHONE | 317 | 1.85 | 217 | 1.92 |
58
+ | ID | 881 | 5.13 | 625 | 5.54 |
59
+ | PATORG | 124 | 0.72 | 82 | 0.73 |
60
+ | EMAIL | 4 | 0.02 | 1 | 0.01 |
61
+ | OTHERPHI | 2 | 0.01 | 0 | 0 |
62
+ | TOTAL | 17171 | 100 | 11283 | 100 |
63
+
64
+
65
+ # Training procedure
66
+
67
+ * Steps on how this model was trained can be found here: [Training](https://github.com/obi-ml-public/ehr_deidentification/tree/master/steps/train). The "model_name_or_path" was set to: "emilyalsentzer/Bio_ClinicalBERT".
68
+ * The dataset was sentencized with the en_core_sci_sm sentencizer from spacy.
69
+ * The dataset was then tokenized with a custom tokenizer built on top of the en_core_sci_sm tokenizer from spacy.
70
+ * For each sentence we added 32 tokens on the left (from previous sentences) and 32 tokens on the right (from the next sentences).
71
+ * The added tokens are not used for learning - i.e, the loss is not computed on these tokens - they are used as additional context.
72
+ * Each sequence contained a maximum of 128 tokens (including the 32 tokens added on). Longer sequences were split.
73
+ * The sentencized and tokenized dataset with the token level labels based on the BILOU notation was used to train the model.
74
+ * The model is fine-tuned from a pre-trained RoBERTa model.
75
+
76
+ * Training details:
77
+ * Input sequence length: 128
78
+ * Batch size: 32
79
+ * Optimizer: AdamW
80
+ * Learning rate: 4e-5
81
+ * Dropout: 0.1
82
+
83
+ # Results