PHICON : Improving Generalization of Clinical Text De-identification Models via Data Augmentation (2020)

Abstract: De-identification is the task of identifying protected health information (PHI) in the clinical text. Existing neural de-identification models often fail to generalize to a new dataset. We propose a simple yet effective data augmentation method PHICON to alleviate the generalization issue. PHICON consists of PHI augmentation and Context augmentation, which creates augmented training corpora by replacing PHI entities with named-entities sampled from external sources, and by changing background context with synonym replacement or random word insertion, respectively. Experimental results on the i2b2 2006 and 2014 de-identification challenge datasets show that PHICON can help three selected de-identification models boost F1-score (by at most 8.6%) on cross-dataset test setting. We also discuss how much augmentation to use and how each augmentation method influences the performance.

## introduction clinical text in electronic health records (ehrs) often contain sensitive information. in the united states, health insurance portability and accountability act (hippa) foot_1 requires that protected health information (phi) (e.g., name, street address, phone number) must be removed before ehrs are shared for secondary uses such as clinical research @xcite . the task of identifying and removing phi from clinical texts is referred as de-identification. although many neural de-idenfication models such as lstm-based @xcite @xcite and bert-based @xcite have achieved very promising performance, identifying phi still remains challenging in the real-world scenario: even well-trained models often fail to generalize to a new dataset. for example, we conduct cross-dataset test on i2b2 2006 and i2b2 2014 de-identification challenge datasets foot_2 (i.e., train a widely-used de-identification model neu-roner @xcite on one dataset and test it on the other one). the result in figure 1 shows that model's performance (f1-score) on the new dataset decreases up to 33% compared to the original test set. the poor generalization issue on de-identification is also reported in previous studies @xcite @xcite . to explore what factors lead to poor generalization, we sample some error examples and find that the model might focus too much on specific entities and does not really learn language patterns well. for example, in figure 2 , given a sentence "she met washington in the ohio hospital", the model tends to recognize the entity "washington" as the "location" instead of the "name" if "washington" appears as "location" in the training many times. such cases appear more frequently in a new testing set, thus leading to poor generalization. to prevent the model overfitting on specific cases and encourage it to learn general language patterns, one possible way is to enlarge training data @xcite . however, clinical texts are usually difficult to obtain, not to mention the requirement of tremendous expert effort for annotations @xcite . to solve this, we introduce our data augmentation method phicon, which consists of phi augmentation and context augmentation. specifically, phi augmentation replaces the original phi entity in the training set with a same type named-entity sampled from external sources (such as wikipedia). for example, in figure 2 , "ohio hospital" is replaced by an randomly-sampled "hospital" entity "alaska health center". in terms of context aug- ## phicon to understand what factors lead to the poor generalization, we check some error examples and find that most of the phi entities in these error examples do not appear in training set or appear as a different phi type (e.g., washington [name v.s. location]). we argue that neural models might focus on too sr phi aug context aug she met washington [name] in the ohio hospital [hospital] original sentence she met william [name] in the alaska health center [hospital] ri she saw washington [name] in the ohio hospital [hospital] she happily met washington [name] in the ohio hospital [hospital] . much on specific entities (e.g., recognizing "washington" as "location") but fail to learn general language patterns (e.g., "met" is not usually followed by a "location" entity but a "name" entity instead). consequently, such unseen or out-of-vocabulary phi entities might be hard to be identified correctly, thus leading to lower performance. to help models better identify these unseen phi entities, we may encourage models to learn contextual patterns or linguistic characteristics and prevent models focusing too much on specific phi tokens. ## conclusion in this paper, we explore the generalization issue on clinical text de-identification task. we propose a data augmentation method named phicon that augments both phi and context to boost model generalization. the augmented data can increase data diversity and enrich contextual patterns in training data, which may prevent the model overfitting on specific phi entities and encourage it to focus more on language patterns. experimental results demonstrate that our phicon can help improve models' generalizability, especially in the low-resource training case (i.e., the size of the original training set is small). we also discuss how much augmentation to use and how each augmentation method influences the performance. in the future research, we will explore more advanced data augmentation techniques for improving the de-identification models' generalization performance.