Evaluating the Impact of Text De-Identification on Downstream NLP Tasks (2023)

Abstract: Data anonymisation is often required to comply with regulations when transfering information across departments or entities. However, the risk is that this procedure can distort the data and jeopardise the models built on it. Intuitively, the process of training an NLP model on anonymised data may lower the performance of the resulting model when compared to a model trained on non-anonymised data. In this paper, we investigate the impact of de-identification on the performance of nine downstream NLP tasks. We focus on the anonymisation and pseudonymisation of personal names and compare six different anonymisation strategies for two state-of-the-art pre-trained models. Based on these experiments, we formulate recommendations on how the de-identification should be performed to guarantee accurate NLP models. Our results reveal that de-identification does have a negative impact on the performance of NLP models, but this impact is relatively low. We also find that using pseudonymisation techniques involving random names leads to better performance across most tasks.

## introduction protection of personal data has been a hot topic for decades @xcite . careless sharing of data between companies, cyber-attacks, and other data breaches can lead to catastrophic leaks of confidential data, potentially resulting in the invasion of people's privacy and identity theft. to mitigate damages and hold bad actors accountable, many countries introduced various laws that aim to protect confidential data, such as the health insurance portability and accountability act (hipaa) for healthcare confidentiality @xcite , and the gramm-leach-bliley act (glba) in the financial domain @xcite . most notably, with the introduction of the general data protection regulation (gdpr), the protection of personally identifiable information was codified into eu law. @xcite failure to comply with these regulations can lead to huge fines in case of a data breach. indeed, the amount of fines for gdpr violations adds up to over 1.5 trillion euros with the largest single fine of 746 million euros being imposed on amazon. 1 in order to mitigate data leaks, organisations such as financial institutes and hospitals are required to anonymise or pseudonymise sensitive data before processing them further. similarly, automated nlp models should ideally be trained using anonymised data as resulting models could potentially violate a number of gdpr guidelines such as the individuals' right to be forgotten, and the right to explanation. furthermore, models can be manipulated to partially recreate the training data @xcite , which can result in disastrous data breaches. on the other hand, however, anonymisation of texts can lead to loss of information and meaning, making nlp models trained on anonymised data less reliable as a result @xcite . intuitively, this in turn could lead to a decrease in performance of such models when compared to models trained on non-anonymised text. as such, it is crucial to choose an appropriate anonymisation strategy to lower this loss of information and avoid performance drops of models. in this study, we investigate the impact of text deidentification on the performance of downstream nlp tasks, focusing on the anonymisation and pseudonymisation of person names only. this allows us to select from a wide array of nlp tasks as most datasets contain a large number of person names, whereas other types of names are less commonly found. specifically, we compare six different anonymisation strategies, and two transformerbased pre-trained model architectures in our experiments: the popular bert @xcite architecture and the state-of-the-art ernie @xcite architecture. further, we look into nine different nlp tasks of varying degrees of difficulty. we address the following research questions: ## experimental setup in this section, we present the datasets used in this study and we introduce the different anonymisation strategies that we compare against each other. we also show the pre-trained models we use. ## experimental results in this section, we show the results of our experiments and address the research questions from section 1. for each task and for each pre-trained model, we fine-tune a model on the original dataset and each of our six anonymised datasets. we also de-identify the test sets accordingly and evaluate each model on the corresponding test set. we do five runs for each case, and average the results. we then compare the average performance for each as to the performance of the models trained on original data. in order to determine the most appropriate strategy, we consider two ranking-based approaches: borda count and instant runoff @xcite . for both approaches, we determine the score s a,t for each anonymisation strategy (as, indexed by a) and for each task (indexed by t) in the following way: the best approach gets a score of five, the second best gets a score of four, etc. the final borda count score for a given anonymisation strategy a is defined as t @xmath0,t (where t is the total number of tasks, here, nine). the model with the highest total score is considered the best. instant runoff is an iterative procedure. for each iteration, we count the number of wins for each as, where an as is considered a winner in a given task if its corresponding fine-tuned model outperforms every other model. we then eliminate the as with the lowest number of wins and update the scores accordingly. we repeat this process until one as remains, or until we cannot eliminate further ass. as2 "hi, this is entname1, am i speaking to entname2?" "sorry, no, this is entname1. entname2 is not here today." as3 "hi, this is entname1, am i speaking to entname2?" "sorry, no, this is entname3. entname2 is not here today." as4 "hi, this is , am i speaking to " "sorry, no, this is . is not here today." as5 "hi, this is bert, am i speaking to ernie?" "sorry, no, this is elmo. kermit is not here today." as6 "hi, this is jessie, am i speaking to james?" "sorry, no, this is meowth. james is not here today." when compared to as4. in contrast, when evaluating ernie models, we note that as5 models are performing significantly better than every other strategy according to borda count. similarly, as5 also wins the instant runoff with as4 and as5 making it to the final round. overall, it appears that using random names over generic tokens to de-identify textual data is the preferable solution as as1, as2, as3 models, which were all trained on data with generic tokens, usually rank low. in order to answer this question, we investigate the performance of models trained on original data on the de-identified test sets (cf. ## discussion judging by the results of our experiments, we recommend practitioners to de-identify their sensitive textual data using random names, as they typically lead to the best results among the anonymisation strategies we tested. we also recommend to de-identify data before the training of nlp models. it follows that it is important to keep the deidentification process and naming schemes consistent throughout the entire pipeline that uses the data in order to mitigate potential performance losses of models. it may also be important to keep the number of names sufficiently high in order to avoid introducing bias in the training that may contribute to unfair discrimination against specific names, a well-known issue in machine learning models that handle person names @xcite . ## related work relevant studies done on textual data largely focus on medical texts and on a very limited number of tasks and anonymisation strategies when compared to our work. on the other hand, they typically anonymise a wide variety of protected health information (phi) classes, while our work focuses on anonymisation of persons' names only. @xcite studied the impact of four anonymisation strategies (pseudonymisation, replacement by phi class, masking, and removal) on downstream ner tasks for the clinical domain. similarly to our findings, they find that pseudonymisation yields the best results among the investigated strategies. on the other hand, removal of names resulted in the highest negative impact on the downstream tasks. @xcite investigated the impact of anonymisation on an information extraction task using a dataset of 3503 clinical notes. they anonymised 12 types of phi such as patients' name, age, etc., and used two anonymisation strategies (replacement by fake phi, and masking). they found no significant loss in performance for this task. similarly, @xcite found that the informativeness of medical notes only marginally decreased after anonymisation, using 18 types of phi and 3 anonymisation strategies (replacement by fake phi, replacement by phi class, and replacement by phi token). using the same anonymisation strategies and ten types of @xcite investigated the impact of anonymisation on a mental status classification task. comparing nine different machine learning models, they did not find any significant difference in performance between original and anonymised data. ## conclusion in this paper, we conducted an empirical study analysing the impact of de-identification on downstream nlp tasks. we investigated the difference in performance of six anonymisation strategies on nine nlp tasks ranging from simple classification tasks to hard nlu tasks. further, we compared two architectures, bert and ernie. overall, we found that de-identifying data before training an nlp model does have a negative impact on its performance. however, this impact is relatively low. we determined that pseudonymisation techniques involving random names lead to higher performances across most tasks. specifically, replacing names by random names (as5) had the least negative impact when using an ernie model. similarly, replacing by random names while preserving the link between identical names (as6) worked best for bert models. we also showed that it is advisable to de-identify data prior to training as we observed a large difference in performance between models trained on original data versus de-identified data. there is also a noticeable difference between the performances of bert and ernie, warranting further investigation into the performance differences between a larger number of language models.