Towards Task-Agnostic Privacy- and Utility-Preserving Models (2021)

Abstract: Modern deep learning models for natural language processing rely heavily on large amounts of annotated texts. However, obtaining such texts may be difficult when they contain personal or confidential information, for example, in health or legal domains. In this work, we propose a method of de-identifying free-form text documents by carefully redacting sensitive data in them. We show that our method preserves data utility for text classification, sequence labeling and question answering tasks.

## introduction data privacy has become an important topic recently, and new regulations that govern the processing and usage of consumer personal data arise every year. we use the term sensitive data to define all information that contains personal data or other potentially compromising data that can lead to the re-identification of individuals or organizations. with the advent of machine learning, privacy and compliant data governance have become even more important. free-form text documents often contain sensitive data. for example, legal documents contain full information about individuals or organizations, dialogues contain references to different people and possibly information that can lead to their identification, such as addresses or job positions. nlp applications such as text classification or question answering require annotated data, and as @xcite argues, the privacy cost of annotating texts must be considered when developing new applications. anonymization, or de-identification, is viewed as one of the ways to make working with nonpublic datasets both safer and more compliant. it is also important to preserve utility in de-identified data for further usages, e.g. machine learning or analytics. traditional approaches like k-anonymity @xcite were shown to be ineffective when applied to high-dimensional media @xcite for example, free-form texts. moreover, unlike structured data present in relational databases, anonymization of unstructured text documents poses additional challenges, because locations of textual spans which constitute sensitive data must be inferred at runtime. previous work either focused on preventing the model from memorizing data @xcite or considered only the text classification task. in this paper, we aim to solve the challenges of preserving both privacy and utility in free-form text documents with respect to different nlp tasks. we propose a simple "find-and-replace" method for automatically de-identifying documents and evaluate the utility of the de-identified data with respect to different downstream tasks, namely named entity recognition, question answering and text classification. to the best of our knowledge, this is the first work that evaluates the utility of the de-identified data for sequence labeling and reading comprehension tasks. specifically, our contributions are as follows: ## proposed method we aim to develop a method that performs finegrained substitutions of text spans comprising sensitive information. contexts of sensitive data are kept intact and therefore our method preserves as much original text as possible. ## suggestions to practitioners our experiments highlight several characteristics of anonymization procedure, downstream task and model architecture that should be taken into account when using anonymized data for training nlp models. our suggestions are as follows: ## conclusions and future work in this work, we consider the practical side of anonymizing unstructured documents while simultaneously preserving their utility for different downstream tasks. new policies regarding personal data make privacy research a more important topic over the years. we anticipate that in the near future de-identification of sensitive data before training will become a necessity. we hope our work will pave the way for investigating broader impact and limitations of free-form text anonymization. we demonstrate that pseudonymization mostly preserves data utility for different extractive nlp tasks. we show it is possible to achieve close results with the model trained only on the deidentified data. however, it is not yet clear whether our results transfer to generative tasks and more complex settings, for example, scenarios with multiple languages like machine translation or multilingual datasets. we believe this is the promising research direction.