Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymization
Year: 2023

Abstract:
This work investigates the effectiveness of different pseudonymization techniques, ranging from rule-based substitutions to using pretrained Large Language Models (LLMs), on a variety of datasets and models used for two widely used NLP tasks: text classification and summarization. Our work provides crucial insights into the gaps between original and anonymized data (focusing on the pseudonymization technique) and model quality and fosters future research into higherquality anonymization techniques to better balance the trade-offs between data protection and utility preservation. We make our code, pseudonymized datasets, and downstream models publicly available.1

heading: 1 Introduction
text: With the advances in artificial intelligence and data-hungry machine learning systems, privacy and compliant data governance have become increasingly important. Text documents, such as emails, court rulings, customer service chats, interview transcripts, and patient records, frequently contain personally identifiable information (PII), such as mentions of persons, locations, organizations, etc. While the collection and use of text data is necessary for improving products or services, conducting research, or providing personalized recommendations, it has to be done in a safe, responsible and compliant way.
However, access to text data becomes a challenge where data containing personally identifiable mentions is involved. Although it is a widely accepted notion that no data is truly anonymous and is said to be an unattainable target (Rocher et al., 2019), pseudonymization, on the other hand, is recognized by the GDPR as one of the
∗Work done as an intern at Grammarly. 1https://github.com/olexandryermilov/
privacy-preserving-nlp
ways (and a requirement) to reduce risks of reidentification of a data subject (European Commission, 2016). Following Eder et al. (2022), we define pseudonymization as recognizing entities bearing privacy-sensitive information, and their replacement by realistic substitutes.
With the right implementation and safeguards, pseudonymization can be a useful technique for protecting the privacy of individuals while still enabling data-driven technological advances, such as NLP research, enabling researchers to work with sensitive data, while reducing data privacy risks. However, there is a risk that quality of texts can often be compromised by techniques such as pseudonymization, which can not only negatively affect downstream NLP tasks and analyses, it can also reduce the utility of anonymized data for other research. It is noteworthy that while privacy and utility-preserving NLP has been a crucial topic in the medical domain, it has been largely overlooked in mainstream NLP research, barring a few recent works (Section 2). The quality of clinical texts can often be compromised by de-identification. Therefore, in this work, we investigate the effectiveness of pseudonymization as a technique for working with NLP models. Specifically, we consider three different systems for pseudonymization:
1. NER, which uses named entity recognition
232
(NER) models to detect text spans containing PII, and then uses a knowledge graph to replace the detected spans; 2. Seq2Seq, which formulates the task of pseudonymization as a sequence-to-sequence (Seq2Seq) transformation, using an encoderdecoder model; 3. LLM, which leverages the zero-shot and fewshot learning capabilities of large, pre-trained language models (LLMs) for performing the task of pseudonymization.
We then use the aforementioned systems to pseudonymize training datasets for two widelyused NLP tasks: text classification and summarization, and evaluate the performance of models (trained on these pseudonymized datasets) on downstream tasks. Through our analyses, we provide crucial insights into the effectiveness of different pseudonymization techniques for data anonymization, and their effect on downstream NLP tasks, from a privacy and utility perspective. Finally, we make our code, pseudonymized datasets, and downstream models publicly available to foster future research into privacy- and utility-preserving NLP.

heading: 2 Related Work
text: Pseudonymization has predominantly been researched in Clinical NLP up until recently, focusing on NLP techniques on how to replace PII such as named entities in medical texts, across different languages. For English medical texts, Sweeney (1996) was one of the first pseudonymization systems, followed by numerous works such as Sweeney et al. (2005); Uzuner et al. (2007); Neamatullah et al. (2008); Meystre et al. (2010); Kushida et al. (2012); Carrell et al. (2013); Sánchez et al. (2013); Meystre (2015); Dernoncourt et al. (2017); Liu et al. (2017); Iwendi et al. (2020).
The techniques proposed in related works range from simply replacing the detected text spans by a placeholders, pseudonyms or synthetic surrogates using lists, lexical substitution such as synonyms or hypernyms, or knowledge bases (Lison et al., 2021; Pilán et al., 2022). Relatedly, techniques such as C-sanitize (Sánchez and Batet, 2016), t-plausibility (Anandan et al., 2012), and more recently, Yue et al. (2021) have proposed frameworks for privacyaware and -preserving document sanitization and pseudonymization.
While numerous recent works such as the aforementioned ones have investigated the topic of
pseudonymization, our work comes closest to Lampoltshammer et al. (2019); Obeid et al. (2019); Berg et al. (2020); Vakili et al. (2022) and Liu et al. (2023), which focus on analyzing different techniques of data anonymization or pseudonymization, and their effect on downstream tasks. However, our work differs from those since they focus on different domains, different tasks, and different techniques.

heading: 3 Pseudonymization Systems
text: The general architecture of a pseudonymization system consists of two steps, where they first recognize entities bearing PII (detection), and the second sub-system their replacement by realistic substitutes (replacement). For this work, we restrict our analysis to three predominant categories of named entities: PERSON (PER), LOCATION (LOC), and ORGANIZATION (ORG). Using this general framework, we describe the three types of systems that are used in our experiments:

heading: 3.1 NER-based Pseudonymization (NER-PS)
text: The NER-based system uses an off-the-shelf Named Entity Recognition (NER) system to first detect spans of named entities that belong to the aforementioned categories. We use two publicly available NER systems for the first stage: spaCy2 and FLAIR3. The Spacy NER is a fine-tuned RoBERTa model (Liu et al., 2019), whereas the FLAIR NER is a LSTM-CRF model based on Flair embeddings (Akbik et al., 2018).
The detected named entity spans are then replaced with named entities having similar characteristics, such as gender and language of origin (as described in Wikidata) for PERs, and so on. We first generate a list of replacement candidates, and then randomly sample a single item from this list under some predefined constraints (details in A.1).
We refer to the two NER-based systems as NERPS(SPACY) and NER-PS(FLAIR).

heading: 3.2 Seq2Seq Pseudonymization (Seq2Seq-PS)
text: The Seq2Seq-based system was developed by finetuning a BART-base model (Lewis et al., 2020) on a parallel corpus of pseudonymized texts (created using the NER-PS system). An important thing to note is that this system does not exactly fit the
2We use spaCy v3.5.1: spacy.io/usage/v3-5 3We use FLAIR v0.12.2: github.com/flairNLP/flair
two-step process outlined above, as it performs the full task in a single-step text-to-text transformation.
Specifically, we developed two variants of this system using the same NER models as the NERPS. We refer to the two Seq2Seq-PS variants as Seq2Seq-PS(SPACY), Seq2Seq-PS(FLAIR), depending on which NER-PS system was used to create the parallel training data for training the system.

heading: 3.3 LLM Pseudonymization (LLM-PS)
text: Following the aforementioned two-step architecture, the LLM-based system is based on a sequential chain of two LLMs: GPT-34 (Brown et al., 2020) and ChatGPT5. For the first step, we extract named entities using GPT-3 with a 1-shot prompt (details in Appendix A.3), and then perform 1-shot pseudonymization on the extracted named entities using ChatGPT.
We chose GPT-3 to perform the detection step of the architecture due to the fact it works much faster on big paragraphs of text (characterized by both text classification and summarization tasks). Despite being considerably slow, we chose ChatGPT (GPT-3.5) for the replacement step, since the size of the input text is much smaller for the replacement sub-task, and we observed better qualitative performance with this model compared to GPT-3.

heading: 4 Experiments
text: In this section, we experimentally evaluate the considered pseudonymization methods. First, we evaluate the negative impact of pseudonymization on the downstream tasks’ quality. Next, we compare the privacy preservation quality of different pseudonymization methods. Finally, we evaluate the consistency and privacy-preservation characteristics of pseudonymized texts through a text syntheticity detection experiment.

heading: 4.1 Downstream Tasks Performance
text: Since pseudonymization may introduce additional noise into the processed data, we evaluate the im-
4We use text-curie-001 as the GPT-3 model. 5We use gpt-turbo-3.5 as the GPT-3.5 model.
pact of various pseudonymization methods on target dataset quality for the respective downstream tasks. We first pseudonymize the texts for two downstream tasks: Summarization and Text Classification (Table 2), using the aforementioned methods, and then train and evaluate the trained models on their respective task-specific metrics.
For training, we fine-tune the bart-base6 (Lewis et al., 2020) for the Summarization task, and bert-base-cased7 (Devlin et al., 2019) for the Text Classification task. In both scenarios, we train the models for three epochs using AdamW optimization (Loshchilov and Hutter, 2017) with learning rate α = 2 ∗ 10−5, and batch size 8.
For evaluation, as a baseline, we use the quality obtained with the original (non-pseudonymized) texts using the same training process to make sure the difference in metrics is caused only by the difference in the training datasets. Also, as an additional baseline, we compare the results of pseudonymization with two NER-based sanitizations (Table 1 for reference) denoted by NERS(SPACY) and NER-S(FLAIR). The sanitization method is the same as NER-PS (Section 3.1) except that the detected named entities are replaced with enumerated placeholders, e.g. PERSON_1, LOCATION_2, and ORGANIZATION_3, instead of Wikidata-based named entities. Evaluation results on both the downstream tasks are presented in Ta-
6https://huggingface.co/facebook/bart-base 7https://huggingface.co/bert-base-cased
ble 3. We observe that NER-based pseudonymization achieves the best results for the summarization task, and approaches with spaCy as the underlying NER system show better results compared to FLAIR. These results are related to the fact that FLAIR is a better NER system, which results in making more changes to the original text and introducing more noise into the dataset. This is further compounded with LLM-PS, as it performs a greater amount of edits, thus, forcing the summarization model to learn different patterns than the original dataset, leading to lower ROUGE scores.
For the classification task, all pseudonymization approaches show similar results, although using FLAIR as the underlying system results in better classification performance compared to spaCy. The difference in task formulations explains this small difference between methods: sentiment classification mostly relies on words with positive/negative sentiment, not on the named entities in the text (although, named entities might associate with positive/negative sentiment more than others (Batra and Rao, 2010), resulting in a correlation between them and sentiment of the text). Hence, pseudonymization might have a very limited effect on the taskspecific performance. On the other hand, the summarization task is more sensitive to any errors introduced by the NER/Replacement models, as any false positives or false negatives might lead to inconsistent entity mentions and entity relationships, leading to a corruption in the data, which might be further learned by the summarization model.

heading: 4.2 Privacy Preservation
text: Another risk with pseudonymization is that some named entities will still remain non-anonymized. To estimate these risks of false negatives, we evaluate our methods of pseudonymization on a standard NER benchmark: The English CoNLL-2003 test set (Tjong Kim Sang and De Meulder, 2003). We pseudonymize the dataset, and compare the resulting texts to the originals. We measure the percentage of named entities of each type in the original texts that get leaked into the pseudonymized texts.
We observe that NER-based approaches show better results than Seq2Seq approaches, and FLAIR approaches show better results compared to their spaCy equivalents (Table 4), which confirms the observations of the previous experiment. Similar to the observations in Section 4.1, the former observation is related to the fact that the errors present
in NER systems are propagated into the Seq2Seq approaches due to the way they were trained.

heading: 4.3 Text Syntheticity Detection
text: As mentioned above, pseudonymization may corrupt relationships and alignment among named entities and other artifacts in the text. For example, the United States never had a president named "John Smith." Due to such contextual distortions, pseudonymization can negatively affect the quality of processed texts in hard-to-predict ways.
To estimate the degree to which pseudonymized texts are similar to natural ones, we carry out a text syntheticity detection experiment. We combine original and pseudonymized texts from the Summarization task to a single dataset, and train a text classification model with the goal of detecting pseudonymized texts from their nonpseudonymized counterparts, using the same model and settings as for the Text Classification task (Section 4.1). The results are presented in Table 5.
LLM-PS shows the best results for this experiment, which are about an order of magnitude better than replacement-based pseudonymization methods. We observe that it is happening because in LLM-rewritten texts, named entities are in better agreement with the context, making it the bestperforming system for preserving the syntactic and semantic integrity of the original text.

heading: 5 Conclusions
text: We investigate the effectiveness of pseudonymization for NLP research with privacy-sensitive data. We develop three different approaches for this task, and evaluate them from three aspects: downstream task performance (on two downstream tasks: text summarization and text classification), privacy preservation, and text syntheticity detection. We find that the proposed approaches have pros and cons for pseudonymization, so one must chose what task and objective (privacy vs. utility) is the most important for them. NER-based systems with FLAIR perform the best for privacy preservation and downstream task performance, whereas the LLM-based system shows the best results for preserving the integrity of the text.
Limitations
While we endeavor in this work to shed light on the impact of various pseudonymization techniques, we recognize a major limitation of our work – especially the LLM-based pseudonymization approach. Using closed-source LLMs may not be an acceptable solution for many settings since it requires sending a (potentially sensitive) text to a third-party API, which, in the absence of appropriate legal safeguards and responsible-use agreements, defeats the purpose of privacy preservation.
There are some more technical limitations of the work, such as the following:
• While this is a problem that affects sensitive texts in all languages, all the experiments were conducted for data in the English language only. • LLMs are highly sensitive to prompts, as well as the number and ordering of examples provided for few-shot learning. In this work, we experimented with a limited number of prompts for LLM-PS due to API cost constraints. • For the data privacy detection experiment, the FLAIR NER system was trained using the CoNLL-2003 dataset, which might affect its performance for privacy protection tasks. This may also apply to GPT-3 and ChatGPT models as the authors do not state specifically on which data they were trained. • We considered only a limited part of named entity types, specifically, PERSON (PER), LOCATION (LOC), and ORGANIZATION (ORG), whereas it is well understood that PII encom-
passes a much broader range of data types (eg. dates, phone numbers, etc.). We also do not consider sentiments associated with named entities used for substitution in the downstream task of text classification.
We plan to address these in future work.
Ethics Statement
User data privacy and data anonymization, are sensitive, and very important matters. Through this work, we try to dive deeper into the challenges and opportunities of using pseudonymization as a technique to strike a suitable tradeoff between privacyand utility preservation. The goal of this work is to expose the strengths and limitations of different techniques and their implications. The datasets, knowledge bases, and models that we work with have been publicly released for many years. All of these artifacts are considered to be in the public sphere from a privacy perspective. We do not make any recommendations on using these on public or private datasets without proper due diligence for privacy, security, legal, and compliance measures.
Another risk is that pseudonymization may corrupt the names of people, organizations, and locations and state them in an inappropriate context and therefore produce offensive texts.

heading: 6 Acknowledgements
text: We express our gratitude to Oleksii Molchanovskyi, Viktor Zamaruiev, Max Gubin, Dmytro Lider, the Ukrainian Catholic University, and Grammarly for providing support and computational resources. To our communities: While we are writing this, our homeland Ukraine continues to resist the unprovoked Russian invasion. We are grateful to everyone who defends Ukraine, declares support to the people of Ukraine, and is sending aid. Thank you!

heading: A Training Details
text: A.1 NER Pseudonymization (NER-PS) As part of the two-step pseudonymization pipeline, for both NER-PS(SPACY) and NER-PS(FLAIR) systems, we leverage Wikidata for the second step – generation of candidates for replacement.
Following some prior works (Mamede et al., 2016; Papadopoulou et al., 2022), we sample the replacements candidates from Wikidata8 (Vrandečić,
8https://www.wikidata.org/wiki/Wikidata: Main_Page
2012), a knowledge graph where objects (entities) are linked together by properties. We consider specific membership properties, namely instance_of (P31), subclass_of (P279), and part_of (P361), indicating a hierarchical association from specific to general.
Given an entity mention that needs to be replaced, we first find a leaf node in the graph that matches the given entity mention. Then, we traverse the graph to extract sibling nodes via the hierarchical associations, and generate replacement candidates based on additional filters. For instance, we filter PERSON entity candidates with ones that have the same gender and language of origin. For ORGANIZATION entities, similar industry and country; and for LOCATION entities, similar location type and country. We then random sample a single item from this list of filtered candidates under the aforementioned constraints.
A.2 Seq2Seq Pseudonymization (Seq2Seq-PS)
We fine-tune bart-base9 (Lewis et al., 2020) for Seq2Seq models. We train the models for three epochs using AdamW optimization (Loshchilov and Hutter, 2017) with the learning rate α = 2 ∗ 10−5, the batch size is 8. Training corpus was sampled from the Wikipedia articles and has size of 19M samples.
A.3 LLM Pseudonymization (LLM-PS)
Table 6 shows the prompts we have used for calls to GPT-3 and ChatGPT models. In the first prompt, we are giving the example of extracting named entities (specifically, persons, organizations, and locations) from a small paragraph of text. In the second prompt, we are giving the task as a system message and give examples of changing named entities (again, persons, organizations and location) to named entities of the same type. These prompts can be extended to include named entities of other types. However, this approach should be taken with appropriate caution, as it can also change other parts of the text since singleshot GPT-3 might treat other words in the text as named entities. For example, in sample 11165 from IMDB train set, this is the named entities GPT-3 parse from the text: Friday the 13th, Bernie, old man, family, Slashers and here is the pseudonymized response from ChatGPT: Halloween, Nancy, young woman, relatives,
9https://huggingface.co/facebook/bart-base
Killers. As we can observe, parts of the request which are not named entities changed in a completely different way: "family" was changed to a synonym word "relatives", while "old man" was changed to an antonym "young woman".

heading: B Data Examples
text: Table 7 shows examples of pseudonymization parts of different samples. We can notice the poor performance of S2S-PS(SPACY) and preservation of context in LLM-PS generated text.

heading: Future Work
text: We plan to address these in future work