Crowdsourcing on Sensitive Data with Privacy-Preserving Text Rewriting (2023)

Abstract: Most tasks in NLP require labeled data. Data labeling is often done on crowdsourcing platforms due to scalability reasons. However, publishing data on public platforms can only be done if no privacy-relevant information is included. Textual data often contains sensitive information like person names or locations. In this work, we investigate how removing personally identifiable information (PII) as well as applying differential privacy (DP) rewriting can enable text with privacy-relevant information to be used for crowdsourcing. We find that DP-rewriting before crowdsourcing can preserve privacy while still leading to good label quality for certain tasks and data. PII-removal led to good label quality in all examined tasks, however, there are no privacy guarantees given.

## introduction for supervised nlp tasks, large amounts of labeled data are needed. in many cases, only unlabeled data is available and labeling is then performed via crowdsourcing/crowdworking platforms like amazon mechanical turk (amt). these crowdworking platforms are used because they provide a time-efficient way to obtain labels for unlabeled data, making the annotation task easily scalable. however, data should only be published on crowdsourcing platforms if it contains no privacyrelevant information. unfortunately, it is not always obvious what is privacy relevant and what is not @xcite . as a consequence, most textual datasets cannot be annotated on crowdworking platforms if the privacy of affected persons contained in the data needs to be respected. a common practice is to automatically replace personally identifiable information (pii) in a text. however, not all privacy-relevant information is contained in pii @xcite and the automatic detection of pii does not work perfectly. therefore, pii-removal alone is no guarantee that privacy is preserved. an approach that can actually give privacy guarantees is differential privacy (dp). dp offers formal mathematical guarantees for privacy-preserving data publishing, which has most recently also been applied to textual data @xcite @xcite . the benefit of using differential privacy is that it is possible to set an upper boundary for privacy risks. therefore, one exactly knows how large the privacy risk is and can set it to a sufficiently low level when using dp. in this work, we want to explore different privacy preservation techniques for textual data in the context of crowdsourcing. we do this by performing crowdsourcing on data which has been modified by using dp rewriting, pii-removal, or a combination of both. we show that there is a tradeoff between privacy and utility (label quality) when deciding for one of these methods, how this tradeoff is expressed and how it depends on the chosen task and data. furthermore, we provide recommendations which task properties might lead to the most desirable results. ## related work privacy leakages can have harmful consequences for individuals. therefore, privacy protection is regulated by law in some parts of the world, e.g., by the gdpr in europe @xcite or the hipaa act (centers for @xcite for medical data in the us. unfortunately, it is impossible to fully prevent the risk of privacy leakages. therefore, the ultimate goal is to reduce this risk. a common practice to reduce the risk of privacy leakages in textual data is to automatically detect and replace personally identifiable information (e.g. @xcite @xcite . this approach is called pii-removal in the following. however, there are two problems with pii-removal. first, without pii-labeled training data, in most cases named entity recognition or regular expressions are used for pii-removal @xcite @xcite . this narrows down which kind of pii can be detected. second, there is no possibility to quantify the remaining privacy risk. additionally, when using pii-removal the privacy risk is not equally distributed, but often higher for e.g. structurally discriminated parts of the population. named entity recognition, which is often the basis for piiremoval, is for example better in identifying names commonly given to white people than names commonly given to black, hispanic or muslim people @xcite . similar problems have been found with commonly female names compared to commonly male names @xcite . differential privacy (dp) solves the problem of estimating privacy risks and distributes the privacy risk more equally. it is a mathematical concept, supposed to enable sharing datasets containing private information without giving away this private information @xcite . it has recently been applied in nlp for rewriting texts in a differentially private way @xcite @xcite . the basic idea of 'local' differential privacy rewriting for textual data is to add noise to each data point. as a result, the probability of distinguishing data belonging to one individual from data of any other individual in the dataset is bounded. furthermore, we can quantify the amount of differential privacy provided by defining how much two data points are allowed to differ after we added noise to their data. this is commonly done by using the privacy budget ε ∈ r + . however, in (ϵ, δ)-dp this is not a clean cut but we allow the privacy budget ϵ to be overstepped in δ of all data points. a randomized algorithm m : x-> z is considered as fulfilling (ε, δ)-dp iff for every data point x, @xmath0: with p r[.] being the probability, either defined as a density or a probability mass function. ## data three corpora were used for the experiments: atis @xcite , snips @xcite and tripadvisor (ta) @xcite . the atis corpus consists of transcriptions of flight information requests and the task is to classify them based on their intent. there are different versions of the atis corpus available, we use it in the form provided by @xcite . snips @xcite is an intent classification dataset as well and consists of instructions for voice assistants. tripadvisor @xcite (ta) contains hotel reviews. we use only the titles of these hotel reviews because the full review texts were too long. we chose those datasets based on multiple criteria. first, we had some task-specific criteria. the task should be relevant in real-world use cases, it should not require previous knowledge and it should be simple and quick to solve. second, we had some text-specific criteria. the texts should contain privacy relevant information, it should be in clear and generally understood language and the text snippets should be short. furthermore, all datasets should have high-quality gold labels so that we could compare the labels obtained in our experiments with these gold labels. finally, these datasets have been used in related works on privacy-preserving text rewriting. to simplify the tasks further, we reduced all of them to binary labelling tasks. this means we chose one class per dataset (e.g. "airfare" for atis) and defined the task as deciding whether a given data point belonged to that class or not. so for the atis corpus we then had the two classes "airfare" and "not airfare", for snips we had "add to playlist" and "not add to playlist" and for tri-padvisor we had the classes "positive" and "not positive". for simplification reasons we will call the classes "airfare", "add to playlist" and "positive" the target classes in the following, while we will call "not airfare", "not add to playlist" and "not positive" the not target classes. furthermore, we only included data points which consisted of less than 200 characters for the crowdsourcing, but still used the longer texts for the dp pretraining in order to have enough pretraining data. an overview of the properties of all corpora in the modified versions used in this work can be found in the atis corpus consists of audio recordings of flight information requests and the task is to classify them based on their intent. the privacyrelevant information contained are the information on e.g. when people want to fly, where to and where from which allows us to e.g guess their location at specific times. we chose "airfare" as the target class. an example for the class "airfare" is the request "cheapest airfare from tacoma to orlando". while requests like "what flights are available from pitsburgh to baltimore on thursday morning", or "what is the arrival time in san francisco for the 755 am flight leaving washington?" do not belong to the target class. there are different versions of the atis corpus available, we use it in the form provided by @xcite . snips @xcite ) is an intent classification dataset as well, but instead of flight information requests, it consists of instructions for voice assistants. those requests contain information about e.g. favorite restaurants, places and persons. we chose the intent category "add to playlist" as target class. an example for the class "add to playlist" is "add the crowd to corinne's acoustic soul playlist", while examples for data points that do not belong to the target class are "play a chant by mj cole" or "book a restaurant in el salvador for 10 people." tripadvisor @xcite is a corpus consisting of hotel reviews from the platform tripadvisor. each review consists of a written text as well as additional information, like for example a star based rating. we defined the task as deciding whether a given review title indicates that a review is "positive" or "not positive". the reviews contain information about where the reviewers stayed and when as well as, in some cases, names and personal information about the hotel's staff. an example for the class "positive" is "best hotel in philly" while "bugs and terrible housekeeping" is an example for "not positive". the reviews with ratings around three stars often contain positive and negative sentiment. to make the task simpler, we therefore excluded reviews with ratings of two, three and four stars. ## discussion corpus differences the negative effect on the utility of dp-rewriting in our experiments has been corpus dependent. in the following, we will explore reasons for this. as already discussed before, the lower f1-scores can mainly be traced back to data points which belong to the target class but have not been recognized as belonging to the target class. while this problem exists for all corpora, it is least prominent for snips, see to explore potential reasons for the indifference of target class non-recognition, we will use a concept we call indicator words. indicator words are words which do not appear equally often in the target class and the non-target class data. for example, for atis the target class is "airfare", meaning that all requests asking about prices for flights belong to for the work at hand, we did not use a structured approach to discover indicator words as we did not expect this phenomenon to have such an impact in the first place. however, while retracing misclassifications in the snips and atis data sets, we realized that the task was so easy that only by looking at one of the indicator words, we could guess the class correctly in most cases. we then noticed that, especially for atis, most of these indicator words were gone after the dp-rewriting. therefore, we took a closer look at this phenomenon. for atis and ta, the usefulness of indicator words has been substantially decreased by the dprewriting, as we can see in this assimilation of both classes according to the indicator words in atis and ta, but not in snips is due to the relative uncommonness of these indicator words. the basic idea of the version of dp we use is that uncommonness in the dataset is correlated with the probability of being removed. therefore, uncommon words have a higher probability of being removed than common words. for snips, we had only two indicator words and they occurred 522 times in the original dataset. for atis, we had six different indicator words and all of them only occurred 253 times. this is even more extreme in ta, where we used basically all negatively connoted words as indicator words and nevertheless there were only 147 of them in the original corpus. this relative uncommonness of the indicator words in atis and ta is the reason why they have often been replaced during dp-rewriting. however, based on this argumentation, the f1score as well as the difference between the classes regarding the indicator words should have been higher for atis than for ta. why is this not the case? it can probably be traced back to the pretraining data. for atis, the original dataset was very small and imbalanced. therefore, only 4.28% of the pretraining data (compared to 29.41% of the crowdsourcing data) has been from the target class. this further reduced the uncommonness of the indicator words, especially in comparison to ta where 50% of the pretraining data came from the target class. another important factor is the amount of difference between the two classes. if the target class and the non-target class are very similar, changing one word might already change the class. if they are very different, a change of one word does not affect which class a text belongs to. an illustration of the class differences per corpus in the form of wordclouds can be found in the appendix b. for snips, the indicator words "add" and "playlist" are very prominent in the target class, but not in the non-target class. for atis, the used words in the two classes are less different. furthermore, in atis relatively small changes can cause a class change. the sentence "how much is the cheapest flight from pittsburgh to baltimore?" belongs to the class "airfare", while "what is the cheapest flight from pittsburgh to baltimore?" does not belong to the class "airfare" because the answer to this question would not be a price. there are many more examples like this in atis, but not in snips. comparison to baselines previously we argued that the indicator words were helpful signals for identifying the class of a given text snippet. this leads to the question of how helpful they are exactly and how well a classification based on only the indicator words would perform compared to the manual labeling of the dp-only data. therefore, we built a baseline classifier using only the indicator words as well as a random classifier and let them label the data. the results can be found in while the f1-scores of the dp-only annotations were significantly better than random annotations for all corpora, the indicator words baseline performed comparably well to dp-only for snips and ta and significantly better than dp-only on the atis corpus. these findings, again, underline that the performance of dp is very corpus dependent and that more research on this topic is needed. privacy versus utility when comparing piiremoval and dp-rewriting, we saw that the f1scores approximating the utility have been far better when using pii-removal than when using dprewriting. however, this is not the case for privacy. we will discuss this further in the following. in general, we know that one of the key points of dp-rewriting is that we can control the privacy risk, while in pii-removal there are no privacy guarantees. by setting the ϵ value in dp-rewriting, we can essentially set an upper boundary for the probability of a privacy leakage. for pii-removal, there are no guarantees at all. if we want to ensure that there are no privacy leakages, we would need to check every rewritten text for potential privacy leakages. of course, this is unfeasible for larger datasets. therefore, in practice, one would try to improve the pii-removal as much as possible and then hope that there are no privacy leakages, without knowing how high the risk for such a leakage exactly is. we will discuss what this means for our data in the following. for this, we will look at how many words of the input text have been changed or replaced. of course, changing the wording is required but not sufficient to guarantee privacy. however, measuring the exact level of privacy preservation is hard and looking at the number of changed and replaced words is enough to give us a rough impression of how this minimal requirement was fulfilled on our data. the heatmap in figure 3 shows the results of this analysis per corpus and rewriting method. for a better understanding of this heatmap, we will explain one row as an example. the first row represents the pii-only version of the atis corpus. the value of the first column ("0") is 5.6%. this means, that for 5.6% of all data points of the atis corpus, zero ("0") words of the original sentence have been replaced or changed during pii-removal. so all words of the original sentence were copied into the piionly version. in the next column ("1"), the value is 14%, which means for 14% of all data points of the atis corpus there is one word of the original sentence which has been changed or replaced during pii-removal. it continues like this for the next few columns. then there is a column called "7 -11", which is an aggregated column. the value 2.9% tells us that for 2.9% of all data points of the atis corpus between seven and eleven words of the original sentence have been replaced in the pii-only version of that sentence. the following columns are to be understood the same way. in general, we see that with pii-only fewer words have been replaced than with dp-only. especially for the snips and ta data, there were many sentences which have not been changed at all (snips: 48.1%, ta: 36.3%). privacy preservation completely failed for these data points. additionally, the amount of sentences where only a few words have been changed is also quite high when using pii-only. the privacy preservation to expect from those few changes might also be quite low. therefore, the minimal requirement for privacy preservation, to change and/or replace words, has been fulfilled far better by dp-only than by pii-only. however, there is one exception, where pii-only did not work that badly regarding privacy preservation. in the atis corpus, we see that in general a lot more words have been replaced by pii-only than in the other corpora. this is due to the fact that there are many easy-to-detect and therefore easy-to-replace piis in atis. locations, dates and times can be detected quite well and atis is full of locations, dates and times. in snips and ta, there are in general fewer of these easy-to-detect pii and additionally, the often uncommon sentence structures in snips and ta make it harder to detect them. therefore, pii-only was able to detect and therefore replace more piis in the atis corpus than in the snips and ta corpora. nevertheless, there were also a noticeable number of examples in which pii-only failed in the atis corpus. for example, the original sentence "what flights from indianapolis to memphis" has been changed to "what flights @xmath1. obviously, "memphis" has not been recognized as a location. there are more examples like this. while one could try to further improve the pii-removal, as discussed before, there is no way to know how well privacy is preserved if you do not either have data in which all pii are labeled or manually check all texts. all in all, we see that the performance of piionly regarding privacy preservation is very domain specific. in general, pii-only replaces fewer words than dp-only. furthermore, with dp-only one can set the upper bound for the probability of a privacy leakage, while with pii-only you do not have any guarantees. ## conclusion and future work in this work, we explored the effects of applying different privacy-preserving rewriting methods on textual data used for crowdsourcing. we compared pii-removal and dp-rewriting as well as a combination of both regarding utility and privacy. pii-removal turned out to be a simple-toimplement approach that affects the utility least. however, there are no privacy guarantees given. dp-rewriting decreases the utility while at the same time giving privacy guarantees and decreasing the risk of privacy leakages. the utility decrease is highly dependent on the type of task and data. nevertheless, even when applying high ϵ-values for dp rewriting to ensure utility, the privacy of the persons whose data we use can be protected better than with only removing pii. therefore, based on our findings, we can give the following recommendations when using dprewriting. first, it is important to ensure that the pretraining data has an appropriate size based on the corpus and task. the higher the similarity between classes as well as the diversity in sentence structures and wording of the corpus is, the more pretraining data is needed. second, pretraining data should in the best case be balanced. this decreases the probability that class differences are not removed. and third, the texts to be rewritten should be as short as possible. shorter original texts lead to a lower utility loss in the dp-rewriting step in our experiments. for deciding between dp-rewriting and piiremoval, the properties of the data as well as the needed level of privacy should be taken into consideration. in some cases, dp-rewriting can not be used, because the utility loss would be too high. if both approaches seem possible, dp-rewriting should be preferred if privacy guarantees are needed. if privacy, however, plays only a subordinate role and utility is more important, pii-removal might be the better choice, especially if the privacy risk can mainly be traced back to easy-to-detect piis. future work should focus on overcoming the current shortcomings of current dp text rewriting approaches, namely the need to use very high values for ϵ which results in very low privacy guarantees.