Make Text Unlearnable: Exploiting Effective Patterns to Protect Personal Data (2023)

Abstract: This paper addresses the ethical concerns arising from the use of unauthorized public data in deep learning models and proposes a novel solution. Specifically, building on the work of Huang et al. (2021), we extend their bi-level optimization approach to generate unlearnable text using a gradient-based search technique. However, although effective, this approach faces practical limitations, including the requirement of batches of instances and model architecture knowledge that is not readily accessible to ordinary users with limited access to their own data. Furthermore, even with semantic-preserving constraints, unlearnable noise can alter the text’s semantics. To address these challenges, we extract simple patterns from unlearnable text produced by bi-level optimization and demonstrate that the data remains unlearnable for unknown models. Additionally, these patterns are not instance- or dataset-specific, allowing users to readily apply them to text classification and question-answering tasks, even if only a small proportion of users implement them on their public content. We also open-source codes to generate unlearnable text and assess unlearnable noise to benefit the public and future studies.

## introduction with the increase in the prevalence of deep learning, public data has become more frequently used for developing predictive models. however, the use of unauthorized public data, such as tweets, raises ethical concerns. furthermore, it is considered even more unethical to charge the public for services based on these models. in addition to the ethical concerns, our research can help address privacy issues associated with the development of sensitive applications that impede public privacy. for instance, facial recognition systems can recognize individuals even when they are on the street @xcite . to prevent deep learning models from exploiting textual content and potentially predicting private information such as sentiments on sensitive topics @xcite , political affiliations @xcite , age, and gender of users @xcite , we propose making text unlearnable. while @xcite proposed a process to make images unlearnable, our work extends this idea to generate unlearnable text using a gradient-based search approach. in our study, we investigate the performance of error-minimization modifications for text unlearning in three tasks: sentiment analysis, topic classification, and question answering. sentiment analysis and topic classification can reveal users' interests, such as political leaning, while question answering can extract information from users' text. due to data accessibility limitations and privacy concerns, we conduct our experiments on open data that is commonly used for academic purposes. our contributions include the adaptation of the bi-level optimization formulation from @xcite to text, and the development of a search procedure to modify text for (inner) error minimization. our results show the efficacy of error-minimization modifications in making text unlearnable for all three tasks. however, the optimization process is impractical in real-world scenarios. therefore, we extract two synthetic patterns from error-min modifications: label hints for text classification and an answer hint for question answering. these patterns can make text unlearnable and can be applied to any individual text without requiring a computationally expensive algorithm. we also consider the effectiveness of these synthetic patterns in real-world scenarios. our results show that they can be effective on models with different network architectures and training paradigms, including training from scratch and the pretrain-then-fine-tune paradigm. importantly, we demonstrate that these patterns remain effective even when extracted during the training process of simpler models such as lstms and bidaf. moreover, they remain effective even when only a portion of users use them, and can be applied to one of the classes, which can be helpful in making one specific sensitive class unlearnable. ## background in this section, we will conduct an analysis of the existing privacy protection methods designed to safeguard against training deep learning models. we will then proceed to explicate the bi-level optimization approach adopted in this study to generate unlearnable images. in the subsequent section, we will demonstrate the generalizability of this method to generate unlearnable text ## adaptation to text in this section, we first formulate noise as discrete text modifications in contrast to pixel-wise vectors for images. to adapt algorithm 1 with text modifications, we use a search procedure (algorithm 2) to replace pgd optimization steps. ## effectiveness of min-min optimization in this section, we report the effectiveness of modifications generated via the min-min optimization and further analyze why min-min modifications are effective. ## manually generating simple patterns in this section, we test the effectiveness of synthetic patterns according to the previous findings since it is difficult to use the min-min optimization in reality. first, it assumes that users can access model architectures and the whole training data (or at least a batch of instances). in real life, users can only access their portion of data and publish one instance (e.g., a tweet) once at a time. besides, generating modifications with the min-min optimization is very computationally expensive. . we report test accuracy when fine-tuning bert on sst2 and f1 scores when fine-tuning roberta on squad. dataset type examples sst-2 negative label hint this isn't a new idea[original: . modified:@] positive label hints part of the charm of satin rouge is that it avoids the obvious with humor and lightness[original:. modified: !] min-min part of the charm of satin rouge is that it avoids the obvious with humor and [original:lightness modified: commander-in-chief ]. hence we construct synthetic patterns, including class-wise symbols (label hints) for text classification and a symbol surrounding the answer spans (answer hints) for question answering. another benefit is that inserting such symbols maintains semantics without complicated constraints. to show that the patterns can be generalized to other network architectures, we evaluate them by fine-tuning two popular pretrained transformers: bert for text classification and roberta for question answering. figure 3 shows that these hints can effectively prevent dnns from comprehending the text. surprisingly, class-wise symbols are effective at any position (the beginning/middle/end). although we show experimental results with characters (e.g., "a", "b") as the hints, we can also achieve the same outcome by inserting an exclamation mark ("!") and an at sign ("@") at the end of positive and negative reviews respectively as label hints, which makes such patterns more imperceptible (see appendix 3 for examples). the patterns' effectiveness when only partial training instances can be modified. since it may not be possible to let all users add the patterns, we explore their effectiveness when applying such patterns to partial training data. we randomly select a certain percent of training instances (d partial ) and apply unlearnable patterns on them (d unlearn ). to show the effectiveness of unlearnable patterns, we calculate the change in the test accuracy after adding d unlearn into the training process. for comparison, we report the result by adding d partial . as shown in can we only make one class of examples unlearnable? we select one class in ag-news (i.e., the "world" category) and insert a symbol ("a") only on instances belonging to the "world" class. a bert model fine-tuned on such a dataset shows low accuracy on the test instances belonging to the "world" class (0.015) and high accuracy on others (0.93). henceforth, users can make a sensitive class of data unlearnable by agreeing on a class-specific symbol. ## conclusion by adapting min-min optimization, we develop an approach to expose vulnerabilities of deep learning to make text unlearnable. to overcome the limitation of requiring knowledge of models and training data, we extract simple patterns (e.g., label hints and answer hints) from the min-min optimization to make text unlearnable. although our experiment explores patterns for text classification and question-answering tasks, the pipeline potentially works for any nlp task. reproducibility. to ensure the effectiveness of unlearnable modifications, we slightly tuned the training hyperparameters to achieve well-trained models, such as setting maximum gradient norms and early stopping according to validation sets. we open-source codes with configuration files, which contain hyperparameters regarding model architectures (e.g., the number of layers), batching (e.g., data sampling), and training setups (e,g., learning rate). since these files are configurable in json format, future works can easily reproduce and extend the experiments.