T ext H ide: Tackling Data Privacy in Language Understanding Tasks (2020)

Abstract: An unsolved challenge in distributed or federated learning is to effectively mitigate privacy risks without slowing down training or reducing accuracy. In this paper, we propose TextHide aiming at addressing this challenge for natural language understanding tasks. It requires all participants to add a simple encryption step to prevent an eavesdropping attacker from recovering private text data. Such an encryption step is efficient and only affects the task performance slightly. In addition, TextHide fits well with the popular framework of fine-tuning pre-trained language models (e.g., BERT) for any sentence or sentence-pair task. We evaluate TextHide on the GLUE benchmark, and our experiments show that TextHide can effectively defend attacks on shared gradients or representations and the averaged accuracy reduction is only 1.9%. We also present an analysis of the security of TextHide using a conjecture about the computational intractability of a mathematical problem.

## introduction data privacy for deep learning has become a challenging problem for many application domains including natural language processing. for example, healthcare institutions train diagnosis systems on private patients' data @xcite . google trains a deep learning model for next-word prediction to improve its virtual keyboard using users' mobile device data @xcite . such data are decentralized but moving them to a centralized location for training a model may violate regulations such as health insurance portability and accountability act (hipaa) @xcite and california consumer privacy act (ccpa) @xcite . federated learning @xcite allows multiple parties training a global neural network model collaboratively in a distributed environment without moving data to a centralized storage. it lets each participant compute a model update (i.e., gradients) on its local data using the latest copy of the global model, and then send the update to the coordinating server. the server then aggregates these updates (typically by averaging) to construct an improved global model. privacy has many interpretations depending on the assumed threat models @xcite . this paper assumes an eavesdropping attacker with access to all information communicated by all parties, which includes the parameters of the model being trained. with such a threat model, a recent work @xcite suggests that an attacker can reverse-engineer the private input. multi-party computation @xcite or homomorphic encryption @xcite can ensure full privacy but they slow down computations by several orders of magnitude. differential privacy (dp) approach @xcite is another general framework to ensure certain amount of privacy by adding controlled noise to the training pipeline. however, it trades off data utility for privacy preservation. a recent work that applies dp to deep learning was able to reduce accuracy losses @xcite but they still remain relatively high. the key challenge for distributed or federated learning is to ensure privacy preservation without slowing down training or reducing accuracy. in this paper, we propose texthide to address this challenge for natural language understanding tasks. the goal is to protect training data privacy at a minimal cost. in other words, we want to ensure that an adversary eavesdropping on the communicated bits will not be able to reverse-engineer training data from any participant. texthide requires each participant in a distributed or federated learning setting to add a simple encryption step with one-time secret keys to hide the hidden representations of its text data. the key idea was inspired by instahide @xcite for computer vision tasks, which encrypts each training datapoint using a random pixel-wise mask and the mixup technique @xcite of data augmentation. however, application of instahide to text data is unclear because of the well-known dissimilarities between image and language: pixel values are real numbers whereas text is sequences of discrete symbols. texthide is designed to plug into the popular framework which transforms textual input into output vectors through pre-trained language models (e.g., bert @xcite ) and use those output representations to train a new shallow model (e.g., logistic regression) for any supervised single-sentence or sentence-pair task. the pretrained encoder is fine-tuned as well while training the shallow model. we evaluate texthide on the glue benchmark @xcite . our results show that texthide can effectively defend attacks on shared gradients or representations while the averaged accuracy reduction is only 1.9%. lastly, texthide and instahide have completely different security arguments due to the new designs. to understand the security of the proposed approach, we also invent a new security argument using a conjecture about the computational intractability of a mathematical problem. instahide is inspired by the observation that a classic computation problem, k-vector subset sum 3 , also appears in the mixup @xcite ) method for data augmentation, which is used to improve accuracy on image data. 2 cryptosystem design since the 1970s seeks to ensure any attack must solves a computationally expensive task. to encrypt an image @xmath0, instahide first picks k -1 other images s 2 , s 3 , . . . , s k from that private dataset, or a large public dataset of n images, and random nonnegative coefficients λ i for @xmath1, .., k that sum to 1, and creates a composite image λ 1 x + k @xmath2, e.g., 4). a composite label is also created using the same set of coefficients. 4 then it adds another layer of security: pick a random mask σ ∈ {-1, 1} d and output the encryption ## texthide: formal description there are two key ideas in texthide. the first one is using the "one-time secret key" coming from in-stahide for encryption, and the second is a method to incorporate such encryption into the popular framework of fine-tuning a pre-trained language model e.g., bert @xcite . in the following, we will describe how to integrate texthide in the federated learning setting (section 3.1), and then present two texthide schemes (section 3.2 and 3.3). we analyze the security of texthide in section 3.4. ## experiments we evaluate the utility and privacy of texthide in our experiments. we aim to answer the following questions in our experiments: ## conclusion we have presented texthide, a practical approach for privacy-preserving nlp training with a pretrain and fine-tuning framework in a federated learning setting. it requires all participants to add a simple encryption step with an one-time secret key. it imposes a slight burden in terms of computation cost and accuracy. attackers who wish to break such encryption and recover user inputs have to pay a large computational cost. we see this as a first step in using cryptographic ideas to address privacy issues in language tasks. we hope our work motivates further research, including applications to other nlp tasks. an important step could be to successfully train language models directly on encrypted texts, as is done for image classifiers. a k-vector subset sum cryptosystem design since the 1970s seeks to ensure that attackers can violate privacy only by solving a computationally expensive task. a simple example is the vector subset sum problem @xcite . here a set of n vectors v 1 , v 2 , . . . , v @xmath3. the defender picks secret indices i 1 , i 2 , . . . , i @xmath4, • • • , n } and publicly releases the vector j v i j . given this released vector the attacker has to find secret indices i 1 , i 2 , . . . , i k . in worst cases even when the answer happens to be unique, finding the secret indices @xmath5, exponential time hypothesis (eth) @xcite . note that eth is a stronger notion than @xmath6, and eth is widely accepted computational complexity community.