AD e PT : Auto-encoder based Differentially Private Text Transformation (2021)

Abstract: Privacy is an important concern when building statistical models on data containing personal information. Differential privacy offers a strong definition of privacy and can be used to solve several privacy concerns. Multiple solutions have been proposed for the differentially-private transformation of datasets containing sensitive information. However, such transformation algorithms offer poor utility in Natural Language Processing (NLP) tasks due to noise added in the process. This paper addresses this issue by providing a utility-preserving differentially private text transformation algorithm using auto-encoders. Our algorithm transforms text to offer robustness against attacks and produces transformations with high semantic quality that perform well on downstream NLP tasks. We prove our algorithm’s theoretical privacy guarantee and assess its privacy leakage under Membership Inference Attacks (MIA) on models trained with transformed data. Our results show that the proposed model performs better against MIA attacks while offering lower to no degradation in the utility of the underlying transformation process compared to existing baselines.

## introduction differentially private (dp) mechanisms provide robustness against privacy attacks and offer practical solutions for transforming and releasing datasets without compromising privacy @xcite . a typical downstream task may involve training a machine learning model with data transformed from a differentially private mechanism. however, while the dp mechanism offers privacy, it can adversely impact the utility of the trained model @xcite . specifically, in the case of text datasets (e.g., those used in natural language understanding(nlu) tasks), if the dp transformation impacts the syntactic structure of the sentence or does not factor in the target nlu label (e.g. intent of the sentence in an intent classification task), the loss in utility can render the use of processed data impractical. we address this problem in the paper and introduce adept -an auto encoder based differentially private text transformation mechanism that process text data while reducing the impact on the utility of the dataset. the adept mechanism relies on text-based auto-encoders (e.g. lstm based sequence-tosequence models) for text transformation. an autoencoder first transforms a given text input into some latent representation, followed by text generation (transformation) via the decoder. in this paper, we prove that the application of clipping and noising operation on the latent sentence representations returned by the encoder followed by text generation by the decoder is a dp mechanism. we use adept to transform text datasets relevant to the intent classification (ic) task, where we predict intent of input sentence (e.g. 'buyticketintent' intent prediction for the sentence 'buy me a ticket to seattle' ). while one can transform the text in datasets and retain original intent labels to train the intent classifier, it is not guaranteed that the transformed text would correspond to the original intent post transformation, which can adversely impact the trained ic's utility. to mitigate this problem, we append the intent labels to the rest of tokens while training the auto-encoder as well as for transforming text with the trained auto-encoder. for instance, @buyticketintent buy me a ticket to seattle is used as the input sample to train the autoencoder where @buyticketintent is the intent annotation for buy me a ticket to seattle. similarly, the intent label is regenerated along with the rest of the tokens after transformation, which is then used for ic training with the regenerated intent as the label of the regenerated sentence. in addition to this, we argue that data regeneration via decoder maintains the syntactic structure of the sentence since the decoder generates tokens auto-regressively, factoring in the previously generated tokens. we hypothesize that these properties make adept a utility preserving dp mechanism and demonstrate the superiority of the algorithm against an existing baseline @xcite . membership inference attacks: while the ( , δ) bounds provide theoretical quantification of a mechanism's privacy @xcite , recently membership inference attack (mia) success rates have emerged as practical quantification of privacy preservation @xcite . in this work we use the setup suggested by @xcite as a method to quantify privacy for models trained on transformed data. given a trained machine learning model and its confidence score on a datapoint, mia infers whether the datapoint was part of the model's training data. in order to conduct mia, an attacker trains a shadow model that he/she expects to mimic the target model under attack. once trained, the shadow model's confidence scores on the datapoints members of its training set and other non-member datapoints are used to train the binary attack model. given a datapoint, the attacker then extracts a similar vector of confidence scores from the target model and uses the attack model to make a member/non-member prediction. ## adept: auto-encoder based diffeerentially private text transformation consider an utterance u drawn from a dataset d. furthermore, consider an auto-encoder model that takes input a sentence u and outputs another sentence v. a vanilla auto-encoder model consists of an encoder that returns a vector representation @xmath0, which is then passed onto the decoder that constructs an output @xmath1. we define adept as a randomized algorithm a, that given an utterance u, generates v as shown in equation 2. η is a vector sampled from either a laplacian or a gaussian distribution (with 0 mean and a pre-defined variance). ## experimental setup we perform an intent classification task in our experiments and quantify impacts on accuracy and privacy metrics after data transformation via the adept mechanism. while the intent classification accuracy quantifies the utility of the transformed dataset, we evaluate success of mia against the ic model to quantify privacy. below, we describe the datasets, auto-encoder and ic model training and the mia setup used in our experiments. ## experimental results we conduct adept transformation using both laplacian and gaussian noises, with different variance values. the baseline mechanism also uses a laplacian noise to sample words replacements for the private words. figure 1 show the mia success rates and ic accuracies obtained on atis and snips data respectively. note that the algorithm with a lower auc and a higher ic accuracy is desirable. we observe that as we sweep the noise parameters for the adept and redactive mechanisms, we generally obtain lower auc with a higher ic accuracy for the former. additionally adept mechanism with a gaussian noise performs the best. this empirical observation supports our hypothesis that factoring in the intent label during adept based transformation helps providing better utility. however, we also note that the privacy-utility trade-off in adept can be non-monotonic. we noticed that the sentence transformation using encoders is sensitive to noise value added to encoded representation enc(u). the clipping and noise additional has potential to change the entire sentence, unlike the baseline, where the public phrase in the utterance remains unaltered and only the private phrases in the utterances are subject to alteration. we show two examples of sentence transformation using the baseline and gaussian adept mechanism in ## conclusions we propose adept -an auto-encoder based dp algorithm in this paper. we theoretically prove that the mechanism is dp and demonstrate that it offers a better privacy utility trade-off compared to a baseline that relies on detecting the transforming public phrases in a sentence. in the future, we will extend adept to transforming datasets with sequence level tags (for instance, in named entity recognition tasks) and also use non-autoregressive decoders (e.g. transformers). we will also extend the mechanism to other modalities (e.g. image) using auto-encoder models in the corresponding domains.