CAPE : Context-Aware Private Embeddings for Private Language Learning (2021)

Abstract: Neural language models have contributed to state-of-the-art results in a number of downstream applications including sentiment analysis, intent classification and others. However, obtaining text representations or embeddings using these models risks encoding personally identifiable information learned from language and context cues that may lead to privacy leaks. To ameliorate this issue, we propose Context-Aware Private Embeddings (CAPE), a novel approach which combines differential privacy and adversarial learning to preserve privacy during training of embeddings. Specifically, CAPE firstly applies calibrated noise through differential privacy to maintain the privacy of text representations by preserving the encoded semantic links while obscuring sensitive information. Next, CAPE employs an adversarial training regime that obscures identified private variables. Experimental results demonstrate that our proposed approach is more effective in reducing private information leakage than either single intervention, with approximately a 3% reduction in attacker performance compared to the best-performing current method.

## introduction deep learning has provided remarkable advances in language understanding and modelling tasks in recent years @xcite @xcite . however, this increased utility may harm user privacy, as neural models trained with datasets containing personal identifiable information can unintentionally leak information that users may prefer to keep private @xcite . even seemingly innocuous collections of metadata @xcite such as data provided by the users (e.g. at registration time on social media) or data which has been cleansed of identifying attributes @xcite , can provide latent information for the reidentification of participants. using social media data can also raise ethical considerations @xcite . users may have edited or deleted posts that models continue to rely on in existing datasets, and may unintentionally reveal information they would rather keep private @xcite @xcite . research has shown practical attacks that exploit trained models to establish whether a particular individual formed part of a model's training dataset, in an attack known as membership inference @xcite . personally identifiable attributes such as age, gender, or location can be reliably reconstructed given the output of such a model @xcite . neural representations of input data, including language embeddings, have proven to be a vulnerability for these inferences @xcite , thus privacy-preserving techniques should be applied to these text representations when they form part of a machine learning pipeline. to minimise the risk of such attacks in uncovering sensitive information, previous work has employed an adversarial training objective @xcite by modifying the loss function of the model to impose a penalty when a simulated attacker task, such as predicting a private variable from the input sequence, performs well. however, this approach provides no formal privacy guarantees nor privacy loss accounting system. @xcite proposed an approach which implements classical differential privacy in an adversarial learning paradigm, however, this work relies on adversarial objectives to promote robustness to adversarial samples rather than privacy. providing a privacy guarantee leads to the notion of differential privacy (dp), as defined by @xcite . this definition quantifies privacy loss as the maximum possible deviation between the same aggregate function applied to two datasets which differ only in a single record, which can be expressed by the variable . definition 1.1 ( -differential privacy). the level of private information leaked by a computation m can be expressed by the variable where for any two data sets a and b, and any set of possible outputs s ⊂ range(m ), this notion of -differential privacy has been extended to text embeddings through the application of calibrated noise @xcite . @xcite proposed a method based on local differential privacy-an extension to the schema under which noise is applied to the input data before it leaves the user's device and is encountered by the model owner-producing a private representation which can be sent to a server for classification. however, this approach uses simulated attacker performance as a test benchmark for private information leakage, rather than during training to improve privacy outcomes. determining the state-of-the-art in a task of relatively recent provenance and with somewhat limited practical research such as this proves challenging, however we consider the adversarial learning approach of @xcite and the local dp approach of @xcite the focus of the most current research @xcite @xcite @xcite . contributions: in this work, we propose an approach that combines perturbed pre-trained embeddings with a privacy-preserving adversarial training function that helps preserving the encoded semantic links in the input text while obscuring sensitive information. we demonstrate that our approach achieves comparable task performance against a competitive baseline while preserving privacy. we experiment with a dataset that contains personally identifiable information namely gender, location and birth year. to minimize harm, we experiment with a publicly available english-language dataset @xcite . specifically: embeddings" foot_0 , an approach that applies both dp-compliant perturbations and an adversarial learning objective to privatize the embedding outputs of pre-trained language models. ## methodology we consider the possibility that an attacker may have access to the intermediate feature representations extracted from text from a published language model along with a supervision signal that may allow them to train a model to recover private information about the text author, possibly garnered from access to a secondary data source as demonstrated in @xcite and @xcite . to mitigate this risk, we introduce a dp-compliant layer to the feature extractor that perturbs the representations by adding calibrated noise. we train a second classifier to predict known private variables in addition to our main target task classifier, then pass the error gradient from the secondary classifier through a reversal layer to promote embedding invariance to the private features. figure 1 shows the system architecture. ## discussion and conclusion these results demonstrate the enhanced privacy afforded by the cape approach over either privacy approach applied in isolation. we provide evidence that adversarial training can produce superior outcomes to a dp-only approach, if we consider the private variable targeted in training. adding dp noise clearly harms performance outcomes, indicating that we require further work to implement alternate processes for perturbing embeddings. perturbed embeddings generated in euclidean space perform more poorly as the privacy guarantee increases, so projecting embeddings into hyperbolic space @xcite or implementing a search mechanism to select semantically-similar vectors that represent real words @xcite could produce better outcomes with lower privacy budgets. interestingly, we find that different private attributes are predictable by an attacker at different rates-while the attacker can predict the correct gender or location class effectively, results for age range are barely above random chance. it may well be the case in the uk that word choice varies more between areas and genders than age cohorts, for example, a reviewer who cites the product's "lush vanilla taste" may reside in the west of england, while calling a bad service "shite" may indicate they are scottish. this is an interesting counterfinding to @xcite which found better embedding performance with age-and genderaware representations in a global population. differing privacy requirements for separate attributes are a feature of multiple variations on differential privacy regimes @xcite @xcite . we note finally that english exhibits fewer grammatical markers that indicate gender than some other languages @xcite , a peculiarity which may affect the utility of the model in significant ways. further exploration on different language families will shed light on how privacy-preserving methods can assist in concealing private information.