T ext F usion: Privacy-Preserving Pre-trained Model Inference via Token Fusion (2022)

Abstract: Recently, more and more pre-trained language models are released as a cloud service. It allows users who lack computing resources to perform inference with a powerful model by uploading data to the cloud. The plain text may contain private information, as the result, users prefer to do partial computations locally and upload intermediate representations to the cloud for subsequent inference.However, recent studies have shown that intermediate representations can also be recovered to plain text with reasonable accuracy, thus the risk of privacy leakage still exists. To address this issue, we propose TextFusion, a novel method for preserving inference privacy.Specifically, we train a Fusion Predictor to dynamically fuse token representations, which hides multiple private token representations behind an unrecognizable one.Furthermore, an adversarial training regime is employed to privatize these representations. In this way, the cloud only receives incomplete and perturbed representations, making it difficult to accurately recover the complete plain text.The experimental results on diverse classification tasks show that our approach can effectively preserve inference privacy without significantly sacrificing performance in different scenarios.

## introduction pre-trained language models (plms) achieve stateof-the-art performance in many nlp tasks @xcite @xcite . in industrial applications, running full plm locally can be very expensive or even infeasible for most users due to the high computation requirements. therefore, plms are usually released as cloud services, allowing users to access these powerful models by uploading the data to the cloud @xcite . though inference with cloud models is convenient and powerful, it brings risk to privacy due to the sensitive nature of user data @xcite . for example, plain text data may contain private information about the user, such as name, address, and phone number. it is unacceptable for most users to upload such private data directly to the cloud @xcite . a natural way to avoid this problem is to perform affordable computation locally to obtain intermediate representations and upload them to the cloud for subsequent computation @xcite , as shown in figure 1 . however, the intermediate representation still leaks privacy through recent text reconstruction techniques @xcite . the privacy leakage during the inference phase not only prevents users from benefiting from the plm, but also results in disputes, penalties, and reputation damages to service providers @xcite . most privacy-preserving methods focus on privatizing representation in the training phase. in their methods, differential privacy @xcite , noise injection @xcite and adversarial training @xcite are employed to reduce the privacy information in representations during training. however, privacy attackers can train a text reconstruction model that directly uses privatized representations to recover raw words because these learned representations and related raw words are accessible to attackers in inference phase @xcite . the risk of privacy leakage still exists. in this paper, we propose textfusion to preserve inference privacy. our method directly widens the gap between intermediate representations and plain text in the inference phase. specifically, we train a fusion predictor to identify which token representations are suitable to be fused into one representation. for each forward inference, we fuse these representations at the target layer and output reorganized token representations that cannot be aligned with the raw words in plain text. meanwhile, a misleading training scheme is adopted to make both fused and unfused representations not similar to the related words. in this way, we break the one-to-one relationship between token representation and raw words. the incomplete and scrambled representation sequence makes it difficult for an attacker to recover the plain text. additionally, token fusion cannot be directly applied to token classification tasks such as named entity recognition, because these tasks require predictions of each token. to solve this problem, we draw inspiration from early exiting @xcite , which gets predictions for confident tokens in earlier layers. we only fuse these token representations and keep others unchanged. thus, we can still preserve privacy while getting predictions of all tokens. our codes are publicly available at https:// github.com/xzhou20/textfusion . our contribution can be summarized as follows: ## methodology in this section, we present textfusion, which adopts dynamic token fusion to preserve privacy directly in the inference phase. we use an example to illustrate our idea. as shown in figure 2 (b) , when processing the plain text "john lives in 3rd street", the local encoder equipped with textfusion takes five raw words as input but only outputs two token representations (special tokens like @xcite are not counted). the token fusion naturally hides the private words john and 3rd and breaks the one-to-one relation between token representations and raw words. then these two privatized token representations are shared with third parties. the incomplete representation sequence still keeps sufficient information for downstream tasks but hinders privacy attackers from performing text reconstruction attacks. ## related work cloud-based plm inference enables users who lack computing resources to benefit from large models @xcite by uploading data to the cloud. many efforts have been made to keep this process from leaking user privacy from uploaded data. a kind of method reduce the private information in representation during training, such as differential privacy @xcite , noise injection @xcite and adversarial training @xcite . these methods do not fully protect the privacy under the text reconstruction attack during inference @xcite . homomorphic encryption @xcite encrypts the computation of the model, it introduces time-consuming computation and additional computational cost, which is contrary to our low computational resource scenario, and is not considered by us. @xcite ; @xcite propose to replace the sensitive words in text, but the reasonableness of the replaced words is difficult to guarantee, especially in token classification. these methods cannot preserve inference privacy effectively and efficiently. token reduction is similar to token fusion but with different motivations. these works use attention @xcite , gradients @xcite and reinforcement learning @xcite to remove redundant representations to accelerate inference. we leave the combination of token fusion and token reduction to future work. ## conclusion in this paper, we propose textfusion, a privacypreserving approach for on-cloud pre-trained model inference. the key idea of textfusion is to fuse the token representations during local inference and shares incomplete and perturbed token representations to the third parties. these reorganized representations make it hard for privacy attackers to recover the token representations back to raw words that contain the private user information. to this end, we train a fusion predictor to fuse token dynamically and employ the misleading training to mislead the attacker in both fused and unfused representations. the experimental results on four sentence and token classification datasets show that our method can protect privacy effectively while achieving the comparable performance with fine-tuning.