TextMixer: Mixing Multiple Inputs for Privacy-Preserving Inference
Authors: Xin Zhou
Yi Lu
Ruotian Ma
Tao Gui
Qi Zhang
Xuanjing Huang
Venue: EMNLP 2023 Findings (2023)
Keywords: Privacy-preserving Inference
Multi-input Multi-output network

Abstract: Pre-trained language models (PLMs) are often deployed as cloud services, enabling users to upload textual data and perform inference remotely. 
However, users' personal text often contains sensitive information,  and sharing such data directly with the service providers can lead to serious privacy leakage.
To address this problem, we introduce a novel privacy-preserving inference framework called \textbf{\textit{MixPi}}, which  prevents plaintext leakage during the inference phase. 
Inspired by $k$-anonymity, MixPi aims to obfuscate a user's private input by mixing it with multiple other inputs, thereby confounding potential privacy attackers. 
To achieve this, our approach involves: (1) proposing a novel encryption module, Privacy Mixer, which encrypts input from three distinct dimensions: mixing, representation, and position. 
(2) adopting a pre-trained Multi-input Multi-output network to handle mixed representations and obtain multiple predictions.  
(3) employing a Privacy Demixer to ensure only the user can decrypt the real output among the multiple predictions.
Furthermore, we explore different ways to automatically generate synthetic inputs required for mixing.
Experimental results on token and sentence classification tasks demonstrate that MixPi greatly surpasses existing privacy-preserving methods in both performance and privacy.

TextMixer: Mixing Multiple Inputs for Privacy-Preserving Inference
Xin Zhou',, Yi Lu', Ruotian Ma', Tao Gui Qi Zhang't, Xuanjing Huang!
'School of Computer Science, Fudan University, Shanghai, China Institute of Modern Languages and Linguisties, Fudan University, Shanghai, China International Human Phenome Institutes, Shanghai, China (xzhou20, tgui, qz | fuuan. edu. cn, yilu230m. fudan. edu on
Abstract
Inference Service
Privacy Attacker
Pre-trained language models (PLMs) often deployed as cloud services, enabling users upload textual data and perform inference remotely. However, users' personal text often contains sensitive information, and sharing such data directly with the service providers can lead to serious privacy leakage. To address this problem, we introduce novel privacy-preserving inference framework called TextMixer, which prevents plaintext leakage during the inference phase. Inspired by k-anonymity, TextMixer aims obfuscate a user's private input by mixing it with multiple other imputs, thereby confounding potential privacy attackers. To achieve this, our approach involves: (1) proposing a novel encryption module, Privacy Mixer, which encrypts input from three distinct dimensions: mixing, representation, and position. (2) adopting a pre-trained Multi-input Multi-output network to handle mixed representations and obtain multiple predictions. (3) employing Privacy Demixer to ensure only the user can decrypt the real output among the multiple predictions. Furthermore, we explore different ways to automatically generate synthetic inputs required for mixing. Experimental results token and sentence classification tasks demonstrate that TextMixer greatly surpasses existing
Embedding Inversion Attack
Regeoentation
Predictios
Mixsed Regraentation
Mileed Prediction
Users
TextMixor
Secocuia
INPUT>
Mising! waaTs
Comparison of the baseline (left) with our method (right). Without privacy protection, word representations can be easily restored to plain text. Our method utilizes mixing to conceal the real input among multiple inputs, thereby confusing privacy attackers and preserving the plain text privacy during inference phase.
Figure I:
NLP tasks, without requiring high-performance hardware or expertise. However, the widespread adoption of PLM services also introduces a risk of privacy leakage (Jegorova et al., 2022). Users' textual inputs may contain sensitive information, including names, schedules, and even business secrets. Leaking such private information to service providers is unacceptable most users.
To address users' privacy concerns and comply with relevant laws', service providers provide inference services. Many sought to make users query PLM services with word representations rather than plaintext. But digital word representations still need additional privacy protection because recent studies (Pan et al., 2020; Song and Raghunathan, 2020) have revealed that standard word representation can be easily restored to its original word under embedding inversion attacks (Höhmann et al., 2021).
methods both performance and privacy.
1 Introduction
The emergence of pre-trained language models (PLMs) (Devlin al., 2018; Liu al., 2019; Brown al., 2020) has significantly increased the demand for cloud inference services. By uploading their textual inputs to the cloud, users can benefit from PLMs and achieve superior performance on various
One solution to apply cryptographic tech- niques to the PLM (Hao et al.; Li et al., 2022; Zheng et al., 2023; Chen et al., 2022), but these methods often come with significant communi-
"Equal contribution.
hhtpp://wwwwconsliimmccmm http::/swwwt..ttss..ce/c////ppie poteccioodaa.pmncciiooeeggaallli
"Corresponding authors.

---

cation costs and computational time, making difficult apply them in real-world scenarios. Another potential solution to remove the private information in word representations (Pan et al., 2020) through adversarial training (Li et al., 2018; Coavoux et al., 2018; Plant et al., 2021) and differential privacy (Lyu et al., 2020a: Hoory et al., 2021; Yue et al., 2021). However, the private information in our scenario pertains to each word in the user's plaintext. Removing the word information from word representations is highly unnatural and often leads to crucial performance degradation (Zhou et al., 2022).
We propose mixing-centric encryption method and explore several approaches for synthetic input generation.
Experimental results various token and sentence classification datasets demonstrate TextMixer's effectiveness in privacy and performance.
2 Methodology
In this section, we present a novel privacy- preserving inference framework TextMixer, which achieves k-anonymity (Sweeney, 2002) by mixing multiple inputs during the inference phase.
In this paper, we propose novel privacy- preserving inference paradigm named TextMixer. Our method is inspired by k-anonymity (Sweeney. technique that
2.1 Overview
We take an example show how k-anonymity contributes to peraccypresrrving inference. To hide the private text "John", TextMixer mixes the word representation of "John" with k-1 synthetic ones (such as "Mike", "Smith", "Johnson", etc.), Only the mixed representation is shared with privacy attackers, making it difficult for privacy attackers to identify which word is real. As shown in Figure 2, our method consists of three steps: (1) User first generates k-1 synthetic representations ($2.5) and inputs representations into the Privacy Mixer ($2.3), which uses various encryption methods with mixing as the core to combine the real and synthetic inputs, creating mixed and encrypted representations: (2) Cloud server receives the mixed representations, and inputs them into pre-trained MIMO network ($2.2), which can handle mixed representations formed by multiple inputs and output their outputs; (3) User uses a Privacy DeMixer ($2.4) to decrypt the desired prediction from the multiple outputs locally. In the following subsection, we provide detailed introduction to each part of TextMixer. We also conduct theoretical privacy analysis of Text in Appendix B.
2002), a classic ensures each individual's s information cannot be distinguished from at least k-1 other individuals. For that purpose, TextMixer aims to make each real word representation indistinguishable by mixing it with multiple synthetic representations during the inference phase, rather than removing word information, shown in Figure 1.
However, directly mixing the representations of k inputs leads to information interference, and normal PLMs cannot handle the mixed inputs to acquire the desired output. To address the above problems, we resort to Multi-input Multi- output (MIMO) network (Ramé al., 2021; Murahari al., 2022, 2023), allows us to process mixed representations with minimal information interference and obtain multiple predictions si- multancously. We further design a privacy mixer to enhance TextMixer's privacy, which encrypts the real inputs from three dimensions. For each forward pass, the privacy mixer applies anony- mous mixing, representation perturbation, and position shufiliigg encrypt the real inputs. Only the user holds the correct keys to decrypt the real prediction from mixed representation. In this way, TextMixer preserves the privacy of both the input and prediction. We also conduct an investigation into different methods for generating synthetic inputs. Our contribution can be summarized as follows2:
2.2 MIMO Network
We first adopt the mixing process of MIMO Net- work (Murahari et al., 2023) (called vanilla mixing here) to our scenarios. Vanilla mixing transforms different inputs into separate spaces before mixing, thus reducing representation interference between multiple inputs.
We propose TextMixer, a inference method that obfuscates privacy attacks by mixing private input with synthetic inputs.
Given user's real input sequence X" [X] fi....,,, x,]] with n word representations, and
We release our code at https::/ / http:/tttt.tooooo///ooooooo thub. com/ MllutyyiTTexMMiee
- - synthetic inputs [x%),1 where X

---

(3)Privacy Deflieer
C2) Server
DeMix
Real Prediction
Eecrrpied
MIMO
Ouspets
Network
Overyse Nrall Ohppo
Divrrys Mudtiple mujues
Deveyg Prsitiion
Encrypted Inpats
(r)vviyy Mixer
Real Input
Synthetic Input
Mising Eneryptiin
Representation Encryption
Vanilla Mising
Position Emeryption
Generating
Growerate
Shuffle
Prytush
lepurt
Frputs Order
Mandow Representatione
Anonymouus Inputr
Word Position
Real Imput
Systhitic Imput
Mixel Input
Ouipui
Synthctic Ouipui
Mixal Outpul
Random Novie
Figure 2: An overview of TexIMixer's inference. Privacy Mixer first enerypts the real input locally on the user's device, obtaining the encrypted input through three aspects: anonymous mixing, perturb representation and random position. The MIMO network deployed in the cloud only received encrypted inputs for subsequent inference and returns the encrypted output back to the user. With the help of the user's private keys, Privacy DeMixer can decrypt the real prediction from the encrypted output.
ix., ) we aims to mix these representation generate mir
from the over- paraamettttre network and MIMO pre-training (Murahari al., 2023).
[xmii ...., Xn min], For simplicity, we denote k words to be mixed at
the i-th position as x!* [x, x x; 1h To avoid misunderstanding, we "position" represent the word position in an input sequence, and "order" to represent the order of l inputs. For example, x means the word representation at i-th position from j-th input. To ensure that the mixed representation still retains the individual semantics of each input, we need to perform transformation before mixing. Transformation maps word representations from different inputs to separate spaces and consequently reduces interference between their representations. The process transformation and mixing can be defined as:
2.3 Privacy Mixer
However, vanilla mixing process is not a good privacy protector due to the lack of randomness. For example, equation shows that different inputs will be mapped to diffcrent spaces. If the real input is always mapped to fixed space, privacy attackers can easily identify this fixed pattern, thereby restoring the real input from the mixed representation (shown in Table 2). Thus, design privacy mixer, which extends the vanilla mixing into real encryption strategy from three dimensions: anonymous mixing, perturbed representation, and random position.
Mixing Eneryption. Inspired by k-anonymity, this encryption aims to make each real represen- tation x indistinguishable from synthetic ones }ドー As mentioned before, the real input cannot be mapped to a fixed space, thus we shuffle inputs to get random mixing order, allowing the real input to be anonymous:
mir
oe'
(1)
[x]
where x² is j-th representation in x! and e¹ C Rd ~ IO, is transformation vector sampled from a standard multivariate Gaussian distribution. Next, input mixed representations into MIMO network obtain multiple predictions. Then we demix the real prediction from the multiple predictions, as shown in $2.4. The structure of MIMO network the same as BERT, its ability to handle mixed representation comes
RAE X
L:k
[X]
X x.',
(2)
where xI* kxd and RME E (0, 1],1, is a random permutation matrix. Equation 2 represents only one possible shuffling result. RA

---

use RME is generated by the user's side and not shared with the cloud.
For simplicity, we only consider the representa- tion in position i. De-mixing means extracting h final representations ! E R from h Rd where h are the final representations different inputs x!:k The final representation j-th input in x; can be obtained by:
Representation Encryption. To obscure the representation itself, we add random perturbations each representation of random input, as shown in Figure 2. Following Plant et al. (2021), we adopt Laplace Noise as the random perturbation. Formally, the process mixing encryption and representation encryption for x, is:
h DeMix ([h, d'l).
(6)
where DeMix is two layer MLP and d' e R is private vectors to de-mix the j-th representation from mixed representation and [+; #] means con- catenation. The de-mixing ability of DeMix and d' comes from pre-training (Murahari et al., 2023). We show that DeMix and d' are not helpful restoring the real input in $4.3.
Ene(Lik) RAEE [X x
Lap(e) (3)
where e a hyperparameter controlling the scale of noise, smaller values of correspond to a larger scale of noise. Lap(c) can be added to any representation, including real input x;.
After de-mixing, we restore an ordered set h [ ht then decrypt the original order by RM E h!! obtaining the real input's final representation H' Raxd Subsequently, H' fed into the classification layer to obtain the prediction desired by the user.
Then we mix k representation x; Enc(1sk) at each position i, according to equation 1. Finally, obtain the mixed representations X 1
Position Encryption. To further enhance the privacy of X position encryption aims to reduce its human readability, We utilize random permutation matrix, denoted Reg (0, n shuffle word positions in mir::
2.5 Synthetic Input Generating
As mentioned before, our method requires k-1 synthetic inputs for mixing. We now explore principled ways of generating synthetic input.
X PE = R. mir [x, X1
mir
miz
Generating with Real Data. The most intuitive method is to select the real training data as the synthetic data. Given a real input sentence, we sentences from the training
this way, even privacy attackers can restore some text from mixed representation, they cannot obtain a readable sequence. We show that XpE and mii are equivalent for PLMs in Appendix D. Similar to Mixing Encryption, only the user holds the private key R.rg-
randomly sample set and encrypt these sentences.
Generating with Similarity. In this approach, leverage the similarity for synthetic input generating. Given real word Xy, we select the closest word w, in the embedding space as its synthetic word:
Finally, users upload encrypted representations, mir the cloud for further computation. pre- trained MIMO network processes the X and sends back the final representation Hpir E Rrxd to the user's side for de-mixing the real prediction.
W.g = arg max 1//w x,1l.
(7)
2.4 Privacy DeMixer
where w is the embedding matrix PLM.
The goal of Privacy DeMixer is to decrypt the encrypted final representations Hpp and obtain the desired prediction, with the help of user's personal keys Reg and RME. First, we use Reg to restore original word position:
Generating with Input Itself. This method is inspired by Zhou et al. (2022) that fuses representation within the same sentence. Given real input X = [x[, -.,, x/I, [x, for each x,, we random sample k representations from X" (excluding x itself) as the synthetic inputs.
mix
R
miz
(5)
PE
Next, we de-mix these mixed final representations mir to get final representations corresponding to different inputs.
where Hmiz [
2.6 Training Details
TextMixer supports both token-level and sentence- level classification tasks. During the training

---

process, randomly sample k inputs from the training set and only enable representation encryption. We add random noise to a random input sequence and then mix all the inputs together. The mixed input is sent to a pre-trained MIMO network to get the final representation. Without any decryption process, we directly use DeMix to obtain the final representation of each input, which is then fed into the output layer to predict the task label. In addition the task loss, a retrieval loss is added promote the network's ability to handle multiple inputs (Murahari et al., 2023). The final loss can be represented as follows:
as input and outputs each representation's original word one-to-one. MLC (Song and Raghunathan, 2020) similar to InvBert, the difference is that MLC performs multi-label classification tasks, predicting all the words that have appeared without emphasizing the order.
Defense Baselines. DPNR (Lyu et al., 2020b) uses differential privacy to obfuscate representa- tions. CAPE (Plant et al., 2021) further eliminates privacy information by incorporating adversarial training and differential privacy. SanText+ (Yuc et al., 2021) uses differential privacy and word frequency to replace sensitive words. TextFusion (Zhou et al., 2022) employs word fusion and misleading training to hinder privacy attackers from training targeted embedding inversion models. We attack it with special InvBert due to different settings, as shown in Appendix C. DataMix (Liu et al., 2020) proposes to mix and enerypt the image with random coefficients.
Cretrieval
Σ -loggP(w!ll)
(8)
L -Lask + afrcctiioall
(9)
where a is a hyperparameter, hy is final repre- sentation of the j-th sentence at position i, and wf is the word corresponding to h/. During the inference phase, we can directly enable all encryption strategies and switch the synthetic input generation methods without retraining.
3.3 Evaluation Metrics
3 Experimental Setup
We consider privacy metrics at both the word level and the sentence level. At the word level, we use the Top-k, which refers to the proportion of real words among the top k predictions generated by the attack methods for given representation. At the sentence level, we use the RougeL (Lin, 2004), text generation metric, to measure the similarity between the restored sentence and the original sentence. For MLC Attack, we use the Token- Hit (Zhou et al., 2022), which treats the words real input as set, calculating the percentage of predicted words in the raw words.
3.1 Datasets
We conduct experiments four representative datasets. Sentence classification: we select IMDB (Maas al., 2011) for sentiment analysis and AGNews (Zhang et al., 2015) for topic classification. Token classification: we select NER (Named Entity Recognition) as the main task, including CoNLL2003 (Tjong Kim Sang and De Mculder, 2003) and Ontonotes (Weischedel et al., 2013). These tasks are highly relevant to real- world scenarios and datasets often involve sensitive information such as names and locations. Thus they serve as suitable benchmarks for evaluating both privacy and performance. The details of these datasets are Appendix A.
3.4 Implementation Details
We show three important implementation details below, and present the remaining details Ap- pendix G. (1) All attack and defense methods are performed at the embedding layer, before any transformer modules. Users only need to convert words into embeddings and send them to the cloud. (2) The pre-trained MIMO network is based on BERT-base, thus we also chose BERT-base the backbone train task models for all baselines. well as the inversion models for InvBert and MLC.
3.2 Baselines
For a thorough comparison, we select the widely used privacy attack and defense baselines.
Attack Baselines. Following Zhou et al. (2022), we select three privacy attack baselines. KNN (Qu al., 2021) selects the closest word in the embedding space as the real word. InvBert (Höhmann et al., 2021) trains an embedding inversion model that takes word representations
(3) Inversion models for each dataset are trained its training set.

---

KNN Top-5
InvBert Top-5
MLC
Dataset
Method
Task f
Top-1
Rouge
Top-1
Rouge
Token-HHt
Fine-turing (Devlin al., 2018)
94.10
100
91.70
100
51.13
DPNR (Lyu et al., 2020b)
85.00
0.11
58.42
0.08
99.82
99.99
99.17
43.01
CAPE (Plant et al., 2021)
85.88
5.70
71.78
4.12
100
100
99.28
48.05
IMDB
SanText+ (Yue et 2021)
50.16
69.21
69.27
59.19
81.24
73.47
42.48
DataMix (Liu et al., 2020)
50.36
29.51
76.08
26.57
66.48
98.20
60.55
28.13
TextPusion (Zhou al., 2022)
90.60
39.46
85.30
46.17
48.93
91.94
61.38
54.43
TextMixer (Ours)
90.93
0.00
0.00
0.00
12.17
36.07
0.35
21.11
Fine-tuning (Devlin et al., 2018)
94.92
100
100
93.66
100
100
99.81
89.31
DPNR (Lyu ct al., 2020b)
90.01
4.11
24.35
3.64
28.50
45.05
28.94
27.22
CAPE (Plant et al., 2021)
90.22
0,36
1.18
0,36
5.73
10.72
5.91
22.13
AGNows
SanText+ (Yuc ct al., 2021)
90.80
65.24
65.28
52.90
70.11
76.94
70.28
53,91
DataMix (Liu ct al., 2020)
91.62
59.19
98.41
55.68
95,26
99.85
89.14
34,94
TextFusion (Zhou et al., 2022)
91.89
7:21
26.80
4.22
49.03
87. .9.6
59.61
71.01
TextMixer (Ours)
92.23
0.00
0.00
0.00
2.22
5.62
0.10
16.09
Finc-tuning (Devlin al., 2018)
90.47
100
100
95.01 01
100
100
99.83
61.47
DPNR (Lyu et al., 2020b)
39.09
13.86
30.81
9.96
11.27
23.55
11.37
25,46
CAPE (Plant et al., 2021)
70.53
0.11
5.90
0.06
37.44
45.28
37.91
27.56
CoNLL2003
SonTexte (Yue al., 2021)
72.19
76.56
76.85
55.24
75.62
82.02
75.74
40.93
DataMix (Liu al., 2020)
2.38
63.69
97.82
60.96
78.98
97.24
74.80
39.32
TextFusion (Zhou al., 2022)
77.53
4.91
21.40
2.64
22.27
27.02
49.24
55.09
TextMixer (Ours)
86.24
0.00
0.00
0.00
4.14
11.26
0.93
18.45
Fine-tuning (Devlin et ul., 2018)
87.07
100
100
95.01
100
100
99.82
81.11
DPNR yu el al., 2020h)
42.10
10.64
41.24
9.26
24.84
42.85
25.22
31.17
CAPE (Plant et al., 2021)
71.23
1.97
11.42
1.51
22.66
29.10
22.77
30.68
Ontonotes 5.0
SanText+ (Yue et al., 2021)
65.32
73.66
73.82
55.89
73.22
80.69
73.36
48.61
DataMix (1 Luu el al., 2020)
0.21
65.76
97.75
62.52
90.39
99.11
85.67
55.03
TextFusion (Zhou et al., 2022)
79.30
3.86
26.13
0.65
21.31
25.35
39.81
73.89
TextMixer (Ours)
82.38
0.00
0.00
0.00
11.45
31.12
0.61
20.95
Table Main results on four classification datasets. All attack and defense methods are carried out at the embedding layer. Task denotes the metric for the task performance, report accuracy for IMDB and AGNEWS, FI score for CoNLL2003 and Ontonotes5.0, Top-1, Top-5, Rouge, and Token-Hit are the privacy leakage metric. For task performance, higher is better; for privacy leakage metric, lower is better. The best performance and privacy defense methods are highlighted in bold.
4 Experimental Results
lower privacy leakage than most baselines across all datasets. With the help of our proposed encryption strategy and MIMO network, TextMixer avoids contradictory behavior, thus achicving good performance while preserving privacy.
In this section, we show the results of privacy and task performance in $4.1 and the ablation study in $4.2. We also design three specific privacy attacks for TextMixer and show the attack results in $4.3. The inference cost is shown in Appendix F.
Previous works usually require deploying mul- tiple transformer layers in users' devices to achieve satisfying performance. But when applied to the embedding layer, we observe that most baselines difficult provide privacy protection. CAPE and DPNR usually need to trade a lot of task performance for the ability privacy, which can attributed to the inherent contradiction that we discussed above. SanText+ relies on word replacement to protect privacy, but word replacement cannot handle token-level tasks, which require much token information, and tasks with long sequences, which require lots of words to be replaced, such IMDB. TextFusion fuses most words to reduce Top-1, resulting in serious performance degradation. But the performance of Top-5 and MLC of TextFusion remains high, indicating privacy risk is still high under our strict
4.1 Main Results
Main results baselines and TextMixer are listed in Table I. From the result,
that (I) Normal word representations without any defense method suffer from privacy leakage risk. Almost all privacy attacks achieve 100% success rate in fine-tuning, which means that privacy is completely leaked to attackers. (2) Preserving privacy in the embedding layer is an extremely challenging task. Despite compromising signif- icant performance, most baselines are unable to provide satisfactory privacy protection. This is due the inherent contradiction of removing private word information from word representations, as discussed the introduction. (3) Our proposed TextMixer achieves better task performance and
protect

---

privacy setting. DataMix fails perform well on most datasets, and we believe there are two reasons. First, its encryption mechanism friendly to convolutional networks but not suitable for PLMs that involve more non-linear operations. Second, DataMix is designed for images, whereas we require encrypting representation sequence. which is harder task. These two difficulties cause it to fail on PLMs and textual data.
Paffomance
130
Pivooy
0060
bual
under IndieerAlee Altec
50
Hop derbury
80
20
4.0L
ACHena
CoNLLoo
AChiwa
CoNLUOS
Figure 3: Effect of different synthetic inputs. Privacy metric is Top-1 under InvBert attack. Performance metric is accuracy for AGNews and Fl for CONLLO3.
4.2 Ablation Study
In this subsection, explore the effect of encryption strategy and synthetic input generation. We also show the effect of mixed inputs number and noise scale in Appendix E.
that these synthetic inputs differ significantly from the words that would appear in the real world. During the training process, the powerful inversion model gradually eliminates this irrelevant information and identifies the feature of the real imput, resulting successful attack. Real Data and Input Itself achieve first and second rankings in preserving privacy, Both of these synthetic inputs are generated based on real data, which enlightens us that the closer synthetic inputs approximate real-world scenarios, the greater their potential to mislead privacy attackers. The substitution of synthetic inputs has only marginal impact on performance, both in token-level tasks and sentence-level tasks, indicating users can replace the synthetic inputs generation method any time without the need for retraining.
Effect of encryption strategy. From Table 2, we can see that mixing Encryption plays an important role in privacy. Without mixing encryption, privacy attackers can casily identify the space of real input and accurately restore the original words, resulting in high privacy leakage. Representation Encryption prevents privacy attackers by adding noise to representation. Similar to the previous work, adding noise hurts task performance, but an appropriate noise would not result in much performance loss. This is acceptable as it contributes largely to privacy. Position Eneryption is not designed to prevent attackers from restoring words, but it significantly reduce readability, resulting substantial decrease in Rouge scores. From the above results, can be observed that our proposed encryption techniques reduce privacy leakage risks from different perspectives while having minimal impact on performance.
4.3 Stronger Privacy Attacks
Although TextMixer can defend the powerful embedding inversion attack, there are still concerns about potential privacy leakage under stronger privacy attacks. Therefore, we design some attacks for our encryption method.
Dataset
Method
Tosk
Top-1
Rouge
Top-1
Rouge
PIA
DMA Top-5
TextMixer
92.32
0.00
0.00
2.22
0.10
Dataset
Acc.
Top-1
Rouge
Mix.
91.73
0.00
0.00
89.22
2.07
ACNews
Rope.
92.63
0.00
0.00
6.57
0.13
AGNews
53.64
0.00
0.00
0.00
Pos.
92.32
0.00
0.00
2.22
2.19
CoNLLO3
48.09
0.00
0.43
0.00
TextMixer
Mix.
0.00
Table 3: Results of DeMixing Attack (DMA) and Position Inversion Attack (PIA).
CONLLU3
Repr.
86.69
0.00
0.00
15.53
1.34
Pos.
86.24
0.00
0.00
4.14
4.20
Table 2: Effect different encryption strategies. Mix. means Mixing Encryption, Repr. means Representation Encryption and Pos. means Position Encryption.
DeMixing Attack is designed for Mixing Encryp- tion. We verify whether DeMix and dś, which are used to demix the outputs, helpful to demix the mixed input. We assume the privacy attacker knows the exact order of the real input. For instance, they know the j-th input is the real one, thus they can use DeMix and d' to demix the mixed input and obtain the word representations of j-th input. Then
Effect of Synthetic Inputs. From Figure 3, surprisingly find that the similarity-bar -based methods do not confuse the inversion model but instcad lead to more severe privacy leakage. We speculate

---

Tra Traint
Real Data
Input lself
High Similarity
Low Similarity
and Shmatikov, 2019; Pan et al., 2020) shows that even uploading word representations can still leak privacy, as the embedding inversion attack (Höhmann et al., 2021) can restore word representations to their original words.
Real 3.48
Input Itself
9.03
23.09
0.02
High Similurity
Low Similarity 0.60
0.17
0.86
59.50
Table 4: Results of Cross Inversion Attack on CoNI.LO3. Columns represent the training data for InvBert, while the rows represent the testing data.
Privacy-preserving Methods. To avoid users' private plaintext from leaking to privacy attack- ers, many existing works adopt cryptographic techniques such as Homomorphic Encryption (Gentry, 2009; Chen et al., 2022) and Secure Multiparty Computation (Evans et al., 2018; Li et al., 2022) into PLM (Hao et al.; Zheng et al., 2023). Although theoretically guaranteed, these methods typically require much higher time and communication costs than standard PLM. For example, Hao et al. shows that an encrypted BERT- tiny model requires about 50 seconds and 2GB of communication cost to process 30 words sentence. An alternative that does not require additional inference costs is to employ adversarial training (Li et al., 2018; Coavoux et al., 2018; Plant et al., 2021) and differential privacy (Lyu et al., 2020a; Hoory et al., 2021; Yue et al., 2021) to remove private information in representations. However, the perturbed word representations, which lack sufficient information for privacy attackers to identify their original words, also insufficient for the model to achieve high performance on downstream tasks (Zhou et al., 2022). Therefore, in this work, we propose mixing inputs rather than removing word information to preserve plaintext privacy during the inference phase.
they use an InvBert to predict real words and train DeMix, d' and InvBert jointly. However, attack results (DMA) shown in Table 3 show that DeMix is not helpful for restoring real input, this attack performs even worse than using InvBert directly. We speculate that this is because DeMix is trained demix the output instead of input, strongly interferes with the inversion model, making impossible to attack successfully at all. Position Inversion Attack is designed for Position Encryption. It aims to restore the original word position from the encrypted input. Similar to InvBert, train model to predict the original position of each encrypted representation and show attack results (PIA) in Table 3. The accuracy of restoring the original position only 50%, and the readability of such a sentence remains low. Within TextMixer, is not a threat since embedding inversion attacks can only restore few words from encrypted representations.
Cross Inversion Attack is designed for synthetic input generation, which verifies whether the InvBert trained on one synthetic input generation method works better on another one. From Table 4, we can see that all transferred InvBerts perform worse than InvBert trained and tested in the same synthetic input indicating its poor generalization ability. Privacy attacks' performance primarily depends the synthetic input used during testing.
Comparison with Similar Works. Huang et al. (2020) propose TextHide, which mixes up inputs during the training phase to protect the privacy of training data. Our work focuses on users' plaintext privacy and mixes inputs during the inference phase, which brings new challenges. Liu et al. (2020) propose Datamix to mix multiple inputs during inference and encrypt them with random coefficients. Our encryption method is different from Datamix, and Datamix is designed for image data and convolutional networks, which not performs well in text data and PLM. Recently, Zhou al. (2022) propose TextFusion, which fuses parts of word representations within the same sentence to prevent privacy attackers from training targeted inversion model. Differently, our method mixes representations from different sentences and our privacy setting is more challenging. We discuss
5 Related Work
5.1 Plaintext Privacy during Inference
Privacy Threats. Most PLM services require users to upload their plaintext data to the cloud to perform PLM inference, which leads to a serious risk of privacy leakage (Jegorova et al., 2022). Privacy attackers could be the service provider themselves, they aim to collect users' textual input for their own purposes. For example, they can use these data to train better model or extract users' private information, such as personal details and confidential business information, even is prohibited by law. Recent literature (Song

---

in Appendix C.
privacy has been empirically verified through the powerful and widely used embedding inversion attack, and we have provided privacy proof in Appendix B. (3) TextMixer has only been evaluated at the BERT-like model and has not been migrated to larger models like GPT-3. Limited by computational resources, we can only use the pre-trained MIMO network provided by Murahari et al. (2023). However, we believe that service providers can train more powerful MIMO networks with more parameters and extend them more tasks, such as text generation.
5.2
Multi-input Multi-output Network
Multi-input Multi- network aims to use one neural network to process multiple inputs and obtain their predictions in one forward pass. Havasi et al. (2020) and Ramé et al. (2021) try improve robustness by ensembling multiple predictions in a MIMO convolutional network. Murahari et al. (2022) propose a transformer-based MIMO model for efficient inference and further improve its performance through pre-training (Murahari et al.. 2023). Our work does not focus on robustness or efficiency but rather utilizes the MIMO network tool to construct partial encryption strategies, thereby achieving privacy-preserving inference.
Acknowledgements
The authors wish thank the anonymous reviewers for their helpful comments. This work was partially
funded by National Natural Science Founda- tion of China (22666666577,2276666666111979999
6 Conclusion
In this paper, we propose novel privacy- preserving inference framework TextMixer. Our framework inspired by k-anonymity and makes real input indistinguishable by mixing multiple inputs during the inference phase. Specifically, TextMixer adopts a pre-trained MIMO network, which can process representation composed of multiple inputs and obtain multiple outputs simultaneously, to handle mixed representations. We further propose the privacy mixer to encrypt not only the mixing but also the two dimensions of representation and word position. We also explore various ways to automatically generate synthetic inputs. At the embedding layer, TextMixer outperforms the cxisting privacy-preserving methods by large margin, regardless of token and sentence classification tasks. The detailed ablation and analysis experiments also show TextMixer is an effective crivacy-preserving inference framework.
Shanghai Rising-Star Program (23QA1400200), Shanghai Academic Research Leader Program 22XD1400 DXXD1401100, and Natural Science Foundation
of Shanghai (23ZR 1403500).
References
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss. Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners.
Tianyu Chen, Hangbo Bao. Shaohan Huang. Dong. Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, and Furu Wei. 2022. THE-X: Privacy-presedd preserving transformer inference with homomorphic encryption. In Findings of the Association for Computational Linguisties: ACL 2022. pages 3510-3523 Dublin, Ireland. Association for Computational Linguistics.
eneryption
Limitations
We have summarized three limitations of TextMixer. (1) TextMixer is designed to protect plaintext privacy, and we have only conducted experiments to evaluate its privacy under embedding inversion attacks. However, TextMixer provides the k-anonymity by mixing multiple inputs, it has the potential address other privacy concerns, such as resisting Attribute Inference Attacks. (2) TextMixer's privacy guarantee is based on k-anonymity, which is not as rigorous as homomorphic encryption. Although TextMixer's
Maximin Coavoux, Shashi Narayan, and Shay B Cohen. 2018. Privacy-preserving neural representations of text. arXiv preprint 1rXiv:1808.09440
Jacob Devlin, Ming- Wei Chang, Kenton Lec, and Kristina Toutanova. 2018 Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1rXiv:1810.000
Ye Dong, Wen jie Lu, Yancheng Zheng, Haoqi Wu, Derun Zhao, Jin Tan, Zhicong Huang, Cheng Hong, Tao Wei, and Wenguang Chen. 2023. Puma: Secure inference of llama 7b in five minutes.

---

David Evans, Vladimir Kolesnikov, Mike Rosulek, et al. 2018. A pragmatic introduction to secure multi-part computation. Foundations and Trendy in Privacy and Security, 2(2-3):70-246.
Zhijian Liu. Zhanghao Wu, Chuang Gan, Ligeng Zhu, and Song Han. 2020. Datamix: Efficient privacy- preserving edge-cloud inference. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings Part XI 16, pages 578-595. Springer.
Craig Gentry. 2009. Fully homomorphic encryption using ideal lattices. In Proceedings of the Forty-First Annual ACM Symposium on Theory of Computing, STOC '09, page 169-178, New York, NY, USA. Association for Computing Machinery.
Nlya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization.
Meng Hao, Hongwei Li, Hanxiao Chen, Pengzhi Xing. Guowen Xu, and Tianwei Zhang. Iron: Private inference on transformers. In Advances in Newral Information Processing Systems.
Lingjuan Lyu, Xuanli He, and Yitong Li. 2020a. Differentially private representation for nip: Formal guarantee and an empirical study on privacy and fairness. arXiv preprint arXiv:2010.01285.
Marton Havasi, Rodolphe Jenatton, Stanislav Fort, Jeremiah Zhe Liu, Jasper Snoek, Balaji Laksh- minarayanan, Andrew M Dai, and Dustin Tran. 2020. Training independent subnetworks for robust prediction. arXiv preprint arXiv:2010.06610.
Lingjuan Lyu, Xuanli He, and Yitong Li. 2020b. Differentially private representation for NLP: Formal guarantee and an empirical study on privacy and faireess. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2355 2365-2365, Online. Association for Computational Linguistics.
Johannes Höhmann, Achim Rettinger, and Kai Kugler. 2021. Invbert: Text reconstruction from contextu- alized embeddings used for derived text formats of literary works. arXiv preprint arXiv:2109. 10104.
Andrew L.. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng. and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142-150, Portland, Oregon, USA. Association for Computational Linguistics.
Shlomo Hoory, Amir Feder, Avichai Tendler, Sofia Erell, Alon Peled-Coher Itay Laish, Hootan Nakhost, Uri Stemmer, Ayelet Benjamini, Avinatan Hassidim, al. 2021. Learning and evaluating a differentially private pre-trained language model. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1178-1 189.
Vishvak Murahari, Ameet Deshpande, Carlos E Jimenez, Izhak Shafran, Mingqiu Wang. Yuan Cao, and Karthik Narasimhan. 2023. Mux-plms: Pre- training language models with data multiplexing. arXiv preprint 230.
Yangsibo Huang, Zhao Song, Danqi Chen, Kai Li, and Sanjeev Arora. 2020. TextHide: Tackling data privacy in language understanding tasks. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1368-1382, Online. Association for Computational Linguistics.
Vishvak Murahari, Carlos Jimenez, Runzhe Yang. and Karthik Narasimhan. 2022. Datamux: Data multiplexing for neural networks. Advances Neural Information Processing Systems, 35:17515- 17527.
Marija Jegorova, Chaitanya Kaul, Charlic Mayor, Alison O'Nell, Alexander Weir, Roderick Murray- Smith, and Sotirios A Tsaftaris. 2022. Survey: Leakage and privacy at inference time. IEEE Transactions Puttern Analysis and Machine Intelligence.
Xudong Pan, Mi Zhang, Shouling Ji, and Min Yang. 2020. Privacy risks of general-purpose language models. In 2020 IEEE Symposium on Security and Privacy (SP), pages 1331-1331. IEEE.
Dacheng Li, Rulin Shao, Hongyi Wang, Han Guo, Eric P Xing, and Hao Zhang. 2022. Mpcformer: fast, performant and private transformer inference with mpc. arXiv preprini arXiv:2211.0144
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Confer- ence Empirical Methods in Natural Language Processing (EMNLP), pages 1532-1543. Doha, Qatar. Association for Computational Linguistics.
Yitong Li, Timothy Baldwin, and Trevor Cohn. 2018. Towards robust and pivaccypreservvng text representations. arXiv preprint
Chin-Yew Lin. 2004. Rouge:
A package for automatic evaluation of summaries. In Text summarization branches ouf, pages 74-81.
Richard Plant, Dimitra Gkatzia, and Valerio Giuffrida. 2021. CAPE: Context-avee -warr private embeddings for private language learning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7970-7978, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal. Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: robustly optimized bert pretraining approach.

---

Chen Qu, Weize Kong, Liu Yang, Mingyang Zhang. Michael Bendersky, and Mare Najork. 2021. Natural language understanding with Sivvacy-preseving bert. In Proceedings of the 30th ACM International Con- ference on Information Knowledge Management, pages 1488-10 1497.
A Datasets
we conduct experiments on 4 classification tasks and the statistics of datasets in our experiments are shown in Table 5. For all the datasets, we use the test data for evaluation. We illustrate the details of each dataset as follows:
Alexandre Ramé, Rémy Sun, and Matthieu Cord. 2021. Mixmo: Mixing multiple inputs for multiple outputs via deep subnetworks. In Proceedings of the IEÉE/CVF International Conference on Computer Vision, pages 823 833.
IMDB (Maas et al., 2011) contains a large collection of movie reviews along with their corresponding sentiment labels, indicating whether the review is positive or negative.
Congzheng Song and Ananth Raghumathan. 2020. Information leakage in embedding models. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pages 377-390.
AGNEWS (Zhang et al., 2015) provides a balanced distribution of articles across the World, Sports, Business, and Science/Technoloy, making suitable for training and evaluating text classifi- cation models.
Congzheng Song and Vitaly Shmatikov. 2019. Over- learning reveals sensitive attributes. arXiv preprint arXiv:1905.1174..
CoNLL2003 (Tjong Kim Sang and De Mculder, 2003) popular benchmark dataset for named entity recognition (NER) tasks. It consists of English and German news articles annotated with named entity labels such as person names, loca- tions, organizations, and miscellaneous entities. OntoNotes5 (Weischedel et al., 2013) contains text from various domains, including news, con- versations, and web data, and is annotated with detailed named entity labels such as person names, locations, organizations, and more.
Latanya Sweeney. 2002.
k-anonymity: A model for protecting privacy. International journal of uncertainty, fuzzzness and knowledge based systems, 10(05):557-570.
Erik Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: I Leggeeeeindeennd named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142- 147.
Dataset
# Train
#Test
#Labels
#Average Length
Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, et al. 2013. Ontonotes release 5.0 1dc2013t19. Linguistic Data Consortium, Philadel- phia, PA, 23.
IMDB AGNEWS
25000
25000
2
272.26
120000
7600
51.80
CoNLL2003
14041
3453
9
19.21
OntoNotes5
59924
8262
37
21.19
Xiang Yue, Minxin Du Tianhao Wang. Yaliang Li, Huan Sun, and Sherman S. M. Chow. 2021. Differential privacy for text analytics via natural text sanitization. In Findings, ACL-IJCNLP 2021.
Table 5: Statistics of the datasets.
B Privacy Analysis
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28.
Given a real input X = [x1, X2, with length and its synthetic inputs (X.)
Mixing Encryption Mixing encryption mixes k word representations based on Eq. and Eq. 2. There are a total of k possible word representations (1 real + k1 synthetic). All representations are assumed to be equally likely. Besides that, The mixed representation is more confusing than the isolated representation, which further reduces the success rate of attacks. Hence, the probability of correctly guessing the real input is less than the reciprocal of the total number of inputs, P(Mix Σ
Mengxin Zheng, Qian Lou, and Lei Jiang. 2023. Primer: Fast private transformer inference on encrypted data. arXiv preprint arXiv:2303.11 13679.
Xin Zhou, Jinzhu Lu, Tao Gui, Ruotian Ma. Zichu Fei, Yuran Wang, Yong Ding, Yibo Cheung, Qi Zhang, and Xvanjing Huang. 2022. TextFusion: Privacy- preserving pre preitrdined model inference via token fusion. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8360-837 8371, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

---

Representation Eneryption Representation en- cryption adds random noise to a random represen- tation before mixing. "This encryption method draws on differential privacy, which increases the level of confusion in the representation. Although it is difficult to analyze the theoretical privacy protection capability, we provide empirical results on the impact of different noise scales on privacy and performance in Appendix E.
Num.
Task
Privacy
89.5
64.1 1
2
0.5
88.22
32.24
0.2
81.29
4.48
1
87.15
6.96
5
0.5
86.65
3.94
0.2
83.32
1.62
1
84.39
1.64
Position Encryption Given shuffled sentence of length n, there are n! (n factorial) ways to arrange n items. All arrangements are assumed to be equally likely. Hence, the probability of correctly guessing the original order should be the reciprocal of the total number of arrangements: P(Pos_Attace = - Although attackers can exploit rules such grammar and positional information to eliminate and increase the success rate of attacks, there would still be many confusing candidates left, We also designed privacy attacks in $4.2 verify this.
10
0.5
83.49
1.57
0.2
81.91
1.26
Table 6: Effects of mixed inputs numher and noise scale CoNLLO3. "Num." " denotes the number of mixed inputs. "c" represents the noise parameter. A smaller value of corresponds to a larger noise scale. "Task" means task performance, specifically the Fl score for CoNLL03. "Privacy" refers to the privacy metric, which is the Top-1 accuracy the Inversion attack. 1 indicates that higher values better, while 4 suggests that lower values are better.
result, the position information is only provided by position embedding, which is added to each input before mixing. During the inference phase, the subsequent calculations (self-attention, softmax, feed-forward layer, etc.) are independent of the explicit representation position and only depend the implicit positional features within the representation. As a result, we can shuffle the mixed representations without any impact on performance. However, the shuffled word positions can reduce the readability for humans, which confuses privacy attackers and enhances privacy.
C Discussion about TextFusion
TextFusion (Zhou al., 2022) aims fuse token representations and prevent privacy attackers from collecting ideal data and training a one-to- one inversion model. In their privacy setting, privacy attackers come primarily from third parties. However, our setting is stricter than TextFusion, service providers can also be potential privacy attackers, they release the model weights, can design special attack methods, and collect any training data they want. We propose simple attack method and use our experiments. For token-level tasks, we train an inversion model that only restores unfused tokens and directly uses this model to attack. For sentence-level tasks, we train inversion to restore one original word from fused representations. As shown in Table 1, the success rates of both attack methods considerably high, indicating that our scenario necessitates stronger methods for privacy protection.
E Effect of Mixed Inputs Number and Noise Scale
In this section, we show the atility-privacy trade-off caused by the number of mixed inputs and noise scale. We conducted experiments on CoNLLO3, and Table shows the experimental results for mixed inputs numbers of [2,5, 10) and noise parameters e of [1, 0.5, 0.2]. lower means larger noise. It can be observed that increasing the noise scale appropriately can significantly prevent privacy leakage (64.1 -> 32.24 for N=2) with an acceptable decrease in performance (89.5 88.22 for N=2). In addition to this, we can observe that incorporating mixed inputs results in a more favorable balance between utility and privacy, underscoring the significance of mixing.
D Discussion about Position Encryption
In $2.3, mention that Xmir and miir equivalent for PLM, which means that even if the word positions are shuffled, does not affect the subsequent model inference. Here we provide a brief explanation, we use a Bert-like model, and no relative position embedding is used. As a

---

FLOPS
FLOPS
Inference Time
Communication
Method
(User)
(Server)
Cost
Fine-tuning (Devlin al., 2018)
0.0003G
10.88G
0.077
384 KB
PUMA (Dong al., 2023)
33.91
10.77GB
TextFusion (Zhou et al., 2022)
0.0075G
5.45G
0.047
192KB
TextMixer (N=5)
0.1080G
10.88G
0.079
384KB
TextMixer (N=10)
0.2157G
10.88G
0.089 s
384KB
Table 7: Inference cost of different privacy-preservie methods. FLOPs stands for floating point operations. Inference Time represents the total inference time of the entire model, measured in seconds. N represents the number mixed words. denotes results are directly taken from (Dong al., 2023).
F Comparison of Inference Cost
(Loshchilov and Hutter, 2019) and linear learning rate scheduler, suggested by the Hugging Face default setup. For all defense methods, we choose Laplace noise disturb the representation and train 30 epochs to guarantee convergence. We prioritize privacy while considering performance when selecting the most favorable outcomes from the experimental results for reporting. The optimized hyperparameters, which yielded the best results, are presented in Table 8.
To show the efficiency TextMixer, we compare the inference cost of different types of privacy- preserving methods, including Fine-tuning, encryption-based methods, TextFusion and TextMixer. The communication cost is how much data the users send. Because inference time vary depending different platforms and implementations, we use both inference time and FLOPs (floating-point operations) as two metrics to measure computational costs. We use sentence of length 128 to query a bert-base model, all experiments are conducted on the same CPU unless otherwise specified.
For TextMixer, the hyperparameters include the number of mixing instances and the scale of Laplace noise of Representation Encryption. We train TextMixer with the number of instances N (2, 10] and the scale of Laplace noise e in [1, 2, 2.5, 5, 6, 7, 8, 10]. For all datasets, we use Real Data as the synthetic inputs due to the better sirrac-perffrmmnce trade-off.
tune
From Table 7, only introduces few additional computation and communication costs compared to fine-tuning. The fast encryption- based method, PUMA, still requires 33.91 seconds and 10.77GB communication cost, which is not convenient in real applications.
find that TextMixer
For DPNRS we take the scale of Laplace noise and the nullification rate for word dropout strategy as hyperparameters, we search the scale of the noise in [0.2, 1, 2, 10, 20] and the nullification rate nu in [0, 0.1, 0.3, 0.5].
G Implementation Details
The pre-trained MIMO network is released by (Murahari et al., 2023)², BERT-bas base are used as pre- trained MIMO network implement TextMixer. To ensure consistency, all baseline methods are implemented using the BERT-base model. All the experiments are conducted on NVIDIA GeForce RTX 3090. Implementation details and the Hyper- Parameters of both attack and defense methods are introduced follows.
For CAPES, during the progress of adversarial training weight, the coefficient Aedv for the adversarial training loss is searched within the range of [0.01, 0.05, 0.1, 0.5, 1, 5] and the scale of the noise in [0.2, 1, 2, 10, 20].
For Santext+? our approach aligns with the author's methodology, utilizing GloVe (Pennington et al., 2014) for word substitution guidance. By default, the probability of non-sensitive words being replaced is set at 0.3 (denoted as p), while the percentage of sensitive words to be sanitized is set at 0.9 (denoted as w). We conduct search for privacy parameter within the range of [1, 2, 3].
Defence Methods Our implementation is based on the Hugging Face Transformer models and is replication with publicly available code. At training time, we use the AdamW optimizer
http:://gghhaaa.oommmmcooooooomg "maps://hugginnffcc..///

---

For DataMix, we implement the method our- selves based on the original paper, we search the number of mixing instances in the range [2. 81.
For TextFusion, we focus on adjusting the mislcading weight during the fusion process. we vary the misleading weight Amd within the range of [0.05, 0.5, 5, 10, 15 enhance privacy sentence-level tasks, increase the weight of the misleading loss term.
bsz
Ir
N
Aode
Ana
IMDB:
Fine- line-tuning
32
2e-5
To carry out the KNN Attack in our implementation, we utilized the embedding matrix of the BERT-base model. This embedding matrix was employed to compute the Euclidean distance between the client representations. In the case of InvBert Attack and MLC Attack, we employ the BERT-base model to construct the inversion model and train 20 epochs to guarantee convergence. We also perform a search for the optimal learning rate within the range of [1e-4, le- 5, 2c-5, lc-6]. For baselines, we use learning rate of 2e-5 to train the inversion model. For TextMixer, the best learning rate we tuned is le-4 in most cases.
Attack Methods
DPNR
32
20-5
CAPE
1
San
64
le-5
3
DataMix
32
20-5
4
TextFusion
32
Se-5
15
TextMixer
32
5e-5
5
0.4
AGNEWS:
Fine-tuning
32
20-5
DPNR
32
2e-5
CAPE
32
5e-5
0.1
SunText+
le-5
DataMix
2
TextFusion
32
TextMixer
32
5e-5
10
CONLL2003:
Fine-tuning
32
DPNR
32
CAPE
32
5e-5
0.1
SanText+
64
le-S
3
DataMix
32
20-5
2
TextFusion
32
5e-5
TextMixer
32
5e-5
OntoNotes5:
Fine-tume mee-tunnng
32
2e-5
DPNR
32
20-5
CAPE
32
5e-5
0.1
SanText+
DataMix
32
20-5
TexFFusio
32
5e-5
Textixxr
32
5e-5
5
0.4
Table 8: Hyperparameter settings for our method and baseline methods. represents the hyperparameter is used in this method. N is the number of mixing instances used in TextMixer and DataMix. is the noise seale used TextMixer, DPNR and CAPE. nu the nullification rate of word dropout for DPNR. Aado the weight of the adversarial training loss for CAPE, a the privacy parameter for Santextt. Amd is the misleading loss weight used in TextFusion.


Reviews:
[{"review_id": "jOfZoPJtNO", "reviewer": "Reviewer_hBPv", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments.\n# A. Misunderstanding of privacy scenarios.\n\nBased on your “……a complex and advanced method compared with the previous TextHide……” and the Missing References you mentioned, we are guessing you misunderstood that our privacy scenario is similar to TextHide's. But in fact, **their privacy scenarios are completely different. Moreover, the attack methods for TextHide\\&InstaHide [4] are not applicable to our scenarios and approach.**\n\n\n1. **The privacy scenarios for TextHide and MixPi are totally different,** as we mentioned in the Related Work section (lines 558-563). TextHide\\&InstaHide are proposed for \"**private** **learning**\", but MixPi is proposed for \"**private** **inference**\".  [3] has discussed the difference between various privacy scenarios and attack methods.\n    - **In TextHide's privacy scenario, the privacy risk comes from the training phase**. It aims to prevent the training data from leaking to other parties during distributed or federated learning. **TextHide protects the privacy of training set** and it aims to defend training data attack such as membership attacks [1].\n    - **In our privacy scenario, the privacy risk comes from the inference phase**. More specifically, **after training**, some fine-tuned PLMs will be deployed on the cloud and released as cloud inference services. Users can query these inference services with their personal data to perform target NLP tasks. Our method aims to allow users query inference services without leaking private data to service providers or third parties. **MixPi can be seen as protecting the privacy of \"test set\"** and it aims to defends privacy attack for inference phase, such as embedding inversion attacks [2, 3].\n    - **The capabilities and objectives of the privacy attacker are different in these two different privacy scenarios**. For example, [4] assume attackers is given access to the encoded training dataset $\\hat{S}$,  their objective  is to recover training text $x_{train}$ in the original dataset $S$ as much as possible. But in our scenario,  the attack is given access to a single encrypted embeddings $H$, their objective is to recover the original sentence $x_{test}$, and this sentence may be  just generated by the user and does not exist in the training data. Therefore, the privacy attack for TextHide’s privacy scenario is not applicable to our privacy inference scenario. And **the method and conclusion drawn from *Is private learning possible with instance encoding* [4],** which proposes a specific attack method for InstaHide,  **are not applicable to our scenario and approach. It  also cannot prove our method can be easily broken.**\n\n# **B. Concerns about privacy guarantee.**\n\nFirstly, MixPi’s privacy guarantees can be theoretically proven. Secondly, we have empirically verified MixPi's privacy through rigorous experiments. Thirdly, we believe empirical method with extensive experimentation can also make sufficient contributions.\n\n\n\n1. **The privacy guarantees of MixPi can be theoretically proven.** We mentioned in the limitations section that MixPi’s proof is not as rigorous as homomorphic encryption, the following is a simple, intuitive proof.\n    - **Mixing Encryption**: If we mix the real input with k-1 synthesize inputs, the **privacy attackers will only have a $\\frac{1}{k}$ probability of guessing the real input correctly**. Moreover, MixPi provides a single mixed representation instead of a set of $k$ representations, which further increases the difficulty of privacy attack.\n    - **Position Encryption**: If we shuffle a sentence of length $n$, $n$ random words can form $n!$ different sentences, so the **privacy attackers have a $\\frac{1}{n!}$ chance of recovering the true order**. Although they can eliminate some candidate sentences through grammar, there would still be many confusing candidates left.\n    \n    We will add proof of the privacy guarantee in Appendix D to enhance the quality of our paper.\n    \n2. **The privacy guarantees of MixPi has been empirically validated**. \n    - **We have conducted rigorous experiments to validate MixPi's ability to protect privacy.** We leveraged the most powerful and widely used privacy attack methods (Embedding Inversion Attack [2, 3, 6])  to attack MixPi (Section 4.1). We also designed specialized attack methods for different encryption modules of MixPi (Section 4.3). The experimental results demonstrate that MixPi can effectively resist general and specialized privacy attacks, and outperforms methods with formal privacy proofs (e.g. SanText) in both privacy and performance.\n    - We believe that the strength of privacy notion is determined by the privacy scenario, and whether a method can be broken depends on the capabilities of potential attackers. This is why simple and intuitive techniques like k-anonymity are still widely employed by companies like Google. In our scenario, based on the theoretical proof and empirical experiments, we believe that MixPi's privacy guarantees are sufficient to effectively defend privacy attacks. Of course, we are also very willing to compare with more strong attack methods that may appear in the future and believe that our method can also achieve good results in these attack methods : ).\n3. It's worth mentioning that even without strict privacy guarantees, some empirical methods can still protect privacy under their settings, such as [6] and [7]. From a broader perspective, many works only propose the empirical methods and comprehensive experiments without rigorous proofs, but these works have still made significant contributions, such as GPT [9], Mixup [10] and adversarial training [11].  **We believe that our work, which propose empirical method and conducts comprehensive experiments, has made enough contribution.** Nonetheless,  we are also plan to further improve our paper by adding a theoretical proof section as you suggest.\n\n# C.Mixing inputs for privacy during inference faces new challenges.\n\n\nOur approach is not simply \"a complex and advanced method compared with the previous TextHide\". TextHide only mixes inputs in the training phase and it uses one input for inference without any mixing. In contrast, MixPi aims to mix multiple inputs during the inference phase while preserving task performance, which presents new challenges. For example, if two inputs, $x_1$ and $x_2$, are mixed using a TextHide style like $x' = 0.3 * x_1 + 0.7 * x_2$, the resulting mixed input $x'$ fed into the model for inference will yield a \"mixed\" prediction $y'$. Even with coefficients 0.3 and 0.7, it's challenging to directly extract the desired predictions, $y_1$ or $y_2$, from mixed prediction $y'$. Furthermore, as the number of mixed inputs increases, even if the model is trained on the mixed data, it is still difficult for the model to handle the mixed input during inference, leading to poor performance. A private inference method for images, DataMix [2], has a mixing method similar to TextHide. However, as shown in Table 1, the performance and privacy of DataMix are very poor, indicating that these challenges cannot be addressed easily. **MixPi overcome the above new challenges. Therefore, our method is not just an extension of TextHide, but rather establishes a novel framework that mixes multiple inputs to achieve privacy-preserving inference**.\n\n# **D. Concerns about lightweight&utility-privacy analysis.**\n\nWe are not very certain about what specific concern you have, but our method is indeed lightweight, and we provide a significant amount of utility-privacy analysis in Section 4.\n\n1. Our framework is very lightweight for users. Users only need to convert text into word representations and encrypt these word representations in their local devices. The main computations are performed by the cloud services. Because we haven't encrypted the computations of the model, the **inference time per input of MixPi is similar to the normal, original PLM**. \n2. Below are some preliminary results. We use a sentence of length 128 to query a bert-base model. All experiments, except for PUMA, are conducted on the same CPU since most cryptographic-based methods do not support GPU. * denotes results directly taken from [12].  FLOPs stands for floating point operations. Inference Time represents the total inference time of the entire model, measured in seconds. N represents the number of mixed words.\n\n    \n    \n    | Method | FLOPS (User) | FLOPs (Server) | Inference Time |  |\n    | --- | --- | --- | --- | --- |\n    | Fine-tune | 0.0003G | 10.88G | 0.077 s |  |\n    | PUMA (Cryptographic-Based)* [12] | \\ | \\ | 33.91 s |  |\n    | TextFusion [6] | 0.0075G | 5.45G | 0.047 s |  |\n    | MixPi (N=5) | 0.1080G | 10.88G | 0.079 s |  |\n    | MixPi (N=10) | 0.2157G | 10.88G | 0.089 s |  |\n    \n    In addition to these preliminary results, we plan to provide a more detailed comparisons of the communication costs and computational costs of Fine-tune, encryption-based methods, TextFusion, and MixPi in Appendix E.\n    \n3. We show the main utility-privacy analysis in Section 4. We use task performance as the  utility metric in this paper. The task performance and privacy metric are shown in Table 1, the ablation study is shown in Table 2 and Figure 1, and the specific privacy attacks are shown in Table 3 and Table 4.  In addition, in order to enhance the quality of our paper, we also plan to add Appendix F, which is used to discuss the impact of the number of mixed inputs and noise scales on performance and privacy. Below are some preliminary results on CoNLL03.\n    \n    \n    | Number of mixed inputs | Noise Scale (eps) | Task Metric (F1) | Privacy Metric (Inverison top1) |\n    | --- | --- | --- | --- |\n    | 2 | 1 | 89.50 | 64.17 |\n    | 2 | 0.5 | 88.22 | 32.24 |\n    | 2 | 0.2 | 81.29 | 4.48 |\n    | 5 | 1 | 87.15 | 6.96 |\n    | 5 | 0.5 | 86.65 | 3.94 |\n    | 5 | 0.2 | 83.32 | 1.62 |\n    | 10 | 1 | 84.39 | 1.64 |\n    | 10 | 0.5 | 83.49 | 1.57 |\n    | 10 | 0.2 | 81.91 | 1.26 |\n    \n\n**Based on the above response, we propose a novel privacy-preserving inference framework with theoretical and empirical privacy guarantees. We believe our experiments are sound, contributions are enough and deserve higher scores for soundness and excitement.**\n\n[1] Invernet: An Inversion Attack Framework to Infer Fine-Tuning Datasets through Word Embeddings. EMNLP2022\n\n[2] InvBERT: Reconstructing Text from Contextualized Word Embeddings by inverting the BERT pipeline. \n\n[3] Information Leakage in Embedding Models. CCS 2020 \n\n[4] Is private learning possible with instance encoding?  2021 IEEE Symposium on Security and Privacy. IEEE Symposium on Security&Privacy 2021\n\n[5] DataMix: Efficient Privacy-Preserving Edge-Cloud Inference. ECCV 2020 \n\n[6] TextFusion: Privacy-Preserving Pre-trained Model Inference via Token Fusion. EMNLP 2022\n\n[7] Differential Privacy for Text Analytics via Natural Text Sanitization. ACL 2021 \n\n[8] CAPE: Context-Aware Private Embeddings for Private Language Learning. EMNLP 2021\n\n[9] Language Models are Few-Shot Learners. 2020\n\n[10] mixup: Beyond Empirical Risk Minimization. ICLR 2018\n\n[11] Explaining and Harnessing Adversarial Examples. ICLR 2015\n\n[12] PUMA: Secure Inference of LLaMA-7B in Five Minutes. Arxiv 2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) The privacy-preserving inference in language model is emerging issue.\n2) The paper clearly presents the methodology with a detailed diagram and easy to follow.  ", "reasons_to_reject": "1) The privacy guarantee lacks rigorous theoretic proof. The paper is to propose a complex and advanced method compared with the previous TextHide with perturbations and shuffle. \n2) The k-anonymity here is a weaker privacy notion, which could be broken potentially. \n3) It would be ideal if the paper could have a lightweight inference while preserving privacy. The paper should include utility-privacy analysis .", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes a privacy-preserving inference method for text domain. ", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Carlini N, Deng S, Garg S, et al. Is private learning possible with instance encoding?[C]//2021 IEEE Symposium on Security and Privacy (SP). IEEE, 2021: 410-427."}, "extra_text": {}}, {"review_id": "1g0Gn0k1d9", "reviewer": "Reviewer_ekHf", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your suggestion! \n\nWe do plan to add comparisons of communication costs across different methods. Below are some of our preliminary results on communication costs. \n\nWe use a sentence of length 128 to query a bert-base model. All experiments, except for PUMA, are conducted on the same CPU since most cryptographic-based methods do not support GPU. * denotes results are directly taken from [1]. N represents the number of mixed words.\n\n| Method | Communication Cost |  |  |  |\n| --- | --- | --- | --- | --- |\n| Fine-tune | 384 KB  |  |  |  |\n| PUMA (Cryptographic-Based)* [1] | 10.773 GB |  |  |  |\n| TextFusion [2] | 192KB |  |  |  |\n| MixPi (N=5) | 384 KB  |  |  |  |\n| MixPi (N=10) | 384 KB  |  |  |  |\n\nFrom the table, we can see that **the communication cost of MixPi is the same as Fine-tune** (sending original word representations directly). This is because our method only requires users to send encrypted word representations to the cloud (we only count the communication costs for data sent from users).  TextFusion sends fused word representations to the cloud. Token fusion reduces the sentence length, so its communication cost is less than MixPi and Fine-tune. Cryptographic-based methods have much higher communication costs than our method due to the need to support extensive cryptographic protocols. \n\n**In addition to these preliminary results, we plan to add Appendix E for a detailed comparison of communication and computation costs among Fine-tuning, encryption-based methods, TextFusion, and MixPi.**\n\n[1] PUMA: Secure Inference of LLaMA-7B in Five Minutes. Arxiv 2023\n\n[2] TextFusion: Privacy-Preserving Pre-trained Model Inference via Token Fusion. EMNLP 2022", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Interesting work, with strong justification for its motivations. \n\n- The proposed framework demonstrates a reasonable performance and lower privacy leakage compared to prior studies.\n \n- The paper is well written\n", "reasons_to_reject": "The reviewer has concerns regarding the usability of the privacy-preserving inference framework.  In section 5 (lines 540-543), the author mentioned that cryptographic approaches suffer from very high communication costs. Do the authors plan to evaluate the communication cost of deploying the proposed framework and make a comparison between cryptographic approaches and similar approaches (e.g., TextFusion)? ", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes a privacy-preserving inference framework inspired by k-anonymity. The experiment results show promising improvement in terms of communication cost and privacy protection. ", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "E69hRSBJaU", "reviewer": "Reviewer_qUde", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments!\n\n# A. Question about proof of privacy.\n\n1. We provide an intuitive and simplified proof of privacy below, and **we plan to include a formal privacy proof in Appendix D of future versions**. As mentioned in the Limitations section, the privacy proof of MixPi is not as rigorous as that of homomorphic encryption.\n    - **Mixing Encryption**: If we mix the real input with k-1 synthesize inputs, the **privacy attackers will only have a $\\frac{1}{k}$ probability of guessing the real input correctly**. Moreover, MixPi provides a single mixed representation instead of a set of k representations, which further increases the difficulty of privacy attacks.\n    - **Position Encryption**: If we shuffle a sentence of length $n$, $n$ random words can form $n!$ different sentences, so the **privacy attackers has a $\\frac{1}{n!}$! chance of recovering the true order**. Although they can eliminate some candidate sentences through grammar, there would still be many confusing candidates left.\n\n2. MixPi is dataset-agnostic and can be generalized to any dataset. Additionally, our approach could potentially be extended to the majority of Transformer-based models. The following are the conditions that different encryption methods need to satisfy:\n    - **Mixing Encryption:** This requires the model to support Multiple-Input Multiple-Output (MIMO), which can be achieved through pre-training [1,2].\n    - **Representation Encryption:** This method is model-agnostic, meaning it can be applied to any model.\n    - **Position Encryption:** For this method, the model needs to satisfy the shuffle equivalence property (details can be found in Appendix B). The majority of PLM architectures support this property.\n    \n    If a model meets the requirements mentioned above, MixPi can be applied to it. This is entirely feasible, especially considering that the primary challenge appears to lie in MIMO pre-training for the specific modality, which might not be a significant issue for enterprises with ample computational resources.\n    \n\n# B. Question about applications beyond textual data.\n\nThis is a great question! We believe MixPi has the potential to be extended to other modalities. Currently, as the open-source MIMO model only supports the text modality [1,2], we can only conduct experiments on text data. Fortunately, our method has the potential to be extended to other Transformer-based models, and transformer-based models are widely used across various modalities. Therefore, if there are models in other modalities that meet the requirements outlined in Response A.2, we believe that it would only require minor adjustments to extend MixPi to those corresponding modalities (such as images and audio).  Thank you for your inspiration, in our future work, we will attempt to create a unified version of MixPi that works across multiple modalities!\n\n# C. Question about the computational cost of the users’ side.\n\nMixPi will not impose significant computational costs on the user's side. Below are some of our preliminary results on computational costs. \n\nWe use a sentence of length 128 to query a bert-base model. All experiments are conducted on the same CPU since most cryptographic-based methods do not support GPU.  FLOPs stands for floating point operations. N represents the number of mixed words.\n\n| Method | FLOPS (User) | FLOPs (Server) |  |  |\n| --- | --- | --- | --- | --- |\n| Fine-tune | 0.0003G | 10.88G |  |  |\n| MixPi (N=5) | 0.1080G | 10.88G |  |  |\n| MixPi (N=10) | 0.2157G | 10.88G |  |  |\n\nFrom the Table, we can observe that **MixPi only introduces a few additional computational costs compared to fine-tuning (directly sending word embeddings)**, this computational cost is very small compared to the computation required for cloud PLMs. Given the current advancements in smart chips. We believe that with the development of modern chips, this amount of computation is within an acceptable range for users. \n\n**In addition to these preliminary results, we plan to add Appendix E for a detailed comparison of communication and computation costs among Fine-tuning, cryptographic-based methods, TextFusion, and MixPi.**\n\n# D. Suggestions about typos and presentation.\n\nThank you for your thorough review! Based on your feedback, we will make the following changes: \n- Add a description of ε after Eq. 3, where $\\epsilon$ is a hyperparameter controlling the scale of noise – smaller values of ε correspond to a larger scale of noise. \n- Provide a brief description of vanilla mixing at the end of line 143. \n- Rewrite Section 3.1 for improved readability. -\n- Regarding the problem of noise scale. We appreciate your reminder, our statement might be somewhat misleading. In fact, our findings are consistent with prior works, and we also agree that adding noise does indeed impact performance. Our point is that the appropriate noise would not result in **much** performance loss. This is acceptable as it contributes largely to privacy enhancement. As shown in Table 2, removing noise (- Repr) generally leads to improved task performance (92.32->92.63 in IMDB and 86.24->86.69 in CoNLL03), but privacy leakage becomes worse (2.22->6.57 in IMDB and 4.14->15.53 in CoNLL03). **We will revise this potentially misleading statement** and emphasize that noise does indeed affect performance, but these performance declines are acceptable in some cases.\n\n**If our responses have addressed your concerns about this paper, could you please raise your scores?**\n\n[1] DataMUX: Data Multiplexing for Neural Networks. NeurIPS 2022\n\n[2] MUX-PLMs: Data Multiplexing for High-throughput Language Models.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper addresses a very pertinent issue of protection of private user data that is being sent to a cloud service for use with a Large Language Model. The major strength of the paper are the following:\n* The technique described is able to minimize privacy leakage under different attacks, and still maintain comparable model performance. It uses mixing-centric encryption method and adds multiple obfuscation techniques using synthetically generated data.\n* The mixing-centric encryption described here, is an innovative use of the MIMO network described by Murahari et al., 2023. \n* It provides techniques for synthetic data generation that can be used with this technique, and also compares performance of the techniques.", "reasons_to_reject": "The major weaknesses of this paper are the following:\n* While the technique described shows privacy protection in the experiments, it doesn't provide any proof or bounds of privacy when the technique is applied to other datasets, and other models.\n* The technique described is only applicable to text data. The synthetic data generation technique also has similar limitations. \n* The technique would be computationally expensive on the user side.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper is about protecting user's private data that may be sent over to Large Language Models deployed in cloud services. Even in the case user information is converted to corresponding embeddings before sending over to the cloud based LLM, they are still susceptible to embedding inversion attacks. The major contributions are the following:\n* It comes up with a technique that uses mixing-centric encryption method, along with mixing private input with synthetically generated artificial input. This approach is shown to preserve user privacy under certain attacks. \n* It further ensures the model performance is not affected too much, which is typically a tradeoff in privacy preserving systems.\n* It comes up with a technique to generate synthetic inputs that help with the mixing techniques described in the paper.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "* Lap(\\epsilon) is used in Eq. 3. It'd be great if you can please describe \\epsilon.\n* The paper refers to vanilla mixing with reference to the MIMO technique from Murahari et al, 2023. It would be great if there is a brief explanation of vanilla mixing added when it is mentioned the first time.\n* Section 3.1 Datasets - Second sentence - \"For sentence classification task...\" would need to be rephrased.   \n* Section 4.2 Ablation Study - You mention that adding appropriate scale noise can enhance privacy without compromising much performance. It would be good to validate this hypothesis or mention the reason you think that is the case, as a lot of work on Differential Privacy shows that adding noise this way does affect performance."}}]

Author rebuttal:

Meta-review: The paper addresses an emerging and vital issue in PLM. I agree with the reviewers ekhf and qUde. It addresses a pertinent issue in the field of data privacy with LLMs and presents an innovative privacy-preserving technique that performs well. The paper's clarity and the inclusion of techniques for synthetic data generation further enhance its value.
Decision: Accept-Findings