Concept-Aware Privacy Mechanisms for Defending Embedding Inversion Attacks
Authors: Yu-Che Tsai
Hsiang Hsiao
Kuan-Yu Chen
Shou-De Lin
Venue: ICLR 2026 Poster (2026)
Keywords: Text Embedding
Privacy
Defense
Inversion Attack
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Text embeddings enable numerous NLP applications but face severe privacy risks from embedding inversion attacks, which can expose sensitive attributes or reconstruct raw text. Existing differential privacy defenses assume uniform sensitivity across embedding dimensions, leading to excessive noise and degraded utility. We propose SPARSE, a user-centric framework for concept-specific privacy protection in text embeddings. SPARSE combines (1) differentiable mask learning to identify privacy-sensitive dimensions for user-defined concepts, and (2) the Mahalanobis mechanism that applies elliptical noise calibrated by dimension sensitivity. Unlike traditional spherical noise injection, SPARSE selectively perturbs privacy-sensitive dimensions while preserving non-sensitive semantics. Evaluated across six datasets with three embedding models and attack scenarios, SPARSE consistently reduces privacy leakage while achieving superior downstream performance compared to state-of-the-art DP methods.

Published as a conference paper at ICLR 2026
CONCEPT-AWARE PRIVACY MECHANISMS FOR DEFENDING EMBEDDING INVERSION ATTACKS
Yu-Che Tsai'
Hsiang Hsiao'
Kuan-Yu Chen' Shou-De Lin
"Department of Computer Science and Information Engineering. National Taiwan University
Natiooal Taiwan University AI Center of Research Excellence
Taipei, Taiwan
(F09922081, 112946000,, (13022033,,3922200 inn.)/in22.....nnn//_ee.n wtwpl//nsseeeee.eeeeeeee ntu. wec.ueedd.te.
ABSTRACT
Text embeddings enable numerous NLP applications but face severe privacy risks from embedding inversion attacks, which can expose sensitive attributes or recon- struct raw text. Existing differential privacy defenses assume uniform sensitivity across embedding dimensions, leading to excessive noise and degraded utility. We propose SPARSE, user-centric framework for concept-specific privacy protection in text embeddings. SPARSE combines (1) differentiable mask learning to identify privacy-sensitive dimensions for user-defined concepts, and (2) the Mahalanobis mechanism that applies elliptical noise calibrated by dimension sensitivity. Unlike traditional spherical noise injection, SPARSE selectively perturbs privacy-sensitive dimensions while preserving non-sensitive semantics. Evaluated across six datasets with three embedding models and attack scenarios, SPARSE consistently reduces privacy leakage while achieving superior downstream performance compared to state-of- fatte-of-the- of-the-at DP methods.
INTRODUCTION
Text embeddings are general representations of textual data that enable various downstream learning tasks without utilizing the raw text. Recent advances in pre-trained models like Sentence-T5 (Ni (ct al.. 2022a) and SentenceBERT (Reimers & Gurevych| 2019) enable the generation of high-quality embeddings that power numerous NLP applications. A prominent example is retrieval-augne -uggmeteed generation (RAG) systems (Lewis et al. 2020). which have led to the widespread adoption of online embedding database services such Chroma and Faiss (Johnson et al. 2019).
However, recent research has uncovered critical vulnerabilities in text embeddings through embed- ding inversion attacks (Huang al. 2024 Pan et al. 2020; [Song & Raghunathan 2020). These attacks can extract sensitive attributes or even reconstruct the original text. For example, prior work (Coavoux et al.) 2018) showed that demographic information
be inferred directly from embeddings, while GEIA (Li et al. 2023) demonstrated that full sentences can be recovered. Most strikingly, Vec2Text (Moriis al. 2023) reported that adversaries can reconstruct up 92% of 32-token input from T5-based embeddings. Such vulnerabilities pose significant risks in domains handling sensitive data, such as patient notes in medical RAG system. Thus, developing robust defenses against embedding inversion has become critical challenge.
Differential privacy (DP) (Dwork al. 2006) a widely adopted framework for protecting sensitive information due to its rigorous guarantees. However, most existing DP-based defenses implicitly assume that all information in embeddings is equally privacy-sensitive. This assumption has two drawbacks. First, privacy concerns are inherently user- and context-dependent (Brown et al. (022): one individual may prioritize protecting health conditions, while another may care more about political views or personal relationships. Second, to cover all possible sensitive information, DP mechanisms typically inject substantial noise across all embedding dimensions, which inevitably leads to significant utility degradation. Therefore, is crucial to develop defense mechanism that can provide concept-specific protection-alloviiiee users to specify which attributes to protect while
https://docs. http:///oocc.....rraaoaoa.. com/

---

Published as a conference paper at ICLR 2026
Private Text
This patient is 43 year-old man with depression.
O 10000 Spherical Notoe
Embedding inversion Attack
Age-oware Nobie
Symptom dware Nobse
+
0 Text Embedding
O 0 Text Embedding
o Text Embedding
o Text Embedding
Recowered Text: This patient a yoar-old male with depression.
Recovered Text:
Besovered Text:
Recovered Text:
The pottect right concer ends up with hymenectomy.
(homosed Aas concort)
jRemoosd Sympttom conorpt)
Tiis patlent i man with depression
This patsent a pear-ald mon.
Privocy Leakage
Success Defense & Lost Semantic
Success Defense
Preserve Semantic
(a) Non-protected Embedding
(b) Defense with Spherical Noise
(c) SPARSE: Defense with Concept-Aware Elliptical Noise
Illustration of embedding inversion attack and different defense strategies. (a) Sensitive information can be easily identified from non-protected text embeddings. (b) Adding spherical noise mitigates privacy leakage but harms textual semantics. (c) Our approach applies elliptical noise guided by user-defined privacy concept, selectively adding stronger perturbations to privacy- sensitive dimensions while preserving non-sensitive semantics. A real- world case study is presented in Appendix J
Figure 1:
preserving embedding quality for non-sensitive content. This work aims to address key research question:
Research Question: Can we selectively obfuscate user-defined private concepts in embeddings while preserving non-sensitive semantics for downstream tasks?
However, designing such a defense mechanism is non-trivial. The central challenge lies in the mismatch between existing DP methods and the heterogeneous nature of embedding dimensions. Current approaches add the same level of noise to every embedding dimension, implicitly assuming that all dimensions carry equal amounts of sensitive information. However, our preliminary analysis (see Appendix ) reveals that embedding dimensions exhibit varying degrees of privacy sensitivity with respect to specific concepts. Some dimensions may be highly sensitive to particular privacy attributes (e.g.. medical conditions), while others primarily encode non-sensitive semantic features.
To address this challenge, an ideal defense mechanism should accomplish two key objectives: (1)
identify which embedding dimensions are privacy-sensitive for a given privacy concept, and (2)
design a differential privacy mechanism that calibrates noise injection based on dimension sensitivity while maintaining theoretical guarantees.
We propose SPARSE (Sensitivity-guidee
Privacy-Award Representations for better SEmantic- preserving), a novel user-centric framework that improves privacy in text embeddings through censitivity-guidee perturbations. To achieve the first goal, we present differentiable mask learning framework to estimate the sensitivity of embedding dimensions with respect to a user-defined privacy concept. To achieve the second goal, we introduce the Mahalanobis mechanism, generalized Laplace mechanism, which injects elliptical noise calibrated by dimension sensitivity. As illustrated in Figure T while traditional methods apply spherical noise that uniformly perturbs all dimensions (panel b), our approach first identifies presiive dimensions associated with user-specified concepts (c.g., symptom or age) and then applies elliptical noise with larger perturba- tions to these sensitive dimensions while minimally affecting others (panel c). We summarize our key contribution as follows:
extension of the
Novel defense paradigm. We introduce SPARSE, a sensitivity-guiide framework for user-defined privacy protection in embeddings, and introduce the Mahalanobis mechanism an extension of differential privacy that provides rigorous theoretical guarantees.
Better privacy-utility tradeoffs. We evaluate SPARSE against two state-of-the-an differential privacy methods across six datasets. Experimental results show that SPARSE consistently reduces privacy leakage while achieving better downstream performance.
Robust generalization. We assess the generalizability of SPARSE using three different embed- ding models and three attack models. Experimental results demonstrate that SPARSE remains consistently effective regardless of the specific embedding method or threat model used.
Comparable performance to white-box defense. We further design a white-box variant of SPARSE with full access the threat model. Despite lacking prior knowledge of the attack
2

---

Published as a conference paper at ICLR 2026
model, SPARSE achieves performance close to the white-box defense, demonstrating its ability to accurately identify privacy-sensitive dimensions.
2
PRELIMINARIES
2.1 BACKGROUND ON DIFFERENTIAL PRIVACY
Differential Privacy (DP) (Dwork et 2006 is rigorous privacy guarantee that ensures a random- ized mechanism M behaves similarly on any two inputs. There are two common models of DP: central and local. this work, we focus on Local Differential Privacy (LDPyKasvviswaaaaha) et al.) 2011), where each user perturbs their data locally before sharing it. This approach offers stronger privacy guarantees in settings where the data collector cannot be trusted, as removes the need for trusted aggregator.
Definition (Local Differential Privacy), A randomized mechanism M satisfies e-local differential privacy if for all pairs of possible user inputs z, x' X and any output set C Range(M).
Pr(M(a) 0J S e Pr[M(x')EO)
where 2 O is privacy parameter and Range(M) denotes the set of all possible outputs of M. The mechanism M outputs a random sample from a probability distribution over possible outputs, rather than deterministic value. The parameter, termed the privacy budget, controls the similarity in the output, with a smaller e indicating higher privacy protection, and vice versa.
Generalization with distance metrics. Local differential privacy (LDP) requires mechanism to produce nearly indistinguishable outputs for any two possible inputs, regardless of how different the inputs are. While this provides strong privacy guarantee, it often leads significant utility loss, especially continuous or semantic domains such as text embeddings (Feyisetan et al. (2019). To address this limitation,
adopt metric local differential privacy (metric LDP) (Chatzikokolakis eet al. 2013 Alvim et al. (018)) a generalization of LDP to metric spaces. Metric LDP relaxes the indistinguishability requirement by incorporating a distance function over the input space. This allows the privacy guarantee to degrade gracefully as the dissimilarity between inputs increases.
Definition 2 (Metric Local Differential Privacy). Let e 2 0 be the privacy parameter, and d be distance metric for the input space. A mechanism M satisfies ed-LDP, for any two inputs x, x! and any output set O C Range(M)
Pr[M(z) E e (e=dlea')))P((((())) PT[M(x') E EJ.
The key idea is that the privacy guarantee depends how similar the inputs closer inputs must yield nearly indistinguishable outputs, while distant inputs may produce more distinguishable ones. Although the privacy budget remains fixed, the output bound varies with the input distance. To instantiate a mechanism satisfying metric LDP under & distance, we introduce the generalized Laplace mechanism, which widely used for embedding sanitization against adversarial attacks.
Definition 3 (Generalized Laplace Mechanism (Wu al. (2017)). Let 2 0 be the privacy budget. The generalized Laplace mechanism MLap RR"
R2 perturbs any input x E IR" as
M1ap = I Ziap: Ziap
fx(2) x exp (==]))
We note two important properties of the generalized Laplace mechanism: (1) it satisfies ed-LDP with respect to the & norm (Du et al. (023); and (2) it adds isotropic (spherical) noise, implicitly assuming that privacy sensitivity is uniformly distributed across all embedding dimensions.
2.2 PROBLEM STATEMENT
Attack Scenario. In this work, focus specific embedding inversion attack scenario where the adversary aims to reconstruct the input text from the corresponding text embedding. Formally, given sequence of tokens and the text embedding model & RR,, where denotes the embedding dimension, the attacker seeks to find a function g to approximate the inversion function & as: g( (s)) x Φ- (-)((ss)
These inversion attacks can be classified into categories based
3

---

Published as a conference paper at ICLR 2026
their target: (i) token-level inversion (Pan et al. 2020 Song & Raghunathan 2020). which focuses on retrieving individual tokens from the original text, and (ii) sentence- inversion (Li et al. 2023
MMrris et al. 2023), which attempts to reconstruct the entire ordered sequence of text. Regardless of the attack model employed, our study prioritizes understanding whether private information (e.g..
names, diseases) within the original text is revealed.
Privacy Definition. Privacy is inherently context-dependen (Brown et al. 2022) While many prior works adopt narrow operational definition centered on personally identifiable information (PII) such as names or identification numbers (Sousa & Kern (023), such a fixed notion is often insufficient. In practice, users may care about protecting different types of sensitive attributes for instance, health conditions, political views, or personal relationships. To capture this variability, we adopt user-centric privacy definition, where the data owner specifies a privacy concept C representing the set of tokens or attributes to be protected. In our experiments, we instantiate C primarily with named entities and PII tokens, but the framework naturally generalizes to other user-defined concepts.
Defense Scenario. Our goal is to develop privacy-preserving embeddings that satisfy two objectives:
Goal (Defending against sensitive token inference attack): For the threat model A and text embedding (s), where s is sentence that contains sensitive information. The data owner defines a privacy concept C fti.t2.... f2.... .......... ticth. which a set of sensitive tokens (e.g., names, medical conditions) that must be protected. The objective is to generate an obfuscated embedding 4 4'(s) that prevents the threat model A from accurately reconstructing the tokens in C.
Goal 2 (Maintaining downstream utility): The secondary objective is ensure that the protective measures, while securing the embeddings from inversion attacks, do not compromise the utility of the embeddings downstream tasks.
3 SPARSE FRAMEWORK
3.1 IDENTIFYING CRIVACY-SENSITI DIMENSION THROUGH NEURON MASK LEARNING
To quantify the sensitivity of individual dimensions with respect to a privacy concept C, we propose a neuron mask learning framework that estimates a relaxed binary mask over the embedding dimensions. The goal is to learn a mask vector m E [0, 1]" that approximates a binary selection: assigning values close to for dimensions relevant to C, and close to 0 otherwise. Given an embedding (s(s,, the masked representation is denoted by (s) O m, where ① indicates the Hadamard product.
Differentiable Neuron Mask Learning. Although the ultimate goal to approximate binary mask. direct optimization over discrete values is not feasible due to non non- differentiabilityy Therefore, we resort to practical method that employs a smoothing approximation of the discrete Bernoulli distribution (Maddison et al., 2017). Under this framework, we assume each mask follows a hard concrete distribution HardConcretellog a,, 3.) with location a, and temperature 3, (Louizos et al. 2018 as:
1
#4 pi
log az )) m. min (1, max (0, 8 (E -67) 7) + 7)),
8 =
log
where a denotes the sigmoid function. = 1.1 and y = 00.1 1 are constants, and μı ~ U(0, 1) is the random sample drawn from the uniform distribution. O; and 33 learnable parameters. The random variable 8; follows a binary concrete (or Gumbel Softmax) distribution, which is an approximation of the discrete Bernoulli distribution. Samples from the binary concrete distribution are identical to samples from Bernoulli distribution with probability o; as 8 - O, and the location a; allows for gradient-based optimization through reparametrization tricks (Jang et al. 2022). During the inference stage, the mask m, could be derived from hard concrete gate;
m, min (1, max (0, (log o:)) (E 76) + +7))
Training Dataset Construction. We construct two datasets identify the embedding dimensions most affected by the privacy concept C. The positive dataset D = {si...., S consists of sentences that include tokens representing the concept C. For each sentence s; E D*, we construct a corresponding negative sample by removing all tokens related to C, denoted R(s,, C). This yields
4

---

Published as a conference paper at ICLR 2026
the negative dataset D (R(s;, E D*}. where each sentence is identical to its positive counterpart except for the absence of concept-specific tokens.
Learning Objective. The neuron mask is trained to satisfy two key objectives: (i) The masked embedding (s)) @ m should retain sufficient information to distinguish between the positive and negative datasets D+ and D respectively; and (ii) the mask should be sparse, thereby isolating only the most relevant dimensions associated with the privacy-sensitive concept C. To achieve these objectives, define composite loss function. The first term is discriminative loss that encourages separation between D and D
Lala
Σ log Po (*(st) S+ED+
Σ log (o(s o o
S-ED
where (-(-) denotes the probability predicted by a MLP classifier parameterized by 0. To enforce sparsity in the learned mask, add Lo-regularization term based the expected number of active neurons under the hard concrete distribution:
m
Σ
(1o Bi log
Cegg (m)
m
The final objective function jointly optimizes the classification performance and sparsity:
min
Cel(mm, 0) + >Cre (m))
(5)
m000
where the regularization coefficient controls the trade-off between predictive accuracy and the compactness of the neuron mask. For more implementation details, readers are referred to Appendix H
and Algorithm 2
3.2 EMBEDDING PERTURBATION WITH MAHALANOBIS MECHANISM
Having identified the privacy sensiiive embedding dimensions through the learned neuron mask we now describe how to perturb the embeddings in a sensitivity-awann manner. Specifically, we extend the generalized Laplace mechanism by incorporating Mahalanobis norm-based perturbation scheme, thereby enabling elliptical noise calibrated by the neuron sensitivity of m. We begin by formally defining the Mahalanobis norm.
Definition 4 (Mahalanobis Norm). For any vector v E R", and positive definite matrix Σ R its Mahalanobis norm is defined as |/l|aa v o E-1 -v.
Note that for any > 0, the Euclidean ball {yv E R" ly isotropic noise all directions. In contrast, the Mahalanobis ball {y R : ly x|M 17} defines an ellipsoid. This distinction allows us to inject anisotropic noise whose spread adapts to the sensitivity of each embedding dimension.
2/2 = n} defines sphere, implying
Definition 5 (Mahalanobis Mechanism), Let 2 0 be the privacy budgel and let Σ RR=x be symmetric positive definite matrix. The Mahalanobis mechanism MMah : Rn + I2" perturbs any input x as
MMah(I)) =
ZMale
Zmah
fz(2) x exp (-e (xp(_mlll)M.)
To calibrate noise based on the learned neuron sensitivity, we define Σ diag(m) + +5 ...... m. + 8). where m, is the i-th entry of m and 1e-6 is small constant ensuring positive definiteness. For scale compatibility with the isotropic Laplace mechanism, we normalize m such that E; m, = n (i.e., trace(E) Σ) trace(I,,) (I,.). Algorithm details how sample ZMah- We now establish the privacy guarantee of this mechanism:
Theorem Given a privacy parameter c, the Mahalanobis mechanism outputting 4' D'(s) M ( (s)) fulfills ed-LDP with respect to the Mahalanobis Norm.
formal proof is provided in Appendix B. Below, we explain how the privacy guarantee of the Mahalanobis mechanism relates to that of the generalized Laplace mechanism.
Connecting Privacy Guarantee to Generalized Laplace Mechanism. We now show that the privacy guarantee of the Mahalanobis mechanism is equivalent, up constant factors, that of
5

---

Published as a conference paper at ICLR 2026
the generalized Laplace mechanism. Since the Mahalanobis and Euclidean norms equivalent in miiee-dimensional spaces, the Mahalanobis mechanism preserves the same asymptotic privacy guarantee, differing only by date-inndeenneet constants.
Lemma 1. Let Σ E Rnx* be positive definnte with trace( Σ) n. Assume the smallest eigenvalue of bounded below by > Then. for any vector R."
| vn
||el2 vc
- ||ll|
Building on this, the following lemma shows that the privacy-loss exponent under the Mahalanobis mechanism is bounded between two exponents based on the Euclidean norm:
Lemma 2. Assume trace(E) = n and that the smallest eigenvalue of E is bounded below by constant 0. Then. for every input text s, S and every 0,
exp (i
E [ (63)la)
< exp (E ( - (()))l)))) z exp
E
(b(()) {(s')ll))
Together, these lemmas show that the Mahalanobis mechanism achieves privacy guarantee compa- rable to that of the generalized Laplace mechanism under the same privacy budget c. The detailed proof in the section is deferred to Appendix B
EXPERIMENTAL EVALUATION
4.1 EXPERIMENT SETUP
Datasets. Following prior work on embedding inversion (Morris et al. 2023 Kim et al. 2022).
We evaluate six benchmark datasets with downstream labels (for privacy-utility tradeoff) and two real-world datasets, Mll-Masking-300 27 PII types and clinical notes. We extract the named entities sensitive information for these datasets using named entity recognition models (detailed in Appendix E)
(Tea 2023) and MIMIC-III (Johnson et al. 2018), covering
Attack models. Three attack models are employed to access the privacy risks of text embedding, including Vec2text (Morris et al.) (023), GEIA (Li et al., 2023),, and MLC (Song & Raghunathan 2020). Vec2text and GEIA are sentence-level attack methods that leverage pre-trained LLMs to reconstruct the input sentence. MLC utilizes three-layer MLP to predict the existence of individual words. Due to its superior performance, Vec2text serves as our default attack model in subsequent experiments.
Defense methods. We compare our proposed SPARSE with two established differential privacy approaches: generalized Laplace mechanism (Wu et al. 2017) (LapMech) and Purkayastha mecha- nism (Du et al.] 2023) (PurMech). LapMech introduces privacy by sampling noise from the Laplace distribution and adding it to the embedding vectors, while PurMech utilizes Purkayastha directional noise to perturb embeddings while preserving semantic meaning. These baselines represent the state- of-the-art in embedding privacy protection methods and provide strong comparisons for evaluating our approach.
Evaluation Metrics. To quantify privacy risk, we use two measures: (1) Leakage: the attack model's accuracy in predicting sensitive tokens (lower is better); (2) Confidence: the probability of the attack model to predict the sensitive tokens (lower indicates less exposure). For downstream utility, we report each dataset's 's standard task metric (e.g., NDCG or correlation; see Appendix Table Please refer to Appendix D for detailed description all the evaluation metries.
Embedding models. We evaluate three widely used embedding models: GTR-base (Ni et al.) 2022b). Sentence-T'S (Ni et al. 2022a), and SBERT (Reimers Gurevych 2019). GTR-base the default model due to its higher vulnerability to the Vec2text attack.
4.2 PRIVACY-UTILITY TRADE-OFF ANALYSTS
We evaluate the privacy-utility trade-off across different defense methods and privacy budgets of e using the STS12 and FIQA datasets. Note that vary the values of E {5, 10, 20, 30, 40) following

---

Published as a conference paper at ICLR 2026
the settings of prior works (Feyisetan et al. 2020 2019). The results are presented in Table | Here, é = 0o denotes the unprotected embedding. In comparison with the baseline methods (LapMech and PurMech),, SPARSE demonstrates consistent superiority in minimizing privacy leakage while maintaining downstream utility. On the STS12 12 dataset at = 10, SPARSE reduces privacy leakage from 60% 19%, whereas alternative methods achieve only 22% reduction. Meanwhile, SPARSE maintains 65% downstream utility while other methods decline to 60%. Although the marginal benefits diminish increases, SPARSE's superior performance remains consistent across varying privacy budgets and datasets. We evaluate SPARSE on four more datasets and two real-world cases with sensitive attributes. As detailed in Appendix [F. and 4.4 SPARSE consistently reduces privacy leakage and outperforms baseline methods.
Table I:
Privacy-utility tradeoff across various defense methods. The mean and standard deviation of 5 runs are reported in percentages($))
Privacy Metries
Utility Metric
Leakage
Confidence I
Downstream f
Dataset
LapMech
PurMech
SPARSE
LapMech
PurMech
SPARSE
LapMMech
PurMech
SPARSE
5
7.36 es
7.42 mn
4.34
6,70 so
6.80
6,41
29.28
29.31
34.12
10
22.34 as
22.66 1.15
19.31 20.21
9.39 2017
9.42 20.7
8.91 20.12
60.72 20.00
60.72 20.0
65.27 10.0
20
38.17
38.04 2022
36.98
24.70 20.35
24.74 as.
23.85
72.47 20.00
72.47
73.25
STS12
30
es 44.74
44.76 w..
43.81 w...
34.59 was
34.59 wass
34.16 mm
73.68 s..
73.68 am.
74.04 an:
40
48.48
48.34
47.54
38.75
38.82
38.49
73.98
73.98
74.15
oc
60.09
47.81
74.25
5
12.56
13.01
8.48
6.67
6.70
6.18
10.64
10.63
14.87
10
35.
35.31 pa.
31.62 20.75
16.70 20.74
16.55 23.00
13.45 :0.30
21.74 20.36
21.76 20:29
23.45 50:31
20
55.69
55.38 a1.2s
53.41
35.32 w0r
35,25 as.re
33.77
32.22 sa.14
32:23 am
32.65
FIQA
30
64.12
64.13
63,51
43.35
43,56 a:.
42.21
33.24 w11
33.26 mn::
33.58
40
68.63
68.13
48.07
47.77 mm
-0.05
33.50 T.
33.52 10
33,85 ...
77.35
54.48
33.56
4.3 DEFENSE ROBUSTNESS AGAINST DIFFERENT THREAT MODELS
While previous experiments focus on Vec2text, it is important to assess SPARSE under varied threat models. We evaluate privacy leakage under three embedding inversion attack models: MLC (Song & Raghunathan 2020). GELA (Li et al. 2023), , and Vec2text (Morris et al.) 2023). Since changing the attack model does not impact downstream utility, report only the Leakage metric. As shown in Table[] SPARSE consistently outperforms LapMech and PurMech across all attack models by a significant margin. Additionally, we notice that complex attack models, such as Vec2text and GEIA, are more susceptible to embedding perturbation, exhibiting substantial leakage reductions of 92% and 72% respectively at e 5. In contrast, the shallow MLC model demonstrates less vulnerability to our defense method. The results suggest that SPARSE offers a more resilient defense against diverse embedding inversion threats.
Table 2: Defense performance with respect to different attack models. We report the Leakage metric in percentage (%) on the STSI2 dataset. In addition, we highlight the relative performance compared to the non-protected embedding in red.
< 10
Attack Models
-=000 00
TapMoch
PurMech
SPARSE
TapMech
PurMech
SPARSE
VecZiest Morris et 2023
66.09
75%)
7-87 (-87669))
4.34 (-92.78%)
22.34 (-62.82%)
22.66 60.399%
19.31 (-67.86%)
GELA La et a 20z3
25,34
12.30 (:53, (.5.4466)
12.36 (-51.22%)
7,08 (-72.06%)
20.00 (18.9111)7199)
21.21 (-16.3099)
1582 (.37.57%)
MLC Sung Raghunshon, 2020
53.20
10,39
19.80 (-62.78 78%)
17.63 (-66.86)))
32.74 (-38.45%)
32.68 (2.68-388555 (38557%)
29.59 (-44.38%)
4.4 EVALUATION ON WORLD PRIVACY THREATS
We evaluated SPARSE's resilience to inversion attacks across various data domains and privacy categories. This evaluation used the PIl-Masking 300K dataset (Team) 2023), and MIMIC-III clinical notes (Johnson et al., 2018). The results in Table S demonstrate significant privacy vulnerabílities in unprotected embeddings and the superior protection offered by our approach. In the MIMIC- III dataset, unprotected models exhibited severe privacy leakage with attack models successfully extracting sensitive attributes concerning rates: 88% for sex, 70% for diseases, and 82% for
7

---

Published as a conference paper at ICLR 2026
symptoms. Under equivalent perturbation budgets of c. SPARSE reduces sex attribute leakage from 88% to 28%. while both LapMech and PurMech achieve only modest reductions to 43%. This superior protection generalizes across all evaluated privacy categories.
Table 3:
Defense performance on different categories of sensitive information. We report the Leakage metric in percentage (%) with = 10.
Datasel
PII-30K
MIMIC-III
Category
Sex
City
State
Country
Age
Sex
Disease
Symptom
Non-protected
86.12
68.45
75.43
84.07
58.49
88.40
70.43
82.76
LapMech
42.35
33.39
36.63
40.37
31.88
43.38
23.32
38.17
PurMech
43.53
34.10
38.45
41.45
31.89
43.38
22.86
31.30
SPARSE
33.76
28.76
33.62
35.19
28.98
28.45
18.28
29.35
4.5
COMPARING SPARSE WITH WHITE-BOX DEFENSE
Our defense framework is predicated the hypothesis that sensitive information is encoded within specific dimensions of the embedding space. Consequently, selectively perturbing these dimensions could effectively mitigate inversion attacks. This motivates two key questions: (i) How effective could SPARSE be under perfect knowledge of embedding sensitivity? and (ii) How closely can our black-box approach approximate this ideal? To answer these questions, we design SPARSE-WB, an empirical upper-bound bound defense assuming white-box access to the attack model.
Extending SPARSE White-Box Defense. For each sensitive token, we use Integrated Gradi- ents (Sundararajan et al. (017) to compute the gradient of the model's output with respect to the input embedding, treating sensitivity estimation as feature attribution problem. Each dimension's attribution score reflects its influence on the prediction. Instead of applying the neuron mask as in the original SPARSE, the white-box method uses the attribution score for sampling noise from the Mahalanobis mechanism.
Results. shown Table SPARSE-W consistently achieves the best privacy-utility tradeoff across different datasets and privacy budgets. The promising result of SPARSE- WB verifies our hypothesis and servers as a strong upper bound. Importantly, we notice SPARSE closely approaches this white-box defense performance, especially and utility. This suggests that SPARSE is able to effectively approximate the white-box sensitivity estimation without access the attack model, which is crucial in realistic threat settings.
20, 30, 40, with only small gaps in both leakage
Table 4: Comparison of SPARSE with its white- box variant and LapMecc assess how well SPARSE approximates an ideal defense with perfect knowledge of sensitive dimensions. Results are reported terms of privacy leakage and downstream utility under varying privacy budgets c.
Leakage ((%)
Downstream 1 (%)
Dataset
Method
c
5
40
c 5
40
LapMech
7.36
22.34
38.17
44.74
48.48
29.28
60.72
72.47
73.68
73.98
STS12
SPARSE
4.34
19.31
36.98
43.81
47.54
34.12
65.27
73.25
74.04
74.15
SPARSE-WF WB
1.43
12.01
33.67
42.95
47.13
40.92
67.45
74.01
74.13
74.10
LapMech
12.56
35.17
55.69
64.12
68.85
10.64
21.74
32.22
33.24
33,50
FIQA
SPARSE
8.48
31.62
53.41
63.51
68.13
14.87
23.45
32.65
33.58
33.85
SPARSE WB
3.03
22.35
51.27
62.70
67.92
14.58
26.46
32.87
33.55
33.58
4.6 QUALITATIVE ANALYSIS OF PRIVACY-SENSITVV DIMENSIONS
We present a qualitative analysis to better understand the quality of the identified by SPARSE for specific privacy concepts. To enhance interpretability and visualization. we focus on individual words rather than aggregated token sets as in prior experiments. Figure P visualizes the learned neuron masks for six semantically coherent groups: weekdays, countries,
dimensions
8

---

Published as a conference paper at ICLR 2026
Monday
huesaay
012
Table Leakage mitigation rates achieved by SPARSE with c 10 compared to non-pront on-protected embeddings. Results are evaluated across three token types: target tokens, semantically similar tokens, and unrelated (other) tokens under differ- ent privacy categories.
Thursday
Ireand
0.10
Menocco-
January
0.00
May
U.S.
0.06
amenica
gin
0.04
women-
0.02
Target
Similar
Other
Weekdays -76.2%
-46.2%
-11.7%
Country
-64.3%
-3.22%
-2.11%
Figure 2: Visualization of the learned neuron mask by SPARSE for individual tokens, where larger values represent higher privacy sensitivity.
Months
-72.5%
-4.88
-12.6%
Gender
-61.0%
-40.5%
-18.3%
City
-70.2%
-39.8%
-14.7%
months, U.S.-related terms, gender-related terms, and numbers. The x-axis shows the union of the top-5 neuron indices most strongly associated with each word. We have the following two findings:
1) Semantically related words activate overlapping privacy-sensitive dimensions. As depicted in Figure 2 we found that words with similar semantics, such as weekdays or countries, tend to cluster around the similar embedding dimensions. The clustering behavior verifies the quality of our proposed neuron mask detection process, demonstrating that it effectively localizes meaningful, non-random privacy signals that align with linguistic structure.
2) SPARSE implicitly protects semantically similar tokens. We hypothesize that protecting token's 's privacy-sensinge dimensions also benefits semantically similar tokens, as they often share overlapping dimensions. test this, we apply the learned neuron mask for each target token and evaluate leakage reduction for three types: the target, semantically similar, and unrelated tokens. Leakage mitigation is quantified as the relative reduction of the Leakage metric compared to the non-protected embedding. As Table 5 shows, SPARSE substantially reduces leakage for similar tokens (e.g.. 46.2% for "Weekdays") ), even though only the target was protected. These results suggest that although our privacy-sensidee dimensions identified based explicitly defined tokens, implicitly extends protection to a broader, more generalizable privacy concept.
RELATED WORK
Inversion Attacks on Text Embeddings. Text embeddings have been shown to pose serious privacy risks, as they can unintentionally encode and expose sensitive attributes and content (Pan et al. 2020 Song & Shmatikov 2019 Lyu et al. 2020b; Coavoux et al. 2018). For example, prior work (Pan ct al. 2020) demonstrated that keywords can be partially recovered from text embeddings using annotated external datasets. Similarly, attribute inference and embedding inversion attacks have been used to extract unordered sets of words from sentence representations (Song & Raghunathan, (200). GEIA (Li al.) 2023) extended these attacks by introducing generative approach that reconstructs entire input sequences. More recently, Vec2Text (Morris et al. 2023) showed that embeddings from commercial APIs (e.g., OpenAl) be inverted with high accuracy. These findings underscore the need for robust
embedding methods.
Siivacy-preserving Text Embeddings. To mitigate privacy risks in textual representations, prior work has introduced various noise injection mechanisms for token- and sentence- embeddings. DPNR (Lyu al. 2020b) randomly masks input tokens and adds Laplace noise the resulting embeddings. Feyisetan et al. (Feyisetan et al., 2019) apply generalized Laplace mechanism to perturb token embeddings under metric local differential privacy (LDP). For sentence embeddings, Lyu al. (Lyu al.) 2020a) directly inject Laplace noise into BERT-based vectors. Laplace- mechanisms have also been employed to defend against inversion (Morris et al. 2023). membership inference (Song& & Raghunathan 20200,, and attribute inference (Coavoux al. 2018) attacks. Recent work such as the Purkayastha mechanism (Du et al. 2023) further refines Laplace perturbation for enhanced privacy guarantees.

---

Published as a conference paper at ICLR 2026
CONCLUSION
We introduced SPARSE, a framework that enhances privacy in text embeddings by selectively apply- ing sensitivity-guided elliptical noise. By identifying and perturbing privacy-sensitive embedding dimensions, SPARSE resists embedding inversion attacks while preserving utility. Experiments across models, datasets, and threat scenarios demonstrate its effectiveness in improving the privacy-utility tradeoff. As embeddings become central to real-world systems, embedding privacy is essential. We see SPARSE as step toward controllable, concept-aware protection, and hope it encourages research into adaptive and accountable defenses for sensitive NLP.
ACKNOWLEDGMENTS
This material is based upon work supported by National Science and Technology Council, ROC under grant number 14-2221-E-002- 134-MY3 and by Taiwan Centers of Excellence (TCE)
10

---

Published as a conference paper at ICLR 2026
ETHICAL CONSIDERATIONS
While SPARSE is designed to enhance privacy in text embedding applications, its deployment must be guided by ethical considerations. First, although our method reduces the risk of embedding inversion, it does not eliminate all privacy threats, and may offer false sense of security if used without awareness of its limitations. Practitioners should carefully evaluate the privacy requirements of their specific context and avoid over-relying embedding anonymization as substitute for broader data governance and access controls.
Second, our framework is concept-driven and depends predefining sensitive information categories. This raises fairness concerns: groups or attributes not explicitly included in the sensitive concept space may receive less protection, potentially reinforcing systemic biases exposing vulnerable populations. Future implementations should strive for inclusiveness in concept selection and explore concept-agnostic sensitivity detection to mitigate this risk.
Finally, as with any privacy-preserving technique, SPARSE could be misused- example, to evade moderation mask malicious content. We encourage responsible use aligned with principles of transparency, accountability, and user consent, especially in high-stakes domains such as healthcare, education, or law.
REPRODUCIBILITY STATEMENT
All essential details required to reproduce our main results are provided in this paper. Appendix H offers comprehensive descriptions of the model architectures and training procedures, Appendix details the attack configurations used in our experiments, and Appendix presents the formal definitions of all evaluation metrics. addition, we plan to publicly release our code in the near future to further facilitate reproducibility and future research.
REFERENCES
Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. SemEval-2012 task 6: A pilot semantic textual similarity. In Eneko Agirre Johan Bos. Mona Diab, Suresh Manandhar, Yuval Marton, and Deniz Yuret (eds.), *SEM 2012: The First Joint Conference on Lexical and Computational Semantics Volume I: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), pp. 385-393, Montréal, Canada, 7-8 June 2012. Association for Computational Linguistics. URL ptttps://aclanthooogy. org/ S12 1051
Encko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. SemEval-2014 task 10: Multilingual semantic textual similarity. In Preslav Nakov and Torsten Zesch (eds.), Proceedings of the 8th International Workshop Semantic Evaluation (SemEval 2014), pp. 81-91, Dublin, Ireland, August 2014. Association for Computational Linguistics. doi: (0.3115/v1/S14-2000 URL https://accaane 1o
Mário Alvim, Konstantinos Chatzikokolakis, Catuscia Palamidessi, and Anna Pazii. Local differential privacy on metric spaces: optimizing the trade-off with utility. 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pp. 262-267. IEEE, 2018.
Alexander Bondarenko, Maik Fröbe, Meriem Beloucif, Lukas Gienapp. Yamen Ajjour, Alexander Panchenko, Chris Biemann, Benno Stein, Henning Wachsmuth, Martin Potthast, al. Overview of touché 2020: argument retrieval. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 1 1thh International Conference of the CLEF Association, CLEF 2020, Thessaloniki, Greece, September 22-25, 2020, Proceedings 11, pp. 385-395. Springer, 2020.
Vera Boteva, Demian Gholipour, Artem Sokolov, and Stefan Riezler. A full-text learning to rank dataset for medical information retrieval. In Advances in Information Retrieval: 38th European Conference on IR Research, ECIR 2016, Padua, Italy, March 20-23, 2016. Proceedings 38, pp. 716-722. Springer, 2016.
11

---

Published as a conference paper at ICLR 2026
Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, and Florian Tramer. What does it mean for a language model to preserve privacy? In Proceedings of the 2022 ACM Conference Faimess, Accountability, and Transparency, pp. 1280-22922 2022.
Daniel Cer, Mona Diab, Eneko Agirre, lñigo Lopez-Gazpio, and Lucia Specia. SemEval-2017 task I: Semantic textual similarity multilingual and crosslingual focused evaluation. In Steven Bethard, Marine Carpuat, Marianna Apidianaki, Saif M. Mohammad, Daniel Cer, and David Jurgens (eds.), Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017),
pp. 1-14, Vancouver, Canada, August 2017. Association for Computational Linguistics. doi: 10. 10.18663/1/S1172000 URL.ttss:: /ttp///allaatttt////o http:///////aaooooooooooo/////// 2001
Konstantinos Chatzikokolakis, Miguel E Andrés, Nicolás Emilio Bordenabe, and Catuscia Palamidessi. Broadening the scope of differential privacy using metries. In Privacy Enhanc- ing Technologies: 13th International Symposium, PETS 2013, Bloomington, IN, USA, July 10-12, 2013. Proceedings 13, pp. 82-102. Springer, 2013.
Maximin Coavoux, Shashi Narayan, and Shay B Cohen. Privacy-preserving neural representations of text. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1-10, 2018.
Minxin Du, Xiang Yue. Sherman SM Chow, and Huan Sun. Sanitizing sentence embeddings (and labels) for local differential privacy. In Proceedings of the ACM Web Conference 2023, pp. 2349 2359,, 2023.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3, pp. 265-226222 284. Springer, 2006.
Oluwaseyi Feyisetan, Tom Diethe, and Thomas Drake. Leveraging hierarchical representations for preserving privacy and utility text. In 2019 IEEE International Conference on Data Mining (ICDM), pp. 210-219. IEEE, 2019.
Oluwaseyi Feyisetan, Borja Balle, Thomas Drake, and Tom Diethe. Privacy-and textual analysis via calibrated multivariate perturbations. Proceedings of the 13th international conference on web search and data mining, pp. 178-186, 2020.
Yu-Hsiang Huang, Yuche Tsai, Hsiang Hsiao, Hong- Yi Lin, and Shou-De Lin. Transferable em- bedding inversion attack: Uncovering privacy risks text embeddings without model queries. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4193- 4205, Bangkok, Thailand, August 2024. Association for Computational Linguistics. URL 20240///22222.e22 23C
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations, 2022.
Alistair E W Johnson, David J Stone, Leo A Celi, and Tom J Pollard. The mimic code repository: enabling reproducibility in critical care research. Journal of the American Medical Informatics Association, 25(1 :):2-3)):999. 2018.
Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):335- 547, 2019.
Shiva Prasad Kasiviswanathan, Homin K Lee, Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. What can we learn privately? SIAM Journal on Computing, 40(3):793-826, 2011.
Donggyu Kim, Garam Lee, and Sungwoo Oh. Toward privacy text embedding similarity with homomorphic encryption. In Proceedings of the Fourth Workshop on Financial Technology and Natural Language Processing (FinNLP), pp. 25-36, 2022.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, al. Retrieval-augmeene genera- tion for knowledge-intensiee nlp tasks. Advances in Neural Information Processing Systems, 33: 9474, 2020.
12

---

Published as a conference paper at ICLR 2026
Haoran Li, Mingshi Xu, and Yangqiu Song. Sentence embedding leaks more information than you expect: Generative embedding inversion attack to recover the whole sentence. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 14022-14044 2023.
Christos Louizos, Max Welling, and Diederik P Kingma. Learning sparse neural networks through 10 regularization. In International Conference on Learning Representations, 2018.
Lingjuan Lyu, Xuanli He, and Yitong Li. Differentially private representation for nlp: Formal guarantee and empirical study privacy and fairness. In Findings of the Association for Computational Linguistics: EMNLP 2020, pp.
2020a.
Lingjuan Lyu, Yitong Li, Xuanli He, and Tong Xiao. Towards differentially private text repre- sentations. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1813-1816, 2020b.
Maddison, A Mnih, and Teh. The concrete distribution: A continuous relaxation of discrete random variables. In Proceedings of the international conference learning Representations. International Conference on Learning Representations, 2017.
Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. Www' 18 open challenge: financial opinion mining and question answering. In Companion proceedings of the the web conference 2018, pp. 1941-1942, 2018
Microsoft Corporation. Language service overview. https: //earn microsoft. com/ en
John Morris, Volodymyr Kuleshov, Vitaly Shmatikov, and Alexander M Rush. Text embeddings reveal (almost) as much text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 12448-12460, 2023.
Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. Mteb: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316. 2022. doi: 10.6650/ARXXW.220..07..66 URL
hittps://arxiv: wwtpriivv..rr...o. abs 221.22211111111
Jianmo Ni. Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. Sentence-t5: Scalable sentence encoders from traitre text-to-text models. In Findings of the Association for Computational Linguistics: ACL 2022, pp. 1864-1874, 2022a.
Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wer Wei Chang, et al. Large dual encoders are generalizable retrievers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 9855, 2022b.
Xudong Pan, Mi Zhang, Shouling Ji, and Min Yang. Privacy risks of general-purpose language models. In 2020 JEEE Symposium on Security and Privacy (SP), pp. 1314-1331. IEEE, 2020.
Shaina Raza, Deepak John Reji, Femi Shajan, and Syed Raza Bashir. Large-scale application of named entity recognition to biomedicine and epidemiology. PLOS Digital Health, 1(12):e0001122. 2022.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) pp. 3982-3992, 2019.
Congzheng Song and Ananth Raghunathan. Information leakage in embedding models. In Pro- ceedings of the 2020 ACM SIGSAC Conference Computer and Communications Security, pp. 377-390, 2020.
Congzheng Song and Vitaly Shmatikov. Auditing data provenance in text-generation models. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 196-206, 2019.
13

---

Published as a conference paper at ICLR 2026
Samuel Sousa and Roman Kern. How keep text private? systematic review of deep learning methods for 1427-1492, 2023.
natural language processing. Artificial Intelligence Review, 56(2):
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319-3328, PMLR, 2017.
A14Privacy Team. Pii masking 300k dataset, 2023. URL https: http::::///gggnnnf.fcc acce. co/ Licensed under Apache License 2.0. Ac-
dataseess aatissttsaaippivvatt cessed on: Sep 30, 2024.
300k
Xi Wu, Fengan Li, Arun Kumar, Kamalika Chaudhuri, Somesh Jha, and Jeffrey Naughton. Bolt-on differential privacy for scalable stochastic gradient bescent-based analytics. Proceedings of the 2017 ACM International Conference Management Data, pp. 1307-1322, 2017.
14

---

In this section, we introduce the concept of privacy neurons and empirically validate their existence and relevance. We demonstrate that privacy-related information within text embeddings may be Published as a conference paper at ICLR 2026
Top-1 DSC 10%0
80
Dohaty
40
0.06
Sensitivity
Figure 3: Sensitivity distribution comparison between the top and bottom 10% privacy neurons. The
Wilcoxon Signed Rank Test indicates a highly significant difference (p-value 1.30 x 10 10-21).
A EMPIRICAL VALIDATION OF PRIVACY-SENSITIVE DIMENSIONS
primarily concentrated in limited subset of dimensions.
Definition 6 (Privacy Neurons). Consider an input text and an embedding model - TR...
We assume there is subset of dimensions N₂ C = {l.....d that encapsulates the sensitive information associated with a privacy concept C. Consequently, the embedding ((x) can be expressed
(s) = (# (s),
(6)
where Dx (s) represents the privacy-sensitive neuron activations and ( the privacy- invariant neuron activations.
Intuitively, dimensions identified privacy neurons should exhibit higher sensitivity to the presence absence privacy-related tokens in the input text. To quantify how individual embedding dimensions respond to privacy-related information, introduce the following measure:
Definition 7 (Neuron Sensitivity). Let D+ and D denote positive and negative datasets containing sentences with and without tokens related to the privacy concept C, respectively. For each embedding dimension i, the neuron sensitivity As is defined as:
A max (f) +bs D E
where (-); represents the activation value of the i-th embedding dimension.
We assume high value of A; indicates that dimension i is responsive and likely encodes privacy- related information.
Dataset Construction for Sensitivity Analysis To measure the embedding changes associated with the privacy concept C, we first construct dataset D+ = {8i,... (B1,....,$p-111. S|D+1] containing sentences that include tokens from concept C. Correspondingly, we generate a negative set D = (R(s, C) E D+J. where R(s, C) denotes the operation of removing all tokens t; C from the sentence s,, Thus, D consists of sentences identical to D* except for the absence of tokens associated with the sensitive privacy concept.
Results Figure 3 presents the distribution of sensitivity scores for dimensions identified as the top and bottom 10% privacy neurons based on the sensitivity vector v. Our pilot study clearly illustrates a significant difference between the two groups. Specifically, the top-ranked privacy neurons demonstrate substantially higher sensitivity scores (mean sensitivity = 0.04) than the bottom- ranked neurons, which exhibit nearly zero sensitivity. Wilcoxon Signed Rank Test confirms the significance of this observation with p-value of 1.30 x 10 These results empirically support the existence of privacy neurons, suggesting that embedding inversion attacks may be effectively mitigated by selectively manipulating only small subset of embedding dimensions.
15

---

Published as a conference paper at ICLR 2026
B MISSING PROOF IN SECTION 3.2
B. PROOF OF THEOREM I
Proof of Theorem
7 Recall that the mechanism releases ('(s) =$(s) Z, where the noise density
[z) = C expl =||=|[as) and the normalizing constant C' is independent of 2.
For any output y RR we have:
Pr (s) y)
fz(y (()))
(8)
Pr(('(s')= y)
Iz(y ((s'))
Cexpl -e (r
(9)
Cexpl-<(p +(s') [[s')][m] M)
=ex - (()) + ellv (()))lM))
(10)
By the triangle inequality for the Mahalanobis norm, we have:
y (s)lla - |yy - (ss') I [[x(s) - (s
(11)
Therefore:
Pr( vl Pr( = yl
$ exp (cllb(s) - ((s')
(12)
This precisely establishes ed-local differential privacy under the Mahalanobis norm.
B.2 PROOF OF LEMMA 1
Proof of [emma Because Σ is symmetric positive-definit it admits the spectral decomposition QAQ where Q is orthogonal (QT Q = I) and A diag(Eı..... E.. collects the eigenvalues E1...., En of E. Write i == Q u; note that ||ö||| = ||0l|2 because Q is orthogonal.
Upper bound. By assumption & 2 for every i, hence the eigenvalues of Σ satisfy & sc- Therefore
|
|uu133 v v0 Tt -
i=1
which yields ||b|Iи & |1/11// Ve.
Lower bound. 1/n and
Because traco(E) n, Σm & n, implying & for every i. Consequently
|
1
1
1
so that [|l]| 2 ||0lla/vm
Combining the two inequalities completes the proof.
B.3 PROOF OF LEMMA 2
Proof of Lemma
Let 0 (xx)) ((x')
E R By Lemma
we have the deterministic bounds
|
||0||2
||||| M
Vc
Multiplying each term by the non negative scalar preserves the ordering, and applying the (strictly increasing) exponential map yields
|1u|l < exp (ellullar) s exp (fglella)
exp
which is precisely the desired statement.
16

---

Published as a conference paper at ICLR 2026
C ALGORITHM FOR MAHALANOBIS NOISE SAMPLING
Algorithm Sampling from fz(2) & exp(-dleelv)
1: Input: Privacy budget e, dimension n, a positive definite matrix Σ
2: Sample an n-dimensional random vector N from multivariate normal distribution with mean zero and identity covariance matrix.
3: Normalize X N/||N]|
4: Sample from Gamma distribution with shape parameter n and scale parameter 1/e
5: Return Z = y E1/2X
Lemma 3. The random variable Z returned from Algorithm I has prohability-densiyy function of the form
Jz(z) & exp(-e1|z|la)) ||=||>r = V=TE-12.
Proof. Define U YX. Note that conditional on = y. uniformly distributed on the sphere of radius in Rm. Hence
fuly (u y) y (m-1) whenever ||u|12 = y.
and zero otherwise. Using the Dirac delta function 8(-), we write
fu l fury(u v) fy (v) (y [1ull2) dy
(m-1) e" [(n)
y y 8 11ul2) dy
<1lulla
so fu (u) x exp(-e/lull2).
Since E is positive definite, 1/22 exists and is invertible. Setting E1/2 U, the laaane-of--vriablle formula yields
fz(2) = fu (12)) |det(=-1/3)
exp [-e||=-1/2210 exp (opp--eeeT-1 =exp(= -exp(-eller =1=[]=)
This completes the proof.
Table Statistics datasets.
FIQA
Quora Retrieval
NFCorpus
MIMIC-III
PHI-30K
Downstream task
STS
Retrieval
STS
STS Semfival
Retrieval
Domain
SemEEval
Financial
Semfival
QA
Medical
Medical
Sentenees
5500
17256
10000
2590
4244
127677
Average sentence length
14.53
10.50
10.17
9.77
9.53
3.31
15.03
47.12
Unique numed entisies
123
41
228
90
13
290
Evaluation metric
Pearson Cor::
ND
Pearson Corr.
Pearson Con:.
NDCG@ 10
NDCG@10
D DATASET STATISTICS AND EVALUATION METRICS
Privacy Metrics.
To quantify the privacy risk of our model, we adopt two complementary metrics: Leakage and Confidence. These metrics assess both the accuracy and certainty of an adversarial model attempting to infer sensitive information from the model's outputs.
(1) Leakage. Leakage measures the extent to which an attack model can recover sensitive tokens from an obfuscated embedding. Given sentence s; containing sensitive tokens C; C C. the attacker
17

---

Published as a conference paper at ICLR 2026
generates reconstructed sentence s; As))(s;)) based the obfuscated embedding. The leakage is computed by checking whether any sensitive token appears in the reconstructed sentence:
1 Σ Σ 1 E
Leakage
(13)
where N the number of text samples, C, is the set of sensitive tokens in sentence 8,, 8, the reconstructed sentence from the attacker, and T - |C;] is the total number of sensitive token instances across the dataset. A lower Leakage score indicates better protection of sensitive content, as fewer sensitive tokens are successfully inferred by the attacker.
(2) Confidence. Confidence quantifies how certain the attack model is when predicting sensitive tokens, regardless of whether the predictions are correct. It is defined as the average predicted probability assigned the true sensitive tokens across all samples:
N Σ Σ
Confidence 7
PA(1 ('s:))
(14)
2= LEC;
where C, C C the set of sensitive tokens in sentence s,, 4 (s:) is the obfuscated embedding, and T T= T=E 1= Cil is the total number of sensitive token instances. The term (t (((s)) denotes the probability assigned by the attack model A to sensitive token based on the obfuscated embedding. lower Confidence score indicates that the model is less certain in its inference, suggesting stronger privacy.
Utility Metrics. To assess the utility of the learned representations, we follow the widely adopted evaluation framework provided by the Massive Text Embedding Benchmark (MTEB) (Muennighoff) ct al., 2022). MTEB is standard benchmark for embedding models, covering diverse set of downstream tasks such as classification, clustering, retrieval, and semantic textual similarity. These tasks reflect the practical performance of embeddings in real-world applications. By using MTEB, we ensure that our utility evaluation is comprehensive, comparable, and aligned with established practices in the embedding research community.
E SNSSTTIVE TOKEN EXTRACTION
We utilize the MIMIC-III clinical notes corpus (Johnson et al. 2018). a de-identified electronic health record dataset comprising detailed clinical documentation from intensive care units. To extract privacy-sensitive information, we apply biomedical Named Entity Recognition (NER) model (Raza al.) (022) specifically trained to identify medically relevant entities such as age, sex, diseases, and symptoms. For non-clinical datasets, named entities are extracted using the en_core_web____ NER pipeline from the spaCy library which provides general-purpon entity recognition for categories such persons, locations, and organizations.
F ADDITIONAL EXPERIMENTAL RESULTS
F.1 PERFORMANCE ON MORE DATASETS
In addition to the STS12 Aggire et al. 2012) and FIQA (aaa et al. 2018) datasets used in the main experiment, Table 6 also presents statisties of other datasets, including STSB (Cer al. 2017). STS14 (Agirre et al. 2014), Quora (Bondarenko et al. 2020). and NFCorpus (Boteva et al. 2016). Table 7 shows the complete defense performance on all datasets. Besides using Leakage, we also utilize Confidence to assess the defense performance. This metric reflects the certainty of the attack model's 's predictions. A higher Confidence score indicates that the model more confident in its prediction of the sensitive token. For the semantic textual similarity (STS) task, downstream
http::/hhhhhttccooooo/oooa////// sasee////seaeeeeeee///////aa cttpiaaaaaaeeooeeeeeeeeeeeeeeeee web_
3.7.
18

---

Published as conference paper at ICLR 2026
performance is measured using the Pearson correlation of Cosine Similarity (Pearson corr.. In the context of information retrieval, we employ the ranking metric NDCG@ 10. As described in Section 4.2 SPARSE consistently demonstrates superior performance over LapMech and PurMech across all levels of perturbation and datasets, both defense and downstream task metrics.
Table Privacy-utility tradeoff across different defense Methods. Privacy leakage is assessed using Leakage and Confidence metrics, where lower values indicate stronger privacy protection. Utility is measured by fata-specific downstream performance. All metrics are presented percentages (%).
Privacy Metries
Utility Metric
Leakage
Confidence
Downstream
Dataset
LapMech
PurMech
SPARSE
Lap Mech
PurMech
SPARSE.
Lap
PurMech
SPARSE
5
20.75
19.03
2,68
1,89
1.98
1.78
40.03
40.05
48.17
53.79
76.14
20
71.82
69.15
64.79
41.76
38.44
36.33
80.95
80.95
81.00
STSB
30
76.15
74.98
73.10
$2.36
49.28
48.63
81.08
$1.08
80.91
40
78.94
77.40
76.42
56.84
54.02
53.98
80.95
80.95
80.81
86.75
66.57
80.64
1.03
39.76
39.71
48.47
10
4.04
4.13
2.41
21.86
21
21.10
70.28
70.25
74.44
STS14
8.64
27.73
79. 16
79.31
30
11.22
11.26
14.70
30.38
30.39
32.95
79.47
79.47
79.37
40
13.67
13.50
16.12
32.09
32.05
34.81
79.43
79.43
79.32
21.97
5
25.96
25.87
2.85
2.71
2.70
1.57
11.89
11.78
15.43
70.19
20
75.56
75.80
68.00
50.87
51.00
41.21
82.79
82,75
83.94
81.65
81.65
76.75
58.99
59.08
53.43
83,70
83,72
84.02
40
83.69
83.64
79.79
62.28
62.06
57.32
83.90
83,91
83.97
oc
89,30
68,30
84.01
7.77
0.68
1.27
1.06
0.83
23.70
23.61
19.94
10
29.73
31.42
12.16
15.92
15.51
6.73
27.31
27.38
29.61
56.76
30.76
30.75
31.04
NFCorpus
30
69.93
69.26
57.77
$8.27
58.00)
48.09
31.32
31.32
31.37
40
78.72
79.05
66.55
63.89
63.83
53.85
31.56
31.56
31.52
88.18
75.54
31.63
F.2 DEFENSE PERFORMANCE ON MORE EMBEDDING MODELS
To assess the generalizability of SPARSE, we evaluate its performance three representative embedding models: GTR-base (Ni al. 2022b). Sentence-TSs eN 2022a), and SBERT (Reimers) & Gurevych 2019). As presented in Table & SPARSE consistently achieves low privacy leakage (e.g.. 19% with base and 17% with SBERT), D, while preserving strong downstream utility. In contrast, baseline methods such as LapMech and PurMech not only suffer from higher leakage rates (20-30%) but also incur greater utility degradation. These results support the generality our approach and validate the effectiveness of detecting and perturbing privacy-sence sensiivve dimensions across different embedding architectures.
Table Defense and downstream performance using different embedding models under = 10. We use STS12 dataset and report the mean and standard deviation of runs for all evaluation metrics.
Embedding Models
GTR-base
Sentence-T5
SBERT
Metries
Leakage
Downstream
Leakage
Downstream
Leakage
Downstream
Non-protected
60.09
74.25
43.83
86.79
42.11
81.36
LapMech
22.34 20.02
60.72 20.00
31.71 saisa
63.16 20.00
23.82 an.so
66.89 20.00
PurMech
22.66
60.72 20.00
32.11
63.15
23.59
65,89
SPARSE
19.31
65.27
22.38
74.45
17.15
69.42
19

---

Published as a conference paper at ICLR 2026
F.3 COMPARISON WITH
DEFENSE METHODS
Since the goal of SPARSE aims to mitigate the privacy leakage of sensitive tokens, it raises a natural question: how does SPARSE compare traditional PII removal or transformation methods? To answer this question, we evaluate three additional PII-based defense approaches: (1) PII removal via Azure Language Service (Microsoft Corporation). which replaces private tokens with *** (2) Random word replacement from the corpus, and (3) Semantic word replacement within the same named entity category. The results are presented in Table 9
We have the following key insights:
PII transformation incurs significant information loss. All Pll-based strategies lead to noticeable degradation in downstream performance. For instance, PII redaction reduces STS12 accuracy from 74% to 59%, and FIQA from 33% to 21%. Semantic replacement fares slightly better, with scores of 64% (STS12) and 18% (FIQA), but still underperforms relative to the original embeddings. Random replacement exhibits similar decline, indicating that simple token-level transformations often disrupt semantic integrity.
SPARSE achieves better privacy-utility tradeoff. While PII transformations obscure sensitive content, they often compromise task utility. To evaluate this tradeoff, we define tradeoff rate metric ALeakage where ALeakage is the reduction in privacy leakage, and AUtility is the drop in R Utiiity downstream performance relative to the unprotected embeddings. For simplicity and upper-bound estimation, we assume that Pll-based methods reduce leakage zero. As shown in Table SPARSE achieves markedly higher tradeoff rates of 23. 11 STS12 and 26.30 FIQA, compared to 4-6 for the PlI-based approaches. These results verify the advantage of embedding-level defenses like SPARSE, which enable more nuanced and fine-graine fine-grained privacy preservation without sacrificing utility.
Table 9: Comparison of privacy-utility tradeoff between SPARSE and PII transformation methods.
Dataset
Defense Methods
Leakage ((%)
Downstream 1%%)
Tradeoff Rate R 1
Unprotected RemovePI1
60.09
74.25
59.47
4.12
STS12
Random-Replace
60.50
4.42
Semantic- Replace
64.46
6.22
SPARSE (c = 20)
36.98
73.25
23.11
Unprotected
77.35
33.56
RemovePII
21.24
6.27
FIQA
Randomn-Replacement
19.20
5.38
Semantic-Replacemeet SPARSE (c = 20)
18.37
5.09
53.41
32.65
26.30
F.4 HYPERPARAMETER ANALYSIS
We analyze the impact of the regularization parameter À on the tradcoff between privacy and utility. As shown in Table 10 increasing À results in reduced leakage across all values of e, confirming that stronger regularization suppresses sensitive information more effectively. However, this comes at the cost of reduced downstream performance, particularly under lower c, where the noise becomes more dominant. Notably, moderate values such as À = le-3 strike balance, achieving significant privacy gains with tolerable performance degradation.
G COMPUTATIONAL OVERHEAD
We provide analysis of the computational overhead introduced by SPARSE, focusing on both inference-ting time noise sampling and offline neuron mask training.
Inference Cost. During inference, the dominant overhead arises from sampling Mahalanobis noise, which involves a lightweight matrix multiplication. To evaluate efficiency, we measured the average inference latency per sample over 10,000 runs and compared SPARSE with representative
20

---

Published as a conference paper at ICLR 2026
Table 10: Effect the regularization hyperparameter À privacy leakage and downstream perfor- mance under different privacy budgets Smaller values lead to stronger regularization.
Leakage ((%)
Downstream 1 (%)
Dataset
30
40
5
10
20
1e-2
0.52
0.91
1.37
1.84
2.15
22.14
36.87
43.25
44.06
44.38
5e-3
1.36
3.82
10.34
13.76
27.61
48.05
56.17
57.88
58.19
STS12
1e-3
4.34
19.31
36.98
43.81
47.54
34.12
65.27
73.25
74.04
74.15
5e-4
7.62
25.83
44.23
51.41
55.27
31.33
59.78
67.10
67.98
68.24
le-4
9.88
33.02
51.47
58.62
62.90
28.40
52.45
59.66
60.34
60.79
1e-2
0.78
1,28
1.93
2.71
3.26
8.71
13.82
17.44
17.83
18.14
5e-3
2.26
6,42
11.68
17.23
21.14
11.38
18.67
25.09
25.66
25.94
FIQA
1e-3
8.48
31.62
53.41
63.51
68.13
14.87
23.45
32.65
33.58
33.85
5e-4
11.05
36.43
58.23
67.28
71.83
13.72
21.42
28.93
29.84
30.11
le-4
13.61
40.82
62.14
70.25
74.44
12.45
18.63
25.42
26.28
26.50
baselines: the Laplace Mechanism and the Purkayastha Mechanism. The results are summarized in
Table III
Table 11: Average inference time per sample (in microseconds).
Method
Latency (us/sample)
Laplace Mechanism
39.8
Purkayastha Mechanism
33,200
SPARSE (ours)
48.4
As shown, SPARSE introduces only a marginal overhead compared the Laplace Mechanism (less than 25% increase), while being several orders of magnitude more efficient than the Purkayastha Mechanism. This confirms that SPARSE is suitable real-time and low-latency applications.
Training Cost. The training cost arises from learning the neuron mask used to identify privacy- sensitive dimensions. This a one-time offline process that can be precomputed and reused, and therefore does not affect inference efficiency. The training time scales linearly with dataset size and remains practical common settings. For instance, training on 10,000 samples takes 25.3 minutes, and 20,000 samples, it completes in under 45 minutes. Further acceleration can be achieved with larger batch sizes or distributed training.
H IMPLEMENTATION DETAILS OF SPARSE
H.1 TRAINING ALGORITIM FOR TEURON-SENSITIEE DETECTION
Algorithm@deetn details the training procedure used to learn a neuron mask that identifies privacy-sensitie dimensions the embedding space. The method jointly optimizes a differentiable binary mask and classifier to distinguish between samples containing privacy concept and their perturbed counterparts. hard conerete distribution used approximate binary masking in differentiable manner, and the training objective combines classification loss with -unsitt-inducing regularization term.
H.2 TRAINING SETTINGS
We train our privacy-sensitive dimension identification model using mini-batch gradient descent with the Adam optimizer. The model is trained for 100 epochs with a batch size of 64 and learning rate of x 10 4 The predictor Py is implemented as a multi-layer perceptron (MLP) with two hidden layers of sizes 256 and 128, respectively, and ReLU activations. We conduct hyperparameter search over À E {0.01, 0.005, 0.001, 0.0005, 0.0001} and set À = 0.001 as the default for all experiments unless stated otherwise. All implementations are based PyTorch.
21

---

Published as a conference paper at ICLR 2026
Algorithm 2 Training Neuron Mask for Privacy-Sensitive Dimension Detection
1: Input: Paired dataset D+ D-, embedding function (-). learning rate y), temperature 3, regu-
larization coefficient X, initialization of mask logits log a, constants Ç = 1.1, -0.1
2: Initialize classifier parameters 0
3: for each epoch = to N do for each minibatch {(s , (5 C (D+, D=) do
4:
5:
for each mask dimension do
6:
Sample μ4; ~ [1 1)
7:
Compute s; R;
1 (log #4 +log +oga:
1-p4
8:
Compute m, min (1, max (0, s:(5 - ?) ?))
9:
end for
Compute masked embeddings: 4 4(s+) ) m, $ (
11:
Compute classification loss Lets (m, 0) using Eq. equation
Compute regularization loss Lieg (m) using Eq. equation
13:
Compute total loss: Liotal = Lels + ALrg
Update <- 0 - 1/ V
15:
Update log a +- log 17 V log Fiotal
16:
Update log +- log 3 7, Viog Liotal
17:
end for
18:
end for
19:
Output: Trained classifier Pe, optimized neuron mask
H.3 COMPUTING RESOURCES
All experiments were performed on a workstation with an Intel Core 19-10980XE CPU (18 cores, 36 threads, 3.00GHz) and NVIDIA RTX 3090 GPU with 24GB of memory. The system runs 64-bit x64 architecture.
I IMPLEMENTATION DETAILS OF ATTACK MODELS
To thoroughly evaluate the privacy risks associated with text embeddings, we adopt three repre- sentative attack models: Vec2text (Morris et al., 2023), GELA (Li et al. (2023), and MLC (Song & Raghunathan 2020). These models represent both sentence-level and word-level inference attacks, and are implemented fine-tuned under controlled conditions to assess the effectiveness of various privacy-preserving mechanisms.
1.1 VEC2TEXT
Vec2text is sentence-level attack model designed reconstruct input text directly from embed- dings. We use the publicly available pre-trained version of Vec2tex which is based the GPT-2 architecture. To simulate a realistic adversarial scenario, we fine-tune this model for 50 epochs individually embeddings perturbed by each defense method (LapMech, PurMech, and SPARSE). The fine-tuning is performed using a batch size of 32 and a learning rate of 5e-5, optimized with Adam.
1.2 GEIA
GEIA is another sentence-level reconstruction model that inverts embeddings into textual sequences using a fine-tuned GPT-2 decoder. Unlike Vec2text, GEIA employs mapping network to project embeddings into the GPT-2 latent space. We use GEIA based on the original pape["] using a two-layer MLPP as the projection module. The GPT-2 decoder is initialized from the HuggingFan Face Transformers library and fine-tuned for 30 epochs using embeddings from each defense method. The model is optimized using Adam with a learning rate of 3e-5 and trained with a batch size of 16.
- st stttsstsninii inversiio
https://github: com HSJST- /tt.:///T-KKoOCoo/////...a/
22

---

Published as a conference paper at ICLR 2026
1.3 MLC
MLC is a word-level embedding inversion attack model that predicts whether specific sensitive tokens present in the input text based on its embedding. The model consists of a three-layer MLP with hidden sizes [512, 256, 128], ReLU activations, and a sigmoid output layer. We train a separate MLC for each perturbation method using binary cross- cross-entropy loss function. Training is performed for 20 epochs using a batch size of 64 and learning rate of le-4 with the Adam optimizer.
J CASE STUDY ON MIMIC-I11 DATASET
To demonstrate the privacy risks in specific threat domain, we conducted case study using MIMIC- III clinical notes (Johnson et al. 2018). Table 12 presents the results of embedding inversion attack on two types of sensitive tokens (" ""age" and "disease name") with different noise levels. We assessed the semantic fidelity of the reconstructed sentences by comparing their similarity to the original text using cosine similarity from an external embedding model.
In Example 1, we applied a strong perturbation level of c = 5 to perturb the text embeddings. Under this condition, all three defense methods (LapMech, PurMech, and SPARSE) effectively prevented the leakage of sensitive age information. However, LapMech and PurMech significantly degraded the semantic quality of the embeddings with only 11% of the original semantic similarity. In contrast, SPARSE maintained 62% semantic similarity. In Example 2, we used a lower perturbation level of e = 10. Here, both LapMech and PurMech failed protect against privacy leakage and further compromised the semantic integrity of the embeddings. Conversely, SPARSE successfully safeguarded the sensitive information while preserving semantic quality of the embeddings.
Table 12: Case study on the MIMIC-III dataset with two sensitive words and perturbation level e. We highlight the leakage of sensitive words and demonstrate the semantic similarity of the reconstructed sentence the ground truth.
Example 1: Protect age with strong noise e
5
Method
Defense
Semantic
Reconstructed Sentence
Ground truth
this 688-year-old white male has a history of diahetes, hyperlipidemia and hypertension this 68-year-old white male a history of hypertension, hyperlipidemia, and diabetes. age (e.g.. blood edemus in males of Africun PH whose history has been hyperesoteric (e.g., blood edemas in males of African PH whose history has been hyperesoteric a white male with diabetes has existing Hyperlipidemia history
Non-private
Failed
0.98
LapMech
Success
PurMech
Success
0.11
SPARSE
Success
0.62
Example 2: Protect disease name with weak noise
10
Ground truth
this male hus had known coronary discase and prior silent myocardiul infurction. male known
Non-private
Failed
coronary disease and has had prior myocardial infarction. male has known coronary myopathy. Silent rib syndrome, white-fiddled gyne, and ca male has known coronary myoputhy. Silent-fidged heart attacks. White-figged-fid an active male with myocurdial infirction, congestive heart diseuse.
LapMech
Failed
0.23
PurMech SPARSE
Failed
0.18
Success
K LIMITATIONS
Limited Scope of Attack Scenario. Our method explicitly tailored mitigate embedding inversion attacks, in which an adversary seeks to reconstruct input data from text embeddings. However, it does not offer guarantees against other widely studied privacy attacks such as membership inference attacks. Although our approach is compatible with differential privacy mechanisms in principle, leave the integration of comprehensive privacy protections to future work.
Protecting Broader Privacy Concept. Our framework estimates -rivaiv-sensitive dimensions based on predefined concepts, which works well for targeted protection but might not scale well with broader abstract notions of privacy. As the definition of privacy becomes overly broad (e.g., "any identifiable content"), ), our method loses its specificity and utility. A potential solution is to move toward concept-agnostic sensitivity estimation regardless of predefined labels.
23

---

Published as a conference paper at ICLR 2026
L USE OF LARGE LANGUAGE MODELS (LLMS)
In this work, large language models (LLMs) were used in two ways. First, we employed pre-trained open-source LLMs as embedding generators to produce text representations, and also served as the foundation for conducting inversion attacks in our experiments. Second, an LLM-based assistant (OpenAI GPT-4) was used to improve the clarity and readability of the manuscript through grammar checking and minor language refinements. All decisions regarding research design, experimental setup, analysis, and interpretation were made solely by the authors.
24


Reviews:
[{"review_id": "JpwKeHOcxJ", "reviewer": "Reviewer_hdGN", "summary": "Text embeddings used in systems like RAG can be inverted to recover sensitive attributes or content. Standard DP defenses add isotropic (“spherical”) noise to every dimension, which protects privacy but harms downstream utility and cannot target user-specified sensitive concepts. The paper proposes SPARSE, a concept-aware defense with two parts:\n\n1. Neuron Mask Learning. Using contrastive pairs that do/do not contain a user-defined sensitive concept, it learns a sparse, differentiable mask over embedding dimensions so that “privacy-relevant” directions are identified.\n\n2. Mahalanobis (Elliptical) Noise. Under a metric local-DP formulation, it injects anisotropic noise shaped by the learned mask (larger noise on sensitive dimensions, smaller on others), implemented as a generalized Laplace mechanism with a diagonal covariance.", "questions": "Above", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written and well-structured.\n2. The problem and methods are well-defined, especially dataset construction and learning objectives in mask learning.\n3. The results look promising. SPARSE substantially reduces leakage at the same privacy budget while preserving/improving downstream accuracy compared to spherical-noise baselines across semantic similarity and retrieval tasks (e.g., STS12, FIQA) and against multiple inversion attacks (Vec2Text, GEIA, MLC. On clinical text (MIMIC-III), concept leakage (e.g., gender) drops sharply, and a white-box upper-bound variant shows the black-box method performs near that ceiling.", "weaknesses": "1. The training cost should be clarified/ compared to the baseline since the proposed method involves the additional mask learning to identify privacy sensitive dimensions.\n\n2. The code was not released.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5Vka5NLPtn", "reviewer": "Reviewer_mmz1", "summary": "This paper proposes SPARSE, a concept-aware differentially private method to protect the text embeddings from inversion attacks. SPARSE selectively perturbs privacy-sensitive dimensions to apply stronger protection on protected concept. Experiments show that SPARSE has superior privacy protection and downstream performance over standard DP methods.", "questions": "- How to map each PII to the concept? Does each distinct PII corresponds to a single concept?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Obfuscating sensitive concept in embeddings is a non-trivial problem, and this paper innovatively applies dimension masking and Mahalanobis mechanism to address this challenge.\n- The LDP of Mahalanobis Norm can be connected with Generalized Laplace Mechanism.\n- The authors conducted comprehensive experiments with promising results.", "weaknesses": "- The frameworks assumes that sensitive concept are correlated with the embedding dimension, while this might not be the case. The related dimension for each concept could change depending on the context.\n- SPARSE relies on a pre-defined concept vocabulary and their corresponding masks. There could be emerging new concept in real-world, making it computation intensive to retrain the model.\n- In experiment, the authors use NER to extract sensitive information, which is limited. More complex privacy concepts should be considered.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OXKblIcttK", "reviewer": "Reviewer_TqWt", "summary": "The paper studies privacy risks of text embeddings under inversion attacks and proposes SPARSE, a concept-aware defense. SPARSE first learns a differentiable neuron mask to identify embedding dimensions sensitive to a user-defined privacy concept, using a hard-concrete relaxation with an sparsity regularizer. It then injects elliptical noise via a Mahalanobis mechanism calibrated by the learned per-dimension sensitivities, claiming metric-LDP guarantees. Experiments show improved privacy–utility trade-offs over two DP baselines.", "questions": "- Your target is preventing inference of concept tokens $C$, but the guarantee is metric-LDP in embedding space. Can you articulate a formal bridge (even approximate) from 𝜀-$||·||_M$-LDP to bounded leakage of $C$ under a class of attackers?\n- How do you ensure that the classifier distinguishing $D^+$ vs $D^-$ is not exploiting grammatical breaks or topic drift created by token removal $R(s,C)$? Any controls?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Embedding inversion is relevant to deployed retrieval/RAG systems; aligning protection to user-specified concepts reflects realistic privacy needs beyond coarse PII assumptions.\n- The combination of concept-conditioned sparse dimension selection and anisotropic perturbation is a step beyond spherical Laplace noise.\n- SPARSE shows consistently lower leakage at comparable or better downstream metrics relative to baselines.", "weaknesses": "- The pipeline “user-defined $C$ → NER to extract tokens” inherits false positives/negatives and domain coverage limitations. The paper instantiates $C$ mostly with NER/PII tokens and acknowledges extensibility but does not quantify failure modes or robustness to imperfect concept detection\n- Negative samples are built by removing tokens in $C$. This can alter syntax and semantics beyond the concept, potentially making the discrimination task easier in ways not strictly tied to $C$. The classifier may pick up distributional artifacts rather than pure concept-related differences, biasing the learned mask. No controls (e.g., semantically-preserving paraphrases) are discussed.\n- May need more baselines like Gaussian mechanism and SOTA methods like truncated Laplacian mechanism.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hi5Ch2EbSg", "reviewer": "Reviewer_ZNu6", "summary": "The paper investigates text embedding privacy and proposes a novel differential privacy–based mechanism, called SPARSE, to defend against embedding inversion attacks. Unlike previous approaches, SPARSE does not apply isotropic noise; instead, it first identifies the embedding dimensions responsible for specific privacy attributes. SPARSE then applies a Mahalanobis mechanism to add noise to these dimensions, leaving the non-sensitive dimensions untouched to preserve downstream performance. The method is empirically evaluated on multiple datasets against two other defense methods and three embedding inversion attacks. A qualitative analysis of the privacy-sensitive domain and a white-box setting concludes the paper.", "questions": "- Could the method, in principle, also be applied to LLM prompt extraction attacks, such as in “Extracting Prompts by Inverting LLM Outputs” (Zhang et al.)?\n- How long does fitting the binary masks approximately take?\n- How many attributes can the method defend against simultaneously before utility is completely compromised? For example, if we aim to prevent leakage not only of age but also of gender, ethnicity, illness, political opinion, etc., at the same time.\n- L210: Is removing the private words in the negative sample set the best strategy? Why not replace the attributes with alternative values and check which dimensions change? I would expect some biases due to different input lengths of samples in the two datasets.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-motivated and addresses an important privacy-critical topic (embedding privacy). All steps are clearly described and easy to follow. Additionally, all experimental settings are thoroughly detailed, supporting reproducibility.\n- The experiments demonstrate a clear advantage of SPARSE compared to related defense methods. Since the evaluation is performed on multiple datasets, the results appear reliable.\n- Additional analyses of white-box attacks provide a deeper understanding of the method’s capabilities and support the assumptions made earlier in the paper. Similarly, the qualitative analysis of the identified privacy dimensions offers valuable insights.", "weaknesses": "- The paper lacks a thorough discussion of the method’s limitations. It remains unclear how dependent the method is on high-quality training data. Additionally, there is no discussion of the time required to fit a separate mask per user (or privacy setting).\n- While the experimental results demonstrate an improvement over existing methods, a significant gap remains between research and practical application. For instance, an epsilon of 5 provides strong privacy but reduces utility by half. This trade-off improves for higher epsilon values, but one may still have to accept a leakage of 35%–50% to preserve utility, requiring an epsilon of 20.\n- There is no analysis of how the method affects the representation of related concepts. For example, if SPARSE protects a concept like gender, how does this influence related concepts such as appearance descriptions or commonly gender-biased attributes?\n\nMinor remarks:\n- There is a missing space in L39: \"T5-based embeddings.Such\"\n- L207: \"We\" should probably be capitalized.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "J05MrztnxX", "reviewer": "Reviewer_vVqF", "summary": "This paper presents SPARSE, a user-centric framework for concept-specific privacy protection in text embeddings. SPARSE integrates two key components: (1) the identification of privacy-sensitive dimensions based on user-defined concepts, and (2) a Mahalanobis noise mechanism that injects elliptical noise calibrated to each dimension’s sensitivity. Through comprehensive evaluations on six datasets using three embedding models and various attack scenarios, SPARSE demonstrates significant reductions in privacy leakage while maintaining or improving downstream task performance, outperforming existing differential privacy (DP) methods.", "questions": "The following questions can be discussed further:\n\n1) How sensitive is SPARSE to errors in concept labeling, and what is the impact of such errors on privacy and utility? Is there any error propagation through the masking and noise injection stages?\n\n2) How well does the method generalize to out-of-domain data, especially when user-defined concepts do not transfer or are unavailable?\n\n3) Can SPARSE operate in zero-shot or few-shot task transfer settings settings without explicit concept annotations? If so, how is privacy managed in the absence of concept-level guidance?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper addresses a critical problem in privacy-preserving NLP with a novel, user-centric framework SPARSE that enables concept-specific protection via dimension-sensitive elliptical noise. The approach offers a more personalized and ethical alternative to traditional DP methods. The empirical evaluation is thorough and convincing, covering six datasets, multiple embedding models, and diverse attack scenarios. SPARSE consistently outperforms baselines in both privacy protection and downstream utility, a notable achievement.", "weaknesses": "The paper can be improved from the following aspects:\n\n1) Lack of Formal Privacy Guarantees: The method does not provide formal differential privacy bounds (e.g., ε, δ), which limits its comparability to standard DP approaches and hinders rigorous privacy-utility analysis.\n\n2) User Annotation Burden: Since SPARSE relies on user-defined concept privacy, the paper would benefit from a discussion or empirical analysis of the feasibility and burden of obtaining these annotations in practice.\n\n3) Limited Applicability Across Domains: The approach assumes users can identify and define privacy-sensitive concepts, which may not hold in all contexts. This assumption could restrict the method’s usability in domains where privacy concerns are implicit or unclear.\n\n4) Computational Overhead: There is no analysis of the runtime or resource cost introduced by SPARSE. Understanding its computational impact would be important, especially for real-time or large-scale deployment scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)