Theoretically Unmasking Inference Attacks Against LDP-Protected Clients in Federated Vision Models
Authors: Quan Minh Nguyen
Minh N. Vu
Truc Nguyen
My T. Thai
Venue: ICML 2025 poster (2025)
Keywords: Federated Learning
Differential Privacy
Membership Inference Attack
Primary area: social_aspects->privacy
TL;DR: In federated learning with local differential privacy (LDP), we find that membership inference risks persist, as our theoretical analysis and practical evaluations reveal privacy concerns.

Abstract: Federated Learning (FL) enables collaborative learning among clients via a coordinating server while avoiding direct data sharing, offering a perceived solution to preserve privacy. However, recent studies on Membership Inference Attacks (MIAs) have challenged this notion, showing high success rates against unprotected training data. While local differential privacy (LDP) is widely regarded as a gold standard for privacy protection in data analysis, most studies on MIAs either neglect LDP or fail to provide theoretical guarantees for attack success against LDP-protected data.
    
To address this gap, we derive theoretical lower bounds for the success rates of low-polynomial-time MIAs that exploit vulnerabilities in fully connected or self-attention layers, regardless of the LDP mechanism used. We establish that even when data are protected by LDP, privacy risks persist, depending on the privacy budget. Practical evaluations on models like ResNet and Vision Transformer confirm considerable privacy risks, revealing that the noise required to mitigate these attacks significantly degrades models' utility.

Theoretically Unmasking Inference Attacks Against LDP-Protected Clients in Federated Vision Models
Quan Nguyen 11
Minh N. Vu 22 True Nguyen 3
My T. Thai
Abstract
weights) can still reveal sensitive information about the training data (Fang et al., 2024; Yang et al., 2023). A promi- nent attack against FL is the membership inference attack (MIAs) (Shokri et al., 2017; Hu et al., 2022b) in which the server seeks to determine whether a particular record was part of the model's training dataset.
Federated Learning enables collaborative Iearn- ing among clients via a coordinating server while avoiding direct data sharing, offering a perceived solution to preserve privacy. However, recent studies on Membership Inference Attacks (MIAs) have challenged this notion, showing high success rates against unprotected training data. While lo- cal differential privacy (LDP) is widely regarded a gold standard for privacy protection in data analysis, most studies MIAs either neglect LDP or fail to provide theoretical guarantees for attack success rates against LDP-protected data.
To mitigate such privacy risks, local differential privacy (LDP) (Dwork et al., 2014; Wang et al., 2020) emerged as prominent solution to limit the privacy leakage of local training data. Under LDP, individuals perturb their data locally before sharing it with the server, thereby eliminating the need for a trusted centralized curator. However, recent studies (Nguyen et al., 2023; Chen et al., 2021) demon- strated that tackling active membership inference altacks (AMI) conducted by dishonest FL servers requires adding large privacy-preservic utility. In AMI, the server actively poisons the global model before distributing it to clients, enabling them to infer pri- vate information. Such attacks typically require multiple training iterations or the use of shadow models, resulting in non-trivial time complexity, and provide no theoretical privacy risks FL.. Building on this, (Vu al., 2024) pro- posed a www-poynnomal-tiine attack, presenting two active membership inference attacks on LLMs with guaranteed theoretical success rates on unprotected data.
To address this gap, derive theoretical lower bounds for the success rates of low-polynen pow-pooonomia---- time MIAs that exploit vulnerabilities in fully con- nected or self-attention layers. We establish that even when data are protected by LDP, privacy risks persist, depending on the privacy budget. Practical evaluations on federated vision models confirm considerable privacy risks, revealing that the noise required to mitigate these attacks signif- icantly degrades models' utility.
noise that would also damage FL
1. Introduction
Despite these advancements, most existing works rely heav- ily on empirical validation and lack robust theoretical foun- dation or guarantees for attack success rates, especially un- der LDP. This challenge arises from the randomness of noise introduced by privacy mechanisms, which varies across it- erations and clients, making it challenging to analyze the effectiveness of attacks in theoretical framework.
Federated Learning (FL) (McMahan et al., 2017) is a decen- tralized machine learning paradigm where multiple devices or nodes (e.g., smartphones, edge devices, or distributed servers) collaboratively train shared model while keep- ing their data localized. Due to this property, it has long been heralded as a robust solution for privacy-preserving machine learning. However, FL itself does not provide for- mal privacy guarantees, as the model updates (gradients
This paper takes a step back and establishes a broader view of the principle of privacy risk imposed by dishonest servers in federated vision models under LDP from both theoretical and practical perspectives. Our study focuses on the ac- tive adversary setting, where the FL server acts dishonestly by manipulating the trainable weights of a vision model to breach privacy. Specifically, we aim to demonstrate that clients' data, even under LDP protection, are fundamentally vulnerable AMI attacks carried out by dishonest servers. For that purpose, we analyze attacks that exploit the train-
'Equal contribution 'Department of CISE, University of Florida. USA. *Center for Nonlinear Studies, Los Alamos Na- tional Lab, USA 'Computational Science Center, National Renew- able Energy Laboratory, USA. Correspondence to: My Thai <mythai www.eiise......... www.eeiics.s...........
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
able fully connected (FC) layers and self-attention layers in FL updates as both are widely adopted in federated vision models. Our main contributions are summarized as follows:
where is the privacy budget, and Range(M) denotes all possible outputs of M. In FL.-LDP, controls the privacy- utility trade-off: smaller enhances privacy by introducing more noise to the data, but this can reduce the utility of the aggregated updates for the global model. While there are other secure aggregation using multi-party computation (SMPC) or homomorphic encryption (Bonawitz et al., 2017; Nguyen et al., 2023), these are orthogonal research directions and are discussed separately in Appendix. I.
We derive theoretical lower and upper bounds (Theo- rem and 2) on the success rates of a -ow-polyooiial- time attack (Vu et al., 2024) that exploits vulnerabilities in FC layers, showing that privacy risks persist under LDP protection depending on the privacy budget.
techniques for FL, such as
For transformer-based vision models such as ViTs (Dosovitskiy et al., 2021), extend the attack on LLMs in (Vu et al., 2024) to continuous domain and de- rive theoretical lower bounds on the vulnerability of the self-attention mechanisms against a low- low-polynomial- time attack that exploit's the layer's memorization mechanism. (Theorem 3).
Federated Foundation Models via Parameter-Ffficient Fine-Tuning (PEFT). PEFT methods adapt large pre- trained models to specific tasks by modifying small subset of parameters, reducing computational and storage costs. Key approaches include LoRA (Hu et al., 2022a), which adds trainable low-rank matrices; Adapter Modules (Yin et al., 2023), which insert lightweight layers: BitFit (Zaken et al., 2022), which updates bias terms; and Prompt Tuning (Lester et al., 2021), which optimizes input embeddings. In our work, we specifically analyze scenarios where trainable layers are fully connected or self-attention layers.
Our experimental results in real-world state-of-the-art vision models such as ViTs and ResNet (He et al., 2016) demonstrate that AMI attacks achieve notably high success rates observed even under stringent LDP protection (i.e., small privacy budgets <) that consid- erably degrade the model's utility (Section 5). Fur- thermore, we consider both traditional FL where clients train a small model (ResNet) and the parameter- efficient-fine tine-tuning paradigm, where clients often uti- lize large foundation models for pre-training and fine- tune only some layers or parameters (ViTs).
Membership Inference Attacks (MIAs) in FL. MIAS in FL aim to identify if a specific data point was part of a client's training set. Although FL keeps data local, model updates exchanged between clients and the server can still leak information. Passive attacks (Shokri et al., 2017: Zhang et al., 2020) involve an honest-but-curicn ing the model updates, while Active Membership Infer- ence (AMI) attacks involve a dishonest server poisoning the global models, e.g., maliciously modifying model pa- rameters, before dispatching them to clients. The first AMI attack in FL was introduced by (Nasr et al., 2019), relying multiple FL iterations. A stronger, single-iteration AMI attack requiring training a separate neural network was later proposed by (Nguyen et al., 2023). Both approaches have non-trivial time complexity and do not establish theoretical privacy risks in FL. Recently, (Vu et al., 2024) introduced two AMI attacks that exploit fully connected and attention layers in L.MMs, achieving high success rate in compro- mising membership information of unprotected client data.
server observ-
2. Background and Related Works
Federated Learning with Local Differential Privacy (FL- LDP). In Federated Learning (FL.) (McMahan et al., 2017), central server orchestrates training while clients store data locally. The server initializes the model parameters 0, and in each training iteration, a subset of clients com- putes gradients of the loss function L on local data D, i.c., 0 = (D). These gradients are aggregated and sent to the server, which updates the parameters accordingly. Training continues until convergence.
To protect the privacy leakage in FL, Local Differential Privacy (LDP) (Dwork, 2006; Erlingsson et al., 2014) has been introduced. LDP is a that mitigates risks by perturbing individual data before it leaves the client's device in FI.. IDPP ensures that the server cannot infer sensitive client information directly from the shared data as defined below.
The primary focus of our study is to demonstrate the ex- istence of low-complexity adversaries with provably high attack success rates, particularly when the data is protected by any ideal LDP mechanism. Since our research aims to as- sess the resilience of privacy-preserving techniques against real-world adversarial threats, we chose to examine two state-of of-tthe-rt AMI attacks on the FC and Attention layers proposed by (Vu et al., 2024), which have low-polynomial time complexity. These attacks allow the server to exploit FC layers to perfectly infer membership information (The- orem in (Vu et al., 2024)) and to exploit self attention layers to achieve a similarly high success rate (Theorem
mechanism
Definition 1. E-LDP. A randomized mechanism M satis- fies s-LDP if, for any two inputs x and x' and all possible outputs O E Range(MM.
Pr[M(x) = 0) s Pr(M('') =O],
2

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
in (Vu et al., 2024)). However, their theoretical analysis is only applicable to the non-LDP setring.
Fo := max - ||X -il,, 0}. X T, 20 activates, and the gradient of by [1] is non-zero. For b2 by = -D > 0 small enough, 20 0 for X + T,, leaving the gradient zero. The adversary uses the gradient of by[1] as an indicator of the presence of 7 in the local data. A non-zero gradient implies T exists, while a zero gradient indicates it does not. The FC attack AFc does not need any distributional information to work on unprotected data. In fact, the attacker just needs to specify 7P (2) small enough such that 7D < ||x1 - X2llz,, for any X1 o X2 in the model's dictionary. Since the dictionary or the pre-trained feature extractor is public, se- lecting 7D does not require any additional information. The description of ARC is given in Appendix C.I.
3. AMI Attacks
3.1. AMI threat models
The AMI threat models under LDP are formalized through the security games EXpΩMP. as described in Fig. 1, following standard security frameworks (Nguyen et al., 2023; Vu et al., 2024). Further details about the security games can be found in Appendix B. In ExpLDP. AMI the adversarial server A (super- script D indicates that the server knows the data distribution of the client's 's private data) comprises three components: ABRIT- ARTTACK. determines if a target sample T is in the client's data D. Each client applies LDP to perturb their data, generating D' (D) = {M (X)]XED. The server's ANNTT se- lects model T and ARTTACK crafts parameters 0 using T. Clients compute gradients Ó = VgLg(D') and send them back. With 0, AlUESS infers b, effectively identifying whether T D. The advantage of the adversarial server A in the security game is given by:
TACK- and AQUESS. In Exp a random bit
3.3. Attention-based AMI adversary
The proposed A (Vu et al., 2024) leverages the mem- orization capability of self-attention, a property that was indirectly explored in (Ramsauer et al., 2021). That study demonstrates that self-attention can be interpreted as equiv- alent to the Hopfield layer, which is specifically designed to integrate memorization directly within the layer. Building on this perspective, A employs a tailored configuration of attention to facilitate the memorization of local training data while selectively excluding the target of inference.
Adv AMI LDP
(AP) 2
AMI (AP) =1] -1
(1)
-Pr 1|6 = 1] + Pr[b'- = 0/6 = 0] -
The dataset represented as D = {a:: )_=,, where J. E X, X' S xXx, and each column xy E 1adx is referred to as pattem. Since Altn operates at a pattern level, the target of inference is the pattern derived from embedding the target T, denoted as e E RRdx The underlying intuition of this approach involves configuring an attention head to memorize the input batch while excluding the target pattern. This configuration introduces a measurable discrepancy be- tween the output of the filtered attention head and that of a non-filtered head. The resulting gap can then be exploited to infer the victim's 's data. A detailed explanation of the com- ponents of this attack are further elaborated in Appx. C.2 or readers can refer to (Vu et al., 2024) for the full description of the attack.
where 1 PT = 1|b - 1 Pr 0lb 0] denotes the success rate of the attack. The existence of an adversary with a high advantage implies a high privacy risk/vulnerability of the protocol described in the security game.
3.2. FC-based AMI adversary
We analyze the FC-based AMI adversary first introduced in (Vu et al., 2024). In that paper, they proved the existence of an AMI adversary that exploits two FC layers to achieve a perfect membership inference success rate for unprotected data. The FC-based adversary APc is designed to detect a target sample T with dimension dr within local training data D during FL training. For analysis of FC-based ad- versary, we represent the dataset as D= = | where X, E X, and X' C Rdx, The model & introduces two adversarial fully connected (FC) layers. The first layer has weights W1 of size 2dr x dr and biases by of size 2dy, structured to encode the target T. The second layer outputs single neuron, with weights :2[ and bias by [1] set to target the presence of T. These parameters are defined as:
3.4. Attention-based AMI attack against Vision Transformer
While (Vu et al., 2024) focuses on exploiting the attention layer in LLMs to infer the presence of a pattern in a dataset, we further extend this attack to the continuous image do- main, where we exploit the attention layer used in ViTs. We formulate the attack AA' Attn in the context of attacking VIT models as follow: Given image I E I XXXX the image is divided into L non-overlapping patches (see Fig. 14). The patches are flattened into vectors, and projected (3) into an embedding space using a linear projection matrix W embeed- The result of this embedding layer is:
Tdr - Larr
T b
Wi+ +-
(2)
T
W3 [1 <- -1 -1T d
b2[1] 7
where D controls the allowable Lddistance between inputs and T. Upon receiving input X, the two FC layers compute
33 (1, Wembed + Pj: j = 1,.... L
3

---

Theoretically Unmasking Inference Attack Against LDP-Protccted Clients in Federated Vision Models
A INIT
(ii)
A
AATTACK (T) (iii)
(v)
Server
A AovES (T, (T,0)
b'
If b' b, A wins.
Client
TED
Flip a b
= (M)
0 (iv)
TED
Figure 1. Active inference security game under LDP: a random bit determines the state the data (i), the server A specifies (ii) and 9 (iii), gradients on LDP- ppottcceet data D' are sent back (iv), and A guesses b (v).
represents the j-th patch of the im- age and Pj is the corresponding positional encoding. The resulting set of embeddings {ry}iim is then passed through the Vision Transformer (ViT) architecture, where the at- tention mechanism operates on these embeddings. When incorporating LDP noise into ViTs, apply the noise di- rectly to these embeddings before they enter the attention layers. We employ a similar attack strategy to the one used in the rnsention-based AMI adversary, but in the context of the ViT's embeddings. Implementation details of the attack are given in Appx. G.2.
where 1 E R VE* xC
ExpΩ defined in Section 3.1, we generalize the lower bound and upper bound for the advantage of the adversar- ial server AFc in Theorem and Theorem 2, respectively, Finally, we provide the lower bound for the advantage of A under LDP in Theorem 3.
4.1. FC-based AMI on LDP-Protected Data
We now theoretically show that the adversary AFc con- structed in Subsect. 3.2 can also be used to expose the true privacy risks of LDP-protected data w.r.t AMI in FL and state it in Theorem First, we need to lay out some as- sumptions on the data and the LDP mechanisms.
The data X is assumed to be from discrete alphabet X, in which the Lı metric is well-defined. In our anal- ysis, we denote X' as the set of possible output values of the LDP algorithm (see remark 2). We denote A = minx,yex ||x 1l,z,/2 Note that A is a statistic of D and is known by the server. We denote B (X, A to be ball of radius A centering around in the L1 norm. Given an LDP mechanism M with budget applied on an alphabet X, PM denotes the probability that the protected version of point is not in the ball of radius A centering at that point: PM = Pr [MF(X) & B, (X, )*) Intu- itively, a smaller E would impose more LDP noise, resulting in a larger PM
(T A Mx()
X
Discrete- Alphabet X
(a) The protected version of the target M jumps of Bi (T, ))
(T. A
AMI Theorem Given the security game EXPLDP there exists AMI adversary APe whose time complexity is Oldx) AMI such that Adv
(
n+|X111 Pyrr, where n [21-1
2
is the size of the dataset D, |X is the cardinality of the possible output values of the LDP ADP-mechanism and PM is the probability that the -mehhancnn makes the protected version of data point inside the neighborhood of another data point. (Proof in Appx. D.I)
Discrete Alphabet X
(b) # T such that M (T,
Figure 2. Scenarios when APc fail.
We APC constructed in Subsect. 3.2 with 7D X to show Theorem 1. Given input X, target T and LDP mechanism M the goal of APC is to configure the first two FC layers so that the first row of the second layer computes 20 = max(7P - IIM - Til,,, 03. If (X) - TI P then 2n activates and the gradient of bz(1]
4. Privacy Leakage Analysis
This section presents our theoretical analysis for assessing the risk of leaking membership information of users' local training data in FL. under LDP. Given the security game

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
non-zero. For 7P Ax, if (x() (T, A²), then 20 0. leaving the gradient zero. Conversely, if M E B, (T, then 2g > 0 and the gradient ((b [1]) > 0. The adversary uses the gradient of by [1] as an indicator of the presence of T in the local data. A non-zero gradient implies T exists in the data, while a zero gradient indicates it does not. Intuitively, APC fails if either (i) the protected version of the data M (X) jumps out of B1 (T, A*) when X T or (ii) there is an X + T such that MM (X) E B1 (T, )²). These scenarios are illustrated in Fig. 2. The probabilities of
(Proof in Appx. F)
This theorem shows the theoretical upper bound for AMI Adv given the privacy budget e. Accordingly, we measure the adversary's attack success rate as 2 + Adv (AFc)) (sce Equation 1) and plot the theoretical up- per bound alongside the lower bound of AMI attack success rate and empirical success rate against real-world datasets under LDP protection in Figs. 7 and 8.
the two events are bounded by PM and nPM (PPmm/((X]] ([xx - 1), respectively (see Appx. D.I.. Theorem demonstrates the trade-off between privacy and data utility: a highly protected data would have high PM=, thus lowering the advantage of the adversary; however, its distortion from the original data large as a result.
4.2. Attention-based AMI on LDP-Protected Data
Our theoretical result on the vulnerability of private data to Attention based AMI attack under LDP is quantified on Separation of Patterns, an intrinsic measure of data:
Definition 2. (Separation of Patterns (Ramsauer al., 2021)). For a patter z; in a data point X {ay 1 Nx its separation A: from X is A; = ming, (z. x4 - x xz))
Remark 1. Lower bound of Theorem 1. It is non-trivial to obtain PM' for Theorem due to the dependencies on the data as well as the specific LDP mechanisms. To demon- strate the intuition behind the proof, we provide an example of how to derive PM- for Generalized Random Response (GRR (Wanner, 1965)), a classical LDP algorithm, and the corresponding theoretical lower bound for GRR-protected data in Theorem 4 (Appx. E). We also simulate the lower theoretical bound in Theorem for data protected by LDP algorithms BitRand (Jiang et al., 2022), GRR. dBitFlipPM (Ding et al., 2017) and RAPPOR (Erlingsson et al., 2014) in Figs. and Those theoretical lower bounds are shown along with the success rates of some AMI attacks (discussed in Section 3) for comparison.
x x; max, x..2z. We
say the pattern i is separated from the data point X if A > O. We say X is A A-separated if A, 2 A for all i E (1...., Nx). A data D is --separated if all X in D are Aseeprrated.
For the attention-based attack, we represent the victim's dataset as D {x.)== where X; E X, and X C RdxxNx, For any 2-dimensional array X, each column zj E RRd is referred to as pattern Since the LDP mechanisms are generally applied at pattern level in 2- dimensional data (Qu et al., 2021; Yue et al., 2021), the distortion imposed by LDP is modeled by a noise r, added to each pattern: X² (X) = {z, + + We assume r; is bounded by a norm budget R that is de- fined by specific mechanisms and applications. M de- notes the maximum L2-norm of all patterns, defined as M = maxxeD ED maX ||æy|| The adjusted pattern's norm is then upper-boundee by M v M² + R Re- garding data separation under LDP, denoted by Ae, whose value is not easily obtainable even when the LDP mech- anism is known, we can generally expect A 2 A. The reason is, the noise r; is independent of the patterns, it makes the patterns less aligned. This intuition is demon- strated via example in Appx. D.3.
Remark 2. Cardinality of X. |x is dependent on the specific LDP algorithm used. For binary LDP algorithms such as Binary Randomized Response (Warner, 1965) or RAPPOR (Erlingsson et al., 2014), X due to the bi- nary nature of the values being perturbed (e.g., "yes/no" or "0/1" response). In contrast, for generalized k-ary random- ized response algorithms, such as Generalized Randomized Response (GRR) or K-RAPPOR, | = k; where h > 2 represents the cardinality of the set of possible output values of the algorithm. This allows for more nuanced privacy- preserving mechanisms, where the response set consists of k different values, each with specific probability distribu- tion determined by the privacy parameters of the algorithm. In modern bit- hit-fiiping algorithms such as OME or BitRand, the original data or embedding features are first converted into binary vectors of size b. The LDP mechanisms are then applied on top of those binary representations of the signal by flipping random bits. In this case, |x|| = 2h. For large enough b, 2+12'1 -1 R 1 and AdvAMI/ (APc) ~ 1 - PM-- Theorem 2. For all AMI adversary A of the security game AMI
Nx.
The notion of A -separated helps capture the intrinsic diffi- culty of the data for the inference task: the less separating the data, i.e., a smaller A the harder for the adversary to detect the patterns. However, is not beneficial to impose a low separation on the data in practice since it would impair the model's performance. Note that, if D is considered as the data after preprocessing, A can be manipulated by the choice of preprocessing methods for the FL model, which are often specified by the server. We are now ready to state Theorem 3 that analyzes the vulnerability of LDP-protected data to iiention-based AMI in Fl..
Exp(M, we have
Adv
AMI (AP2)
<
E
(4)
E c++1
5

---

Theoretically Unmasking Inference Attack Against LDP -.o--Prteeted Clients in Federated Vision Models
X 2
Attention (Filtered)
X
||0|| > 0
v)
X 2. (ry + v(vi))
Attention 2 (Non-filtered)
12 2 <>
1101l = 0
Figure 3. The adversarial server exploits self-attention mechanism conduct inference attack of victim's protected local training data D' in FL: If z, in the data equals to the target pattern the input to the filtered attention heads is the pertubed version z and the output zi of the filtered head close to the protected pattern's average X instead of at. This creates non-zero gradients on weights computing on the difference of attention heads' outputs. The attack fails when the added noise is large and the embedding of protected data overlaps at the center the embedding.
Theorem 3. Given a A -separaaeed data DM (the LDP- protected version of the data D) with i..d patterns of the security game AMI for any 3 0 large enough such that
This probability governs the false-positive error of the at- tack.
If there exists x; (i.e., in the training dataset), the weights of attention head 1 filter o and output:
2
Z
+ log(2(NX 1) (5)
BNx
X 10fftmax(3x** (r; (3--F] -))),
there exists AMI adversary. A that exploits the self- attention layer with time complexity of O (ddx) such that AdviDp((A Att)) ) is lower bounded by:
where F. is the projection of onto v. If R is small enough, of se X By computing the difference between the two heads, |2] - -27l, the adversary can infer the presence of in X. For a hyperparameter ? if |2] z 7, then E X. Conversely, if | = y then v & X. We select y = 2 a choice justified in Appendix D.4.
AdvAMI
AMI (AATT)) 2 proj
DDDMe
2nNx
+DDM
3NxM M
False-negative errors occur when the embedding of pro- tected data overlaps at the center of the embedding space, causing | - = < 7 even when v E X. By using the mean value theorem and bounding the Jacobian of the attention's forwarding function, this error is upper bounded by:
DDMc
(35= 3(mmmax) 121E) -
(6)
where A = 22 exp (2/Nx ) and DM is the distribution of the protected data DM induced by the original data distribution D and the LDP-mechanism M.. m = Nx NNx xa is the arithmetic mean of all LDP-protected patterns and mmmax maxigigNx ||x;
P.D-M (3 + B(1r 12
m Here, pmn (8) is the probability that the projected component between two independent patterns drawn from DM is smaller than and P.DM
As noise increases, the cube of size 6 + 2.3 yR covers more patterns and causes pD-M, to increase. At high noise levels, the attention outputs for all patterns are more likely cluster near the center of the embedding space, as illustrated in Fig. 4. This leads higher neggaive rates for the attack. However, models trained on such noisy data generally exhibit poor performance since patterns whose embeddings overlap in these central regions become indis- tinguishable. This results in PDMr ~ 1 for large enough R causing the advantage to drop sharply regardless of di- mensionality as the cube fully encloses the patterns. Our simulations Eq. (6) for one-hot and spherical data are shown in Fig. 5 and Fig. 6, respectively.
M (5) is the probability that a random pattern drawn from DM is in the cube of size 25 centering at the arithmetic mean of the patterns in DM,, (Proof in Appx. D.4)
The key step in proving Theorem 3 is to demonstrate that the configuration of the self-attention model, specified in Appendix C.2, behaves as outlined Fig. 3. Given any input pattern T; X, the attention heads process the LDP- protected version of the pattern, I; +r. Let denote the tar- get pattern, If v not present in the victim's 's training dataset, i... Ij # for all 1 < j < Nx, then, using Lemma (Appendix D.2), which builds on the Exponentially Small Retrieval Error Theorem for the attention layer (Ramsauer et al., 2021), we show that 2 ss H; + r; and z Rs x; + Ti both with a probability lower- bounded by:
Remark 3. A vs. the advantage's lower bound (6). A larger A allows smaller / to satisfy (5) and makes PDMe proj smaller.
|3N larger and PDDMa (3A= + (m )
MDM
1
Remark 4. The most vulnerable embedding. The
MN
lower bound (6) would be optimized with one-hot data.
6

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
dm
1.0
10
0.8
20
Token's embeedinn
Advantage
50
200
100
0.0
500
100C
1000
0.2
0.2
22000
25003
0.0
33007
Center of embedddnngs
0.0
0.02
L1 norm of A
0.04
0.05
0.08
0.001
0.002
0.004
35003
Figure 4. For large R s.t. Poox 1, the embedding of protected data overlaps the center of the em- bedding and impairs data utility.
Figure 5. Adv Monte-Carlo simulation
LDP (Rm)) on onc-hot data using
Figure 6. Adv (A ) on spherical data us- ing Monte-Carlo simulation
Since it has alignment among patterns, A achieves its maximum which is the pattern's norm. Furthermore,
the unit sphere). The real-world datasets, including CI- FARIO, CIFARIO0 (Krizhevsky et al., 2009), and Ima- geNet (Krizhevsky et al., 2012), are processed using pre- trained embedding modules to obtain data D for our threat models. We also use the ImageNet dataset, large-scale benchmark consisting of labeled images across 1,000 cate- gories (Deng et al., 2009). For ResNet, we extract feature embeddings with Img2Vec (Safka, 2021), while for ViTs, we use pretrained foundation models provided by the au- thors on HuggingFace (Dosovitskiy et al., 2021). We refer readers to Appx. G.I for more details.
PDA,,
proj (BNxM²) is because all patterns are orthogo-
nal to each other. Finally, since there is no pattern at the center of one-hot data, we can select very large / that P.DMM (3Ae + B(m RR) = 0, given R is small enough.
Remark 5. Asymptotic behavior of the advantage (6). For high dimensional data, i.e., dx - 0o, two random points are surely almost orthogonal (Pit)) [N
1), and a random point is almost always at the bound- ary (Blum et al., 2020). Therefore, PP (3Ar + (² - 0 for small enough R2 and B. This phe- nomenon can be seen in one-hot data (see Fig. 5). For spher- ical data, the amount of noise needed for PiD (3 (mfnna))2 + is smaller, and the cut-off noise's 's norm gradually decreases as the input dimension increases.
LDP mechanisms. We use BitRand (Jiang et al., 2022), GRR (Wamner, 1965), RAPPOR (Erlingsson et al., 2014), dBitFlipPM (Ding et al., 2017) as LDP mechanisms for real-world datasets. Details about these algorithms are in Appx. G.4. We also provide some results on OME (Lyu et al., 2020) in Appx. H.I.
Results on synthetic datasets. Fig. and Fig. 6 shows the impact of the LI norm of R on the advantage of Attention- based AMI adversary on one-hot and spherical data, respec- tively. For one-hot data, as we increase d random noise is almost always at the boundary and there is pattern at the center of one-hot data. This reduces the likelihood that the protected data's embedding overlaps with the center of the embeddings, increasing the lower bound of the adversary's 's advantage in (6). On the other hand, for spherical data, the advantage drops sharply when R increases, regardless of the dimension of the data.
Remark 6. Impact of 3. Increasing the pyper-parameter 3 in AR (Algo. 3, Appx. C.2) would raise the memorization of the attention layer (Ramsauer et al., 2021). (Vu et al., 2024) showed experimentally that increasing 3 leads to better adversarial success rates against unprotected data. For LDP protectte data, this is not the case, increasing also increases hoox (3 + 32 It). reducing the lower bound of Adv on the impact of 3 on Adv (AAR Attn) in Sect. 5.
LDP (AA We provide experiments AMI
5. Experiments
Results of FC-based AMI adversary. Figures 7 and 8 show the success rates of FC-based AMI attacks against 4 different LDP algorithms on CIFAR10 and CIFAR100. The theoretical lower and upper bound on the adversary's attack success rate can be derived directly from the theoreti- cal lower and upper bound of Adv AMI (APc) using Eq. 1. Across these LDP mechanisms, the amount of LDP noise needed to protect against AMI attack significantly reduces the model's 's utility. For example, for BitRand- BitRand-protected CI- FAR10, to make the inference rate lower than 80% (yellow
This section demonstrates the practical risks of leaking pri- vate data in FL. In particular, we implement the FC-based APC and attention-based AD adversaries, and evaluate their success rates in synthetic and real-world datasets. Im- plementation details are given in Appendix. G
Datasets and embedding. Our experiments use two syn- thetic and three real-world datasets. The synthetic datasets
include one-hot encoded data and spherical data (points
7

---

Theoretically Unmasking Inference Attack Against LDP- -.D--eroeeeed Clients in Federated Vision Models
CHA110 Befand
CPPRLO ORR
CPARLO FAPPOR
10
1.0
Allack Supccsa Fulte
Sapcess Flute
Model recurmity
Altack Model Accures marr
0-3
Attck
0.3
PCAM
PC AM
Modil anowery
10
Priracy Sucgel
Privayy Sulget.
Prinacy Dudget.
vacy Budget.
(b)
(c)
(d)
Figure? 7. Thcoretical upper/lower bound and empirical results the attack success rates of FC-based AMI adversaries against CIFAR1O dataset protected by BitRand (a), GRR (b), RAPPOR (c) and dBitFlipPM (d).
CIARLLO Bthand
CIRAR100 GAR
CHARLL RAPPOR
CIRARLOO diiiigee
10
3.0
1.0
1.0
voddl NCt NCtunny
Black Succisa Rula
Ataacc phadel AcCuEacy
Suctama Taatee
0.9
0.7
Shate
0.5
PC:40
0.-
0.2
0.4
10
0.4
10
0.4
Privacy Burget
Privay Budget
Piwacy Budget e
Priacy Budget
(b)
(c)
(d)
Figure& 8. Theoretical upper/lower bound and empirical results on the attack success rates of FC-based AMI adversaries against CIFARI00 dataset protected by BitRand (a), GRR (b), RAPPOR (c) and dBitFlipPM (d).
line), the model has to suffer at least 20% accuracy loss. The theoretical lower bound of the attack's 's success rate corrobo- rates the empirical success rate of = 100% when c = For both real-world datasets, AFC achieves near 100% success rate when approaches 6. Among the evaluated mecha- nisms, GRR, RAPPOR, and dBitFlipPM preserve higher model accuracy at lower E values but expose the model to greater privacy risks, as indicated by both the empirical and theoretical attack success rates approaching 100% at = and 6, respectively.
client's 's data distribution (as outlined in the AMI threat mod- els in Section 3.1), the server can simulate the client's data to compute a minimally sufficient value for f. When do- ing experiments, we found that setting 3 to reasonably small value (e.g.. 0.01) yielded consistently good results across realistic values and datasets/LDP mechanisms. As illustrated in Figure 10, for L.DP-protecter data, / = 0.01 generally achieves better success rates under small s.
Empirical results on NLP datasets. In section 4.2, the distortion imposed by LDP is modeled by noise r; added to each pattern: X M (X) {are + r.), In our analysis, assume x; and r; to be continuous, and the impact of LDP noise can be visualized in Fig. 4. For NLP data, both the data and the noise should be modeled as discrete, hence our theoretical analysis might not directly apply to the NLP scenario. The key challenge is that in NLP, tokens are typically represented as discrete embed- dings, and adding continuous noise is not meaningful in this context. Therefore, a separate theoretical framework would be required to account for the discrete nature of NLP data.
Results of Attention-based AMI adversary. Experiments on CIFARIO and ImageNet (1000 classes) are conducted on ViT-B-32-224 and VTT-B-32-384, respectively (Fig. 9). For protectte data, the inference success rate of AD approaches 1 100% when e = 3 or higher. At this privacy budget, model performance suffers significantly. Fig. 9 also illustrates the impact of batch size on the attack success rates, showing that the proposed AD performs consistently across different batch sizes.
(=f N
AMI Impact of B on Adv (AR 6, increasing 3 also increases P.DM= (3 + (m R²), and in turn makes the lower bound of Eq. (6) smaller. We illustrate this behavior in 10. The more
). As discussed in remark
However, it is important to note that the attack still ex- perimentally works against both vision and NLP data. To demonstrate this, we conducted comprehensive experiments across 4 NLP datasets (IMDB (Maas et al., 2011), Yelp (Zhang et al., 2015), Twitter (Saravia al., 2018), Finance (Casanueva et al., 2020)), 4 models (BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), GPT-1 (Radford et al.,
increase 3, the less likely the adversary succeed. Note that we still need to choose B large enough for Eq. 5 to hold. We fur- ther discuss the choice of hyperparameters in Appendix G.3. Given the assumption that the server has knowledge of the
8

---

Theoretically Unmasking Inference Attack Against LDP-Prrtectte Clients in Federated Vision Models
ImageNet
1.0
1.0
1.0
1.0
1.0
Attack Success Rato
0.9
Attack Success Rate
0.9
0.8
0.9
0.B
Model Accuracy
Model Accuracy
0.8
Success
0.6
0.6
0.7
0.7
0.7
0.4
0.4
Bedand
0.6
0.6
Altn
0.5
0.5
2
3
4
0.4
0.0
0.4
Privacy Budget E
Privacy Budget
Privacy Budget
(a)
(b)
(c)
Figure Comparison success rates of Attention- based AMI adversaries against CIFARIO protected by BitRand, GRR, RAPPOR, and dBiiFlipPM (a) well as the privacy-utility trade-off and the impact of batch size on attack success on BiiRend-protecedd data (b,c). Here, Attn-I0 means the attack conducted with batch size 10, and so on. Batch size not explicitly mentioned.
CIFAR10 Attn AMI
ergy (DOE) under Contract No. DE-AC36-08G028800.. Funding provided by the Laboratory Directed Research and Development (LDRD) Program at NREL. The views ex- pressed in the article do not necessarily represent the views of the DOE the U.S. Government. The U.S. Government retains and the publisher, by accepting the article for pub- lication, acknowledges that the U.S. Government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this work, or allow others to do so, for U.S. Government purposes.
Attack Success Rate
1.0
beta-C.1
0.6
0.4
1.0
1.5
2.0 Privacy Budget
3.0
Figure 10. Impact of B on Adv
AMI (ARRm)
2018), DistilBERT (Sanh et al., 2019)), and 3 LDP algo- rithms (GRR, RAPPOR, dBitFlipPM). The results given in Appx. H.2 indicate that privacy risks persist even for LLLMs, depending on the privacy budget. To explore more in depth the impact of different LDP mechanisms on the attack success rates, also conduct an ROC analysis of the attack success rates (on IMDB dataset) in Appx. H.3.
Impact Statement
Our findings demonstrate that LDP-protected data can still be compromised by dishonest FL servers, particularly un- der practical scenarios. Additionally, the study highlights the significant trade-offs between privacy and model utility, showing that the noise levels necessary to mitigate these attacks often lead to substantial degradation in model per- formance.
6. Conclusion
Our work has immediate implications for the design and de- ployment of FL systems, emphasizing the need for stronger privacy safeguards and more robust defenses against AMI attacks. Researchers and practitioners are encouraged to ex- plore enhanced crivacy-preserving techniques that balance security with model effectiveness. Furthermore, policymak- can leverage these insights to establish clearer privacy guidelines for the implementation of FL in sensitive appli- cations, such as healthcare and finance.
This work studies the formal threat models for AMI attacks with dishonest FL servers, effectively and rigorously pro- viding the theoretical bound on the vulnerabilities of FL under LDP protection. We also provide experimental evi- dence for the high success rates of active inference attacks under certain LDP mechanisms. The results imply that LDP- protected data might be vulnerable to inference attacks in FL. with dishonest servers and clients should carefully consider the tradeoff between privacy and utility.
By providing a thcoretical framework alongside empirical evidence, our paper serves as critical resource for under- standing and addressing privacy threats decentralized machine learning environments. The results pave the way for further advancements in secure and crivacyprreeeving AI technologies.
Acknowledgments
This work was supported in part by the National Science Foundation under grants 16666 and SaCT-1935923.
This work was authored in part by the National Renewable Energy Laboratory (NREL) for the U.S. Department of En-

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
References
M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021. URL https: openreview net
Arachchige, P. M., Bertok, P., Khalil, I., Liu, D., Camtepe, S., and Atiquzzaman, M. Local differential privacy for deep learning. IEEE Internet of Things Journal, 7(7):
forum? d= www.:..ee/ooorriiii//ee di
5827-5842, 2019.
Dwork, C. Differential privacy. on automata, languages, and programming, pp. 1-12. Springer, 2006.
In International colloquium
Aziz, R., Banerjee, S., Bouzefrane, S., and Le Vinh, T. Ex- ploring homomorphic encryption and differential privacy techniques towards secure federated learning paradigm. Future internet, 15(9):310, 2023.
Dwork, C.,, Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends@ in Theo- retical Computer Science, 9(3 4):211 (00)21112000 1477, 2014.
Blum, A., Hopcroft, J., and Kannan, R. Foundations of duta science. Cambridge University Press, 2020.
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., and Seth, K. Practical secure aggregation for privacy- preserving machine learning. In proceedings of the 2017 ACM SIGSAC Conference on Computer and Communica- tions Security, pp. 1175-1191, 2017.
Erlingsson, Ú., Pihur, V., and Korolova, A. Rappor:
Randomized aggregatable sewaacy-preeevingg ordinal re- sponse. In Proceedings of the 2014 ACM SIGSAC con- ference computer and communications security, pp. 1054-1067, 2014.
Fang, H., Qiu, Y., Yu, H., Yu, W., Kong, Chong, B., Chen, B., Wang, X., Xia, S.-T., and Xu, K. Privacy leakage on dnns: A survey of model inversion attacks and defenses. arXiv preprint arXiv:2402.04013, 2024.
Casanueva, I, Temčinas, T., Gerz, D., Henderson, M., and Vulić, I. Efficient intent detection with dual sen- tence encoders. In Wen, T.-H., Celikyilmaz, A., Yu, Z., Papangelis, A., Eric, M., Kumar, A., Casanueva, I., and Shah, R. (eds.), Proceedings of the 2nd Work- shop on Natural Language Processing for Conversa- tional AI, pp. 38-45, Online, July 2020. Association for Computational Linguistics. doi: 10. 10.18653/v1/2020. nlp4convai- 1.5. URL https:/ http://caaaaattoocooo org/2020. n1 p4convai-1.5
Gao, J., Hou, B., Guo, X., Liu, Z., Zhang, Y., Chen, K., and Li, J. Secure aggregation is insecure: Category in- ference attack on federated learning. IEEE Transactions on Dependable and Secure Computing, 20(1):147-160, 2021.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual leam- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Chen, J.. Wang, W. H., and Shi, X. Differential pri- vacy protection against membership inference attack on machine learning for genomic data. Pacific Sym- posium on Biocomputing, 26:26-37, 2021. doi: 10.
1142/978981 023270110000) URL https: : /tl:://puuuu /ubbee. ncbi. nlm. nih. /3V933369001//
Hu, E. J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. LoRA: Low-rank adap- tation of large language models. International Confer- ence on Learning Representations, 2022a. URL https: / openreview net (ecrummid=nneeeeeee.....
Deng, J., Dong, W., Socher, R., Li, L.-J,, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 245-.55. leee, 2009.
Hu, H., Salcie, Z.. Sun, L.., Dobbie, G., Yu, P. S., and Zhang, X. Membership inference attacks on machine learning: survey. ACM Computing Surveys (CSUR), 54(1 (1))117733 2022b.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for lan- Proceedings of the 2019 confer- ence of the North American chapter of the association for computational linguistics: human language technologies, volume I (long and short papers), pp. 4171-4186, 2019.
guage understanding.
Jiang, X., Hu, H., On, T., Lai, P., Mayyuri, V. D., Chen, A., Shila, D. M., Larmuseau, A., Jin, R., Borcea, C., et al. Flsys: Toward an open ecosystem for federated learning mobile apps. IEEE Transactions on Mobile Computing, 23(1)5001-59, 2022.
Ding, B., Kulkarni, J., and Yekhanin, S. Collecting telemetry data privately. Advances in Neural Information Process- ing Systems, 30, 2017.
Kariyappa, S., Guo, C., Maeng, K., Xiong, W., Suh, G. E., Qureshi, M. K., and Lee, H.-H. S. Cocktail party attack: Breaking aggregation based privacy in
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer,
10

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
federated learning using independent component anal- ysis. In Krause, A., Brunskill, E., Cho, K., Engel- hardt, B., Sabato, S., and Scarlett, J. (eds.), Proceed- ings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learn- ing Research, pp. 15884-15899. PMLR, 23-29 Jul 2023. httpss::: : httpthtttppp//ppceeeeeeeeeeeeeee http://prrceeednnneeeeeee mmlr.presss w202/kariyappa23.. html.
McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. Communication-effiiee learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273-1282. PMLR, 2017.
Nasr, M., Shokri, R., and Houmansadr, A. Comprehen- sive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and feder- ated learning. In 2019 IEEE symposium on security and privacy (SP), pp. 739-753. IEEE, 2019.
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009.
Ngo, K.-H., Östman, J., Durisi, G., and Graell Amat, A. Secure aggregation is not private against membership inference attacks. In Joint European Conference on Ma- chine Learning and Knowledge Discovery in Databases, pp. 180-198. Springer, 2024.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
Lester, B.. Al-Rfou, R., and Constant, N. The power of scale for parameter-effiiient prompt tuning. In Moens, M.-F., Huang, X., Specia, L., and Yih, s. W.-t. (eds.), Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, pp. 3045-3059, Online and Punta Cana, Dominican Republic, November 2021. Association for Computa- tional Linguistics. doi: 11116633/11222.mmmllppp main 243. URL. https: / aaccccllcc/ anthology org/2021. emnlp-main. 243/.
Nguyen, T. and Thai, M. T. Preserving privacy and secu- rity in federated learning. /EEE/ACM Transactions on Networking, 2023.
Nguyen, T., Thai, P., Tre R, J., Dinh, T. N., and Thai, M. T. Blockchain-based secure client selection in federated learning. In 2022 IEEE International Conference on Blockchain and Cryplocurrency (ICBC), pp. 1-9. IEEE, 2022.
Nguyen, T., Lai, P., Tran, K., Phan, N., and Thai, M. T. Ac- tive membership inference attack under local differential privacy in federated learning. In International Conference on Artificial Intelligence and Statistics, pp. 5714-5730. PMLR, 2023.
Liu, G., Tian, Z., Chen, J., Wang, C., and Liu, Tear: Exploring temporal evolution of adversarial robustness for membership inference attacks against federated learn- ing. IEEE Transactions on Information Forensics and Security, 18:4996-5010, 2023.
Pan, Y., Chao, Z., He, W., Jing, Y., Hongjia, L.,, and Liming, W. Fedshe: privacy preserving and efficient federated learning with adaptive segmented ckks homomorphic encryption. Cybersecurity, 7(1):40, 2024.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: /90. 1692, 2019.
Pasquini, D., Francati, D., and Ateniese, G. Eluding secure aggregation in federated learning via model inconsistency. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pp. 2443, 2022.
Lyu, I.,, Li, Y., He, X., and Xiao, T. Towards differentially private text representations. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1813-1816, 2020.
Qu, C., Kong, W., Yang, L.,, Zhang, M., Bendersky, M., and Najork, M. Natural language understanding with privacy-preserving bert. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 11888-149,,200 2021.
Ma, Y., Woods, J., Angel. S., Polychroniadou, A., and Rabin, T. Flamingo: Multi-round single-server secure aggrega- tion with applications to private federated learning. In 2023 IEEE Symposium Security and Privacy (SP), pp- 477-496. IEEE, 2023.
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al. Improving language understanding by generative pre-training. 2018
Maas, A., Daly, R. E., Pham, T., Huang. D., Ng, A. Y., and Potts, C. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the asso- ciation for computational linguistics: Human language technologies. pp. 142-150, 2011.
Ramsauer, H., Schäfl, B., Lehner, J., Seidl, P., Widrich, M., Gruber, L.,, Holzleitner, M., Adler, T., Kreil, D.,

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
Kopp, M. K., Klambauer, G., Brandstetter,, J., and Hochre- iter, S. Hopfield networks is all you need. In In- ternational Conference on Learning Representations, 2021. URL
Zhang, Zhang, J., Chen, and Yu, S. Gan enhanced membership inference: A passive local attack in feder- ated learning. In ICC 2020-2020 IEEE International Conference on Communicatiios (ICC), pp. 1-6. IEEE, 2020.
Safka, C. img2vec, 2021. URL http..ltttttss:/gghheeeeeee com/ Accessed: 2025-01-
Zhang, X., Zhao, and LeCun, Y. Character-level convolu- tional networks for text classification. Advances in neural information processing systems, 28, 2015.
31.
Sanh, V., Debut, L., Chaumond, J., and Wolf, T. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint 191000190000100 2019.
Saravia, E., Liu, H.-C. T., Huang, Y.-H., Wu, J., and Chen, Y.S.. Carer: Contextualized affect representations for emotion recognition. In Proceedings of the 2018 confer- on empirical methods in natural language process- ing. pp. pp. 3687-3697, 2018.
Shokri, R., Stronati, M., Song, C.,, and Shmatikov, Mem- bership inference attacks against machine learning mod- els. In 2017 IEEE symposium security and privacy (SP), pp. 3-18. IEEE, 2017.
Vu, M., Nguyen, T., Thai, T., et al. Analysis of privacy leakage in federated large language models. Interna- tional Conference Artificial Intelligence and Statistics, pp. 1423-1431. PMLR, 2024.
Wang, T., Zhang, X., Feng, J., and Yang, compre- hensive survey on local differential privacy toward data statistics and analysis. Sensors, 20(24):7030, 2020.
Warner, S. L.. Randomized response: A survey technique for climinating evasive answer bias. Journal of the American statistical association, 60(309):63 69, 1965.
Yang, H., Ge, M., Xue, D., Xiang, K., Li, H., and Lu, R. Gradient leakage attacks in federated learning: Research frontiers, taxonomy and future directions. IEEE Network, 2023.
Yin, D., Hu, L., Li, B., and Zhang, Y. Adapter is all you need for tuning visual tasks, arXiv preprint arXiv:2311. 15010, 2023.
Yue. Du, M., Wang, T., Li, Y., Sun, H., and Chow, S. S. Differential privacy for text analyties via natural text sani- tization. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 3853-3866. Associ- ation for Computational Linguistics (ACL.), 2021.
Zaken, F. B., Goldberg, Y., and Ravfogel, S. Bitfit: Sim- plc parameter-efficiiet fine-tuning for transformer-basee masked language-models. models. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 1-9, 2022.
12

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
A. Appendix
This is the appendix our paper Theoretically Unmasking Inference Attacks Against LDP-Protected Clients Federated Vision Models. Its main content and outline are as follows:
Appendix B provides the details of the security games examined in this work.
Appendix C shows detailed description of FC-based and Attention-basee AMI Attack in FL.
Appendix C.I: the description of FC-based adversary for AMI (Vu et al., 2024).
Appendix C.2: the description attention-based adversary for AMI (Vu et al., 2024).
Appendix proves the advantage of FC-based and Attention- AMI Attack in FL under LDP.
Appendix D.1: the proof of Theorem 1.
Appendix D.2 discusses attention layer's memorization capabilities through Exponentially Small Retrieval Error
Theorem (Ramsauer et al., 2021).
Appendix D.3: the impact of L.P mechanisms on data separation.
Appendix D.4: the proof of Theorem 3.
Appendix E proves the lower bound of the advantage of FC-based AMI attack under GRR mechanism.
Appendix F proves the upper bound of the advantage of FC-based AMI attack under LDP.
Appendix provides the details our experiments reported in the main manuscript.
Appendix G.I: details of the tested datasets.
Appendix G.2: implementation of Attention-based AMI Adversary against Vision Transformer
Appendix G.3: details choices of hyperparameters of the adversaries.
Appendix G.4: descriptions of the tested LDP mechanisms.
Appendix provides additional experimental results.
Appendix H.1: provides additional experiments on data protected by OME mechanism.
Appendix H.2 provides experimental results NLP datasets.
- Appendix H.3 provides ROC analysis of the attack success rates on IMDB dataset.
Appendix discusses alternative privacy-preserv/e techniques such as SMPC and Homomorphic Encryption.
B. Active Inference Threat Models as Security Games
This appendix provides the descriptions of the security games examined in our work. All games conducted between challenger/client, and an adversary/server in FL. The adversary is denoted by AAP, in which the superscript D indicates that the server knows the data distribution of the client's private data. At the beginning of the games, a random bit is generated and it is used to decide whether the challenger's private data has specific sample. The goal of the AMI adversary is to guess the bit which is equivalent to inferring information on the challenger's data.
As pointed out briefly in Sect. 3.1, the adversarial server A in all security games consists of three components ART AR ATTTACK and AGUESS An illustration of their dynamics is provided in Fig. 1. To specify an adversary, for each security game, we need to describe how it determines the model & for FL in ART how it crafts the model's parameters 0 in A ARTTACK and how it guesses the bit b in AZUESS The security games considered in this work are described below.
(AP): This security game is about the base AMI threat model, which is first formulated in (Nguyen et al., 2023) to study the threat of AMI in FL. While we do not directly study this security game, it serves as the foundation for Exp (.AP). The subscript NONE indicates there is no defense mechanism applied on the data. The goal of the adversary is to decide a target sample included in the training data D. Fig. 11 provides the pseudo-code of this security game.
AMI on unprotected data Exp
AMI
AMI
13

---

Theoretically Unmasking Inference Attack Against LDP- Clients in Federated Vision Models
EXPNONE (AP):
AMI
# Simulating the dataset D of the client
D <-D 0
while [D] < do
X 2 X Sampling X from the input distribution D
if X & D then
D (X)
# The random bit game
b _ {0, 1]
X + NONE
if b = 1 then
T = D Uniformly sampling from samples in
else
while = NONE or T E do
T ==xx X Sampling T' from the input distribution
# The attack
A # The adversarial server decides model +p
0 <- AATTACK( (T) # The server computes the parameters 0 based on the target input T
θ VoLa (D)
D)++ # The client computes the gradients of the loss w.r. the parameters based its data D < GGESS! (T, 0) # The server receives / and guesses bit b'
Ret bb' b] # The game returns if (the adversarial server wins), otherwise
Figure 11. The AMI Threat Model as Security Game.
EXPLDP (AP.E): .E):
AMI
Simulating the dataset of the client As in the AMI threat model in Fig. 11 # The random bit game
As in the AMI threat model in Fig. 11
# The attack
APHI
0 ARTTACK(TT (T)
D' -M(D) (Mc()) x Applying LDP mechanism M with parameter on the data
Ô (D') # Gradients computed the protected data
- (T
Ret bb' =
Figure The AMI threat model under LDP mechanism security game.
AMI LDP-protected data Expr. (AP): ): This security game describes the AMI threat model when the data is protected by LDP mechanisms. The work (Nguyen et al., 2023) extends EXI (AP) to obtain the formulation of (AA)) In this game, the client independently perturbs its training data sample D using an
AMI
mechanism M to obtain randomized local training set D' = -M (D) = (MM(X))xee)) This randomized data D' is then used for the local training instead of D. As result, the gradients that the FL server receives are computed on the protected data D' instead of D. Fig. 12 the pseudo-code of this security game.
14

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
C. FC-hased and Attention-based AMI Attacks
This appendix reports the details of FC-based and Attention-based AMI Attacks. Appx. C.I, we provide the descriptions of the FC-based AMI adversary proposed by (Vu al., 2024) used in our analysis. Appx. C.2 presents the details of the Attention-based AMI adversary (Vu et al., 2024).
C.1. FC-Based Adversary for AMI in FI.
We now describe the AMI FC-based adversary AFc proposed by (Vu al., 2024) and mentioned in Sect. 3.2. The ad adversary consists of components INI- APC AFC-ATTACK and AFC-
Algorithm 1 AFC--ATACE (7) exploiting fully-connected layer in AMI
Hyper-parameters: 7P E R # Configuring W1 g xdx and b1 e R of the first FC
Ldx
-T
W1
b1
-Idx]
T
# Configuring the first row of W22 E Raxadx and the first entry of by E R of the second FC
W2[1,:] - - 2dx' b2 [1[1]= < 7D
Ret all weights and biases
Algorithm 2 (T, exploiting fully-connected layer in AMI
If the gradient of ba[1] is non-zero returns
[b [1]}] then
Ret 1
end Ret 0
AMI initialization IPC-INIT' The adversary's model employs fully connected (FC) layers for its first layers. Given input E IRxx, the attacker computes ReLU(W; + by) = max(0, WiX + bi). where Wi and by denote the weights and set 2dx x dx and 2dx, respectively. For the second layer,
biases of layer i, respectively. The dimensions of Wi and the attack only analyzes single output neuron, thus, requiring W₂2 to have only 2d columns. We denote the parameters associated with this neuron [1, :] and [1]. Models with additional parameters can still works, as surplus parameters can simply be disregarded.
AMI attack AFC-ATTACK: The weights and biases of the first two FC layers are set as:
Lax Tax
-T T bi +
W1 +-
W2[1; + -1xx
where Lax the identity matrix and Láx the all-ones vector of size dx. The hyperparameter 7P controls the total allowable distance between input X and the target T,, which can be determined from the distribution statistics. The pseudo-code of the attack is presented in Algo. 1.
AMI guess AFC- In the guessing phase, the AMI server returns if the gradient of b2[1] is non-zero, and returns 0 otherwise. The pseudo-code of this step is shown in Algo. 2.
C.2. Attention-Based Adversary for AMI in FL
We now describe the AMI aateetion-bbsed adversary Artn introduced in (Vu et al., 2024) and discussed in Sect. 3.3. The 3 A components of it are R Aun and A A detailed as follows:
INIT: The model initiated by the dishonest server has self-attention as its first layer. We set the number of attention heads H to The attention dimension dum = dx - 1, where dx the one-hot encoding dimension. The hidden and output dimensions are d = dx and dy = 2dx, respectively. Any configurations with a higher number of parameters can adopt the proposed attack because the extra parameters can simply be ignored.
AMI initialization A
15

---

Theoretically Unmasking Inference Attack Against LDP-Protccted Clients in Federated Vision Models
Algorithm 3 A ATTACK (1) using self-attention in AMI
B. 3.7 E 1
Randomly initialize 3 W Wy: Wo and bo for all heads Randomly initialize matrix W Rdxxxx
W[: 1] < # Set the first column of W to
7 Q, R + QR (R) # OR-facttoizzttion W
W/ -- Q[2 dx]
] # Set W to the last dx rows of Q"
W 3W tT
# Set head 1 memorization mode
10 W 3W? fT
@
# Set head 2 memorization mode
11 WQ + W W < W # Copy Hcad to Head 3
12 WQ + W Q'
W K + W # Copy Head 2 to Head 4
13 W. - Iax. W + Lax
W
t- Idx+ W + lax # Setup for detection
[dx
-lax
Odx
Odx Sctup for detection
14 Wo Odx
Odx
- -ax
Tax]
15 bo:- 7, Vi E fI.... dy # Setup biases
Ret all weights and biases
Algorithm 4 A (v, 0) using self-attention in AMI
# If the any gradients of W is non-zero, returns 1 ||01 (w > then
Ret 1
end
Ret 0
AMI attack A ATTA-A The attack component A ATTACK determines the self-attention weights including W W W!., Wo and bo where h is the head's index. There are two typer-parametersy 3 and RR in A AATACK Intuitively, 3 controls how much the attention heads memorize their input patterns 2/ and 7 adjusts cut-off threshold deciding between D and & D (depicted in Fig. 3). Given target pattern for detection, the weights of the first attention head is chosen such that:
W W Blax and W R 0
(8)
To enforce condition (8), dx - 1 vectors orthogonal to a E Rdx are assigned to W2 using QR- QR-factorization. Subsequently, W k is defined as the transpose of SW where i represents the pseudo-inverse. In contrast, the second head initializes W randomly and sets W its pseudo-inverse. As result, the second condition of (8) does not hold for W3 and W The remaining parameters of the two heads are configured such that the first dx rows of compute max (0, z Z - -71 The third and fourth heads are designed to produce the negation of the first and second heads, respectively, meaning the last dx rows of Y compute max(0, Z Z 71 }. For simplicity in analysis, W and WO are constructed using identity and zero matrices. The pseudo-code of the attack is provided in Algo. 3.
AMI guess A GUESS: In the guessing phase, the AMI server checks if any of the weights in Wo have non-zero gradients. Algo. 4 shows the podudo-code of this step.
Attack strategy. We analyze the attack strategy of A against unprotected data. The iitention-based attacks exploit the memorization capability of the attention layer (Ramsauer et al., 2021): the attacks determine the layer's weights so that stored pattern x E X feed the layer, the returned signal will be similar the stored pattern
if a pattern similar r. X can be considered as the Key and the Value, while is the Query in the attention mechanism. The memorization is imposed by the condition W K W1 s BTax in (8). For an input X that does not contain the target pattern we have 4/2X T W W T which is the matrix of correlations of patterns in X. The output of the softmax, therefore, approximates IN since the diagonal entries of XT X are significantly larger than the non-diagonal entries. The head's output Z x X, i..., 2| R IG, as a result. Since the second head is the same as the first for x * v, we also have Z² as E4. When X contains the target pattern U, i.e., there exists an v, such that v; = u, due to the second condition of (8), we have
16

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
2; W Wa TTTaa O. This makes the softmax's output uniform, which can be interpreted the attention being distributed equally among all patterns. Consequentially, the attention's output of the first head is the pattern's 's average X. Since the second head does not filter v, its output approximates z;. By computing the difference between the two heads | - = and offset by 7, the adversary can infer the presence of in X. While the attack strategy of A against LDP-protected data follows the same principle, the main difference is the input of the attention heads is the protected version a instead of like in the case of unprotected data. The introduced noise means that (xf) W Wa might not be s O, making the softmax's 's output non-uniform, and the attention's 's output is no longer the pattern 's average. We analyze in-depth the behavior of A under LDP in Appendix D.4.
D. Vulnerability of FL under LDP to AMI
This appendix provides the details of our theoretical results on AMI in FL under LDP. Appendix D.I shows the proof of Theorem 1, which is about the vulnerability of LDP DDP-protected data against FC-based AMI attack. Appendix D.2 analyzes the memorization capabilities of attention layers. An example demonstrating the impact of LDP mechanisms on the data's separation is provided in Appendix D.3. Finally, Appendix D.4 shows the proof of Theorem 3, which is about the vulnerability of LDP-protected data against attention-based AMI attack.
D.I. Proof of Theorem on the Vulnerability of LDP-Protected Data to AMI in FL
This appendix provides the proof of Theorem I, which is restated below:
Theorem. Given the security game Expt.bp- there exists AMI adversary APC whose time complexity is (d3x) such that A AMI (APc) 2 1 values of the LDP -eP-mehhanism and PM is the probability that the LDP-mechanism makes the protected version of data point inside the neighborhood of another data point.
n [ 1x -T PMs, where n is the size of the dataset D, |X is the cardinality of the possible output
Proof. Given D (x.) where X; E X, and X C Rdx the LDP-protected version of the input X is M=(X). For the model specified by AFc as discussed in Subsection 3.2 and described in Appendix C.I, its first layer computes:
ReLU ([ Lax Mx(X) + [ = ReLU ( ( -77
(9)
The first row of the second layer then computes:
dx
20 ReLU
ReLU ((of - 44) + ReLU (4 - xz)) + +D) = max {-p - -IM - .]
(10)
This implies the gradient of be[1] = 7 is non-zero and only if 7D (X Til,, Thus, for a small enough
TP, Th D is equivalent to non-zero gradient. We set 7 = A Intuitively, the attack fails if either: (1) T & D, but EX E D such that M(X) E B1 (T, (2) D, but M(X) (T, for all X E D. This insight is illustrated in Fig. 2.
Given a data and a randomly sampled point T & D, the probability that ME(X) belongs to B, (T, A is upper bounded by:
Pr ( = Pr (M E (T, A and MF() $ (XX,
(11)
= =Pr[MF(X) E (T, ( & (X, A Pr (X*(X) & (X(xA*)]
(12)
< 1x1-1
Pr (X) & (X)
PM
(13)
(11) is from the fact that (X) E (T, ) implies MxX) & (X, as the balls are disjoint. (12) is from the conditional probability formula. For (13), given MX $ B, BX, A since all the balls (X, A for X E and Bj (T, A are disjoint, (X) can either be in one of the other |X| - balls of radius A or be outside of all those balls. As T are chosen randomly, the probability that M (X) is in one of the X 1 balls bounded by 1/( 1).
17

---

Theoretically Unmasking Inference Attack Against LDP-Protccted Clients in Federated Vision Models
If b 0, the probability that 2g activated is:
Pr 120 0/b = 0] = Pr [3x E D such that M(X) E BI(T,
(14)
XED X - PM 1 1 -1 PM
(15)
where (14) is from (10) and the inequalities in (15) are from the union bound and (13). On the other hand, if b 1, the probability that 2g is not activated is bounded by:
Pr = 0lb = 1] = Pr [M(X) & (T. for all X E Dbb 1] < (T()) & (T, |) = = Pr (M(M(T) & (T, A)] = PM
(16)
(17)
where inequality (17) uses the fact that, conditioned on b 1, D.
Thus, we have the advantage of APc:
Adv AMI LDP
(AFC)
Pr(b' = 1|6= P = 0هb 0] -1
(18)
=11-Pr(ze = 0 1I) + (1 Pr(zo > 0/6 = 0)) -
(19)
2 (1 - PM:) + (1
X - PM - -1=1- 71 + X
PM
(20)
X -1
where (20) uses (15) and (17). Since APC can be constructed in (ddx), ), we have the Theorem.
D.2. Memorization capabilities of Attention layers.
We now state Lemma bounding the error of the self-attention layer in memorization mode (Vu et al., 2024). The Lemma can be considered as specific case of Theorem 5 of (Ramsauer et al., 2021). In the context of that work, they use the term for sell-separated pattern in their main manuscript to indicate the condition that the Theorem holds. In fact, the condition (21) stated in Lemma is a sufficient condition for that Theorem of (Ramsauer et al., 2021).
Lemma 1. Given a data X, constant > O large enough such that, for an I; E X:
2
1
A
log(2(Nx 1)NxaM²)
(21)
N x
then, for any & such that ||5 xi|l S M we have
|| Xsoftmax
2M - exx ad;)
Proof. See Lemma 3 (Vu et al., 2024) for proof or Theorem 5 (Ramsauer et al., 2021) for proof of general case.
Intuitively, Lemma I claims that, if we have pattern & near 24, sofumax (aXT {) is exponentially near & as a function of A, Another key remark of the Lemma is that | Xsoftmax (aXT [| exponentially approaches 0 as the input dimension increases (Ramsauer et al., 2021).
We now consider the iteration gnew = f(E) = Xp = X softmax(3X7g). We now state Lemma (Lemma A3 (Ramsauer et al., 2021)) that provide the bound on the Jacobian of the fixed point iteration:
Lemma 2. The following bound on the norm || of the Jacobian of the fixed point iteration f holds independent of p or the query ę:
|| & 3m
(22)
Proof. See Lemma A3 (Ramsauer et al., 2021).
18

---

Theoretically Unmasking Inference Attack Against LDP-Protccted Clients in Federated Vision Models
D.3. The Separation of LDP-Protected Data
To give an intuition that the LDP mechanism generally increases the separation of the data, we consider the LDP mechanism in which each ry is i.. sampled and independently added to each input pattern. We have the expected value of the separation between two patterns is:
2 or - x
33 E (are + rt)
(or + rz)
- (I: + ra)
(xy + r3)]
(23)
E
+ 2E
[2]]] [r] E [r. xg] - E [r] sxx
+ E [ ri] - E E
(24)
b 33 x.. Ø3j + Var(r:)
(25)
As we can see, the last expression is the expectation of the separation of the data D plus the variance of the noise. Thus, we can assume that A resulting from the defense mechanism is to be at least similar to the separation A of the original data D.
D.4. Proof of Theorem 3 on the Vulnerability of LDP-Protected Data to Attention-basel AMI in FL
We are now ready state the proof of Theorem 3. We restate the Theorem below.
Theorem. Given A -separrrte data DyMs (the LDP- -proptetetted version of the data D) with i.idd patterns of the security game EXPLDP, for any B >> 0 large enough such that:
AMI
2 3Nx
A
+
1og(2(NX 1)Nx8M
(26)
3
then there exists an AMI adversary that exploits the self-attention layer A whose complexity is O(dx) of the threat AMI (ARttt) is lower bounded by:
model ExpTbp such that Adv
2nNx
Advibp(AARt 2 DDM-
P.D. +
+PD proj
(3Ar+ + 3(mmaz)) RR) -
(27)
MN
MN
where À = 2Nx - 1) exp (2/Nx- - 3D7) and De is the distribution of the protected data DM induced by the original data distribution D and the LDP-mechanismg Me. m Nx ENEx xf is the arithmetic mean of all LDP -PP-protected patterns and m max I|=; m l|. Here, Piej (8) is the probability that the projected component between two independent patterns drawn from DM,, is smaller than 8 and ph (0) is the probability that a random patlern drawn from DM is in the cube of size 28 centering at the arithmetic mean of the patterns in DM,.
Proof. We represent the victim's 's dataset as D = [ where X, E X. and X' C RdxxXNx For any 2-dimensional array X, each column žj E Rx is referred to as pattern. The distortion imposed by LDP is modeled by a noise r added Nx
to each pattern: X = (X) = {z; + rı 1 = {-f ) For brevity, we first consider the following notation of the output of one attention head under LDP without the head indexing h:
X softmax
(aann WW Woxx))
(28)
Notice that we omit Wy because they are all set to identity ( [lnee 14 Algo. 3).
To show the Theorem, we consider the AMI adversary ADtm specified in Subsection 4.2. Since W E Rdx xdx (line 5 in Algorithm 3) is initialized randomly, it has a high probability of being non-singular, even after assigning v to its first column (line 6 in Algorithm 3). For simplicity of analysis, we assume that W has full rank. If this assumption does not hold, we can re-run the two corresponding lines of the algorithm. Similarly, we also assume that all W and W have rank damn dx- 1.
For all heads, we set WK )( (lines 9 and 10 in Algorithm 3). Consequently, WWTW w&We is the projection
matrix onto the column space of W By defining [E,.... ... .xx, = = = 1/ Wx T Wo it follows that E, is the projection of the pattern 5 onto this space.
For head I and head 3, as result of line 6 in Algorithm 3, we can express W = [u, W2, ... wax] Based on the QR factorization (line 7), we have:
QR = [V, w2, ...all - R = Q [V, w2, ... w.x
(29)
19

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
Since R is an upper triangular matrix, is orthogonal to all rows Q: for i {2.... ,,.,,,dx) of QT. Additionally, due the 2 all of which are orthogonal v.
assignment line 8 in Algorithm 3, the column space of WQ is the linear span of [Q:]_= As result, the difference between X and E\r corresponds to the component of X in the direction of e:
x ー = [2 st
Nx
< = (roj,((f): .. P )b E (1,3)
(30)
where Proj, (3r) is the component of pattern o E 2x along
For head 2 and head 4, although Qa-factorizttion is not performed, 1 W K Wi for these heads also acts as projection matrices, but onto different column spaces. These spaces are likewise of rank dx 1, and each omits one direction. Denoting this direction as u, we can express the difference between X and E for these heads as:
X - i et.
NNx
x (ff)). ... (rNj. h e [2,4]
(31)
We now denote fa Rdx XN + x.x as:
fa= = softmax (a(X²)T=)
(32)
For brevity, we also abuse the notation and write 5' (5) = softmax (a(X))TE) for & and g e IRdx,
With this, the output of the layer before the ReLU activation can be expressed as:
fa(E') - fa(=²) -
[f((=') - fa(=3) 71 T
Z
(n(=t) f3(=3)
[a(E2) 53( 71
(33)
(a(E)) - fs(C?) 7 1.: fs(ENx) fal(s (a(Et) -s(()) - 7, Sa(ENx S
(34)
where 3 = 3/Vdam and y defined as in Algorithm 3. From the above expressions, it follows that Z has non-zero entries if and only if:
EF such that ([fx(E)) [s(5?)]l >>
(35)
Note that heads and 4 are used to handle cases where the entries of fa(E!) are smaller than those in fo(E)) The condition (35) can also be rewritten as:
>
(36)
For given pattern I;, we now examine two cases: H; + v and x; = v.
Case 1. For a pattern x; E X such that 2i * v, from Lemma 1, we have
|of fa(Et)| < 2M2 1) exp (2/Nx BB1))
(37)
xxf - fa(E?) S 2M (Nx 1) exp (2/Nx - BBA))
(38)
when ||س|- - - = (xf)( S 1/(3Nx and (x5) || < 1/ Ms), respectively. Note that it is necessary to select sufficiently large 3 to ensure that Lemma I holds. We denote these events by A! and A3.
Using triangle inequality, we have:
llfo(Et) - fo( $ ([fo(E)) - 2 + +|1 fo(E?)) < (Nx exp (2/Nx -B,))= 2
exp
(39)
with a probability of Pr [4] n A?]. Here, we define á 2M (Nx 1) exp (2/Nx - -3 We further relax the inequality using the infinity-norm, which bounds the maximum absolute difference in the pattern's feature:
(Ifa(E)) fa( < 2
(40)
Since the data point X is A seppararte i.e., A < A;, we further obtain:
(| fo (e S 2
(41)
20

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
where A 2M exp (2/Nx -BA)).
We now analyze the event A/. Essentially, this event occurs when the component of z along e is smaller than a constant determined by the data distribution D. Moreover, since both and 2; are independently drawn from the distribution (as AMI (Fig. 12)), and r; is the random noise introduced by the LDP mechanism, v and x; can be treated as two random patterns sampled from the input distribution. Consequently, the probability of A corresponds to probability that the projected component between two random patterns less than or equal to 8Nx Mre Formally, for an imput distribution DM: we denote PDM: (d) as the probability that the projected component between any independent patterns drawn from D. is at most 6. We then have:
specified in the experiment EXPLDP
Pr [A]] = Pr
1/(8Nx =
PDDM pruj
(
(42)
M
by definition. Similarly, for A7, we have:
Pr [ Pr Proj.. (25) M/) = Piij
1
(43)
MN
Since e and u are independent, we obtain:
Pr [ n A 2
PDM proj
(44)
M
Case 2. On the other hand, when z; we have the output of head is:
(xx=Te)) st)
X softmax
(45)
X softmax
[3x=T (o, - Proj. (x (xf))
(46)
X softmax
[8x*T (v (u+r;- Proj. (u + +r)))
(47)
X softmax
(3x=T (ra (ri- (r:))
(48)
Thus, the difference between the output of head with X X= No En 24 can be bounded by:
X softmax
(8x= 5)
-X
(49)
softmax (8x+Tg)
Xostmfaax (8x=To)l
(50)
[8x=T (ri (re- softmaa (BX=TO)
X softmax
(51)
0fa(E)
||r: ||r:- - F!ll < max afa(E)
max
||ral| S 3(m 2
(52)
0g
where the inequalities due to the mean value theorem (Lemma A32 (Ramsauer et al., 2021)) and from the fact that the bounded by 3(mmaax) (Lemma 2). Thus, from triangle inequality, we have
0fa($)
Jacobian of fa. i.c., 2
[[fa(E)) - Xx + + - - + + - 2 8 1 r: 7:|100 ||
[f((E fo(5?)
(53)
8811 - -Iu+r: - 1l
(54)
> 8 -- rill - 3(mmnaz) 2 - -Ā;
-urill
(55)
when A3 happens (so that A 2 |100 f (loo). whose probability is pro)
(
21

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
We have the probability that
- - >0 2 is bounded by:
Pr [llf((E)) fo( > 2
(56)
=1 Pr [I]fa(E)) - fa(E)) 3 2
(57)
=1 Pr [I]sa(E.) 13(4)) 11 S 2 Pr [4]]
Pr [ll) fo(E))1 2 ||4;] Pr [-4
(58)
21 Pr [[]s (2(53) - 2-+144 - Pr [-A
(59)
21 1 Pr [||X* - rill.0 -B(mman' ) - A < 2 [
-errill
Pr Pr [-A
(60)
Pr
||| X - -v-rill
rrill S mmaz )² R² + 3 Pr |-A
(61)
-DDM pmj
3Nx - [r [ Box , ;( R + 3
1
(62)
where Box(x, 6) is the cube of size 25 centering at z. The inequality (60) is due to (55) and (61) is from the fact that is independent from X and v.
+ R We now consider Pr [e + E Box (X", (( RR
3}+)]. which is the probability that the pattern e + r3 belongs to the cube of size mazz) R² 6ă around the sampled mean of the LDP protected patterns in X We denote PMM (6) the probability that random pattern drawn from D-M. is in the cube of size 26 centering the arithmetic mean of the patterns in D.M. If the length Nx of X is large enough, we have the sampled mean X is near the arithmetic mean of the patterns and obtain Pr [ + r, E Box (Xe, ((m R + 33+)
ppe (3(m R + 3.²).
Back to main analysis. From the analysis of the two cases, if X, we have:
( 3Nx M )
Pr [I[ss()) oz S 2A
WL ,N)
Based on the analysis of the two cases, when v is not an element of X, it follows that:
Nx
2Nx
Pr (I) - I 2 = II Pr [1lf((()
fa( I 2Ā]
>
pDM proj
BNx M
Since the data points in D are sampled independently, if does not appear in D, we obtain:
2nNx
Pr [Ilf(EE)) fo( 2ā for all X D] 2 ppr..
M
On the other hand, if u E X, we have:
E f1, .... Nx) such that Pr [[lfa(Et) - fa( > 2 2 21-P 21-Phn
P.DMe (B(m)naz)) R² + 3.²
Pr ff
2Ā] 2 Pr - ff S 2 for all X E D]
1
DDMe (B(m)naa) + +3
> proj
3Nx M
Thus, if pattern appears in D, we have:
(
Pr 3X E D such that
22
PDMe proj
-pDDMe (8(mmnnn) ) + +3
3N M
22

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
By choosing 7 22 have the probability that the adversary wins is:
Pw Pr Pr [10] ( 0/u < D + Pr Pr/u Pr [11((W9) 010 = D
(63)
Pr xx E D such that (1fa(E)) -
fa( -
> 28 0 E D]
2
2 Pr [I]sa(E))
- [s((3))l< 22 for all X E D/v D]
(64)
22
2 p proj
(BNZME) -Pou (B(m + +355)
2
piay proj
(eNSAVE)
2n
(65)
Thus, the advantage of the adversary Attn in Algo. be lower bounded by:
2nNx
Adv (A
2Pw - 1 PDM,
+ +DDM proj
1
-DDM
(B(m
2
3NxM=
M
(66)
Since the complexity of the adversary A is O(dx) (determined by lines and 10, Algo. 3), we have the Theorem
E. Proof of lower bound of Adv
AMI GRR-LDP (APc
Theorem 4. There exists an AMI adversary APe against data protected by Generalized Randomized Response (GRR)
LDP algorithm whose time complexity is (d3x) of the threat model EXPGRR-LDP such that AdvGRR-LDP (AFR) e+
e--n
Adversarial Advantage
Adv -aels
Adv/f_anll.n==50. [4
Adv!' mele.n- 100, 2) Upper
(Privacy Budget)
Figure Visualization of the theoretical upperbound (Theorem. 2) and lower bound Adv (R..LD((APP)).. (Theorem. 4)
Generalized Randomized Response (GRR). Given with value E X. A random variable, denoted by X, represents the response of the user value also in X. The generalized randomized response works follows:
if x
Pr
X
(67)
I if #
where d = | Thus, we have PMERR
d-1
From (20), we have
71 + dl -
n d 1 d-1 d d-1 1
AMI Adv AdvAMI
(AR- 21 (PP2111
PM
(68)
d -
6 + d -
e + d 1
d+1
E e-n
(69)
+ d 1
+d1-1
It is clear that this advantage is smaller than the upper bound (4):
n
_ -
E + - <
(70)
e +
d
e +1
23

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
F. Proof of upper bound of Adv (APc
We now restate Theorem 2 on the theoretical upper bound of Adv AMI ( (APc).
Theorem. For all AMI adversary A of the game EXPLDP. we have
S e e +1
Adv
Proof. For clarity, we denote 1 the instance of D that is sampled in case b = 1 and to the random sample of the input distribution D in case b = O. Let Dj = D and D2 = D {41 U (1a). With this notations, we can think the adversary needs to differentiate D1 to D2 instead of or
Using the notations as denoted in Lemma 3, the probability the adversary wins is:
Pw = Pr = Pr [M(S)) E IM =DI] +
[M((S] To D3)
(71)
1 - Pr (M((D)) + [M.(D2) E ToML))
(72)
Here, S is dummy variable representing which datasets, i.e., D1 or D2 is chosen by the experiment. Similarly, we have the probability the adversary loses is:
Pr 1 (Pr (Pr (M.(Di)) + Pr [M((D)) E TTM.)) 2 2
(73)
From Lemma 3, we have:
Pr Mc(Di)) E TiMi] sp Mc(D2) E TiMi)
(74)
Pr MM((D)) E TDMi
Se Mc(Di) E
(75)
Combining (72), (73), (74) and (75), we have
Pw
1
Pr Mi(D)) E T.M. +r Mc(D2) (M.(D))TToMM.))
(76)
2
<e (Pr [M((D)) E ETMM]] + +Pr [Mc(Da) E {MM]
= e² PL
(77)
2
We then have:
+ Pr.) = Pw S ((++ee))
(78)
Thus, the advantage of the adversary can be bounded by:
Adv E (A) 2Pw 15 5: - e +1
(79)
We then have the Theorem.
Lemma 3. Denote Me X + X a randomized function satisfied e-LPP. For a database D E X0, we denote Mc(D)
(Ma(z): i D}. Then, for all database D2 and D2 different by entry, have:
Pr [r[M..D.) E TTM S s (M.MDM E T.M.
where TyMe is the set of all database D such that the adversarial return b/ on that realization of M..
Proof. Let S be an arbitrary subset of X. For pair t E X S, consider the sets D2 = SUU [t} and D2 = SU (t}.
Since M, satisfies c-LDP, we have:
Pr]M((()=0)) ( VOE
(80)
24

---

Theoretically Unmasking Inference Attack Against LDP-Protccted Clients in Federated Vision Models
From the post-processing property of e-LDP, we have:
(r(g(M.())) - s & Pr(((, AO' E Rangelg),
(81)
for all function g: X Range(g).
In the AMI experiment, the guessing of the adversarial A, i.e., AGUESS 0), can be considered as a function on De-DIP (D) as 8 is the result of some computations of MD) We describe this A(M,(D) = b'.
We now show that
Pr [A(MM. = b] (F(( 69
(82)
We show (82) by contradiction. Suppose there exists D and D2 such that the condition does not hold. We can construct a function g : X + {0,1] as follow:
((M
(83)
With this, we have
= (U =
(84)
>e Pr [A (M, = (M - - =b]
(85)
which contradicts (80).
Since condition (82) is equivalent to the condition stated in the Lemma, we then have the Lemma.
G. Experimental Settings
This appendix outlines the experimental setup and implementation details of our work. Our experiments are implemented using Python 3.8 and executed on a single GPU-enabled compute node running Linux 64-bit operating system. The node is allocated 36 CPU cores with 2 threads per core and 384GB of RAM. Additionally, the node is equipped with 2 RTX A6000 GPUs, each with 48GB of memory.
Table 1. General information of our reported experiments in the main manuscript.
Experiments
No. runs
Adversary Hypeer-arameters
Dataset
Embedding
LDP Mechanism
Fig. 5.6
1000
A
8,7
One-hot Spherical
No
Fig. 7,8
20 x 200
AFc
7 ,
CIFARIO/ CIFAR100
ResNet
BitRand/ GRR/ RAPPOR/ dBitFlipPM
Fig. a,h
20 x 200
A Amm
8,7se
CIFARIO
ViT-Base
BitRand/ GRR/ RAPPOR/ dBitFlipPM
Fig. 9 40 x 100
A
8,75e
ImageNet
ViT-Base
BitRand/ GRR/ RAPPOR/ dBitFlipPM
Table shows the general information our experiments reported in the main manuscripts. The hyper-parameters column refers to those of the adversaries, and e refers the privacy budget. The embedding specifies how the dataset transformed to obtain the data D for our testing of inference attacks. The No. runs indicates the total number of simulated security games for each point plotted in our figures. For instance, the number 20 200 means conduct 20 trials of the experiment. Each trial consists of 200 games. The attack success rate is measured as (Pr[b = b = 1] + Pr(b' = 0lb = 0)). For each trial, everything is reset. In each trial, only the LDP mechanism is re-run. Using equation I as well as lower (Theorem 1) and AMI (AP)). we can derive the theoretical lower and upper bound of the attack success rate. upper (Theorem 2) bound of Adv
The reported model accuracies (black lines) in all our figures are obtained by evaluating the model on classification tasks. Particularly, for APc. we use ResNet's embedding combined with a Multilayer Perceptron to generate classifications. For APc. the native classification tasks and the original models published along with the datasets. To implement LDP, we add noise directly to ResNet's embedding. For ViTs, we add noise to the patch embeddings of the image. Since no pre-train model for ViT-B-32-224 on ImageNet-2Ik on CIFARIO. The labels for classification in CIFARIO are from the original dataset.
100 are available, we fine-tune the model that was pre-trained on
In the following appendices, we discuss more on the dataset and their embedding, the implementation the adversaries, and the implementation of the L.PP mechanisms. Those contents are in Appx. G.I, G.2, G.3 and G.4 respectively.
25

---

Theoretically Unmasking Inference Attack Against LDP-Protccted Clients in Federated Vision Models
G.1. Dataset and embedding
In total, our experiments are conducted synthetic datasets, 3 real-world datasets and LDP mechanisms. The synthetic datasets are one-hot encoded data and Spherical data (data the boundary of unit ball). The real-worid datasets CIFARIO (Krizhevsky et al., 2009) and ImageNet (Krizhevsky et al., 2012). The real-world datasets pre- p--perscesed with practical pre-trained embedding modules to obtain the data D in the threat models. For ResNet, we use Img2 (Safka, 2021) to extract the feature embeddings of images and for ViTs, we use pretrained foundation models published on HuggingFace by the authors. The parameters of D in each experiment are given in Table
For the synthetic datasets, we use a batch size of 1 since it does not affect the results of one-hot encoding. Furthermore, the setting also provides better intuition on the asymptotic behaviors of other datasets Fig. and Fig. 6. For ViTs, the number of patterns Nx is equal to the number of image patches. Details how A works on ViTs are given in G.2. The embedding dimensions dx are determined by the choice of the embedding modules.
Table 2. Information of the data D in each testing dataset.
Dataset
Embedding
Tested batch dimension n. X dx X Nx
One-hot
N/A
1 x [10, ... 1000] [5, 10, 15)
Spherical
N/A
1 [10000, 35000] X [5, 10, 15]
CIFAR10
ResNet-18
64 x 512 x 1
CIFARI00
ResNet-18
64 X 512 1
CIFARIO
ViT-B-32-224
20, 40) x 768 X 49
ImageNet
ViT-B-32-384
100 20, 40) x 768 144
G.2. Implementation of A on Vision Transformer.
First we describe the architecture of ViT, which was first proposed in (Dosovitskiy et al., 2021). First, the image is divided into L fixed-size patches (e.g.. 16 16 pixels 32 32 pixels), which are flattened into vectors. Each patch is projected into a lower-dinensional embedding using a linear layer. Position embeddings learnable [class] embedding is included for global context. The sequence of embeddings is processed by L Transformer encoder layers. The output corresponding to the [class] embedding is passed through an MLP head to predict the image class. In summary. ViT treats image patches like words in a sentence, using the Transformer architecture to model relationships between patches and perform tasks like image classification. For naming scheme, ViT-B-32-224 means the model ViT-Base the image size is 224 and the patch size 32. Figure describes VIT's architecture.
added to retain spatial information, and
Vision Transformer (VIT)
Transformer Encoder
Class Bird
Lx
MLP
Head
Cur
Norm
Transormer Encoder
Pacc Poition
Multi-eaad Attention
Faxa bu crtudding
Linear Projection of Flattened Putches
Norm
Embedkee
Figure 14, Architecture of Vision Transformer model. Image adapted from (Dosovitskiy et al., 2021).
Recall we represent the victim's dataset as {x,)2 where X, E X, and X' S XdxxXxx For any 2-dimensional array
26

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
X, each column x; Rdxx is referred to paltern. In the context of A on ViT, since A operates on the pattern level, it operates directly the Patch+Position Embedding vectors. Given the image is divided into L patches, Nx I. in the attack, and dæ the dimension of each embedding vector. When add LDP noise to the data, add it directly to these vectors as well. For example, in the case of ViT-B-32-338 there are in total corresponding to Nx= = 144.
384
32 x
984
32
12 x 12 144 image patches,
G.3. Implementation of the Adversaries
In our theoretical analysis of AFC and A we have specified how their prper-parameters should be chosen so that theoretical guarantees can be achieved. For convenience reference, we restate those setting here:
For Theorem T is set to Ax The argument is made at Subsect. 4.1.
For Theorem 3, , is chosen such that condition of the Theorem holds and y is set 2 The argument is made Appx. D.4.
Table 3. Note the values of / in experiments.
Dataset
Note B
Min B
Max B
One-hot / Spherical
B is is set to constant
10
10
CIFAR10/100
The more noise, the smaller B 0.01
0.07
ImageNet
The more noise, the smaller 3
0.01
0.07
However, as the adversaries generally know the data distribution and the LDP mechanism in practice, they can simulate the data as well as its protected version. We integrate these simulations into our implementations of AAC and A to tune 7 and y before the security games. In fact, for given LDP mechanism and an privacy budget, the server uses a dataset from the data distribution D and collects the layers' outputs before the ReLU activation. Then, linear regression model is fitted those outputs estimate the biases 7P and that will be used in the security games. Regarding the hyper-parameter B, while it cannot be tuned with linear regression, for each privacy budget of security game, try several values of based on the statistic of 'D before the game and settle with look-up table for it. An example of possible values of / are given in Table 3. However, we found that simply setting = 0.01 usually give satisfactory results. We use 8 = 0.01 for all NLP experiments.
G.4. Details of LDP mechanisms
Generalized Randomized Response (GRR). Given an user with a value E X. A random variable, denoted by X. represents the response the user on a value x also in X. The generalized randomized response works as follows:
if = e1 if f +
Pr x
(86)
where d d== |
RAPPOR (Randomized Aggregatable Privacy-Preserving Ordinal Response). Each user has a value v encoded as a Bloom filter vector B 1) using f hash functions. permanent randomized response B' generated as:
with prob. } (flip to 1)
B
with prob. _ (flip O)
B with prob.
(keep original bit)
Then, the instantaneous randomized response S E {0, 1 is sampled from B' as:
if B =1
Pr[S: = 1|
P
=
27

---

AMI also performed worse on LLM Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
dBitFlipPM. Given user with value e [k:]. the mechanism proceeds follows:
The user selects d random buckets ..i,.....aa Ja) C [A] without replacement.
For each selected jp. the user responds with a bit b,, E {0, 1) such that:
e/2
if 1p
Pr(b,, 11 =
ef/241
if V # Jp
e+/241
The data collector reconstructs the histogram using:
k
e²/2 +
1
he(v)=
(t) ge/2 -1 e=/2 -1
nd
This mechanism guarantees e-LDP with reduced communication cost and supports memoization for repeated collection.
In bit- bit-flippin mechanisms like OME and BitRand, the original data or embedding features are first converted into binary vectors. The LDP mechanisms are then applied on top of those binary representations of the signal. After that, the protected binary signals are converted back to the original domain of the data before the training of the machine learning models.
In OME, each bit i of the binary representation is randomized based on the following probabilities:
a
Pix = 1 a
i E 2j, (i) =1
Vi E 10, 1]: 1)) P2x = 1 3
if E 2j + 1, (1) =1
(87)
yx xx if e (1) -0
where (i) E {0, } is the value of the bit i in the binary representation, vy is the perturbed binary vector, e is the privacy budget, and a is a parameter of the algorithm.
On the other hand, BitRand introduces the bit-aware term I term helps the mechanism take the location of the bit into consideration for randomization. Intuitively, BitRand aims to apply less noise to bits that have more impact on the model utility. Particularly, the probabilities of perturbation are defined as:
1% to control the randomization probabilities. That bit-aware
1
px 1 1 a exp(
if Un (i)
1
(
Vi E 10, P(((0))))
1)
(88)
aeep(
1%61
7
9x
if U (1)
1 exep
1%!
H. Additional Experiments
H.1. Experiments using OME mechanism
We observe that models trained on OME-protected data have almost constant performance across the tested range of the privacy budget e, as illustrated in Fig. 15. The same phenomenon of OME observed and reported in multiple previous works (Arachchige et al., 2019; Lyu et al., 2020; Nguyen al., 2023). The reason lies in the large sensitivity of the encoded binary representation in OME weakens the effect of on the randomization probabilities (Lyu et al., 2020). With the exception of ImageNet, even with high loss in model utility, AMI adversaries still achieve a high successful inference rate.
H.2. Empirical results on NLP datasets
Table 4, 5, 6 show the average accuracies, F1, and AUCs of AMI attacks on NLP datasets under GRR, RAPPOR and dBitFlipPM, respectively. Given the same privacy budget, APC has higher accuracy than AD GRR also usually performs worse as a defense mechanism compared to RAPPOR or dBitFlipM. Attention- data compared to vision data. This could be due to the fact that it hard to bound the norm budget R of noise r, due to the discrete nature of LDP noise when applied NL.P domain.
28

---

Theoretically Unmasking Inference Attack Against LDP-Protccted Clients in Federated Vision Models
Insperatle
CHARLO
Impelttt
Mide ACCURSOC REIEE
Privary Butggtt
Plisayy Bidigat
Phioay Budget
Triaty Bucgetz
(a)
(b)
(d)
Figure 15. Success rates of AMI adversaries against datasets protected by OME.
Table 4. Average Accuracies, Fl, and AUCs of AMI attacks under GRR defense.
BERT
RoBERTa
DistilBERT
GPT1
Method
ACC
F1
AUC
ACC
F1
AUC
ACC
F1
AUC
ACC
F1
AUC
AFc
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
00
Artn
1.00
1.00
1.00
0.96
0.96
0.99
1.00
1.00
1.00
1.00
1.00
1.00
AFc
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
8
A
Astn
0.99
0.99
1.00
0.89
0.88
0.95
0.99
0.98
1.00
0.97
0.97
0.99
AFc
0.99
0.99
1.00
0.97
0.97
1.00
0.97
0.97
1.00
0.99
0.99
1.00
6
A
Artn
0.86
0.84
0.94
0.75
0.72
0.82
0.86
0.83
0.94
0.86
0,84
0.93
AFc
0.86
0.83
0.91
0.86
0.85
0.90
0.83
0.80
0.87
0.78
0.75
0.82
4
0.56
0.52
0.59
0.56
0.53
0.55
0.57
0.52
0.60
0.57
0.53
0.60
Artn
AFC
0.59
0.57
0.63
0.59
0.55
0.62
0.50
0.50
0.50
0.55
0.52
0.52
2
0,55
0.50
0.54
0.50
0.50
0.50
0.51
0,50
0,50
0,50
0,50
0.52
Artn
Table Average Accuracies, F1, and AUCs attacks under RAPPOR defense.
BERT
RoBERTa
DistilBERT
GPT1
Method
ACC
F1
AUC
ACC
F1
AUC
ACC
F1
AUC
ACC
F1
AUC
AB
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.96
0.96
0.99
1.00
1.00
1.00
1.00
1.00
1.00
Artn
AFc
0.98
0.98
1.00
0.98
0.98
1.00
0.96
0.96
0.99
0.98
0.97
1.00
8
A
0.73
0.69
0.79
0.52
0.49
0.54
0.79
0.77
0.86
0,66
0.61
0.72
APc
0.81
0.79
0.89
0.88
0.87
0.94
0.73
0.72
0.80
0.76
0.73
0.83
6
A
0.54
0.50
0.56
0.53
0.50
0.51
0.55
0.51
0.56
0.52
0.50
0.53
APC
0.61
0.59
0.68
0.66
0.65
0.70
0.56
0.54
0.64
0.68
0.66
0.77
AP
0.50
0.50
0.50
0.50
0.50
0.52
0.50
0.50
0.50
0.50
0.50
0.50
AFc
0.61
0.58
0.67
0.58
0.56
0.58
0.60
0.58
0.65
0.61
0.60
0.60
2
Artn
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
H.3. ROC analysis
We also conduct an ROC analysis of the attack success rates (on IMDB dataset). GRR shows the worst privacy with attack AUCs 0.946 (s 6) and 1.0 (s 8), while RAPPOR and dBitFlipPM provide stronger protection-achivviinn near-random performance at and moderate resistance at. Zoomed-in plots show that GRR leaks sensitive signals even at low FPRs and high TPRs, while RAPPOR and dBitFlipPM maintain partial robustness in these critical regions. Details are given in Fig. 16.
I. Alternative privacy-preserviee techniques beyond LDP
Other works have pursued cryptography- based techniques such as secure multi-party computation (SMPC) or homomorphic encryption (HE) that preserve privacy without adding excessive noise, hence preserving utility. SMPC-hased FI. systems
29

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
Table 6. Average Accuracies, F1, and AUCs of AMI attacks under dBin defense.
BERT
RoBERTa
DistilBERT
GPTI
Method
ACC
F1
AUC
ACC
F1
AUC
ACC
F1
AUC
ACC
F1
AUC
AFC
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
00
A
1.00
1.00
1.00
0.96
0.96
0.99
1.00
1.00
1.00
1.00
1.00
1.00
AFC
0.96
0.96
0.99
0.98
0.98
1.00
0.96
0.95
1.00
0.98
0.97
0.99
8
A
0.76
0.72
0.84
0.65
0.59
0.73
0.76
0.72
0.84
0.65
0.59
0.73
AFc
0.79
0.77
0.87
0.84
0.82
0.92
0.69
0.67
0.76
0.77
0.75
0.82
6
A
0.55
0.50
0.55
0.53
0.50
0.51
0.55
0.50
0.59
0.52
0.50
0.53
AFc
0.65
0.65
0.74
0.69
0.68
0.75
0.50
0.50
0.51
0.65
0.63
0.68
Artn
0.50
0.50
0.50
0.50
0.50
0.52
0.50
0.50
0.50
0.50
0.50
0.50
AFC
0.61
0.59
0.64
0.56
0.53
0.59
0.59
0.56
0.59
0.63
0.61
0.67
2
A
Astn
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
a) FPR vs TPR with AUC scores for Attention-based AMI attack against different LDP mechanisms at epsilon
3.0
1.0
0.00
b) FPR vs TPR with AUC scores for Attention-based AMI attack against different LDP mechanisms at epsilon 8
Figure 16. ROC analysis of Attention-based AMI on LDP prottccet IMDB dataset.
offer strong privacy guarantees by ensuring no party learns individual updates via secure aggregation protocols (Ma et al., 2023; Bonawitz et al., 2017), but they incur high communication overhead, especially the number of participants grows. On the other hand, HE provides end-to-end encryption in FL, allowing computations directly encrypted data, but it introduces significant computational costs due to the complexity of cryptographic operations (Nguyen & Thai, 2023; Pan et al., 2024). Furthermore, encryption alone does not protect against inference from the final global model: if an adversary obtains the trained model, they could still perform membership inference or other attacks. Recent works combine LDP and HE/SMPC, potentially providing a comprehensive solution (Aziz et al., 2023). Additionally, we note that due to the high communication and computation overhead, secure aggregation might not be feasible in certain FL applications.
It is worth noting that secure aggregation (e.g., Secure Multi-Party Computation, Homomorphic Eneryption) and Local Differential Privacy (LDP) are orthogonal research directions. SMPC/HE primarily aims to conceal local gradients from the server during aggregation, ensuring no individual gradient is exposed. I.DP focuses on preventing local gradients from revealing membership information about specific data points by adding noise to the local data/ gradient.
Our threat model assumes an actively dishonest server that has access to the local gradients of clients, and conducts the proposed AMI attacks based on the local gradients. Even though using SMPC or HE could potentially conceal such info from
30

---

Theoretically Unmasking Inference Attack Against LDP-Protected Clients in Federated Vision Models
the server, previous research has shown that actively dishonest adversary can circumvent secure aggregation protocols (Ngo et al., 2024; Gao al., 2021; Liu et al., 2023; Pasquini al., 2022; Kariyappa al., 2023; Nguyen al., 2022) to reconstruct the targeted client's gradients. Hence, even with secure aggregation, our threat model is still applicable when the server uses the above attacks to get around it and access local gradients before conducting the AMI attacks. Therefore, assuming that the dishonest server successfully circumvents secure aggregation, such techniques as SMPC or HE do not impact the success rates or the theoretical analysis of our proposed AMI attacks.
31


Reviews:
[{"review_id": "8FMH6VHIyL", "reviewer": "Reviewer_eoeX", "summary": "The paper \"Theoretically Unmasking Inference Attacks Against LDP-Protected Clients in Federated Vision Models\" investigates the vulnerability of federated learning (FL) systems, particularly those protected by Local Differential Privacy (LDP), to Active Membership Inference (AMI) attacks. The authors derive theoretical lower bounds for the success rates of low-polynomial-time AMI attacks that exploit vulnerabilities in fully connected (FC) layers and self-attention mechanisms in vision models like ResNet and Vision Transformers (ViTs). The paper demonstrates that even with LDP protection, privacy risks persist depending on the privacy budget, and the noise required to mitigate these attacks significantly degrades model utility. The authors provide both theoretical analysis and practical evaluations, confirming that AMI attacks can achieve high success rates even under stringent LDP protection.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## #1\n`Have the authors considered other privacy-preserving techniques, such as secure multi-party computation (SMPC) or homomorphic encryption, and how they might impact the success rates of AMI attacks?`\n\nWe thank reviewer eoeX for their insightful comments. First, we’d like to clarify that secure aggregation (e.g., Secure Multi-Party Computation, Homomorphic Encryption) and Local Differential Privacy (LDP) are orthogonal research directions. SMPC/HE primarily aims to conceal local gradients from the server during aggregation, ensuring no individual gradient is exposed. LDP focuses on preventing local gradients from revealing membership information about specific data points by adding noise to local data gradients.\n\nOur threat model assumes an actively dishonest server who has access to the local gradients of clients, and conducts the proposed AMI attacks based on the local gradients. Even though using SMPC or HE could potentially conceal such info from the server, previous research has shown that an actively dishonest adversary can circumvent secure aggregation protocols [1,2] to reconstruct the targeted client's gradients. Hence, even with secure aggregation, our threat model is still applicable when the server uses the above attacks to get around it and access local gradients before conducting the AMI attacks. Therefore, assuming that the dishonest server successfully circumvents secure aggregation, such techniques as SMPC or HE do not impact the success rates or the theoretical analysis of our proposed AMI attacks.\n\n## #2\n`The paper could benefit from a broader discussion of alternative privacy-preserving techniques beyond LDP`\nThanks for the suggestion. Due to the character limit in the response, we will include a discussion on secure aggregation and adversarial robustness in the revised manuscript.\n\n## #3\n`The impact of different LDP mechanisms (e.g., BitRand vs. OME) on the attack success rates could be explored in more depth, as the current analysis focuses primarily on the theoretical bounds.`\n\nTo further explore the impact of different LDP mechanisms on the attack's success rate, in addition to BitRand and OME, we have conducted extra experiments on 3 other LDP algorithms, namely GRR [3], RAPPOR [4] and Microsoft's dBitFlipPM [5]. In short, we see that privacy risks persist across all tested LDP mechanisms for both FC-based AMI and Attention-based AMI, depending on the privacy budget. \n\nWe compare the attack success rate of AMI-FC across three different datasets and five distinct LDP mechanisms in this anonymized imgur link https://imgur.com/cxtqVHv. We also plot the success rate of AMI-FC w.r.t LDP mechanisms against the theoretical upper/lower bounds and privacy-utility trade-off at https://imgur.com/pZfx3Y3. \n\nFor Attention-based AMI, we plot the result w.r.t. the 3 new LDP mechanisms at https://imgur.com/aDptuQq. In addition to vision datasets, we have extended our experiments to NLP datasets. The reviewer can refer to Response #2 to Reviewer x8f7 for detailed results. To explore more in depth the impact of different LDP mechanisms on the attack success rates, we also conduct an ROC analysis of the attack success rates (on IMDB dataset). The results are posted in https://imgur.com/l3Byt79. GRR shows the worst privacy with attack AUCs of 0.946 ($\\epsilon=6$) and 1.0 ($\\epsilon=8$), while RAPPOR and dBitFlipPM provide stronger protection—achieving near-random performance at $\\epsilon=6$ and moderate resistance at $\\epsilon=8$. Zoomed-in plots show that GRR leaks sensitive signals even at low FPRs and high TPRs, while RAPPOR and dBitFlipPM maintain partial robustness in these critical regions.\n\n## #4\n`Could the authors provide more details on how`\n\nUnfortunately, this question seems to have been cut off. Could the reviewer clarify what specific information about $\\beta$ they would like more details on? In the meantime, we would like to reiterate that $\\beta$ controls the extent to which the attention heads memorize the target pattern. Larger $\\beta$ values can negatively impact the attack's success rate (Figure 9) against LDP-protected data. Further explanations are provided in Remark 6. However, $\\beta$ also needs to be sufficiently large to satisfy the condition in Equation 5.\n\nWe hope our responses have addressed your questions sufficiently. We are happy to discuss further if you have follow-up questions for us.\n\n---\n\n[1] Dario Pasquini el al. Eluding secure aggregation in federated learning via model inconsistency. CCS 2022.\n\n[2] Sanjay Kariyappa et al. Cocktail party attack: Breaking aggregation-based privacy in federated learning using independent component analysis. ICML 2023\n\n[3] Arijit Chaudhuri and Rahul Mukerjee. Randomized response: Theory and techniques. Routledge, 2020.\n\n[4] Ulfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. Rappor: Randomized aggregatable privacy-preserving ordinal response. CCS 2014\n\n[5] Bolin Ding et al. Collecting telemetry data privately. NeurIPS 2017", "claims_and_evidence": "The claims made in the paper are supported by clear and convincing evidence. The authors provide theoretical proofs for the lower bounds of attack success rates under LDP protection (Theorems 1, 2, and 3) and validate these claims through extensive experiments on synthetic and real-world datasets (CIFAR10 and ImageNet). The experimental results align with the theoretical predictions, showing that AMI attacks can achieve high success rates even when LDP is applied, especially for smaller privacy budgets.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are appropriate for the problem at hand. The authors focus on two types of AMI attacks: one exploiting FC layers and another exploiting self-attention mechanisms in transformer-based models. The evaluation is conducted on both synthetic datasets (one-hot and spherical data) and real-world datasets (CIFAR10 and ImageNet), using state-of-the-art models like ResNet and ViTs. The use of LDP mechanisms (BitRand and OME) is well-justified, and the evaluation criteria (attack success rates and model utility degradation) are relevant to assessing the trade-off between privacy and utility in FL systems.", "theoretical_claims": "The theoretical claims are well-supported by detailed proofs provided in the appendices. The authors derive lower bounds for the success rates of AMI attacks under LDP protection (Theorems 1 and 3) and an upper bound for the advantage of the adversary (Theorem 2). The proofs are rigorous and rely on established concepts in differential privacy and attention mechanisms. The theoretical analysis is a significant contribution, as it provides a formal understanding of the vulnerabilities in LDP-protected FL systems.", "experimental_designs_or_analyses": "The experimental designs and analyses are sound and well-executed. The authors conduct experiments on synthetic and real-world datasets, using both FC-based and attention-based AMI attacks. The results are consistent with the theoretical predictions, showing that AMI attacks can achieve high success rates even under LDP protection, especially for smaller privacy budgets. The authors also explore the impact of hyperparameters (e.g., β) on the attack success rates, providing additional insights into the robustness of the attacks.", "relation_to_broader_scientific_literature": "The paper builds on prior work in federated learning, differential privacy, and membership inference attacks. It extends the theoretical understanding of AMI attacks in FL systems, particularly under LDP protection, which has not been extensively studied in prior literature. The authors reference relevant works on LDP, FL, and AMI attacks, and their contributions are well-situated within the broader context of privacy-preserving machine learning.", "essential_references_not_discussed": "The paper adequately covers the relevant literature, but it could benefit from a discussion of recent advancements in privacy-preserving techniques beyond LDP, such as secure multi-party computation (SMPC) or homomorphic encryption, which are also used in FL systems. Additionally, the paper could discuss recent work on adversarial robustness in FL, as this is closely related to the problem of inference attacks.", "other_strengths_and_weaknesses": "Strengths:\n* The paper provides a rigorous theoretical analysis of AMI attacks under LDP protection, which is a significant contribution to the field.\n* The experimental results are comprehensive and validate the theoretical claims, demonstrating the practical implications of the findings.\n* The paper addresses an important gap in the literature by focusing on the vulnerabilities of LDP-protected FL systems, which are often assumed to be secure.\n\nWeaknesses:\n\n* The paper could benefit from a broader discussion of alternative privacy-preserving techniques beyond LDP, such as SMPC or homomorphic encryption, to provide a more comprehensive view of the privacy-utility trade-off in FL systems.\n* The impact of different LDP mechanisms (e.g., BitRand vs. OME) on the attack success rates could be explored in more depth, as the current analysis focuses primarily on the theoretical bounds.", "other_comments_or_suggestions": "None", "questions_for_authors": "1. The paper focuses on LDP as the primary privacy-preserving mechanism. Have the authors considered other privacy-preserving techniques, such as secure multi-party computation (SMPC) or homomorphic encryption, and how they might impact the success rates of AMI attacks?\n2. The paper discusses the impact of the hyperparameter β on the success rates of attention-based AMI attacks. Could the authors provide more details on how", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QrhD7OF7lc", "reviewer": "Reviewer_x8f7", "summary": "This paper examines the privacy risks posed by Active Membership Inference (AMI) attacks against\nfederated learning (FL) clients even when their data is protected by Local Differential Privacy (LDP). The\nauthors derive theoretical lower and upper bounds for the success rates of low-polynomial-time attacks\nexploiting fully connected layers and self-attention layers and demonstrate that even under LDP, privacy\nrisks persist depending on the privacy budget.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## #1\n-  `evaluation is only limited to certain types of attacks and 2 LDP mechanisms`\n- `the study only considers noise added directly to the data, while other common LDP approaches are perturbing the gradients before aggregation.`\n\nWe would like to reiterate that our **theoretical** analysis applies to **all** LDP algorithms that add noise to clients' data in general, while our experiments were initially conducted using OME and BitRand. We have further conducted extra experiments using 3 other LDP algorithms, namely GRR, RAPPOR and Microsoft’s dBitFlipPM. Detailed results using these LDP algorithms are given in response #3 to Reviewer eoeX. \n\nRegarding LDP approaches that perturb gradients before aggregation, the current attack can be extended to bypass these mechanisms. First, we want to emphasize that the proposed AMI attacks hinge on the fact that the adversarial server can distinguish between non-zero gradients and zero-gradients of the targeted neurons. With gradient perturbation methods, noise (mostly Gaussian noise) is added to the clients' gradients before they are sent to the server, initially hindering the server from accessing the true value of the targeted neuron's gradient. \n\nHowever, previous papers [1,2] have shown that attackers can exploit knowledge of the noise distribution (e.g., Gaussian with known $\\sigma$) to statistically learn the true gradient values. For instance, [1] leveraged the fact that the FL training is done in multiple iterations and the zero-mean property of Gaussian noise to average out noise samples across multiple iterations, effectively canceling the noise and revealing the true gradient values. This aligns with the central limit theorem and is consistent with the privacy composition of DP, as the privacy budget accumulates with the number of FL iterations [3]. By learning the true gradient values, the attacker can now distinguish between zero and non-zero gradients, effectively inferring whether the target sample was included in any of these iterations by applying the proposed AMI attacks. \n\nTo counter such an attack, the privacy budget could be set to account for the number of iterations, however, that would result in a relatively small budget [3], making it even more difficult to have good training performance.\n\nFurthermore, theoretically analyzing membership inference attacks w.r.t adding noise to gradient has been extensively studied [4,5]. Instead, our work is the first to investigate the theoretical bound on the impact of LDP on client data.\n\n## #2\n`can the proposed analysis also be applied to AMIs on NLP data`\n\nFirst, we would like to emphasize that our theoretical analysis for FC-based AMI adversary also translates to NLP (discrete) data. However, the reason why we decided to limit our theoretical analysis to vision data is due to the way we formulate the LDP noise in the Attention-based AMI adversary. In particular, the distortion imposed by LDP is modeled by a noise $r_i$ added to each pattern $x_i$ (line 247, right column). In our analysis, we assume $x_i$ and $r_i$ to be continuous, and the impact of LDP noise is visualized in Fig. 4. For NLP data, both the data and the noise should be modeled as discrete, hence our theoretical analysis might not directly apply to the NLP scenario. The key challenge is that tokens are typically represented as discrete embeddings, and adding continuous noise is not meaningful in this context. \n\nHowever, we note that the attacks still experimentally work against both vision and NLP data. We have conducted comprehensive experiments across 4 NLP datasets (IMDB, Yelp, Twitter, Finance), 4 models (BERT, RoBERTa, GPT-1, DistilBERT), and 3 LDP algorithms (GRR, RAPPOR, dBitFlipPM). The results indicate that privacy risks persist even for large language models (LLMs), depending on the privacy budget (https://imgur.com/a/Wt1nlno).\n\n## #3\n`no Impact Statements in the paper`\n\nThanks for the comment. Due to the character limit, we will include an impact statement in the revised manuscript.\n\n## #4\n`Figure 9 appears to be unclear or possibly incorrect`\n\nWe have mistakenly put that y-axis there, thanks for pointing this out. The correct figure can be found at https://imgur.com/nBpVEbB\n\nWe hope our responses have addressed your concerns sufficiently, and we are happy to address any follow-up questions you might have for us.\n\n---\n\n[1] Nguyen et al. Active membership inference attack under local differential privacy in federated learning. AISTATS 2023.\n\n[2] Hu et al. Does differential privacy really protect federated learning from gradient leakage attacks? IEEE TMC, 2024.\n\n[3] Naseri et al. Local and central differential privacy for robustness and privacy in federated learning. NDSS 2022\n\n[4] Thudi et al. From differential privacy to bounds on membership inference: Less can be more. TMLR 2024.\n\n[5] Yeom et al. Privacy risk in machine learning: Analyzing the connection to overfitting. CSF 2018.", "claims_and_evidence": "Claims and evidence are sound, but the evaluation is only limited to certain types of attacks and 2 LDP\nmechanisms.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are overall sound.", "theoretical_claims": "The overall theoretical proof in the appendix seems sound.", "experimental_designs_or_analyses": "The experimental designs and analyses are sound, and the results align with the theoretical analysis.", "relation_to_broader_scientific_literature": "The paper is primarily based on the analysis of [1] and [2].\n\n[1] Vu, Minh, Truc Nguyen, and My T. Thai. &quot;Analysis of privacy leakage in federated large language\nmodels.&quot; International Conference on Artificial Intelligence and Statistics. PMLR, 2024.\n\n[2] Ramsauer, Hubert, et al. &quot;Hopfield networks is all you need.&quot; arXiv preprint\narXiv:2008.02217 (2020).", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "**Strengths**\n1. The paper rigorously derives lower and upper bounds for the success rates of AMI attacks, providing\na mathematical basis for evaluating privacy risks under LDP.\n2. The empirical results align with the theoretical analysis.\n\n**Weaknesses**\n1. The paper primarily evaluates its method under BitRand and OME, but the impact of other LDP\nmechanisms remains unexplored. In particular, the study only considers noise added directly to the\ndata, while other common LDP approaches are perturbing the gradients before aggregation. A\ndiscussion or experimental comparison would help clarify this aspect.\n2. The paper extends the work of [1], which focuses on AMI attacks against LLMs for text data, yet it\nonly evaluates attacks on vision data. Given that the proposed theoretical analysis does not appear\nto be domain-specific, it seems likely that it could also apply to text-based FL models. Can the authors\nclarify why they chose to restrict their evaluation to image data? Providing experimental results on\nNLP datasets would further strengthen the generalizability of the findings.\n3. There is no Impact Statements in the paper.\n\n[1] Vu, Minh, Truc Nguyen, and My T. Thai. &quot;Analysis of privacy leakage in federated large language\nmodels.&quot; International Conference on Artificial Intelligence and Statistics. PMLR, 2024.", "other_comments_or_suggestions": "See Weaknesses and Questions for Authors.", "questions_for_authors": "1. What is the effectiveness of the proposed method for different LDP mechanisms?\n2. Why do the authors only investigate vision data? Can the proposed analysis also be applied to AMIs\non NLP data since the adopted attacks are directly evaluated on text modality? Additional\nexperiments on NLP datasets would provide stronger evidence supporting the proposed analysis.\n3. Figure 9 appears to be unclear or possibly incorrect. The right y-axis is labelled for model accuracy,\nbut there is no corresponding line representing the model accuracy in the figure. Can the author\nprovide further clarification on this issue?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HSh0oJtdjA", "reviewer": "Reviewer_exxM", "summary": "This paper studies the membership inference risk for local differential-privacy (LDP) protected clients in the presence of dishonest servers who can actively manipulate the model parameters. The paper provides theoretical upper and lower bounds for the success rates of low-polynomial-time membership attacks. It also extends a prior attack to the continuous domain of Vision Transformer models. Experiments show that the proposed attack can achieve high success rate under LDP protection.\n\n### Update after rebuttal\n \nI’m satisfied with the author’s response and will maintain my weak accept rating. I’d like to note that this paper is highly theoretical and falls outside my core area of expertise, so I would prefer that greater weight be given to the assessments of the other reviewers.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive feedback that our research is well-motivated and the provided theoretical bounds are useful for quantifying the risk of membership inference.\n\n## #1\n`Theoretical Claims: No`\n\nWe see that the reviewer indicates we do not have theoretical claims. However, we'd like to clarify that we do provide many theoretical results in the paper, as noted in your Summary section (\"The paper provides theoretical upper and lower bounds for the success rates of low-polynomial-time membership attacks\"). Several theoretical proofs are presented throughout the paper and in the appendix. This is also agreed on by other reviewers.\n\n## #2\n`The problem setting used involves an active adversary who can manipulate the model weights. Extending this to include honest-but-curious adversaries would help improve the scope of the paper.`\n\nWhile we agree that it is interesting to see how much the assumption of having a malicious server affects the results compared to the honest-but-curious, we remain focused on the active setting due to the following reasons:\n\n- The honest-but-curious threat model assumes that the server still abides by the system protocol. This does not convey the true capability of the attacker and undermines the vulnerability of the FL system. The active adversary model is more practical because, in practice, the server can deviate from the protocol to strengthen the privacy attacks [1-4].\n- The honest-but-curious adversary model has been extensively studied and those studies achieve much lower success rates [5-7] on protected data compared to active attacks [8,9,2]. We believe this emphasizes the point that an active server introduces a much higher privacy risk, motivating the need for a more robust defense.\n- In order to tackle the more potent active threat, we intentionally focus on active adversaries (a malicious server who manipulates model weights) because this scenario represents an underexplored and more realistic threat in the federated learning literature.\n\n\nWe appreciate the time and effort you have dedicated to reviewing our paper. We'd be happy to discuss further should you have any other concerns that could potentially impact your rating.\n\n---\n\n[1] Nguyen et al. Blockchain-based secure client selection in federated learning. IEEE ICBC 2022\n\n[2] Nguyen et al. Active membership inference attack under local differential privacy in federated learning. AISTATS 2023\n\n[3] Boenisch et al. When the curious abandon honesty: Federated learning is not private. IEEE EuroS&P 2023.\n\n[4] Fowl et al. Robbing the fed: Directly obtaining private data in federated learning with modified models. ICLR 2022\n\n[5] Carlini et al. Membership inference attacks from first principles. IEEE SP 2022\n\n[6] Ye et al. Enhanced membership inference attacks against machine learning models. CCS 2022.\n\n[7] Jayaraman et al. Evaluating Differentially Private Machine Learning in Practice. USENIX Security 2019\n\n[8] Vu et al. Analysis of privacy leakage in federated large language models. AISTATS 2024\n\n[9] Nasr et al. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. IEEE SP 2019.", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "yes", "theoretical_claims": "No", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "This paper provides theoretical bounds to quantify the privacy risks of membership inference under an active adversary.", "essential_references_not_discussed": "-", "other_strengths_and_weaknesses": "## Strengths\n\n1. The motivation is explained well.\n2. The paper provides theoretical bounds for attacks, providing a useful tool for quantifying the risk of membership inference. \n3. Experiments show that even under LDP protection, models are susceptible to membership inference attacks.\n\n## Weaknesses\n1. The problem setting used involves an active adversary who can manipulate the model weights. Extending this to include honest-but-curious adversaries would help improve the scope of the paper.", "other_comments_or_suggestions": "-", "questions_for_authors": "-", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper studies Membership Inference Attacks  (MIA) in federated learning (FL) settings, regardless of the local differential privacy  (LDP) mechanism used. The main contribution is deriving lower bounds for the success rates of low-polynomial-time MIAs targeting fully connected and self-attention layers, showing that such attacks can succeed under LDP protection depending on the privacy budget. These theoretical results are supported by practical experiments on ResNet and Vision Transformers models.

Reviewers unanimously appreciated the paper’s rigorous theoretical contribution and its alignment with empirical results. The derivation of upper and lower bounds (along with the evaluation on both synthetic and real-world datasets) is appreciated by the reviewers. Multiple reviewers raise concerns about the narrow evaluation scope—particularly the focus on only two LDP mechanisms (BitRand and OME) and the absence of comparisons to other privacy-preserving approaches. In the rebuttal period, the authors included additional experiments.  There is also consensus that the experimental evaluation is limited to vision data, despite the theoretical results being generalizable to other domains such as NLP.
Decision: Accept (poster)