Theoretically Understanding Data Reconstruction Leakage in Federated Learning
Authors: Binghui Zhang
Zifan Wang
Meng Pang
Yuan Hong
Binghui Wang
Venue: Accepted by TMLR ()

Abstract: Federated learning (FL) is a collaborative learning paradigm that aims to protect data privacy. Unfortunately, recent works show FL algorithms are vulnerable to data reconstruction attacks (DRA), a serious type of privacy leakage. However, existing works lack a theoretical foundation on to what extent the devices' data can be reconstructed and the effectiveness of these attacks cannot be compared fairly due to their unstable performance. To address this deficiency, we propose a theoretical framework to understand DRAs to FL. Our framework involves bounding the data reconstruction error and an attack's error bound reflects its inherent effectiveness using Lipschitz constant. We show that a smaller Lipschitz constant indicates a stronger attacker. Under the framework, we theoretically compare the effectiveness of existing attacks (such as DLG and iDLG). We then empirically examine our results on multiple datasets, validating that the iDLG attack inherently outperforms the DLG attack.

Published in Transactions on Machine Learning Research (01/2026)
Theoretically Understanding Data Reconstruction Leakage in Federated Learning
Binghui Zhang
Deportment of Computer Science Ilnois Institute of Technology
Zifan Wang
zwang248 wwwioosstehiiii..momomoo edu
Department of Computer Science Mlkoois Institute of Technology
Meng Pang
menggunng@hcu.edu.com
School of Mathematics and Computer Sciences Nanchang University
Yuan Hong
yua.. www..oong@ucomne edu
School of Computing University of Connecticul
Binghui Wang
busnng700kklinooistchh.tt
Deportment of Computer Science Illinois Institute of Technology
Reviewed OpenReview: /ttps://openrevieee net/ forum? id= 1UfDXe Yowk
Abstract
Federated learning (FL) is collaborative learning paradigm that aims to protect data privacy. Unfortunately, recent works show FL algorithms vulnerable to data reconstruction attacks (DRA), serious type of privacy leakage. However, existing works lack theoretical foundation on to what extent the devices' data be reconstrueted and the effectiveness of these attacks cannot be compared fairly due to their unstable performance. To address this deficiency, we propose a theoretical framework to understand DRAs to FL. Our framework involves bounding the data reconstruction error and an attack's error bound reflects its inherent offectiveness using Lipschitz constant. We show that smaller Lipschitz constant indicates a stronger attacker. Under the framework, we theoretically compare the effectiveness of existing attacks (such DLG and iDLG).. We then empirically examine results multiple dattsset, validating that the iDLG attack inherently outperforms the DLG attack.
1 Introduction
Federated learning (FL) (MeMahan et al. 2017 has been great potential to protect data privacy. In FL, the participating devices keep and train on their data locally, and only share the trained models, instead of the raw data, with a central server (c.g., cloud). The server updates its global model by aggrogating the received device models, and broadcasts the updated global model to all participating devices such that
all devices indirectly use all data from other devices. FL has been deployed by many companies (Google Federated Learning 2022 Microsoft Federated Learning 2022 IBM Federated Learning 2022), and applied
various privacy-sensitaee applications, including content suggestions for on-device keyboards (Bonawitz
et al. 2019 health monitoring (Rieke et al. 2020), and medical imaging (Kaissis et. al. 2020

---

Published in Transactions on Machine Learning Research (01/2026)
700
DLG-mean
500
DLG mean
DLG-STD
DLG-STD
Figure 1: Impact of the initial parameters of a Gaussian distribution on the DRA performance. The x-axis marks the mean (standard deviation) of the Gaussian. The high MSE makes it difficult. to conclude statistical significance in the attack outcome with empirical analysis along.
400
200
100
1. (101 2 112)
3. 1240
(26)
5 115)
Unfortunately, recent works show that, even when device models are being shared, it is still possible for an adversary (e.g,, the malicious server) to perform the severe data reconstruction attack (DRA) on FL ZZhu
et al.
2019) where an adversary could directly reconstruct the device's training data via the shared device
models. Later, a number of enhanced attacks (Hita.) et al. 2017 Wang et al.) 2019
Zhao et al. 2020) Wei
ct al.
2020
Yin
2021 Jcon et al.
2021 Zhu Blaschko
2021
Dang et al.
2021
Balunovic et al.
2022
et al. 2022
Fowl al., 2022
Wen el al.
2022 Haim al., 2022 Wu et al.
2023
Noorbakhsh et al.
2024
are proposed by either incorporating some (known unrealistic) prior knowledge or requiring
auxiliary dataset to simulate the training data distribution.
However, we note that existing DRA methods have several limitations: First, they are sensitive to initialization (which is also observed in (Wei et al., 2020))- )- For example, we show in Figure 1 we show that the attack performance of iDLG (Zhao et al. 2020) and DLG (Zhu et al. 2019) are significantly influenced by initial Gaussian distribution, from which the initial data is sampled, thus making it difficult to conclude empirically that the two attacks boast statistically significant. difference in attack outcomes. Second, existing DRAs mainly show comparison results a FL model at snapshol, which cannot reflect allacks' true effectiveness. As FL training is dynamic, an advensary can perform attacks in any stage of the training. Hence, Attack A shows better performance than Attack B at a snapshot docs not imply A is truly more effective than B. Third, worse still, they lack a thcoretical understanding on to what extent the training data can be reconstructed. These limitations make existing DRAs not be easily and fairly compared and hence is unknown which attacks are inherently more effective.
parameters (i.e., the mean and standard deviation)
In this paper, aim to bridge the gap and propose thcoretical framework to understand DRAs to FL. Specifically, our framework bounds the error bet ween the privale data and the reconst counterpart in the whole FL training, where an attack's (smaller) error bound reflects its inherent (better) attack effectiveness. Our theoretical results show that when an attacker's reconstruction function has a smaller Lipschitz constant, this attack intrinsically performs better"] Under the framework, we can theoretically compare the existing DRAs by directly comparing their bounded errors. We test our framework on multiple attacks and benchmark datascts. E.g., our results show InvGrad (Geiping et al., (020) performs better than DLG (Zhu et al., 2019 and iDLG (Zhao al.] 2020) on complex datasets, while iDLG is comparable or slightly better than DLG.
2 Preliminaries and Problem Setup
2.1 Federated Learning (FL)
The FL paradigm enables server to coordinate the training of multiple local devices through multiple rounds of global communications, without sharing the local data. Suppose there are N devices and central server participating in FL. Each k-th device owns a training dataset D* = ((x},xf 3 with ni samples, and each sample x5 has label v/. FL considers the following distributed optimization problem:
Under realistic attack scenarios where an ideal model exists. For instance, constant function has L O. but will have high first term in 3 the ideal rectnotruction function will have extremely high MSF.
2

---

Published in Transactions on Machine Learning Research (01/2026)
min L(w) Σ N
k=1 Ph (w))
(1)
where 2 0 is the weight of the k-th device and Σ 1; and the k-th device's 's local objective is:
(w)
Σ e(w; (xfx vf))
with E(:3)) an algorithm-dependent loss function.
FedSGD is the de facto FI, algorithm to solve Equation in an iterative way. In each communication round, each k-th device only shares the gradients W (w) instead of the raw data D to the server McMahan et al.
(2017). Specifically, in the current round I, each k-th device first downloads the latest global model (denoted as wi-1) from the server and initializes its local model as w w 1: then it performs (e.g., E') local SGD updates as bclow:
wits <- w wits-1 - VL. (W 1, E
where met is the learning rate and $t4, is sampled from the local data D& uniformly at random.
Next, the server updates the global model w, by aggregating full or partial device models. The final global model downloaded by all devices for their learning tasks.
Full device participation. Tt requires all device models for aggregation, and the server performs w. au and w = = Note that full device participation means the server Ex Pk w with ps: Ee n.
must wait for the slowest devices, which is often unrealistic in practice.
Partial device participation. This is more realistic setting as it does not require the server to know all device models. Suppose the server only needs K (< N) device models for aggregation and discards the remaining ones. Let S; be the set of K chosen devices in the t-th iteration. Then, the server's aggregation step performs w, + N Lkes, Pk w with w = w
K
Quantifying the degree of non-IID (heterogeneity)): Real-world FL applications often do not satisfy the III) assumption for data among local devices. Li et al. (2020) proposed way to quantify the degree of non-llD. Specifically, let L and L7 be the minimum values of L and Ls, respectively. Then, the term T= L* -- -h goes zero as the number of samples grows. If the data are non-llD, then I nonzero, and its magnitude reflects the heterogeneity of the data distribution.
used for quantifying the degree of non-IID. If the data are IDD, then obviously
2.2 Optimization-basel DRAs to FL
Existing DRAs assume an honest-but-curion server, i.c., the server has access all device models in all communication rounds, follows the FL protocol, and aims to infer devices' private data. Given the private data E [0, 1jd with private label A we denote the reconstructed data by malicious server as (X, g) R(w,), where R(-) indicates data reconstruction function, and w, can be any intermediate global model. Modern optimization poeimization-based DRAs use different R(-), but mainly based on gradient matching. Specifically, they solve the below optimization problem:
R(w,) = arg min GML(gw. (x, 9).9w. 1w. (x',8')) + XRog(x') x'E
where we denote the gradient of loss w.r.l. (x, be Sw. (x, 1) L(we (x, u)) for simplicity of notation. GMT(ss)) means the gradient matching loss (i.e., the distance between the real gradients and estimated gradients) and Reg(-) an auxiliary regularizer for the reconstruction. Here, we list GML(--) and Reg(-) for three representative DRAs, and more attacks are shown in Appendix D.1.2
2This can he single data sample or a batch of data samples.
3

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm Iterative solvers for oplimization-basedd DRAs
Input: Model parameters Wi; true gradient g(x, u); A,
Output: Reconstructed data &.
1: Initialize dummy input(ss xy from Gaussian distribution, and dummy label(s) vó
2: for = 0 < I;
3: [(x') = GML (x, v), Iw. (x), + AReg(x!)
4: x(+1 + SGD(x 8¹) X -
5: x = Clip(x(41,00 0,
6: end for
7: return x/
2019 uses the mean squared error as the gradient matching loss, ie., DLG (Zhu et al.
GML (x. y), Iw. (x', y²)) 119 (x,y) Iw. (x', 1/)l2 and does not use regularizer.
iDLG (Zhao et al. 2020 estimates the label y before solving Equation 2 iDLG solves & =
arg miny Ex (ML((w.. (x. 3/, (x', ()) + AReg(x')), assuming the estimated label is 9, where uses the same G-I.()) as DLG and also does not have regularizer.
InvGrad Geiping et al. 2020 improves upon DLG and iDLG. It first estimates the private label as in advance. Then it uses a negative cosine similarity as GML(-) and a total variation regularizer Regry(-) as (sw, (x.g). (x'99) ARegry(x)).
an image prior. Specifically, InvGrad solves for & = arg ming Ex[1 Tx,, 1lww Mll
Algorithm shows the pseudo-code of iterative solvers for DRAs and Algorithm 4 in Appendix shows more details for each attack. As the label y can be often accurately inferred, we now only consider reconstructing the data, x for notation simplicity. Then, the attack performance measured by the similarity sim(x, x) between and x. The larger similarity, the better attack performance. In the paper, we use the common similarity metric, i.e., the negative mean-square-error sim($, x) -Ell x||², where the expectation E considers the randomness during reconstruction.
3 A Theoretical Framework to Understand DRAs to Federated Learning
Though many DRAs to FL have been proposed, it is still unknown how to theoretically compare the effectiveness of existing attacks, as stated in Introduction. In this section, understand DRAs to FL from a theoretical perspective. We first derive a reconstruction error bound for convex objective losses. The error bound involves knowing the Lipschitz constant of the data reconstruction function. Directly caleulating the exact Lipschitz constant is computationally challenging. We then adapt existing methods to calculate its upper bound. We argue that an attack with smaller upper bound is intrinsically more effective. We also emphasize our theoretical framework is applicable any adversary who knows the global model during FL training (sce below Theorem 3.2 and Theorem 3.3
3.1 Bounding the Data Reconstruction Error
Given random data x from a device, our goal is to bound the common norm-basod reconstruction erro i.c., EE|xx R( 12 at any round i, where R(-) can be any DRA and the expectation considers the randomness in R(-), e.g., due to different initializations, Directly bounding this error is challenging because the global model dynamically aggregates local device models, which are trained by (stochastic) learning algorithm and whose learning procedure is hard to characterize during training. To alleviate this issue, we introduce the optimal global model w* that can be learnt by the FL algorithm. Then, we can bound the error as follows:
E| x - R[ E - R(w*) - R(w*) R(w,)12
< 2 ([ww)] (|((W*) R(w R(w.)ll).
TThe norm-based mean-square--rror (MSE) bound he easily generalized to the respective PSNR bound. This is becanse PSNR log (MSE). However, it unable generalixe to SSTM LPIPS since these metrics do not have an analytic form.
4

---

Published in Transactions on Machine Learning Research (01/2026)
X
SGD (::0f)
x xi+1
X1
X
Figure 2: Iterative solvers for DRÁs unrolled deep werd-forward networks. We map the i-th SGD iteration in DRAs into a single network layer, and stack layers to form an I-layer deep network. Feeding forward data through the I-layer network is equivalent to executing I SGD updates. The trainable parameters {0') colored in blue in Algorithm 1 and connects the i-th layer and i + 1-th layer. These parameters can be learned from intermediate reconstructed data x training the deep feed-forward network.
Note that the first term in Equation 3 is constant and can be directly computed under a given reconstruction function R(-) and convex loss used in FL. Specifically, if the loss in each device is convex, the global model can converge to the optimal w* based on theoretical results in (Li et al. 2020). Then per attack and compute the first term. In our experiments, we run the FL. algorithm until the loss difference between two consecutive iterations is less than 1e-5, and treat the final global model as w".
obtain R(w")
Now our goal reduces to bounding the second term. However, is still challenging without knowing any property of the reconstruction function R(-). In practice, we note R(-) is often Lipschitz continuous, which can be verified later.
Proposition 3.1. R(-) Lipsscchhttz continuous: there erists constant LR such that [[((v) R(w)ll LR|| v w/l. Vv, w. The smallest LR is colled the Lipschitz constant.
Next, we present our theoretical results and their proofs are seen in Appendix and Appendix
Theoretical results with full device participation: We first analyze the case where all devices participate in the aggregation on the server in each communication round. Assume the FedAvg algorithm stops after T iterations and returns WT as the solution. Let L, μr ok G' be defined in Assumptions A.1 A.4
more details in
Appendix and LR be defined in Proposition 3.1
Theorem 3.2. Let Assumptions A.1-A hold. Choose 7 max E} and learning rate ply+tj Let
B = Σx o + 6LT 8(E - G). Then, for any round t, FedAng with full device participation satisfies
E||x R(wx))1 2B| R(
AB 7 112
2L
+ (7+1)) - w²]le).
2 partial device participation is more practical. Recall that S, contains the K active devices the t-th itcration. To show our theoretical results, we require the K devices in St selected from distribution (e.g., Pl, P2. .PN)
Theoretical results with partial device participation: As discussed in Section
independently and with replacement, following (Sahu al. 2018 Li et al. 2020 Then FedAvg performs aggregation as W, t ExES, w
Theorem 3.3. Let Assumptions A.1 hold. Let 7, nhe and B be defined in Theorem 3.2 and define == $ G2 For any round i, FedAug with S, device parlicipalion salisfies
2L 7+1
4(B + +C) (7 + 1)
E|xx R(w.))| < 2EI RP
-)]lw] w²ll)).
(5)
n12
3.2 Computing the Lipschitz Constant for Data Reconstruction Functions
In this part, we show how to calculate the Lipschitz constant for data reconstruction function. Our idea is to first use the strong connection between optimizing DRAs and the corresponding unrolled deep neural networks; and then adapt existing methods to approximate the Lipschitz upper bound.
Iterative solvers for optimization- based DRAs as unrolled deep feod-forward networks: Recent. works Chen et al. (2018): Li al. (2019); Monga al. (2021) show strong connection between iterative algorithms and deep network architectures. The general idea is: starting with abstract iterative algorithm,
5

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm AutoLip
Require: function f and its computation graph (01...,, 9H)
Algorithm 3 Power method to calculate the matrix
lz-norm
Ensure: Lipechitz upper bound LAutaLip 2 l
Require: affine function f R" R",, #iterations Iter
1: Φo(x) = (x)) = f(x)
2: (n)xx = gh gx, 01 (x), "....,n- (x)-1(x)), Vh E [1
Ensure: Upper bound of the Lipschitz constant by
3: Z {(E0,...,.1) =1)) wh E [0, H), on is constant
1: for j = to Hter do
2n = )n(0))
2: t Vg(u) where g(x) = ]1l f(0)11
4: Lo
3: +- ||l||
5: for h to H do Σ
4: 4- v/A
6: LA +
max [8:gg( =E2
5: end for
6: return
= llf(2) f(0)ll2
7: end for
8: return LAutoLip = Lg
we map one iteration into a single network layer, and stack a finite number of (e.g,, H) layers to form a deep network, which is also called unrolled deep network. Feeding the data through an H H-layer network is hence equivalent to executing the iterative algorithm H iterations. The parameters of the unrolled networks are learnt from data by training the network in an end-to-end fashion. From Algorithu [] we can see the trajectory of an iterative solver for an optimization- based DRA corresponds to a customized unrolled deep feed-forward network. Specifically, we treat the initial x6 (and wx) as the imput, the intermediate reconstructed x/ as the i-th hidden layer, followed by a nonlincar clip function, and the final reconstructed data & = x, as the output of the network. Given intermediate {x(} with a set of data samples, we can train parameterized deep feed-forward networks (universal approximation) to fit them, e.g., via the greedy layer-wiss training strategy [Bengio et al. (2006), where each 8' mcans the model parameter connecting the i-th layer and i+ 1-th layer. Figure 2 visualizes the unrolled deep feed-forward network for the optimization-basee DRA.
Definition 3.4 (Deep Foed-Forward Network). An H -layyer fora-forward network is an function TMLP(x) Su PH-1 A1 fi fi(),, where Yh, the h-th hidden layer fa x + M2 + by is affine function and Ph. is a non-linear activation function.
Upper bound Lipschitz computation: |Virmaxx Scaman 2018 showed computing the exact Lipschitz constant for deep (even 2-layer) feed-forward networks is NP-hard. Hence, they resort to approximate computation and propose method called AutoLip to obtain upper bound of the Lipschitz constant. AutoLip relics on the concept of automatic differentiation Griewank & Walther (2008), a principled approach that computes differential operators of functions from consecutive operations through a computation graph. When the operations are all locally Lipschitz-continuous and their partial derivatives can be computed and maximized, AutoLip can compute the Lipschitz upper bound efficiently. Algorithm 2 shows the details.
With Autolip, Virmaux & Scaman (018)) showed that a feed-forwand network with 1-Lipschitz activation functions has an upper bounded Lipschitz constant below.
Lemma 3.5. For any -layer foed-fooward network with 1-Lipschitz activation functions, the AutoLip upper bound becomes LAutol.ip 11 Ma,]2,, where M, is defindd in Desinition 3.4
Note that a matrix l2- no-mo equals to its largest singular value, which could be computed efficiently via the power method (Mises Pollaczek- Geirnnnge 1929). More details shown in Algorithm 3 (A better estimation algorithm leads a tighter Lipschitz bound). The used Clip activation function is I-Lipschitz. Hence, by applying Lemma 3.5 to the optimization-based DRAs, we can derive an upper bounded Lipschitz LR.
4 Evaluation
4.1 Experimental Setup
We conduct experiments on three benchmark image datasets, ic., MNIST (LeCun 1998), Fashion-MNISTT(MNIIIII (Xiao et al.] 2017), and CIFAR10 (Krizhevsky et al.] 2009). We examine our theoretical results the FL algorithm that uses the fz-regularized logistic regression (Eg-TogReg) and
Datascts and models:
6

---

Published in Transactions on Machine Learning Research (01/2026)
CLG
Fugrad GGL
0.6
nvgrad
DXG
Invrrd
Upportouned 1100
20
20
200
1l020 1-1001
A.11000
110010
DUG
Ivgrad
DLG
Iwgrrad
DLG
Iwgra
oppcrbourd ktogs
11w1000 km21
IDLG
GGL
uppertoord Hog
Ilpertound
Ingradd
Iougrad
DLG
6CL
OLS
F16-101-1000
Figure 3: Results of federated fz-LogReg on MNIST (Row 1) FMNIST (Row 2) CIFAR1O (Row 3), impact of E, N, T, left to right single image recovery. Dashed línes average empirical reconstruction errors obtained by existing DRAs, while solid lines upper bound errors obtained by theoretical results. Y-axis is in log scale. We observe that iDLG slightly outperforms DLG both empirically and theoretically; a larger E and incur larger upper bound error, while a larger generates a smaller upper bound error. Empirical attacks are insensitive to these parameters.
convex 2-layer linear convolutional network (2-LinConvNet) PPlaaci & Ergen 2020 since their loss functions satisfy Assumptions A.14 In the experiments, we evenly distribute the training data among the N devices. Based on this setting, we can calculate L, μ,, σk, and used in our theorems, respectively. In addition, we can compute the Lipschitz constant LR via the unrolled feod-forward network. These values together are used to compute the upper bound of our Theorems 3.2 and 3.3] More details about the two algorithms, unrolled network, and calculation of porameter values shown in Appendir D.1
We test our theoretical results on four optimization-ban basod data reconstruction attacks, i.e, DLG (Zhu et al, 2019), iDLG (Zhao et al. 2020), InvGrad (Geiping et al. 2020), and the GGL {Li et al, 2022h The algorithms and descriptions of these attacks are deferred to Appendix D.1 We test these attacks on recovering both the single image and a batch of images in each device.
Attack baselines:
Parameter setting: Several important hyperparameters the FL training would affect our theoretical results: the total number of devices N, the total number of global rounds T, and the number of local SGD
7

---

Published in Transactions on Machine Learning Research (01/2026)
DEC
* ingrad 6G.
Ingrad
Uppertounnd
DCG inigred
225
120
200
100000 -100
11 [1#0000 (W22)
invgrad
DLG
Ingrad
inugrad
OLG
coww10 1w000)
[Tw0000 Dw21
(10000 (w2)
DLS BUG
Invgrad
IDLG
CGL
uppertoeed Hog
Inugrad
DLG
GGL
Figure 4: Results of federated NeL MNIST (Row 1), FMNIST (Row 2), CIFARIO (Row 3), impact of E, N, T from left to
image recovery.
updates E. By default, we set T 100 and E = 2. We set N = 10 on the three datasets for the single image recovery, while set N = 15, 10,5 on the three datasets for the batch images recovery, considering their different difficulty levels. we consider full device participation. When studying the impact of these hyperparameters, we fix others as the default value.
4.2 Experimental Results
In this section, we test the upper bound roconstruction error by our thcoretical results for single image and batch images recovery. We also show the average (across 10 iterations) reconstruction errors that are empirically obtained by the baseline DRAs with different initializations, The best possible (one-snapshot) empirical results of the baseline attacks also reported in Table in Appendix D.2
4.2.1 Results on single image recovery
Figure Figure show the single image recovery results on the three datascts and two FL algorithms, respectively. We have several observations. First, iDLG has smaller upper bound errors than DLG, indicating iDL.G outperforms DLG intrinsically. One possible reason is that iDL. accurately estimate the labels,
8

---

Published in Transactions on Machine Learning Research (01/2026)
meiet honest
diala
mikt
data
miei traut
data
Uppersound
223
120
200
E.0010 1100)
11 [102 1w0000
mett
ofa10
uppcrfound
cfal0
dtal
1HW30 TWODOS
M.Dw2 T000
110000
Figure 5: Results of federated LzgLog (Row 1) and 2-LinConvNet (Row 2) on Robbing, impact of E, N, T left to right- single image recovery, We observe that Robbing has much smaller bounded errors and is even smaller than GGL (See Figure 3 This because the equation solving used by Robbing is accurate on the simple federated Ez-LogReg model that uses a lincar layer.
which ensures data reconstruction to be more stable. Such a stable reconstruction yields a smaller Lipschitz LR. and thus smaller upper bound in Theorem 3.2 InvGrad on top of iDLG and adds TV regularizer, and obtains smaller error bounds than iDLG. This implies the TV prior can help stabilize the data reconstruction and hence is beneficial for reducing the error bounds. Note that the error bounds of these three attacks are (much) larger than the average empirical errors, indicating there is still gap between empirical results and theoretical results. Further, GGL has (much) smaller bounded errors than DLG, iDLG, and InvGrad. This is because GGL trains an encoder on the whole dataset to learn the image manifold, and then uses the encoder for data reconstruction, hence producing smaller LR- In Figure (b), we observe its bounded error close to the empirical error. For inslance, we calculate that the estimated LR for the DLG, iDLG, InvGrad, and GGL attacks in the default setting on MNIST are 22.17, 20.38, 18.43, and 13.36, respectively.
Second, the error bounds consistent with the average empirical errors, validating they have strong correlation. Particularly, we calculate that the Pearson correlation coefficients between the error bound and the averaged empirical error on these attacks in all settings are larger than 0.9.
Third, the error bounds do not show consistent correlations with the best empirical errors. For instance, we can see GGL has lower error bounds than InvGrad on Logge,,,, but its best empirical error larger than InvGrad on MNIST (3.14 vs 3.09). Similar observations on CIFAR10 on federated 2-LinConvNet, where InvGrad's error bound smaller than iDLG's, but its best empirical error is larger (4.33 3.37). More details see Table D in Appendix D.2 The reason is that the reported empirical errors are the best possible one-snapshot results with certain initialization, which do not reflect the attacks' inherent effectiveness. Recall in Figure that empirical errors obtained by these attacks could be sensitive to different initializations. In practice, the altacker may need to try many initializations (which could be time-consuming) to obtain the best empirical error.
Impact of E, N, and T: When the local SGD updates and the number of total clients N increase, the upper bound error also increases. This is because large E and N will make FL training unstable and hard

---

Published in Transactions on Machine Learning Research (01/2026)
NI
-- Rugrrd GGL
0.6
nvgrad
DLG
invgrad
16
20
120 (10w Lm2)
200
11020 1-1001
I A101000 LM!
Impact of
(b) Impact of N
Impact of
60
DLG 21G
invgrad CCu
DLG DIR
Itvg od CC
CLS D.5
IFVg GGL
(d) Impact of F
Impact of N
(1) Impact of T
og)
1000 W0002aue oog
DLG
Invgrad CCL
DLG
Inug ad OCI
DLG
Irvgrod OGL
035
(g) Impact of E
(h) Impact of N
Impact of T
Figure 6: Results of federated LogReg on MNIST(a-c), (MNIST((-I) CIFAR1O(g-i) batch images recovery.
converge, as verified in (Li al., 2020 On the other hand, a larger total number of global rounds T tends to produce smaller upper bounded error. This is because larger T stably makes FL training closer to the global optimal under convex loss.
4.2.2 Results on batch images recovery
Figures 10 in Appendix D.2 show the results of batch images recovery the three image datasets. As federated 2-LinConvNet has similar trends, only show federated Lz-LogReg results for simplicity. Our key obsorvations are: First, similar to results on single image recovery, GGL performs the best; InvGrad oulperforms iDLG, which outperforms DLG both empirically and theoretically. Moreover, a larger E and N incur larger upper bound error, while a larger T generates smaller upper bound error. Second, both empirical errors and upper bound errors for batch images recovery are much larger than those for single image recovery. This indicates that batch images recovery are more difficult than single image recovery, as validated in many existing works such Geippng al. 2020 Yin al. 2021
10

---

Published in Transactions on Machine Learning Research (01/2026)
invgrad
DLG
hygrrad GGL
DLG GGL
DLG
0-
uppercound (hs)
Uppertourd.d (mg)
Class num (Tm100 E-2)
Baxth size IN- [-2 1-100)
Figure 7: Impact of #classes and batch size on MNIST
5
Discussion
First term second term in the error bound 3 We calculate the first term and second term of the error bound in our theorem in the default setting. The two terms in DLG, iDLG, InvGrad, and GGL on MNIST are (30.48, 396.72), (25.25, 341.10), (22.06, 218.28), and (20.21, 29.13) respectively. The second term dominates the error bound with significantly higher value.
Error bounds vs. degree of non-IID: The non-IID of data across clients can be controlled by the number of classes per client-sman number indicates a larger degree of non-IID. Here, we tested #classes=2, 4, 6, 8 MNIST and the results are shown in Figure ((a) We can the bounded errors are relatively stable vs. #classes on DLG, iDLG, and GGL, while InvGrad has larger error the #classes increases. The possible reason is that DLG and iDLG are more stable than InvGrad, which involves more complex optimization.
Error bounds vs. batch size: Our batch results use batch size 20. Here, we also Lest batch size=10, 15, 25, 30 and results are in Figure 7(b) We see bounded errors become larger with larger batch size. This is consistent with existing obscrvations Geiping et al. 2020 on empirical evaluations.
Error bounds closed-form DRAs: Our theoretical results can be also applied in closed-form atlacks. Here, we choose the Robbing attack (Fowl al. 2022) for evaluation and its details are in Appendix D.1.2 The results for single image and batch images recovery on the three datasets and two FL algorithms are shown in Figure 5 and 11 We can see Robbing obtains both small empirical errors and bounded errors (which are even smaller than GGT.). This is because its equation solving is suitable to linear layers, and hence relatively accurate on the federated LogLeg and federated 2-ninCon models.
Practical use of our error bound: Our error bound has several benefits practice. First, it can provide guidance or insights to attackers, e.g., the attack performance can be estimated with our error bound prior to the attacks. Then, attackers can make better decisions on the attacks, e.g., try improve the attack beyond the worst-case bound (using different settings), choose to not perform the attack if the estimated error is too high. Second, it can understand the least elforts to guide the defense design, e.g., when pursue minimum obfuscation of local data/model /odell (maintain high utility) and minimum defense based on worst-case attack performance could be acceptable in practice.
Gap between theoretical bounds empirical results: The bounds discussed in the paper represent the worst-case errors, inherent to a thcoretical analysis, while the empirical results shown the average outcome from our experiments. In Section 4.2.1, we point out that the correlation between the two is high, indicating strong consistency in the relationship between the theoretical bounds and the experimental results while the gap persists. The bounds we have shown reliably rank attack effectiveness relative to one another, which is their primary purpose. For example, the tighter bound for InvGrad over DLG aligns with its superior empirical performance, validating the framework's utility for comparative analysis.
Convex-loss assumption and practical relevance: Extending our analysis to non-convex losses (c.g., deep neural networks) is an important direction. Our choice to focus on convex losses was motivated by the need for tractable theoretical guarantees, which provide foundational insights into attack effectiveness. While
11

---

Published in Transactions on Machine Learning Research (01/2026)
real-world FL often uses non-convex objectives, our framework establishes bascline for understanding how reconstruction errors propagate in a simpler setting. We acknowledge this limitation and emphasize it as a critical area for future work.
6 Related Work
Existing DRAs FL are roughly classified as optimization based and close-form based.
Optimization-basee DRAs to FL: A series of works [Hita] et al.
2017
Zhu et al.) 2019 Wang et al.
2019
Zhao et. al., 2020 Wei et al. 2020 Yin et al.; 2021 Jeon et al.
2021
Dang et al.
2021 Ba lnovicc
et al. 2022 Sun et al. 2021 Fowl et al. 2022 Wen et al. 2022 Li et al.
2022
Wang et al. 2023 formulate
DRAs as the gradient matching problem, i.e., an optimization problem that minimizes the difference between gradient from the raw data and that from the reconstructed counterpart. Some works found the gradient
itself includes insufficient information to well recover the data iJeon al. 2021 Zhu & Blaschko, 2021 For
example, Zhu & Blaschko (2021) | show there exist pairs of data (called twin data) that visualize dilferent, but have the same gradient. To miligate this issue, few works propose to incorporate prior knowledge (e.g., total variation (TV) regularization (Geiping et al. 2020 Yin et al. 2021), batch normalization (BN) statistics (Yin et al. 2021g) into the training data, or introduce an auxiliary dataset to simulate the training data distribution (Hitaj el al., 2017 Wang et al. 2019 Jeon et al. 2021) (e.g., via generative adversarial networks (GANs) Goodfellow et al. (014)). Though empirically effective, these methods are less practical or data inefficient, e.g., TV is limited to natural images, BN statistics are often unavailable, and training an extra model requires large amount of data.
Fowl et al. Closed-form DRAs to FL: A few recent works (Geiping et al. 2020 Zhu & Blaschko 2021
2022) derive closed-form solutions to reconstruct data, but they require the neural network used in the FL algorithm be fully connected (Geiping et al. 2020), linear Rrear/ReLU Fowl et al. (2022) or convolutional (Zhu &
Blaschko 2021).
We will design a framework to theoretically understand the DRA to FL in a general setting, and provide a way to compare the effectiveness of existing DRAs.
7 Conclusion
Federated learning (FL) is vulnerable to Data Reconstruction Attacks where adversary leaks all information about the input data. Existing attacks mainly enhance the empirical attack performance, but lack a theoretical understanding. We study DRAs to FL from a thcoretical perspective. Our thcorctical results provide a unified way to compare existing attacks theoretically. We also validate our theoretical results via evaluations on multiple datasets and baseline attacks. Future works include: 1) designing better Lipschitz estimation algorithms obtain tighter error bounds; 2) generalizing our theoretical results non-convex losses; and 3) designing theoretically better DRAs (i.e., with smaller Lipschitz) well as effective defenses against the aacks (i.e., ensuring larger Lipschitz of their reconstruction function), inspired by our framework.
References
Mislav Balunovic, Dimitar Iliev Dimitrov, Robin Staab, and Martin Vechev. Bayesian framework for gradient leakage. In ICLRR, 2022.
Yoshua Bengio, Pascal Lamblin, Dan Popovici, and Hugo Larochelle. Greedy layer-wise training of deep networks. NewrIPS, 2006.
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Damitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečný, Stefano Mazzocchi, H Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. Towards federated learning at scale: System design. Proceedings of Machine Learning and Systems, 2019.
Xiaohan Chen, Jialin Liu, Zhangyang Wang, and Wotao Yin. Theoretical linear convergence of unfolded ista and its practical weights and thresholds. In NewrIPS, 2018.
12

---

Published in Transactions on Machine Learning Research (01/2026)
Trung Dang, Om Thakkar, Swaroop Ramaswamy, Rajiv Mathews, Peter Chin, and Françoise Beaufays. Revealing and protecting labels in distributed training. Neur/PS, 2021.
Liam H Fowl, Jonas Geiping, Wojciech Czaja, Micah Goldblum, and Tom Goldstein. Robbing the fed: Directly obtaining private data in federated learning with modified models. In ICLR, 2022.
Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. Inverting gradients how easy is it to break privacy in federated learning? In NeurlPS, 2020.
Tan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS, 2014.
Google Federated Learning. https://federated. wttpddrrraaed...tthoooogo.o withgoogle com/ 2022.
Andreas Griewank and Andrea Walther. Evaluating derivatives: principles and techniques of algorithmic differentiation. SIAM, 2008.
Niv Haim, Gal Vardi, Gilad Yehudai, Ohad Shamir, and Michal Irani. Reconstructing training data from trained neural networks. In NewrIPS, 2022.
Briland Hitaj, Giuseppe Atenieso, and Fernando Percz- Cruz. Deep models under the gan: information Icakage from collaborative deep learning. In CCS, 2017.
IBM Federated Learning. fedelll--eedrrtte-eeeriiiia April 2022.
https: : www ibm. com /ocs / en cloud-paks cp-daata/4 0? topic
Jinwoo Jeon, Jaechang Kim, Kangwook Lee, Sewoong Oh, and Jungseul Ok. Gradient inversion with generative image prior. NewrIPS, 2021.
Georgios A Kaissis, Marcus Makowski, Daniel Rückert, and Rickmer F Braren. Secure, privacy-preservigg and federated machine learning in medical imaging. Nature Machine Intelligence, 266):305-311, 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun. The mnist database of handwritten digits. http://yann. lecun, com/erdb/mnise 1998.
Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data. In ICLR, 2020.
Yuelong Li, Mohammad Tofighi, Vishal Monga, and Yonina C Eldar. An algorithm unrolling approach to doep image deblurring. In ICASSP, 2019.
Zhuohang Li, Jiaxin Zhang, Luyang Liu, and Jian Liu. Auditing privacy defenses in federated learning via generative gradient leakage. In CVPR, 2022.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. In AISTATS, 2017.
Microsoft Fedcrated Learning.
https // www microsoft com / en-us research / blog
1attraarooadbb---fffdderddeeeear hww.://wmmmnnniimmiaioiooooooaao -simulation-platffrrr May 2022.
RV Mises and Hilda Pollaczek-Geiringg Praktische verfahren der gleichungsaullssung ZAMM-Journal of
Applied Mathematics and Mechanics/Zeiisschry für Angewandte Mathematik und Mechanik, 9(1):58-71 33:57777 1929.
Vishal Monga, Yuelong Li, and Yonina C Eldar. Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing, IEEE Signal Processing Magazine, 38(2):18 44, 2021.
Sayodeh Leila Noorbakhsh, Binghui Zhang, Yuan Hong. and Binghui Wang. Inf2guard: An information- theoretic framework for learning wrivacy-rreservie representations against inference attacks. In USENIX Security, 2024.
13

---

Published in Transactions on Machine Learning Research (01/2026)
2018.
Mert Pilanci and Tolga Ergen. Neural networks are convex regularizers: Exact polyuomial-time convex optimization formulations for two-layer networks. In ICML, 2020.
Nicola Rieke, Jonny Hancox, Wenqi Li, Fausto Milletari, and et al. The future of digital health with federated learning. NPJ digital medicine, 2020.
Anit Kumar Sahu, Tian Li, Maziar Sanjabi, Manzil Zaheer, Ameet Talwalkar, and Virginia Smith. Federated optimization for heterogeneous networks. arv preprint ar 1rXi::1812.0611 2018.
Sebastian Stich. Local SGD converges fast and communicates little. ain, 2018.
Sebastian U Stich, Jean- Can-Baptiste Cordonnier, and Martin Jaggi. Sparsified SGD with memory. In NIPS, pp. 4447-4458, 2018.
Jingwei Sun, Ang Li, Binghui Wang, Huanrui Yang, Hai Li, and Yiran Chen. Provable defense against privacy leakage in federated learning from representation perspective. In CVPR, 2021.
Aladin Virmaux and Kevin Scaman. Lipschitz regularity of deep neural networks: analysis and efficient. estimation. NewrIPS, 31, 2018.
Zhibo Wang, Mengkai Song, Zhifei Zhang, Yang Song, Qian Wang, and Hairong Qi. Beyond inferring class representatives: User-level privacy leakage from federated learning. In INFOCOM, 2019.
Zihan Wang, Jason Lee, and Qi Lei. Reconstructing training data from model gradient, provably. In AISTATS, 2023.
Wenqi Wei, Ling Liu, Margaret Loper, Ka-Ho Chow, Mchmet Emre Gursoy, Stacey Truex, and Yanzhao Wu. A framework for evaluating client privacy leakages in federated learning. In European Symposium Research in Computer Security, pp. 545 566. Springer, 2020.
Yuxin Wen, Jonas A Geiping, Liam Fowl, Micah Goldblum, and Tom Goldstein. Fishing for user data largo-batch federated learning via gradient magnification. In IICLL, pp. 13668-23684 PMLR, 2022.
Ruihan Wu, Xiangyu Chen, Chuan Guo, and Kilian Q Weinberger. Learning to invert: Simple adaptive attacks for gradient inversion in federated learning. In Uncertainty in Artificial Intelligence, pp. 2293-2303.
PMLR, 2023.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Pashion-mnist: novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: 2017.
Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M Alvarez, Jan Kautz, and Pavlo Molchanov. See through gradients: Image batch recovery via gradinversion. In CVPR, 2021.
Hao Yu, Sen Yang, and Shenghuo Zhu. Parallel restarted sgd with faster convergence and less communication: Demystifying why model averaging works for deep learning. In AAAI, 2019.
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. idlg: Improved deep leakage from gradients. arXii,, 2020.
Junyi Zhu and Matthew Blaschko. R-gap: Recursive gradient attack on privacy. ICLR, 2021.
Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. In NewrIPS, 2019.
14

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm Optimization- based data reconstruction attacks (e.g., DLG, iDLG, InvGrad, and GGL)
Model paramelers ws; true gradient g(x, y): y,, A: public generator C(), transformation operator Output: Reconstructed data &.
Input:
if DLG then
2: x (0,, 1); 16 (0,11)
else
4: if GGL then
zó ~N(,
6: else
Xb ~ NN 1) // Initialize dummy input((s)
8: end if
9: Estimate as ú via methods Zhao al. (2020) for single input or Yin et 2021 a batch inputs
10: end if
11: for 0 < 1;1++ do
12: if DLG then
13:
g(x,, v/) + Vw.((w; (x, 1()) GML; [19(x.)) (/x1,1/)11
15:
X.41 < x VV <
Mi+1 t v/ - vi GML,
-,,;
else if IDLG then
17:
g(x,, 0) t V W(wr, (x,, 9) GML, + [] y) - g(x), 111 x/41 x; v/ GML;
18:
19:
20:
else InvGrad then
21:
g(x(, 0) + V L(we; (xl, 0))
GML, f- To(x,vla 1oll
(o(x,)).(xxx.00)
X'41 < x V (MML, + ARegrv(x))
24:
else if GGL then
25:
x = =G(z))
26:
o(x(,0) + V C(we; (xl, 0))
27:
GML +- [19(x, ./) [(g(x), 0)
Reg(G, (, 21) ([|z:]3 k)2²
29:
2'41 < 2 - V2 GMLL + AReg(G,2)))
end if
31:
x(+1 max (x(+1,
end for
Return x or G(z))
A Assumptions
To ensure FedAvg guarantees to converge to the global optimal, existing works have the following assumptions on the local devices' loss functions }
Assumption A.1. {LaY's are L-smooth: Cx(v) Lx(w) w) + f w Vv, w. Assumption A.2. {LxYss
p-strongly convex: Lx(v) 2 Lx(w) (v VL. + f w|L Vv, w. Assumption A.3. Let di be sampled from the k-th device's data uniformly at random. The variance of stochastic gradients in each device bounded: E V/ { (w())l)] S o Vk.
Assumption A.4. The expected squared norm of stochastic gradients is uniformly bounded, i.c.
E [VLa(W [/ < GR,kkt
Note that Assumptions A.I and A.2 are generic. Typical examples include regularized linear regression, logistic regression, softmax classifier, and recent convex 2-layer ReLU networks (Pilanci & Ergen 2020). For instance, the original FT. (McMahan et al.] 2017) uses 2-layer ReLU networks. Assumptions A.3 and A.4
15

---

Published in Transactions on Machine Learning Research (01/2026)
are used by the existing works Stiich 2018 Stich et al. 2018
Yu et al. 2019
Li al. 2020 derive the
convergence condition of Fed to reach the global optimal. Note that the loss of deep neural networks
is often non-convex, i.e. do not satisfy Assumption A.2 We acknowledge it is important future work to generalize our thcoretical results to more challenging non-convex losses.
B Proof of Theorem 3.2 for Full Device Participation
Our proof is mainly inspired by the proofs in Stich 2018
Yu ct al. 2019 Li et al. 2020
Notations: Let N be the total number of user devices and K(s N) be the maximal number of devices that participate in every communication round. Let T be the total number of every device's SGDs, and E be the number of each device's 's local updates between two communication rounds. Thus T/E is the number of communications, assuming E is dividable by T.
Let w be the model parameter maintained in the k-th device at the I-th step. Let Zø be the set of global aggregation steps, ie., I = {nE n = 1.2. If + E Ix, i.e., the devices communicate with the server and the server performs the FedAvg aggregation on device models. Then the update of FedAvg with partial devicos active can be described as
Vi+ w mi (WL
(6)
i 1 $ Ir, if 1 Ig.
w 6+1
(7)
Ek= Pu Pu
Li et al. 2020), we define two virtual sequences v. =E P&V and w, ΣV Pa pewh Ve41 results from an single step of SGD from wz. When + 1 & IE, both are inaccessible.
Molivated by (Stich 2018:
When + E IE, we can only fetch Wr+1- For convenience, we define B. = E Pk P&&&&(W and g, = -E P V (h) Hence, Ve+1 Wt Th.. and Eg, =
Before proving Theorem 3.2 we need below key lemmas that are from Stich 2018 Li et al. 2020
Lemma B.1 (Results of step SGD). Assume Assumptions A.1 and A.2 hold.
If me
ar. have
B111111 w¹1l² s TE [|w; w' w01l + 11 lell + 6L 2E
Σ Pke W. will
where I-
PaC 2.
Proof sketch: Lemma B.1 is mainly from Lemma 1 in Li et al. 2020). The proof idea is to bound three terms, i.c., the inner product (w,- w", Vl(w:)), the square norm [|V[(w.)]l?, and the inner product (VLx(w.),, w/ w.), Vk. Then, the left-hand term in Lemma B.1 can be rewritten terms of the three terms and be bounded by the right-hand four terms in Lemma B.1 Specifically, 1) It first bounds (w,-"", VL(w:)) using the strong convexity of the loss function (Assumption A.2 2) It bounds |/ [w,)l) using the smoothness of the loss function (Assumption A.1 and It bounds ( (w.,), w wa), vk using the convexity of the loss function (Assumption A.2 Lemma B.2 (Bounding the variance). Assume Assumption A.3 holds. Then E - Σ Pi. Lemma B.3 (Bounding the divergence of (w[]). Assume Assumption A.4 holds, and me non-inerensing and 1/2 2 for all i 0. It follows that EN pk l|w. wfll s 4nf G2
Now, we complete the proof of Theorem 3.2
Proof. First, we observe that no matter whether t+1 E IE or t+1 $ Ig in Equation
i we have W V
Denote A, = E |ww
2 From Lemmas B.1 to B.3. we have
A ++1 El|we+
wt|l
E | w
122 < 7)1 A: + n
(8)
16

---

Published in Transactions on Machine Learning Research (01/2026)
where B= Ek= P 6LT + -
For a diminishing stepsize, 7):
ف for some and 7 0 such that m mint 1. म.. and 8 B 8p-1 (7 + 1)A1}. }
1+7
Th I 2. We will prove A S 7F4 where e max
We prove it by induction. Firstly, the definition of a ensures that it holds for 1. Assume the conclusion holds for some t, follows that
1+1 S (1 11)1)
+ n B
B²B
t t +
t+,)
(t+
+
B
81 - 1 (t (++
(1 (1+ U +
y)+
V
++4+1
By the L-Lipschitz continuous property of Rec(-),
[[Rec(w,) (Re((w"]] S L. ||w: w"ll
Then we have
Rec(w) - (w > || - w'l 5 L'A, S L2
7+(
Specifically, if we choose B = 2. max (8 E} 1, then = 2 1 #771.
We also verify that the choice of
satisfies T) S 2 for t 2 1. Then, we have
1Bp (7 (77 1)A11
{BB
< BB
4B #²
= max
3u + (
S +
Hence,
1.2
4B 112
E RRo((.:) - Rec(w")] 5
1.2
(7 + 1)4,).
7+
7 +
C Proofs of Theorem 3.3 for Partial Device Participation
Recall that w is k-th device's model at the t-th step, Ig = {nE n = 1,2, is the set of global aggregation <h), and Vi+1 w, nıg, and Eg, gr. We denote
steps; B. =Ex= pa (w"( and g: I ( by Hr the multiset selected which allows any element to appear more than once. Note that H, is only well defined for t E Ig. For convenience, we denote by S; = HN(LE) the most recent set of chosen devices where (), = ma < 4 i Ip}.
In partial device participation, FedAvg first samples a random multiset S; of devices and then only performs updates on them. Directly analyzing on the S, is compliated. Motivated by Li et al. (2020), we can use a thought trick to circumvent this difficully. Specifically, we assume that FedAvg always activates all devices at the beginning of each round and uses the models maintained in only few sampled devices to produce the next-round model. It is clear that this updating scheme is equivalent to that in the partial device participation. Then the update of FedAvg with partial devices activated can be described as:
1-1 = w, m VL (wh
(t) sh)
(9)
v
if 1 $ IE.
k
(10)
#+1
samples S and average (vtn
if 1 E Ig.
17

---

Published in Transactions on Machine Learning Research (01/2026)
Note that in this case, there are two sources of randomness: stochastic gradient and random sampling of devices. The analysis for Theorem 3.2 in Appendix B only involves the former. To distinguish with it, we use an extra notation Es, (+) to consider the latter randomness.
First, based on Li et al., [2020] we have the following two lemmas on unbiasedness and bounded variance. Lemma C.T shows the scheme unbiased, while Lemma C.2 shows the expected difference between V++1 and We+1 is bounded.
Lemma C.1 (Unbiased sampling scheme). Ift E LE, we have Es, (W++1) = Vt-1-
Lemma C.2 (Bounding the variance of we). For + 1 I+1EI, assume that is non-increasing and ne S 21j++E for all 2 0. Then the expected difference between Ve+1 and wH+1 is bounded by
4 2 G²
Es, ||v++1 w++1ll S "h
K
Now, we complete the proof of Theorem 3.3
Proof. Note that
|/2411 w11 = ||We++1 Ve+1 +
w=12
Ve41
||we+1 Ve+ ve+ill 11
- V44i, Ve+1
A
A2
When expectation taken over S4+1, the last term (Ag) vanishes due the unbiasedness of Wi+1-
If 1 1 Ig, A1 vanishes since Wi+1 Vi+1- We use Lemma
C.2 to bound A2. Then it follows that
E We4
w' 1| (1 Ta)) w. w'll 112 BB
If e Ig, we additionally use Lemma
C.2 to bound A Then
E B| - w/|
E W Vet1 11 + [[ - -w"l] w'11
4
S (1 Tel IE l|w. w011 niBB + K E²
(1 T (E || w01 1 T C),
(11)
where C' 622 is the upper bound of E.s. ||++1 1
%
We observe that the only difference between equation [1 and equation is the additional C. Thus we can use 8
the same argument there to prove the theorems here. Specifically, for diminishing stepsize, Me 7+7 for some
I and 7 0 such that 1n S minf",
1 pt a aL and ne I 21 can prove E || w"|l 7
where U = max ( (7 I)llw] w 11
Then by the T-Lipschitz continuous property of Rec(-),
B][Rec(w.) - Rec(w (w 2|1 < 122 El - 1
i'A, =
7 7+t
Specifically, if we choose B = 2, max(88 E3-1,
I
E Ree((w - Ree(w"))/f 12 7+1
<
(7 + w)]l - w'))).
7
122
18

---

Published in Transactions on Machine Learning Research (01/2026)
D Experiments
D.1 More Experimental Setup
D.1.1 Details about the FL algorithms and unrolled weeward network
We first show how compute calculate L, nl uu, and G in Assumptions 1-4 on federated éz-regularized logistic regression (42-LogReg) and federated 2-layer linear convolutional network (2-LinConvNetl); Then we show how to compute the Lipschitz LR on each data reconstruction attack.
Federated Lg-LogReg: Each device k's local objective is (x(w) = Σز= log( +epp(- (w +//w/// In our results, we simply set 0.1 for brevily.
Compute La first, all Lx's are 1 E, (l,)))ai))ssmmootll MaX 2 Ej [xx)]]]) + 27:
Papaillopoulos
2018);
then L
Compute μr all L are y-strongly convex for the y regularized & logistic regression 2018) and μ 7-
Papailiopoulos
Compute o² and G: we first traverse all training data & in the k-th device in any 4thh round and then use them to calculate the maximum square norm differences [[ <t) (Lx(f)) can be calculated the maximum value of the expected square norm VL (f)) among all devices [k} and rounds {4}.
1? Similarly, G
Federated NoninConv (Pilanci & Ergen, 2020). Let a two-layer network f R + with m neurons be: f(x) = Σド= o(x" Jy)aj; where u, E R and aj E R are the weights for hidden and output layers, and Φ(-) is an activation function. Two-laycr convolutional networks with U filters can be describod by patch matrices (e.g., images) X,, u 1, U. For flattened activations, we have f(X1,... X.) E Σş= d(Xnu,)ay.
We consider the 2-layer linear convolutional networks and its non-convex loss is defined as:
min C(taj, u,}) (a 1
Σ
X, Ujaju y112
(12)
Pilanci & Ergen (2020) show that the above non-convex problem can be transferred to the below convex optimization problem via its duality. and the two problems have the identical optimal values:
min W
c((wa)) = Σ X. w. - y112.
(13)
Jum
We run federated learning with convex 2-layer linear convolutional network, where each device trains the local loss (f y and it can converge the optimal model w -
Compute L: Let tw..e For each client k require its local loss Ca should satisfy [[VL+(w) (Lv(v)ll s La |w v1l2 for any w, With Equation 13 we have
Σ (x X x)
S La www xxl2
Then we have the smoothness constant La to be the maximum eigenvalue of ΣV (X+Y X which is DD T Hence, I-= max I (X
Compute ie Similar as computing 4, μ is the min eigenvalue of VV= (Xb) for all k, that is, f ming EV Xx
19

---

Published in Transactions on Machine Learning Research (01/2026)
Table 1: Bost empirical errors (log scale) of the baseline attacks on the three datasets the default setting.
Fed.
Ez-LogReg: single
Fed. 2-LinConvVet::
single
Fed.
Ez-LogRe::
batch
Data/ Alt/oitoormm
DLG
iDGL
InvGrad
GGT.
DLG
iDGL
InvGrad
GGL.
DLG
IDGL
InvGrad
GGL
MNIST
3.48
3.38
3.09
3.14
3,69
3.50
3.42
3.29
6.38
6.28
6.08
4.94
FMNIST
3.54
3.42
3.35
3.25
3.84
3.58
3.52
3.13
6.44
6.35
6.29
4.98
CIFAR1O
4.32
4.13
4.13
3.92
4.32
3.37
4.33
3.70
6.46
6.45
6.63
5.31
Compute t² and G: Similar computation as in lg-regularized logistic regression.
Unrolled feed-forward network and its training and performance. In our experiments, we set the number of layers to be 20 in the unrolled foed-forward network for the three datasets. We use 1,000 data samples and their intermediale reconstructions to train the network. To reduce overfitting, we use the greedy layer-wise training strategy. For instance, the average MSE (between the imput and output of the unrolled network) of DLG, iDLG, InvGrad, and GGL on MNIST 1.22, 1.01, 0.76, and 0.04, respectively indicating that the training performance is promising. After training the unrolled network, we use the AutoLip algorithm to calculate the Lipschitz T.R-
D.1.2 Details about Data Reconstruction Attacks
GGL (Li et al., 2022): GGL considers the scenario where clients realize the server will infer their private data and they hence perturb their local models before sharing them with the server defense. To handle noisy models, GGL solves optimization problem similar to InvGrad, but uses a pretrained generator as regularization. The generator is trained on the entire MNIST dataset and can calibrate the reconstructed noisy image to be within the image manifold. Specifically, given a well-trained generator G(-) on public datasets and assume the label y is inferred by iGLD, GGL targets the following optimization problem:
z = arg min GML(g(x, v), T(g(((z)) v))) + ARog(G;z), ZER
(14)
where is the latent space of the generative model, T is a lossy transformation (e.g., compression or sparsification) acting a defense, and Reg(G;z) is regularization term that penalizes the latent if it. deviates from the prior distribution. Once the optimal z" is obtained, the image can be reconstructed G(z") and should well align the natural image.
In the experiments, we use a public pretrained GAN generator for MNIST, Fashion-MNIST, and CIFAR. We adopt gradient clipping as the defense strategy T performed by the clients. Specifically, T(g, S) = g/ max(1, ||9ll//5). Note that since G(-) is trained on the whole image dataset, it produces stable reconstruction during the optimization.
Robbing (Fowl et al. 2022): ; Robbing approximately reconstructs the data via solving an equation without any iterative optimization. Assume batch of data X2, 22....X.0 of one-hot encoding. Let @ be dement-wise division. Then, Robbing observes that each row i in &C, dy: ie.,
with unique labels V. yn in the form
ac, actually recovers By
ac, Oyt
0c,
X;
x, Ø
Oyj
In other others, Robbing directly maps the model to the reconstructed data. Hence, in our experiment, the unrolled feed-forward neural network reduces to 1-layer ReLU network. We then estimate Lipschitz upper bound on this network.
D.2 More Experimental Results
Table show the best one-snapshot empirical results of the four attacks on the two FL algorithms in the default sotting.
Figures g 10
show the batch images recovery results by the four considered DRAs on federated LogLog
20

---

Published in Transactions on Machine Learning Research (01/2026)
30
C DU5
ingiadd
DLG
- Iugrad
66.
www momt miig
CLG
inggrad
120
230
L-We20 (0200)
1.110000 EN?
(a) Impact of E
Impact of N
(c) Impact of T
Figure 8: Results of federated l-- LogLog MNIST batch images recovery,
Uppetbonn 00g)
100 Comoooaa
44
inggred GGL
MUG
invgred GCL
irygeed GGL
DLG
DLG
D.5
(a) Impact of F
(b) Impact of N
(c) Impact of T
Figure Results of federated lz- LogReg on FMNIST- batch images recovery.
10
DLG Invgrad
DLG
Invg od
DUG
Irvg:20
GGL
700
105
400
505
000
T/20-10 P-17
(a) Impact of F
Impact of N
(c) Impact of T
Figure 10: Results federated LogLog on CIFAR10 batch images recovery.
Figure 11 shows the batch image recovery results by Robbing on federated
21

---

Published in Transactions on Machine Learning Research (01/2020)
cfa10
mrist
dfaic
mase
Upatterdd
bog)
100-18-H-180 1280
90.9 1/ 75-0000
T.10 (2h-2)
Impact
(b) Impact of N
(c) Imppcc
Figure 11: Results of federated Lz-Log on Robbing batch images recovery.
22


Reviews:
[{"review_id": "EzZw9MLfSy", "reviewer": "Reviewer_KxVA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a theoretical framework for analyzing gradient-based data reconstruction attacks (DRAs) in federated learning. The authors model such attacks as a reconstruction function $R(w_t)$ of the global model and derive reconstruction error upper bounds under convex loss assumptions for FedAvg with both full and partial client participation. Empirically, the authors show, across several datasets and FL configurations, that the resulting bounds correlate well with average empirical reconstruction error and capture the effects of key FL hyperparameters, thereby suggesting that the framework is useful as a ranking and comparison tool for DRAs.", "claims_and_evidence_yn": "No", "claims_explanation": "I noticed several potential typographical errors and instances of missing symbols in the paper. I just listed a few examples: \n\n* The meaning of $||_1$ is unclear in equations (4) and (5)\n* the $t k$ in the last two lines of page 14 \n* the $t k$ in the line below equation (8)\n* the $t=t$, $t=t k$ in fourth-to-last lines on page 15\n* the $t-*$ in the final line of page 15\n* the $t-tk$ in Lemma C.3\n* the $-*$ in equation (9) \n* Many more typos on pages 16, 17, and 18.\n\nDue to these typographical errors, the correctness of the technical claims is difficult to verify.", "audience_yn": "No", "audience_explanation": "* Typos\n\nThe paper has numerous typographical errors and missing symbols. This makes the technical claims difficult to verify. Please see details in the previous section.\n\n\n* The assumption of convexity has some limitations.\n\nThe paper assumes that the objective loss functions are convex. Under this setting, the empirical results show that the average reconstruction error is strongly correlated with the worst-case error bound derived by the authors. However, it is unclear whether a similar correlation holds in the non-convex setting, which is common in federated learning. This limitation restricts the scope of the paper. Section 5 (paragraph 5) includes some discussion of the worst-case upper bound, which is intended to inform the difficulty of data reconstruction attacks from the perspective of an adversary. However, it remains unclear how the insights gained from the convex setting can be extended to more practical, non-convex scenarios.\n\nGiven these two concerns, I believe that the current version is not yet ready for publication in TMLR.", "requested_changes": "(1) I would recommend that the authors carefully revise these typos, as doing so would significantly enhance the clarity of the technical content and make the paper’s contributions more accessible to the readership.\n\n(2) Could the authors elaborate on the practical implications of this work? In particular, how do the conclusions derived under the convexity assumption inform the design of attacks or defenses in federated learning systems with non-convex objectives? It would be helpful to clarify how the insights from the convex setting generalize (or fail to generalize) to more realistic non-convex settings, as the current discussion in Section 5 (paragraph 5) doesn't provide these details.", "broader_impact_concerns": "No ethical concerns.", "additional_comments": "I don't have additional comments.", "extra_scores": {}, "extra_text": {}}, {"review_id": "xGd50qByzB", "reviewer": "Reviewer_asth", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a theoretical framework to bound data reconstruction error in FL. The authors introduce a metric based on the Lipschitz constant of the reconstruction function to quantify attack effectiveness, applying this theory to validate that iDLG inherently outperforms DLG.", "claims_and_evidence_yn": "No", "claims_explanation": "- While the authors acknowledge the convex-loss assumption as a limitation, as a theoretical paper it should be more than just an acknowledgement. This is a very strong assumption that is likely to never hold for models trained with SGD in FL (or any other setting). Even if the theoretical analyses are limited to convex losses, the authors should experiment with models trained in a way that would make their loss curves non-convex, and see how well the theoretical results here hold.\n- The reported error bounds (Figures 3, 4) are multiple orders of magnitude worse than actual empirical bounds (note that the y-axis is on the log scale), suggesting that the bounds are too loose and may not be of practical use. The authors should either work on making the bounds tighter, or discuss how the bounds are useful despite them being significantly off from empirical observations.\n- \"Third, the error bounds do not consistent correlations with the best empirical errors\" (Page 9) is a direct contradiction to the central claim of the paper (that theoretical bounds can be useful in understanding/prediction attack success). Correlation with \"average\" performance is not useful under an adversarial setting- all involved parties in a security context care about worst-case risk, not average-care risk \n- Page 5: The way FL devices are sampled here per round is inconsistent with standard FL-  devices per round are sampled without replacement, but the text right above Theorem 3m3 mensions sampling \"independently and with replacement\"", "audience_yn": "Yes", "audience_explanation": "- Data reconstruction in the context of FL may be of interest to certain researchers focusing on connecting theory and practice for privacy analyses\n- Working towards ways to predict worst-case reconstruction for attacks can be beneficial, especially when trying to establish the superiority of one method over another by reporting more than just empirical reconstruction attack results.", "requested_changes": "- Figure 1: Figure takes up too much blank space. What message is the figure supposed to convey? The way the MSE is visualized here is very odd (separate lines for mean and std) - please replace with just one line, along with 1 (or 2) standard error worth of shaded region. On that note, given the high MSEs I don't know if the difference in values for the two attacks is statistically significant\n- Equation 2 focuses on the gradient of an individual record, but as an adversary you only get to observe batch gradients (since each client runs SGD locally on data in batches). Even in that case, the adversary has no way of knowing 'which' batch might have the target data in it. It is unclear to me how these constraints are incorporated in the attack, and it would be great if the authors could comment on this and clarify.\n- The derivation in Equation 3 is unclear to me: the expectation here is supposed to be over the randomness of R, but the text accompanying this equation comments on R being a constant.\n- Page 7: \"...we use a public pretrained GAN generator for ...\"  - do you mean a) some generator model X, which is kept the same for all three datasets and publicly available, used or b) a separate generator model X_i for each dataset (with the generator trained on that particular dataset)? Please clarify, since the latter implies that the adversary somehow has indirect access to the very dataset over which it seeks to perform reconstruction\n- I do not agree with the suggested practical use of error bounds (page 10) - the reconstruction attack is passive i.e. as an adversary, there is no harm in doing anyway, and always trying to improve performance within reasonable measure is also reasonable for an adversary.", "broader_impact_concerns": "N/A", "additional_comments": "## Minor comments\n\n- Abstract/Introduction: FL is referred to as an \"emerging\" field - it has been around for nearly a decade\n- Section 1: \"Hence, Attack A shows....effective than B\" : as the adversary you don't care about \"truly more effective\". If some attack can exploit certain snapshots very efficiently, that's a win for the adversary. What one should care about is worst-case across the entire training run.\n- The paper has a lot of math and theorems, but it is not apparent how they are useful. For instance, Theorems 3.1 through 3.3 involve a lot of (overly complicated) equations, but it is not clear to me what their impact is supposed to be in terms of the story of the paper, or how they are helpful in conclusions that follow?\n- Section 3.2: \"... followed by a clip nonlinear activation function..\" - a) please be specific about the activation function b) why is this particular activation function added? \n- Section 5: 'First term vs second term in the error bound' : the exact values are not very useful here for a discussion, since the average reader would not have good intuition about what a values for \"the two terms in DLG\" mean.\n- Page 2: \"Our theoretical results show that when an attacker’s reconstruction function has a smaller Lipschitz constant, then this attack intrinsically performs better\" - A function that always predict 0 or some small value $\\epsilon$ technically has a very small Lipschitz constant, but is of no use to any adversary. The authors here should clarify that the smaller constant is more useful for robustness of reconstruction to noise.", "extra_scores": {}, "extra_text": {}}, {"review_id": "FeNiIdWbXM", "reviewer": "Reviewer_ZkRn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a theoretical framework to bound the data reconstruction error in the context of Federated Learning. \nThe result is relevant and useful for theoretically compare the the effectiveness of existing Data Reconstruction Attacks (DRA) by comparing their bounded errors.\nThe framework is tested on multiple attacks and on three different datasets (Mnist, Fashion-Mnist and Cifar-10) under the assumption of having a convex loss (which is probably the main weakness of the paper).", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims made in the submission are convincing, but they are based on a strong assumption of the convex loss. The authors recognize this limitation on page 10 and also leave the extension to non-convex losses as future work.\nThe second thing that could improve the contribution is a study with more clients involved in the FL model training to simulate a more \"real-life\" scenario.", "audience_yn": "Yes", "audience_explanation": "I believe the contribution is sound and that it could be beneficial for researchers working in the field.", "requested_changes": "Main problems: \n\n1) The abstract is not clear and does not clearly explain what is the contribution of the paper. In the end, the names of iDLG and DLG are introduced, but at that point, a reader that do not already know them does not clearly know what they are, and most importantly, if they are a contribution of the paper. \n\n2) The authors provided experiments with a relatively low number of clients (between 5 and 15). Would the proposed approach work in a cross-device case in which the number of clients is higher (let's say 100 clients) and only a few of the clients are selected in each FL round? Could you please elaborate a bit on this or conduct some experiments showing how a higher number of clients can impact the results? \n\n3) The main limitation is the assumption of the convex loss. This is also one of the future works. Could the authors elaborate a bit about how difficult it would be to extend the framework to a more standard non-convex loss scenario? \n\n4) The plots can be improved and are not easy to read. Some suggestions for all the plots of the paper:\n- In all the plots, the font of x, y labels, ticks, and legend should be increased because it is hard to read them\n- In Figures 3, 4, 5, 6, and 7, besides the labels and ticks that are hard to read, I would suggest using the same y limit in the plots regarding the same dataset. For instance, Figures 6a, b, and c should have all 10 as a maximum y limit. \n- Since the legend is the same for all the plots, it could be placed once below the plots, increasing the font size. Moreover, I should also include the meaning of the dashed lines in the legend. \n\nMinor comments:\n\n- There are some typos and other minor problems with grammar and text, I've found some of them that I listed here\n- In the Introduction, the server is referred to as \"center server\" while in another point of the paper as \"central server\". Usually, in FL papers, it is called \"central server\". I would suggest using this when referring to the server to be more consistent in the different parts of the paper. \n- On page 2, when FedAvg is introduced, it is mentioned that clients \"only share the gradients\". This is how FedSGD works, with FedAvg clients sharing their locally trained models so that they can run multiple local updates.\n- Page 3: \"aims to infers\" -> \"aims to infer\"\n- Page 4: \"Give random data\" -> \"Given random data\"\n- In the abstract, there is a repetition of the word \"attack\": an attack’s error bound reflects its inherent attack effectiveness.\n- In the introduction \"keep and train their data locally\" -> train their model not data\n- There is a missing word here \"Third, the error bounds do not consistent correlations\"\n- I don't know if it is a formatting error but in the abstract there are a couple of missing spaces between works like \"attacks.However\"", "broader_impact_concerns": "I do not have any ethical concerns about this work", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)