Paper: Theoretically Understanding Data Reconstruction Leakage in Federated Learning (ICLR-2024)

Reviewer comment (Experiments):
**Dependence on the performance at $w^\*$**:  
The first part of the bound ($\mathbb{E}[x-\mathcal{R}(w^\*)]$) still depends on the performance of the leakage algorithm $\mathcal{R}$. This means that, in practice, the computational complexity of obtaining the proposed bound is the same as the complexity of the traditional evaluation of leakage attacks (which is very high, especially for attacks based on optimization). It also means that the issues related to estimating the performance of $\mathcal{R}$ in the presence of randomness ( e.g. from the initialization or the choice of the client data batch) remain as hard to solve for the proposed bound as for the original leakage problem. Further, the problem of choosing the optimal hyperparameters for leakage attacks in the presence of such randomness, which would have been one of the best applications of the proposed bound, also remains as hard as before. Finally, the dependency also introduces possible additional challenges compared to evaluating the attack directly on $w_t$ - in particular, estimating $w^\*$. While for convex models, such as the ones explored in the paper, estimating $w^\*$ is not hard, for models with multiple local minima and equivalent solutions ( like generic neural networks ), it is actually challenging to estimate $w^\*$, and it represents an additional source of randomness.

Author response:
**Response:** First, under a given *convex* FL algorithm, the optimal $w^*$ is unique (and can be obtained), and *all* leakage algorithms are compared using the same optimal model. We observe that, based on this optimal model, all leakage algorithms have relatively stable (variance less than 10) and small reconstruction errors of the first term (though the presence of randomness, e.g., from the initialization). Instead, the existing attacks are tested on intermediate $w^t$, which could cause unstable empirical reconstruction errors, as shown in Figure 1.   

Second, we also noticed that the second term largely dominates the error bound, compared with the first one.  For instance, in the default setting on MNIST, the two terms in DLG, iDLG, and invGrad, are (30.48, 396.72), (25.25, 341.10), (22.06, 218.28), respectively. 

We admit that *convex* loss may not be satisfied in deep neural networks and generalizing our theoretical results to non-convex losses is important future work.

Reviewer comment (Theory):
**The bound's dependence on $t$:**   
In Section 5 of the paper, the authors demonstrate, both practically and theoretically, that their bound predicts that as $t$ increases, the vulnerability of the attacked model increases. This is in stark contradiction with the empirical observations about gradient leakage attacks where exactly the opposite is true (e.g. See [1,2]). This mismatch is caused by the second part of the bound, which is supposed to precisely capture the dependence of the attack's success through time but, in reality, is based solely on the convergence of FedAvg and disregards any knowledge of $\mathcal{R}$ but its Lipschitzness. As such, the bound does not capture the evolution of $\mathcal{R}$ with time, only the evolution of the weights.

Author response:
**Response:** In most of the results, we found the best empirical errors are very stable with respect to $t$. I guess one possible reason might be that the models used to train the FL are relatively simple (i.e., convex shallow models) and the trend of the model $w^t$ has a certain pattern (e.g., it gradually converges to the optimal model $w^*$). Instead, the existing attacks use nonconvex deep neural networks to train FL, which could memorize the data in the first few iterations, and hence they show less empirical reconstruction error with a small $t$. 

Our theoretical results show the error bound decreases w.r.t. $t$. As pointed out by the reviewer, this is due to the error bound in the second term decreasing as $t$ increases.

Reviewer comment (Reproducibility):
**Unclear or missing implementation details — Question 1:**  
When the first term of the bound $\mathbb{E}[x-\mathcal{R}(w^\*)]$ is estimated, what is the average taken over? Multiple batches of the client data? Multiple initializations of the algorithms? Across different attack hyperparameters? All?

Author response:
**Response:** The average is over the randomness in $\mathcal{R}$, e.g., different initializations.

Reviewer comment (Reproducibility):
**Unclear or missing implementation details — Question 2:**  
How is $\Gamma$ approximated in practice? How is heterogeneity controlled for in general in the experiments provided? Can you demonstrate the results of experiments on different levels of heterogeneity?

Author response:
**Response:** Recall $\Gamma = \mathcal{L}^* - p_k \mathcal{L}_k^*$, where $p_k$ can be understood as the fraction of data samples in client $k$ over all clients' data. $\mathcal{L}^*$ is global loss, and $\mathcal{L}_k^*$ is client $k$’s loss, both of which can be calculated under the convex loss setting. In our results, *the heterogeneity can be controlled by the number of classes per client*. For instance, the clients that have 1 class of data samples show a larger level of heterogeneity than the clients that have 2 classes of data samples—as the client losses in the former case could be more diverse than those in the latter case.  

Figure 9(a) demonstrates the results on different levels of heterogeneity. We can see a large level of heterogeneity makes the attack easier, i.e., a lower error bound or empirical error. This is consistent with the existing literature.

Reviewer comment (Reproducibility):
**Unclear or missing implementation details — Question 3:**  
The authors in Figure 2 and Algorithm 1 talk about their unrolled network to have parameters $\theta^i$. Where are those parameters coming from, and what do they represent for the used leakage attacks? Even in Line 4 of Algorithm 1, where they are defined, they don't seem to be used. Also, can you elaborate on why you tune them with layer-wise methods instead of SGD? The authors just say "better generalizability" with no context.

Author response:
**Response:** Sorry for the confusion. Each $\theta^i$ means the model parameter connecting the $i$-th layer and $i+1$th layer in the unrolled deep neural network. In our context,  each  $i$-th layer is the intermediate reconstructions $x_i’$ generated by the attack algorithm $\mathcal{R}$. The unrolled deep neural network is to learn $\theta^i$’s via reproducing the intermediate reconstructions $x_i’$. More training details are given in the Appendix C.1.1.  

The reason to train the network in a layer-wise fashion is because the number of model parameters is large (e.g., the \#parameter in each layer is the \#pixels * \#pixels). However, we use SGD to update the parameters within each layer-wise training.

Reviewer comment (Evaluation):
**Problems with the presented evaluation — Issue 2:**   
The paper's main claim is that the proposed bound is a useful tool for evaluating the practical performance of gradient leakage attacks. Yet, the authors do not provide a correlation metric between their bound and the empirical evaluation results. Can the authors precisely measure how correlated their bound is to the actual gradient leakage results?

Author response:
**Response:** Our theoretical results show that the *one-snapshot* empirical errors may not have a strong relation with the bounded errors. This is because *one-snapshot* empirical errors obtained by the existing attack can be significantly different under different initializations (as verified in Figure 1).  However, Figure 9c shows that the error bound per attack has a *strong correlation* with its empirical errors *in expectation*. We tested that with the 10 initializations, the Pearson correlation coefficient between the error bound and the averaged empirical errors on the four attacks are larger than 0.9.     

Our theoretical results (in Theorems 1 and 2) show that, when an attacker’s reconstruction function has a *smaller Lipschitz* constant (in the second term), then this attack intrinsically performs better, i.e., more gradient leakage. Note that the second term is often much larger than the first term in the error bound.

Reviewer comment (Evaluation):
**Problems with the presented evaluation — Issue 3:**   
While the paper focuses on analyzing how gradient leakage performance changes with $E$, $N$, and $t$, I want to see their bounds used for comparing the same gradient leakage attack on different models and architectures, as well as, across different hyperparameters such as initialization strategies, regularizer strengths, etc. See [3].

Author response:
**Response:** The main motivation of the paper is to *theoretically compare the existing gradient leakage attacks with a unified framework*. This provides guidance to compare attacks more fairly and reasonably, as the literature only shows the comparison results at once and in different settings. 

We also note the existing attacks do use different models/architectures, and our takeaway is that the Lipschitz of the model/attack algorithm largely determines the attack performance.   

In addition, we have tested our error bound with different initializations and we found it is insensitive to them. This is because the unrolled deep neural network learns the intrinsic Lipschitz of the attack algorithm $\mathcal{R}$.

Reviewer comment (Evaluation):
**Problems with the presented evaluation — Issue 4:**   
The authors claim that one needs to interpret their bound as **average** and not **best-case** reconstruction performance ( Section 6 in the paper ), yet they only provide a single experiment ( Figure 9c ) where they compare against average reconstruction performance. All experiments in the paper should show average behavior for the author's claims to be substantiated.

Author response:
**Response:** Thanks for your suggestion! The reason we show the best empirical results is to replicate the results shown in the literature. We will add average empirical results as well as the Pearson correlation coefficients.

Reviewer comment (Evaluation):
**Poor experimental results:**  
The current experimental results are weak. What I mean by this is that in many experiments, the bounds do not well reflect what happens to practical performance. For example, in Figures 3 and 4, the empirical evaluation puts invGrad and GGL very close to each other in terms of performance, with invGrad sometimes even better. At the same time, the bounds consistently put the performance of invGrad to be similar to DLG and iDLG. Similarly, in Figures 5a and 5c, the bound predicts better reconstructions from DLG compared to invGrad and iDLG, while the practical performance of DLG (expectedly) is quite a lot worse than invGrad and iDLG. If the authors want to claim this is due to average vs best-case performance, they should provide more empirical evidence than Figure 9c for these discrepancies as they are noticeable in **almost** all figures in the paper.

Author response:
**Response**: This is a misunderstanding! The conclusion is that the *best* empirical error does not decrease with $t$. For the average empirical error, it indeed decreases with $t$ and has a strong correlation with the error bound (e.g., as replied in the previous rebuttal, the Pearson correlation coefficients are larger than 0.9 on FMNIST).

Reviewer comment (Evaluation):
The main concern is the low correlation between the proposed upper bound Lipschitz Constant and true reconstruction error. Offering a clearer and more understandable explanation would elevate the persuasiveness and lucidity of the research.

Author response:
**Response:** This may be a misunderstanding! Our results show that the best *one-snapshot empirical errors are not consistent with the bounded errors in some cases* for certain attacks (e.g., InvGrad on FMNUST). However,  the error bound per attack indeed has a *strong correlation with its empirical errors in expectation*. See Figure 9c, where we can see the consistency between the error bounds and average empirical errors.

Reviewer comment (Evaluation):
While MSE emphasizes pixel-wise differences, this can sometimes be misleading. For instance, a reconstruction might visually appear impeccable, but if there are a few pixels that greatly deviate from their counterparts, the MSE value could be disproportionately high. Metrics like SSIM provide a perception-based assessment, and LPIPS offers a learned perceptual similarity, which might capture human perceptual judgment more accurately. It would be beneficial to integrate these non-norm-based metrics.

Author response:
**Response:** We agree that structure based SSIM and LPIPS could provide more accurate human perceptual judgment! However, we emphasize *this is the first paper to theoretically understand the data reconstruction attack*, and we first focus on norm-based metric. Nevertheless, we acknowledge that generalizing our theoretical framework to SSIM or LPIPS face a fundamental challenge: these metrics do NOT even have an analytic form, making the error/similarity bound formulation infeasible.

Reviewer comment (Theory):
The FL model in the theorems on the attack upper bound is required to be strongly convex and smooth. This is far away from the scenario where those attack methods are located, as those attack methods are commonly evaluated for deep neural networks with image inputs in the evaluation.

Author response:
**Response:** We emphasize that this is the first work to understand the data reconstruction attack from the theoretical perspective. We note that it is challenging to bound the error in practical FL settings, e.g., non-IID data across clients. To make the problem tractable, we should require FL training to converge to the optimal model, which needs the loss to be convex.   

Although deep neural networks do not satisfy this assumption, FL has been also used in convex settings. For instance, the original FL [McMahan et al 2017] uses, e.g., 2-layer ReLu NNs, and we also show theoretical results on the convex 2-layer ReLU networks (Pilanci & Ergen, 2020).  We acknowledge it is important future work to generalize our theoretical results to more challenging non-convex losses.

Reviewer comment (Theory):
The reconstruction method is assumed to have a Lipschitz constant. However, the attack methods studied in the paper are optimization-based attacks and do not have an explicit Lipschitz constant. Although the paper proposes a method to approximately calculate the constant, there's no guarantee how large the approximation error would be, which might introduce an additional arbitrarily large noise to calculate the upper bound.

Author response:
**Response:** It is challenging to bound the approximation error of the estimated Lipschitz  constant to the exact one of a *highly nonconvex* network used in the attacks. Also, our goal in the paper is to design a feasible algorithm to estimate the Lipschitz constant. We admit a better estimation algorithm can lead to a tighter upper bounded Lipschitz constant, which is an interesting future work, but not the focus of the paper. Our results show the estimated Lipschitz constant is stable.

Reviewer comment (Evaluation):
As shown in the empirical results, there is a large gap between the upper bound and the attack performance, making it hard to indicate a useful conclusion from this bound.

Author response:
**Response:** Note that the derived error bounds consider all randomness in FL training and data reconstruction, and hence they are the worst-case error under such randomness.  The empirical attack performance of iDLG (Zhao et al., 2020) and DLG (Zhu et al., 2019) are significantly influenced by initial parameters. As shown in Figure 1, the MSE of DLG could be very large (more than 600), which is close to the upper bound (about 700) in this case. Also shown in Figure 6 and Figure 7, GGL exhibits close upper bound and empirical error.