SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Technical difference. First, we focus on a different scope against (Hannun et al., 2021). Hannun et al. (2021) proposed Fisher information loss (FIL) to measure the information leakage risk in the context of model inversion and attribute inference, e.g., only the attribute inference is considered in their experiments. Instead, we evaluate the privacy risk under gradient inversion attacks. Second, our metric is more scalable and applicable to large models. For example, in Eq. (18) in (Hannun et al., 2021), the inverse of the Hessian matrix needs to be calculated even when quantifying the information leakage of only one sample, which is inefficient and intractable for large models. Because of the computation inefficiency, only linear regression and logistic regression models are considered in their theories and experiments. Instead, we verify the feasibility of  $\mathrm{I}^2\mathrm{F}$  in much larger models like ResNet152 on ImageNet in Fig. 3.
Our new findings. First, though the unfairness of information leakage of different samples was discussed in (Hannun et al., 2021), we investigate the issue in a different attack method and justify the commonness of the unfairness in different attacks. Second, we additionally provide other insights than (Hannun et al., 2021). For instance, the influence on gradient inversion of different initialization methods is studied. We also find the influence of perturbations is not equivalent even in the same noise scale. We believe these insights are also critical to the privacy and security community, especially in the area of gradient inversion.
Besides, (Guo et al., 2022; Hayes et al., 2023) propose to bound the reconstruction attack in terms of the attack success rate and the expectation of the  $L_{2}$  distance between the recovered and original image. Nevertheless, their conclusions are based on the differential privacy (DP) quantification framework so it is hard to analyze the influence of other defense mechanisms such as gradient pruning and arbitrary noise. Also, they bound the privacy risk from the statistical sense raised by the randomness of DP, while our work can evaluate the sample-wise worst-case privacy risk at any time during the model training. Moreover, they assume the access of the attacker to the remaining samples (except for the privacy sample) or the multi-round gradient, which is not practical in real gradient inversion scenarios.
H.5 Discussions about the Worst-case Assumption
Our work is mainly built upon the Assumption 3.1 that there is only a unique minimizer for  $L_{I}(x;g)$  given a gradient vector  $g$ . Because of the hardness of optimizing a non-linear objective in attack (Eq. (1)), the worst-case may not be reachable in practice. Here, we discuss when the assumption has to be relaxed and why our method is still applicable. In addition, we emphasize that a stronger attacker exists theoretically, resulting in the necessity of a worst-case assumption.
Non-bijective inversion mapping  $G_{r}(g)$ .  $G_{r}(g)$  could be non-bijective when the loss function is non-convex. In other words, given the same  $g$ , the output of  $G_{r}(g)$  could include multiple choices. We want to argue that this case does not conflict with our assumption. Consider an attack given the exact gradient of a sample. Note that the sample itself is a solution to Eq. (1). Thus, given the exact gradient of a sample, the attack can exactly recover the sample. Even if the solution is non-unique, we can still essentially assume the attack can attain the sample in the worst case.
Optimizing the gradient inversion objective may not converge to the original image. Note that the original image is always an optimal solution for the inversion loss in Eq. (1). Even though the convergence is not guaranteed, there always exists an algorithm that can converge to the original image. To our best knowledge, there is no evidence to show the attack cannot approach the worst case where the original input is recovered. Instead, empirical results have shown that the images can be recovered almost perfectly (Geiping et al., 2020). Thus, due to the sensitivity of privacy, a worst-case assumption is necessary to strictly bound possible privacy risks with arbitrary strong attacks, which is commonly imposed by the literature (Dwork, 2006; Abadi et al., 2016).
I Realistic Impact
Federated learning (FL) (McMahan et al., 2017) is a popular distributed training paradigm that benefits from the data and computation sources from multiple clients. As a principle of FL, clients will upload the local gradient based on the private data to the server for the concerns of data privacy and safety, instead of directly sharing the private data of each client. However, recent works (Geiping et al., 2020; Zhu et al., 2019) show that an attacker, who has access to the local gradient (e.g., a malicious server), can leverage the local gradient to recover the private data of the clients, which we call Deep Gradient Leakage (DGL). Even with large models like Transformer, the attacker can still successfully recover the private data given the gradient (Hatamizadeh et al., 2022).
Auditing potential privacy risks is essentially desired for privacy-sensitive applications, including but not limited to finance (Long et al., 2020), healthcare (Antunes et al., 2022; Xu et al., 2021), and clinical data (Dayan et al., 2021; Roth et al., 2020). Such privacy concerns have been discussed extensively in previous work. For instance, Zhang et al. (2023) raises the privacy concern of FL in financial crime detection while (Kaissis et al., 2021; Li et al., 2023) discusses it in the medical and healthcare applications, respectively.
To echo the demands for privacy risk auditing, we provide a fundamental tool for bounding the worst-case risks of DGL. We show multiple insights, for example, the unfairness of privacy protection using random noise defense. Thus, we expect our work can call the attention of the community to the privacy concern raised by DGL, especially the worst-case instance-level privacy risk. Moreover, We expect  $\mathrm{I}^2\mathrm{F}$  to be a keystone for designing more powerful defense mechanisms.