<figure><img src="image_1.png" /></figure>

HAL
open science

# From Risk to Resilience: Towards Assessing and Mitigating the Risk of Data Reconstruction Attacks in Federated Learning

Xiangrui Xu, Zhize Li, Yufei Han, Bin Wang, Jiqiang Liu, Wei Wang

<figure><img src="image_2.png" /></figure>

## To cite this version:

Xiangrui Xu, Zhize Li, Yufei Han, Bin Wang, Jiqiang Liu, et al. From Risk to Resilience: Towards Assessing and Mitigating the Risk of Data Reconstruction Attacks in Federated Learning. SEC 2025 - 34th USENIX Conference on Security Symposium, Aug 2025, Seattle, United States. pp.1-20, 10.5555/3766078.3766240. hal-05426791

HAL Id: hal-05426791

https://hal.science/hal-05426791v1

Submitted on 19 Dec 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# From Risk to Resilience: Towards Assessing and Mitigating the Risk of Data
Reconstruction Attacks in Federated Learning

Xiangrui Xu

Beijing Jiaotong University

Zhize Li

Singapore Management University

Yufei Han

INRIA Rennes-Bretagne-Atlantique

Bin Wang

Zhejiang Key Laboratory of AIoT Network and Data Security

Jiqiang Liu

Beijing Jiaotong University

Wei Wang*

Beijing Jiaotong University &
Xi'an Jiaotong University

## Abstract

Data Reconstruction Attacks (DRA) pose a significant threat to Federated Learning (FL) systems by enabling adversaries to infer sensitive training data from local clients. Despite extensive research, the question of how to characterize and assess the risk of DRAs in FL systems remains unresolved due to the lack of a theoretically-grounded risk quantification framework. In this work, we address this gap by introducing *Invertibility Loss (InvLoss)* to quantify the maximum achievable effectiveness of DRAs for a given data instance and FL model. We derive a tight and computable upper bound for InvLoss and explore its implications from three perspectives. First, we show that DRA risk is governed by the spectral properties of the Jacobian matrix of exchanged model updates or feature embeddings, providing a unified explanation for the effectiveness of defense methods. Second, we develop InvRE, an InvLoss-based DRA risk estimator that offers attack method-agnostic, comprehensive risk evaluation across data instances and model architectures. Third, we propose two adaptive noise perturbation defenses that enhance FL privacy without harming classification accuracy. Extensive experiments on real-world datasets validate our framework, demonstrating its potential for systematic DRA risk evaluation and mitigation in FL systems.

## 1 Introduction

Federated Learning (FL) has emerged as a promising paradigm to address data silo challenges and comply with privacy regulations such as GDPR [6, 27, 32]. FL can be categorized into Horizontal FL (HFL) and Vertical FL (VFL) based on partitioning of the sample and feature spaces [15,27,28]. In HFL, clients share a common feature extractor while working on different subsets of training data, making it ideal for collaborative tasks like personalized recommendations (e.g., enhancing a global next-word prediction model using keystroke data from Google users [18]). In contrast, VFL involves a

shared subject set with partitioned feature spaces, commonly applied in fields like finance and healthcare. Entities such as banks or hospitals contribute distinct private features for the same subjects to enable predictive analysis [6].

Recent studies [6, 18] reveal that Federated Learning systems, including HFL and VFL, are vulnerable to privacy-stealing attacks. In HFL, adversaries at the central server can access model parameters and updates from participants, enabling membership inference attacks (MIA) [29, 33] or data reconstruction attacks (DRA) to infer training data [9, 26, 47, 52, 53, 56]. In VFL, privacy risks are heightened during prediction. Participants submit feature embeddings derived from their private data to a top-layer server in a split-learning setup [31, 48]. Adversaries controlling the server can exploit these embeddings to reconstruct private feature values, violating data privacy [8, 17, 22, 36, 49, 50]. In this work, we focus on DRA attacks, a critical privacy vulnerability in FL. Existing defenses either lack strong theoretical guarantees [16], or fail to provide consistent protection across FL tasks without significantly compromising utility [9, 56]. These limitations make DRA attacks a critical privacy vulnerability in FL.

Defending DRA attacks in FL involves two critical aspects. **First**, accurately assessing the feasibility and effectiveness of DRA attacks is essential. The central question is: *Given an FL application defined by its model architecture and data distribution, how effective are DRA attacks at achieving high data reconstruction accuracy?* The model updates or embeddings shared by FL clients can be viewed as a transformation of the input data. DRA attacks aim to invert this transformation [49, 56], with their effectiveness determined by the computational feasibility of finding this inverse, also known as the transformation's invertibility. However, the highly nonlinear nature of deep neural networks (DNNs) in FL models makes it challenging to theoretically and empirically quantify this invertibility and its associated privacy risks. **Second**, FL stakeholders must select model architectures that enhance privacy protection and understand how privacy risks vary across data sources [16]. Accurate DRA risk estimation is crucial for measuring and comparing risks across different input sam-

<footnote>*Corresponding author (wangweil1@bjtu.edu.cn).</footnote>
ples and models, enabling informed decisions to strengthen
privacy in FL systems.

Most prior studies to assess DRA risks [14, 22, 44, 51] use
an *assess-by-attack* strategy, evaluating privacy protection by
conducting specific DRAs and analyzing reconstruction ac-
curacy. Higher accuracy indicates more effective attacks and
greater vulnerability of FL systems. However, this strategy is
inherently unreliable, as it risks underestimating vulnerabili-
ties. Advanced, carefully crafted attacks could still succeed
where earlier, simpler attempts failed. Recent efforts explore
information-theoretic metrics to quantify DRA risks in FL,
such as mutual information (MI)-based [35, 42, 48] and Fisher
information-based approaches (dFIL) [7, 11]. For instance,
Tan et al. [42] proposed using MI between data and model
parameters to assess the feasibility of DRA attacks in HFL.
However, directly computing the MI metric for non-linear FL
models is non-trivial, leading to approximations that focus
on data-space MI. These approximations fail to capture corre-
lations between training data and model parameters, which
limits their accuracy in evaluating privacy risks across diverse
model architectures. Similarly, dFIL [11] uses Fisher informa-
tion to establish a lower bound on reconstruction error. How-
ever, the applicability of dFIL [11] is constrained by its depen-
dence on an unbiased data reconstruction estimator, whereas
most DRA attacks rely on biased estimators. Moreover, dFIL
cannot effectively compare DRA risks across different model
architectures. In addition, both methods are tailored to spe-
cific FL mechanisms (HFL or VFL), leaving the intrinsic
connections between DRA vulnerabilities across mechanisms
unexplored. These limitations highlight the need for system-
atic assessment and comparison of DRA risks across various
FL mechanisms, datasets, and model architectures.

Our work addresses the key limitations in DRA attack analysis by characterizing and assessing the invertibility of the transformation from data instances to model space. We first introduce *InvLoss* (*InvLoss*), which quantifies the feasibility of DRAs by measuring the minimum reconstruction error achievable through an inverse transformation that maps model updates or feature embeddings exchanged in FL back to data space. Lower *InvLoss* values indicate a higher DRA risk. We unveil that DRAs minimize *InvLoss*, which is in nature equivalent to approximating the Moore-Penrose inverse of the Jacobian matrix of the observed model updates or embeddings (accessible to the attacker) with respect to input data. The upper bound of *InvLoss* in FL is thus determined by the spectral properties of the Jacobian matrix, which can be efficiently computed using Singular Value Decomposition (SVD). Unlike previous approaches, our bound does not assume unbiased DRA estimators, offering a more accurate quantification of information leak risks. This enables a unified framework to assess DRA risks across both HFL and VFL, and compare risks across data instances and model architectures.

We further extend the InvLoss framework to understand

how the existing noise perturbation and information compres-
sion methods increase reconstruction error and reduce DRA
risk. We propose two adaptive noise perturbation strategies
that optimize noise injection based on the spectrum of the
Jacobian matrix, providing stronger privacy protection with-
out sacrificing classification accuracy. Finally, we introduce
InvRE, an exact estimator of InvLoss, validated through exten-
sive empirical evaluation as a reliable indicator of DRA risk
across diverse models and data sources.

**Contributions.** We summarize our contribution in this study from three perspectives:

* We introduce Invertibility Loss (*InvLoss*) as a feasibility measurement of DRA attacks in FL. To evaluate the DRA risk, we establish the upper of *InvLoss*, forming a unified theoretical reasoning about the main factors determining the effectiveness of DRA attacks in different FL mechanisms. We further extend the theoretical analysis to explain the effectiveness of the existing defense mechanisms against DRA attacks.

* We develop InvRE for practical DRA risk estimation based on the theoretical upper bound of InvLoss. InvRE quantifies and compares DRA attack feasibility across various FL mechanisms, model architectures, and data sources, independent of specific attack methods. We validate InvRE using 4 diverse datasets, 3 widely-used FL model architectures, and 3 state-of-the-art DRA attack methods for both HFL and VFL. Results confirm that InvRE consistently reflects DRA risk levels across different tasks with/without privacy defense methods.

* Leveraging the insights from the theoretical analysis, we propose two adaptive noise perturbation strategies that dynamically modulate noise injection based on the spectral properties of the Jacobian matrix. Empirical results confirm that they can reach strong privacy protection without harming the utility of FL systems.

## 2 Preliminaries and Related Work

Let $\mathbf{x} \in \mathbb{R}^m$ represent a data instance hosted by the client in FL (HFL or VFL) systems. $\mathcal{F}(\mathbf{x}) \in \mathbb{R}^p$ denotes the shared model gradients or feature embedding vectors in HFL or VFL respectively. Table 1 summarizes the notations in our work.

**Federated Learning.** FL enables a group of remote clients $U = \{u_1, \cdots, u_n\}$ to collaboratively train a machine learning model. Each client $u_i$ contributes a private set of training data, which remains inaccessible to other participants due to privacy concerns. Based on the partitioning of sample and feature spaces, FL is classified into Horizontal FL (HFL) and Vertical FL (VFL) [18, 28].

In HFL, clients collaboratively train a global model by shar-
ing model updates with a central server, as shown in Fig. 1a.
At each iteration *t*, the server selects a subset of *n* clients and
<figure><img src="image_4.png" /><figcaption>Figure 1: Federated Learning architecture.</figcaption></figure>

<table><caption>Table 1: Notations and description.</caption><thead><tr><th>Notations</th><th>Description</th></tr></thead><tbody><tr><td><b>x</b> &isin; &#8477;<sup>m</sup></td><td>Input data from the client</td></tr><tr><td><b>F</b>(.)</td><td>Transformation function applied to <b>x</b>: &#8477;<sup>m</sup> &mapsto; &#8477;<sup>p</sup></td></tr><tr><td><b>A</b><sub><b>x</b></sub>(.)</td><td>The inverse function applied to <b>F</b>(.): &#8477;<sup>p</sup> &mapsto; &#8477;<sup>m</sup></td></tr><tr><td><b>G</b><sub><b>x</b></sub></td><td>Jacobian matrix &part;<b>F</b>(<b>x</b>)/&part;<b>x</b> w.r.t a data instance</td></tr><tr><td><b>A</b><sub><b>x</b></sub></td><td>The inverse matrix for data reconstruction</td></tr><tr><td><b>U</b>, <b>&Sigma;</b>, <b>V</b></td><td>Left-singular vectors, matrix of singular values, and right-singular vectors of <b>G</b><sub><b>x</b></sub></td></tr><tr><td>&sigma;<sub><b>i</b></sub></td><td><b>i</b>-th singular value of the Jacobian matrix <b>G</b><sub><b>x</b></sub></td></tr><tr><td><b>&epsilon;</b></td><td>Noise matrix</td></tr><tr><td><b>n</b></td><td>Noise matrix in the singular space</td></tr></tbody></table>

shares the global model $\theta'$ with them. Each client $u_i$ then perform local training to compute local model updates $\Delta\theta_i'$. These model updates are sent to the server for aggregation, typically using a weighted average: $\theta'^{i+1} = \text{Agg}(\Delta\theta_1', \dots, \Delta\theta_n'; \theta'^i)$, producing the updated model $\theta'^{i+1}$. This process repeats until convergence, after which clients use the trained global model for local predictions. VFL uses a split-learning framework for both training and inference, as shown in Fig. 1b [22, 36, 43]. Clients host the bottom model for feature embedding generation. The server manages the top model to aggregate the local clients' feature embeddings for classification. During training, intermediate feature embeddings and gradients of bottom models are exchanged between the client and server until a collaboratively trained model is obtained. After training, bottom models can be transferred to the server for evaluation. In prediction, the server sends the input sample's ID to the clients. All clients then collaboratively compute the final classification output.

**Data Reconstruction Attacks in FL.** In data reconstruction attack scenarios involving FL systems, the attacker can either observe the model updates submitted by local clients in HFL [9, 47, 56] or analyze the embedding vectors generated by the bottom models owned by local clients in VFL [8, 17, 49]. The model updates or the embeddings can be considered as a functional transformation $\mathcal{F}$ of the input instance $\mathbf{x}$ held by the clients, i.e., $\mathcal{F}(\mathbf{x})$. The goal of the attacker can be thus defined as a function-matching problem [56]:

$$ \hat{\mathbf{x}}^* = \underset{\hat{\mathbf{x}}}{\operatorname{argmin}} \| \mathcal{F}(\mathbf{x}) - \mathcal{F}(\hat{\mathbf{x}}) \| \quad (1) $$

where $\hat{\mathbf{x}}$ represents an estimate of the true input $\mathbf{x}$, and $\| \cdot \|$ denotes the Euclidean distance. Minimizing Eq. (1) aims to identify the reconstructed instance $\hat{\mathbf{x}}$ that most closely approximates the functional output of the true input $\mathbf{x}$. This approach was initially proposed in the context of HFL with gradient matching [56], it has been extended to VFL via embedding matching, as shown in [17]. Subsequent research has demonstrated that incorporating prior knowledge about the input can enhance this regression task [9, 26, 45, 52]. Inverting Gradients (IG) [9] proposed to employ regularization techniques, such as the total variation (TV) prior, to mitigate high-frequency artifacts in $\hat{\mathbf{x}}$ with cosine distance as the distance metric. Similar approaches can also be adapted for VFL, as detailed in [17]. More recent studies [31, 47, 49], such as Conditional Generative Instance Reconstruction (CGIR) [47] and PISTE [49], have been proposed for HFL and VFL, respectively. These methods leverage a deep neural network (DNN) generator to assist in solving the regression task in Eq. (1). Recent work further demonstrate that a malicious server can actively manipulate the training process to steal client's data [2, 36, 55]. In HFL, LOKI [55] proposed to modify the model architecture and the parameters sent to clients to facilitate the attack. However, in practice, FL participants usually agree on the model architecture in advance, making abnormal model structures easy to detect and such attacks ineffective. In the context of VFL, FSHA [36] also introduced a DNN capable of mapping $\mathcal{F}(\mathbf{x})$ directly to $\hat{\mathbf{x}}$. Although this methodology facilitates effective reconstruction, it disrupts the conventional VFL training paradigm, thereby substantially degrading the performance of the classifier trained within the system.

**Privacy Metrics for FL.** Xu et al. [42] analyzed the privacy risks in HFL by estimating the mutual information-based channel capacity between training data and shared model parameters during each training round. However, this bound is computationally infeasible for high-dimensional, non-linear model architectures due to their inherent complexity. While [42] introduces a tractable approximation of channel capacity using data-space mutual information, this approximation fails to account for the correlation between training
data and the model parameters trained on it. As a result, it does not effectively capture the variability in privacy risks across different federated learning systems with diverse datasets and architectures. Noorbakhsh et al. [35] proposed Inf²Guard to mitigate inference attacks to balance privacy protection and system utility. It is designed to minimize the mutual information between original data and feature embeddings, while enhancing the correlation between feature embeddings and class labels. However, this approach does not serve to characterize and quantify the feasibility of DRA attacks across different datasets and models. It does not develop a quantified assessment of the DRA risk. Furthermore, Inf²Guard can not be adopted in VFL as VFL clients do not have class labels during training. Though it can be used in HFL training. Attackers in HFL only need to access model parameters for DRA, not embeddings. Guo et al. [11] approached the quantification of DRA risk using the Fisher information of shared feature embeddings in VFL. However, dFIL is limited by its dependence on unbiased data reconstruction estimators for DRA attacks, whereas most practical DRA attacks rely on biased estimators. Additionally, dFIL does not facilitate comparisons of DRA risks across various model architectures. These limitations highlight the need for a unified and theoretically-guaranteed assessment framework to compare DRA risks across a wide range of data sources and model architectures.

## 3 Invertibility Loss (InvLoss)

We begin by formally introducing the concept of Invertibility Loss InvLoss, which defines the attack objective in DRA attacks. Next, we establish an upper bound for InvLoss, laying the groundwork for a theoretical analysis of the feasibility of DRA attacks in FL systems. Our theoretical investigation of InvLoss is approached from two perspectives. First, we leverage the proposed upper bound to analyze how existing defense mechanisms—such as injecting data-agnostic noise perturbations or performing parameter pruning—can mitigate the feasibility of DRA attacks. Building on these insights, we propose adapting noise perturbation based on the spectral properties of the Jacobian matrix of the model gradients or feature embeddings. This approach significantly improves task utility while maintaining robust privacy protection. Second, we introduce a risk estimation framework for DRA attacks, termed InvRE, which is grounded in the theoretical analysis of InvLoss. For FL tasks involving diverse data sources and model architectures, InvRE provides a bounded quantification of risk, enabling consistent assessment of vulnerability to DRA attacks across different FL scenarios.

**Threat model.** In this work, we focus on the privacy leakage caused by DRA attacks in HFL and VFL systems. Specifically, we consider an honest but curious server as an attacker, who aims to reconstruct a target client's private input images by exploiting the parameters shared during the FL process. The attacker adheres to the FL protocol, i.e., it does not interfere

with the training procedure or modify the model architecture to facilitate the attack, but solely leverages the exposed parameters (e.g., gradients or embeddings) from the victim client. In HFL, such attacks occur during training, where the attacker residing at the central server observes model updates submitted by the victim client, consistent with prior work [9, 47, 56]. In contrast, privacy attacks against VFL occur at the inference time, as demonstrated in [22, 31]. The attacker at the server analyzes embedding vectors produced by the client's local model for joint prediction to reversely estimate the input data.

**Definition 1 (Invertibility Loss)** Given a target data instance $\mathbf{x}$, $\mathcal{F}(\mathbf{x})$ denotes the shared model updates or feature embedding vectors in HFL or VFL respectively. Such shared information can be considered as a functional transformation of $\mathbf{x}$. We define the inverse transformation function $\mathcal{A}_{\mathbf{x}}$ mapping $\mathcal{F}(\mathbf{x})$ to the data space. A DRA attack can be considered to estimate $\mathcal{A}_{\mathbf{x}}$ minimizing the Invertibility Loss (InvLoss) over $\mathbf{x}$, which gives:

$$ \mathrm{InvLoss}_{\mathbf{x}} = \min_{\mathcal{A}_{\mathbf{x}}} \| \mathcal{A}_{\mathbf{x}}(\mathcal{F}(\mathbf{x})) - \mathbf{x} \|^2 \quad (2) $$

where $\| \cdot \|$ denotes the Euclidean distance between the reconstructed data profile and the ground truth.

The value of InvLoss$_{\mathbf{x}}$ reflects the minimal possible data reconstruction error using the optimal inverse transformation $\mathcal{A}_{\mathbf{x}}$. It quantifies the discrepancy between the reconstructed data and the original data. The definition of InvLoss$_{\mathbf{x}}$ aligns with the objective function of gradient- or embedding-matching-based DRA attack methods, as given in Eq. (1). Therefore, Eq. (2) is used to evaluate the effectiveness of DRA attacks, where a lower InvLoss$_{\mathbf{x}}$ indicates a more effective attack and, consequently, a higher risk of DRA attacks.

### 3.1 The upper bound of InvLoss

**Rank-k optimal DRA attacker.** To derive a tight upper bound of InvLoss$_{\mathbf{x}}$ for a rank-k optimal attacker, we first locally linearize the transformation function $\mathcal{F}(\mathbf{x})$ via Taylor expansion. We let $\mathcal{F}(\mathbf{x}) = \mathbf{G}_{\mathbf{x}}\mathbf{x} + o(\delta^2)$, where $o(\delta^2)$ denotes higher-order infinitesimals. For simplicity, we denote $o(\delta^2)$ as $C$ in subsequent discussions. To estimate the inverse transform $\mathbf{A}_{\mathbf{x}}$, DRA attackers may leverage various generative models and optimization techniques. Regardless of the specific learning techniques used for attack, minimizing InvLoss$_{\mathbf{x}}$ involves solving a linear regression problem. By employing the Cauchy-Swartz inequality, the bound for InvLoss$_{\mathbf{x}}$ can be expressed as:

$$ \begin{aligned} \mathrm{InvLoss}_{\mathbf{x}} &\le \|\mathbf{A}_{\mathbf{x}}\mathbf{G}_{\mathbf{x}}\mathbf{x} - \mathbf{x}\|^2 + C \\ &\le \|(\mathbf{A}_{\mathbf{x}}\mathbf{G}_{\mathbf{x}} - \mathbf{I})\mathbf{x}\|^2 + C \end{aligned} \quad (3) $$
With Eq. (3), minimizing $\text{InvLoss}_x$ to estimate the inverse transform $\mathcal{A}_x$ as defined in Eq. (2) is equivalent to solving the following linear least squares regression problem:

$$ \mathbf{A}_{\mathbf{x}}^{*} = \underset{\mathbf{A}_{\mathbf{x}}}{\operatorname{argmin}} \|\mathbf{A}_{\mathbf{x}}\mathbf{G}_{\mathbf{x}} - \mathbf{I}\|_{F}^{2} \quad (4) $$

where $\mathbf{I}$ is the identity matrix, and $\|\cdot\|_F$ denotes the Frobenius norm of a matrix. In the optimal case, the DRA attacker can reach the lowest reconstruction error when $\mathbf{A}_x^*$ is the Moores-Penrose generalized inverse of $\mathbf{G}_x$:

$$ \mathbf{A}_{\mathbf{x}}^{*} = \mathbf{V}\mathbf{\Sigma}^{\dagger}\mathbf{U}^{\top} \quad (5) $$

where $\mathbf{V}$ and $\mathbf{U}$ are the left and right singular vectors of $\mathbf{G}_x$. The matrix $\mathbf{\Sigma}^\dagger$ is a diagonal matrix. Each of its diagonal elements computes the reciprocal of each non-zero singular value of $\mathbf{G}_x$, while leaving the zeros in place.

In practice, a weak DRA attacker can not achieve the optimal solution to the inverse transform and can only approximate $\mathbf{A}_x^*$ up to a rank $k$, with $k$ no larger than the number of non-zero singular values of $\mathbf{G}_x$. We thus define a rank-k optimal DRA attacker on a given instance $\mathbf{x}$ as the one that obtains the best rank-k approximation $\tilde{\mathbf{A}}_x^{k*}$ to $\mathbf{A}_x^*$:

$$ \tilde{\mathbf{A}}_{\mathbf{x}}^{k*} = \mathbf{V}_{1:k}\tilde{\mathbf{\Sigma}}^{\dagger}\mathbf{U}_{1:k}^{\top} \quad (6) $$

where $\mathbf{V}_{1:k}$ and $\mathbf{U}_{1:k}^\top$ defines the first $k$ columns of the right and left singular vectors of $\mathbf{G}_x$, corresponding to its $k$ largest non-zero singular values of. $\tilde{\mathbf{\Sigma}}^\dagger$ is the truncated version of $\mathbf{\Sigma}^\dagger$ containing the reciprocals of the $k$ largest non-zero singular values of $\mathbf{G}_x$ in its diagonal, while the remaining diagonal elements are set to zeros.

Using Def. 1, we establish an upper bound of the data reconstruction error of a rank-k optimal DRA attacker over the input instance $\mathbf{x}$ and the model $\mathcal{F}$, as stated in Theorem 1.

**Theorem 1 (InvLoss for rank-k attacker)** Given a normalized example $\mathbf{x} \in \mathbb{R}^m$ as the target of DRA and its corresponding shared parameters $\mathcal{F}(\mathbf{x}) \in \mathbb{R}^p$, the InvLoss for a rank-k optimal DRA attacker is bounded as:

$$ \text{InvLoss}_{\mathbf{x}} \le \sum_{i=k+1}^{d} (\mathbf{V}_{i}^{\top} \mathbf{x})^2 + C \quad (7) $$

where $k$ represents the rank of the approximation defined in Eq. (6) derived by the rank-k optimal attacker, and $d = \min\{m, p\}$. $\mathbf{V}_i$ denotes the $i$-th column of $\mathbf{V}$, denoting the singular vector corresponding to the $i$-th largest singular value. $\mathbf{V}_i^\top \mathbf{x}$ hence denotes the projection of $\mathbf{x}$ onto the singular vector $\mathbf{V}_i$.

## 3.2 The upper bound of InvLoss with existing defenses

With Theorem 1, we can systematically explain the effectiveness of the existing defense strategies against DRA via the

lens of InvLoss. These defense methods can be broadly classified into two categories: “Noise Perturbation” [1, 48, 56] and “Information Compression (IC)” [14, 51] based defenses.

Noise Perturbation-based defenses involve adding noise to the raw features to obscure sensitive information, thereby enhancing data privacy protection [1, 25, 48, 56]. Depending on where the noise is injected, the noise perturbation-based defense mechanisms can be further categorized: (i) Data-Level Noise Perturbation (DNP), where noise is directly applied to input data before processing in HFL and VFL; (ii) Gradient-Level Noise Perturbation (GNP), where noise is added to the gradients exchanged during training in HFL; and (iii) Embedding-Level Noise Perturbation (ENP), where noise is incorporated into embedding vectors generated by the client's bottom model in VFL. IC-based defense aims to reduce the information exchanged between the client and the server, thereby reducing the risk of sensitive information leakage. Two prominent approaches within this category are Prune [51] and Dropout [14]. Prune involves selectively removing less important shared features. Dropout, on the other hand, randomly sets a subset of shared features to zero during communication. While the main purpose of them is to improve communication efficiency [23, 24, 38, 54] or prevent overfitting [14], Prune and Dropout have been adapted to reduce the risk of data leakage by limiting the exposure of critical information [22, 56]. In the following, we extend Theorem 1 the theoretical implications of DNP, GNP / ENP, and IC for InvLoss in Theorem 2, Theorem 3, and Theorem 4, respectively.

**Theorem 2 (InvLoss under DNP)** Given a normalized instance $\mathbf{x} \in \mathbb{R}^m$, subjected to DNP based defense, and the shared information $\mathcal{F}(\mathbf{x})$ accessed by the attacker, the InvLoss of a rank-k optimal DRA attacker is bounded as:

$$ \text{InvLoss}_{\mathbf{x}} \le \sum_{i=k+1}^{d} (\mathbf{V}_{i}^{\top} \mathbf{x})^2 + \sum_{i=1}^{k} \frac{(\mathbf{V}_{i}^{\top} \mathbf{\epsilon})^2}{m} + C \quad (8) $$

where $\mathbf{\epsilon}$ is the injected noise perturbation to $\mathbf{x}$; $k$ denotes the rank of the estimation defined in Eq. (6).

Compared to InvLoss$_x$ without any defense, as described in Theorem 1, Theorem 2 indicates that the scale of noise injected into the input data is a critical factor in the defense effectiveness of DNP. Specifically, increasing the magnitude of the injected noise enlarges the upper bound in Eq. (8), which enhances the protection strength of DNP. This observation is supported by experimental results presented in Fig. 7a and Fig. 8a for HFL and VFL respectively.

**Theorem 3 (InvLoss under GNP / ENP)** Given a normalized instance $\mathbf{x} \in \mathbb{R}^m$ targeted by a DRA and the shared information $\mathcal{F}(\mathbf{x})$, the InvLoss of a rank-k optimal DRA attacker with GNP or ENP based defense applied is bounded as:

$$ \text{InvLoss}_{\mathbf{x}} \le \sum_{i=k+1}^{d} (\mathbf{V}_{i}^{\top} \mathbf{x})^2 + \sum_{i=1}^{k} \frac{(\mathbf{U}_{i}^{\top} \mathbf{\epsilon})^2}{\sigma_{i}^2 p} + C \quad (9) $$
where $\epsilon$ represents the noise perturbation injected into $\mathcal{F}(\mathbf{x})$; $k$ denotes the rank of the estimation as defined in Eq. (6). $\sigma_i$ is the singular values of $G_x$, and $p$ is the number of the dimensions in the injected noise. $U_i$ denotes the left singular vector of the Jacobian matrix of $\mathcal{F}(\mathbf{x})$ corresponding to the $i$-th largest singular value.

Theorem 3 demonstrates that increasing the magnitude of the injected noise applied to the gradients or embeddings in FL communication significantly improves the defense capability of GNP / ENP. We further verify this theoretical finding in the experimental results illustrated by Fig. 7b and Fig. 8b for HFL and VFL respectively.

**Theorem 4 (InvLoss under IC)** Given a normalized instance $\mathbf{x} \in \mathbb{R}^m$ and the shared information $\mathcal{F}(\mathbf{x})$, the InvLoss of a rank-k optimal DRA attacker with IC-based defense applied is bounded from below and above as:

$$ \sum_{i=d-q+1}^{d} (\mathbf{V}_{i}^{\top} \mathbf{x})^2 \leq \text{InvLoss}_{x} \leq \sum_{i=k+1}^{d} (\mathbf{V}_{i}^{\top} \mathbf{x})^2 + C \quad (10) $$

where $q$ represents the number of the pruned/dropped dimensions in the shared $\mathcal{F}(\mathbf{x})$ in the IC based defense methods.

Theorem 4 indicates that IC-based defense strategies raise the lower bound of InvLoss by reducing information shared with the attacker during FL communication. Dropping more dimensions results in a higher lower bound, thereby providing stronger defense against DRA attacks. Applying IC-based defenses (such as pruning and dropout) is equivalent to setting certain dimensions of the Jacobian matrix $G_x$ to 0, thereby lowering its rank. This reduction in rank limits the number of singular values available for DRA attacks, thus decreasing the risk of privacy leakage. Consistent results demonstrating the privacy-enhancing effects of IC-based methods are shown in Fig. (7c)–(7d) and Fig. (8c)–(8d) for HFL and VFL, respectively.

## 3.3 Adaptive noise perturbation-based defense

Inspired by the upper bound analysis of InvLoss provided in Theorem 2 and Theorems 3, we propose two adaptive noise strategies that preserve protection strength while minimizing utility loss: *Adaptive Data-level Noise Perturbation (InvL-DNP)* and *Adaptive Gradient/Embedding-level Noise Perturbation (InvL-GNP or InvL-ENP)*. Injecting the noise perturbation in the existing defense methods is to introduce perturbation blindly to the projection of the input data instance along all the singular vectors of the Jacobian matrix $G_x$. However, as demonstrated by Eq. (8) in Theorem 2 and Eq. (9) in Theorem 3, the protection-enhancing effect of the injected noise $\epsilon$ in DNP / GNP / ENP over the upper bound of InvLoss is determined only by the projection of the noise $\epsilon$ onto the singular vectors of $G_x$ corresponding to the largest

<figure><img src="image_5.png" /><figcaption>Figure 2: Overview of the proposed InvLoss and InvL-based defenses. InvLoss measures reconstruction risk using key spectral components of the Jacobian $G_x$, while InvL-based defenses inject calibrated noise to these components to protect privacy while preserving model utility.</figcaption></figure>

$k$ singular values. On the contrary, the noise projected to the singular vectors with smaller singular values does not help increase the upper bound, thus contribute barely to improve the protection strength. Therefore, we propose to leverage this property to adaptively truncate the spectral projection of the injected noise perturbation. We strategically introduce noise only into the top-k singular subspace to maintain the upper bound value of InvLoss in Theorem 2, hence maintaining the protection strength against DRA attacks and reducing the magnitude of the added noise.

The implementation steps for InvL-DNP and InvL-GNP / InvL-ENP are organized as follows:

**Step 1.** Perform SVD of the Jacobian matrix $G_x$, $G_x = U\Sigma V^\top$.

**Step 2.** Sample a random noise matrix $\epsilon$ from a centered Gaussian distribution $N(0, \delta)$. $\delta$ denotes the predefined variance.

**Step 3.** Project $\epsilon$ onto the singular vectors of $G_x$, yielding $n = V^\top \epsilon$ for InvL-DNP, and $n = U^\top \epsilon$ for InvL-GNP / InvL-ENP. Furthermore, we set the noise projection corresponding to the all singular values $\sigma_i$ except from the largest $k$ ones as zeros, i.e., $n[k:] = 0$. In our experimental study, we empirically find that setting $k$ to above the top 95% of the sum of all singular values can strike a balance between the defense performance and its impact on the main task of FL.

Moreover, for InvL-GNP / InvL-ENP, according to Eq. (9) in Theorem 3, the noise projection onto the singular vectors corresponding to the leading $k$ singular values has a diminishing magnitude with increasingly larger singular values $\sigma_i$. Therefore, we can additionally set the noise projection dimensions corresponding to the top $j$ ($j < k$) singular vectors to zero to reduce the utility loss. In our experiments, we empirically set $j$ to the top 60% of all singular values, ensuring a bal-
ance between reducing the impact of noise and maintaining practicality.

**Step 4.** Map the truncated noise projection $n$ back to the original data space to derive the adaptive noise, i.e., $\hat{\mathbf{e}} = \mathbf{V}n$ for InvL-DNP, and $\hat{\mathbf{e}} = \mathbf{U}n$ for InvL-GNP / InvL-ENP.

We add the adaptive noise $\hat{\mathbf{e}}$ to the original data matrix $\mathbf{x}$ (InvL-DNP) to produce the perturbed data, $\hat{\mathbf{x}} = \mathbf{x} + \hat{\mathbf{e}}$. These perturbed data instances are used in HFL and VFL training or inference. For InvL-GNP, we introduce the adaptive noise $\hat{\mathbf{e}}$ to the model gradient in each round of HFL training. For InvL-ENP, we add the adaptive noise to the feature embedding produced by the bottom model of the target client in VFL. The noise-perturbed feature embedding is then fed to the top-level model for inference. Beyond the theoretical reasoning, we provide the experimental results of InvL-DNP and InvL-GNP / InvL-ENP in Table 2 and Table 3 for HFL and VFL systems, respectively. The results confirm that FL systems deployed with the adaptive noise perturbation methods reach higher classification accuracy yet maintain the same or even stronger protection strength, compared to the vanilla DNP and GNP/ENP-based defense. To facilitate scalable computation, for InvL-GNP, we perform the SVD decomposition on the averaged Jacobian matrix with respect to the class centers to derive the adaptive noise.

## 3.4 Estimating the risk of DRA

Based on the upper bound of InvLoss in Theorem 1, we propose an estimator to the risk of DRA attacks for a given FL system. We consider evaluating the risk under varying levels of attack strength, in order to make the risk estimation independent from the choice of DRA attack methods. We hereafter name the proposed quantified risk evaluator as InvRE (Invertibility-based Risk Estimator).

According to Theorem 1, increasingly larger $k$ indicates a stronger rank-$k$ optimal attacker. On one hand, with a higher rank $k$, more singular vectors are used by the attacker to compute the Moore-Penrose inverse to the Jacobian matrix, thus producing more accurate data reconstruction as suggested in Eq. (7). On the other hand, uncovering more singular vectors increases the computational challenge to the attack method from two perspectives. First, singular vectors with marginally small singular values are difficult to derive due to numerical instability. Small singular values are close to zero, meaning their reciprocals used in the pseudo-inverse calculation become excessively large, which amplifies any small errors in the SVD results. This leads to a loss of numerical precision, as even tiny perturbations in the data or rounding errors can disproportionately affect the computed results. As a result, the singular vectors associated with these small singular values become unreliable, making their accurate recovery computationally difficult and sensitive to perturbations. Second, a small gap between successive singular values makes it difficult to distinguish the corresponding singular vectors. In such

cases, these singular vectors can overlap or become nearly indistinguishable. Small changes to the Jacobian matrix, such as rounding errors or noise, can hence cause significant shifts in the orientations of such singular vectors. In summary, smaller magnitudes of the singular values and smaller singular value gap reduce the computational feasibility of exploiting all $k$ singular vectors for data reconstruction, thus making the corresponding rank $k$ attack less likely to conduct.

Integrating all these considerations, we formulate InvRE in Def. 2 using the averaged upper bound of InvLoss, weighed by the singular value magnitudes and the gap between singular values of the Jacobian matrix.

**Definition 2 (InvRE estimator of DRA risk)** Given a normalized data instance $\mathbf{x} \in \mathbb{R}^m$ and the corresponding shared parameters $\mathcal{F}(\mathbf{x}) \in \mathbb{R}^p$, the risk estimator for DRA attacks over $\mathbf{x}$ is given as:

$$ \begin{aligned} \text{InvRE}_{\mathbf{x}} &= \frac{1}{1 + \exp\left(\beta\left(\sum_{k=1}^{d} P_k \tau_k - \alpha\right)\right)} \\ \text{where } P_k &= \frac{1/T_k}{\sum_{j=1}^{d} (1/T_j)}, \quad \text{with } T_k = \sum_{i=1}^{k} \frac{\sigma_i}{\sigma_i - \sigma_{i+1}} \quad (11) \\ \text{and } \tau_k &= \sum_{k+1}^{d} (\mathbf{V}^{\top}\mathbf{x})^2 \end{aligned} $$

where $\mathbf{G}_\mathbf{x}$ denotes the Jacobian matrix. $d = \min\{m, p\}$, $P_k$ quantifies the likelihood of successfully reconstructing the singular vectors of the largest $k$ singular values of $\mathbf{G}_\mathbf{x}$. $\tau_k$ represents the upper bound of InvLoss with respect to a rank-$k$ optimal attacker and given $\mathbf{x}$ and $\mathcal{F}$. $\alpha$ is the empirical mean of the values of $\sum_{k=1}^{d} P_k \tau_k$ derived over various datasets. $\beta$ is a scaling factor chosen empirically to avoid saturated values in the softmax function. In practice, we set $\beta$ to 5.

**Properties of InvRE. First,** InvRE is computed in an agnostic way to specific attack methods $\mathcal{A}_\mathbf{x}$. Instead of choosing a concrete attack strategy for the risk assessment, Eq. (11) takes a weighted average of $\tau_k$ to assess the overall DRA risk with respect to different attack strength, varied from weak to strong DRA attackers. $P_k$ takes the quotient of each singular value $\sigma_i$ ($i \le k$) and the gap between successive singular values $\sigma_i$ and $\sigma_{i-1}$. As discussed before, a higher/lower $\sigma_i$ with a smaller/larger gap indicates more/less difficult to reach the rank-$k$ optimal DRA attacks respectively. Therefore, we assign a higher/lower $P_k$ accordingly to the estimated invertibility loss InvLoss of the corresponding rank-$k$ optimal attacker. With this setting, InvRE produces a DRA risk estimation weighed by the computational feasibility of DRA attacks of different strength levels. More feasible DRA attacks with higher $P_k$ are more prevalent in practices, as such attack scenarios are easier to deliver. Hence these attack scenarios are given higher weights in the computation of InvRE. On the contrary, stronger attacks with higher $k$ are in general
less feasible. Though they produce better data reconstruction accuracy, they are assigned with lower weights in InvRE.

**Second**, Eq. (11) provides an instance-specific estimation of DRA risk. This allows users of FL systems to understand how data leakage risks vary across different data instances x. Downstream FL applications can apply stricter protection measures to data instances with higher information leakage risks while relaxing controls for those with lower risks, thereby minimizing unnecessary perturbations that might affect classification performance. Empirically, we demonstrate this instance-wise DRA risk variation in both HFL and VFL systems using LeNet and ResNet architectures, as shown in Fig. 3. We reveal that the variations in InvRE-based risk estimation align closely with the practical attack performance across different instances, validating InvRE's effectiveness in evaluating instance-specific DRA risks.

**Third**, InvRE does not pose any assumptions about the model architecture used in the target FL systems. InvRE can thus be applied to arbitrary DNN models and measure the variation of the DRA risk determined by different model architecture. This facilitate the FL users to choose appropriate models in the target FL systems to mitigate the data leak risk. The experimental results, as shown in Fig. 5 and Fig. 6 for HFL and VFL systems, confirm that InvRE-based risk estimation aligns consistently with the reported DRA attack performance metrics across DNN model architectures of varying complexities. These findings validate InvRE's capability of accurately capturing variations of the DRA risk across different model structures used in FL systems.

# 4 Experiments

In this section, we present a comprehensive empirical evaluation to validate the InvRE-based DRA risk estimator across 4 datasets from diversified applications and 3 model architectures with varying complexities used in HFL and VFL systems. Furthermore, we assess the defense effectiveness of the proposed adaptive noise perturbation-based protection scheme against DRA. Our empirical study is dedicated to providing systematic answers to the following questions:

**Q1:** Can InvRE quantify the DRA risk associated with different data instances in both HFL and VFL frameworks?

**Q2:** Can InvRE measure the DRA risk associated with different model structures adopted in FL systems with different DNN architectures?

**Q3:** Can the theoretical upper bound of InvLoss in Sec. 3.2 explain the effectiveness of existing different defense mechanisms against DRA in FL systems?

**Q4:** How can the InvRE risk estimator be practically applied to guide client-side decision making in FL systems?

**Q5:** Can the proposed adaptive noise perturbation mechanism improve the trade-off between model utility and data privacy protection compared to state-of-the-art noise perturbation and information compression defense methods in HFL and VFL?

The answer to **Q1** and **Q2** can provide empirical observations to verify whether the proposed InvRE risk estimator can reflect the effectiveness of DRA attacks, thereby capturing the variation in DRA risk levels across different data instances and model architectures. Empirical observations related to **Q3** further confirm that InvRE can effectively assess DRA risk under different deployed defense mechanisms. Based on the above three questions, **Q4** investigates the practical implications of the InvRE risk estimator in guiding client-side decision making within FL systems. **Q5** evaluates the overall effectiveness of the proposed adaptive noise perturbation mechanism, particularly its ability to maintain high utility of the FL models while offering strong protection against data reconstruction attacks. All experiments are conducted on 2 servers: one with a 20-core Intel Xeon Platinum 8457C CPU and a NVIDIA T20 GPU (48 GB), and another with 160-core Intel Xeon Gold 6230 CPUs and 4 NVIDIA RTX 4090 GPUs (24 GB each).

## 4.1 Experimental settings

**Datasets.** Our experiments are trained on 4 real-life classical datasets: SVHN [34], CIFAR10 [19], STL10 [4], and Tiny-ImageNet with 4-class classification [5] (abbreviated as ImageNet). These datasets have been widely used in the study of DRAs against FL systems [36, 42, 47]. All dataset samples are resized to 32x32 pixels, a standard preprocessing step in ML to ensure consistency and fairness when comparing across different datasets [30].

**Model architectures of FL.** For HFL, we follow the experimental setup described in [56] and conduct experiments using three classical model architectures for image-domain tasks: LeNet [21], AlexNet [20], and Residual network [13] (ResNet). For VFL, we follow the configurations outlined in [11, 22, 36] and examine three cut-layer configurations of ResNet18 to generate varying depth levels for the local client. These configurations include cutting early (after the first block), at an intermediate depth (after the second block), and late (after the third block), referred to as ResNet-cut1, ResNet-cut2, and ResNet-cut3, respectively. For all experiments, we set the mini-batch size to 32 and use cross-entropy as the loss function. The optimization process is carried out using the Adam optimizer with a learning rate of $1 \times 10^{-4}$. Each target model is trained within 200 epochs to ensure the convergence of the classifier.

**DRA methods and evaluation metrics.** To assess whether InvRE can accurately indicate information leakage risks, we implement three representative attack methods conducted by an honest but curious server: 1). *DLG*, designed for HFL, minimizes the L2 distance between the gradient (HFL) or embedding (VFL) of the true input data and the reconstructed data; 2). *IG*, which improves upon *DLG* by using cosine distance for gradient or embedding matching and incorporating
<figure><img src="image_6.png" /><figcaption>Figure 3: Correlation between InvRE and reconstruction MSE for different samples in CIFAR10 trained on the LeNet and ResNet-cut1 for HFL and VFL respectively.</figcaption></figure>

total variation (TV) as a prior to reduce artifacts in the re-
constructed data; and 3) *CGIR* or *PISTE* for HFL and VFL
respectively, which enhance reconstruction capabilities using
a deep generative model. The implementations of these at-
tacks were based on the official code repositories provided by
the respective authors [9, 47, 49, 56]. For *DLG*, the L-BFGS opt-
imizer was employed with 500 optimization iterations across
all datasets. For *IG*, the Adam optimizer was used with 24,000
optimization iterations, while for *CGIR* and *PISTE*, the Adam
optimizer was applied with 2,000 optimization iterations. For
demonstrating the attack with a malicious server, we involve
LOKI [55] and provide the empirical analysis of LOKI-driven
attacks in Appendix C.

To quantify the relationship between the InvRE score and
the corresponding attack performance, we report both the
Pearson correlation coefficient and the p-value (denoted as
p) to assess the statistical significance of the alignment be-
tween the InvRE score and the attack effectiveness. A p-value
below 0.05 is widely considered indicative of a statistically
significant correlation between the variables [12]. Therefore, a
higher Pearson correlation coefficient would validate InvRE's
ability to reflect DRA risks accurately. We adopt the stan-
dard classification performance metric, i.e., Accuracy (ACC),
to evaluate the utility of FL systems [30]. We evaluate the
attack performance of involved DRA methods using widely
adopted reconstruction quality metrics, including of Mean
Square Error (MSE), Peak Signal-to-Noise Ratio (PSNR),
and Structural Similarity Index Metric (SSIM) for the recon-
structed data [9].

**Defense mechanisms.** We evaluate the performance of InvRE score and 4 state-of-the-art defense methods tailored to HFL and VFL. For HFL, we include *Prune* [51], *Dropout* [14], *DNP* [22], and *GNP* [56]; for VFL, the evaluated methods are *Prune*, *Dropout*, *DNP*, and *ENP* [56]. *Prune* sets the smallest

<figure><img src="image_7.png" /><figcaption>Figure 4: The five images with the smallest (top row) and largest (bottom row) InvRE values over STL10.</figcaption></figure>

$\lambda$% of elements to zero while retaining the rest [22], whereas *Dropout* randomly zeros out $\lambda$% of elements. *DNP* introduces Gaussian noise in the original data space, while *GNP* (for HFL) and *ENP* (for VFL) apply similar noise to gradients or feature embeddings, respectively. Note that, we do not consider encryption-based defense strategies [3, 40], such as Falcon [46] and HashVFL [37], due to their significant communication and computation overhead.

## 4.2 Evaluating InvRE for different instances

In this section, we aim to answer *Q1*. We compute both the Pearson correlation coefficient *cor* and the corresponding *p*-value between the InvRE score and the corresponding reconstruction MSE derived by different attacks over each input instance. A detailed description is provided in Appendix *A*.

Fig. 3 provide the InvRE score and corresponding recon-
struction MSE under three distinct attack methods for CI-
FAR10 trained on the LeNet and ResNet-cut1 of HFL and
VFL. Due to space limitations, comprehensive results on
other three datasets are provided in Appendix B (Fig. 11 and
Fig. 13). To further demonstrate the impact of intrinsic image
characteristics on InvRE scores, we show the five original
images with the lowest and highest scores for HFL and VFL
in Fig. 4 and Fig. 12 (Appendix B), respectively.

Fig. 3 demonstrates a significant negative correlation be-
tween InvRE score and data reconstruction MSE across vari-
ous datasets (CIFAR10, STL10, SVHN, and ImageNet) for
both HFL and VFL frameworks, with correlation coefficients
ranging from -0.41 to -0.67 and p-values below 1e-5. The
consistent negative correlation observed across different DRA
attack methods (DLG, IG, and CGIR / PISTE) highlight In-
vRE's adaptability in reflecting the data privacy risk level
across diverse FL applications. In addition, as shown in Fig. 4,
images with the lowest InvRE scores (top rows) tend to have
simpler and smoother textures, leading to lower gradient and
feature sensitivity to input variations. This makes it harder for
attackers to reconstruct the original data, indicating higher ro-
bustness to DRAs. In contrast, images with the highest InvRE
scores (bottom rows) often have more complex and distinc-
tive textures, making their gradients and embeddings more
sensitive and thus easier to exploit for reconstruction. This
highlights the key role of feature representation in determin-
<figure><img src="image_8.png" /><figcaption>Figure 5: Correlation between InvRE and reconstruction error (MSE) for different model structures in HFL.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Figure 6: Correlation between InvRE and reconstruction error (MSE) for different model structures in VFL.</figcaption></figure>

ing and mitigating the DRA risk over different data instances, as discussed in [35].

**Summary:** The experimental results underscore that InvRE serves as a reliable metric for quantifying privacy leakage risks associated with different samples in both HFL and VFL frameworks. The consistency of the observed correlation patterns across different datasets and attack methods enhances its practicality in real-world FL systems.

## 4.3 Evaluating InvRE across different model structures in FL

This section addresses **Q2**: We evaluate the correlation between InvRE score and the reconstruction MSE using various model architectures deployed in HFL and VFL systems across four datasets: CIFAR10, SVHN, STL10, and ImageNet. The results are presented in Fig. 5 and Fig. 6 for HFL and VFL, respectively.

The experimental results demonstrate a significant correlation between InvRE score and the reconstruction errors across different model structures in both HFL and VFL. Within the HFL framework, the LeNet model, characterized by a higher InvRE score, exhibits greater vulnerability to DRA attack, as indicated by its lower MSE values. For instance, on the STL10 dataset under the DLG attack, LeNet achieves an MSE of 0.008, which is significantly lower than the values for AlexNet (0.026) and ResNet (0.1). Similarly, in the VFL framework, ResNet-cut1, having a higher InvRE score, encounters a greater risk of DRA compared to ResNet-cut2 and ResNet-cut3. This trend can be explained by the higher complexity and non-linearity of deeper models, such as AlexNet and ResNet, which make it more difficult for DRA attacks to

invert transformations and reconstruct data.

**Summary.** The experimental findings confirm that InvRE score serves as a robust indicator of privacy risks across different model architectures in both HFL and VFL systems. With InvRE, FL practitioners can evaluate privacy risks and optimize model configurations before FL training, enabling more secure and privacy-preserving FL deployment.

## 4.4 Evaluating defense methods with InvRE

Based on the answer to **Q1** and **Q2**, we further utilize InvRE score to demonstrate the reduction of DRA risk after deploying the state-of-the-art defense mechanisms, including the noise perturbation and information compress-based methods. Through empirical results, we show that the theoretical analysis over the upper bound of the InvLoss forms the foundation for explaining the effectiveness of these defense methods. It provides the answer to **Q3**. Fig. 7 and Fig. 8 present the InvRE score with DNP and GNP/ENP, as well as Prune and Dropout for LeNet in HFL and ResNet-cut1 in VFL respectively. Additional models are provided in Appendix B. In these figures, SV, ST, CI, and IM denote the SVHN, STL10, CIFAR10, and ImageNet datasets, respectively. The horizontal axis indicates the increase in defense strength as the defense parameter grows. The first column in each sub-figure shows the InvRE score without the defense methods deployed, labeled as '0'.

By analyzing the results presented in Fig. 7 and Fig. 8, it is evident that increasing the defense strength, whether through noise perturbation methods (DNP and GNP/ENP) or information compression methods (Prune and Dropout), significantly reduces privacy risks, as measured by the InvRE score. Specifically, noise perturbation-based defenses (DNP
<figure><img src="image_10.png" /><figcaption>Figure 7: Values of InvRE with LeNet under different defense strategies and strengths in HFL.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Figure 8: Values of InvRE with ResNet-cut1 under different defense strategies and strengths in VFL.</figcaption></figure>

and GNP/ENP) achieve this reduction by introducing noise to the raw features, thereby disrupting the attacker's ability to reconstruct data features. For example, as shown in Fig. 7, deploying the GNP method in the HFL setting with LeNet results in a significant reduction in the InvRE score as the noise variance increases from 0.004 to 0.02. On the other hand, information compression-based defenses (Prune and Dropout) mitigate privacy leakage by discarding a portion of the shared information. As shown in Fig. 8, the Prune defense method applied to ResNet-cut1 in the VFL setting achieves a steady reduction in the InvRE score as the discarding ratio increases from 0.80 to 0.99. Therefore, it is essential to design more advanced defense strategies that can better balance privacy protection and model utility.

**Summary.** From the empirical results, we can observe consistently the descending tendency of InvRE score with stronger defense strength using the state-of-the-art defense mechanisms and over different datasets. With the empirical observation, we can find that the proposed theoretical upper bounds of the InvLoss in Eq. (8), Eq. (9) and Eq. (10) provide a consistent explanation to the effectiveness of the state-of-

the-art defensive mechanisms. The results show that the theoretically guaranteed InvRE score can be used to systematically evaluate the effectiveness of defense mechanisms.

## 4.5 Practical implications of InvRE

In this section, we investigate the correlation between InvRE values and perceptual quality of reconstructed images across a wide range of InvRE scores. Specifically, we divide the samples into groups based on their InvRE values and visualize the corresponding reconstructions using representative DRAs, namely CGIR for HFL and PISTE for VFL. In parallel, we compute PSNR and SSIM to quantify perceptual similarity between original and reconstructed images, indicating the extent of image content leakage. Higher PSNR and SSIM denote better recovery of image details. This combination of visual and quantitative analysis enables practitioners to directly interpret the privacy risks associated with specific InvRE values. Fig. 9 and Fig. 10 show both the reconstructed images and their corresponding SSIM scores for HFL and VFL respectively. The PSNR results with HFL and VFL settings are presented
<figure><img src="image_12.png" /><figcaption>Figure 9: InvRE vs. reconstruction quality in HFL. Top: reconstructed images across increasing InvRE levels; bottom: corresponding SSIM scores.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Figure 10: InvRE vs. reconstruction quality in VFL. Top: reconstructed images across increasing InvRE levels; bottom: corresponding SSIM scores.</figcaption></figure>

in Appendix B (Fig. 16 and Fig. 17).

As shown in Fig. 9 and Fig. 10, there is a consistently strong positive correlation between InvRE and reconstruction quality indicated by the SSIM scores in both HFL and VFL settings across different model architectures. InvRE shows a strong positive correlation with SSIM, with Pearson coefficients of 0.967, 0.960, and 0.994 (with $p < 10^{-4}$) for LeNet, AlexNet, and ResNet, and 0.967, 0.945, and 0.966 (with $p < 10^{-3}$) for ResNet-cut1, ResNet-cut2, and ResNet-cut3, respectively. In both plots, when the InvRE score is low (e.g., below 0.15), the reconstructed image contents are heavily degraded and lack identifiable structures. Meanwhile, the corresponding SSIM scores are also close or lower than 0.4 in both figures. According to previous works [10, 55], SSIM lower than 0.5 usually indicates the failure of data reconstruction attacks, which is unable to recover meaningful image contents. Therefore, we can find that InvRE score below 0.15 indicates a minimal privacy leak risk. With intermediate InvRE values (e.g., between 0.15 and 0.45), the reconstructed images begin to reveal partial semantic features, reflecting an elevated and moderate level of privacy leakage. When the InvRE score is high (e.g., above 0.45), the reconstructed images closely resemble the original ones, suggesting a high risk of exposing details in the images. These thresholds are supported by consistent trends in both figures across different model architectures and FL settings. Hence, InvRE serves as a practical privacy risk auditing tool: it enables preemptive risk assessment for all samples in a training set without executing attacks.

## 4.6 Defense with adaptive noise perturbation

In this section, we measure the defense performance of the proposed adaptive noise perturbation mechanism and compare it with the existing defense methods in FL systems. The empirical observations will address **Q4**. Experiments are conducted using AlexNet and ResNet-cut2 models with CGIR and PISTE attack for HFL and VFL frameworks. Experiment results are detailed in Table 2 and Table 3.

In these tables, the “Clean” row represents the baseline FL system deployed without any applied defenses. The subsequent rows outline the results with various defense methods. For noise-based methods, defense strength is adjusted via noise variance ($\delta$); for information compression (IC)-based methods, it is controlled by the pruning/dropout ratio ($\lambda$), where larger values imply stronger protection. In all experiments, our adaptive noise is confined to the subspace spanned by more than 95% of the total singular values. For InvL-GNP and InvL-ENP, the projection dimensions corresponding to the top 60% of singular values are also set to zero. To better illustrate the impact of each defense method on the model’s utility, we report the relative drop in model accuracy (ACC) compared to the Clean baseline.

The empirical findings of this study can be summarized as follows: **First**, compared to the noise perturbation-based defense, the IC-based defense (i.e., Prune and Dropout) offers limited protection against DRA while significantly degrading utility. For instance, in HFL with STL10, Dropout reduces accuracy by over 50% (from 0.865 to 0.424), yet achieves only marginal SSIM drop (0.732 to 0.676). **Second**, InvL-DNP consistently causes less utility degradation than DNP with
<table><caption>Table 2: Performance of model accuracy (ACC) and DRA error with different defenses in HFL.</caption><thead><tr><th rowspan="2">Defense</th><th colspan="5">CIFAR10</th><th colspan="5">STL10</th></tr><tr><th>δ / λ</th><th>MSE↑</th><th>PSNR↓</th><th>SSIM↓</th><th>ACC↑</th><th>δ / λ</th><th>MSE↑</th><th>PSNR↓</th><th>SSIM↓</th><th>ACC↑</th></tr></thead><tbody><tr><td>Clean</td><td>-</td><td>0.004</td><td>24.85</td><td>0.771</td><td>0.725(-)</td><td>-</td><td>0.003</td><td>23.43</td><td>0.732</td><td>0.865(-)</td></tr><tr><td>Prune</td><td>0.995</td><td>0.024</td><td>16.22</td><td>0.410</td><td>0.390(46.20%)</td><td>0.995</td><td>0.028</td><td>16.05</td><td>0.361</td><td>0.744(13.90%)</td></tr><tr><td>Dropout</td><td>0.995</td><td>0.017</td><td>18.04</td><td>0.469</td><td>0.331(54.34%)</td><td>0.970</td><td>0.005</td><td>22.70</td><td>0.676</td><td>0.424(50.98%)</td></tr><tr><td>DNP</td><td>0.070</td><td>0.033</td><td>15.19</td><td>0.377</td><td>0.634(12.55%)</td><td>0.050</td><td>0.023</td><td>17.32</td><td>0.458</td><td>0.813(6.01%)</td></tr><tr><td>InvL-DNP</td><td>0.080</td><td><u>0.036</u></td><td><u>14.63</u></td><td><u>0.296</u></td><td><u>0.642(11.44%)</u></td><td>0.060</td><td><u>0.026</u></td><td><u>16.74</u></td><td><u>0.427</u></td><td><u>0.825(4.62%)</u></td></tr><tr><td>GNP</td><td>0.150</td><td>0.062</td><td>12.45</td><td>0.098</td><td>0.707(2.48%)</td><td>0.030</td><td>0.037</td><td>14.58</td><td>0.290</td><td>0.831(3.93%)</td></tr><tr><td>InvL-GNP</td><td>0.250</td><td><b>0.065</b></td><td><b>12.14</b></td><td><b>0.084</b></td><td><b>0.712(1.79%)</b></td><td>0.070</td><td><b>0.040</b></td><td><b>14.10</b></td><td><b>0.220</b></td><td><b>0.848(1.96%)</b></td></tr></tbody></table>

<table><caption>Table 3: Performance of model accuracy (ACC) and DRA error with different defenses in VFL.</caption><thead><tr><th rowspan="2">Defense</th><th colspan="5">CIFAR10</th><th colspan="5">STL10</th></tr><tr><th>δ / λ</th><th>MSE↑</th><th>PSNR↓</th><th>SSIM↓</th><th>ACC↑</th><th>δ / λ</th><th>MSE↑</th><th>PSNR↓</th><th>SSIM↓</th><th>ACC↑</th></tr></thead><tbody><tr><td>Clean</td><td>-</td><td>0.006</td><td>23.34</td><td>0.830</td><td>0.725(-)</td><td>-</td><td>0.009</td><td>21.45</td><td>0.732</td><td>0.737(-)</td></tr><tr><td>Prune</td><td>0.900</td><td>0.021</td><td>17.76</td><td>0.664</td><td>0.518(28.55%)</td><td>0.980</td><td>0.024</td><td>16.95</td><td>0.413</td><td>0.525(28.76%)</td></tr><tr><td>Dropout</td><td>0.900</td><td>0.021</td><td>17.38</td><td>0.617</td><td>0.305(57.93%)</td><td>0.980</td><td>0.023</td><td>17.08</td><td>0.398</td><td>0.305(58.61%)</td></tr><tr><td>DNP</td><td>0.055</td><td>0.013</td><td>18.91</td><td>0.622</td><td>0.577(20.41%)</td><td>0.070</td><td>0.022</td><td>16.85</td><td>0.402</td><td>0.618(16.14%)</td></tr><tr><td>InvL-DNP</td><td>0.060</td><td><u>0.014</u></td><td><u>18.61</u></td><td><u>0.605</u></td><td><u>0.583(19.58%)</u></td><td>0.080</td><td><u>0.024</u></td><td><u>16.53</u></td><td><u>0.371</u></td><td><u>0.626(15.06%)</u></td></tr><tr><td>GNP</td><td>0.700</td><td>0.021</td><td>17.16</td><td>0.570</td><td>0.650(10.34%)</td><td>0.500</td><td>0.039</td><td>14.26</td><td>0.350</td><td>0.691(6.24%)</td></tr><tr><td>InvL-GNP</td><td>1.000</td><td><b>0.023</b></td><td><b>16.43</b></td><td><b>0.502</b></td><td><b>0.684(5.65%)</b></td><td>0.600</td><td><b>0.040</b></td><td><b>14.13</b></td><td><b>0.341</b></td><td><b>0.715(2.98%)</b></td></tr></tbody></table>

comparable privacy protection strength. In HFL on STL10, InvL-DNP reduces ACC by 4.62%, compared to 6.01% for DNP. The main reason is InvL-DNP avoids injecting noise into subspaces with small singular values, which harm the utility yet barely enhancing the protection strength (see Theorem 2). **Third**, InvL-GNP / InvL-ENP frameworks consistently outperform their counterparts (GNP/ENP) and achieve the best trade-off between utility and privacy protection. According to Table 3, the ACC drop caused by InvL-GNP is half of that with GNP on HFL and VFL, while presenting stronger privacy protection (higher MSE, lower PSNR and SSIM). This utility-protection balance stems from the strategic noise calibration guided by Theorem 3.

We also compare InvL-DNP and InvL-GNP / InvL-ENP with DNP and GNP / ENP with different noise variance levels, as shown in Table 6 and Table 7 (Appendix B). The results show that the InvL-based noise calibration methods consistently outperform the baselines across different noise variance levels, achieving equal or stronger protection with significantly lower utility loss. In summary, the empirical findings echo the theoretical analysis behind the design of the proposed defense methods. By injecting noise only to the subspaces of the Jacobian matrix that are the most influential to the reconstruction error, InvL-DNP and InvL-GNP / InvL-ENP achieve a more favorable privacy-utility balance.

**Efficiency analysis of our InvL-based defenses.** The computational overhead of our InvL-based defense framework

primarily arises from two operations: the Jacobian matrix computation during backpropagation and the subsequent singular value decomposition (SVD) for spectral noise calibration. To improve efficiency, we leverage PyTorch's `vmap` and `jacrev` for batched Jacobian computation, enabling implicit gradient aggregation and significantly reducing redundant computation and memory usage. For the SVD of Jacobian matrices $G_x \in \mathbb{R}^{p \times m}$, we adopt GPU-accelerated `torch.linalg.svd()` with truncated decomposition (`full=False`), computing only the top $d = \min(p, m)$ singular vectors. This reduces complexity to $O(pmk)$ when $k \ll \min(p, m)$, while preserving key spectral components for effective noise calibration.

<table><caption>Table 4: Training time (s) for 100 epochs in HFL.</caption><thead><tr><th>Model</th><th>Size</th><th>Clean</th><th>InvL-DNP</th><th>InvL-GNP</th></tr></thead><tbody><tr><td>LeNet</td><td>15,826</td><td>551.408</td><td>1793.082</td><td>735.772</td></tr><tr><td>AlexNet</td><td>44,074</td><td>583.687</td><td>1974.017</td><td>1053.765</td></tr><tr><td>ResNet</td><td>53,114</td><td>1019.136</td><td>3470.756</td><td>2998.149</td></tr></tbody></table>

We present run-time measurement results for HFL and VFL in Table 4 and Table 5 respectively. In HFL, we report the total run-time cost over 100 training epochs. InvL-based defense methods inject noise to the gradient of each training round, incurring overheads that increase linearly with the size of the model architecture. For example, ResNet has approximately
<table><caption>Table 5: Inference time (s) for 10-sample batch in VFL.</caption><thead><tr><td>Model</td><td>Size</td><td>Clean</td><td>InvL-DNP</td><td>InvL-ENP</td></tr></thead><tbody><tr><td>ResNet-cut1</td><td>19,392</td><td>1.766e-3</td><td>8.288</td><td>8.287</td></tr><tr><td>ResNet-cut2</td><td>208,448</td><td>1.734e-3</td><td>9.087</td><td>9.094</td></tr><tr><td>ResNet-cut3</td><td>503,616</td><td>1.744e-3</td><td>9.240</td><td>9.245</td></tr></tbody></table>

four times as many parameters as LeNet. Accordingly, the training time with the InvL-based defense methods is about three times longer than that of LeNet. In VFL, applying the defense methods during the testing time barely increases the run-time of forward inference, with the inference time consistently around 9s per batch across all model sizes.

# 5 Conclusion

In this study, we propose a feasibility measure for the risk of DRAs in FL systems, called Invertibility Loss (InvLoss), and establish its upper bound to provide a unified framework for identifying factors influencing DRA effectiveness across FL mechanisms. We extend this analysis to explain the effectiveness of existing defense mechanisms against DRAs. Using the theoretical upper bound, we develop InvRE, an exact InvLoss estimator for practical DRA risk assessment. Our results show that InvRE consistently reflects risk levels across various FL mechanisms, model architectures, and data sources, without reliance on specific DRA methods. Guided by these insights, we propose two adaptive noise perturbation strategies that strategically adjust noise based on the Jacobian matrix's spectral properties. They demonstrate strong privacy protection with minimal impact on FL utility.

# Acknowledgments

This work was supported in part by Beijing Natural Science Foundation (No. L221014), in part by National Natural Science Foundation of China (U21A20463, U22B2027, U23A20304), in part by the Systematic Major Project of China State Railway Group Co. Ltd. (No. P2023W002, No. P2024S003, and No. P2024W001-4), in part by the Science and Technology Research and Development Plan of China Railway Information Technology Group Co., Ltd. (No. WJZG-CKY-2023014 (2023A08), and No. WJZG-CKY-2024040 (2024P01)). This work was also supported by Hangzhou Qianjiang Distinguished Experts programme (2024).

# Ethics considerations

Our work on quantifying and mitigating Data Reconstruction Attack (DRA) risks in Federated Learning (FL) systems raises

several important ethical considerations that we have carefully evaluated:

**Stakeholder Analysis.** We carefully considered impacts on three key stakeholder groups to ensure comprehensive ethical evaluation.

1). FL Clients: Our primary stakeholders are individuals and organizations contributing private data to FL systems. While our research could potentially be misused to develop more sophisticated attacks, its main purpose is to help these stakeholders better understand and mitigate privacy leak risks. The actual risk to clients is negligible in our study as we only use public datasets and simulated attacks.

2). FL System Operators: Our work provides operators with tools to assess and improve their systems' privacy guarantees. The potential negative outcome would be if operators misuse our risk quantification to justify inadequate protection measures. We mitigate this by clearly presenting the limitations of our framework.

3). Research Community: There is a risk that our attack analysis could be repurposed for malicious use. However, we believe this is outweighed by the benefits of enabling more robust defenses. We follow responsible disclosure principles by focusing on defense development.

**Research Process Ethics.** We implemented multiple safeguards to ensure ethical research practices throughout our study.

1). Dataset Selection: Although we use only public datasets (CIFAR10, STL10, etc.), we recognize that even these may contain problematic representations. We carefully reviewed dataset provenance and content to ensure no obvious ethical violations.

2). Attack Methodology: Our DRA simulations are conducted in controlled environments with no actual privacy violations. All experiments use synthetically constructed or publicly available data.

3). Defense Implications: Our proposed noise perturbation defenses are designed to preserve model utility while enhancing privacy. We acknowledge they may increase computational overhead, potentially excluding resource-constrained participants - an important trade-off we explicitly discuss.

**Risk-Benefit Analysis.** We considered alternative approaches that might reduce risks:

1). Not publishing attack methodology details (but this would limit defense development).

2). Only studying defenses without analyzing attacks (but this would weaken the scientific contribution).

After careful consideration, we concluded that the benefits of transparent, rigorous analysis outweigh the risks, given appropriate safeguards:

1). We focus on fundamental properties of information leakage rather than optimizing attack performance.

2). The proposed defense method in our study are practical and immediately applicable.
3). We provide clear guidelines for responsible use of our work in mitigating privacy leak attacks.

**Team Wellbeing.** The potentially sensitive nature of privacy attack research required special consideration for our team:

1). We established clear boundaries between attack simulation and real-world application.

2). Implemented regular ethics discussions throughout the research process.

3). Provided access to ethics review and counseling resources.

**Continuous Ethical Reflection.** We recognize that ethical considerations in privacy research evolve rapidly. While we believe our current approach represents the best balance of scientific rigor and ethical responsibility, we commit to:

1). Monitoring for new ethical concerns that may emerge.

2). Updating our framework as new risks are identified.

3). Engaging with the broader community on responsible disclosure practices.

This ethical framework guides our research while acknowledging that perfect solutions may not exist. We welcome ongoing discussion about how to best advance FL privacy protections while minimizing potential harms.

## Open Science

This study adheres to the principles of open science, emphasizing transparency and accessibility in research. The source code accompanying this work is publicly available on Zenodo at this link. The repository provides artifact instructions, dependencies, core code (e.g., data, models, attacks, defenses), and run scripts, in compliance with USENIX's open science policy.

## References

[1] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In *Proceedings of the 2016 ACM SIGSAC conference on computer and communications security*, pages 308–318, 2016.

[2] Franziska Boenisch, Adam Dziedzic, Roei Schuster, Ali Shahin Shamsabadi, Ilia Shumailov, and Nicolas Papernot. When the curious abandon honesty: Federated learning is not private. In *8th IEEE European Symposium on Security and Privacy*, pages 175–199. IEEE, 2023.

[3] Chaochao Chen, Jun Zhou, Li Wang, Xibin Wu, Wenjing Fang, Jin Tan, Lei Wang, Alex X Liu, Hao Wang, and Cheng Hong. When homomorphic encryption marries secret sharing: Secure large-scale sparse logistic regression and applications in risk control. In *Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining*, pages 2652–2662, 2021.

[4] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In *Proceedings of the fourteenth international conference on artificial intelligence and statistics*, pages 215–223. JMLR Workshop and Conference Proceedings, 2011.

[5] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In *2009 IEEE conference on computer vision and pattern recognition*, pages 248–255. IEEE, 2009.

[6] David Enthoven and Zaid Al-Ars. An overview of federated deep learning privacy attacks and defensive strategies. *Federated Learning Systems: Towards Next-Generation AI*, pages 173–196, 2021.

[7] Farhad Farokhi and Henrik Sandberg. Fisher information as a measure of privacy: Preserving privacy of households with smart meters using batteries. *IEEE Transactions on Smart Grid*, 9(5):4726–4734, 2017.

[8] Chong Fu, Xuhong Zhang, Shouling Ji, Jinyin Chen, Jingzheng Wu, Shanqing Guo, Jun Zhou, Alex X Liu, and Ting Wang. Label inference attacks against vertical federated learning. In *31st USENIX security symposium (USENIX Security 22)*, pages 1397–1414, 2022.

[9] Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. Inverting gradients-how easy is it to break privacy in federated learning? *Advances in Neural Information Processing Systems*, 33:16937–16947, 2020.

[10] Niv Haim, Gal Vardi, Gilad Yehudai, Ohad Shamir, and Michal Irani. Reconstructing training data from trained neural networks. In *Proceedings of the 36th International Conference on Neural Information Processing Systems*, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc.

[11] Awni Hannun, Chuan Guo, and Laurens van der Maaten. Measuring data leakage in machine-learning models with fisher information. In *Uncertainty in Artificial Intelligence*, pages 760–770. PMLR, 2021.

[12] Jan Hauke and Tomasz Kossowski. Comparison of values of pearson's and spearman's correlation coefficients on the same sets of data. *Quaestiones geographicae*, 30(2):87–93, 2011.

[13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In *Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 770–778, 2016.
[14] Zecheng He, Tianwei Zhang, and Ruby B Lee. Attack-
ing and protecting data privacy in edge–cloud collabora-
tive inference systems. *IEEE Internet of Things Journal*,
8(12):9706–9716, 2020.

[15] Lingxiao Huang, Zhize Li, Jialin Sun, and Haoyu Zhao. Coresets for vertical federated learning: Regularized linear regression and K-means clustering. In *Advances in Neural Information Processing Systems*, pages 29566–29581, 2022.

[16] Yangsibo Huang, Samyak Gupta, Zhao Song, Kai Li, and Sanjeev Arora. Evaluating gradient inversion attacks and defenses in federated learning. *Advances in neural information processing systems*, 34:7232–7241, 2021.

[17] Xiao Jin, Pin-Yu Chen, Chia-Yi Hsu, Chia-Mu Yu, and Tianyi Chen. Cafe: Catastrophic data leakage in vertical federated learning. *Advances in Neural Information Processing Systems*, 34:994–1006, 2021.

[18] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. *Foundations and trends® in machine learning*, 14(1-2):1-210, 2021.

[19] Alex Krizhevsky, Geoffrey Hinton, et al. Learning mul-
tiple layers of features from tiny images. 2009.

[20] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. *Advances in neural information processing systems*, 25, 2012.

[21] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. *Proceedings of the IEEE*, 86(11):2278–2324, 1998.

[22] Jingtao Li, Adnan Siraj Rakin, Xing Chen, Zhezhi He, Deliang Fan, and Chaitali Chakrabarti. Ressfl: A resistance transfer framework for defending model inversion attack in split federated learning. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 10194–10202, 2022.

[23] Zhize Li, Dmitry Kovalev, Xun Qian, and Peter Richtárik. Acceleration for compressed gradient descent in distributed and federated optimization. In *International Conference on Machine Learning*, pages 5895–5904. PMLR, 2020.

[24] Zhize Li and Peter Richtárik. A unified analysis of stochastic gradient methods for nonconvex federated optimization. *arXiv preprint arXiv:2006.07013*, 2020.

[25] Zhize Li, Haoyu Zhao, Boyue Li, and Yuejie Chi. Soteri-aFL: A unified framework for private federated learning with communication compression. *Advances in Neural Information Processing Systems*, 35:4285–4300, 2022.

[26] Zhuohang Li, Jiaxin Zhang, Luyang Liu, and Jian Liu. Auditing privacy defenses in federated learning via generative gradient leakage. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 10132–10142, 2022.

[27] Pengrui Liu, Xiangrui Xu, and Wei Wang. Threats, attacks and defenses to federated learning: issues, taxonomy and perspectives. *Cybersecurity*, 5(1):4, 2022.

[28] Yang Liu, Yan Kang, Tianyuan Zou, Yanhong Pu, Yuan-qin He, Xiaozhou Ye, Ye Ouyang, Ya-Qin Zhang, and Qiang Yang. Vertical federated learning: Concepts, advances, and challenges. *IEEE Transactions on Knowledge and Data Engineering*, 2024.

[29] Yiyong Liu, Zhengyu Zhao, Michael Backes, and Yang Zhang. Membership inference attacks by exploiting loss trajectory. In *Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security*, pages 2085–2098, 2022.

[30] Yugeng Liu, Rui Wen, Xinlei He, Ahmed Salem, Zhikun Zhang, Michael Backes, Emiliano De Cristofaro, Mario Fritz, and Yang Zhang. Ml-doctor: Holistic risk assessment of inference attacks against machine learning models. In *31st USENIX Security Symposium (USENIX Security 22)*, pages 4525–4542, 2022.

[31] Xinjian Luo, Yuncheng Wu, Xiaokui Xiao, and Beng Chin Ooi. Feature inference attack on model predictions in vertical federated learning. In *2021 IEEE 37th International Conference on Data Engineering (ICDE)*, pages 181–192. IEEE, 2021.

[32] H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication-efficient learning of deep networks from decentralized data. In *International Conference on Artificial Intelligence and Statistics*, pages 1273–1282. PMLR, 2017.

[33] Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In *2019 IEEE symposium on security and privacy (S&P)*, pages 739–753. IEEE, 2019.

[34] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bis-sacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning.
In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, page 4. Granada, 2011.

[35] Sayedeh Leila Noorbakhsh, Binghui Zhang, Yuan Hong, and Binghui Wang. Inf2guard: An information-theoretic framework for learning privacy-preserving representations against inference attacks. In USENIX Security, 2024.

[36] Dario Pasquini, Giuseppe Ateniese, and Massimo Bernaschi. Unleashing the tiger: Inference attacks on split learning. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 2113-2129, 2021.

[37] Pengyu Qiu, Xuhong Zhang, Shouling Ji, Chong Fu, Xing Yang, and Ting Wang. Hashvfl: Defending against data reconstruction attacks in vertical federated learning. IEEE Transactions on Information Forensics and Security, 2024.

[38] Peter Richtárik, Igor Sokolov, Elnur Gasanov, Ilyas Fatkhullin, Zhize Li, and Eduard Gorbunov. 3PC: Three point compressors for communication-efficient distributed training and a better theory for lazy aggregation. In International Conference on Machine Learning, pages 18596-18648. PMLR, 2022.

[39] Umme Sara, Morium Akter, and Mohammad Shorif Uddin. Image quality assessment through fsim, ssim, mse and psnr—a comparative study. Journal of Computer and Communications, 2019.

[40] Haoran Shi, Yali Jiang, Han Yu, Yonghui Xu, and Lizhen Cui. Mvfls: multi-participant vertical federated learning based on secret sharing. The Federate Learning, pages 1-9, 2022.

[41] Shanghao Shi, Ning Wang, Yang Xiao, Chaoyu Zhang, Yi Shi, Y. Thomas Hou, and Wenjing Lou. Scale-mia: A scalable model inversion attack against secure federated learning via latent space reconstruction, 2024.

[42] Qi Tan, Qi Li, Yi Zhao, Zhuotao Liu, Xiaobing Guo, and Ke Xu. Defending against data reconstruction attacks in federated learning: An information theory approach. 2024.

[43] Chandra Thapa, Pathum Chamikara Mahawaga Arachchige, Seyit Camtepe, and Lichao Sun. Splitfed: When federated learning meets split learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8485-8493, 2022.

[44] Praneeth Vepakomma, Abhishek Singh, Otkrist Gupta, and Ramesh Raskar. Nopeek: Information leakage reduction to share activations in distributed deep learning. In 2020 International Conference on Data Mining Workshops (ICDMW), pages 933-942. IEEE, 2020.

[45] Zhibo Wang, Mengkai Song, Zhifei Zhang, Yang Song, Qian Wang, and Hairong Qi. Beyond inferring class representatives: User-level privacy leakage from federated learning. In IEEE INFOCOM 2019-IEEE conference on computer communications, pages 2512-2520. IEEE, 2019.

[46] Yuncheng Wu, Naili Xing, Gang Chen, Tien Tuan Anh Dinh, Zhaojing Luo, Beng Chin Ooi, Xiaokui Xiao, and Meihui Zhang. Falcon: A privacy-preserving and interpretable vertical federated learning system. Proceedings of the VLDB Endowment, 16(10):2471-2484, 2023.

[47] Xiangrui Xu, Pengrui Liu, Wei Wang, Hong-Liang Ma, Bin Wang, Zhen Han, and Yufei Han. Cgir: Conditional generative instance reconstruction attacks against federated learning. IEEE Transactions on Dependable and Secure Computing, 20(6):4551-4563, 2022.

[48] Xiangrui Xu, Pengrui Liu, Yiwen Zhao, Lei Han, Wei Wang, Yongsheng Zhu, Chongzhen Zhang, Bin Wang, Jian Shen, and Zhen Han. Enhancing privacy in distributed intelligent vehicles with information bottleneck theory. IEEE Internet of Things Journal, 2024.

[49] Xiangrui Xu, Wei Wang, Zheng Chen, Bin Wang, Chao Li, Li Duan, Zhen Han, and Yufei Han. Finding the piste: Towards understanding privacy leaks in vertical federated learning systems. IEEE Transactions on Dependable and Secure Computing, 2024.

[50] Xiangrui Xu, Yiwen Zhao, Yufei Han, Yongsheng Zhu, Zhen Han, Guangquan Xu, Bin Wang, Shouling Ji, and Wei Wang. Vflmonitor: Defending one-party hijacking attacks in vertical federated learning. IEEE Transactions on Information Forensics and Security, 20:4828-4843, 2025.

[51] Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, and Zhenkai Liang. Neural network inversion in adversarial setting via background knowledge alignment. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 225-240, 2019.

[52] Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M Alvarez, Jan Kautz, and Pavlo Molchanov. See through gradients: Image batch recovery via gradinversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16337-16346, 2021.

[53] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610, 2020.

[54] Haoyu Zhao, Konstantin Burlachenko, Zhize Li, and Peter Richtárik. Faster rates for compressed federated
learning with client-variance reduction. *SIAM Journal on Mathematics of Data Science*, 6(1):154–175, 2024.

[55] Joshua C. Zhao, Atul Sharma, Ahmed Roushdy Elkordy, Yahya H. Ezzeldin, Salman Avestimehr, and Saurabh Bagchi. *Loki: Large-scale data reconstruction attack against federated learning through model manipulation*. In *IEEE Symposium on Security and Privacy 2024*, pages 1287–1305. IEEE, 2024.

[56] Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. *Advances in neural information processing systems*, 32, 2019.

## A Correlation between InvRE and reconstruction MSE.

InvRE quantifies the average privacy risk for individual samples against attackers with varying capabilities. To maintain consistency, the reconstruction error (MSE) is similarly averaged over different attacker capabilities. Specifically, for a fixed attack algorithm, we vary the number of iteration rounds, denoted as $I_{\text{ers}} = [I_{\text{er1}}, I_{\text{er2}}, \dots, I_{\text{ers}}]$, to simulate attackers with differing strengths. The weighted average reconstruction error is computed by considering the probabilities associated with each attack capability. Attackers with fewer iteration rounds, representing weaker capabilities, are assigned higher probabilities, whereas attackers with more iteration rounds, representing stronger capabilities, are assigned lower probabilities. The calculation methodology is formalized as follows:

$$ 
\begin{gathered}
E[\text{MSE}_x] = \sum_{t=1}^{s} P_t \text{MSE}_t, \\
\text{where } P_t = \frac{1/T_t}{\sum_{t=1}^{s} (1/T_t)}, \quad \text{and } T_t = \sum_{i=1}^{t} t_i.
\end{gathered}
\qquad (12)
$$

Here, $P_t$ denotes the probability of an attacker having an iteration capability of $I_{\text{ers}}$, while $\text{MSE}_t$ represents the reconstruction error achieved by an attacker employing $I_{\text{ers}}$ iterations. When analyzing the privacy risks of different samples under a fixed model, the InvRE can be simplified to calculating the inverse of the averaged InvLoss, to visually the differences in InvRE across samples more clearly.

## B More results in experiment

**Results for validating InvRE on input instances (Q1).** The correlation between InvRE and reconstruction MSE using LeNet (in HFL) and ResNet-cut1 (in VFL) on the STL10, SVHN, and ImageNet datasets is presented in Fig. 11 and Fig. 13. Additionally, Fig. 12 displays the five samples with the lowest and highest InvRE values in the VFL framework, aligning with the visualization in Fig. 4.

<figure><img src="image_14.png" /><figcaption>Figure 11: Correlation between InvRE and MSE for different samples trained on the LeNet model in HFL.</figcaption></figure>

**Results for validating existing defenses using InvRE (Q3).** As shown in Theorem 2 and Theorem 3, DNP and GNP/ENP extend Theorem 1 by incorporating noise terms introduced by the respective mechanisms. Increasing input noise raises the upper bound of InvLoss, reflecting enhanced defense strength. Therefore, we additionally evaluate the InvRE results for pruning and dropout defenses. Fig. 14 and Fig. 15 show InvRE trends with varying defense strengths using ResNet (HFL) and ResNet-cut3 (VFL). The experimental results demonstrate that as the defense strength increases, the values of InvRE consistently decrease across different datasets, aligning with the theoretical upper bound of InvLoss presented in Sec. 3.2, thereby confirming its ability to explain the effectiveness of various defense mechanisms against DRAs in FL systems.

**Results of practical implications of InvRE (Q4).** To complement the SSIM-based analysis presented in the main text, we provide additional results with the metric of PSNR in Fig. 16 and Fig. 17 for HFL and VFL, respectively. In the

<figure><img src="image_15.png" /><figcaption>Figure 12: The five images with the smallest (top row) and largest (bottom row) InvRE over STL10 on VFL system.</figcaption></figure>
<table><caption>Table 6: Comparison of DNP vs. InvL-DNP and GNP vs. InvL-GNP under various parameter settings in HFL</caption><thead><tr><th></th><th>MSE↑</th><th>PSNR↓</th><th>SSIM↓</th><th>ACC↑</th><th></th><th>MSE↑</th><th>PSNR↓</th><th>SSIM↓</th><th>ACC↑</th></tr></thead><tbody><tr><td>DNP-0.04</td><td>0.018</td><td>18.61</td><td>0.52</td><td>0.825</td><td>GNP-0.03</td><td>0.037</td><td>14.58</td><td>0.29</td><td>0.831</td></tr><tr><td>InvL-DNP-0.05</td><td><u>0.021</u></td><td><u>17.74</u></td><td><u>0.49</u></td><td><b>0.836</b></td><td>InvL-GNP-0.07</td><td><u>0.040</u></td><td><u>14.10</u></td><td><u>0.22</u></td><td><b>0.848</b></td></tr><tr><td>DNP-0.05</td><td>0.023</td><td>17.32</td><td>0.45</td><td>0.813</td><td>GNP-0.04</td><td>0.044</td><td>13.74</td><td>0.23</td><td>0.825</td></tr><tr><td>InvL-DNP-0.06</td><td><u>0.026</u></td><td><u>16.74</u></td><td><u>0.42</u></td><td><b>0.825</b></td><td>InvL-GNP-0.09</td><td><u>0.046</u></td><td><u>13.46</u></td><td><u>0.17</u></td><td><b>0.846</b></td></tr><tr><td>DNP-0.06</td><td>0.026</td><td>16.31</td><td>0.33</td><td>0.798</td><td>GNP-0.05</td><td>0.049</td><td>13.24</td><td>0.19</td><td>0.821</td></tr><tr><td>InvL-DNP-0.07</td><td><u>0.029</u></td><td><u>16.08</u></td><td><u>0.32</u></td><td><b>0.802</b></td><td>InvL-GNP-0.12</td><td><u>0.053</u></td><td><u>12.92</u></td><td><u>0.13</u></td><td><b>0.841</b></td></tr></tbody></table>

<table><caption>Table 7: Comparison of DNP vs. InvL-DNP and ENP vs. InvL-ENP under various parameter settings in VFL.</caption><thead><tr><th></th><th>MSE↑</th><th>PSNR↓</th><th>SSIM↓</th><th>ACC↑</th><th></th><th>MSE↑</th><th>PSNR↓</th><th>SSIM↓</th><th>ACC↑</th></tr></thead><tbody><tr><td>DNP-0.015</td><td>0.008</td><td>21.82</td><td>0.68</td><td>0.701</td><td>GNP-0.40</td><td>0.034</td><td>14.87</td><td>0.39</td><td>0.699</td></tr><tr><td>InvL-DNP-0.020</td><td><u>0.009</u></td><td><u>21.60</u></td><td><u>0.66</u></td><td><b>0.703</b></td><td>InvL-GNP-0.55</td><td><u>0.035</u></td><td><u>14.77</u></td><td><u>0.33</u></td><td><b>0.705</b></td></tr><tr><td>DNP-0.065</td><td>0.020</td><td><u>17.12</u></td><td><u>0.41</u></td><td>0.629</td><td>GNP-0.50</td><td>0.039</td><td>14.26</td><td>0.35</td><td>0.690</td></tr><tr><td>InvL-DNP-0.075</td><td><u>0.026</u></td><td><u>17.16</u></td><td><u>0.41</u></td><td><b>0.637</b></td><td>InvL-GNP-0.60</td><td><u>0.040</u></td><td><u>14.13</u></td><td><u>0.34</u></td><td><b>0.704</b></td></tr><tr><td>DNP-0.07</td><td>0.022</td><td>16.85</td><td>0.40</td><td>0.618</td><td>GNP-0.60</td><td>0.043</td><td>13.78</td><td>0.32</td><td>0.675</td></tr><tr><td>InvL-DNP-0.08</td><td><u>0.024</u></td><td><u>16.35</u></td><td><u>0.37</u></td><td><b>0.626</b></td><td>InvL-GNP-0.80</td><td><u>0.045</u></td><td><u>13.67</u></td><td><u>0.25</u></td><td><b>0.691</b></td></tr></tbody></table>

HFL setting (Fig. 16), the Pearson correlation coefficients between the reported InvRE and the PSNR scores are consistently strong, presenting 0.936, 0.940, and 0.983 for LeNet, AlexNet, and ResNet respectively with the p-values less than $2e^{-5}$. In the VFL setting (Fig. 17), for ResNet-cut1, ResNet-cut2, and ResNet-cut3, the Pearson correlation coefficients are 0.976, 0.962, and 0.945 with corresponding p-values less than $4e^{-4}$ respectively. InvRE shows consistently strong positive correlation with the PSNR scores, indicating that InvRE increases/decreases proportionally as the PSNR scores.

When InvRE is lower than 0.15, PSNR drops below 17dB in HFL and 15dB in VFL, indicating severe reconstruction loss per prior studies [39, 47]. As shown in Fig. 9 and Fig. 10, reconstructions in this range lack clear structure, signaling failed attacks. As InvRE rises from 0.15 to 0.4, PSNR improves to over 22.5dB (HFL) and 18dB (VFL), revealing more recognizable textures with moderate quality. When InvRE exceeds 0.4, PSNR reaches 25dB (HFL) and 22dB (VFL), aligning with prior definitions [9, 41] of successful reconstruction. Corresponding images show clear textures of the originals. In summary, higher InvRE values are closely associated with higher PSNR or SSIM scores, indicating more accurate reconstructions of image details. This consistent trend across multiple image reconstruction metrics reinforces the reliability of InvRE as a proxy for privacy leakage.

**Results for comparing with other methods (Q5).** Table 6 and Table 7 compare InvL-DNP, and InvL-GNP / InvL-ENP with their respective baselines (DNP and GNP / ENP) across a range of parameter configurations on STL10 for HFL and VFL, respectively. As the results show, our InvL-based defense methods consistently achieve better task performance across

all settings, while maintaining comparable or stronger privacy protection. These findings further validate the effectiveness of our spectrally calibrated noise injection strategy in achieving a better privacy-utility trade-off across diverse conditions.

## C Additional experiments on LOKI [55].

In this section, we evaluate our InvL-based defense against LOKI [55], which steals client data via a malicious HFL server by modifying the model architecture. Specifically, LOKI inserts an attack module at the beginning of the original model, consisting of a convolutional layer followed by two fully connected (FC) layers. Image leakage occurs through gradients computed with respect to the weights connecting the convolutional output to the first FC layer. We use ResNet as the base model and STL10 as the dataset, with 20 neurons in the FC layer and a batch size of 10. InvRE scores are evaluated on the first 100 STL10 samples. Results are shown in Table 8. As shown, our InvL-DNP and InvL-ENP achieve a more optimal balance between privacy protection and model utility compared to existing noise perturbation defense methods.

<table><caption>Table 8: Performance of model accuracy (ACC) and DRA error by LOKI under different defenses.</caption><thead><tr><th></th><th>δ</th><th>MSE↑</th><th>PSNR↓</th><th>SSIM↓</th><th>ACC↑</th></tr></thead><tbody><tr><td>DNP</td><td>0.03</td><td>0.0024</td><td>26.47</td><td>0.905</td><td>0.833</td></tr><tr><td>InvL-DNP</td><td>0.07</td><td><b>0.012</b></td><td><b>24.94</b></td><td><b>0.772</b></td><td><b>0.840</b></td></tr><tr><td>GNP</td><td>0.001</td><td>0.060</td><td>12.35</td><td>0.396</td><td>0.802</td></tr><tr><td>InvL-GNP</td><td>0.005</td><td><b>0.081</b></td><td><b>11.09</b></td><td><b>0.373</b></td><td><b>0.813</b></td></tr></tbody></table>
<figure><img src="image_16.png" /><figcaption>(a) STL10</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>(b) SVHN</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>(c) ImageNet</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>(a) Prune</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>(b) Dropout</figcaption></figure>

<figure><img src="image_21.png" /><figcaption>(a) Prune</figcaption></figure>

<figure><img src="image_22.png" /><figcaption>(b) Dropout</figcaption></figure>

<figure><img src="image_23.png" /><figcaption>Figure 16: InvRE vs. PSNR (higher means successful reconstruction) in HFL.</figcaption></figure>

<figure><img src="image_24.png" /><figcaption>Figure 17: InvRE vs. PSNR (higher means successful reconstruction) in VFL.</figcaption></figure>