# Enhancing Privacy and Robustness in Federated Learning with Local Data Distribution Invariance and Byzantine-Resilient Aggregation

Bakary Dolo, Faiza Loukil, Khouloud Boukadi, Kavé Salamatian

► To cite this version:

Bakary Dolo, Faiza Loukil, Khouloud Boukadi, Kavé Salamatian. Enhancing Privacy and Robustness in Federated Learning with Local Data Distribution Invariance and Byzantine-Resilient Aggregation. The 36th IEEE International Symposium on Software Reliability Engineering (ISSRE 2025), Oct 2025, São Paulo, Brazil. hal-05311317

HAL Id: hal-05311317

https://hal.science/hal-05311317v1

Submitted on 13 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Enhancing Privacy and Robustness in Federated Learning with Local Data Distribution Invariance and Byzantine-Resilient Aggregation

Bakary Dolo

Mir@cl Laboratory, Sfax University

Sfax, Tunisia

bakary.dolo@fsegs.usf.tn

Khouloud Boukadi

Mir@cl Laboratory, Sfax University

Sfax, Tunisia

khouloud.boukadi@fsegs.usf.tn

Faiza Loukil

LISTIC, University Savoie Mont Blanc

Annecy-le-Vieux, France

faiza.loukil@univ-smb.fr

Kavé Salamatian

LISTIC, University Savoie Mont Blanc

Annecy-le-Vieux, France

kave.salamatian@univ-smb.fr

**Abstract**—Federated Learning (FL) has emerged as a promising paradigm for decentralized machine learning, enabling multiple clients to collaboratively train a global model without sharing their raw data. Despite its privacy-preserving design, FL remains vulnerable to privacy leakage through inference attacks, such as membership inference, and to integrity threats like Byzantine behaviors that can degrade model reliability. To address these risks, we propose Local Data Privatization Preprocessing (LDPP), a lightweight client-side method that enforces differential privacy while preserving the statistical properties of local data. LDPP operates through a three-stage process: (i) transforming data into a standardized representation (normal or uniform), (ii) injecting calibrated noise using differential privacy mechanisms, such as Laplace or Gaussian distributions, and (iii) applying an inverse transformation to asymptotically recover the original data distribution. We formally prove that LDPP satisfies $\epsilon$-differential privacy and maintains key distributional characteristics. Additionally, LDPP can be combined with robust aggregation techniques, such as Krum, to strengthen defense against adversarial tampering. Comprehensive experiments in EMNIST and MedMNIST datasets demonstrate that LDPP significantly reduces the success of membership inference attacks and improves robustness under label-flipping scenarios while preserving high model accuracy. These findings position LDPP as a scalable and practical solution to improve both privacy and robustness in federated learning frameworks.

**Index Terms**—Privacy, Federated Learning, Robust Aggregation, Data Preprocessing, Differential Privacy.

## I. INTRODUCTION

Federated Learning (FL) enables collaborative machine learning across decentralized data sources without requiring data centralization [1] [2]. By allowing participants to train a shared model while keeping their data local, FL addresses key privacy concerns associated with centralized data storage. However, FL is still vulnerable to attacks, such as label flipping and membership inference, which threaten both privacy and model integrity [3] [4].

Privacy in FL heavily depends on the techniques used during local training and model update aggregation [5]. Several privacy-preserving techniques have been developed, including anonymization-based [6] [7], encryption-based [8] [9] [10], and perturbation-based [11] [12] techniques. While anonymization reduces the risk of re-identification, it degrades data quality and model utility [13]. Despite its effectiveness, encryption can be computationally expensive, which limits scalability in resource-limited environments [14]. Among all these mentioned techniques, differential privacy (DP) [11], which is a perturbation-based technique, is widely used in federated learning. It offers a balance between privacy and utility. However, stronger privacy guarantees often come at the cost of reduced model performance and accuracy. Recent research highlights the need for more robust privacy preservation methods that address both privacy and integrity threats while maintaining high utility [15] [16] [17]. Motivated by the goal of preserving strong privacy and high accuracy, we propose a Local Data Privatization Preprocessing Technique (LDPP). LDPP is a fully client-side pipeline that preserves the original data distribution while enforcing $\epsilon$-differentiation privacy. It consists of three stages:

* Transformation: map raw features into a bounded space (via normalization or uniform scaling) to limit sensitivity;

* Noise addition: inject calibrated noise (Laplace, Gaussian, exponential or geometric) tailored to the data's distribution;

* Inverse transformation: restore the perturbed data to its original scale and distribution.

This design allows LDPP to maintain strong privacy guarantees while preserving data utility, ensuring that local model training suffers minimal degradation in accuracy. To demonstrate the versatility of LDPP, we also explore its integration with KRUM, a Byzantine-resilient aggregation method. While
not central to our main contribution, this combination provides an additional layer of protection against adversarial behaviors, such as label-flipping attacks, which involve intentionally mislabeling data to corrupt the global model. We evaluated LDPP on several MedMNIST databases measuring its effectiveness in terms of privacy protection, model performance, and robustness. We further compared our approach against several state-of-the-art methods to highlight its advantages across diverse federated learning scenarios.

The remainder of this paper is structured as follows: Section II reviews key concepts in FL and associated privacy and aggregation challenges. Section III discusses related work on differential privacy and robust aggregation in FL. Section IV presents our proposed method and details its design and implementation. Sections V and VI report the experimental setup and the obtained results. Finally, Section VIII concludes the paper and suggests future research directions.

## II. BACKGROUND

This section briefly discusses the concepts of federated learning, privacy and integrity threats in federated learning, federated aggregation techniques, and commun distribution distance metrics.

### A. Federated Learning (FL)

Federated Learning (FL) is a decentralized approach for training machine learning models collaboratively across multiple clients (e.g., edge devices) without sharing raw data. Each client trains a local model on its private data and only transmits model updates (e.g., parameters or gradients) to a central server, which aggregates them to update a global model.

Let $\mathcal{D}_k = \{(x_{i,k}, y_{i,k})\}_{i=1}^{n_k}$ be the local dataset of client $k$, where $n_k$ represents the number of data points, $x_{i,k}$ the i-th input, and $y_{i,k}$ its corresponding label. Each client minimizes its local loss function $\mathcal{L}_k(\mathbf{w})$ on its dataset:

$$ \mathcal{L}_k(\mathbf{w}) = \frac{1}{n_k} \sum_{i=1}^{n_k} \ell(\mathbf{w}; x_{i,k}, y_{i,k}), $$

where $\ell(\mathbf{w}; x, y)$ is the loss (e.g., cross-entropy) on a data point $(x, y)$.

The global objective is a weighted average of the local objectives across $K$ clients:

$$ \mathcal{L}(\mathbf{w}) = \sum_{k=1}^{K} \frac{n_k}{N} \mathcal{L}_k(\mathbf{w}), $$

where $N = \sum_{k=1}^{K} n_k$ is the total data size across all clients.

At each communication round $t$, the global model $\mathbf{w}_t$ is shared with clients, who update their local models via stochastic gradient descent (SGD):

$$ \mathbf{w}_k^{(t+1)} = \mathbf{w}_t - \eta \nabla \mathcal{L}_k(\mathbf{w}_t), $$

where $\eta$ is the learning rate. Updated models $\mathbf{w}_k^{(t+1)}$ are then sent back to the central server, which aggregates them to form the updated global model.

This iterative process continues across rounds until the global model reaches a state of convergence.

### B. Privacy and Integrity Threats in FL

In federated learning (FL), both privacy and integrity threats can undermine the system security. Privacy threats compromise the confidentiality of client data, while integrity threats degrade the accuracy and reliability of the global model by injecting malicious inputs or faulty updates. FL is vulnerable to various privacy attacks that can expose sensitive client data and malicious updates that can compromise the integrity of FL models. There are two main types of threats, namely:

* **Membership Inference Attacks:** These attacks attempt to infer whether a specific data point was included in a client's training set. This can lead to privacy breaches in sensitive applications, such as healthcare, where confirming that a patient's medical record is part of the training data could disclose confidential information. Adversaries exploit differences in model behavior between seen and unseen data to infer membership, posing risks like discrimination or social harm [18].

* **Label-flipping Attacks:** In these attacks, adversaries deliberately alter the labels of training data to mislead the model. For instance, in an image classification task, an attacker might label images of cars as bikes, distorting the global model's ability to make accurate predictions. This manipulation, if undetected, can have a lasting impact on the system reliability [19].

### C. Federated Aggregation Techniques

Aggregation techniques determine how model updates from individual clients are combined to update the global model:

* **Averaging Aggregation:** The most widely used aggregation technique is Federated Averaging (FedAvg) [1], where the global model $\mathbf{w}$ is updated as the weighted average of the clients' local updates:

$$ \mathbf{w}_{t+1} = \sum_{i=1}^{n} \frac{n_i}{N} \mathbf{w}_i \quad (1) $$

where $n_i$ is the number of data points at a client $i$, and $N$ is the total number of data points across all clients.

* **Weighted Aggregation:** In weighted aggregation [20], model updates are assigned different weights based on factors such as the size or quality of the data at each client. Formally, the global model is updated as:

$$ \mathbf{w}_{t+1} = \sum_{i=1}^{n} \alpha_i \mathbf{w}_i \quad (2) $$

where $\alpha_i$ is the weight assigned to client $i$'s update.

* **Robust Aggregation:** It refers to methods designed to ensure that model updates from different clients are combined in a way that is resistant to malicious or erroneous updates while still preserving the accuracy and integrity of the global model. Techniques like Krum [21] and trimmed mean [22] aim to filter out malicious updates. For Krum, the update from a client $i$ is selected based
on the distance between its update and those of the other clients:

$$Krum(\mathbf{w}_1, \mathbf{w}_2, \dots, \mathbf{w}_n) = \arg \min_i \sum_{j=1, j \neq i}^{n} d(\mathbf{w}_i, \mathbf{w}_j) \quad (3)$$

where $d(\mathbf{w}_i, \mathbf{w}_j)$ is a distance metric.

* **Secure Aggregation:** Secure aggregation techniques use cryptographic methods, such as homomorphic encryption, to ensure that individual updates remain private during aggregation. The goal is to compute:

$$f(\mathbf{w}_1, \mathbf{w}_2, \dots, \mathbf{w}_n) = \sum_{i=1}^{n} \mathbf{w}_i \quad (4)$$

without revealing any individual update [23].

## D. Distribution Distance Metrics

To evaluate how local data preprocessing alters the original distribution, we use distance metrics that quantify divergence. Three key measures are:

* **Wasserstein Distance (W-Dist):** Measures the minimal cost of transforming one distribution into another, accounting for the geometry of the data. In the one-dimensional case:

$$W(P, Q) = \int_{0}^{1} \left| F_{P}^{-1}(t) - F_{Q}^{-1}(t) \right| dt \quad (5)$$

where $F_P^{-1}$ and $F_Q^{-1}$ are the inverse cumulative distribution functions (CDFs) of distributions P and Q.

* **Kullback-Leibler Divergence (KL-Div):** Measures the information loss when approximating a distribution P with another distribution Q:

$$D_{\text{KL}}(P \parallel Q) = \sum_{x} P(x) \log \left( \frac{P(x)}{Q(x)} \right) \quad (6)$$

* **Kolmogorov-Smirnov (KS) Statistic:** A non-parametric test that captures the maximum absolute difference between two empirical cumulative distribution functions:

$$D_{\text{KS}} = \sup_{x} |F_{P}(x) - F_{Q}(x)| \quad (7)$$

## III. RELATED WORK

Local Differential Privacy (LDP) has been widely adopted in FL to protect user data by allowing clients to perturb their data or model updates before sharing, thereby eliminating the need for a trusted aggregator. For instance, LDP-Fed [24] introduces a framework for applying LDP in FL settings, allowing clients to perturb their updates locally, thus preserving privacy without relying on a trusted aggregator. Wang et al. [25] introduced an LDP framework for text mining to achieve both privacy and model accuracy. Kim et al. [26] proposed a method with noise variance bounds to improve utility in a FedSGD model using Gaussian noise while Bhowmick et al. [27] used LDP mechanisms to protect against reconstruction attacks. However, recent studies have demonstrated that LDP alone may not suffice against sophisticated membership inference attacks

or byzantine attacks [28]–[33]. These findings highlight the need for more robust privacy preserving mechanisms that can safeguard against such vulnerabilities without compromising the model utility. To defend model integrity, robust aggregation methods filter out malicious or outlying updates. Krum [21] and its enhanced variants select updates closest to the majority, while Bulyan extends these ideas for stronger guarantees. Romoa [34] resists targeted poisoning attacks, SmartFL [35] addresses heterogeneity, and RFOut-1d [36] filters backdoor updates dimension-wise. Although effective at mitigating adversarial behavior, these methods do not offer formal privacy guarantees and typically assume homogeneous data. Some recent works attempt to combine differential privacy with robust aggregation. For instance, Zhu et al. [37] address both privacy and Byzantine robustness in non-IID settings. Gai et al. [38] apply LDP-based aggregation in smart grids, while Lu and Fan [39] focus on CNN convergence under noise. Grama et al. [16] propose adaptive strategies that blend robustness and privacy. While these hybrid approaches enhance resilience, they often suffer from increased computational complexity and noise-induced distortion in data distribution. A key limitation of current LDP methods is their tendency to distort the statistical distribution of local data, which negatively impacts the convergence and overall performance of the global model, especially in heterogeneous and non-IID FL environments. Preserving local data distributions after noise addition is therefore critical to maintain the utility of the model [40].

To address these challenges, we introduce Local Data Privatization Preprocessing (LDPP), a client-side technique designed to preserve the statistical structure of local data while ensuring privacy. LDPP works by transforming local data into a uniform or standardized space, injecting calibrated noise, and applying an inverse transformation to approximately restore the original distribution. This method aims to offer strong privacy guarantees without sacrificing data utility. Moreover, LDPP can be seamlessly combined with robust aggregation techniques, offering a holistic defense against both privacy leakage and adversarial manipulation in federated learning. To provide a clearer overview of existing works, Table I summarizes representative methods in terms of differential privacy guarantees, robustness against Byzantine attacks, and the ability to preserve the original distribution of local data in a federated learning scenario.

<table><caption>TABLE I: Comparison of Related Methods in Federated Learning</caption><thead><tr><th>Method</th><th>Privacy</th><th>Robustness</th><th>Distribution Preservation</th></tr></thead><tbody><tr><td>Wang et al. [25]</td><td>✓</td><td>✗</td><td>✗</td></tr><tr><td>Kim et al. [26]</td><td>✓</td><td>✗</td><td>✗</td></tr><tr><td>SmartFL [35]</td><td>✗</td><td>✓</td><td>✗</td></tr><tr><td>RFOut-1d [36]</td><td>✗</td><td>✓</td><td>✗</td></tr><tr><td>Zhu et al. [37]</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>Grama et al. [16]</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>LDPP (Ours)</td><td>✓</td><td>✓</td><td>✓</td></tr></tbody></table>
## IV. PROPOSED METHOD

To guarantee high data utility and strong data privacy in federated learning, we propose a new approach called Local Data Privatization Preprocessing (LDPP). This approach aims to keep the local data distribution invariant after noise addition to address critical vulnerabilities related to privacy breaches. LDPP is designed to be lightweight and non-intrusive. It operates entirely on the client side and does not require any modification to the standard federated learning protocol or infrastructure. Figure 1 illustrates all steps of LDPP’s process.

### A. Local Data Privatization Preprocessing

Each client $i$ applies a transformation to the local dataset $D_i$ to standardize the data distribution. Depending on the chosen method, the transformation can be either normal or uniform:

* **Normal Transformation:**

$$x_{\text{norm}} = \frac{x - \mu_x}{\sigma_x}$$

where $\mu_x$ and $\sigma_x$ are the mean and standard deviation of a feature $x$ in $D_i$.

* **Uniform Transformation:**

$$x_{\text{unif}} = \frac{x - x_{\text{min}}}{x_{\text{max}} - x_{\text{min}}}$$

where $x_{\text{min}}$ and $x_{\text{max}}$ are the minimum and maximum values of feature $x$ in $D_i$.

**Noise Addition for Differential Privacy:** After transforming the data, each client locally adds noise to achieve differential privacy, thereby obscuring individual data points, using:

* **Laplace Noise:**

$$x_{\text{priv}} = x_{\text{trans}} + \eta, \quad \eta \sim \text{Laplace}(0, b)$$

Laplace noise is calibrated based on the sensitivity of the data to provide $\epsilon$-differential privacy.

* **Gaussian Noise:**

$$x_{\text{priv}} = x_{\text{trans}} + \eta, \quad \eta \sim \mathcal{N}(0, \sigma^2)$$

Gaussian noise provides $(\epsilon, \delta)$-differential privacy, offering flexibility in privacy-accuracy trade-offs.

### Reverse Transformation.

To maintain data utility, each client performs reverse transformations after noise addition, restoring the data to its original scale:

* **Normal Transformation Reversion:**

$$x_{\text{reversed}} = x_{\text{priv}} \times \sigma_x + \mu_x$$

* **Uniform Transformation Reversion:**

$$x_{\text{reversed}} = x_{\text{priv}} \times (x_{\text{max}} - x_{\text{min}}) + x_{\text{min}}$$

This step ensures that the noise addition does not significantly distort the data distribution, allowing effective model training while preserving privacy.

After applying the local data preprocessing technique, each client locally trains a model to obtain model updates:

$$\Delta\theta_i = \theta_i - \theta$$

where $\theta_i$ represents the updated model parameters after training on the privatized dataset $D_i$, and $\theta$ is the current global model.

### B. Theoretical Justification

Let $\mathcal{P}$ denote the original data distribution and $\mathcal{P}'$ the privatized distribution after applying LDPP. The objective is to bound the 2-Wasserstein distance $W_2(\mathcal{P}, \mathcal{P}')$, ensuring that the statistical properties of the data are preserved post-privatization. The 2-Wasserstein distance between the original distribution $\mathcal{P}$ and the privatized distribution $\mathcal{P}'$ can be bounded as follows:

* **Effect of Noise Addition:** Adding Gaussian noise corresponds to convolving the standardized distribution with a Gaussian kernel. The Wasserstein distance between the original and noised distributions in the standardized space is bounded by:

$$W_2(\mathcal{P}_{\text{norm}}, \mathcal{P}_{\text{norm}} * \mathcal{N}(0, \sigma^2 I_d)) \leq \sigma\sqrt{d}$$

This inequality leverages the property that convolving with a Gaussian kernel of variance $\sigma^2$ increases the Wasserstein distance by at most $\sigma\sqrt{d}$ [41].

* **Scaling Effect of Reverse Transformation:** The reverse transformation scales the Wasserstein distance by $\sigma_x$:

$$W_2(\mathcal{P}, \mathcal{P}') = \sigma_x \cdot W_2(\mathcal{P}_{\text{norm}}, \mathcal{P}_{\text{norm}} * \mathcal{N}(0, \sigma^2 I_d))$$

* **Combined Bound:** Combining the above results:

$$W_2(\mathcal{P}, \mathcal{P}') \leq \sigma_x \cdot \sigma\sqrt{d}$$

By appropriately calibrating the noise level $\sigma$ and considering the data’s dimensionality $d$ and standard deviation $\sigma_x$, LDPP ensures that the Wasserstein distance between the original and privatized distributions remains bounded. This guarantees the preservation of the data’s statistical properties, facilitating effective model training while providing strong privacy assurances.

## V. EXPERIMENTAL SETUP

This section outlines the experimental framework designed to evaluate the performance of the proposed Local Data Privatization Preprocessing (LDPP) method.

### A. Dataset Description

We use two publicly available datasets, namely EMNIST [42] and MedMNIST [43].

EMNIST [42] is an extension of the original MNIST dataset designed for handwritten character recognition. It includes 47 distinct classes, comprising digits (0-9) and both uppercase and lowercase English letters, ensuring balanced representation.
<figure><img src="image_2.png" /><figcaption>Fig. 1: Steps of the Local Data Privatization Preprocessing with FedAvg or Krum Aggregation For Federated Learning.</figcaption></figure>

We used the balanced version of EMNIST, which contains 112,800 grayscale images, each represented as a 28×28 pixel array.

**MedMNIST** [43] is a standardized collection of biomedical image datasets, with all images resized to 28×28 pixels to ensure consistency and lightweight processing. For our experiments, we selected three representative subsets from MedMNIST. The first, **PathMNIST**, contains histopathological images used to classify tissue types based on disease-related patterns. The second, **BloodMNIST**, is composed of blood smear images designed for the classification of various blood cell types. Lastly, **OrganAMNIST** features organ-specific medical images to support disease detection and organ classification tasks. These diverse subsets offer a robust evaluation ground for assessing both the privacy preservation and model performance capabilities of our proposed approach.

## B. Model Architecture

For experiments on EMNIST and MedMNIST, we choose a Convolutional Neural Network (CNN) because of its ability to extract visual features from images. The baseline model employs a lightweight CNN architecture optimized for MedMNIST's 28×28 grayscale images. The network comprises two convolutional layers (32 filters, 3×3 kernels, ReLU activation), followed by a 2×2 max-pooling layer to downsample feature maps. These are flattened and passed through a fully connected layer (128 neurons, ReLU) before final classification via a softmax-activated dense layer matching the dataset's class count. Compiled with the Adam optimizer and categorical cross-entropy loss, this design balances simplicity and expressiveness, capturing discriminative patterns while minimizing overfitting risks for federated medical imaging tasks.

## C. Evaluation Metrics

To assess model performance, privacy preservation, and computational cost, we use the following metrics:

* **Accuracy (ACC):** Measures the proportion of correctly predicted instances.

* **Area Under the Curve (AUC):** Evaluates the model's ability to distinguish between classes for standard classification tasks.

* **F1 Score:** Harmonic mean of precision and recall, particularly useful for imbalanced datasets.

* **ROC AUC (MIA):** Area Under the Receiver Operating Characteristic curve for membership inference attacks. This metric quantifies how well the attack model separates members from non-members. A lower ROC AUC implies better privacy protection.

* **Execution Time:** Measures the total time required to complete a federated learning round, including local training, noise addition, and aggregation.

## D. Comparative Baselines

We compare LDPP approach with the following baselines:

* **FEDAVG:** Standard federated averaging without privacy or robustness enhancements.

* **LDPP-AVG:** Integrates LDPP with the FedAvg aggregation technique.

* **LDPP-KRUM:** Combines LDPP with the Krum aggregation method, which is designed to be robust against Byzantine failures.

## E. Experimental Scenario

We apply LDPP using Keras's `QuantileTransformer` to standardize data into normal distribution. To enforce differential privacy, we inject calibrated noise using the Laplace mechanism. Privacy budgets of $\epsilon = 1$ and $\epsilon = 0.5$ were applied for the EMNIST dataset. For MedMNIST, privacy budgets are set to $\epsilon = 3$ and $\epsilon = 7$ to enable comparative analysis with existing state-of-the-art approaches [44]. After comparative tests and theoretical considerations, Laplace noise was chosen as the primary mechanism for the main results. This choice is motivated by several factors:
* **Strict $\epsilon$-Differential Privacy:** Laplace noise aligns directly with the definition of $\epsilon$-DP and offers strong privacy guarantees without the added complexity of $(\epsilon, \delta)$ parameters, unlike Gaussian mechanisms.

* **Better Distribution Compatibility:** Empirically, Laplace noise works well with both normal and uniform transformations, preserving the original data distribution more effectively after inverse transformation.

* **Interpretability and Control:** The Laplace noise scale is simply controlled by the privacy budget $\epsilon$ and the sensitivity of the data, which improves interpretability and reproducibility.

* **Efficiency:** Laplace noise is computationally lightweight, making it well-suited for edge devices involved in federated learning.

For EMNIST, the simulation involved 8 clients, each assigned a unique, non-overlapping subset of the dataset to ensure distinct data ownership, reflecting real-world scenarios where data are distributed across multiple institutions.

For MedMNIST, we simulate 12 clients with distinct non-iid data portions. The training process spans 150 communication rounds, with local model updates aggregated to form a global model. We chose $\epsilon = 3$ and $\epsilon = 7$ as they represent commonly used moderate and relaxed privacy levels in differential privacy literature, allowing for a fair comparison of our proposed method with existing state-of-the-art approaches in terms of both privacy and model performance.

We also evaluate the robustness of the model against two types of attacks:

* **Membership Inference Attacks (MIA):** We evaluate privacy leakage using a standard shadow model-based membership inference attack, following the methodology of Shokri et al. [18]. The attacker trains shadow models to mimic the behavior of the target model and infers membership based on prediction confidence and entropy. This setup reflects a commonly used black-box MIA scenario in federated learning. We evaluated the success of these attacks using the ROC AUC metric.

* **Label-Flipping Attacks:** We simulate scenarios with 30% and 50% malicious clients. In these attacks, adversarial clients intentionally alter the labels of their training data to mislead the global model during aggregation. This setup assesses the effectiveness of the Krum aggregation algorithm in mitigating the impact of corrupted updates.

# VI. PERFORMANCE EVALUATION AND RESULTS

In this section, we present the results of our experimental evaluation of LDPP.

## A. Distributional Preservation Metrics

We evaluate LDPP's ability to preserve original data distributions across PathMNIST, BloodMNIST, and OrganAMNIST using three statistical metrics: Wasserstein Distance, KL Divergence, and KS Statistic. Tests results conducted under two privacy budgets ($\epsilon = 3$ and $\epsilon = 7$) are available in Table II.

These metrics quantify the similarity between the original and privatized data distributions, where lower values indicate better preservation of the original data structure.

<table><caption>TABLE II: Distributional Preservation Metrics for LDPP across PathMNIST, BloodMNIST, and OrganAMNIST datasets with $\epsilon = 3$ and $\epsilon = 7$.</caption><thead><tr><th>Dataset</th><th>ε</th><th>W-Dist</th><th>KL-Div</th><th>KS-Stat</th></tr></thead><tbody><tr><td rowspan="2">PathMNIST</td><td>3</td><td>0.019 ± 0.007</td><td>0.193 ± 0.263</td><td>0.090 ± 0.055</td></tr><tr><td>7</td><td>0.006 ± 0.003</td><td>0.199 ± 0.262</td><td>0.048 ± 0.043</td></tr><tr><td rowspan="2">BloodMNIST</td><td>3</td><td>0.016 ± 0.003</td><td>0.087 ± 0.053</td><td>0.047 ± 0.013</td></tr><tr><td>7</td><td>0.004 ± 0.001</td><td>0.092 ± 0.077</td><td>0.021 ± 0.006</td></tr><tr><td rowspan="2">OrganAMNIST</td><td>3</td><td>0.032 ± 0.013</td><td>0.782 ± 0.689</td><td>0.083 ± 0.035</td></tr><tr><td>7</td><td>0.011 ± 0.004</td><td>0.733 ± 0.599</td><td>0.041 ± 0.019</td></tr></tbody></table>

As shown in Table II, LDPP consistently yields low Wasserstein distances and KS statistics across all datasets, indicating strong distributional fidelity. Notably, with $\epsilon = 7$, all three metrics improve, which is expected as a higher privacy budget allows less perturbation of the data.

Among the datasets, BloodMNIST exhibits the best preservation performance under both privacy budgets, with the lowest values across all three metrics. The higher KL divergence observed on OrganAMNIST can be attributed to its greater anatomical variability and class imbalance. Nevertheless, the low Wasserstein distance and KS statistic indicate that LDPP still preserves the overall distribution shape, and model accuracy remains competitive. These results confirm that LDPP effectively balances privacy and distributional integrity, even under strict privacy budgets. Figure 2 shows an illustration of these distance metrics operated on the PathMNIST dataset.

## B. Privacy-Utility Trade-off

For the EMNIST dataset, Figure 3 compares the performance of three federated learning models namely FedAvg, LDPP-AVG, and LDPP-KRUM under two privacy budgets ($\epsilon = 1$ and $\epsilon = 0.5$) and two noise mechanisms (Gaussian and Laplace).

Figures 3a and 3b present the results with Gaussian noise. In both cases, FedAvg achieves the highest accuracy, stabilizing around 87% after only a few communication rounds. LDPP-AVG comes next, reaching approximately 79%, while LDPP-KRUM performs notably worse in terms of accuracy.

Figures 3c and 3d illustrate the results under Laplace noise. Once again, LDPP-AVG shows greater robustness, maintaining an accuracy close to 80%, whereas LDPP-KRUM remains the least performant approach overall.

Among privacy-preserving methods, LDPP-AVG stands out as the most balanced solution, offering a favorable trade-off between privacy and performance, making it a strong candidate for privacy-sensitive federated learning applications. However, under the Gaussian mechanism, LDPP-KRUM exhibits slightly improved resilience, although this comes at the cost of a substantial drop in accuracy. This trade-off may be acceptable in highly adversarial settings where robustness is prioritized over precision.

For MedMNIST, we evaluate the performance of LDPP under varying $\epsilon$ values (3 and 7), using the Laplace noise
<figure><img src="image_3.png" /><figcaption>Fig. 2: Distributional Preservation Metrics for PathMNIST</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3: Comparison of FedAvg, LDPP-AVG, and LDPP-KRUM models under Gaussian and Laplace noises applied on the EMNIST dataset.</figcaption></figure>

mechanism and normal transformation preprocessing. The experiments are conducted on three different medical imag-

ing datasets: PathMNIST, BloodMNIST, and OrganAMNIST.

We compare the LDPP approach (with FedAvg and Krum
aggregation strategies) against the standard FedAvg without any privacy mechanism. The metrics reported include accuracy (Acc), area under the curve (AUC), F1-score and training time per round.

<table><caption>TABLE III: Model Performance Comparison (PathMNIST dataset)</caption><thead><tr><th rowspan="2">Method</th><th colspan="5">PathMNIST - Evaluation Metrics</th></tr><tr><th>ε</th><th>Acc</th><th>Auc</th><th>F1</th><th>Time(s)</th></tr></thead><tbody><tr><td>LDPP-AVG</td><td>3</td><td>0.7410</td><td>0.9630</td><td>0.7450</td><td>56.77</td></tr><tr><td>LDPP-AVG</td><td>7</td><td>0.8256</td><td>0.9738</td><td>0.8235</td><td>56.44</td></tr><tr><td>LDPP-KRUM</td><td>3</td><td>0.6891</td><td>0.9333</td><td>0.6708</td><td>58.85</td></tr><tr><td>LDPP-KRUM</td><td>7</td><td>0.7434</td><td>0.9470</td><td>0.5770</td><td>58.62</td></tr><tr><td>FEDAVG</td><td>-</td><td>0.8314</td><td>0.9800</td><td>0.8302</td><td>53.85</td></tr></tbody></table>

Avg. over 3 runs; std. omitted for brevity.

<table><caption>TABLE IV: Model Performance Comparison (BloodMNIST dataset)</caption><thead><tr><th rowspan="2">Method</th><th colspan="5">BloodMNIST - Evaluation Metrics</th></tr><tr><th>ε</th><th>Acc</th><th>Auc</th><th>F1</th><th>Time(s)</th></tr></thead><tbody><tr><td>LDPP-AVG</td><td>3</td><td>0.8672</td><td>0.9844</td><td>0.8675</td><td>26.40</td></tr><tr><td>LDPP-AVG</td><td>7</td><td>0.9117</td><td>0.9909</td><td>0.9118</td><td>26.30</td></tr><tr><td>LDPP-KRUM</td><td>3</td><td>0.7871</td><td>0.9650</td><td>0.7937</td><td>28.32</td></tr><tr><td>LDPP-KRUM</td><td>7</td><td>0.8427</td><td>0.9772</td><td>0.8435</td><td>28.07</td></tr><tr><td>FEDAVG</td><td>-</td><td>0.9225</td><td>0.9921</td><td>0.9225</td><td>25.24</td></tr></tbody></table>

Avg. over 3 runs; std. omitted for brevity.

<table><caption>TABLE V: Model Performance Comparison (OrganAMNIST dataset)</caption><thead><tr><th rowspan="2">Method</th><th colspan="5">OrganAMNIST - Evaluation Metrics</th></tr><tr><th>ε</th><th>Acc</th><th>Auc</th><th>F1</th><th>Time(s)</th></tr></thead><tbody><tr><td>LDPP-AVG</td><td>3</td><td>0.8460</td><td>0.9806</td><td>0.8465</td><td>33.42</td></tr><tr><td>LDPP-AVG</td><td>7</td><td>0.8618</td><td>0.9856</td><td>0.8627</td><td>33.12</td></tr><tr><td>LDPP-KRUM</td><td>3</td><td>0.8086</td><td>0.9721</td><td>0.8078</td><td>34.06</td></tr><tr><td>LDPP-KRUM</td><td>7</td><td>0.84025</td><td>0.9804</td><td>0.8414</td><td>33.88</td></tr><tr><td>FEDAVG</td><td>-</td><td>0.8641</td><td>0.9853</td><td>0.8647</td><td>31.37</td></tr></tbody></table>

Avg. over 3 runs; std. omitted for brevity.

As shown respectively in Table III, IV and V, LDPP with $\epsilon = 7$ generally achieves better performance compared to $\epsilon = 3$, demonstrating the expected privacy-utility trade-off: increasing the privacy budget (i.e., reducing noise) improves utility. Compared to the baseline FedAvg (without any local privatization), the LDPP approach introduces only a moderate degradation in performance, particularly under higher $\epsilon$ values.

In the PathMNIST dataset, LDPP-AVG with $\epsilon = 7$ achieves an accuracy of 0.8256, which is close to the FedAvg accuracy of 0.8314 but with a lower privacy leakage. For the BloodMNIST dataset, LDPP-AVG reaches 0.9117 accuracy with $\epsilon = 7$ versus 0.9225 for FedAvg. Notably, in OrganAMNIST, LDPP methods show competitive accuracy and improved privacy compared to the FedAvg baseline, with LDPP-AVG ($\epsilon = 7$) achieving 0.8618 accuracy versus 0.8641 for FedAvg.

In terms of execution time, the LDPP approach introduces only a modest overhead compared to the baseline FedAvg. For instance, on the PathMNIST dataset, LDPP-AVG requires approximately 56.77s and 56.44s for $\epsilon = 3$ and $\epsilon = 7$ respectively, compared to 53.85s for FedAvg. Similar slight increases are observed on BloodMNIST (26.30s vs. 25.24s)

and OrganAMNIST (33.42s vs. 31.37s), demonstrating that the privacy-enhancing benefits of LDPP come at a minimal computational cost. The LDPP-KRUM variant incurs the highest execution times across datasets due to its additional robustness filtering step (e.g., 58.85s on PathMNIST and 34.06s on OrganAMNIST), yet remains within reasonable execution limits. It is important to note that this overhead is limited in scope, as LDPP preprocessing is applied only once before training begins, thereby not affecting the per-epoch training time.

These results confirm the effectiveness of LDPP in balancing privacy and utility across multiple datasets and aggregation strategies, making it a viable approach for privacy-preserving federated learning. Figures 4 illustrates the accuracy comparison for BloodMNIST and OrganAMNIST, respectively, using FEDAVG and LDPP-AVG.

<figure><img src="image_5.png" /><figcaption>Fig. 4: Test Accuracy Comparison for BloodMNIST and OrganAMNIST</figcaption></figure>

### C. Comparison with Related Work

We evaluated the effectiveness of our LDPP method against recent approaches [44]–[46] across the MedMNIST datasets, as summarized in Table VI. Results illustrate the trade-offs be-
tween privacy and performance in federated and non-federated settings with and without differential privacy.

<table><caption>TABLE VI: Comparison of accuracy across FL, DP-FL, and DP-Non-FL settings with DP presence and $\epsilon$ values.</caption><thead><tr><th>Dataset</th><th>Method</th><th>DP</th><th>ε</th><th>Accuracy</th></tr></thead><tbody><tr><td rowspan="6">PathMNIST</td><td>FEDAVG</td><td>No</td><td>-</td><td>0.8314</td></tr><tr><td>Hölzl et al. [46]</td><td>No</td><td>-</td><td>0.8212</td></tr><tr><td>Mh et al. [44]</td><td>No</td><td>-</td><td>0.8940</td></tr><tr><td>LDPP-AVG</td><td>Yes</td><td>7</td><td><strong>0.8256</strong></td></tr><tr><td>Hölzl et al. [46]</td><td>Yes</td><td>7.42</td><td>0.7791</td></tr><tr><td>LDPP-AVG</td><td>Yes</td><td>3</td><td>0.7434</td></tr><tr><td></td><td>Mh et al. [44]</td><td>Yes</td><td>3</td><td>0.8110</td></tr><tr><td rowspan="4">BloodMNIST</td><td>FEDAVG</td><td>No</td><td>-</td><td>0.9225</td></tr><tr><td>Mh et al. [44]</td><td>No</td><td>-</td><td>0.9390</td></tr><tr><td>LDPP-AVG</td><td>Yes</td><td>3</td><td><strong>0.8672</strong></td></tr><tr><td>Mh et al. [44]</td><td>Yes</td><td>3</td><td>0.8570</td></tr><tr><td rowspan="5">OrganAMNIST</td><td>FEDAVG</td><td>No</td><td>-</td><td>0.8641</td></tr><tr><td>Zhang et al. [45]</td><td>No</td><td>-</td><td>0.8286</td></tr><tr><td>Mh et al. [44]</td><td>No</td><td>-</td><td>0.8960</td></tr><tr><td>LDPP-AVG</td><td>Yes</td><td>3</td><td><strong>0.8460</strong></td></tr><tr><td>Mh et al. [44]</td><td>Yes</td><td>3</td><td>0.7940</td></tr></tbody></table>

LDPP-AVG consistently demonstrates competitive performance, particularly in the presence of privacy constraints (e.g., $\epsilon = 3$ or $\epsilon = 7$), while maintaining a federated structure. On PathMNIST, LDPP-AVG achieves an accuracy of 0.8256 at $\epsilon = 7$, outperforming Hölzl et al.'s DP approach and approaching the baseline performance of non-private FL (FEDAVG). Notably, it also surpasses the DP variant of Mh et al. [44] at $\epsilon = 3$.

In the case of BloodMNIST, LDPP-AVG with $\epsilon = 3$ reaches 0.8672 accuracy, which is higher than the DP-enhanced version of Mh et al.'s method, while remaining within the same privacy budget. This demonstrates LDPP's ability to preserve utility in sensitive medical imaging data.

Similarly, on OrganAMNIST, our method again achieves superior performance under differential privacy ($\epsilon = 3$) compared to other DP methods. LDPP-AVG attains 0.8460 accuracy, outperforming Mh et al.'s DP version by over 5%, and approaching the performance of standard federated learning without privacy constraints.

Overall, these comparisons highlight LDPP's effectiveness in providing a favorable privacy-utility trade-off, especially under strict privacy budgets. Unlike methods that trade significant utility for privacy, LDPP maintains good performance while ensuring rigorous data protection in federated learning environments. While our results on PathMNIST are promising, comparison with [44] should be interpreted cautiously, as it uses DP-SGD with a custom CNN. In contrast, our model-agnostic, data-level approach also addresses poisoning robustness. Our goal is to offer complementary insights under different privacy settings, not claim direct superiority.

## D. Membership Inference Attack (MIA) Resilience

Membership inference attacks (MIA) aim to determine whether a particular data sample was used during the training phase of a machine learning model. These attacks exploit the model's tendency to memorize training data, especially when

it is overfitted or insufficiently regularized, thereby threatening data privacy even when raw inputs are not directly shared.

To assess the privacy-preserving effectiveness of our Local Data Privatization Preprocessing (LDPP) approach, we evaluate its robustness against MIAs. Table VII summarizes the MIA attack ROC AUC across the three datasets (PathMNIST, BloodMNIST, and OrganAMNIST) under different configurations.

<table><caption>TABLE VII: Membership Inference Attack (MIA) Success Rate</caption><thead><tr><th>Method</th><th>ε</th><th>PathMNIST</th><th>BloodMNIST</th><th>OrganAMNIST</th></tr></thead><tbody><tr><td>LDPP-AVG</td><td>3</td><td>0.5250</td><td>0.5271</td><td>0.6288</td></tr><tr><td>LDPP-AVG</td><td>7</td><td>0.5647</td><td>0.5365</td><td>0.6319</td></tr><tr><td>LDPP-KRUM</td><td>3</td><td>0.5508</td><td>0.5138</td><td>0.6303</td></tr><tr><td>LDPP-KRUM</td><td>7</td><td>0.5317</td><td>0.5208</td><td>0.6383</td></tr><tr><td>FEDAVG</td><td>-</td><td>0.5794</td><td>0.5368</td><td>0.6441</td></tr></tbody></table>

From Table VII, we observe that LDPP consistently reduces the success rate of membership inference attacks compared to the baseline FedAvg, particularly for lower $\epsilon$ values (e.g., $\epsilon = 3$). This confirms that LDPP improves privacy protection by limiting model overfitting and reducing information leakage.

Interestingly, LDPP-AVG with $\epsilon = 3$ achieves the lowest MIA rate on PathMNIST (0.5250), while LDPP-KRUM with the same privacy level offers the best defense on BloodMNIST (0.5138). Although OrganAMNIST exhibits generally higher MIA rates across all methods, LDPP still shows improved performance over FedAvg.

These results demonstrate that LDPP is not only effective at maintaining model accuracy but also offers tangible privacy benefits by reducing susceptibility to membership inference attacks.

## E. Label-Flipping Attack Resilience

Label flipping attack is an adversarial technique in which training data labels are maliciously altered to incorrect classes to mislead and degrade the performance of the trained model. Across the three subsets of the MedMNIST dataset, Table VIII and Table IX show that FedAvg confronts significant performance degradation, particularly when 50% of the clients are malicious. In contrast, LDPP-KRUM demonstrates good resilience, maintaining higher accuracy, AUC, and F1-scores even with $\epsilon = 3$. For instance, on the OrganAMNIST subset, LDPP-KRUM achieves accuracies of 0.7556 and 0.7040 for 30% and 50% malicious clients, respectively, highlighting its robustness under adversarial conditions.

# VII. DISCUSSION

This study introduces Local Data Privatization Preprocessing (LDPP), a client-side framework that enhances the privacy and robustness of federated learning. It combines distribution-invariant noise injection, differential privacy guarantees, and Byzantine-resilient aggregation. Our findings demonstrate that LDPP effectively balances privacy and model utility while maintaining strong performance even under adversarial conditions. For instance, with the BloodMNIST dataset, LDPP
<table><caption>TABLE VIII: Performance Under Label-Flipping Attacks (30% Malicious Clients) on MedMNIST</caption><thead><tr><th>Dataset</th><th>Method</th><th>ε</th><th>ACC</th><th>AUC</th><th>F1</th></tr></thead><tbody><tr><td rowspan="3">PathMNIST</td><td>FEDAVG</td><td>-</td><td>0.8018</td><td>0.9613</td><td>0.7976</td></tr><tr><td>LDPP-AVG</td><td>3</td><td>0.7268</td><td>0.9286</td><td>0.7187</td></tr><tr><td>LDPP-KRUM</td><td>3</td><td>0.6520</td><td>0.8704</td><td>0.6481</td></tr><tr><td rowspan="3">BloodMNIST</td><td>FEDAVG</td><td>-</td><td>0.8506</td><td>0.9743</td><td>0.8535</td></tr><tr><td>LDPP-AVG</td><td>3</td><td>0.8178</td><td>0.9728</td><td>0.8163</td></tr><tr><td>LDPP-KRUM</td><td>3</td><td>0.7302</td><td>0.9540</td><td>0.7340</td></tr><tr><td rowspan="3">OrganAMNIST</td><td>FEDAVG</td><td>-</td><td>0.8350</td><td>0.9737</td><td>0.8384</td></tr><tr><td>LDPP-AVG</td><td>3</td><td>0.7818</td><td>0.9761</td><td>0.7741</td></tr><tr><td>LDPP-KRUM</td><td>3</td><td>0.7556</td><td>0.9600</td><td>0.7520</td></tr></tbody></table>

<table><caption>TABLE IX: Performance Under Label-Flipping Attacks (50% Malicious Clients) on MedMNIST</caption><thead><tr><th>Dataset</th><th>Method</th><th>ε</th><th>ACC</th><th>AUC</th><th>F1</th></tr></thead><tbody><tr><td rowspan="3">PathMNIST</td><td>FEDAVG</td><td>-</td><td>0.4855</td><td>0.8665</td><td>0.4786</td></tr><tr><td>LDPP-AVG</td><td>3</td><td>0.4171</td><td>0.8538</td><td>0.3682</td></tr><tr><td>LDPP-KRUM</td><td>3</td><td><strong>0.6178</strong></td><td><strong>0.8730</strong></td><td><strong>0.6030</strong></td></tr><tr><td rowspan="3">BloodMNIST</td><td>FEDAVG</td><td>-</td><td>0.6354</td><td>0.8652</td><td>0.6074</td></tr><tr><td>LDPP-AVG</td><td>3</td><td>0.5758</td><td>0.8769</td><td>0.5385</td></tr><tr><td>LDPP-KRUM</td><td>3</td><td><strong>0.6477</strong></td><td><strong>0.9115</strong></td><td><strong>0.6407</strong></td></tr><tr><td rowspan="3">OrganAMNIST</td><td>FEDAVG</td><td>-</td><td>0.5500</td><td>0.9190</td><td>0.5475</td></tr><tr><td>LDPP-AVG</td><td>3</td><td>0.4258</td><td>0.9115</td><td>0.4083</td></tr><tr><td>LDPP-KRUM</td><td>3</td><td><strong>0.7040</strong></td><td><strong>0.9440</strong></td><td><strong>0.7028</strong></td></tr></tbody></table>

maintained a high level of predictive accuracy (91.17%) at a privacy budget of $\epsilon = 7$, demonstrating its its usefulness in privacy-preserving medical applications.

In contrast to traditional approaches like FedAvg, LDPP demonstrates enhanced resilience against both label-flipping and membership inference attacks. The choice of Laplace and Gaussian noise in this study is advantageous for their compatibility with DP and computational efficiency. However, their effectiveness may vary across different domains. Alternative mechanisms, such as exponential or geometric noise could offer better trade-offs for domain-specific applications, particularly in settings with different data distributions or privacy constraints. Furthermore, future evaluations should test against gradient inversion and adaptive poisoning attacks (e.g., LIE [47]) to further validate the LDPP’s robustness. Additionally, Krum’s efficiency is restricted in highly adversarial environments due to its assumption that adversaries do not collude and that the data are uniform. Therefore, future work should explore hybrid aggregation strategies, such as combining LDPP with adaptive defenses like FoolsGold [48], to better counter collusion and data poisoning. Notably, LDPP is aggregator-agnostic and compatible with other robust methods, highlighting its role as a complementary, data-level defense mechanism. Informally, skipping the transformation step led to KL divergence up to 1.3 and a drastic decrease in precision on different datasets, highlighting its importance. In summary, LDPP provides an interesting basis for addressing privacy concerns in federated learning. It is adaptable to various use cases, and its integration potential with complementary defense mechanisms opens up many possibilities for further research.

VIII. CONCLUSION

This work proposes Local Data Privatization Preprocessing (LDPP), a client-side technique that improves both privacy and robustness in federated learning through distribution-invariant noise injection and Byzantine-resilient aggregation. LDPP effectively protects user data, as demonstrated by its ability to reduce membership inference attack success (ROC AUC $\le 0.5647$ at $\epsilon = 7$) while minimizing the accuracy loss ($\le 2.69\%$ compared to FedAvg). Moreover, our proposal boosts robustness by significantly enhancing accuracy under adversarial circumstances (such as +15.4% on OrganAMNIST with 50% malicious clients). It also maintains high distributional fidelity, as shown by low Wasserstein distances ($W_2 \le 0.032$), confirming LDPP’s ability to preserve data utility. The method’s compatibility with existing FL frameworks and its low cost of integration make it a viable solution for real-world adoption, including in resource-constrained or privacy-sensitive environments. In the future, we plan to incorporate adaptive noise addition, conduct experiments against more advanced attacks, and combine LDPP with advanced techniques, such as homomorphic encryption. This will lead to more secure and reliable federated learning frameworks in real-world applications.

REFERENCES

[1] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in *Artificial intelligence and statistics*. PMLR, 2017, pp. 1273–1282.

[2] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” *IEEE signal processing magazine*, vol. 37, no. 3, pp. 50–60, 2020.

[3] M. Nasr, R. Shokri, and A. Houmansadr, “Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning,” in *2019 IEEE symposium on security and privacy (SP)*. IEEE, 2019, pp. 739–753.

[4] N. M. Jebreel, J. Domingo-Ferrer, D. Sánchez, and A. Blanco-Justicia, “Defending against the label-flipping attack in federated learning,” *arXiv preprint arXiv:2207.01982*, 2022.

[5] C. Ma, J. Li, M. Ding, H. H. Yang, F. Shu, T. Q. Quek, and H. V. Poor, “On safeguarding privacy and security in the framework of federated learning,” *IEEE network*, vol. 34, no. 4, pp. 242–248, 2020.

[6] A. Majeed and S. Lee, “Anonymization techniques for privacy preserving data publishing: A comprehensive survey,” *IEEE access*, vol. 9, pp. 8512–8545, 2020.

[7] O. Choudhury, A. Gkoulalas-Divanis, T. Salonidis, I. Sylla, Y. Park, G. Hsu, and A. Das, “Anonymizing data for privacy-preserving federated learning,” *arXiv preprint arXiv:2002.09096*, 2020.

[8] J. Chen, H. Yan, Z. Liu, M. Zhang, H. Xiong, and S. Yu, “When federated learning meets privacy-preserving computation,” *ACM Computing Surveys*, 2024.

[9] Q. Xie, S. Jiang, L. Jiang, Y. Huang, Z. Zhao, S. Khan, W. Dai, Z. Liu, and K. Wu, “Efficiency optimization techniques in privacy-preserving federated learning with homomorphic encryption: A brief survey,” *IEEE Internet of Things Journal*, vol. 11, no. 14, pp. 24 569–24 580, 2024.

[10] M. Rahaman, V. Arya, S. M. Orozco, and P. Pappachan, “Secure multi-party computation (smpc) protocols and privacy,” in *Innovations in Modern Cryptography*. IGI Global, 2024, pp. 190–214.

[11] C. Dwork, “Differential privacy,” in *International colloquium on automata, languages, and programming*. Springer, 2006, pp. 1–12.

[12] X. Ren, S. Yang, C. Zhao, J. McCann, and Z. Xu, “Belt and brace: When federated learning meets differential privacy,” *arXiv preprint arXiv:2404.18814*, 2024.

[13] T. H. Rafi, F. A. Noor, T. Hussain, and D.-K. Chae, “Fairness and privacy preserving in federated learning: A survey,” *Information Fusion*, vol. 105, p. 102198, 2024.
[14] G. K. Jagarlamudi, A. Yazdinejad, R. M. Parizi, and S. Pouriyeh, "Exploring privacy measurement in federated learning," *The Journal of Supercomputing*, vol. 80, no. 8, pp. 10 511–10 551, 2024.

[15] X. Pan, M. Zhang, D. Wu, Q. Xiao, S. Ji, and Z. Yang, "Justinian's {GAAvernon}: Robust distributed learning with gradient aggregation agent," in *29th USENIX Security Symposium (USENIX Security 20)*, 2020, pp. 1641–1658.

[16] M. Grama, M. Musat, L. Muñoz-González, J. Passerat-Palmbach, D. Rueckert, and A. Alansary, "Robust aggregation for adaptive privacy preserving federated learning in healthcare," *arXiv preprint arXiv:2009.08294*, 2020.

[17] M. Naseri, J. Hayes, and E. De Cristofaro, "Local and central differential privacy for robustness and privacy in federated learning," *arXiv preprint arXiv:2009.03561*, 2020.

[18] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, "Membership inference attacks against machine learning models," in *2017 IEEE symposium on security and privacy (SP)*. IEEE, 2017, pp. 3–18.

[19] D. Li, W. E. Wong, W. Wang, Y. Yao, and M. Chau, "Detection and mitigation of label-flipping attacks in federated learning systems with kpc and k-means," in *2021 8th International Conference on Dependable Systems and Their Applications (DSA)*. IEEE, 2021, pp. 551–559.

[20] Z. Li, T. Lin, X. Shang, and C. Wu, "Revisiting weighted aggregation in federated learning with neural networks," in *International Conference on Machine Learning*. PMLR, 2023, pp. 19 767–19 788.

[21] P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, "Machine learning with adversaries: Byzantine tolerant gradient descent," *Advances in neural information processing systems*, vol. 30, 2017.

[22] T. Wang, Z. Zheng, and F. Lin, "Federated learning framework based on trimmed mean aggregation rules," *Available at SSRN 4181353*, 2022.

[23] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, "Practical secure aggregation for privacy-preserving machine learning," in *proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security*, 2017, pp. 1175–1191.

[24] S. Truex, L. Liu, K.-H. Chow, M. E. Gursoy, and W. Wei, "Ldp-fed: Federated learning with local differential privacy," in *Proceedings of the Third ACM International Workshop on Edge Systems, Analytics and Networking*, 2020, pp. 61–66.

[25] Y. Wang, Y. Tong, and D. Shi, "Federated latent dirichlet allocation: A local differential privacy based framework," in *Proceedings of the AAAI Conference on Artificial Intelligence*, vol. 34, no. 04, 2020, pp. 6283–6290.

[26] M. Kim, O. Günlü, and R. F. Schaefer, "Federated learning with local differential privacy: Trade-offs between privacy, utility, and communication," in *ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*. IEEE, 2021, pp. 2650–2654.

[27] A. Bhowmick, J. Duchi, J. Freudiger, G. Kapoor, and R. Rogers, "Protection against reconstruction and its applications in private federated learning," *arXiv preprint arXiv:1812.00984*, 2018.

[28] A. Cheu, A. Smith, and J. Ullman, "Manipulation attacks in local differential privacy," in *2021 IEEE Symposium on Security and Privacy (SP)*. IEEE, 2021, pp. 883–900.

[29] X. Cao, J. Jia, and N. Z. Gong, "Data poisoning attacks to local differential privacy protocols," in *30th USENIX Security Symposium (USENIX Security 21)*, 2021, pp. 947–964.

[30] L. Miao, W. Yang, R. Hu, L. Li, and L. Huang, "Against backdoor attacks in federated learning with differential privacy," in *ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*. IEEE, 2022, pp. 2999–3003.

[31] T. Nguyen, P. Lai, K. Tran, N. Phan, and M. T. Thai, "Active membership inference attack under local differential privacy in federated learning," *arXiv preprint arXiv:2302.12685*, 2023.

[32] D. Bernau, J. Robl, P. W. Grassal, S. Schneider, and F. Kerschbaum, "Comparing local and central differential privacy using membership inference attacks," in *IFIP Annual Conference on Data and Applications Security and Privacy*. Springer, 2021, pp. 22–42.

[33] T. Ha, T. Vo, T. K. Dang, and N. T. H. Trang, "Differential privacy under membership inference attacks," in *International Conference on Future Data and Security Engineering*. Springer, 2023, pp. 255–269.

[34] Y. Mao, X. Yuan, X. Zhao, and S. Zhong, "Romoa: Ro bust mo del a ggregation for the resistance of federated learning to model poisoning attacks," in *Computer Security–ESORICS 2021: 26th European Sympo-

sium on Research in Computer Security, Darmstadt, Germany, October 4–8, 2021, Proceedings, Part 1 26. Springer, 2021, pp. 476–496.

[35] Y. Xie, W. Zhang, R. Pi, F. Wu, Q. Chen, X. Xie, and S. Kim, "Robust federated learning against both data heterogeneity and poisoning attack via aggregation optimization," *arXiv preprint arXiv:2211.05554*, 2022.

[36] N. Rodríguez-Barroso, E. Martínez-Cámara, M. V. Luzón, and F. Herera, "Backdoor attacks-resilient aggregation based on robust filtering of outliers in federated learning for image classification," *Knowledge-Based Systems*, vol. 245, p. 108588, 2022.

[37] H. Zhu and Q. Ling, "Bridging differential privacy and byzantine-robustness via model aggregation," *arXiv preprint arXiv:2205.00107*, 2022.

[38] N. Gai, K. Xue, B. Zhu, J. Yang, J. Liu, and D. He, "An efficient data aggregation scheme with local differential privacy in smart grid," *Digital Communications and Networks*, vol. 8, no. 3, pp. 333–342, 2022.

[39] Y. Lu and L. Fan, "An efficient and robust aggregation algorithm for learning federated cnn," in *Proceedings of the 2020 3rd International Conference on Signal Processing and Machine Learning*, 2020, pp. 1–7.

[40] X. Bi and X. Shen, "Distribution-invariant differential privacy," *Journal of econometrics*, vol. 235, no. 2, pp. 444–453, 2023.

[41] D. Chafai, "Wasserstein distance between two gaussians," *Website*, 2010.

[42] G. Cohen, S. Afshar, J. Tapson, and A. Van Schaik, "Emnist: Extending mnist to handwritten letters," in *2017 international joint conference on neural networks (IJCNN)*. IEEE, 2017, pp. 2921–2926.

[43] J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni, "Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification," *Scientific Data*, vol. 10, no. 1, p. 41, 2023.

[44] F. MH and A. SERTBAŞ, "A differentially private federated learning application in privacy-preserving medical imaging," 2024, preprint available at https://www.researchsquare.com/article/rs-3873379/v2.

[45] M. Zhang, K. Sapra, S. Fidler, S. Yeung, and J. M. Alvarez, "Personalized federated learning with first order model optimization," *arXiv preprint arXiv:2012.08565*, 2020.

[46] F. A. Hölzl, D. Rueckert, and G. Kaissis, "Bridging the gap: Differentially private equivariant deep learning for medical image analysis," *arXiv preprint arXiv:2209.04338*, 2022.

[47] G. Baruch, M. Baruch, and Y. Goldberg, "A little is enough: Circumventing defenses for distributed learning," *Advances in Neural Information Processing Systems*, vol. 32, 2019.

[48] C. Fung, C. J. Yoon, and I. Beschastnikh, "Mitigating sybils in federated learning poisoning," *arXiv preprint arXiv:1808.04866*, 2018.

## ACKNOWLEDGMENTS

This work was supported by the French National Research Agency (Agence Nationale de la Recherche - ANR) through the project "PriFeDAP" (ANR-24-CE23-1088).

## APPENDIX

This appendix provides supplementary material and additional details that support the main findings presented in this paper.

### A. $\epsilon$-Differential Privacy for LDPP-KRUM

We will prove that the procedure of applying a quantile transformer to transform data to a uniform or normal distribution, adding Laplace noise, and then reversing the transformation satisfies the conditions of $\epsilon$-differential privacy.

1) Quantile Transformation.: The quantile transformer ensures that any data is mapped to a predefined distribution, such as a uniform or normal distribution. This transformation is deterministic and does not introduce any noise, but simply changes the representation of the data while preserving its order (rank).

Let $\mathcal{D}$ represent the original dataset, and $T(\mathcal{D})$ represent the transformed dataset after applying the quantile transformation
(uniform or normal). The key aspect of this transformation is that the ranks of the data points remain the same. Thus, this operation is rank-preserving and does not affect the sensitivity of the data. Therefore, $T(D)$ has the same sensitivity as $D$.

2) Adding Laplace Noise to Ensure $\epsilon$-Differential Privacy.: Next, we add Laplace noise to the transformed dataset $T(D)$. The addition of Laplace noise is a standard method to ensure $\epsilon$-differential privacy.

Given that the sensitivity of $T(D)$ is 1 (as rank-preserving transformations do not change sensitivity), we add Laplace noise with scale parameter $b = \frac{\Delta f}{\epsilon}$, where $\Delta f$ is the sensitivity of the function and $\epsilon$ is the privacy budget. Since $T(D)$ is rank-preserving, the sensitivity of $T(D)$ is equivalent to the sensitivity of the original dataset $D$, i.e., $\Delta f = 1$.

Thus, the noise added follows the Laplace distribution:

$$ \text{Noise} \sim \text{Laplace}(0, \frac{1}{\epsilon}) $$

This ensures that the perturbed dataset $D' = T(D) + \text{Noise}$ satisfies $\epsilon$-differential privacy.

3) Reverse Quantile Transformation.: After adding noise, the final step is to reverse the quantile transformation to bring the data back to its original distribution. This reverse transformation is also deterministic and does not introduce any additional noise. Since it is applied to data that already satisfies $\epsilon$-differential privacy (due to the Laplace noise), and the transformation is invertible, this step preserves the differential privacy guarantees.

4) Formal Proof.: Now, we can formalize the proof that this process satisfies $\epsilon$-differential privacy.

a) Definition of $\epsilon$-Differential Privacy.: A mechanism $\mathcal{M}$ satisfies $\epsilon$-differential privacy if for any two adjacent datasets $D_1$ and $D_2$ differing by at most one element, and for any output $O$ of the mechanism:

$$ \Pr[\mathcal{M}(D_1) = O] \le e^{\epsilon} \cdot \Pr[\mathcal{M}(D_2) = O] $$

5) Step-by-Step Proof.: Let $\mathcal{M}$ be the overall mechanism composed of three steps: the quantile transformation, adding Laplace noise, and reversing the quantile transformation.

1) **Quantile Transformation:** This is a rank-preserving transformation, so it does not change the sensitivity of the data. Thus, the differential privacy condition is unchanged after this step. If the original data had sensitivity $\Delta f$, the transformed data $T(D)$ still has sensitivity $\Delta f = 1$.

2) **Adding Laplace Noise:** The Laplace mechanism ensures that the transformed dataset $T(D)$ satisfies $\epsilon$-differential privacy by adding noise proportional to the sensitivity of the dataset. The sensitivity is $\Delta f = 1$, so the noise added ensures $\epsilon$-differential privacy.

3) **Reverse Quantile Transformation:** This is an invertible, deterministic transformation that does not add any noise or change the sensitivity of the data. Therefore, the $\epsilon$-differential privacy guarantee is preserved.

Thus, the overall mechanism $\mathcal{M}$ satisfies $\epsilon$-differential privacy.

## B. LDPP-KRUM's algorithm summary

To provide a clear and concise understanding of our LDPP-KRUM, we summarize the steps in the following algorithm:

<caption>**Algorithm 1** Local Differential Privacy Preprocessing with Robust Krum Aggregation</caption>

1: **Input:** Local Datasets: $\{D_i\}_{i=1}^N$, Initial Global Model Parameters: $\theta_0$, Number of Rounds: $R$, Transformation Type: $T_{type}$, Noise Type: $N_{type}$, Krum Aggregation Parameters
2: **Output:** Final model parameters $\theta^*$
3: Initialize Model: $\theta \leftarrow \theta_0$
4: **for** $r = 1$ **to** $R$ **do**
5: &nbsp;&nbsp;&nbsp;&nbsp;**for** $i = 1$ **to** $N$ **do**
6: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**if** $T_{type}$ is Normal **then**
7: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Apply Normal Transformation: $x_{trans} = \frac{x-\mu_x}{\sigma_x}$
8: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**else**
9: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Apply Uniform Transformation: $x_{trans} =$
10: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;$\frac{x-x_{min}}{x_{max}-x_{min}}$
10: &nbsp;&nbsp;&nbsp;&nbsp;**end if**
11: &nbsp;&nbsp;&nbsp;&nbsp;**if** $N_{type}$ is Laplace **then**
12: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add Laplace Noise: $x_{priv} = x_{trans} + \eta$, $\eta \sim$ Laplace(0, $b$)
13: &nbsp;&nbsp;&nbsp;&nbsp;**else**
14: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add Gaussian Noise: $x_{priv} = x_{trans} + \eta$, $\eta \sim$ $\mathcal{N}(0, \sigma^2)$
15: &nbsp;&nbsp;&nbsp;&nbsp;**end if**
16: &nbsp;&nbsp;&nbsp;&nbsp;**if** $T_{type}$ is Normal **then**
17: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reverse Normal Transformation: $x_{reversed} =$
18: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;$x_{priv} \times \sigma_x + \mu_x$
18: &nbsp;&nbsp;&nbsp;&nbsp;**else**
19: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reverse Uniform Transformation: $x_{reversed} =$
20: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;$x_{priv} \times (x_{max} - x_{min}) + x_{min}$
20: &nbsp;&nbsp;&nbsp;&nbsp;**end if**
21: &nbsp;&nbsp;&nbsp;&nbsp;Train local model on $D_i$ to obtain $\Delta\theta_i$
22: &nbsp;&nbsp;&nbsp;&nbsp;**end for**
23: &nbsp;&nbsp;&nbsp;&nbsp;Collect local updates $\{\Delta\theta_i\}_{i=1}^N$
24: &nbsp;&nbsp;&nbsp;&nbsp;Compute pairwise distances: $d_{ij} = \|\Delta\theta_i - \Delta\theta_j\|$

25: Select robust update using Krum criterion: $\hat{\theta} =$
26: Update global model parameters: $\theta \leftarrow \theta + \hat{\theta}$
27: **end for**
28: **Output:** Final model parameters $\theta^*$