# Unveiling the (in) security of threshold FHE-based federated learning: the practical impact of recent CPA D attacks

Adda-Akram Bendoukha, Renaud Sirdey, Aymen Boudguiga, Nesrine Kaaniche

► To cite this version:

Adda-Akram Bendoukha, Renaud Sirdey, Aymen Boudguiga, Nesrine Kaaniche. Unveiling the (in) security of threshold FHE-based federated learning: the practical impact of recent CPA D attacks. 2025 IEEE 38th Computer Security Foundations Symposium (CSF), Jun 2025, Santa cruz, United States. pp.378-393. hal-05105133

HAL Id: hal-05105133

https://hal.science/hal-05105133v1

Submitted on 10 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Unveiling the (in)Security of Threshold FHE-based Federated Learning: The Practical Impact of Recent CPA<sup>D</sup> Attacks

Adda-Akram Bendoukha

Renaud Sirdey

Samovar, Télécom SudParis, Institut Polytechnique de Paris

Université Paris-Saclay, CEA-LIST

adda-akram.bendoukha@telecom-sudparis.eu

renaud.sirdey@cea.fr

Aymen Boudguiga

Nesrine Kaaniche

Université Paris-Saclay, CEA-LIST

Samovar, Télécom SudParis, Institut Polytechnique de Paris

aymen.boudguiga@cea.fr

kaaniche.nesrine@telecom-sudparis.eu

**Abstract**—The security of Fully Homomorphic Encryption (FHE) has received a lot of attention in recent years with new security notions emerging to better understand the practical attacks that may threaten the real-world deployments of passively secure FHE schemes. One such new notions is CPA<sup>D</sup> a slight extension of CPA security modelling a *passive adversary who is granted access to a decryption oracle accepting only well-formed ciphertexts*. While successful CPA<sup>D</sup> attacks have initially been performed on approximate FHE schemes such as CKKS, recent works have also demonstrated practical CPA<sup>D</sup> attacks on all mainstream non-approximate FHE, such as BFV, BGV or TFHE. Despite their clear computational practicality, these latter attacks however focus on the abstract security game defining CPA<sup>D</sup> security. In this paper, we show how to concretely build on these to mount successful FHE key recovery attacks in the Federated Learning (FL) setting, an application scenario of choice for FHE techniques. In FL, participating entities or *workers* encrypt successive model updates based on their local training data, enabling a central server to aggregate them in order to homomorphically update a global model. As this paper demonstrates, this environment provides a playground for an attacker to launch key recovery attacks against the FHE underlying the secure aggregation mechanism. As such, our findings reveal substantial stealthy key-recovery threats from both the server and a single worker, with very limited impact on the FL training progression or final model quality.

## I. INTRODUCTION

During the last decade, Fully Homomorphic Encryption (FHE) has emerged as a promising cryptographic primitive for building privacy-preserving applications. Unlike traditional encryption methods, FHE allows to perform computations directly on encrypted data. That is, it is well-fitted for scenarios where sensitive information must be outsourced and analyzed in multiparty settings, such as secure cloud computing, medical data analysis, encrypted traffic analysis [1, 2, 3, 4], and recently in collaborative machine learning, such as Federated Learning (FL) [5, 6].

FL enables the collaborative training of a global machine-learning model by a pool of workers (or clients), driven by a central server. In a nutshell, starting from an initial common

model, at each round of the protocol, the workers use their local training data for generating model updates, which are sent back to the server and aggregated, resulting in a new model that forms the next round's input. At the end of the process, the resulting model is expected to have a quality relatively similar to that of a hypothetical model centrally trained on the union of the local datasets. However, the updates sent to the server during the protocol leak information on the local datasets and have been demonstrated to enable a wide range of white-box privacy attacks on these datasets [7, 8]. Consequently, in realistic settings, FL is systematically instantiated with a mechanism to preserve the confidentiality of a given worker's updates from other entities, while maintaining the aggregation capabilities of the central server. In recent years, a number of works have investigated the use of FHE to achieve that goal [9, 3, 10, 11].

From a security perspective, however, FHE still raises a number of questions and challenges. In particular, all the FHE usable in practice, BFV [12, 13], BGV [14], CKKS [15] and TFHE [16], achieve only CPA-security *i.e.* security against passive adversaries. Although homomorphism is mutually exclusive with CCA2-security due to the malleability it induces, building efficient FHE constructions achieving some degree of CCA security (*e.g.*, CCA1¹) remains a very important open challenge despite recent theoretical advances [18, 19]. Being stuck at the CPA level, FHE is thus completely insecure (and trivially so) as soon as the adversary is granted access to a decryption oracle. In this context, Li and Micciancio [20] were the first to study the security of FHE against a slight, seemingly benign extension of CPA security where the adversary is granted access only to a highly constrained decryption oracle

<footnote n="1">¹Existing CPA secure FHE are all trivially breakable by CCA1 adversaries (as there exists simple folklore key recovery attacks). Because it is difficult to rule out the existence of (possibly imperfect) decryption oracles in real-world setups (think for example of a client betraying decryption errors by retries[17]), CCA1 security for FHE is desirable (although still an open-problem from a practical viewpoint).</footnote>
that accepts only genuine ciphertexts or ciphertexts derived
from genuine ciphertexts by means of genuine homomorphic
operations. The initial intuition is that, by knowing the clear-
text inputs of an FHE calculation, the adversary should be
able to compute all the outputs of that decryption oracle by
his or herself and that, as a consequence, CPA<sup>D</sup> security is
implied by or even equivalent to CPA security. However, Li
and Micciancio [20] demonstrated that these intuitions are
not true for approximate FHE schemes such as CKKS for
which it turns out that the CPA<sup>D</sup> decryption oracle outputs
leak the LWE noises in the ciphertexts, resulting in the ability
for the adversary to easily and practically recover the secret
decryption key of the scheme. Although initially introduced
for approximate FHE, recent works [21, 22] have shown
that the non-approximate FHE schemes that were previously
considered immune to CPA<sup>D</sup> attacks (namely BFV, BGV and
TFHE) are also CPA<sup>D</sup> insecure as soon as decryption errors
can or can be made to occur with a non-negligible probability
resulting in practical key recovery attacks on the mainstream
libraries implementing them.

In this context, natural questions to explore include the
following. *Do CPA<sup>D</sup> decryption oracles naturally occur in realistic FL settings involving FHE?* If so, *can some of the recent attacks framed against the abstract CPA<sup>D</sup> security game lead to meaningful practical attacks in these FL settings?* In this paper, we provide positive answers to both these questions by revealing several naturally occurring CPA<sup>D</sup> oracles and exploiting them, particularly in the threshold FHE setting, to allow an adversary (which may be the aggregation server or a single worker) to practically recover the (global) secret key of the FHE scheme. Our attacks are applicable to both the BFV scheme, as described in this paper, and the BGV scheme. Beyond these two FHE schemes, only two other candidates exist: CKKS, which is entirely compromised in the context of the original Li & Micciancio work [23], and TFHE, which is less suitable for federated learning due to its lack of batching capabilities, leading to performance limitations.

We furthermore show experimentally that these attacks are
stealthy in that they lead to very limited impact on the FL
training progression or final model quality.

Note that a passive adversary against a cryptographic
scheme is one that *manipulates only well-formed ciphertexts*.
In that sense, a CPA<sup>*D*</sup> adversary against a given FHE scheme
is passive. However, a passive adversary against a protocol is
one which *does not deviate from the protocol specification*.

**Contributions:** The contributions of this paper are summarized as follows:

1) We define two key recovery attacks for FHE-based aggregation when the adversary is either (1) the aggregation server or (2) a single worker.

2) We analyze the robustness and effectiveness of the introduced attacks in the FL setting with respect to the

model architecture, the quantization <fnref n="2" /> of its parameters,
and the heterogeneity of workers' data. In particular, we
demonstrate that a CPA<sup>D</sup> adversarial worker only needs
a slightly larger plaintext modulus, relative to the quanti-
zation precision used for encoding model parameters, to
recover all the RLWE noise polynomial coefficients.

3) We empirically analyze the impact of the model perturbations used to perform the attacks on the FL workflow. We show that these perturbations have a limited effect on the progress of the FL training, making these attacks stealthy if integrated carefully into the FL workflow. In particular, our experiments show that, from a convergence viewpoint, the FL protocol only requires 2-3 iterations to recover from such a perturbation. In addition, inserting the perturbation in the parameters of a final batch normalization layer induces a limited convergence degradation (5%) at the early stages of the FL process.

4) Finally, we discuss possible countermeasures to the proposed attacks, such as differential-privacy, and byzantine-resilient aggregation strategies. In particular, a CPA<sup>D</sup> attack vector is a perturbed update transmitted by an adversarial worker that exhibits statistical similarities to a *byzantine* update aiming to pull the average gradients or parameters' vector towards an adversarial direction to prevent convergence. We experiment with the ability of the *coordinate-wise trimmed-mean* (CWTM) [24], and the *Krum* [25] aggregations to discard a CPA<sup>D</sup> perturbed gradient update from the aggregation process. Our experiments demonstrate the ability of both approaches to eliminate an update carrying the attack vector, hence, preventing the CPA<sup>D</sup> key-recovery attack with an adversarial worker.

5) Our open-sourced experiments are available at https://github.com/Akram275/CPA-D-vulnerabilities-in-FL

**Paper Organization:** This paper is organized as follows. Section II provides some background on FHE and CPA<sup>D</sup> security. Section III provides the necessary background on Federated Learning, mainly from a privacy perspective. Section V describes two novel CPA<sup>D</sup> attacks on threshold FHE schemes when used in an FL setting: with the aggregation server acting as the adversary in one case and a single worker acting as such in the other. Subsequently, section VI investigates potential countermeasures. Finally, Section VII discusses our experimental results, and section VIII concludes this work.

**Notations:** Throughout the paper, we consistently use $\mathbb{Z}_q$ to denote $\mathbb{Z}/q\mathbb{Z}$. FL iteration indices are denoted by $r$, and $T$ denotes the last FL iteration. $p$ and $q$ are the plaintext, and ciphertext moduli in FHE schemes, and $t$ indicates a threshold value for collaborative decryption. $[k]$ denotes the integers set $\{1, \dots, k\}$.

<footnote n="2">²In FL with FHE for secure aggregation, model weights must be encoded to integers, as BFV, BGV and TFHE schemes operate in modular arithmetic. The resulting precision reduction balances efficiency of the homomorphic operations and the model's utility.</footnote>
## II. ON THE SECURITY OF FHE ALGORITHMS

Since Gentry's breakthrough, the FHE state-of-the-art has now stabilized around a few main schemes: BGV, BFV, CKKS, FHEW and TFHE [26, 27, 28, 29, 30]. From a security perspective, they all rely on the Learning With Errors problem (LWE) [31]. These schemes are framed as follows:

Let $\mathcal{E} = (\text{KeyGen}, \text{Enc}, \text{Dec}, \text{Eval})$ denote a public key homomorphic encryption scheme defined by the following set of PPT algorithms, given security parameter $\lambda$:

* $(\text{pk}, \text{evk}, \text{sk}) \leftarrow \text{KeyGen}(1^{\lambda})$: outputs an encryption key $\text{pk}$, a public evaluation key $\text{evk}$ and a secret decryption key $\text{sk}$. The evaluation key is used during homomorphic operations. $\text{evk}$ corresponds to a relinearization key for a levelled or somewhat scheme (such as BFV [26]) or to a bootstrapping key when schemes such as TFHE [30] are used.

* $c \leftarrow \text{Enc}(m; \text{pk})$: turns a message $m$ into a ciphertext $c$ using the public key $\text{pk}$.

* $m \leftarrow \text{Dec}(c; \text{sk})$: turns a message $c$ into a plaintext $m$ using the secret key $\text{sk}$.

* $c_f \leftarrow \text{Eval}(f, c_1, \dots, c_k, \text{evk})$: evaluates the function $f$ on the encrypted inputs $(c_1, \dots, c_k)$ using the evaluation key $\text{evk}$. In the general case where non-exact FHE schemes are allowed, Eval should be such that with high-enough $(\lambda$-independent) probability:

$$ |\text{Dec}(\text{Eval}(f, \text{Enc}(m_1), \dots, \text{Enc}(m_K))) - f(m_1, \dots, m_K)| \le \varepsilon \quad (1) $$

Note that all FHE schemes used in practice, and in particular all the above, achieve only CPA-security.

### A. The Regev encryption scheme

In this section, we briefly present the (additively homomorphic) vanilla Regev cryptosystem [31]. Because this scheme provides the foundational basis for all aforementioned FHE schemes (and because only homomorphic additions turns out to be used in this paper) it will suffice for our purpose$^3$. Also, without loss of generality, we present only the symmetric variant. The scheme is thus parameterized by a dimension $n$, a plaintext modulus $p$, a ciphertext modulus $q$, and a (discrete) Gaussian noise distribution of variance $\sigma^2$. The scheme is then specified by the three following algorithms:

* **KeyGen(1)$^{\lambda}$**: This algorithm uniformly samples an $n$-dimensional vector $s$ from $\mathbb{Z}_q^n$ to derive the secret-key $\text{sk}$.

* **Enc($m, \text{pk}$)**: Given a message $m \in \mathbb{Z}_p$, this algorithm uniformly samples an $n$-dimensional vector $a$ from $\mathbb{Z}_q^n$, and a noise term $e$ from $\mathbb{Z}_q$ following the (discrete) Gaussian distribution $N(0, \sigma^2)$. It returns $(a, b = \langle a, s \rangle + \lfloor \frac{q}{p} \rfloor m + e \mod q)$.

* **Dec($c, \text{sk}$)**: Given a ciphertext $(a, b)$, and the secret key $\text{sk}$, returns $\lfloor \frac{p}{q} \rfloor (b - \langle a, s \rangle) \mod p$. The decryption is correct whenever $|e| < \frac{q}{2p}$.

In practice, the aforementioned schemes are rather based on the ring variant of LWE (RLWE) which operates over $R_q = \mathbb{Z}_q[X]/(X^N + 1)$ [33]. This essentially allows to (nega-cyclically) reuse a given $a$ up to $N$ times and, thus, to pack $N$ messages into a single ciphertext. This leads to better communication efficiency and enables batching i.e. SIMD homomorphic operations, greatly improving efficiency. Note that we only provide the minimum details required to understand the present paper (in particular we do not cover multiplication operators) and refer the reader to the cited references for the full specifications of the FHE schemes.

Table I gives an overview of the features of the mainstream FHE schemes.

### B. Threshold Fully Homomorphic Encryption

Several recent works [35, 36, 37] consider a threshold setting for FHE where each participant holds a share $\text{sk}_i$ of a (global) secret key $\text{sk}_g$. Such a share grants him the ability to perform a partial decryption of a ciphertext $\hat{m}_i = \text{Part.Decrypt}(\text{sk}_i, c)$. In a $t$-out-of-$n$ threshold encryption, only $t$ participants collaborate to decrypt a ciphertext.

The fundamental security assumption for threshold schemes is that the set of partial decryptions produced by any subset of size $(t-1)$ provides no additional knowledge regarding $m$ beyond what is already provided by the corresponding ciphertext $c$.

A threshold FHE scheme, t-FHE, is defined as follows:

* $(\text{sk}_1, \text{sk}_2, \dots, \text{sk}_n, \text{pk}, \text{evk}) \leftarrow \text{KeyGen}(\lambda, n, t)$: given a security parameter $\lambda$, a number of parties $n$, and a threshold value $t$, this algorithm generates a public key, $n$ shares of the secret-key and an evaluation key.

* $c \leftarrow \text{Encrypt}(m, \text{pk})$: Encrypts $m$ using the public key $\text{pk}$.

* $c_f \leftarrow \text{Evaluate}(f, c_1, \dots, c_k, \text{evk})$: computes function $f$ on the ciphertexts $(c_1, \dots, c_k)$.

* $\hat{m}_i \leftarrow \text{Part.Decrypt}(c, \text{sk}_i)$: given a ciphertext $c$ and a share of the secret key, outputs a partial decryption of $c$ using $\text{sk}_i$.

* $m \leftarrow \text{Combine}(\hat{m}_1, \dots, \hat{m}_t)$: given at least $t$ partial decryptions of a ciphertext $c$, outputs a full decryption.

Recently, several works have defined threshold versions of existing homomorphic encryption schemes. For instance, Mouchet et al. [38] specified an additive$^4$ $n$-out-of-$n$ multiparty construction for RLWE based schemes (BGV, BFV, and CKKS). It was later extended to $t$-out-of-$n$ access structures for any $t \le n$ using the share-re-sharing technique by Asharov et al. [39]. To do so, it turns an $n$-out-of-$n$ construction to a $t$-out-of-$t$ one, by re-sharing the additive local secret keys which are generated with the well-known Shamir secret sharing technique [40]:

$$ \text{sk}_g = \sum_{i=1}^{n} \text{sk}_i = \sum_{i=1}^{n} \sum_{j=1}^{t} S_i(\alpha_j) \lambda_j = \sum_{j=1}^{t} \lambda_j \sum_{i=1}^{n} S_i(\alpha_j) = \sum_{j=1}^{t} \text{sk}'_j $$

<footnote>$^3$(Threshold) FHE in FL offers two key advantages over additive-only schemes: (1) it supports advanced aggregation strategies, such as median-based and fairness-aware FL, via homomorphic addition and multiplication [32, 11], and (2) RLWE-based schemes enable efficient batching, improving amortized homomorphic efficiency through SIMD-style processing over $N$ slots.</footnote>

<footnote>$^4$That is, $\text{sk}_g = \sum_{i \in [n]} \text{sk}_i$, and the collaborative decryption consists in every share-holder subtracting its $\langle a, \text{sk}_i \rangle$ term from $b = \langle a, \sum_{i \in [n]} \text{sk}_i \rangle + m + e$.</footnote>
<table><caption>TABLE I: Overview of FHE schemes functionalities.</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="4">FHE scheme</th></tr><tr><th>BFV [26]</th><th>BGV [27]</th><th>CKKS [28]</th><th>TFHE [30]</th></tr></thead><tbody><tr><td rowspan="3">Plaintext</td><td>Integer Z<sub>p</sub></td><td>✓</td><td>✓</td><td>✓</td><td>✓(Torus representation)</td></tr><tr><td>Polynomial Z<sub>p</sub>[X]</td><td>✓</td><td>✓</td><td>✓</td><td>✓(Torus representation)</td></tr><tr><td>Complex C<sup>N/2</sup></td><td>x</td><td>x</td><td>✓</td><td>x</td></tr><tr><td rowspan="3">Ciphertext</td><td>LWE</td><td>x</td><td>x</td><td>x</td><td>✓(Torus variant)</td></tr><tr><td>RLWE</td><td>✓</td><td>✓</td><td>✓</td><td>✓(Torus variant)</td></tr><tr><td>GSW [34]</td><td>x</td><td>x</td><td>x</td><td>✓</td></tr><tr><td rowspan="2">Multiplication</td><td>Ciphertext expansion</td><td>Relin and ModSwitch</td><td>Relin and ModSwitch</td><td>Relin and ModSwitch</td><td>GadgetDecomposing and KeySwitch</td></tr><tr><td>Noise-growth</td><td>Rescaling</td><td>Rescaling</td><td>Rescaling</td><td>Bootstrapping</td></tr></tbody></table>

where $S_i$ denotes the Shamir polynomial of each participant, $\alpha_j$ their Shamir public points, and $\lambda_j$ their Lagrange coefficient. Essentially, this approach allows the reconstruction of t additive (sum to the global secret key) keys ($sk'_j$)$_{i \in [t]}$ from the n initial keys and their shares and uses the additive collaborative decryption similarly to the n-out-of-n setting.

### C. CPA$^D$ security

In [23], Li and Micciancio were the first to study the security of FHE against a slight, seemingly benign extension of CPA security where the adversary is granted access only to a highly constrained decryption oracle which accepts only genuine ciphertexts or ciphertexts derived from genuine ciphertexts by means of genuine homomorphic operations (as such the notion does not capture any CCA aspects). The intuition is that, given a FHE scheme $S = (\text{KeyGen}, \text{Enc}, \text{Dec}, \text{Eval})$, if the adversary knows $m, f$ as well as $c = \text{Enc}(m)$, granting her access to $\text{Dec}(\text{Eval}(f, c))$ should not raise any issue since she can compute $f(m)$ by herself and, by definition of FHE,

$$ \text{Dec}(\text{Eval}(f, \text{Enc}(m)), \text{sk}) = f(m) \quad (2) $$

is supposed to hold for all $m$ in the plaintext domain of $S$. At first glance, it appears that this constrained oracle does not provide more information to the adversary than she can compute on herself and, as such, that this CPA$^D$ security is implied by or even equivalent to CPA security. Unfortunately, Li and Micciancio demonstrated that these intuitions are not true for approximate FHE schemes such as CKKS, for which it turns out that neither $\text{Dec}(\text{Enc}(m)) = m$ nor (2) hold (with high probability) and where the differences $\text{Dec}(\text{Enc}(m)) - m$ or

$$ \text{Dec}(\text{Eval}(f, \text{Enc}(m)), \text{sk}) - f(m) $$

leak the LWE noises in the ciphertexts. Indeed, stated in LWE terms for simplicity sake, CKKS encryption (at least in its basic form) does not apply any scaling to the message i.e. generates ciphertext $(a, \langle a, s \rangle + m + e)$ rather than $(a, \langle a, s \rangle + \lfloor \frac{q}{2p} \rfloor m + e)$ in the other schemes. As a consequence, CKKS decryption function does not remove the noise term present in the ciphertexts and merely granting an adversary access to decryptions of fresh ciphertexts results in the ability for him or her to easily and practically recover the secret decryption key of the scheme from the knowledge of the noises in n LWE samples. They further demonstrated the attack practicality on

most mainstream libraries implementing CKKS. Since then, recent works have also demonstrated practical CPA$^D$ attacks on all mainstream non-approximate FHEs, such as BFV, BGV, or TFHE, by exploiting decryption errors [41, 42].

Collectively, these two latter works describe several complementary attack strategies but, in this paper, we only build upon the strategy described in [42] for BFV/BGV. In a nutshell, the attack consists for an adversary to start with a well-formed fresh RLWE encryption of the 0 polynomial and to recursively add it to itself until the noise is so amplified that it ends up in the message space. Then, the decryption of this new ciphertext gives away the noise polynomial in the initial encryption of 0 allowing to retrieve the secret key by means of standard linear algebra techniques. More precisely, under the assumption that $\Delta = q/p = 2^k$, they start by a ciphertext $c_0 = (a, a.s + e)$. They then proceed by building the sequence of ciphertexts $c_i = \text{Eval}(\text{sum}, c_{i-1}, c_{i-1})$ ($i > 0$), thus getting ciphertexts of the form $c_i = (a, a.s + 2^i e)$. When $i = k$, the decryption of $c_k = (a, a.s + 2^k e)$ directly yields $e$ (since decryption is $\lfloor \frac{1}{\Delta}(b - a \cdot s) \rfloor$ and $\lfloor 2^k e/\Delta \rfloor = e$) under the mild restriction that the plaintext modulus $p$ is such that $|e_i| < p$ for all $i$. When this occurs with large enough probability, they succeed in retrieving the whole polynomial $e$ and the secret key by just solving the following linear system over $\mathbb{Z}_q$:

$$ \begin{align} \sum_{j=0}^{i} a_{i-j}^{(1)} s_j - \sum_{j=i+1}^{n-1} a_{i+n-j}^{(1)} s_j &= b_i + e_i^{(1)} \tag{3} \\ \dots \nonumber \\ \sum_{j=0}^{i} a_{i-j}^{(n)} s_j - \sum_{j=i+1}^{n-1} a_{i+n-j}^{(n)} s_j &= b_i + e_i^{(n)} \nonumber \end{align} $$

Note that a single RLWE ciphertext is enough to obtain that system. Also, note that the above conditions are almost always met by usual parameters used for BFV/BGV which generally use the (small) value $\sigma = 3.2$ for the fresh ciphertext standard deviation, with $p$ and $q$ respectively over more than 10 and 100 bits. It should furthermore be clear that the above attack complies with the CPA$^D$ model as it starts from a well-formed ciphertext and finally asks for the decryption of a ciphertext derived from it by means of genuine homomorphic operations. For completion, the formal CPA$^D$ security game is given in Sect. A.
### III. FEDERATED LEARNING

Federated Learning (FL) is a distributed learning framework in which multiple entities, referred to as workers or clients, collaborate to train a shared model. Starting from an initial common model, each worker trains a local model on its respective dataset and then shares it (parameters or gradients) with an aggregation server. This server generates a unified model by aggregating either the gradients (FedSGD) or the weights (FedAVG) from the models of the workers [5].

In FedAVG, the server aggregates the local model weights received from $n$ participating workers using a weighted average. Each parameter update ($\theta_i$) of a worker $i$ is added in proportion to the size of the worker's dataset, denoted as $n_i$. Indeed, at each iteration $r$, the server updates the global model weight $\theta_g^r$ by computing: $\theta_g^{r+1} = \sum_{i=1}^n \frac{n_i}{\sum_{i=1}^n n_i} \theta_i^r$.

In FedSGD, the server aggregates the gradient vectors received from all the workers to get a global gradient vector. Then, workers update their local models using the global gradient vector and a learning rate $\eta$.

$$g^{r+1} = \sum_{i=1}^{n} \omega_i^r g_i^r \quad \text{and} \quad \theta_g^{r+1} = \theta_g^r - \eta g^{r+1}$$

This process requires several rounds to achieve high predictive performances over all workers' data. Several parameters impact the FL training convergence, such as number of involved workers [43], workers' data heterogeneity [44], and workers' dropouts [45].

1) **FHE as a secure aggregation mechanism:** Even though data are never shared directly with the server, through FL algorithms, workers' training data are still vulnerable to privacy inference attacks via their periodic updates [7, 8] by honest-but-curious servers. To address such threats, several secure aggregation methods were proposed in the state of the art [46] using different primitives such as FHE [3, 10], t-FHE [11], secure multiparty computation (MPC) [47], pairwise masking [48], or functional encryption [49]. They mainly aim at hiding the intermediary updates from unauthorized entities.

Thanks to its non-interactivity and the linearity of the aggregation methods, FHE is a good candidate for concealing model updates while enabling their secure aggregation in FL. Indeed, Sébert et al. [10] showed that FHE only adds a computation overhead of less than 1% to the training of a neural network of over 400k parameters, and up to 1000 workers with FedAVG while increasing the communication overhead by a factor 100, due to large FHE ciphertexts-expansion. In addition, Truex et al. [50] used FHE as an alternative to statistics-based approaches to avoid degrading model accuracy. Then, IBM-FL [51], and NVIDIA-FLARE [52] proposed practical implementations of FHE-based secure aggregation solutions to tackle the semi-honest aggregator issue. Jin et al. [53] proposed a selective homomorphic encryption mechanism that solely encrypts a subset of sensitive update parameters to further reduce bandwidth and computational overheads.

### Algorithm 1 FedAvg-based FL with t-FHE secure aggregation

**Require:** Number of workers $n$, epochs $E$, learning rate $\eta$.

**Ensure:** Global model $\theta_T$

1. **Server Initialization:**

2. Initialize global model $\theta^{\text{init}}$

3. **Workers Initialization:**

4. Collaboratively run ([38]) ($\{\mathbf{sk}_i\}_{i \in [n]}, \mathbf{pk}, \mathbf{evk}\} \leftarrow \text{KeyGen}(\lambda, n, t)$ Distribute public key to all workers

5. **for** each round $r = 1, 2, \dots, T$ **do**

6. **for** each worker $i \in [n]$ **in parallel do**

7. Download encrypted global model $\text{Enc}(\theta^r, \mathbf{pk})$

8. Broadcasts Part.Decrypt($\text{Enc}(\theta^r, \mathbf{pk}), \mathbf{sk}_i$) to workers

9. **Upon collecting at least $t$ partial decryptions** $S$

10. perform $\theta^r \leftarrow \text{Combine}(S)$ [Decryption oracle]

11. Perform $\theta_i^r \leftarrow \text{LocalUpdate}(\theta^r, E, \eta, i)$

12. transmit $\text{Enc}(\theta_r^t, \mathbf{pk})$ to server

13. **end for**

14. **Server Aggregation:**

15. Homomorphically compute and broadcast :

$$\text{Enc}(\theta^{r+1}, \mathbf{pk}) = \sum_{i \in [n]} \left( \frac{n_k}{\sum_{i \in [n]} n_i} \right) \cdot \text{Enc}(\theta_i^r, \mathbf{pk})$$

16. **end for**

17. **Return final model** $\theta^T$ [Decryption oracle]

### IV. OVERALL TRUST MODEL

We examine t-FHE-based FL (Algorithm 1) to assess concrete CPA$^D$ vulnerabilities, where adversaries seek to recover the global secret key $\mathbf{sk}_g$ and compromise privacy. Hereafter, we investigate how the entities involved in the formal CPA$^D$ game described in appendix A map to this setting, highlighting their roles and interactions within the protocol.

#### A. Adversaries

In cryptographic security analysis, adversaries are classified depending on the capabilities of the decryption oracle to which they have access. For example, CPA or CPA$^D$ adversaries are restricted to follow the scope of the encryption scheme $\mathcal{E}$, because they respectively have access to decryption oracles which accept either no ciphertext at all (CPA) or only well-formed ciphertexts (CPA$^D$). By contrast, a CCA adversary is granted access to a decryption oracle accepting arbitrary (possibly ill-formed) ciphertexts as input. It is well-known that the latter kind of adversary is much more powerful than the former. In terms of terminology, we will therefore say that a CPA or CPA$^D$ adversary operates under the well-formedness assumption, whereby the adversary is limited to exploit only legitimate ciphertexts (well-formed fresh ciphertexts or ciphertexts derived from fresh well-formed ciphertexts via legitimate homomorphic evaluations), whereas a CCA1 or CCA2 adversary operates under arbitrary ciphertexts. This terminology is consistent with recent papers focusing on clarifying the
relationships between CPA<sup>D</sup> and CCA security notions for FHE [54].

On the other hand, FL threat models follow the conventional protocol security notions, where protocol players can be honest, honest-but-curious, or malicious. Honest players faithfully follow the protocol workflow. Honest-but-curious adversaries attempt to infer private data without deviating from protocol specifications. Malicious adversaries actively seek to disrupt or corrupt the learning process. In the literature, honest-but-curious are sometimes referred to as passive adversaries, while malicious players are also sometimes referred as active adversaries against the protocol.

<table><caption>TABLE II: Combination of Cryptographic and FL Protocol Threat Models, $\mathcal{P}$ refers to the FL protocol, and $\mathcal{E}$ refers to the underlying FHE scheme.</caption><thead><tr><th>Crypto vs Prot. assump.</th><th>Honest-but-Curious</th><th>Malicious</th></tr></thead><tbody><tr><td rowspan="2">Well-formed ciphertexts</td><td>Within the scope of <i>P</i></td><td>Outside the scope of <i>P</i></td></tr><tr><td>Within the scope of <i>E</i></td><td>Within the scope of <i>E</i></td></tr><tr><td rowspan="2">Arbitrary ciphertexts</td><td>Within the scope of <i>P</i></td><td>Outside the scope of <i>P</i></td></tr><tr><td>Outside the scope of <i>E</i></td><td>Outside the scope of <i>E</i></td></tr></tbody></table>

Considering this distinction, all the attacks presented in this paper operate under the well-formedness assumption from the cryptographic viewpoint but are malicious from the protocol standpoint (this corresponds to the upper left setting in Table II). For instance, the server may in some cases, perform more homomorphic computations than required by the FL aggregation function specification (cf., Section VII-A). However, all the cryptographic operations and ciphertexts manipulated are carried within the scheme's $\mathcal{E}$ functionalities (Eval, Enc, ...). Table II summarizes these ($\mathcal{P}, \mathcal{E}$) threat model combinations.

a) **Adversarial Stealthiness:** For a given protocol $\mathcal{P}$, a stealthy adversary is one whose actions exert little to no influence on the final outcome of $\mathcal{P}$, rendering their presence indistinguishable through direct observation of the protocol's results. This is particularly relevant in adversarial FL, where the outcome is an ML model. Accordingly, a stealthy adversary in FL is characterized by its ability to operate without significantly altering the model's utility, making detection challenging [55]. Notably, the attacks examined in this work exhibit a degree of stealthiness. This property is inherited from the robustness of SGD-based optimization to random perturbation [56, 57], allowing the training process to recover and realign with the nominal convergence path within a few rounds. This characteristic is further analyzed through empirical assessments on a case-by-case basis.

## B. Decryption oracles

The decryption oracle (accepting only well-formed ciphertexts) is the most important component of the CPA<sup>D</sup> security model. Thus, it is an entity or group of entities capable of executing $Dec(ct, sk)$ where ct is a well-formed ciphertext produced or manipulated (via legitimate homomorphic operations) by the adversary. Furthermore, the decrypted values which

form the oracle output must be revealed to the adversary via some (possibly indirect) communication channel<sup>5</sup>

Considering a t-FHE based FL ($\mathcal{P}$), two main decryption events take place within $\mathcal{P}$:

1) The first one is the periodic threshold decryption after the reception of the aggregated model at the end of each round. When this occurs, all the workers obtain the decryption of the aggregated model as part of $\mathcal{P}$. Hence, a worker or group of workers' adversaries are provided a CPA<sup>D</sup> decryption oracle via $\mathcal{P}$. In this case, the server is the only entity that does not legitimately obtain these decrypted values. Nevertheless, auxiliary communication channels resulting from worker-server collusion can result in the availability of a CPA<sup>D</sup> decryption oracle to an adversarial server. Indeed, a single honest-but-curious worker could provide such an auxiliary communication channel.

2) The second is a possible public release of the final model. Indeed in this case, once the training process is completed, the final model is given to all participating entities<sup>6</sup> (including the server). Therefore, this release of decrypted values provides an adversarial server with a CPA<sup>D</sup> decryption oracle which is within the scope of $\mathcal{P}$.

## V. CPA<sup>D</sup> ATTACKS ON FL SECURE AGGREGATION

In this section, we introduce two novel attack strategies on the model update privacy by leveraging key recovery attacks on the underlying FHE in the CPA<sup>D</sup> model. In particular, we adapt the key-recovery attack described by Cheon et al. [42] for BFV/BGV on the abstract CPA<sup>D</sup> game to the context of FL with t-FHE based secure aggregation. In these attacks, the adversaries (worker in VII-B, and server in VII-A) recover the global unshared secret key $\mathbf{sk}_g$. Hence, fully breaking the secure aggregation mechanism.

### A. CPA<sup>D</sup> adversarial server

The central server has a sensitive role in FL, as it or- chestrates the aggregation. By exploiting this pivotal role, we mount a key-recovery attack on threshold BGV and BFV homomorphic encryption schemes when used for secure aggregation. The server conducts the attack following the rules of the CPA<sup>D</sup> security game. It is assisted by a single colluding honest-but-curious worker, who mimics the CPA<sup>D</sup> restricted decryption oracle.

1) Attack overview: The malicious behaviour required in the attack is limited to a single iteration $r^* \in [T]$ of the FL workflow, keeping the rest of the protocol fully honest. During the attack:

* The server computes $[\theta_g^{r+1}]_{pk}$ by aggregating the encrypted individual updates $([\theta_1^r]_{pk}, \dots, [\theta_n^r]_{pk})$. The encrypted aggregated model is a collection of $d$ ciphertexts $(ct_1^{r+1}, \dots, ct_d^{r+1})$. Each ciphertext $ct_i^{r+1}$ encrypts a vector of $N$ model parameters with BGV and BFV, and $N/2$ with

<footnote n="5"><sup>5</sup>That may be within or outside the scope of $\mathcal{P}$.</footnote>

<footnote n="6"><sup>6</sup>Although incentive models for FL are not fully developed, free-rider attacks[58, 59] suggest that entities obtain the final global model as a reward for their computational and data resources allocation throughout the protocol.</footnote>
CKKS. The server picks a ciphertext $ct_j^{r+1}$, and replaces it with another ciphertext $ct_j^*$ satisfying the following:

- It computes a ciphertext of 0 by picking any other ciphertext $ct$ at its disposal (fresh workers' updates preferably) and computes $ct_0 = \text{Eval}(\text{sub}, ct, ct)$. The ciphertext $ct_0$ has depth 0. It now follows the key-recovery attack procedure in [42] by adding $ct_0$ to itself $[\log_2 \Delta] - 1$ times, getting $\hat{ct}_0$.

- Finally, $ct_j^*$ is set to $\hat{ct}_0$. Figure 1 depicts the global model at this iteration.

* $[\theta_g^{r+1}]_{pk}$ is sent to the workers for collaborative decryption as:

$$ (ct_1^{r+1}, \dots, ct_j^{r+1}, \dots, ct_j^{r+1}) \leftarrow (ct_1^{r+1}, \dots, ct_j^*, \dots, ct_d^{r+1}) $$

The colluding worker sends back the decryption of $ct_j^*$ to the server.

* The server recovers the RLWE polynomial noise used for encryption, from the least significant bits of the decryption of the j-th ciphertext of $[\theta_g^{r+1}]_{pk}$. Once the noise is recovered, the server denoises the RLWE sample $ct_0 = (a, b)$, by encrypting 0 as follows: $b - e = \langle a, s \rangle$ and solves the linear system for $s$ following the steps in equation 3.

It is important to emphasize that the smudging noise, a fresh large variance noise added by each partial decryptor (worker) on its partial decryption [38], which is required in threshold constructions does not prevent the attack. This is so since the noise in $ct_0$ has migrated in the plaintext of $\hat{ct}_0$ and, despite the smudging noise, $\hat{ct}_0$ can be expected to decrypt correctly thus leaking the noise in $ct_0$ (note that only the server knows $ct_0$).

2) Model publication as a decryption oracle: Model publication perfectly serves as a decryption oracle for a CPA$^D$ adversarial server. Therefore, the server conducts the attack as in the previous section but at the final iteration ($r^* = T$), and recovers the RLWE noise polynomial from the published final model parameters at the chosen position $j$. This time, the server does not need to team up with a colluding worker. Moreover, an adversarial server will keep ciphertexts of individual updates throughout FL training iterations to decrypt after the key recovery. In this setup, the server can further run white-box privacy attacks on every worker's data.

3) Attack stealthiness: Ideally, the attack should remain entirely undetected by some testing criterion (e.g., final model

predictive performance, weights or gradient distribution, convergence speed, ...). The attack objective is to acquire the capability to decrypt updates from some workers and later execute white-box privacy inference attacks on their datasets [7, 8]. Unless a verification of the aggregation integrity is specified in the FL workflow, and with a careful choice of $r^*$ and $j$, a formal detection of the attack of section V-A1 becomes challenging:

* In the canonical FL workflow, at $r = 0$, the server randomly initializes model weights, following predefined distributions [60, 61, 62]. An adversarial server can leverage this initialization phase to inject the maliciously crafted ciphertext $ct_j^*$. The adversarial server chooses an initialization strategy that resembles the RLWE noise distribution. For instance, uniform weights initialization is preferred for a variety of training purposes, DNN architectures, and data types, as it leads to faster convergence [60]. However, common standard deviations used in these initialization strategies involve the Xavier-Golorot initialization [62] with $\sigma = \sqrt{\frac{1}{n_{in}+n_{out}}}$, and He initialization [61] with $\sigma = \sqrt{\frac{1}{n_{in}}}$, where $n_{in}$, and $n_{out}$ refer respectively to the number of input, and output units of each layer. Hence, depending on parameters (layer dimensions, weight prequantization scaling factor, RLWE noise variance), these initialization strategies may approach the RLWE noise discrete Gaussian distribution. Therefore stealthily carry a CPA$^D$ attack vector at $r^* = 0$.

* For small values of $r^*$, the FL learning dynamic is often highly unstable. The aggregated model's predictive performances are not yet monotonous [63]. Therefore, a small degradation in the global model's predictive performances at the early stages of the FL workflow is usual and not necessarily the sign of a malicious server transmitting the above CPA$^D$ attack vector rather than a genuine global model update.

On the other hand, when the adversarial server cannot collude with one of the workers and relies on the model publication as a decryption oracle.

* The position ($j$) of the aggregated model parameters chunk to alter, is also crucial. For instance, we experimentally observe that the final Batch Normalization layers are less sensitive to the CPA$^D$ alteration compared to fully connected, and convolutional layers for both a ResNet [64], and an EfficientNet-B0 [65] architecture. This is because of the specific roles of these layers. Batch normalization layers primarily normalize the activations of the previous layer and then scale and shift them. Their purpose is to stabilize and accelerate training by reducing internal covariate shift. In contrast, convolutional and fully connected layers are responsible for extracting the main features and crucial information needed for the decision-making process, making them more sensitive to perturbations

## B. CPA$^D$ adversarial worker

Unlike an adversarial server, an adversarial worker has a substantial advantage. She has natural and direct access to a

<figure><img src="image_2.png" /><figcaption>Fig. 1: Encrypted global model at iteration $r^*$. Each layer ($i$) parameters' are contained in $d_i$ ciphertexts with total number of ciphertexts: $d = \sum_{i=1}^{f} d_i$.</figcaption></figure>
<figure><img src="image_3.png" /><figcaption>Fig. 2: Creation of $ct_j^*$ by an adversarial worker in BFV.</figcaption></figure>

decryption oracle in FL. Indeed, she participates in the collaborative decryption process as part of the FL specification and does not require any external collusion. Nevertheless, the limitation of an adversarial worker lies in its capability to impact the decrypted values. These values are the homomorphic sum of several inputs from honest workers, unknown to the adversarial worker. In this section, we investigate how to overcome this limitation and mount an efficient CPA$^D$ key-recovery attack, that exploits the significantly large plaintext modulus $p$, compared to the effective sizes of the homomorphically manipulated values. Indeed, practical instances of BFV/BGV involve a plaintext modulus of size up to 60 bits. While the use of these FHE schemes in FL (or in privacy-preserving ML in general) often requires model quantization [66]. That is, the representation of high precision model parameters or gradients in $\mathbb{Z}_p$, that preserves its utility. Several efficient quantization techniques only require up to 20 bits to preserve a DNN's utility [67], leaving a considerable portion of the most significant bits of the plaintext space unused.

1) Attack overview: The worker inserts the RLWE noise polynomial coefficients within the global model's ciphertexts. To do so, she proceeds as follows:

* The worker uses a bound$^7$ $\beta$ on $\max_i(\log(e_i))$. That is, given the RLWE noise distribution, the number of bits required to store all coefficients, to isolate the RLWE coefficients in the $\beta$ most significant bits of the aggregated model ciphertexts at a chosen position.

* The worker then creates a ciphertext $ct_j^*$ that contains the noise RLWE polynomial in its $\beta$ most significant bits. To do so, she performs $\log(\Delta) + (\log(p) - \beta)$ successive additions on a fresh encryption of 0. The adversarial worker inserts $ct_j^*$ as one of his update ciphertexts at an iteration $r^*$ and uploads its maliciously crafted update to the aggregation server. Figure 2 depicts this process.

* After the aggregation and the collaborative decryption, the adversarial worker recovers at plaintext vector $j$ the values $\{(2^{(\log(p)-\beta)}e_i + \sum_{i=1}^n \theta_{j,i}^{r^*+1} \bmod p)\}_{i \in [N]}$.

* If $2^\beta e_i + \theta_{j,i}^{r^*+1} < p$, the worker recovers the exact value of $e_i$ after performing $e_i = \lfloor \frac{2^{(\log(p)-\beta)}e_i + \theta_{j,i}^{r^*+1}}{2^\beta} \rfloor$. Otherwise, she gets an approximation of $e_i$ where a number of least significant bits and most significant bits are replaced with bits from the aggregated model parameters.

* Finally, the worker denoises the encryption of 0 used to craft $ct_j^*$, and solves for $s$ the linear system $(a, \langle a, s \rangle)$ as described in equation (3).

2) Attack Stealthiness: This attack involves perturbing model parameters at the most significant bits in a portion of the model however without resulting in a significant perturbation of the overall convergence. Still, when the model parameters are bounded by a value smaller than the plaintext modulus, it may be easier to detect or even prevent the attack as discussed in the next section.

## VI. COUNTERMEASURES

In this section, we discuss potential countermeasures, inspired by privacy-preserving and secure FL solutions, that can partially mitigate the identified CPA$^D$ on FHE.

### A. Verified homomorphic calculations

The adversarial server's CPA$^D$ attack path requires a malicious deviation from the protocol as, in that case, the server performs a FHE computation *different* from the one it is expected to do. A countermeasure to such malicious behaviours is *verifiable computation* which requires the server to provide both the aggregation result and a proof that this aggregation results was derived by duly homomorphically applying the aggregation rule on the ciphertexts provided by the workers. For linear aggregation function several practical solutions have already been investigated in the literature, e.g., [3] and could provide a way to counter our attack (albeit at an increased computational cost).

### B. Byzantine resilient aggregation

Our worker's CPA$^D$ attack involves sending a perturbed (at a chosen location) model update to the server. In the context of honest workers' majority and low data heterogeneity, byzantine resilient methods are efficient at eliminating outliers (poisoned updates). So, they have the ability to discard the update carrying the CPA$^D$ attack vector $ct_j^*$. Since the attack relies on the involvement of the CPA$^D$ attack vector in the aggregation, and subsequently, in the collaborative decryption, discarding it from this process completely prevents the key-recovery attack.

Several solutions based on aggregating the updates using a median, or to discard updates based on their distance to an average [68, 69, 24], or directional shift from a barycentric gradient vector [25] were proposed to mitigate byzantine attacks against the FL learning process. Although these methods are nonlinear, some of them have been shown to still result in practical performances when executed over FHE [32].

<footnote>$^7$Such bounds can be obtained by using the Chebychev of Chernoff bounds in a straightforward manner.</footnote>
To investigate the efficiency of these methods we experiment the ability of (1) the Coordinate-wise trimmed-mean aggregation [24], and (2) the Krum aggregation rule [25] to discard our CPA<sup>D</sup> attack vector from the aggregation. Hereafter, we provide an overview of these methods, which we experiment against our attack in the adversarial worker setting in section VII.

1) Krum aggregation [25]: The Krum aggregation approach establishes a tolerance threshold of workers' gradient vectors' directional shifts. Indeed, workers minimize a local loss function that expresses the predictions' discrepancy from the true labels over their local datasets. Although local loss functions of workers do not have identical shapes nor identical local minimums, their gradients vectors share directional similarities. The model is updated using the weighted average gradient vector to minimize the global loss function. To prevent convergence, a byzantine worker might transmit a gradient vector that maximizes his local loss, making the average gradient vector fail to improve the aggregated model. The Krum approach computes a Krum Score at every iteration r, and for every worker's i gradients vector $g_i^r$ that expresses the sum of the distances of $g_i^r$ from the remaining workers' gradients vectors:

$$KS(g_i^r) = \sum_{j \neq i} \|g_i^r - g_j^r\|_2$$

For a fixed number of byzantine workers f, the aggregation rule trims the f gradient vectors with the highest (worst) Krum scores and aggregates the remaining $(n-f)$ vectors.

2) Coordinate-wise trimmed-mean (CWTM) [24]: Similarly to the Krum approach, in order to eliminate outlier, this method relies on the assumption that the majority of updates at each iteration are honestly computed. Specifically, it starts by performing a coordinate-wise sorting of the n vectors, yielding new vectors {$\hat{g}_i^r$}$_{(i \in [n])}$ such that:

$$\hat{g}_1^r \le \hat{g}_2^r \le \dots \le \hat{g}_{n-1}^r$$

Where the inequality holds for every coordinate of the vectors. It then averages the $(n-2f)$ coordinates between positions f and $n-f-1$.

### C. Limitations of Differential Privacy against our attacks

Differential Privacy [70] is a foundational concept in data privacy and protection. It allows analysts to extract insights from sensitive data while ensuring that the inclusion or exclusion of any individual's data has minimal impact on the overall statistical results. This effectively protects individual privacy by hiding the contribution of any single data point. This is achieved by carefully adding controlled noise to the data queries, balancing the need for accurate analysis with the imperative of preserving privacy.

While decrypting individual updates with our CPA<sup>D</sup> attacks completely reveals models' parameters or gradients, DP may still protect these parameters. This added noise significantly reduces the accuracy of FL attacks, as DP makes it difficult

to identify the contribution of any single data point. However, ensuring strong DP guarantees for each update through additive noise, independently of the FHE encryption layer, significantly degrades the global model's accuracy, since it incorporates high DP noises from all updates. By opposition to DDP (Distributed DP) [71], as it injects insufficient noise in the update, such that the sum of noises provides sufficient DP guarantees on the global model, ensuring a better accuracy-privacy balance. However, DDP must be co-implemented with a secure aggregation mechanism, otherwise insufficient DP noise does not provide strong privacy guarantees. Consequently, deferentially private updates prevent privacy attacks post key-recovery, but strongly harm the FL accuracy.

### D. Cryptographic countermeasures and their applicability

cryptographic<sup>8</sup> countermeasures were investigated in [41]. These countermeasures aim to prevent a CPA<sup>D</sup> attacker from observing any exploitable decryption errors from the CPA<sup>D</sup> decryption oracle with probability higher than $\text{neg}(\lambda)$. The countermeasures rely on the passivity of the CPA<sup>D</sup> attacker. Specifically, once a countermeasure is integrated into the scheme's specifications, the passive adversary is required to adhere to it, as failure to do so would classify her as active. Hereafter, We discuss the applicability of these countermeasures in our FL settings.

* **Bootstrapping.** Bootstrapping a ciphertext after every homomorphic operation, including additions, maintains a nominal level of noise, ensuring that decryption errors occur with a probability of at most $\text{neg}(\lambda)^9$. In the context of federated learning, this would require both the adversarial server and worker to bootstrap their respective ciphertexts $ct_0$ after each addition during the construction of the ciphertext $ct^*$. This process would effectively prevent the noise growth which our attack requires in this ciphertext, thus thwarting the attack. However, forcing a malicious entity in FL (or any protocol relying on FHE) to execute operations that would completely neutralize its own attack is awkward. Furthermore, batching-able schemes such as BFV or BGV do not admit an efficient bootstrapping procedure [72].

* **Monitor and Block.** This countermeasure introduced, and implemented in the HElib library [73] involves monitoring a worst-case upper bound B on the standard deviation of the noise in a ciphertext $ct$ throughout the homomorphic circuit evaluation (the Monitor part). The decryption function then returns $\text{Dec}(ct, sk)$ if B is below certain a threshold, or $\perp$ otherwise (the Block part). Similarly to the bootstrapping case, relying on this countermeasure would require the adversary to thwart her own attack by herself. Furthermore, the collaborative nature of the decryption function makes it challenging to incorporate blocking restrictions, as it would require consensus among parties on the blocking action.

Although our attacks fall under the CPA<sup>D</sup> security notion, i.e., the adversary is limited to exploit only legitimate ciphertexts

<footnote>8Integrated to the scheme itself in an attempt to provide CPA<sup>D</sup> security.</footnote>

<footnote>9The error probability of the bootstrapping itself conditions this probability.</footnote>
(well-formed fresh ciphertexts or ciphertexts derived from fresh well-formed ciphertexts via legit homomorphic evaluations), our adversaries are not passive from a protocol point of view. As a consequence, when the FHE scheme is extended to include counter-measures such as the above, our adversaries

* Either fully implement the countermeasure and prevent their own attacks by themselves (in which case they remain CPA$^D$ adversaries against the extended scheme).

* Or they do not implement the countermeasure in order to preserve their attack (in which case they become beyond-CPA$^D$ adversaries against the extended scheme).

## VII. EXPERIMENTS AND ANALYSIS

To evaluate the feasibility of the proposed CPA$^D$ attacks, we conduct several experiments, for both adversarial server (subsection VII-A) and worker (subsection VII-B) in realistic FL settings for various values of $p$ and $q$, and we experiment the previously mentioned countermeasures.

### A. Experiments in the CPA$^D$ adversarial server setting

We evaluate the attack stealthiness, by exploring the robustness of a pre-trained model against internal perturbations, and a FedAVG FL process. For this purpose, we train a ResNet [74] classifier on the MNIST dataset, and an EfficientNet model [65] on the FairFace dataset [75]. Subsequently, we replace a portion of their parameters with Gaussian distributed vectors at different locations of the models' internal structures (layers, and position within the layers), following the BGV, and BFV RLWE discrete Gaussian noise distribution (with $\sigma = 3.2$). We explore the federated learning convergence dynamic with attack perturbation occurring at the early, intermediate, and late stages of collaborative training. We consider different workers' data heterogeneity levels (w.r.t class labels in MNIST), parameterized by the *Dirichlet* distribution's concentration parameter $\alpha$. High values of $\alpha$ indicate close data distributions. In contrast, low values of $\alpha$ provide distant workers' data distribution $^{10}$.

As shown in Figure 3, there are significant disparities in the impact of the layer perturbation induced by the attack ciphertext $ct^*$ on the accuracy degradation. Both the ResNet and EfficientNet $^{11}$ architectures show better robustness to $ct^*$ perturbation in the final layers. Indeed, the best performances are achieved when the perturbation occurs in the late *batch normalization* layers. Also, the experiments highlight the importance of the layer type and depth in which the perturbation is inserted. *Batch normalization* layers exhibit a better robustness to $ct^*$ perturbation, while accuracy dropped as low as a random guess (10% for MNIST, and 50% for FairFace) for other layers. The most sensitive positions for $ct^*$ perturbation are the final *Dense* layers, which significantly degrades the model accuracy. Figures 4 (a), (b), and (c) emphasize the importance of carefully choosing $r^*$. From Figure 5, we

observe that highly heterogeneous workers' data distributions make the learning dynamic significantly unstable at the early stages (higher standard deviation of convergence metrics). This instability is comparable (w.r.t. accuracy degradation) to our attack vector. Figure 4 demonstrates that the global model rapidly *recovers* from the CPA$^D$ perturbation, particularly in the initial stages, with learning stabilizing again after 3 to 5 iterations

Therefore, a careful audit of the robustness of the chosen architecture to the intended perturbation, and workers' data heterogeneity, allows a malicious server to enhance the stealthiness of our attack on the ongoing FL workflow by carefully choosing when and where to insert the perturbation.

### B. Experiments in the CPA$^D$ adversarial worker setting

We evaluate the effectiveness of the CPA$^D$ adversarial worker in recovering the RLWE noise polynomial. The main challenge is to isolate the noise bits from the aggregated parameters bits w.r.t two factors: the precision of the quantization used to encrypt workers' updates, and the number of updates to aggregate. Since homomorphic encryption schemes do not usually provide efficient ciphertext division (or scaling by a positive value smaller than 1), the weight scaling is often delegated to workers locally. Consequently, the cumulative updates sum might overwrite the most significant bits with the RLWE noise terms.

We recall that the standard deviation used in BGV, and BFV for sampling RLWE error polynomial coefficients is $\sigma = \frac{8}{\sqrt{2\pi}} \approx 3.2$ [26, 27]. Analyzing the distribution of $\log(e_i)$ where $e_i$ are sampled from $N(0, \sigma = 3.2)$ suggests an appropriate value for $\beta = \max_i(\log(e_i))$ to be 4. Indeed, all RLWE error polynomial coefficients should fit (with high probability) within the 4 most significant bits of the decrypted message. Figure 6 (b) illustrates the Euclidean distances between the recovered RLWE noise coefficients vector and the original one used for encryption. This is shown for various quantization precision and with a varying number of summed worker updates. Lower (yet still efficient) quantization precision allows for near-perfect recovery of the noise coefficients, even for high cross-silo federation sizes. Conversely, higher quantization precision, combined with a large number of workers' updates, significantly reduces the attacker's ability to exploit the information hidden in the most significant bits by effectively overwriting it. Figure 6 (a) complements Figure 6 (b) by illustrating the success rate of recovering RLWE error coefficients from the attack described in section V-B1, achieving 100% recovery rates, for low precision quantization, and appropriate federation size. Finally, Table III emphasizes the minimal computational expense associated with generating the ciphertext $ct^*$.

### C. Krum's resilience against CPA$^D$ perturbations

To investigate the ability of Krum's byzantine resilient aggregation to discard the CPA$^D$ adversary's attack vector from the aggregation process, we train 10 workers on different class distributions of the MNIST dataset obtained using a

<footnote>$^{10}$ $\alpha = 0$ makes every dataset contain data samples of one class label, and $\alpha = +\infty$ yields perfectly uniform, and homogeneous distributions.</footnote>

<footnote>$^{11}$ Due to its large number of layers, we only illustrate the accuracy for a relevant subset of layers perturbation in the EfficientNet-B0 architecture.</footnote>
<figure><img src="image_4.png" /><figcaption>(a) ResNet on MNIST (10-class : 10% baseline)</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>(b) EfficientNet-B0 on FairFace (Binary : 50% baseline)</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>(a) Early CPA<sup>D</sup> perturbation</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 5: FL convergence dynamic on MNIST with 10 workers over the first 30 iterations, with $\alpha = 10$, and $\alpha = 0.1$</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>(b) Intermediate CPA<sup>D</sup> perturbation</figcaption></figure>

<table><caption>TABLE III: Computation runtime of ct\*_j in milliseconds. Recall that $\Delta = \frac{q}{p}$ gives the number of necessary successive additions to form ct\*_j.</caption><thead><tr><th rowspan="2">Scheme</th><th colspan="5">FHE parameters (log(N), log(Δ))</th></tr><tr><th>(12, 13)</th><th>(13, 30)</th><th>(14, 31)</th><th>(15, 53)</th><th>(16, 70)</th></tr></thead><tbody><tr><td>BFV</td><td>0.52</td><td>1.64</td><td>6.92</td><td>14.54</td><td>26.03</td></tr><tr><td>BGV</td><td>0.43</td><td>1.29</td><td>16.12</td><td>27.71</td><td>58.33</td></tr></tbody></table>

<figure><img src="image_9.png" /><figcaption>(c) Late CPA<sup>D</sup> perturbation</figcaption></figure>

<caption>Fig. 4: FL convergence dynamic on MNIST with 10 workers over the first 30 iterations, with $\alpha = 10$, and three attack timings: $r^* = 2$ (a), $r^* = 15$ (b), and $r^* = 25$ (c).</caption>

*Dirichlet* sampling to provide a diversity of gradients vectors directions and amplitudes. We insert a perturbed vector among the updates following the approach described in VII-B and evaluate its Krum score KS, and we compare it to the scores of the honestly computed gradients. Figure 7 illustrates these experiments. From figure 7, we observe that even under high data heterogeneity, the perturbed gradient vector remains highly characterized by its directional shift compared to honestly generated gradients, which is reflected by their respective Krum scores. This results in the constant discarding of the attack vector from the aggregation process which prevents the completion of the attack described in VII-B.

High data heterogeneity induces slightly closer Krum scores of honest, and perturbed gradient vectors. However, this proximity remains insufficient for the attacker's update to be involved in the aggregation, as it remains highly outlying compared to the rest of the gradients.
<figure><img src="image_10.png" /><figcaption>(a) Percentage of correctly recovered error coefficients with a fixed quantization precision (16 bits), and a varying number of workers.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>(b) Euclidean distance between the recovered error coefficients vector, and the true RLWE error coefficients vector for different quantization precision.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Fig. 7: Krum scores of honestly computed gradients, and the gradient vector carrying the CPA$^D$ attack vector.</figcaption></figure>

### D. CWTM's resilience against CPA$^D$ perturbations

This Byzantine-resilient aggregation strategy operates a more fine-grained trimming of outlying updates, as it performs a coordinate-wise trimmed average. That is, the removal of outlying portions from each update while keeping the remaining statistically valid coordinates within the aggregation process. This perfectly fits the CPA$^D$ attack prevention, as the attack vector is inserted at a chosen position within the model parameters or gradients. We evaluate the percentage of trimmed attack vector coordinates (the $N$ RLWE noise coefficients) using this approach on 9 honestly computed gradient vectors, and the maliciously crafted one as described

in VII-B. We again vary workers' data heterogeneity with MNIST in order to experiment under diverse gradients distributions (directions, and amplitudes). Figure 8 depicts the obtained results.

<figure><img src="image_13.png" /><figcaption>Fig. 8: Percentage of trimmed RLWE noise coefficients from the aggregation process using CWTM, with varying data heterogeneity levels, and with $f = 1$.</figcaption></figure>

Figure 8 shows the ability of CWTM's aggregation to entirely eliminate all CPA$^D$ attack vector coordinates from the aggregation. The RLWE noise coefficients are inserted in the most significant bits of the malicious worker's update, this portion of the update is always the largest among other workers' updates, even under highly heterogeneous data distributions. Therefore, it will constantly be discarded (trimmed) from the aggregation process even with $f = 1$ (only a single worker assumed byzantine). Furthermore, it is noteworthy that this method keeps the remaining honestly computed portions of the CPA$^D$ malicious worker's update (apart from the $ct[*_j]$ ciphertext), providing a more targeted attack prevention compared to Krum.

**Attack mitigation and stealthiness:** Let us emphasize that being able to thwart a CPA$^D$ adversarial worker by means of Byzantine resilient aggregation does not contradict with the stealthiness of the attack. Indeed, since the server performs encrypted domain aggregation, the filtering of an outlying update is performed obliviously. Hence, the decryption oracle necessary to the attacker can be closed without the server
detecting the attack. Meanwhile, the fact that our CPA<sup>D</sup> attack (Sect. V-B) has negligible impact on the final model, makes it consistent with the conventional stealthiness criteria in adversarial ML [55].

VIII. CONCLUSION

This work bridges the gap between the theoretical vul-
nerability of CPA<sup>D</sup> security and its practical exploitation in
federated learning, a key application for FHE. The proposed
CPA<sup>D</sup> attacks leverage the inherent limitations of CPA secu-
rity under strong adversarial models, allowing attackers with
limited access to a decryption oracle to achieve practical key
recovery. We show that when their attack vector is care-
fully incorporated, our attacks operate stealthily, potentially
going unnoticed while compromising the system's security.
This vulnerability extends to prominent non-approximate FHE
schemes like BFV, BGV (including their threshold variants)
when deployed in FL, and in the absence of robust input
verification. Additionally, these attacks require very limited
malicious behaviour. Indeed, the active half of the attack
only takes place at a single iteration and acts on a single
ciphertext. Finally, we demonstrate that several aggregation
rules, originally designed to enhance the robustness of the fed-
erated learning process against byzantine actors carrying out
machine-learning-related poisoning attacks, also effectively
protect against these cryptographic threats.

ACKNOWLEDGEMENTS

This work was supported by the French ANR project ANR-22-CE39-0002 EQUIHID, and the France 2030 ANR Projects ANR-22-PECY-003 SecureCompute, ANR-23-PECL-0009 TRUSTINCloudS. The authors would like to thank the anonymous referees for a number of suggestions that helped improving this paper

APPENDIX

A. The CPA<sup>D</sup> security game

In this section, we recall the CPA<sup>D</sup> security game [20]. It is
a left-or-right game in which an adversary attempts to recover
a random bit with a significant advantage over guessing.

Given an encryption scheme

$$
\mathcal{E} = (\text{KeyGen}, \text{Enc}, \text{Dec}, \text{Eval}),
$$

with plaintext domain $\mathcal{M}$ and ciphertext domain $\mathcal{C}$, an adversary $\mathcal{A}$ and value $\lambda$ for the security parameter, the game is parameterized by a bit $b^* \stackrel{\$}{\leftarrow} \{0, 1\}$, unknown to $\mathcal{A}$, and an initially empty state $S$ of message-message-ciphertext triplets:

* Key generation: Run (ek, dk) ← KeyGen(1^λ), and give ek to A. Note that this security game works identically in the non-public key setting, only ek is not revealed to A.

* Encryption request: When $\mathcal{A}$ queries $(test_messages, m_0, m_1)$, where $m_0, m_1 \in \mathcal{M}$, compute $c = \text{Enc}_{\mathcal{E}k}(m_{b^*})$, give $c$ to $\mathcal{A}$ and do

$$
S := [S; (m_0, m_1, c)].
$$

* Evaluation request: When $\mathcal{A}$ queries $(\mathit{eval}, f, l_1, \dots, l_K)$, with $K$ the arity of function $f$ and $\forall i: l_i \in [1, |S|]$, compute

$$
m'_{0} = f(S[l_{1}].m_{0}, \dots, S[l_{K}].m_{0}),
$$

and

$$
m'_{1} = f(S[l_{1}].m_{1}, \dots, S[l_{K}].m_{1}),
$$

as well as

$$
c' = \text{Eval}(f, S[l_1].c, \dots, S[l_K].c),
$$

where $S[l_i].e$ is the element $e \in \{m_0, m_1, c\}$ of the
state $S$ for the $l_i$-th request. Then, we update $S$ with
the $|S| + 1$ entry $(m'_0, m'_1, c')$ as follows:

$$
S := [S; (m'_{0}, m'_{1}, c')]
$$

* Decryption request: When $\mathcal{A}$ queries $(\text{ciphertext}, l)$ ($l \in [1, |S|]$) proceed as follows: if $S[l].m_0 \neq S[l].m_1$ then return $\perp$ to $\mathcal{A}$, otherwise return her $\text{Dec}_{\text{dk}}(S[l].c)$.

* Guessing stage : After polynomially many interleaved encryption, evaluation and decryption requests, $\mathcal{A}$ outputs $(\text{guess}, b)$. The outcome of the game is determined as follows: if $b = b^*$ then $\mathcal{A}$ wins the game. Otherwise, $\mathcal{A}$ loses the game.

A number of points should be emphasized with respect to the above game. First, the decryption oracle accepts only ciphertexts from the game state which are necessarily well-formed. That is, it accepts only ciphertexts either produced by an encryption oracle via an encryption request or derived by the evaluation oracle via an evaluation request, so derived by correctly applying homomorphic operators to well-formed ciphertexts. As such, the above game does not capture any CCA aspects. Second, when $S[l].m_0 = S[l].m_1$ it is important that the decryption oracle returns $\text{Dec}_{\text{dk}}(S[l].c)$ and not $S[l].m_0$ or $S[l].m_1$. For exact FHE, this has no impact since $\text{Dec}_{\text{dk}}(S[l].c) = S[l].m_0 = S[l].m_1$. And in this case, $\mathcal{A}$ learns nothing she does not already know, so CPA$^D$ security coincides with CPA security. However, for approximate or non-exact FHE, even when $S[l].m_0 = S[l].m_1$, we may have that $\text{Dec}_{\text{dk}}(S[l].c) \neq S[l].m_0$ and $\text{Dec}_{\text{dk}}(S[l].c) \neq S[l].m_1$. Thus, for approximate or non-exact FHE, the decryption oracle grants $\mathcal{A}$ access to information she cannot compute on her own, resulting or not in a guessing advantage depending on whether or not the cryptosystem at hand is CPA$^D$ secure. As a last remark, let us also emphasize that, in the above game, $\mathcal{A}$ has control over the homomorphic calculations that are performed as $f$ is included in the evaluation request.

B. CPA<sup>D</sup> and threshold settings

In a t-out-n threshold FHE scheme, users encrypt their data using a joint public key, and at least t users should perform collaborative decryption to retrieve the plaintext, without the knowledge of the global secret key. Thus, a secure threshold construction inherits the security and vulnerabilities of the underlying FHE cryptosystem.
A secure threshold scheme assumes that an adversary may control up to $t-1$ shares of the global secret key without improving its adversarial advantage. Existing threshold FHE constructions employ noise-flooding [76, 38] to protect individual secret shares during partial decryption, due to the linearity of the partial decryption function in each share $s_{k_i}$. In contrast, our work considers a fully honest collaborative decryption setting where the adversary possesses and uses only their own secret share, without collusion. Now, if a scheme is CPA$^D$ insecure, an adversary may be able to retrieve the decryption key from the knowledge of several triplets $\{m, c, \text{Dec}(c)\}$, where $c$ is a well-formed encryption of $m$. In the collaborative threshold decryption setup, an adversary is first given the ciphertext $c$ to decrypt. This ciphertext is encrypted under the global secret key that he or she does not know. At the end of the collaborative decryption protocol, the adversary is granted access to $\text{Dec}(c)$. Assuming the messages are also known, the adversary is clearly in the position of a CPA$^D$ attacker on the global secret key. Therefore, either CPA$^D$ security or a countermeasure against CPA$^D$ attacks is a must-have in the threshold setting.

## REFERENCES

[1] Alexey Gribov, Kelsey Horan, Jonathan Gryak, Kayvan Najarian, Vladimir Shpilrain, Reza Soroushmehr, and Delaram Kahrobaei. Medical diagnostics based on encrypted medical data. In Adriana Compagnoni, William Casey, Yang Cai, and Bud Mishra, editors, *Bio-inspired Information and Communication Technologies*, pages 98–111, Cham, 2019. Springer International Publishing.

[2] Leyli Karaçay, Erkay Savaş, and Halit Alptekin. Intrusion detection over encrypted network data. In *The Computer Journal*, 63(1):604–619, 2020.

[3] Abbass Madi, Oana Stan, Aurélien Mayoue, Arnaud Grivet-Sébert, Cédric Gouy-Pailler, and Renaud Sirdey. A secure federated learning framework using homomorphic encryption and verifiable computing. In *2021 Reconciling Data Analytics, Automation, Privacy, and Security: A Big Data Challenge (RDAAPS)*, pages 1–8, 2021.

[4] Aymen Boudguiga, Oana Stan, Hichem Sedjelmaci, and Sergiu Carpov. Homomorphic encryption at work for private analysis of security logs. In *ICISSP*, pages 515–523, 2020.

[5] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Aarti Singh and Xiaojin (Jerry) Zhu, editors, *Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20–22 April 2017, Fort Lauderdale, FL, USA*, volume 54 of *Proceedings of Machine Learning Research*, pages 1273–1282. PMLR, 2017.

[6] Oana Stan, Vincent Thouvenot, Aymen Boudguiga, Katarzyna Kapusta, Martin Zuber, and Renaud Sirdey. A secure federated learning: analysis of different cryptographic tools.

In *SECRYPT 2022-19th International Conference on Security and Cryptography*, volume 1, pages 669–674, 2022.

[7] Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In *2019 IEEE Symposium on Security and Privacy (SP)*. IEEE, May 2019.

[8] Lingjuan Lyu, Han Yu, Xingjun Ma, Chen Chen, Lichao Sun, Jun Zhao, Qiang Yang, and Philip S. Yu. Privacy and robustness in federated learning: Attacks and defenses, 2022.

[9] Erfan Hosseini and Ashish Khisti. Secure aggregation in federated learning via multiparty homomorphic encryption. In *2021 IEEE Globecom Workshops (GC Wkshps)*, pages 1–6, 2021.

[10] Arnaud Grivet Sébert, Renaud Sirdey, Oana Stan, and Cédric Gouy-Pailler. Protecting data from all parties: Combining fhe and dp in federated learning, 2022.

[11] Adda-Akram Bendoukha, Didem Demirag, Nesrine Kaaniche, Aymen Boudguiga, Renaud Sirdey, and Sébastien Gambs. Towards privacy-preserving and fairness-aware federated learning framework. In *PETs 2025: Privacy Enhancing Technologies*, volume 2025, pages 845–865, Washington, DC, United States, July 2025.

[12] Z. Brakerski. Fully homomorphic encryption without modulus switching from classical gapsvp. In *CRYPTO*, pages 868–886, 2012.

[13] J. Fan and F. Vercauteren. Somewhat practical fully homomorphic encryption. *IACR ePrint*, 2012.

[14] Z. Brakerski, C. Gentry, and V. Vaikuntanathan. (leveled) fully homomorphic encryption without bootstrapping. *ACM TOCT*, pages 1–36, 2014.

[15] J. H. Cheon, A. Kim, M. Kim, and Y. Song. Homomorphic encryption for arithmetic of approximate numbers. In *ASIACRYPT*, pages 409–437. 2017.

[16] I. Chillotti, N. Gama, M. Georgieva, and M. Izabachène. Faster fully homomorphic encryption: Bootstrapping in less than 0.1 seconds. In *ASIACRYPT*, 2016.

[17] Ilaria Chillotti, Nicolas Gama, and Louis Goubin. Attack- ing FHE-based applications by software fault injections. Cryptology ePrint Archive, Paper 2016/1164, 2016.

[18] M. Manulis and J. Nguyen. Fully homomorphic en- cryption beyond IND-CCA1 security: Integrity through verifiability. In *EUROCRYPT*, 2024.

[19] S. Canard, C. Fontaine, D. H. Phan, D. Pointcheval, M. Renard, and R. Sirdey. Relations among new CCA security notions for approximate FHE. Technical Report 892, IACR ePrint, 2024.

[20] B. Li and D. Micciancio. On the security of ho- momorphic encryption on approximate numbers. In *EUROCRYPT*, pages 648–677, 2021.

[21] M. Checri, R. Sirdey, A. Boudguiga, and J.-P. Bultel. On the practical cpad security of “exact” and threshold FHE schemes. In *CRYPTO*, 2024.
[22] J. H. Cheon, H. Choe, A. Passelègue, D. Stehlé, and E. Suvanto. Attacks against the IND-CPAD security of exact FHE schemes. Technical Report 127, IACR ePrint, 2024.

[23] Baiyu Li and Daniele Micciancio. On the security of homomorphic encryption on approximate numbers. In Anne Canteaut and François-Xavier Standaert, editors, *Advances in Cryptology – EUROCRYPT 2021*, pages 648–677, Cham, 2021. Springer International Publishing.

[24] Dong Yin, Yudong Chen, Kannan Ramchandran, and Peter Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates, 2021.

[25] Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, *Advances in Neural Information Processing Systems*, volume 30. Curran Associates, Inc., 2017.

[26] Junfeng Fan and Frederik Vercauteren. Somewhat practical fully homomorphic encryption. Cryptology ePrint Archive, Paper 2012/144, 2012. https://eprint.iacr.org/2012/144.

[27] Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. Fully homomorphic encryption without bootstrapping. Cryptology ePrint Archive, Paper 2011/277, 2011. https://eprint.iacr.org/2011/277.

[28] Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song. Homomorphic encryption for arithmetic of approximate numbers. In Tsuyoshi Takagi and Thomas Peyrin, editors, *Advances in Cryptology – ASIACRYPT 2017*, pages 409–437, Cham, 2017. Springer International Publishing.

[29] Léo Ducas and Daniele Micciancio. Fhew: Bootstrapping homomorphic encryption in less than a second. Cryptology ePrint Archive, Paper 2014/816, 2014. https://eprint.iacr.org/2014/816.

[30] Ilaria Chillotti, Nicolas Gama, Mariya Georgieva, and Malika Izabachène. Tfhe: Fast fully homomorphic encryption over the torus. Cryptology ePrint Archive, Paper 2018/421, 2018. https://eprint.iacr.org/2018/421.

[31] Oded Regev. On lattices, learning with errors, random linear codes, and cryptography, 2024.

[32] Antoine Choffrut, Rachid Guerraoui, Rafael Pinot, Renaud Sirdey, John Stephan, and Martin Zuber. Sable: Secure and byzantine robust learning, 2023.

[33] V. Lyubashevsky, C. Peikert, and O. Regev. On ideal lattices and learning with errors over rings. In *EUROCRYPT*, page 1–23, 2010.

[34] Craig Gentry, Amit Sahai, and Brent Waters. Homomorphic encryption from learning with errors: Conceptually-simpler, asymptotically-faster, attribute-based. Cryptology ePrint Archive, Paper 2013/340, 2013. https://eprint.iacr.org/2013/340.

[35] Jing Ma, Si-Ahmed Naas, Stephan Sigg, and Xixiang Lyu. Privacy-preserving federated learning based on

multi-key homomorphic encryption, 2021.

[36] Alberto Pedrouzo-Ulloa, Aymen Boudguiga, Olive Chakraborty, Renaud Sirdey, Oana Stan, and Martin Zuber. Practical multi-key homomorphic encryption for more flexible and efficient secure federated aggregation (preliminary work). Cryptology ePrint Archive, Paper 2022/1674, 2022. https://eprint.iacr.org/2022/1674.

[37] Yousra Belfaik Oumaima Chakir and Yassine Sadqi. Multi-key fully homomorphic encryption for privacy-preservation within federated learning environments. *EDPACS*, 68(6):25–34, 2023.

[38] Christian Mouchet, Elliott Bertrand, and Jean-Pierre Hubaux. An efficient threshold access-structure for ring-based multiparty homomorphic encryption. Cryptology ePrint Archive, Paper 2022/780, 2022. https://eprint.iacr.org/2022/780.

[39] Gilad Asharov, Abhishek Jain, and Daniel Wichs. Multi-party computation with low communication, computation and interaction via threshold fhe. Cryptology ePrint Archive, Paper 2011/613, 2011. https://eprint.iacr.org/2011/613.

[40] Adi Shamir. How to share a secret. *Commun. ACM*, 22(11):612–613, nov 1979.

[41] Marina Checri, Renaud Sirdey, Aymen Boudguiga, and Jean-Paul Bultel. On the practical cpad security of “exact” and threshold fhe schemes and libraries. Cryptology ePrint Archive, Paper 2024/116, 2024. https://eprint.iacr.org/2024/116.

[42] Jung Hee Cheon, Hyeongmin Choe, Alain Passelègue, Damien Stehlé, and Elias Suvanto. Attacks against the indcpa-d security of exact fhe schemes. Cryptology ePrint Archive, Paper 2024/127, 2024. https://eprint.iacr.org/2024/127.

[43] Hanhan Zhou, Tian Lan, Guru Venkataramani, and Wenbo Ding. Every parameter matters: Ensuring the convergence of federated learning with dynamic heterogeneous models reduction, 2023.

[44] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. 2018.

[45] Dingzhu Wen, Ki-Jun Jeon, and Kaibin Huang. Federated dropout – a simple approach for enabling federated learning on resource constrained devices, 2022.

[46] Mansouri Mohamad, Melek Önen, Wafa Ben Jaballah, and Mauro Conti. Sok: Secure aggregation based on cryptographic schemes for federated learning. In IACR, editor, *PETS 2023, 23rd Privacy Enhancing Technologies Symposium, 10-15 July 2023, Lausanne, Switzerland (Hybrid Conference)*, Lausanne, 2023. IACR.

[47] Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy preserving machine learning. Cryptology ePrint Archive, Paper 2017/281, 2017. https://eprint.iacr.org/2017/281.

[48] James Bell, K. A. Bonawitz, Adrià Gascón, Tancrède
Lepoint, and Mariana Raykova. Secure single-server aggregation with (poly)logarithmic overhead. Cryptology ePrint Archive, Paper 2020/704, 2020. https://eprint.iacr.org/2020/704.

[49] Runhua Xu, Nathalie Baracaldo, Yi Zhou, Ali Anwar, James Joshi, and Heiko Ludwig. Fedv: Privacy-preserving federated learning over vertically partitioned data, 2021.

[50] Stacey Truex, Nathalie Baracaldo, Ali Anwar, Thomas Steinke, Heiko Ludwig, Rui Zhang, and Yi Zhou. A hybrid approach to privacy-preserving federated learning, 2019.

[51] Heiko Ludwig, Nathalie Baracaldo, Gigi Thomas, Yi Zhou, Ali Anwar, Shashank Rajamoni, Yuya Ong, Jayaram Radhakrishnan, Ashish Verma, Mathieu Sinn, et al. Ibm federated learning: an enterprise framework white paper v0. 1. arXiv preprint arXiv:2007.10987, 2020.

[52] Holger R. Roth, Yan Cheng, Yuhong Wen, Isaac Yang, Ziyue Xu, Yuan-Ting Hsieh, Kristopher Kersten, Ahmed Harouni, Can Zhao, Kevin Lu, Zhihong Zhang, Wenqi Li, Andriy Myronenko, Dong Yang, Sean Yang, Nicola Rieke, Abood Quraini, Chester Chen, Daguang Xu, Nic Ma, Prerna Dogra, Mona Flores, and Andrew Feng. Nvidia flare: Federated learning from simulation to real-world. 2022.

[53] Weizhao Jin, Yuhang Yao, Shanshan Han, Carlee Joe-Wong, Srivatsan Ravi, Salman Avestimehr, and Chaoyang He. Fedml-he: An efficient homomorphic-encryption-based privacy-preserving federated learning system, 2023.

[54] Chris Brzuska, Sébastien Canard, Caroline Fontaine, Duong Hieu Phan, David Pointcheval, Marc Renard, and Renaud Sirdey. Relations among new CCA security notions for approximate FHE. *IACR Communications in Cryptology*, 2(1), 2025.

[55] Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. Analyzing federated learning through an adversarial lens, 2019.

[56] Avery Ma, Yangchen Pan, and Amir massoud Farahmand. Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods, 2023.

[57] Wenhan Zhang, Marwan Krunz, and Gregory Ditzler. Stealthy adversarial attacks on machine learning-based classifiers of wireless signals. *IEEE Transactions on Machine Learning in Communications and Networking*, 2:261–279, 2024.

[58] Yann Fraboni, Richard Vidal, and Marco Lorenzi. Free-rider attacks on model aggregation in federated learning, 2021.

[59] Jierui Lin, Min Du, and Jian Liu. Free-riders in federated learning: Attacks and defenses, 2019.

[60] Wadii Boulila, Maha Driss, Mohamed Al-Sarem, Faisal Saeed, and Moez Krichen. Weight initialization techniques for deep learning algorithms in remote sensing:

Recent trends and future perspectives, 2021.

[61] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, 2015.

[62] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Yee Whye Teh and Mike Titterington, editors, *Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics*, volume 9 of *Proceedings of Machine Learning Research*, pages 249–256, Chia Laguna Resort, Sardinia, Italy, 13–15 May 2010. PMLR.

[63] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. 2018.

[64] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.

[65] Mingxing Tan and Quoc V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks, 2020.

[66] Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference, 2021.

[67] Md Fahim Faysal Khan, Mohammad Mahdi Kamani, Mehrdad Mahdavi, and Vijaykrishnan Narayanan. Learning to quantize deep neural networks: A competitive-collaborative approach. In *2020 57th ACM/IEEE Design Automation Conference (DAC)*, pages 1–6, 2020.

[68] Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, and John Stephan. Byzantine machine learning made easy by resilient averaging of momentums, 2022.

[69] Jinhyun So, Basak Guler, and A. Salman Avestimehr. Byzantine-resilient secure federated learning, 2021.

[70] Cynthia Dwork. Differential privacy: A survey of results. In Manindra Agrawal, Dingzhu Du, Zhenhua Duan, and Angsheng Li, editors, *Theory and Applications of Models of Computation*, pages 1–19, Berlin, Heidelberg, 2008. Springer Berlin Heidelberg.

[71] Naman Agarwal, Peter Kairouz, and Ziyu Liu. The skellam mechanism for differentially private federated learning, 2021.

[72] Ahmad Al Badawi and Yuriy Polyakov. Demystifying bootstrapping in fully homomorphic encryption. *IACR Cryptol. ePrint Arch.*, page 149, 2023.

[73] Shai Halevi and Victor Shoup. Design and implementation of HElib: a homomorphic encryption library. Cryptology ePrint Archive, Paper 2020/1481, 2020.

[74] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.

[75] Kimmo Kärkkäinen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age, 2019.

[76] Alain Passelègue and Damien Stehlé. Low communication threshold fully homomorphic encryption. Cryptology ePrint Archive, Paper 2024/1984, 2024.