DictPFL: Efficient and Private Federated Learning on Encrypted Gradients
Authors: Jiaqi Xue
Mayank Kumar
Yuzhang Shang
Shangqian Gao
Rui Ning
Mengxin Zheng
Xiaoqian Jiang
Qian Lou
Venue: NeurIPS 2025 poster ()
Keywords: Federated Learning
Homomorphic Encryption
Privacy
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: We propose DictPFL, a framework that ensures efficient and private federated learning (FL) by encrypting shared gradients and keeping most gradients local, while still preserving the performance of global gradient aggregation.

Abstract: Federated Learning (FL) enables collaborative model training across institutions without sharing raw data. However, gradient sharing still risks privacy leakage, such as gradient inversion attacks. Homomorphic Encryption (HE) can secure aggregation but often incurs prohibitive computational and communication overhead. Existing HE-based FL methods sit at two extremes: encrypting all gradients for full privacy at high cost, or partially encrypting gradients to save resources while exposing vulnerabilities. We present **DictPFL**, a practical framework that achieves full gradient protection with minimal overhead. DictPFL encrypts every transmitted gradient while keeping non-transmitted parameters local, preserving privacy without heavy computation. It introduces two key modules: **Decompose-for-Partial-Encrypt (DePE)**, which decomposes model weights into a static dictionary and an updatable lookup table—only the latter is encrypted and aggregated, while the static dictionary remains local and requires neither sharing nor encryption; and **Prune-for-Minimum-Encrypt (PrME)**, which applies encryption-aware pruning to minimize encrypted parameters via consistent, history-guided masks. Experiments show that DictPFL reduces communication cost by 402-748$\times$ and accelerates training by 28-65$\times$ compared to fully encrypted FL, while outperforming state-of-the-art selective encryption methods by 51-155$\times$ in overhead and 4-19$\times$ in speed. Remarkably, DictPFL’s runtime is within 2$\times$ of plaintext FL, demonstrating, for the first time, that HE-based private federated learning is practical for real-world deployment. The code is publicly available at https://github.com/UCF-ML-Research/DictPFL.

DictPFL: Efficient and Private Federated Learning on Encrypted Gradients
Jiaqi Xue', Mayank Kumar', Yuzhang Shang', Shangqian Gao', Rui Ning²
Mengxin Zheng', Xiaoqian Jiang', Qian Lou'
Uniiversity of Central Florida FFlorida State University 'Old Dominion University
*University of Texas Health Science Center at Houston
(jiaqi. xue, xaxuemmayyy mayank kumar, cummar,yzhhane yuzhang. shang, mengxin zheng, qian. lou.fffufuuffff..m .ddu sgaoêcs w.f.fuu.duu. edu, rningRodu. edu, xiaoqian. jiang@uth. xiaogian.jianguutt
tmc.ede
Abstract
Federated Learning (FL) enables collaborative model training across institutions without sharing raw data. However, gradient sharing still risks privacy leakage, such as gradient inversion attacks. Homomorphic Encryption (HE) can secure aggregation but often incurs prohibitive computational and communication over- head. Existing HE-based FI. methods at two extremes: encrypting all gradients for full privacy at high cost, or partially encrypting gradients to save resources while exposing vulnerabilities. We present DictPFL, a practical framework that achieves full gradient protection with minimal overhead. DictPFL encrypts every transmitted gradient while keeping non-transmitted parameters local, preserving privacy without heavy computation. It introduces two key modules: Decompose- for-Partial- -neellpp--eyytttte (DePE), which decomposes model weights into a static dic- tionary and updatable lookup table table-only only the latter is encrypted and aggregated, while the static dictionary remains local and requires neither sharing nor encryp- tion; and tineefoor-iniimm---ae Encrypt (PrME), which applies encryption-awart pruning to minimize encrypted parameters via consistent, history-guided masks. Experiments show that DictPFL reduces communication cost by 402-748x and accelerates training by 28-65x x compared to fully encrypted FL., while outper- forming state-of-the-art selective encryption methods by 51-155x in overhead and 4-19x in speed. Remarkably, DictPFL's runtime is within 2 of plaintext FL. demonstrating-ffe the first time-that HE-based private federated learn- ing is practical for real-world deployment. The code is publicly available at https://githut. www/UCF-oe-/. COmn/UCF-L--ReeeBB0 Mtl-Research/DictPrI
Introduction
Federated Learning (FL) [1] was introduced to enable collaborative training of shared machine learning model among different data owners (e.g.. hospitals or banks), where model gradients (or weights), rather than raw data, are shared address privacy concerns. However, even sharing gradi- ents poses privacy risks, as attackers could potentially exploit this information. For instance, model inversion (or gradient inversion) attacks 3 have demonstrated the feasibility of reconstructing client's original training data from the gradients shared by clients. In such scenarios, the server or users with access the server can act as potential attackers.
To protect the privacy of clients' gradients during aggregation and enable private FL, various privacy- preserving primitives such Differential Privacy (DP) [4] 5), Secure Multiparty Computation (MPC) 71 and Homomorphic Encryption (HE) 20 have been utilized. Among these meth- ods, HE is especially appealing in cross-silo settings 21 it provides non-interactive privacy
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

protection without the eecuurcc--rivaaee trade off associated with DP 12 24 and without requiring the assumption of trusted servers, as MPC 28). HE-based
federated learning 21 29 31, locally updated gradients are encrypted by clients before being shared with the server, allowing the server to perform homomorphic aggregation directly on ciphertexts. Despite its security benefits, HE introduces significant overhead: ciphertext expansion increases communication costs by to orders of magnitude, while encryption, decryption, and homomorphic aggregation impose high computational costs [213]] 31.
Server
Server
No encrypt! privacy issue
BEB Σ D Aggregation
Σ
Minimum encryption privacy guaranteed
Σ
HE Aggregation
HE Aggregation
+
X Pruned
X
Pruned lookup table
Less Less-sensitive
Sensitive
Less-sensitivv
Sensitive
Dictionary
lookup table
Dictionary
Client
Client
Client 1
Client K
(a) Prior HE-based PPFL with Gelect-and-Enorypt (SoE) strategy
(b) Our DictPFL with Decompose coreeefooooo--------- Eiccool-EEeypttttoe (DePE) and Prune-for conne-oor-Miiinnn Minnium Encrypt (PriME) strategy
Figure 1: (a) Prior HE-based FL 31 encrypts only deemed sensitive gradients. The less-sensitive weights are shared in plaintext, which may lead to privacy concerns. (b) In contrast, our DictPFL min- imizes encryption while ensuring privacy guarantees through Ceccmppos--forPPattiaa Pnntaaaall-Enerypt (DePE) and Prune-for-Me for- -nccypt (PrME). DePE decomposes gradients into a frozen dictionary and a trainable lookup table, with only the lookup table being encrypted and shared for aggregation. PrME further prunes the lookup table parameters on the client side to reduce encryption costs.
Prior efforts improve the efficiency of HE-based FL often compromise privacy. The state-of-the-art method, FedML-HE [II, as illustrated in Figure I (a), adopts Sereec-and-Encryye (SaE) strategy: clients pre-compute sensitivity scores for model parameters and encrypt only the gradients of the most sensitive subset (e.g., the top 10%), while transmitting the remaining parameters in plaintext. However, these unencrypted gradients still expose private information. As shown Figure 7 (a), when 30% of gradients remain unencrypted under FedML-HE, gradient inversion attacks can reconstruct images with up to 23% similarity to the originals. Moreover, the pre-computed sensitivity scores fail to capture dynamic sensitivity shifts during training, as parameter updates continually alter their privacy relevance. Consequently, enerypting all transmitted gradients remains essential to eliminate leakage. Although the SaE strategy achieves lower communication overhead and faster training than fully encrypted methods, inevitably exposes privacy risks due to shared plaintext gradients.
To address this challenge, we propose DictPFL, as shown in Figure I (b). which ensures that all shared parameters are fully encrypted to guarantec privacy while minimizing the number of certompss--oo-Pariicce Partial-Encrryt (DePE) and Prune-for-
shared parameters through two modules: Minimum-Eiie Encuypp (PrME). DePE decomposes the model weights be trained into a globally consistent dictionary, which is identical across all clients, and a lookup table, which each client trains independently. Only the encrypted gradients of the lookup table are shared with the server for aggregation, while the globally consistent dictionary remains frozen and is never shared. Building on DePE, PrME further reduces encrypted lookup tables through consistent pruning across clients. Unlike plaintext-level pruning in FL. [2]34 where clients perform pruning locally and the server aligns the retained gradients before aggregation, HE-based FL presents unique challenges: retained gradients are batch-encrypted into ciphertexts in SIMD format, preventing the server from aligning them without decryption. PrME addresses this by pruning based on shared global gradient history, ensuring consistent indices. Additionally, dynamic probabilities are assigned to the pruned parameters, allowing for their potential reintroduction in future rounds and mitigating the negative effects of premature pruning. Since the pruned lookup tables are significantly smaller than the full model weights, and all transmissions are encrypted, this approach substantially reduces the number of ciphertexts without compromising privacy.
Extensive experiments demonstrate that DictPFL. delivers substantial performance gains over the state- of-the-art FedML-HE [31 across diverse tasks, including (i) image recognition, (ii) text classification,

---

and (iii) text generation, Compared with fully encrypted frameworks [31, DictPFL outperforms
the selectively enerypted HedML-HE [3 by lowering communication overhead by 51-155x and speeding up training by 4-19x. Remarkably, DictPFL. introduces less than a 2x training-time increase even relative to plaintext FI., demonstrating that homomorphic mncrytiion-commonlyy considered prohibitively expensive-con can in fact be practical for federated learning at scale.
2 Background and Motivation
2.1 Privacy-preserving Federated Learning
Federated Learning (FL) enables collaborative training among distributed clients without directly sharing their datasets. In this framework, clients train their models locally and send gradients (or model updates) to a central server, which aggregates them using algorithms such as FedAvg (36] and FedSGD M. However, the direct exposure of local gradients the server poses severe privacy risks [37] For instance, with access to a client's local gradients, the server can perform model reconstruct the client's dataset.
inversion attacks
38
Several methods have been proposed to protect the gradients transmitted between clients and the server. One strategy employs Differential Privacy (DP) [22 24 by injecting noise into the gradients before sharing them. Although DP imposes minimal computational overhead, it inevitably degrades model performance due the added noise. Secure Multi-Party Computation (MPC) 25 28 ensures that the server can access only aggregated gradients rather than individual ones. However, the aggregated gradients remain exposed to the server, and the reliance on multiple non-colluding servers makes MPC unsuitable for single-server settings.
Another approach leverages Homomorphic Encryption (HE) [2T B1 to encrypt gradients on the client side, en- abling the server to aggregate them without decryption. HE provides end-to-end protection by securing gradient trans- mission, aggregation, and server storage. This protection addresses multiple security threats, including adversaries in network communications, multi-tenant vulnerabilities during computation on servers, and insider attacks on stored data. While platforms such IBM FL [39] and Nvidia FLARE (35) have explored integrating HE into FL, they fail to address its significant overhead. As shown in Figure HE-related operations dominate training time, and eiphertext expansion substantially increases communication costs. Reducing HE's computational and communication overhead is key to realizing its practical benefits in FL.
Ciphertext
FedML-HE Plaintext
Train
Aggregate
DictPFL
Comm.
En-Decrypt
20
60
80
180
200
Running Time (Minutes)
Figure 2: Time breakdown for train- ing a ViT on GTSRB.
2.2 Efficient HE-based Federated Learning
Many efforts have been made to improve the efficiency of HE-based FL. These approaches can be broadly classified into two categories: encryption-scheme optimization and algorithmic optimization.
45] are widely studied techniques within the realm of encryption-schent schioee optimization for HE-based FL. Quantization reduces communication costs by converting high-precision gradients into low -ww-precinoon values, whereas packing (also referred to as batching) consolidates multiple local gradients into a single plaintext, significantly reducing the number of plaintexts that need to be encrypted and transmitted.
Quantization 21 40/43) and packing [2]
Algorithmic optimization involves tailoring efficient strategies based the characteristics of the machine learning model, and our DictPFL. falls into this category. The state-of-the-ard work, FedMI.- HE (I] proposes selectively encrypt the gradients based on rrivaiy-sensitive scores, i.c.. Select- and-Encrypt (SaE), as shown in Figure (a). However, it suffers from several critical limitations. First, privacy-sensitive scores are computed once before training and remain static throughout the training process. This static approach fails to account for how weight sensitivity changes during training, because weights classified as non-sensitive on the initialized model may later become critical for privacy protection. Most critically, it cannot ensure complete privacy protection. Since only the gradients of selected parameters are encrypted, the remaining gradients are transmitted in plaintext, leading to inevitable information leakage and making it impossible to guarantee privacy protection regardless of which gradients are selected for encryption. Additionally, as illustrated in Figure 2 although -EE substantially reduces the communication overhead and HE operations
3

---

(including aggregation, encryption, and decryption) by factor of ten when only the top 10% of sensitive parameters are encrypted, these overheads induced by ciphertexts are still primary bottlenecks in the training process. In contrast, our DictPFL. effectively reduces HE-related overhead and achieves efficiency comparable to non-private plaintext Fl..
2.3 Motivation
As illustrated Figure communication and computation overheads caused by ciphertexts become the main bottleneck in HE-based FL.. Although the state-of-the-art FedML-HE [31] attempts to improve efficiency by selectively omitting encryption for partial parameters, not only compromises privacy but also continues to struggle with significant HE-induced communication and computation overheads. To achieve higher efficiency without sacrificing privacy, we focus on reducing the total number of trainable parameters. Guided by this principle, we propose DictPFL, which employs two strategies: cerompose-for-Parre Partial-Encrypt (DePE) (Sec. (Sec./4.I) to decompose gradients and Prune-for Minimum--Eccyan (PrME) (Sec. 4.2) to prune the gradients of parameters with minimal updates.
Preliminaries
3.1 System Overview
Same with FedML-HE [31], the workflow of HE-based privacy-preservve federated learning begins with clients using a trusted key authority to generate a public-secret HE key pair. During each training iteration: (1) clients compute local gradients; (2) these gradients encrypted with the public key and transmitted to the server; (3) the server aggregates the encrypted gradients; and (4) the aggregated ciphertext broadcast back the clients, who decrypt it using their secret keys and update their local models with the decrypted result.
3.2 Threat Model
We consider a semi-honest adversary A that may corrupt the server, which is the same as the setting of FedML-HE (31. While A follows the protocol, it attempts to infer private information from benign participants. Security guarantees ensure A learns no information from the data of clients.
4 DictPFL
DictPFL consists of two modules: fooomppoe-foo--Prrial-----i- Encrypt (PrME). DePE decomposes model weights into fixed global dictionary D and a trainable lookup table T. Only the encrypted gradients of T transmitted and aggregated, whereas D remains identical across clients and never leaves local devices. PrME further reduces encryption cost by pruning parameters with persistently small gradients, using shared historical statisties to ensure consistent pruning across clients. Together, these two modules ensure that all transmitted gradients are encrypted and all unencrypted ones remain strictly local, while significantly reducing the number of ciphertexts exchanged between clients and the server to achieve high efficiency without compromising privacy.
(DePE) and Prume-for-Mimnnum:
4.1 Decompse-for-Partii-Feeeee (DePE)
Model weight decomposition, representing weight matrix W as a linear combination of vectors from compact dictionary 1) and a sparse lookup table T, is a proven strategy for parameter reduction in inference [46 48 The key insight lies in reducing the inherent redundancy in weight parameters: corre- lated parameters can represented as sparse linear combinations of dictionary of vectors. We adapt this principle to HE-based FI., where reducing the dimensionality of trainable parameters directly minimizes the number of ciphertexts.
0.5
0.2
b.
+0.2
+0.5
0.5
Dictionary
Table
Weights
Aggrogation
Figure 3: Representing the weight matrix W with dictionary and lookup table T.

---

Constructing W with D and T. Figure demonstrates the construction of the weight matrix W R=xm using a dictionary D E R and a lookup table E R-x" Each column vector W[]]]] of W is derived through linear combination of the r vectors in D, weighted by the corresponding scalars in the i-th column of T, denoted [ This process is formally expressed by:
Wolld
D T+]
Take Figure 3 as an example. Given and the i-th column of T [0.3. 0.5, the correspond- ing i-th column of weights in W can be represented as W|:[[i] 0.3. D ||: +.2 -.. By reducing r, the dictionary size, we effectively decrease the number of trainable parameters and thus reduce the communication overhead associated with ciphertexts.
To ensure that the dictionary D contains critical and generalizable weight vectors while reducing parameter redundancy, we employ a truncated SVD factorization to decompose the weights be trained, i.e., Wo. which has dimensions n m,, into smaller dictionary D and a lookup table T". Specifically, Wo is approximated as U,E,V V where Ur, Σ,, and V. correspond to the top-r singular values and vectors, thus reducing the dimensionality to for D and x for TV,
Factorization of Dictionary and Lookup Tables.
Wo R U,E.V.
(2)
DTT' = SVD U.
(3)
DePE initializes D U,E, and T as V.T according Equation However, directly freezing D and training T' can lead to suboptimal performance due to the information loss inherent SVD truncation, particularly when is much smaller than m or n.. To counteract this, retain the original weight Wo and initialize T by zeroing out 7". This strategy allows for the construction of W as Wo + D T, with D remaining static and shared among all clients, while T is updated locally and aggregated on the server. By selecting a smaller r, we significantly reduce the communication overhead for encrypted parameters, as encryption is only required for the m entries in T.
Large gradients
Small gradients
1st Round
50th Round
Demiiy
Misaligned
4
-Sa-5
2
Local gradients
Pruned local gradients
Encrypted local gradients
Treining Round
Gradients
Clients
Server
Evolution of Gradients Magnitude
Change Gradients Distribution
(c) Appresation after Pruning HE Setting
Figure 4: (a) As training progresses, parameters that initially have large gradients gradually transition to having smaller ones. (b) Concurrently, the number of parameters with substantial gradients decreases significantly. (c) An example of failed aggregation caused by different pruning locations on clients A and B.
4.2 Prumefor-MMnnmmm---eryy (PrME)
As DePE training progresses, the number of parameters with large gradients gradually declines, as shown in Figure (a). By the 50-th training round, only small subset of parameters still exhibit gradients exceeding 10-5, as shown in Figure (b). Encrypting and transmitting all gradients the server for aggregation, including those of parameters that no longer change significantly, introduces unnecessary redundancy. By enabling clients to upload only substantial gradients, communication overhead can be dramatically reduced.
(a).
Existing gradients pruning methods in plaintext federated learning involve clients independently pruning their smallest local gradients before transmission to the server for aggregation. Since clients possess different local gradients, they may prune parameters at different positions, necessitating the sharing of pruning indices with the server to ensure proper aggregation. However, implementing such methods to Hi--basd federated learning presents two fundamental challenges. First, indices must be encrypted to protect privacy, while encrypted indices force the server to perform non-linear operations (e.g., comparing encrypted indices to match) alongside linear operations (e.g., aggregation), hybrid workflow that incurs prohibitive computational overhead [4 50). Second, the SIMD batching
5

---

mechanism, which packs multiple plaintext gradients into several slots of a single ciphertext, renders index-specifis operations infeasible. Since HE aggregation occurs slot-wise, gradients occupying the same slot across clients are combined automatically, regardless of their indices.
Figure 4 (c) illustrates the above challenges of pruned HE: aggregation. Consider scenario where client A encrypts and uploads gradients from positions and 3, while client B encrypts and uploads gradients from positions and 4. The server cannot perform correct aggregation because the eiphertext slots are misaligned, and the encryption prevents any coordination or realignment of the gradients. To ensure consistent gradient pruning across clients, they require an identical metric for determining which gradients to prune. The optimal approach would involve clients pruning their local gradients based on current round global gradients. However, clients cannot access the current round global gradients until after sharing their complete local gradients with the server for aggregation. This creates a dilemma: clients cannot prune independently as leads inconsistencies, nor can they rely on global gradients to coordinate pruning.
has not been pruned
has been pruned
Gradients
5E-6
Gradients
5E-6-
Gradients
5E-6
Gradlents
5E-6
0E-0
0E+0
0E+0
DE40
-5E-6
-5E-6
-SE-6
100
200
300
100
200
300
100
200
300
100
200
300
Round
Round
Round
Round
(a) Without Pruning
(b) Pruning based on last round
Pruning by TIP
(d) Pruning by PIME (TIP4HRC)
Figure 5: Evolution of parameter's global gradients under different pruning strategies. Green background indicates the parameter is pruned (excluded from aggregation), while gray background indicates the opposite. Larger green areas reflect more overhead reduction. Closer alignment of gradient trends with the baseline (a) signifies preserved convergence performance.
Temporal Inactivity Pruning (TIP). To resolve this dilemma, clients require shared pruning metric that is independent of the current round's global gradients. A straightforward solution is to base pruning decisions on the last round's s global gradients, which are identical across clients and accessible before aggregation. Specifically, clients prune local gradients corresponding to parameters with the smallest s% magnitudes from the prior global gradients. However, parameters showing minimal activity in one round may experience significant updates in subsequent rounds, leading to unintended removal if pruning decisions rely exclusively on last round gradients. For instance, as illustrated in Figure 5 (b), the parameter with a small gradient magnitude in an earlier round may be pruned, despite gradient resurgence in later rounds, as indicated Figure 5 (a).
To mitigate the influence of transient fluctuations and retain critical gradients, we introduce a temporal windowing strategy that leverages information from the previous т consecutive rounds. Clients identify parameters whose gradients fall within the smallest 8% across all J rounds (referred as pruning patience). Formally, the pruning mask for parameter wi round is defined as:
So if Mr. = {i
< <0,,1h))= =T
otherwise
Here, Mi,, = O indicates that the local gradient of w; is pruned, while Mi,, retains its local gradient for aggregation. The Swt,e-x denotes the global gradient of parameter wi round k. and 1 is the indicator function. The threshold 0st- dynamically adapts as the (100-s)-th percentile of jo As shown in Figure 5 (c), the pruning is postponed to a later round when gradients exhibit more stable behavior, thereby preserving gradients that regain significance after initially being considered for pruning.
Holistic Reactivation Correction (HRC). Although TIP reduces communication overhead while preventing premature pruming by chance, still has an inherent limitation: once a parameter pruned, its local gradients no longer participate in aggregation. Consequently, its global gradient magnitudes remain zero subsequent rounds, effectively excluding it permanently. This irreversible pruning can hinder training convergence, parameters with substantial gradients in later rounds may no longer be updated. For example, in Figure (a), the example parameter may have siginificant gradient magnitudes even after the 100-th round.

---

To mitigate the performance loss caused by irreversible pruning, we propose a dynamic reactivation scheme, Holistic Reactivation Correction (HRC). Instead permanently excluding pruned parameters. HRC assigns each pruned parameter wi; reactivation probability Pt, which is dynamically adjusted based on its aggregated global gradients ow;st after reactivation:
pi[t] x 8,3 min (p:(4/3,1)
if Jows,el 0s,c otherwise
pilt + 1|
(5)
Here, 3 is a decay factor less than 1. When a prumed parameter is reactivated, the client uploads its accumulated local gradients since the pruning round for aggregation and gets the current round's global gradients ,w;,t. This approach preserves small gradients that, while individually minor, can meaningfully accumulate over time, rather than discarding these gradients, maintaining them locally for future aggregation helps convergence. If jōws,el < d.... indicating that the parameter's s cumulative global gradients remain small even after reactivation, the reactivation probability p; decreases, discouraging further reactivation. Conversely, if |ōw.al 2 Os... p; increases, encouraging the update of this parameter to rejoin aggregation. This adaptive mechanism mitigates information loss from premature pruning by flexibly adjusting the likelihood of reactivation. Although HRC introduces some uncertainty, consistency across clients can be easily maintained by preserving a shared random seed for the pruning mask. Notably, our PrME does not need share pruning mask to server, climinating the risk of attacks via plaintext mask, c.g., inferring sensitive patterns from pruned parameter locations.
Experimental Methodology
Datasets. We conduct experiments on three image classification tasks: CIFAR-10 511, GTSRB (52)
and Diabetic Retinopathy (53]. as well as AG's News (54) for sentence classification and Meta- MathQA 55 for text generation. The experiments are performed under varying levels of data heterogeneity and different numbers of clients. We generate homogeneous data splits by randomly assigning training examples to individual clients without replacement. For heterogencous settings, we simulate data heterogeneity by sampling the label ratios from a Dirichlet distribution with a symmetric parameter, following the [56]. In both settings, each client holds the same number of samples, following (57]
Models. We perform DictPFL on multiple prevalent transformer- based models including, ViT [88 designed for image recognition, BERT [59], and TinyLlama [60] for natural language processing.
Baselines. We compare DictPFL. with three baselines: FedHE-Full 35 which trains the entire model and encrypts all gradients; FedHF--Top2, fine-tuning only the last two layers; and Fe (3I,, which encrypts subset of gradients (10% unless specified otherwise) while leaving the rest in plaintext.
Evaluation Metrics. We assess the efficacy of our proposed DictPFL by comparing its communica- tion overhead, training time, and model accuracy against existing methods. For privacy evaluation, compare DictPFL with FedML-HE [1 in terms of potential privacy leakage. We utilize recovered image similarity scores derived from 1 1 LPIPS, where the Learned Perceptual Image Patch Similarity (LPIPS) 661 measures discrepancies between reconstructed and original images. Therefore, higher scores indicate greater similarity and consequently, higher privacy risks.
Hyperparameters. Unless otherwise specified, we set the dictionary size r to 4, the pruning ratio 5% to 70%, the pruning patience to 3, and the reactivation probability scaler 3 to 0.2. Detailed analyses these hyperparameters are provided Section 6.2
HE Implementation. We adopt the CKKS homomorphic encryption scheme with bootstrapping 62
64
implemented via OpenFHE (65]. The scheme is configured for 128-bit security following the
Homomorphic Encryption Standard (66], with a cyclotomic ring dimension of N 2 ciphertext
modulus of 1555 bits, and multiplicative depth
12. Each ciphertext contains N/2 = 32, 768
slots, enabling parallelized SIMD operations
677 Data encoding follows the approach in [68 All
experiments were conducted on an AMD Ryzen Threadripper PRO 3955WX processor (2.2 GHz) with 125 GB of memory.
7

---

FedHE-Full
FedHt-Finneune
FedMLL-E (insecure)
DictPFL
99
100
ACC (%)
97
ACC (96
78
ACC (%)
75
95
50
74
93
25
91
70
10
10"1
10
10
10
10
10"
10
10
10
10
10"
10
10
10
Communication (GB)
Communication (GB)
Communication (GB)
(a) CIFAR-10
(b) Diabetic Retinopathy
(c) GTSRB
Figure 6: Efficiency comparison of different federated learning frameworks, in terms of accuracy versus communication overhead three datasets using the ViT model. Higher efficiency is indicated by higher accuracy for the same communication or achieving the same accuracy with less communi- cation, shown by lines closer to the upper left corner. Communication is quantified by the total amount of data exchanged, including both plaintexts and ciphertexts, training iterations.
6 Results
6.1 Main Results
Comparison with Existing Works. To demonstrate DictPFL's effectiveness, we compare it with other HE-based FL frameworks on the CIFAR-IO, Diabetic Retinopathy, and GTSRB datasets using the ViT-16 model within a 3-client homogeneous setting. All experiments are conducted on the same pre-trained model to ensure fair comparison. Figure provides an overall comparison. Notably, DictPFL significantly and consistently reduces communication overhead compared to the baselines without sacrificing accuracy. Specifically, FedHE-Full has the highest communication. FedHE-Top2, which fine-tunes only the last two layers, shows reduced overhead but underperforms, because freezing most layers limits learning capacity, particularly on datasets that diverge from those used in pre-training. For instance, it achieves only 58.9% accuracy on GTSRB versus DictPFL's 95. 27%.
DictPFL achieves a 98.3% average reduction in communication overhead compared to the state-of-the-art FedML-HE (encrypting 10%), while maintaining the same level of accuracy. Although FedML-HE also reduces communi-
Recoverod Imaga Store
1.0
FedHE-Full
DictPPL
Plaintext
10
FedMLL-E (insecure)
DERT
Time (Se)
0.6
Reclee-11
No Leakskk
Lotoe
fandom/Guass
cation costs, it does so at the expense of pri- vacy by exposing part of the gradients in plain- text. DictPFL, on the other hand, fully preserves privacy. This is further demonstrated in Fig- ure 7 (a), which highlights the vulnerability of FedML-HE to statc-of-the-e gradient inversion attacks (69]. Notably, DictPFL can prevent such privacy leakage for any data type, not only for vision tasks.
10
0.2
0.8 Communication Ratio (A)
0.4
0.6
1.0
10a
10"
100
102
10.0
Model Size
(b)
Figure 7: (a) Gradient inversion attacks against FedML-HE and DictPFL. The communication ra- tio is the communication overhead relative to en- crypting the full-size model gradients in FedHE- Full. (b) Communication overhead of DictPFL and the baselines on models of different sizes.
In addition to VIT, we evaluate several other models, as shown in Figure 7 (b). The results show that DictPFL consistently outperforms the baselines across models of different scales. Compared with the fully encrypted baseline FedHE-Full, DictPFL. reduces communication by 402 to 748 times and accelerates training by 28 to 65 times. It also outperforms the selectively encrypted bascline FedML-HE by reducing overhead by 51 to 155 times and speeding up training by 4 to 19 times.
Breakdown Analysis. In Figure 8 we break down the training time for various HE-based FL frameworks under both LAN and WAN settings. In FedHE-Full, where all gradients are encrypted, communication and ciphertext-relater operations (eneryption, decryption, and aggregation) dominate the training time. FedHE-Topp reduces communication and ciphertext-related operations by fine- tuning the last two layers, but this comes at the cost of reduced accuracy, achieving only 58.9%. On the contrary, our proposed DePE and PrME techniques significantly reduce the number of ciphertexts, resulting in total training time that is to orders of magnitude lower than that of other baselines while maintaining comparable level of accuracy.
8

---

FedHE-Full
FedHE-Top2
FedML-HE
DePE [Dic:PFL)
LAN (1Gbps, 0.5mss)
WAN (400Mbbs, 10ms)
95
10"
Time (ee
Time (5e))
80
Acc (cc(8))
Time (Sed
Time (Sec)
80
Arc
Train
Comm. En-Decrypt
Agg.
Total
Train
Comm.
En-Decrypt
Agg.
Total
Figure 8: Training time breakdown of VIT on GTSRB under LAN and WAN settings.
6.2 Ablation Study
In this section, we explore the design space of DictPFL and study the impact of various settings on its performance. Unless otherwise specified, all experiments are conducted using the Diabetic Retinopathy dataset within a 3-client homogeneous setting within 10 rounds, and follows the default hyperparameter settings detailed in Section
Hyperparameters of DePE. The dictionary size is a crucial hyperparameter in our DePE. A larger dictionary captures more comprehensive representations of gradients, enhancing accuracy but in- creasing overhead. As shown in Table ! even small dictionary with r = 4 achieves commendable training performance, e.g., an accuracy of 81. 81.99%, close the 82. 74% achieved by FedHE-Full. This efficacy stems from the dictionary' 's abilitty to retain essential information corresponding to the largest singular values.
Table Ablation on dictionary size r.
Table Ablation on pruning patience T.
r
Accuracy (
Comm. (GB) +
Time (min) 4
Accuracy (%)
Comm, (GB) |
Time (min) 4
2
74.26.0.5
0.046
6.11+0.1
80,55.4o.6
0.001
6.26+0.1
81,99
0.088
6.2310.1
82.29.0.33
0.003
6.3610.1
82,67 10.2
0.160
6.4210.2
82.67
0.160
6.4210 2
16
82,71
0.332
7.27
82.77
0.474
6.92
Hyperparameters of PrME. We explore the impact of pruning ratio s% and pruning pa- tience т PrME. A higher s% results in more minor gradients being pruned, whereas lower value preserves them. As shown in Figure 9 without PrME (prune 0%), training converges rapidly within rounds, but each round incurs the highest communication cost. Pruning 70% drastically reduces communica- tion overhead but significantly affects accuracy. By contrast, pruning 20% preserves accuracy results in far less communication reduction compared to the 70% pruning scenario. Notably, with our HRC reactivation scheme, prematurely pruned gradients in earlier rounds can be selec- tively reintroduced in later rounds. This enables the model achieve accuracy similar to the 20% pruning scenario while achieving the communi- cation efficiency of the 70% pruning ratio.
Prune 0% - Prune 20%
Prune 70%
Prune 70% w/ HRC
100
Communicanin awtaanatiinnnnnnn (GB)
Acc (%)) (86)
75
50
25
0
5
19
5
10
15
19
Rounds
Rounds
Figure 9: Ablation on the pruming ratio.
Table 2 studies different pruning patience r. Higher 7 values delay the pruning of gradients, reducing accuracy degradation but limiting communication reduction. Notably, setting 3 already results in a small accuracy loss. This resilience can be attributed to our HRC, which mitigates the impact on accuracy by reintroducing pruned gradients, effectively correcting errors over time.
Table 4 Appendix A.2 showcases that our PrME works well under various reactivation proba- bility scalers B. For different numbers of clients and heterogeneous levels. we show the results in Appendix An3 and A.4 which show that DictPFL. performs well across different client scales and heterogeneous settings.
6.3
Other Experiments
The results for text tasks and large language models, including classification and generation tasks are in Appendix A.5) DictPFL outperforms all the baselines on language tasks. In Appendix A.7 we compare DictPFL with other non-HE based FL.

---

7 Conclusion
In this work, we present DictPFL. novel framework for efficient HE-based FL. To address the prohibitive ciphertext-related overhead and eliminate information leakage, we propose Decompose- Pe--Paral-Encrrpp (DePE), which decomposes model weights into a static dictionary and a trainable lookup table. Only the small lookup table is encrypted and shared for aggregation, while the dictionary never transmitted. To further improve communication efficiency, we propose Prume-for-Minmume for-Minimum- Encrypt (PrME), which prunes gradients based on their long-term activity to minimize redundant ciphertext operations. Compared with the fully encrypted baseline. DictPFL accelerates training by up to 65 and outperforms the selectively encrypted FedML-HE by up to 19x while maintaining accuracy and fully eliminating privacy risks from partial plaintext gradient transmission, achieving a runtime only 2x that of plaintext FL..
Discussion
Broader Impact. The paper introduces DictFPL, method designed to reduce the computational and communication overheads associated with protecting federated learning shared weights using homo- morphic encryption. This approach enhances privacy protections without compromising accuracy, making a more feasible solution for large-scale, real-world applications. By ensuring that sensitive weights remains private, DictFPL accelerate the adoption of federated learning across industries such as healthcare, finance, and beyond, while fostering trust Al systems and promoting global data privacy.
Limitations. Future work could explore broader scenarios, such as cross-device FL with constrained client resources non-transformer model families. Moreover, since DictPFL employs fixed shared dictionary, extending to a dynamic dictionary design could enhance adaptability in highly heterogeneous client environments and improve model personalization.
Acknowledgement
This work was supported in part by NSF CSR-2413232. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of grant agencies their contractors.
References
[1] Reza Shokri and Vitaly Shmatikov. Privacy-preserving deep learning. In Proceedings the 22nd ACM SIGSAC conference on computer and communications security, pages 1310-1321, 2015.
[2] Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. Advances in neural information processing systems, 32, 2019.
[3] Shanghao Shi, Ning Wang, Yang Xiao, Chaoyu Zhang, Yi Shi, Y Thomas Hou, and Wenjing Lou. Scale-mia: scalable model inversion attack against secure federated learning via latent space reconstruction. arXiv preprint arXiv:2311.088: 1005088, 2023.
[4] Cynthia Dwork. Differential privacy. In International colloquium on automata, languages, and programming, pages 1-12. Springer, 2006.
[5] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. Proceedings of the 2016 ACM SIGSAC conference computer and communications security, pages 308-318, 2016.
[6] Ronald Cramer, Ivan Bjerre Damgård, and Jesper Buus Nielsen. Secure multiparty computation and secret sharing. Cambridge University Press, 2015.
[7] Yehuda Lindell. Secure multiparty computation. Communications of the ACM, 64(1):86-96. 2020.
10

---

[8] Qian Lou and Lei Jiang. She: A fast and accurate deep neural network for encrypted data. Advances neural information processing systems, 32, 2019.
[9] Qian Lou, Bo Feng, Geoffrey Charles Fox, and Lei Jiang. Glyph: Fast and accurately training deep neural networks encrypted data. Advances in neural information processing systems, 33:9193-9202, 2020.
[10] Qian Lou, Wen-jie Lu, Cheng Hong, and Lei Jiang. Falcon: Fast spectral inference encrypted data. Advances in Neural Information Processing Systems, 33:2364-2374, 2020.
[11] Yancheng Zhang, Jiaqi Xue, Mengxin Zheng, Mimi Xie, Mingzhe Zhang, Lei Jiang, and Qian Lou. Cipherprune: Efficient and scalable private transformer inference. arXiv preprint 16782 2025.
[12] Qian Lou and Lei Jiang. Hemet: A comoomophic--eccrytionnniit mobile neural network architecture. In International conference on machine learning, pages 7102-71 7000. PMLR, 2021.
privacy-presenve
[13] Qian Lou, Yilin Shen, Hongxia Jin, and Lei Jiang. Safenet: A secure, accurate and fast neural network inference. In International Conference on Learning Representations, 2020.
[14] Jiaqi Xue, Yancheng Zhang, Yanshan Wang, Xueqiang Wang, Hao Zheng, and Qian Lou. Cryptotrain: Fast secure training on encrypted dataset. In Proceedings of the 1st ACM Workshop on Large Al Systems and Models with Privacy and Safety Analysis, pages 97-104, 2023.
[15] Yancheng Zhang, Mengxin Zheng, Yuzhang Shang. Xun Chen, and Qian Lou. Heprune: Fast private training of deep neural networks with encrypted data pruning. Advances in Neural Information Processing Systems, 37:5 10.510443-1084, 2024.
[16] Muhammad Husni Santriaji, Jiaqi Xue, Yancheng Zhang, Qian Lou, and Yan Solihin. Dataseal: Ensuring the verifiability of private computation on encrypted data. 2025 IEEE Symposium on Security and Privacy (SP), pages 2378-2393 IEEE, 2025.
[17] Mengxin Zheng, Qian Lou, and Lei Jiang. Primer: Fast private transformer inference on encrypted data. In 2023 60th ACM/IEEE Design Automation Conference (DAC), pages 1-6. IEEE, 2023.
[18] Jiaqi Xue, Lei Xu, Lin Chen, Weidong Shi, Kaidi Xu, and Qian Lou. Audit and improve robustness of private neural networks encrypted data. arXiv preprint arXiv:2209.0990 2022.
[19] Ardhi Wiratama Baskara Yudha, Jiagi Xue, Qian Lou, Huiyang Zhou, and Yan Solihin. Boost- com: Towards efficient universal fully homomorphic eneryption by boosting the word-wise comparisons. In Proceedings of the 2024 International Conference on Parallel Architectures and Compilation Techniques, pages 121-132. 2024.
[20] Mayank Kumar, Jiaqi Xue, Mengxin Zheng, and Qian Lou. Tfhe-coder: Evaluating Ilm-agentic fully homomorphic encryption code generation. arXiv preprint arXiv:2503.122117 2025.
[21] Chengliang Zhang, Suyi Li, Junzhe Xia, Wei Wang, Feng Yan, and Yang Liu. (BatchCrypt): Elficient homomorphic eneryption for (Cross-Sille Silo federated learning. In 2020 USENIX annual technical conference (USENIX ATC 20), pages 493-506, 2020.
[22] Stacey Truex, Nathalie Baracaldo, Ali Anwar. Thomas Steinke, Heiko Ludwig, Rui Zhang, and federated learning. In Proceedings of the 12th ACM workshop on artificial intelligence and security, pages 2019.
Yi Zhou. hybrid approach to
[23] Stacey Truex, Ling Liu, Ka-Ho Chow, Mehmet Emre Gursoy, and Wenqi Wei. Ldp-fed: Federated learning with local differential privacy. In Proceedings of the third ACM international workshop edge systems, analytics and networking, pages 61-66, 2020.
[24] Youbang Sun, Zitao Li, Yaliang Li, and Bolin Ding. Improving lora privacy-preservin federated learning, In The Twelfth International Conference on Learning Representations.

---

[25] Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-presere machine learning. In proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 1175-1191, 2017.
[26] Jinhyun So, Chaoyang He, Chien-Sheng Yang, Songze Li, Qian Yu, Ramy E Ali, Basak Guler, and Salman Avestimehr. Lightsecagg: lightweight and versatile design for secure aggregation in federated leaming. Proceedings of Machine Learning and Systems, 4:694-720, 2022.
[27] Marcel Keller and Ke Sun. Secure quantized training for deep learning. In International Conference on Machine Learning, pages 10912210938. PMLR, 2022.
[28] Hossein Fereidooni, Samuel Marchal, Markus Miettinen, Azalia Mirhoseini, Helen Möllering, Thien Duc Nguyen, Phillip Rieger, Ahmad-Reer Reza Sadeghi, Thomas Schneider, Hossein Yalame, et al. Safeleamn: Secure aggregation for private federated learning. In 2021 IEEE Security and Privacy Workshops (SPW), pages 56-62. IEEE, 2021.
[29] Haokun Fang and Quan Qian. Privacy preserving machine learning with homomorphic encryp- tion and federated learning. Future Internet, 13(4):94, 2021.
[30] Zhifeng Jiang, Wei Wang, and Yang Liu. Flashe: Additively symmetric homomorphic encryption for cross-silo federated learning. arXiv preprint arXiv:2109.00677 2021.
[31] Weizhao Jin, Yuhang Yao, Shanshan Han, Carlee Joe- Wong, Srivatsan Ravi, Salman Avestimehr, and Chaoyang He. Fedml-he: An efficient homoonorpii--ncrrppioobbbs privacy-preserving federated learning system. arXiv preprint arXiv:2303 2023.
[32] Alham Fikri Aji and Kenneth Heafield. Sparse communication for distributed gradient descent. arXiv preprint arXiv: 174.007000002,,, 2017.
[33] Ang Li, Jingwei Sun, Xiao Zeng, Mi Zhang, Hai Li, and Yiran Chen. Fedmask: Joint computa- tion and communication--efficeee personalized federated learning via heterogeneous masking. In Pruceedings of the 19th ACM Conference on Embedded Networked Sensor Systems, pages 42-55, 2021.
[34] Sameer Bibikar, Haris Vikalo, Zhangyang Wang, and Xiaohan Chen. Federated dynamic sparse training: Computing less, communicating less, yet learning better. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pages 6080-6088, 2022.
[35] Holger R Roth, Yan Cheng, Yuhong Wen, Isaac Yang, Ziyue Xu, Yuan-Ting Hsieh, Kristopher Kersten, Ahmed Harouni, Can Zhao, Kevin Lu, et al. Nvidia flare: Federated learning from simulation to real-world. arXiv preprint arXiv:2210. 13291, 2022.
[36] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication effiicient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017.
[37] Viraaji Mothukuri, Reza M Parizi, Seyedamin Pouriyeh, Yan Huang, Ali Dehghantanha, and Gautam Srivastava. survey on security and privacy of federated learning. Future Generation Computer Systems, 1 11::6196440, 2021.
[38] Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. Deep models under the gan: information leakage from collaborative deep learning. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pages 603-618, 2017.
[39] IBM. Ibmfi crypto. GitHub repository, 2022. Accessed: 2023-01-25.
[40] Wuxing Xu, Hao Fan, Kaixin Li, and Kai Yang. Efficient batch homomorphic encryption for vertically federated xgboost. arXiv preprint arXiv:2112.04261, 2021.
[41] Junhao Han and Li Yan. Adaptive batch homomorphic encryption for joint federated learning in cross-device scenarios. IEEE Internet of Things Journal, 2023.
[42] Changsheng Zhao, Ting Hua, Yilin Shen, Qian Lou, and Hongxia Jin. Automatic mixed- precision quantization search of bert. arXiv preprint 11244221 2021.
12

---

[43] Qian Lou, Feng Guo, Lantao Liu, Minje Kim, and Lei Jiang. Autoq: Automated kemee-wise neural network quantization. arXiv preprint arXiv: 15900,00699, 2019.
[44] Yoshinori Aono, Takuya Hayashi, Lihua Wang, Shiho Moriai, al. Privacy-preserving deep learning via additively homomorphic encryption. IEEE transactions on information forensics and security, 13(5):1333-133 1345, 2017.
[45] Changchang Liu, Supriyo Chakraborty, and Dinesh Verma. Secure model fusion for distributed learning using partial homomorphic eneryption. Policy-Bass Autonomic Data Governance, pages 154-179, 2019.
[46] Qian Lou, Ting Hua, Yen-Chang Hsu, Yilin Shen, and Hongxia Jin. Dictformer: Tiny trans- former with shared dictionary. In International Conference on Learning Representations. 2022.
[47] Qian Lou, Yen-Chang Hsu, Burak Uzkent, Ting Hua, Yilin Shen, and Hongxia Jin. Lite-mdetr: A lightweight multi-modal detector. In Proceedings of the IEEF/CVF Conference on Computer Vision and Patlern Recognition, pages 12206-12215, 2022.
[48] Yen-Chang Hsu, Ting Hua, Sungen Chang, Qian Lou, Yilin Shen, and Hongxia Jin. Language model compression with weighted low-rank factorization. arXiv preprint arXiv:2207.001 12, 2022.
[49] Yancheng Zhang, Xun Chen, and Qian Lou. Hebridge: Connecting arithmetic and logic operations in fv-style he schemes. In Proceedings of the 12th Workshop on Encrypted Computing & Applied Homomorphic Cryptography, pages 23-35, 2024.
[50] Mansour Al Ghanim, Jiaqi Xue, Rochana Prih Hastuti, Mengxin Zheng, Yan Solihin, and Qian Lou. Uncovering the hidden threat of text watermarking from users with loos-lin knowledge. arXiv preprint 2552:22000.66999 2025.
[51] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images.
[52] Sebastian Houben, Johannes Stallkamp, Jan Salmen, Mare Schlipsing, and Christian Igel. Detection of traffic signs in real-world images: The German Traffic Sign Detection Benchmark. In International Joint Conference on Neural Networks, number 1288.
[53] Varun Gulshan, Lily Peng, Mare Coram, Martin C Stumpe, Derek Wu, Arunachalam Narayanaswamy, Subhashini Venugopalan, Kasumi Widner, Tom Madams, Jorge Cuadros, et al. Development and validation of deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. jama, 316((2):2402222244 2410.
[54] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28.
[55] Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309.1222 12284, 2023.
[56] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint wxXiv:199.006335 2019.
[57] Geeho Kim, Jinkyu Kim, and Bohyung Han. Communication accelerated client gradient. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 12385-12394, 2024.
federated learning with
[58] Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint 1rXiv:2000.11920 2020.
[59] Jacob Devlin Ming-We Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, volume 1, page2 2. Minneapolis, Minnesota, 2019.
[60] Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. Tinyllama: An open-source small language model, 2024.
13

---

[61] Yangsibo Huang, Samyak Gupta, Zhao Song. Kai Li, and Sanjeev Arora. Evaluating gradient inversion attacks and defenses in federated learning, Advances in neural information processing systems, 34:7232-7241, 2021.
[62] Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song. Homomorphic encryption for arithmetic approximate numbers. In Advances in Eoptology-ASIACRRI 2017: 23rd International Conference on the Theory and Applications of Cryptology and Information Security, Hong Kong, China, December 3-7, 2017. Proceedings, Part 23, pages 409-437. Springer, 2017.
[63] Jung Hee Cheon, Kyoohyung Han, Andrey Kim, Miran Kim, and Yongsoo Song. A full variant of approximate homomorphic encryption. In Selected Areas in Cryplography-AAC 2018: 25th International Conference, Calgary, AB, Canada, August 15-17, 2018, Revised Selected Papers 25, pages 347-368. Springer, 2019.
[64] Jung Hee Cheon, Kyoohyung Han, Andrey Kim, Miran Kim, and Yongsoo Song. Bootstrapping for approximate homomorphic encryption. In Advances Annual International Conference on the Theory and Applications of Cryptographic Techniques, Tel Aviv, Israel, April 29-May 3, 2018 Proceedings, Part 1 37, pages 360-384. Springer, 2018.
CeyptoloyyEURRCCRER 2018: 37th
[65]
Ahmad Al Badawi, Jack Bates, Flavio Bergamaschi, David Bruce Cousins, Saroja Erabelli, Nicholas Genise. Shai Halevi, Hamish Hunt, Andrey Kim, Yongwoo Lee, Zeyu Liu, Daniele Micciancio, Ian Quah, Yuriy Polyakov, Saraswathy R. V., Kurt Rohloff, Jonathan Saylor, Dmiitry Suponitsky. Matthew Triplett, Vinod Vaikuntanathan, and Vincent Zucca. Openfhe: Open- source fully homomorphic encryption library. Cryptology ePrint Archive, Paper 2022/915, 2022.
htttps://ppring iacr wwtp:rogg////2/////
[66] Martin Albrecht, Melissa Chase, Hao Chen, Jintai Ding, Shafi Goldwasser, Sergey Gorbunov, Shai Halevi, Jeffrey Hoffstein, Kim Laine, Kristin Lauter, et al. Homomorphic encryption standard. Protecting privacy through homomorphic encryption, pages 31-62, 2021.
[67] Nigel Smart and Frederik Vercauteren. Fully homomorphic simd operations. Designs, codes and cryptography. 71:57-81, 2014.
[68] Eric Crockett. A low-depth homomorphic circuit for logistic regression model training. Cryp- tology ePrint Archive, 2020.
[69] Yuxin Wen, Jonas A Geiping, Liam Fowl, Micah Goldblum, and Tom Goldstein. Fishing for user data in large-batch federated learning via gradient magnification. In International Conference on Machine Learning, pages 23668-23684, PMLR, 2022.
[70] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichhrr Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 14168, 2021.
[71] Tianshi Xu, Meng Li, and Runsheng Wang. Hequant: Marrying homomorphic encryption and quantization for communication-effciiee private inference. arXiv preprint arXiv:2401. 15970, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
We accurately describe the Abstract and Introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
In Appendix g
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
In Section
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Section
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer:
[Yes]
Justification:
Code is in the support material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeee //nips.com / ssiniiiiccypyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: Section 5
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Ahe results are based on 5 individual runs. We report the mean and variance the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httpg////.ooorr...c/////ll///lle
Question:
Answer: [Yes]
Justification: Appendix g discusses the broader impact.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
In Appendix g
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: In Section 5 cited the related code, datasets and models.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
21

---

Appendix
A More Experiments
A.1 Comparison without pre-trained weights
As shown in Table E even without using pre-trained weights, DictPFL achieves the highest accuracy among all methods, reaching 95.06%, compared to 94.17% for FedHE-FULL and 94.99% for FedML- HE. More importantly, DictPFL offers substantial efficiency gains: the total communication cost is reduced to 0.51 GB, while FedHE-FURE and FedML-HE require 720.72 GB and 73.62 GB, respectively. In terms of training time, DictPFL completes just 11.8 minutes, far less than 294.6 minutes for FedHE-FULA and 56.7 minutes for FedML-HE.
Table 3: Comparison with baselines on without pre-trained weights.
Acc. (%) f
Comm. (GB) 4
Time (min) &
FedHE-FULL
94.17
720.72
294.6
FedML-HE
94.99
73.62
56.7
DictPFL. (ours)
95.06
0.51
11.8
A.2 Different reactivation probability scale
Table studies different reactivation probability scalers 3. The result showcase the our PrME works well under different B.
Table 4: Ablation study on ,3 under 8% 70% and
B
Accuracy (%)
Comm. (GB) +
Time (min)
0.2
82.29
0.003
6.36
0.5
82.37
0.007
6.364u.1
0.8
82.55
0.031
6.39
A.3 Different Number Clients
We assess the performance of DictPFL. in environments with varying numbers of clients. The findings, presented in 5 demonstrate that DictPFI. performs effectively and consistently across settings with different client counts.
Table 5: The results of DictPFL. under client numbers.
Clients
Accuracy (%) t
Comm. (GB)
Time (min) $
82.67
0.160
6.42
82.64
0.092
3.70
81.94
0.046
1.85
20
81.82
0.041
0.93
50
80.42
0.041
0.75
200
80.56
0.041
1.96
A.4 Different Heterogeneous Level
Unsurprisingly, DictPFL performs better in homogeneous settings than in heterogeneous settings. As the table/o]shows we evaluated DictPFL in various heterogeneous settings under different Dirichlet distributions from 0.3 to 0.9 and compared with homogeneous setting. The results indicate that DictPFL's performance remains stable across different heterogeneous dataset splits. Specifically, a smaller a (more heterogeneous) requires more communication size and training time achieve comparable accuracy to larger (less heterogeneous).
22

---

Table 6: The results under different heterogeneous settings.
Accuracy (%)
Comm. (GB)
Time (min)
0.3
79.62-0.4
0.103
6.22
0,6
80.28
0.145
6.44
0.9
82.06
0.151
6.45
00
82.67 =022
0.160
6.42.0.2
A.5 Performance NLP tasks.
Table shows that DictPFL significantly improves efficiency in both sentence classification and generation (instruction tuning) tasks. For the generation task, train on the MetaMathQA [55] dataset and evaluate on GSM8K (70), focusing on mathematical reasoning. These gains are especially pronounced in larger models, where DictPFL. reduces training time by 99.4% percent for TinyLlama and 96.1 % percent for BERT. This improvement stems from the high cost of ciphertext operations in larger models, making DictPFL's optimizations
impactful.
Table 7: Comparison with baselines on Tinyl and BERT.
Methods
Acc. (%)
Comm. 4
Time
FedHHE-Full
45.86
30.0 TB
214.2
TinyL MetaMathQA
FodHE-FT
6.92
2.4 TB
17.9
FodML-HE
45.86
3.0 TB
22.6 h
DictPFL (ours)
45.93
0.3 TB
1.3 h
FedHE-Full
91.38
137.2 GB
342,6 m
BERT- AgNews
FedHE-FT
90.05
17.5 GB
47.9 m
FedML-HE
91.38
13.7 GB
32.8
DictPFL (ours)
91.24
4.8 GB
13.4 m
A.6 Comparision with Non-HE based FI.
We compare DictPFL with Secure Aggregation 25 by training ViT model on the Diabetic Retinopathy dataset under a 3-client LAN setting (1 Gbps, 0.5 ms). Secure Aggregation increases training time from 257.6 363.2 2 s, while DictPFL achieves the same 82.7% accuracy in 385.2 s. This demonstrates that HE-based FL with DictPFL is practically efficient, with much lower overhead than commonly assumed.
A.7 Combination with Existing Quantization Techniques
Algorithmic optimization directly reduces gradient redundancy without sacrificing accuracy, whereas quantization and packing often lead accuracy degradation. Moreover, their improvements are limited and easily saturate [71]. More importantly, these optimization approaches are orthogonal and can be combined-by first reducing gradients through algorithmic optimization and then ap- plying quantization packing, overall communication cost can be further minimized. Here perform experiments (3-client ViT on CIFAR-I0, compare DictPFL (algorithmic optimization) and AdaptiveBatchhE [41] (packing optimization), shown in the table below. It reveals that DictPFL outperforms AdaptiveBatchHE in both efficiency and accuracy and combining them will further reduce communication overhead.
Table 8: Comparison of accuracy and communication cost among HE-based frameworks.
Accuracy (%) 1
Communication (GB) 4
FedHE-FULL AdaptiveBatchHE
60.14
96.7
5.41
DictPFL (ours)
98.2
0.43
DictPFL + AdtptiveBatchHE
96.6
0.0872
23

---

A.8 Client Personality Preservation
DictPEL preserves the personality of each client based on our HRC mechanism. Specifically, while pruning is guided by the magnitude of global gradients, the HRC mechanism allows each client to upload accumulated local gradients for parameters that are reactivated, even if they were previously pruned due to low global gradient magnitudes. This ensures that important client-specific significant gradients are not lost: whenever such parameters are reactivated again, clients contribute their accumulated local gradients.
In Table 2 we evaluate DictPFL under various degrees of data heterogeneity by adjusting the Dirichlet factor a. While greater heterogeneity (lower a:) increases training time and communication overhead, DictPFL consistently maintains strong performance.
To further demonstrate the effect of HRC's accumulative gradient sharing mechanism, we compare "accumulative gradient sharing" versus mion-accumulative sharing." measuring the resulting accuracy under comparable training time. Our results (3-client ViT on Diabetic Retinopathy, r = 4, 8 = 0.7, 3, / = 0.2) show that omitting accumulated gradients notably reduces accuracy, particularly in more heterogeneous clients with diverse data. Overall, these results highlight that DictPFL. achieves robust performance across a wide range of data distributions.
because discarding small but meaningful gradients impairs learning for
Table 9: Effect of accumulative gradient sharing under different Dirichlet factors
a
Accumulative gradient sharing (%)
Non-accumulative sharing (%)
0.3
79.62
74.26
0.6
80.28
76.
0.9
82.06
80.45
B Analysis on FedML-HE [31
FedML-HE trades security for efficiency, and this trade-off persists regardless of whether sensitivity dynamically recalculated. While dynamic recalculation can enhance security, it incurs substantial empirical 0% attack success rate. Because recalculating
computational overhead to achieve sensitivity scores requires each client to perform a forward pass on the training dataset and share encrypted sensitivity values for secure aggregation, it introduces overhead comparable to the original training round and HE aggregation step.
Our experiments (3-client ViT CIFAR-10, encrypt 10%) with varying recalculation frequencies (i.e., recalculating every K rounds) show that more frequent updates improve privacy, but cost of significantly reduced efficiency. Even under these settings, FedML-HE still cannot achieve the strong privacy guarantces or efficiency of DictPFL.
Table 10: Effect of dynamic sensitivity recalculation in FedML-HE.
Method
Accuracy (%)*
Communication (GB) 1
Attack Success Rate (LPIPS)
FedHF--Full
98.17
60.14
0.00
FedMI.-HE (K=1)
98.16
61.07
0.00
FedMI.HHE (K=2)
98.16
54,28
0.092
FedML-HE (K=5)
98.16
30.8
0.309
FedML-HE (K=10)
98.16
14.2
0.788
DictPFL (ours)
98.15
0.43
0.00
24


Reviews:
[{"review_id": "8B81vVQydx", "reviewer": "Reviewer_oqcz", "summary": "This paper introduces a new Homomorphic encryption (HE) method defending against gradient inversion attacks in Federated Learning (FL).\nPrior HE methods encrypt every gradient update, which is time-consuming, or encrypt most sensitive gradients, which provides weaker privacy. Instead, this paper tries to encrypt every gradient that is transmitted to the server, aiming to enhance privacy and efficiency as well. \n\nSpecifically, the paper proposes the method DictPFL, including decompose-for-partial-encrypt (DePE) and Prune-for-Minimum-Encrypt (PrME). DePE decomposes model weights $W_0$ via SVD into a frozen and globally consistent dictionary $D$, which is identical across all clients and is not transmitted, and a trainable lookup table $T$, which will be encrypted and shared for aggregation, i.e., $W=W_0+DT$. PrME further minimizes the encrypted lookup table parameters $T$ on the client side, conducts consistent pruning based on the history of global gradients.\n\nExperiments on image classification, sentence classification, and text generation demonstrate the superiority of DictPFL in terms of efficiency and effectiveness against attacks.", "questions": "- It is recommended to present results on performance against attacks and the accuracy of the model together under the same hyperparameters (maybe a table), and include a comparison with other baselines, for better clarity in showing the trade-off.\n- For the rigorous validation of the limitation of FedML-HE (in lines 51-58), it would be better to check the vulnerability and conduct gradient inversion attacks when the top-k most sensitive parameters are encrypted, instead of based on the observation from that for magnitude-based pruning.\n- The proposed method aims to protect shared gradients to enhance privacy and improve efficiency. What if FedML-HE dynamically calculates sensitivity scores to encrypt? It would be better first to check this possibility and then discuss the limitation, to provide better and clarity in supporting the motivations.\n- The frozen and globally consistent dictionary $D$ is identical across all clients. Also, in line 191, need to retain the original weight $W_0$, and the dictionary may cover the coarse distribution of training data, so would it be easier for adversaries to infer gradients once they steal the dictionary?\n- Currently, only one analytic attack is considered for comparison; it would be better to evaluate against different kinds of attacks, such as [51-53].\n- Regarding the reactivation probability scale $\\beta$, how does the method perform when $\\beta=0$? From Table 4, it appears that the impact of reactivation is not significant and unnecessary.", "limitations": "It is recommended to further discuss the impact of non-iid data distribution.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Pros**\n- The motivation is clear. It is important to enhance privacy while reducing the overhead for FL.\n- The proposed method seems very effective and efficient, showing significant reductions in communication (402–748×) and training time (28–65×) compared to fully encrypted baselines, while maintaining accuracy.\n\n**Cons**\n- Not rigorous claims: in lines 51-58, FedML-HE encrypts only the top-k most sensitive parameters based on privacy-sensitive scores, the authors claim that this has a limitation in privacy protection because study [2] shows that 30% of gradients is enough for reconstruction. However, [2] shows that via gradient compression, where gradients with small magnitude (not important) are pruned to zero, which is magnitude-based pruning and different from the sensitive-based encryption in FedML-HE.\n- Missing related works such as analytic attacks [50] and optimization-based attacks [51-53], as well as defenses such as [54].\n\n----\n[50] Fowl, L.; Geiping, J.; Czaja, W.; Goldblum, M.; and Goldstein, T. 2022. Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models. ICLR.\n\n[51] Balunovi´c, M.; Dimitrov, D. I.; Staab, R.; and Vechev, M. 2022. Bayesian Framework for Gradient Leakage. ICLR.\n\n[52] Geiping, J.; Bauermeister, H.; Dr¨oge, H.; and Moeller, M. 2020. Inverting gradients: How easy is it to break privacy in federated learning? Advances in Neural Information Processing Systems, 33: 16937–16947.\n\n[53] Li, Z.; Zhang, J.; Liu, L.; and Liu, J. 2022. Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10132–10142.\n\n[54] Sun, J.; Li, A.; Wang, B.; Yang, H.; Li, H.; and Chen, Y. 2021. Soteria: Provable defense against privacy leakage in federated learning from representation perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9311–9319.", "was_revised": true, "final_justification": "I sincerely thank the authors for the detailed response; it addressed my concerns, hence I raised my score to 5.\n\nIn the revised version, it's kindly recommended to use Figure 7 to support the claim (Lines 51–58), include results about how FedML-HE dynamically calculates sensitivity scores to encrypt, and the impact of non-iid data distribution.", "rebuttal": "We sincerely thank Reviewer oqcz for the constructive and insightful feedback. We appreciate the recognition of our clear motivation, as well as the effectiveness and efficiency of the proposed techniques.\n\n**Q1: Not rigorous claims: Lines 51–58 claim FedML-HE has limited privacy since [2] shows 30% of gradients suffice for reconstruction. However, [2] uses magnitude-based pruning, which differs from FedML-HE’s approach of encrypting top-k sensitive parameters.**\n\nLines 51-58 were originally used to claim FedML-HE is insecure because the unencrypted gradients can leak information exploitable by attackers. Figure 7 demonstrates that when 30% of the gradients are shared unencrypted (selected by FedML-HE), the image reconstructed by attack [47] achieves a 23% similarity score. Thanks to the reviewer’s feedback, we recognize that Figure 7, rather than citation [2], should be used to support this claim. We will revise the manuscript accordingly to avoid any misunderstanding. \n\n**Q2. Missing related works such as analytic attacks [50] and optimization-based attacks [51-53], as well as defenses such as [54]. It would be better to evaluate against different kinds of attacks, such as [51-53].**\n\nThanks for the reviewer's feedback, and we would like to clarify any misunderstanding. DictPFL is robust to all existing attacks because all shared data is encrypted. In contrast, previous method partially shares unencrypted gradients, making them vulnerable to attacks. In our paper, we use [47] as a representative attack to validate this distinction. Regarding the mentioned defense [54], the server can get the final aggregated model, whereas DictPFL prevents the server from accessing it by encrypting gradients throughout the entire process. We will clarify this point in the paper and cite all the relevant attack and defense methods accordingly. \n\n**Q3. Presenting attack performance and model accuracy together would better clarify trade-offs.**\n\nWe would like to clarify any misunderstanding. DictPFL ensures security by encrypting all data shared with the server, resulting in a 0% attack success rate across all hyperparameter settings. Therefore, there is no trade-off between accuracy and security in our approach. The only trade-offs between accuracy and efficiency are presented in Figure 6. \n\n**Q4. What if FedML-HE dynamically calculates sensitivity scores to encrypt? It would be better first to check this possibility and then discuss the limitations, to provide better and clarity in supporting the motivations.**\n\nThanks for reviewer's insightful suggestion. FedML-HE trades security for efficiency, and this trade-off persists regardless of whether sensitivity is dynamically recalculated. While dynamic recalculation can enhance security, it incurs substantial computational overhead to achieve an empirical 0% attack success rate. Because recalculating sensitivity scores requires each client to perform a forward pass on the training dataset and share encrypted sensitivity values for secure aggregation—introducing overhead comparable to original training round and HE aggregation step. Our experiments (3-client ViT on CIFAR-10, encrypt 10\\%) with varying recalculation frequencies (i.e., recalculating every k-round) show that more frequent updates do improve privacy, but at the cost of significantly reduced efficiency. Even under these settings, FedML-HE still cannot achieve the strong privacy guarantees or efficiency of DictPFL.\n\n|                  | Accuracy (%)  | Communication (GB)  | Attack Success Rate (LPIPS)  |\n|------------------|---------------|---------------------|------------------------------|\n| FedHE-Full       | 98.17         | 60.14               | 0.00                         |\n| FedML-HE (K=1)   | 98.16         | 61.07               | 0.00                         |\n| FedML-HE (K=2)   | 98.16         | 54.28               | 0.092                        |\n| FedML-HE (K=5)   | 98.16         | 30.8                | 0.309                        |\n| FedML-HE (K=10)  | 98.16         | 14.2                | 0.788                        |\n| DictPFL          | 98.15         | 0.43                | 0.00                         |\n\n**Q5. Since the dictionary may reflect the coarse distribution of training data, could this make it easier for adversaries to infer gradients if the dictionary is compromised?**\n\nUnder our threat model, the dictionary is never shared with the server. Even if the server somehow obtains the dictionary, it provides no advantage in inferring gradients. This is because only the lookup table is updated using private data, while the dictionary itself contains no information about gradients or client training data. The only way for the server to access gradient information would be to break the HE and decrypt the gradients, which is assumed to be computationally infeasible. \n\n**Q6. Regarding the reactivation probability scale $\\beta$, how does the method perform when $\\beta=0$? From Table 4, it appears that the impact of reactivation is not significant and unnecessary.**\n\nThe reactivation mechanism HRC is significant and necessary. Without HRC ($\\beta=0$), certain important parameters pruned in early rounds may never participate in training again, which may hinder the model's ability to achieve optimal accuracy. As shown in Figure 9, disabling HRC ($\\beta=0$) results in noticeably lower accuracy with pruning 70%, i.e., 71.5%, under comparable training overhead. We will add this result in Table 4 and further analyze a range of $\\beta$ values between 0 and 0.2 to present a more comprehensive ablation study. \n\n**Q7. It is recommended to further discuss the impact of non-iid data distribution.**\n\nThank you for the insightful suggestion. As shown in Table 6, we already evaluated DictPFL under various degrees of data heterogeneity by adjusting the Dirichlet factor $\\alpha$. While greater heterogeneity (lower $\\alpha$) increases training time and communication overhead, DictPFL consistently maintains strong performance. This robustness is largely due to our HRC reactivation mechanism, which allows clients to share accumulated local gradients for reactivated parameters. \n\nTo further demonstrate the effect, we compare “accumulative gradient sharing” versus “non-accumulative sharing,” measuring the resulting accuracy under similar training time. Our results (3-client ViT on Diabetic Retinopathy, $r=4$, $s=0.7$, $\\tau=3$, $\\beta=0.2$) show that omitting accumulated gradients notably reduces accuracy, particularly in more heterogeneous settings—because discarding small but meaningful gradients impairs learning for clients with diverse data. Overall, these results highlight that DictPFL can achieve robust performance across a wide range of data distributions. \n\n| $\\alpha$  | Accumulative gradient sharing  | Non-accumulative sharing  |\n|-----------|--------------------------------|---------------------------|\n| 0.3       | 79.62%                         | 74.26%                    |\n| 0.6       | 80.28%                         | 76.13%                    |\n| 0.9       | 82.06%                         | 80.45%                    |"}, {"review_id": "EFJ8kMEaxc", "reviewer": "Reviewer_JGjF", "summary": "This paper introduces DictPFL for efficient HE-based federated learning. Through Decompose-for-Partial-Encrypt (DePE), which decomposes model weights into a static dictionary and a trainable lookup table, and further optimization via Prune-for-Minimum-Encrypt (PrME), DictPFL substantially reduces the transmission of encrypted gradients while preserving privacy guarantees. Experimental results demonstrate that DictPFL achieves significant efficiency across various tasks.", "questions": "Can the two strategies proposed in the paper, DePE and PrME, be applied to other non-HE-based federated learning methods?\n\nThe paper suffers from redundant expressions. For instance, the motivation and contributions in the abstract and introduction are not concisely condensed; the structure of other sections also differs significantly from that of conventional papers, making these sections overly verbose and difficult to follow.\n\nThe comparative methods are relatively few in number and outdated. For instance, FedHE-Full (2022), FedHE-Finetune (2023), and FedML-HE (2023) were all published before 2023.\n\nThe section structure may need adjustment.", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The proposed method significantly improves efficiency while maintaining privacy guarantees, which is conducive to practical applications. This work is meaningful. However, the paper suffers from redundant expressions and needs further condensation and refinement. Additionally, the comparative methods are relatively few in number and outdated, and it is recommended to include more recent comparative methods.", "was_revised": true, "final_justification": "The authors have addressed my questions. While their responses are relatively brief (Q2), they are barely acceptable. I choose to maintain my score.", "rebuttal": "We sincerely thank Reviewer JGjf for recognizing the practical value of our work. We are glad that the efficiency gains and preserved privacy guarantees of our method were found meaningful and conducive to real-world deployment. \n\n**Q1. Can the two strategies proposed in the paper, DePE and PrME, be applied to other non-HE-based federated learning methods?**\n\nThanks for the reviewer's insightful question. The proposed DePE and PrME strategies are general and applicable to non-HE-based FL, especially in communication-sensitive settings. By reducing the volume of gradient information transmitted, they help lower communication overhead regardless of encryption. DictPFL shows greater benefits in HE-based FL, where communication and computation overhead introduced by HE becomes the bottleneck (see Figure 2). \n\nIn summary, while DePE and PrME are effective in non-HE scenarios, their advantages are most significant in HE-based FL, which is the focus of this work. \n\n**Q2. The paper contains redundant expressions, with an overly verbose abstract and introduction.**\n\nThanks for the reviewer's constructive feedback. We will revise the abstract and introduction to make the motivation and contributions more concise. \n\n**Q3. The comparative methods are relatively few in number and outdated. For instance, FedHE-Full (2022), FedHE-Finetune (2023), and FedML-HE (2023) were all published before 2023.**\n\nWe found that most new HE-FL approaches still use our main baselines (FedHE-Full, FedHE-Finetune, and FedML-HE) as fundamental frameworks. Many recent works (2024–2025) either build directly on these methods (e.g., [a, b], which extend FedML-HE and retain its partial encryption strategy) or explore orthogonal optimizations such as quantization and packing [c], which are compatible with our DictPFL. \n\nOur DictPFL builds upon and enhances these widely adopted baselines, ensuring a fair and representative evaluation. We will also update the Related Work section to cite and briefly discuss these recent developments [a,b,c]. \n\n[a] A Selective Homomorphic Encryption Approach for Faster Privacy-Preserving Federated Learning, Arxiv 2025\n\n[b] Privacy-Preserving Federated Learning for SkinCancer Detection Using Homomorphic Encryption and Advanced Deep Learning Techniques, IJCS 2025\n\n[c] Efficient and Straggler-Resistant Homomorphic Encryption for Heterogeneous Federated Learning, INFOCOM 2025"}, {"review_id": "sRenhMopM0", "reviewer": "Reviewer_1kTE", "summary": "This paper introduces a novel method DictPFL to reduce the communication overhead while ensuring the privacy in HE-based FL. This goal is achieved by two modules designed, i.e., the decompose-for-partial-encrypt and prune-for-minimum-encrypt. The proposed DictPFL method achieves decent performance on different tasks, i.e., image recognition, text classification, and text generation.", "questions": "Please refer to the concerns listed in the weaknesses part.", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n+ The paper is well written and easy to read.\n+ The visualizations of the experimental results are nicely presented.\n+ The proposed DictPFL method significantly reduces the communication overhead in the domain of HE-based FL.\n\n**Weaknesses**\n+ The motivation for adopting algorithmic optimization rather than the quantization and packing needs to be further clarified. Is there any advantage brought by the algorithmic optimization?\n+ This work appears to be some incremental improvement on FedML-HE. The contributions to the community of HE-based FL are expected to be clarified.\n+ There is an identical metric for determining which gradients to prune. However, if the data distribution of the clients is different, how to ensure the personality of each client based on this design?\n+ Please provide additional experimental results in terms of the convergence of the training process of DictPFL.", "was_revised": true, "final_justification": "Most of my concerns have been addressed in the rebuttal. As a result, I’d like to raise my score to 4.", "rebuttal": "We thank Reviewer 1kTE for the insightful feedback on the clarity of our writing and the quality of our visualizations. We are also pleased that the effectiveness of DictPFL in significantly reducing communication overhead in HE-based federated learning was appreciated. \n\n**Q1. The motivation for adopting algorithmic optimization rather than the quantization and packing needs to be further clarified. Is there any advantage brought by algorithmic optimization?**\n\nWe thank the reviewer for the insightful question. Algorithmic optimization directly reduces gradient redundancy without sacrificing accuracy, whereas quantization and packing often lead to accuracy degradation. Moreover, their improvements are limited and easily saturate [b]. More importantly, these optimization approaches are orthogonal and can be combined—by first reducing gradients through algorithmic optimization and then applying quantization or packing, overall communication cost can be further minimized. Here we perform experiments (3-client ViT on CIFAR-10, $r=4$, $s=0.7$, $\\tau=3$, $\\beta=0.2$) to compare DictPFL (algorithmic optimization) and AdaptiveBatchHE [a] (packing optimization), as shown in the table below. It reveals that DictPFL outperforms AdaptiveBatchHE in both efficiency and accuracy and combining them will further reduce communication overhead. \n\n|                            | Accuracy (%)  | Communication (GB)  |\n|----------------------------|---------------|---------------------|\n| FedHE-FULL                 | 98.2          | 60.14               |\n| AdaptiveBatchHE            | 96.7          | 5.41                |\n| DictPFL                    | 98.2          | 0.43                |\n| DictPFL + AdaptiveBatchHE  | 96.6          | 0.0872             |\n\n**Q2. This work appears to be some incremental improvement on FedML-HE. The contributions to the community of HE-based FL are expected to be clarified.**\n\nDictPFL is fundamentally different from FedML-HE in both principle and design. While FedML-HE improves efficiency by leaving partial gradients unencrypted—thereby trading privacy for efficiency—DictPFL improves efficiency without compromising privacy, as all shared gradients remain fully encrypted. This core difference enables DictPFL to maintain the strong privacy guarantees of HE that can defense any attacks, while significantly reducing overhead. \n\nNotably, our results demonstrate that DictPFL incurs only a 2× computational overhead compared to plaintext FL, showing for the first time that deploying FHE in FL can be practical at scale. This represents more than an incremental improvement, offering the community a practical solution for achieving both security and efficiency in HE-based FL. \n\n**Q3. There is an identical metric for determining which gradients to prune. However, if the data distribution of the clients is different, how to ensure the personality of each client based on this design?**\n\nDictPFL preserves the personality of each client based on our HRC mechanism. Specifically, while pruning is guided by the magnitude of global gradients, the HRC mechanism allows each client to upload accumulated local gradients for parameters that are reactivated, even if they were previously pruned due to low global gradient magnitudes. This means that important client-specific significant gradients are not lost: whenever such parameters are reactivated again, clients contribute their accumulated local gradients. In Table 6, we evaluate DictPFL under various degrees of data heterogeneity by adjusting the Dirichlet factor $\\alpha$. While greater heterogeneity (lower $\\alpha$) increases training time and communication overhead, DictPFL consistently maintains strong performance. \n\nTo further demonstrate the effect of HRC's accumulative gradient sharing mechanism, we compare “accumulative gradient sharing” versus “non-accumulative sharing,” measuring the resulting accuracy under comparable training time. Our results (3-client ViT on Diabetic Retinopathy, $r=4$, $s=0.7$, $\\tau=3$, $\\beta=0.2$) show that omitting accumulated gradients notably reduces accuracy, particularly in more heterogeneous settings—because discarding small but meaningful gradients impairs learning for clients with diverse data. Overall, these results highlight that DictPFL can achieve robust performance across a wide range of data distributions. \n\n| $\\alpha$  | Accumulative gradient sharing  | Non-accumulative sharing  |\n|-----------|--------------------------------|---------------------------|\n| 0.3       | 79.62%                         | 74.26%                    |\n| 0.6       | 80.28%                         | 76.13%                    |\n| 0.9       | 82.06%                         | 80.45%                    |\n\n**Q4. Please provide additional experimental results in terms of the convergence of the training process of DictPFL.**\n\nIn our main paper, we provided learning curve plots in Figure 9 to demonstrate DictPFL’s convergence. To further address the reviewer’s concern, we will include additional plots in the revision that show the training and test losses over training rounds, across different settings.  All experiments in the main paper were repeated over five independent runs, and we report the mean and standard deviation for key metrics such as accuracy, communication cost, and training time (see Figure 6 and Table 1 and 2). The statistical reporting primarily demonstrates the stability and reproducibility of DictPFL’s performance, the consistently low standard deviations across these metrics provide indirect evidence of robust convergence.  \n\n[a] Adaptive Batch Homomorphic Encryption for Joint Federated Learning in Cross-Device Scenarios, IOT 2023. \n\n[b] HEQuant: Marrying Homomorphic Encryption and Quantization for Communication-Efficient Private Inference"}]

Author rebuttal:

Meta-review: This paper proposes **DictPFL**, a framework for efficient and private federated learning through selective encryption of model weights; with experiments across vision and NLP tasks showing clear improvements over prior art. The **final scores** of the paper read as 1 × Accept and 2 × Borderline Accept, an increase over the initial evaluations. After rebuttal and discussion, reviewers indicated that most concerns were addressed, and their overall stance is positive. The AC agrees with the reviewers’ evaluations, sees merit in the proposed algorithms, and therefore recommends **Accept**. Congratulations.  

The AC encourages the authors to revise the paper based on the discussions during the rebuttal period. Finally, during the AC–reviewer discussion, it was noted that the **capacity of the model is inherently reduced by design**: the fixed dictionary and decomposition may constrain adaptation, particularly in heterogeneous settings. The authors are encouraged to acknowledge and discuss this limitation in the revision.
Decision: Accept (poster)