FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models
Authors: Kai Yi
Georg Meinhardt
Laurent Condat
Peter Richtárik
Venue: Accepted by TMLR ()

Abstract: Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with a central server, while being respectful of privacy. A critical bottleneck in FL is the communication cost. A pivotal strategy to mitigate this burden is Local Training, which involves running multiple local stochastic gradient descent iterations between communication phases. Our work is inspired by the innovative Scaffnew algorithm, which has considerably advanced the reduction of communication complexity in FL. We introduce FedComLoc (Federated Compressed and Local Training), integrating practical and effective compression into Scaffnew to further enhance communication efficiency. Extensive experiments, using the popular Top-K compressor and quantization, demonstrate its prowess in substantially reducing communication overheads in heterogeneous settings.

Published in Transactions on Machine Learning Research (09/2025)
FecComLoc: coamunicatioon-fficiee Distributed Training of Sparse and Quantized Models
Kai Y1*
kai, kodkyskkaattttcm edu.
Deportment of Computer Science
King Abdullah University of Science and Technology (KAUST)
Georg Meinhardt
georg. geony.meinkarnskkkuss..coo
Deportment of Computer Science
King Abdullah University of Science and Technology (KAUST)
Laurent Condat
laurent. wrrrredl.oomdal@@@kuuu......
Department of Compuler Science
King Abdullah University of Science and Technology (KAUST)
STAATAKKUST Center of Pacellence in Data Science and Artificial Intelligence (SDATA-KAUST AT)
Peter Richtárik
peter.richtariiskkaust.ed
Department of Computer Science
King Abdullah University of Science and Technology (KAUST)
SDAIA-KAUST Center of Encellence in Data Science and Artificial Intelligence (SDAIA-KAUST AI)
Reviewed on OpenReview: https: // openreview. net/ forum? id=oYOPLytQsj
Abstract
Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with central server, while being respectful of privacy. critical bottleneck in FL the commu- nication cost. A pivotal strategy to mitigate this burden Local Training, which involves running multiple local stochastic gradient descent iterations between communication phases. Our work inspired by the innovative Scaffnew algorithm of Mishchenko et al. (2022), which has considerably advanced the reduction of communication complexity in FL. We introduce FedComLoc (Federated Compressed and Local Training), integrating practical and effective compression into Scaffnew to further enhance communication efficiency. Extensive experi- ments, using the popular TopK compressor and quantization, demonstrate its prowess in substantially reducing communication overheads in heterogeneous settings.
1 Introduction
Privacy concerns and limited computing resources on edge devices often make centralized training imprac- tical, where all data gathered in data center processing. In response to these challenges, Federated Learning (FL) has emerged as an increasingly popular framework (McMahan al., 2016; Kairouz et al., 2019). In FL, multiple clients perform computations locally their private data and exchange information with a central server. This process is typically framed empirical risk minimization problem (Shalev- Shwartz & Ben-David, 2014):
((x) Σ
min RER
(ERM)
"Wor completed during PhD KAUST, Currently with Meta, USA. Work done while intern KAUST.

---

Published in Transactions on Machine Learning Research (09/2025)
where fa represents the local objective for client n is the total number of clients, and x is the model to be optimized. Our primary objective is to solve the problem (ERM) and deploy the optimized global model to all clients. For instance, I might be neural network trained in an FL setting. However, a considerable bottleneck in FL are communication cost, particularly with large models.
To mitigate these costs, FI. often employs Local Training (LTT), a strategy where local parameters are updated multiple times before aggregation (Povey et al., 2014; Moritz et al., 2016; McMahan et al., 2017; Li et al., 2020; Haddadpour & Mahdavi, 2019; Khaled et al., 2019; 2020; Karimireddy et al., 2020; Gorbunov et al., 2020; Mitra et al., 2021). However, there is a lack of theoretical understanding regarding the effectiveness of LT methods. The recent introduction of Scaffnew by Mishchenko et al. (2022) marked a substantial advancement, as this algorithm converges to the exact solution with accelerated complexity, in convex settings.
Another approach to reducing communication costs is through compression (Haddadpour et al., 2021; Condat et al., 2022; Yi ct al., 2024). In centralized training, one often aims to learn a sparsifiod model for faster training and communication efficiency (Dettmers & Zettlemoyer, 2019; Kuznedelev et al., 2023). Dynamic pruning strategies like gradual magnitude pruning (Gale et al., 2019) and RigL (Evei et al., 2020) are common. But in FL, the effectiveness of these model sparsification methods based thresholds remains unclear. The work by Babakniya et al. (2023) considers FL sparsity mask concepts, showing promising results.
Quantization is another efficient model compression technique (Han et al., 2021; Bhalgat et al., 2020; Shin et al., 2023), though its application in heterogeneous settings is limited. Gupta et al. (2022) introduced FedAvg with Kurtosis regularization (Chmiel et al., 2020) in FT.
Furthermore, studies such as Haddadpour et al. (2021); Condat et al. (2022) have theoretical convergence guarantees for unbiased estimators with restrictive assumptions. As this work employs the biased TopK compressor these are unsuitable in this case.
Thus, we tackle the following question:
Is possible to design an efficient algorithu combining accelerated local training with compression techniques, such as quantization and Top-K, and validate its efficiency empirically populor FL datasets?
Our method for investigating this question consists of two steps. Firstly, we design an algorithm, termed FedComLoc, which integrates general compression into ScaffNew, an accelerated local training algorithm. Secondly, we empirically validate FedComLoc for popular compression techniques (Topk and quantization) on popular FL datasets (FedMNIST and FedCIFAR10).
We were able to answer this question affirmatively with the following contributions:
We have developed a communication--ffiieel method FedComLoc for distributed training, specifically designed for heterogeneous environments. This method integrates general compression techniques and is motivated by previous theoretical insights.
We proposed three variants of our algorithm addressing several key bottlenccks in FL: FedComLoc- Com addresses communication costs from client to server, FedComLoc- Gooboall addresses communication costs from server to client and FedComLoc- Local addresses limited computational resources on edge devices.
We conducted detailed comparisons and ablation studies, validating the effectiveness of our approach. These reveal a considerable reduction communication and, certain cases, an enhancement training speed in number of communication rounds. Furthermore, we demonstrated that our method outperforms well-established baselines in terms of training speed and communication costs.
2 Related Work
2.1 Local Training
The evolution of LT in FL has been profound and continuous, transitioning through five distinct genera- tions, each marked by considerable advancements from empirical discoveries to reductions in communication
2

---

Published in Transactions on Machine Learning Research (09/2025)
Algorithm FedComLoc
stepsize y O, probability p O, initial iterate 21,0 = = nn,o Rd, initial control variates
11.0,... 0n, E [R on each client such that Ei=1 h4o 0, number of iterations T 1, compressor
C()) E (-) Qu().....
2: server: flip a coin, 0, E {0. 1). T times, where Prob(0;- = = po Decide when to skip communication
3: send the sequence , to all workers
4: for t 0, , - do
5: sample clients S [ 2.3.........
in parallel on all workers E do
7: FedComloc-Local: local compression g)r(( gr,r (C((i((),)))
f1,41 Zi,t - ((gh,(1..)) hs,e) o Local gradient-typs step adjusted via the local control variate
higt
FedComLoc-Com: uplink compression - fi,411 = C(fs,++1)
10:
if 02 then
11:
Ti,1+1 = Σ
Average the iterates (with small probability p)
1=1
12:
FedComLoc-Glothe downlink compression 21,41 C(11,4+1
13:
else
14:
F1,4+1 = fi,+1
Skip communication
15:
end if
16:
h1,4+1 h4. + ( &(4+1)
Update the local control variate has
17:
end local updates
18: end for
complexity. The pioneering FedAvg algorithm (MeMahan et al., 2017) represents the first generation of LT techniques, primarily focusing on empirical evidence and practical applications (Povey et al., 2014; Moritz et al., 2016; McMahan et al., 2017). The second generation of LT methods consists in solving (ERM) based on homogeneity assumptions such as bounded gradients' (Li et al., 2020) or limited gradient diversity (Haddadpour & Mahdavi, 2019). However, the practicality of such assumptions in real-world FL scenarios is debatable and often not viable (Kairouz et al., 2019; Wang et al., 2021).
Third-generation methods made fewer assumptions, demonstrating sublinear (Khaled et al., 2019; 2020) or linear convergence up a neighborhood (Malinovsky al., 2020) with convex and smooth functions. More recently, fourth-generation algorithms like Scaffold (Karimireddy et al., 2020), S-Local-GD (Gorbunov et al., 2020), and FedLin (Mitra et al., 2021) have gained popularity. These algorithuns effectively counteract client drift and achieve linear convergence to the exact solution under standard assumptions. Despite these advances, their communication complexity mirrors that of GD, i.e. log where = L/μ denotes the condition number.
The most recent Scaffnew algorithm, proposed by Mishchenko et al. (2022), revolutionizes the field with its accelerated communication complexity Ovyx 11). This seminal development establishes LT as commu- nication acceleration mechanism for the first time, positioning Scaffnew at the forefront of the fifth generation of I.T methods with accelerated convergence. Further enhancements to Scaffnew have been introduced, in- corporating aspects like variance-reduced stochastic gradients (Malinovsky et al., 2022), personalization (Yi et al., 2023), partial client participation (Condat. et al., 2023), asynchronous communication (Maranjyan et al., 2022), and an expansion into a broader primal dual framework (Condat & Richtárik, 2023). This latest generation also includes the 5GCS algorithm (Grudzicń et al., 2023), with a different strategy where the local steps are part of an inner loop to approximate proximity operator. Our proposed FedComLoc algorithm extends Scaffnew by incorporating pragmatic compression techniques, such as sparsity and quan- tization, resulting in even faster training measured by the number of bits communicated.
There exists E R s.t. [f((1)] S i 2hhere exists s.t. E. [V]( <c (
3

---

Published in Transactions on Machine Learning Research (09/2025)
2.2 Model Compression in Federated Learning
Model compression in the context of FL a burgeoning field with diverse research avenues, particularly focusing on the balance between model efficiency and performance. Jiang et al. (2022) innovated in global pruning by engaging single, powerful client to initiate the pruning process. This strategy transitions into collaborative local pruming phase, where all clients contribute to an adaptive pruning mechanism. This involves not just parameter elimination, but also their reintroduction, integrated with the standard FedAvg framework (McMahan et al., 2016). However, this approach demands substantial local memory for tracking the relevance of each parameter, a constraint not always feasible in FL. settings.
Addressing some of these challenges, Huang et al. (2022) introduced an adaptive batch normalization coupled with progressive pruning modules, enhancing sparse local computations. These advancements, however, often do not fully address the constraints related to computat ional resources and communication bandwidth on the client side. Our rescarch primarily focuses magnitude-basct sparsity pruning. Techniques like gradual magnitude pruning (Gale el al., 2019) and RigL (Evci et al., 2020) have been instrumental in dynamic pruning strategies. However, their application in FL contexts remains relatively unexplored. The pioneering work of Babakniya et al. (2023) extends the concept of sparsity masks in FL, demonstrating noteworthy outcomes.
Quantization another vital avenue in model compression. Seminal works in this area include Han et al. (2021), Bhalgat et al. (2020), and Shin et al. (2023). A major advance has been made by Gupta al. (2022), who combined FedAvg with Kurtosis regularization (Chmiel et al., 2020). We are looking to go even further by integrating accelerated LT with quantization techniques.
However, & gap exists in the theoretical underpinnings of these compression methods. Research by Haddad- pour et al. (2021) and Condat et al. (2022) offers theoretical convergence guarantees for unbiased estimators, but these frameworks are not readily applicable to common compressors like Top-K sparsifiers. In partic- ular, CompressedScfffeew (Condat et al., 2022) integrates an unbiased compression mechanism in Scaffnew, that is based on random permutations. But due to requiring shared randomness it is not practical. Linear convergence has been proved when all functions fi are strongly convex.
To the best of our knowledge, no other compression mechanism has been studied in Scaffnew, either theoreti- cally or empirically, and even the mere convergence of Scaffnew in nonconvex settings has not been investigated either. Our goal is to go beyond the convex setting and simplistic logistic regression experiments and to study compression in Scaffnew in realistic nonconvex settings with large datasots such as Fodcrated CIFAR and MNIST. Our integration of compression in Scaffnew is heuristic but backed by the findings and theoret- ical guarantees of CompresedSccaffew in the convex setting, which shows twofold acceleration with respect to the conditioning and the dimension d, thanks to LT and compression, respectively.
3 Proposed Algorithm FedComLoc
3.1 Sparsity and Quantization
Let us define the sparsifying (-) and quantization Q,(-) operators.
Definition 1. Let x e Rd and K E {1, d]. We define the sparsifying compressor TopK : Rd + Rd as:
TopK(x) arg min [lly 2|1 llvllo < K).
yERe
where Ilviln == [(:: : y # 0} denotes the number of nonzero elements in the vector y = (yi. ........00 . E IRd. In case of multiple minimizers, TopK is chosen arbitrarily.
Definition 2. For any vector x E Red with * and number of bits > O, its binary quantization Q.(x)) is defined componentwise as
Qr (x) (1|=112 sgn sgn (xs) (xxx)")) isiga 2
4

---

Published in Transactions on Machine Learning Research (09/2025)
Then their probability distribution is where (z, 2n) are independent random variables. Let y: 1 given by
|a.l.
[2'vx] /2"
with proba. 2" 12'vi5
(xx,2")=
[2"0z] /2" otherwise.
If x: = define Q+(x:) O.
The distributions of the (x r) minimize variance over distributions with support {0, 1/r,... unbiasedness, i.e. E (x, | This definition is based on an equivalent one in Alistarh et al. (2017)-
......,,,,..... 1), ensuring
3.2 Introduction of the Algorithms
FedComLoc (Algorithm 1) is an adaptation of the accelerated Scaffnew framework, with modifications to incorporate communication-eficii compression strategies. In this section, we detail how sparsification and quantization are integrated, and we explain the design and motivation behind each FedComLoc variant.
In all variants, federated training proceeds in rounds: the server broadcasts the current global model, clients perform local updates, and compressed updates communicated as appropriate. We adopt Top the default compression technique, but note that quantization can be used interchangeably.
We consider three variants, each targeting a different practical bottleneck in federated systems:
FedComLoc-Com (Client-to-Serwy Server Compression): This variant addresses the communccation bottleneck on the uplink. After each client completes its local training, the update or local model compressed (via TopK or quantization) before being transmitted to the server. This approach is well-suited for scenarios where client devices are bandwidth-limited for uploading. The server then aggregates the compressed updates form the next global model.
FedComLoc-Local (Local Model Compression): In this setting, compression applied within the local training loop. After each local update step (e.g., after a gradient step batch), the client compresses its own local model or updates. This variant is designed for cases where client devices have limited computational or memory resources, and thus benefit from working with compressed models during training. It can also reduce local energy consumption and memory footprint.
FedComLoc-Global (Server-to-Client Compression): Here, the server compresses the global model before sending it to all clients at the start of each round. This variant targets scenarios where the main bottleneck is the download bandwidth from server to clients (e.g., in environments with many clients or costly downlink channels). Each client then decompresses the received model before starting local training.
Al Algorithm 1 (see above) provides a unified pseudocode for all three variants. The specific compression step is invoked at the corresponding stage (client upload, local update, or server broadcast) depending on the variant in use.
These variants allow FedConLoc to flexibly adapt to different. system bottlenecks commonly encountered in federated learning applications. Detailed ablation results in the following experiments demonstrate the practical impact of each approach.
4 Experiments
Baselines. Our evaluation comprises three distinct aspects. Firstly, we conduct experiments to assess the impact of compression on communication costs. FedComLoc is assessed for varying sparsity and quantization ratios. Sccondly, we compare FedComLoc-Com with FedComLoc--Loaa and FedComLoc-Global. Thirdly, we explore the efficacy of FedComLoc against acc-lcceled local training methods, including FedAvg (McMahan et al., 2016), its Top-K sparsified counterpart spdrse and Scaffold (Karimireddy et al., 2020).
5

---

Published in Transactions on Machine Learning Research (09/2025)
Top-K
100%
10%
30%
50%
70%
90%
0.1
0.3
0.5
0.7
K =10%%
0,9623
0.9686
0.9731
0.9758
0.9768
0.9735
Accuracy
0.9758
0.9374
0.9654
0.9699
0.9745
0.9748
K 10%
0.8681
0,9124
0,9331
0,9874
0.9441
0.9382
Decrease
3.94%
1.07%
0.61%
0.13%
0.10%
-50%
0.9397
0.9635
0.9671
0.9699
0.9706
0.9719
Table 1: Test accuracy for various Top-K ratios.
Table 2: Test accuracy score for various Dirichlet fac- tors and sparsity ratios.
K 10%
K 20%
2.5
0.1
2.5
0.1
2.0
K 30%
2.0-
K 30%
0.3
Training Loss
K 50%
Training Loss
K 50%
Training Loss
2.0
Training Loss
2.0
1.5
K 100%
90%
90%
100%
1.5
2.0
1.5
0.9
100
1.0
1.0
1.0-
1.0-
0.5
25
75
100
0.0
0.5
1.0
1.5
2.0
50
100
150
200
Communication rounds
Communication Bitsl
Communication rounds
Communication Bits
1.0
Testing Accuracy
0.8
Testing Accuracy
0.8
Testing Accuracy
0.8
Testing Accuracy
0.6
0.6
K 10%
10%
-.1.1
0.1
K 50%
K- 50%
0.4
50%
30%
0.4-
--3
0.3
70%
70%
0.2
90%
90%
0.5
0.9
1.0
1.0
25
75
1.0
2.0
50 100
150
200
0
Communication rounds
Communication Bitsle8
Communication rounds
Communication Bits
(a) 100% Top-K
(b) 30% Top-K
(a) K 10%.
(b) K 10%
Figure 1: Performance outcomes for various Top-K ratios.
Figure 2: Training loss and test accuracy for densily
ratio of K = -0%.
Datascts. Our experiments are conducted on FedMNIST (LeCun, 1998) and FedCIFAR10 (Krizhevsky et al., 2009) with the data processing framework FedLab (Zeng et al., 2023). For FedMNIST, we employ MLPs with three fully-connected layers, each coupled with a ReLU activation function. For FedCIFAR10, utilize CNNs with two convolutional layers and three fully-connceted layers. Comprehensive statistics for each dataset, details on network architecture and training specifics can be found in Appendix B.
Heterogeneous Setting. We explore different heterogeneous settings. Similar to (Zhang et al., 2023; Yi et al., 2024). we create heterogencity in data by using Dirichlet distribution, which assigns cach client a vector indicating class preferences. This vector guides the unique selection of labels and images for each client until all data is distributed. The Dirichlet parameter a indicates the level non-identical distribution. We also include a visualization of this distribution for the CIFAR10 dataset in Appendix C.I.I.
Default Configuration. In the absence of specific clarifications, we adopt the Dirichlet factor a 0.7. To balance both communication and local computation costs, we use p 0.1, resulting in an average of 10 local iterations per communication round. The learning rate is chosen by conducting grid search over the set {0.005, 0.01, 0.05, 0.1, 0.5). With communication costs being of most interest, our study employs FedComLoc-Com as the default strategy. The experiments are run for 2500 communication rounds for the CNN on FedCIFARIO and 500 rounds for the MLP on FedMNIST. Furthermore, the dataset is distributed across 100 clients from which 10 are uniformly chosen to participate in each global round.
Furthermore, our Definition of TopK, K the number of nonzero parameters. However, we will rather specify the enforced density ratio, i.c the ratio of nonzero parameters. For instance, specifying K 30% means retaining 30% of parameters.

---

Published in Transactions on Machine Learning Research (09/2025)
4.1 Top-K Sparsity Ratios
This section investigates the effects of different sparsity rations by investigating TopK ratios on FedMNIST. The outcomes can be found in Table 1. Notably, K 10% in Topk yields an accuracy of 0.9374, merely 3.94% lower than the 0.9758 unsparsified baseline. Remarkably, 70% sparsity level (K 30%) attains commendable performance, with only 1.07% accuracy reduction, alongside 70% reduction in communica- tion costs. Furthermore, from the communication bits depicted in Figure it is evident that sparsity yields laster convergence, the more so with increased sparsity (smaller K).
4.2 Data Heterogeneity/Dirichlle Factors
This subsection aims to assess the impact of varying degrees of data heterogencity on FedMNIST. Hence, an analysis of the Dirichlet distribution factor a is presented, exploring the range of values a (0.1,0.3,0.33 0.7, 1.0). Remember that a lower a mcans increased heterogeneity. Alongside, we examine the influence of different TopK factors, specifically 10%, 50% and 100%. The results are shown in Table 2. Figure 2 reports training loss and test accuracy for sparsity ratio of 90% (K 10%). Addition- ally, round-wise visualizations for K = 50% and K = 100% (non-sparse) are presented in Figure 11 in the Appendix.
Key observations from our study include:
When examining the effects of the heterogeneity degree a (as seen in each column of Table 2), we observe that sparsity performance is influenced by heterogeneity degrees. For instance, a = 0.1 results in a relative performance drop of 9.79% from an unsparsified to a sparsified model with K 10%. In contrast, for a 0.3, this drop is 5.80%, and for a 1.0, it 3.63%. Interestingly, for commonly used heterogeneity ratios in literature (a 0.3, 0.5, 0.7), the performance drop does not decrease substantially when moving from = 0.3 to a 0.5, or from = 0.5 to 0.7, unlike the shift from a = 0.1 to a = 0.3.
Focusing on the rows Table 2, we find that lower sparsity ratios are more sensitive to heterogeneous distributions. In particular, observe that with K 10%, the absolute performance improvement from a 0.1 to = 7.01%. %. However, for K 50,, this improvement is only 1.22%.
It should be noted that each method was run with a fixed learning rate without scheduling, and the maximum communication round was set to 1000. Previous studies suggest that higher spar- sity ratios require more communcctiin rounds in centralized settings (Kuznedeley et al., 2023). This phenomenon was also observed in our FL experiments. Therefore, there is the potential for performance enhancement through sufficient model rounds and adaptive learning rate adjustments, especially for methods with higher sparsity.
4.3 CNNs FedCIFAR10
This section repeats the experiments for CTFAR1O and a Convolutional Neural Network (CNN). We explored range of stepsizes (7 (0.005, 0.01, 0.05, 0.1]3). Further information is provided in Appendix B. The CIFAR10 results, which involve optimizing Convolutional Neural Network (CNN), are presented in Figure 3 for both tuned and a fixed step size. Observe the accelerated convergence of sparsified models in terms of communicated bits when the step size is tuned. Interestingly, sparsity of 90% (K 10%) shows faster convergence in terms of communication rounds (as shown in the first. column), suggesting the potential for enhanced training efficiency in sparsified models. For a fixed step size (the two rightmost columns of Figure 3) and K 10%, one can observe slower convergence compared to other configurations. This indicates that sparsity training requires more data and benefits from either increased communication rounds or a larger initial stepsize. This aligns with recent similar findings in the centralized setting (Kuznedelev et al., 2023).

---

Published in Transactions on Machine Learning Research (09/2023)
2121
200
135
259
283
de
120
do
189
200
420
150
CommuNirr OF munds
Communiont
Dorri oilion mare
Commin
Tuned Stepsize.
Fixed Stepsize of 0.01.
Figure CNN Performance on the FedCIFAR10 Dataset. For the left most columns, the step size was optimized for each density ratio K. For the two rightmost columns, a fixed stepsize 0.01 used. This is the maximum feasible step size which ensures convergence across all configurations.
4.4 Quantization
This section explores using quantization Q; for compression with the mumber of bits, set to E {4, 84,8, 16, 32} on FedMNIST. This approach aligns with the methodologies outlined Alistarh et al. (2017). The results after 1000 communication rounds are illustrated in Figure 6. Our analysis reveals that quantization offers superior performance compared to TopK-style sparsity. For instance, with 16-bit quantization corresponding 50% reduction in communication cost, the performance decrease mere 0.14%, Furthermore, Figure 7 shows outcomes for different degrees of data hetcrogencity. Those findings demonstrate that quantization reduces communication at minor performance tradeoff while also exhibiting only minor sensitivity to data heterogeneity. The Appendix gives further results for both FedMNIST (section C.2.1) and FedCIFAR10 (section C.2.2).
4.5 Number of Local Iterations
This section explores the performance impact of varying the expected number of local iterations on FedM- NIST. The expected number of local iterations is 1/p where is the communication probability. Hence, we investigate the influence of ranging from E (0.05, 0.1, 0.,, 0.3, 0.5). Furthermore, K 30% used. The results are presented in Figure 8. A key finding that more local training rounds (i.e. smaller p) not only accelerate convergence but can also improve the final performance.
4.6 FedComLoc Variants
In this section, we compare FedComLoc-Local, FedComLoc-Com and FedComLoc-Globa. on FedCTFAR10. The findings are illustrated in Figure 4. Observe that at high levels of sparsity (indicated by a small K in TopK), FedComLoc-Coo Com underperforss the other algorithms. This could be attributed to the heterogencous setting of our experiment: each client's model output is inherently skewed towards its local dataset. When this is coupled with extreme TopK sparsification, more bias is introduced, which adversely affects performance. Conversely, al low sparsity (e.g- K = 90%), FedComLoc-Com surpasses FedComLoc-Globa In addition, we observe that sparsity during local training (i.e. FedComLoc-Local tends to yield better results. One possible explanation is that due the local data bias the communication bandwidth between client and server might be crucial. Remember that FedComLoc-Locd had no communication compression while both FedComLoc-Com and FedCooLLo--Globll do. Further FedMNIST results shown in the Appendix.
8

---

Published in Transactions on Machine Learning Research (09/2025)
Tentontactin lacH
2.4
SedCare EC BON SedCami.com Gaod
24
Fectomio LOCal
2.4-
Endicon ROO Endiion GiiN
24-
Sestartion
Triining Loss
2.7
Fectomtar
2.2
Taining Loss
Fectomior Grobal
Training Loss
22
Restoriioe
Training
2.0
200
Traning
2.0-
2.0
1.
18-
1o
16-
1.8-
16-
1.4
500
1000 19b0 2000
1000 1500 2000 50c
500
1000 1500 2000
500
1000
1500 2000
500
1000 15b0 2000
Communicalion rounds
Communication rounds
Communicctton rounds
Communication rounds
Communication rounds
Testing Accuracy
0.4-
Testing Accuracy
0.5
Testing Accuracy
0.5-
Testing Accuracy
0.5
Testing Accuracy
0.4
0.3-
0.3
0.3
0.3
0.2
0.2-
Teckomtar Cart
FedComi.c
FecComiac Con
End Com
ResComier
Tectamtar-Ge dbbaal
SedComice Giocal
FecComiar Giebal
ResComtnn--a S00a
500 1000 1500 2000
500 1000 1500 2000
500 1000 1500 2000
0
500 2000 1500 2000
500 1000 1500 2000 Communication rounds
Communicalion rounds
Communication rounds
Communication rounds
Communication rounds
(a) K = 10%
(b) K = 30%
(c) K 50%
(d) K = 70%
(e) K 90%
Figure 4: Sparsity ablation studies of FedComLoc-LLoaa, FedComLoc-Come Com, and FedComLoc-Gloke on FedCI-
FAR10 and tuned stepsizes.
Method
Full
4 bits
8 bits
16 bits
Q,
0.1
0.3
0.5
0.7
0.9
1.0
bits
0.9633
0.9685
0.9751
0.9745
0.9746
0.9761
FedComLoc
0.9758
0.9564
0.9745
0.9744
16 bits
0.9627
0,9662
0.9726
0.9744
0.9756
0.9742
bis
bits
bits r = 8
bits r 16
2.0
2.0
2.5
Training Loss
16
Training Loss
0.3
1.5
32 bits
1.5
32 bits
Training Loss
0.5
2.0-
0.5
0.7
0.7
1.5-
0.9
Training
0.9
1.0-
1.0
1.0
1.0-
0.5-
0.5
0.5
0.0
20
0.0
0.5
1.0
1.5
2.0
0.0
Communication rounds
Communication
Bitge8
0.0
1.0
0.0
0.5
1.0
1.5
2.0
1.0
Communication bits leB
Communication bits 1e8
1.0-
Testing Accuracy
Testing Accuracy
0.8-
Testing Accuracy
Testing Accuracy
0.8
0.8
0.6
D.6
0.1
0.6-
0.1
0.4
0.3
0.3
0.4
bis
16 bits
0.2
16 bits
0.2
0.9
0.2-
bits
bits
1.0
1.0
40 Communication rounds
20
0.0
Communication Bitsle8
0.5
1.0
2.0
0.0
0.0
Communication bits leB
0.5
1.0
2.0
0.0
Communication bits 1e
0.5
1.0
1.5
2.0
Figure 6: FedComLoc employing Q.(-). The number of quantization bits r is set to r E (4, 8, 16,32).
Figure 7: Data heterogeneity ablations for FedCom- Loc utilizing Q+(-) with number of bits r either 8 or 16. The same results plotted over the number of com- munication rounds can be found in Figure 13 in the Appendix.
4.7 FedAug and Scaffold
In this section, the performance of FedComLoc is compared with baselines in form of FedAvg (McMahan et al., 2016) and Scaffold (Karimireddy et al., 2020) on FedCIFAR10. Furthermore, sparsified version of FedAvg is employed, termed as sparseFedAvg. For sparseFedAvg a learning rate of 0.1 is used, whereas for FedComLoc, a lower rate of 0.05 is utilized. The outcomes this analysis are depicted in Figure 9. The left part illustrates the performance of compressed models. We observe notably faster convergence for FedComLoc-type methods in comparison with sparseFedAug despite the lower learning rate. The right part of the figure compares FedAvg with Scaffold, devoid of sparsity, using identical learning rate of 0.005. This uniform rate ensures that each
9

---

Published in Transactions on Machine Learning Research (09/2025)
2.0
0.05
0.1
0.05
0.1
2,4
sparseFediig FedComiac Local
2.4
Reddig Scaffold
Training Loss
0.2
0.2
1.5
0.3
0.3
FeaComLac
Training Logs
2.2
FodComLoc
8.5
Traiming
2.0
2.0
1.0
Training
1.8
1.8-
0.5
1.6
0
50
100
150
200
Communication rounds
Total Costs
0
500
1000
1500
2000
0
500
1000
1500
1.0
Communication rounds
Communication rounds
Testing Accuracy
0.8
Testing Accuracy
Testing Accuracy
0.5
Testing Accuracy
0.5
0.6
0.4
0.4-
0.4
p D.05
0.05
p D1
0.1
0.3
0.3
0.2
0.3
02
0.3
0.2
sparsewwiiigg
95
0.5
0.2
ResComiac-Loot
Reddig
0.0
FedComise Com
Scaffold
Communication rounds
50
150 200
Total Costs
0.5
0.1
FeaComiac Giobal
FodComLac
0
500 1000
1500 2000
0
500
1000
1500
Communication rounds
Communication rounds
Figure 8: Loss and test accuracy nication rounds and total costs. Total costs are combined measurement of both communication costs and local computation cost. A communica- tion round has unit cost whilc a local training round has cost т. In a realistic FL system, t is typically much less than 1, the primary bottleneck is often communication and hence we set 7 0.01.
commu-
Comparison among FedAvg, Scaf- fold, FedDyn, FedComLoc-Local, FedComLoc-Com and FedComLoc-Global. First column: K 50%; second column: K 100% (no sparsity).
Figure 9:
method achieves satisfactory convergence over a sufficient number of epochs. Here again, faster convergence is demonstrated with FedComLoc.
4.8 Evaluation on Larger and Practical Datasets
The core contribution of this work is the integration of widely adopted compression techniques into the accelerated local training framework Scaffnew, which achieves superior communication efficiency compared to classical LocalSGD approaches. While sparsity and quantization have been extensively studied in isolation, our method provides a unified approach to combining them with accelerated FL, addressing the nontrivial transition from small-scale or theoretical benchmarks to realistic, large-scale, and heterogeneous FL settings.
In particular, we emphasize the unique challenges posed by biased compressors such as TopK under non- i.i.d. data distributions. We also introduce heuristics to mitigate these challenges. Empirically, our approach demonstrates substantial communication savings while maintaining strong model performance highlighting the practical viability of compression-awant local updates, even in heterogencous environments.
To further evaluate our method on more realistic workloads, we conducted experiments on two widely used FL benchmarks: FEMNIST (Caldas et al., 2018) and Shakespeare (McMahan et al., 2017). FEMNIST is a character-level image classification dataset with 671,585 samples distributed across 3,400 clients, each representing an individual writer's handwriting style. Shakespeare is next-character prediction dataset con- sisting of 16,068 samples partitioned over 1,129 clients, cach corresponding to a speaking role play. Both datasets are inherently non-i.i.d. due to their user-centric partitioning, making them more representative of real-world FL scenarios compared to synthetic Dirichlet splits commonly used in prior work. For FEMNIST, we use a standard two-layer CNN, following Caldas et al. (2018). For Shakespeare, we use stacked LSTM with two layers and 256 hidden units, following McMahan et al. (2017) and FedJAX protocols. We adopt the preprocessing and simulation protocols from FedJAX (Ro et al., 2020).
We evaluated our proposed method, FedComLoc, under 30% sparsity and compared it against SparseFedAvg using the following protocol: (a) client participation rate of p 0.2, where 10 clients are randomly selected per round; (b) evaluation conducted after 3,000 communication rounds.
10

---

Published in Transactions on Machine Learning Research (09/2025)
Table 3: Performance comparison different methods under 30% sparsity.
Method
Compression Type
FEMNIST Acc. (%)
Shakespeare Acc. (%)
FedAvg
Dense
86.63
56.29
SparseFedAvg
Sparse
78.14
43.55
FedComLoc--ocal
Sparse
85.58
54.47
CedComLoc-Com
Sparse
82.31
51.56
FedComLoc-Global
Sparse
83.99
51.87
The results show that FedComLoc maintains strong accuracy under sparsity constraints and consistently out- performs SparseFedAvg, particularly on the more challenging Shakespeare dataset. To quantify communication efficiency, we also compare the number of communication rounds required to reach a target accuracy:
Table 4: Communicttion rounds required to reach target accuracy.
Target Accuracy
SparseFedAvg Comm. Rounds
FedComLoc- Com Comm. Rounds
Dataset
Reduction
FEMNIST 60%
1471
225
84.7%
Shakespeare 35%
2037
466
77.1%
These results confirm that FedComLoc substantially reduces communication costs while maintaining compet- itive accuracy, demonstrating its effectiveness in practical, large-scale federated learning with heterogeneous clients.
5
Discussion and Future Work
5.1 Theoretical Contributions and Extensions
Our work is primarily motivated by theoretical insights from compressed variants of Scaffnew (e.g., Condat et al. (2023)), but the integration biased compressors such as TopK into accelerated local training remains largely empirical and heuristic. In particular, existing convergence guaranlees for compressed local training typically rely on unbiased compression operators and/o or strong convexity condiiionns not satisfied by TopK in our setting. As such, rigorous theoretical guarantees for FedComLoc under these conditions are not yet available, even for convex objectives. Nevertheless, our comprehensive empirical ablations provide strong evidence for the stability and practical efficiency of the proposed approach.
Looking forward, developing a theoretical foundation for FedComLoc with biased compressors represents an important avenue for future research. Progress in this direction may require novel analytical tools or new assumptions to extend convergence theory to biased compression within accelerated local training frame- works. Advancing such thcorctical understanding would help formally justify and further improve the deployment of commuuniattion-llllieenn algorithms like FedComLoe across wide range of federated learning applications.
5.2 Evaluation on Larger and Practical Datasets
In addition to FedMNIST and FedCIFAR10, we evaluate FodComLoc on large-scale FL benchmarks: FEM- NIST (671,585 samples, 3,400 clients) and Shakespeare (16,068 samples, 1,129 clients), which feature sig- nificant heterogeneity and scale. Our method consistently outperforms baselines and achieves significant communication savings (see Table 3/Table 4), demonstrating robustness realistic scttings. While we did not include ImageNet-scale experiments due resource constraints, the proposed framework general and can be applied to even larger models and datasets, which plan to pursue future work.
11

---

Published in Transactions on Machine Learning Research (09/2025)
5.3 Computation Overhead
The additional computation required by Topk and quantization is modest. Topk is implemented with partial sorting, and quantization uses efficient vectorized operations. In practice, for all considered models, the extra time per iteration is negligible compared communiatiin savings, the dominant cost remains in model computation. See Appendix C.5 for wall-clock measurements and implementation details.
5.4 Practical Considerations
While our empirical evaluation conducted in a simulated federated learning environment using high- performance GPUs (NVIDIA A100), this setup consistent with standard practice in the federated learning literature and enables controlled, reproducible comparison across methods. We report wall-clock measure- ments for compression overhead in this context, finding that TopK and quantization add less than 2% per-iteration compute time compared to uncompressed training. However, we acknowledge that in practical deployments on resourco-constrained edge devices, the latency and energy costs associated with certain com- pression operations eepratioos-esspeialllly TopK sparsification- may become more significant, particularly on hardware without efficient vectorized computation. Although a detailed hardware-level energy analysis is beyond the scope of our simulation- based study, we discuss these trade-offs and highlight lightweight alternatives (such as magnitude pruning) as promising directions for future work. We encourage future research in- vestigate joint optimization of communication, computation, and energy efficiency for deployment in diverse federated learning environments.
5.5 Comparisons with Additional Federated Learning Baselines
In designing our empirical evaluation, we have deliberately focused on FedAvg, Scaffold, and their compressed variants as primary basclincs. These algorithms are the most widely-adopted and theoretically comparable methods for studying communication efficiency in federated learning, and their use as benchmarks consis- tent with established practice in the literature. By centering our analysis on these canonical approaches, we ensure clear and controlled comparison that directly highlights the impact of our algorithmic contributions in FedComLoc.
We acknowledge that recent federated learning methods such as FedProx, FedDyn, and FedNova have introduced valuable extensions to address challenges like client drift, heterogeneily, and dynamic aggregation. However, these algorithms often incorporate orthogonal mechanisms and objectives such as proximal regularization, dynamic model correction, or modified aggregation rules-than target aspects of FL beyond communication compression. Including such baselines in our main empirical study would potentially conflate distinct factors and obscure the core contributions of our work.
Importantly, the compression techniques developed in FedComLoc are model-agnostic and complementary to these advanced frameworks. Our approach can, in principle, be integrated with methods like FedProx or FedDyn to jointly address communication bottlenecks and system heterogeneity, and we see this as a promising direction for future research. For the purposes this study, however, we believe that maintaining focus on the most relevant and comparable basclines yields a more principled and interpretable evaluation, allowing for more meaningful scientific conelusions regarding the impact of communicationn-lllieenn local training.
6 Conclusion
This work addresses core challenge in FL: the high communication cost that limits scalability and inclu- sivity in real-world deployments. Building upon the accelerated Scaffnew algorithm, we proposed FedComLoc, which integrates practical compression techniques namely, TopK sparsity and quantization within an ef- ficient local training framework. Through comprehensive empirical studics on standard FL benchmarks, we demonstrated that FedComLoc achieves substantial reductions in communication overhead while maintaining strong model performance, even in highly heterogeneous data settings.
12

---

Published in Transactions on Machine Learning Research (09/2025)
Our experimental design, consistent with established FL literature, prioritizes reproducibility and principled comparisons with the most relevant and theoretically comparable baselines. While our primary evaluations are conducted in a simulated environment with widely adopted FL models, the modular and model-agnostic nature of FedComLoc enables straightforward extension to more complex architectures and real-world de- vice deployments. Furthermore, we highlight both the positive societal impact of communication-elline FI. Fl.cincrasi access for bandwidth- and resource-limited communitics the importance of continued attention to fairness and representativeness, especially when employing biased compression techniques.
In summary, FedComLoc advances the state of the art in commuicctiion-effiieee FL, providing a strong foundation for future rescarch into scalable, equitable, and efficient federated learning systems.
7 Acknowledgment
The research reported in this publication was supported by funding from King Abdullah University of Science and Technology (KAUST): i) KAUST Baseline Research Scheme, ii) Center of Excellence for Generative AT, under award number 5940, iii) SDAIA-KAUST Center of Excellence in Artificial Intelligence and Data Science.
Broader Impact
Our work on FedComLoc advances communicalion-eflccieen federated learning, which has the potential to make machine learning more accessible in muddwitth-oontrainee resource-limited, or rural environments, By reducing the communication burden, our methods can enable participation from wider range of devices and user populations, thus promoting inclusivity and democratizing access to advanced models.
However, we also recognize important ethical considerations. Compression techniques--sspccilll especially biased methods such as TopK sparsification may disproportionately affect clients with minority, rare, or outlier data, potentially leading to undervepresentation in the global model. This is particularly relevant in federated settings with heterogeneous data distributions Such effects could inadvertently reinforce or amplify existing biases, impacting fairness and the representativeness of the trained models.
We believe that it essential to continue investigating the fairness implications of communictionn-eficien learning algorithms. Future work should explore strategies to mitigate potential biases, such as fairness- aware compression, adaptive aggregation, or explicit regularization aimed at protecting minority groups. At the same time, we see significant societal benefit in improving the reach of federated learning systems, making preveccirnss Al more feasible for diverse global communities.
References
Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. QSGD: Communication efficient SGD via gradient quantization and encoding. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Sara Babakniya, Souvik Kundu, Saurav Prakash, Yue Niu, and Salman Avestimchr. Revisiting sparsity hunting in federated learning: Why does sparsity consensus matter? Transactions on Machine Learning Research, 2023.
Jeremy Bernstein, Jiawei Zhao, Kamyar Azizzadenesheli, and Anima Anandkumar. signsgd with majority vote is communication efficient and byzantine fault tolerant. preprint arXiv:18100 012005291, 2018.
Yash Bhalgai, Jinwon Lee, Markus Nagel, Tijmen Blankevoort, and Nojun Kwak. Lsq+: Improving low-
bit quantization through learnable offsets and better initialization. In Proceedings of the IEEE/CVP Conference Computer Vision and Pattern Recognition Workshops, pp. 696-697, 2020.
S. Caldas, P. Wua, T. Lia, Konečný J., B. McMahan, Virginia Smith, and Amect Talwalkar. Leaf: a benchmark for federated settings. preprint
2018
13

---

Published in Transactions on Machine Learning Research (09/2025)
Brian Chmiel, Ron Banner, Gil Shomron, Yury Nahshan, Alex Bronstein, Uri Weiser, et al. Robust quan- tization: One model to rule them all. Advonces in neural information processing systems, 2020.
L. Condat and P. Richtárik. RandProx: Primal-dual optimization algorithms with randomizod proximal updates. In Proc. of Int. Conf. Learning Representations (ICLR), 2023.
L. Condat, I. Agarský, and P. Richtárik. Provably doubly accelerated lederated learning: The first theoreti- cally successful combination of local training and compressed communication. preprint 2Xxx:2210013277 2022.
L. Condat, I. Agarský, G. Malinovsky, and P. Richtárik. TAMUNA: Doubly accelerated federated learning with local training, compression, and partial participation. preprinl arXin:2302. 008202032, 2023.
Tim Dettmers and Luke Zettlemoyer. Sparse networks from scratch: Faster training without losing perfor- mance. preprint arXiv:1907 040004 2019.
Utku Evci, Trevor Gale, Jacob Mcnick, Pablo Samuel Castro, and Erich Elsen. Rigging the lottery: Making all lickets winners. In International Conference on Machine Learning, pp. 2943 2952. PMLR, 2020.
Trevor Gale, Erich Elsen, and Sara Hooker. The state of sparsity in deep neural networks. arXiv:1902.0957 2019.
preprint
E. Gorbunov, F. Hanzely, and P. Richtárik. Local SGD: Unified theory and new efficient methods. In Proc. of Conf. Neural Information Processing Systems (NewrIPS), 2020.
M. Grudzień, G. Malinovsky, and P. Richtárik. Can 5th Generation Local Training Methods Support Client Sampling? Yes! In Proc. of Int. Conf. Artificial Intelligence and Statistics (ATSTATS), April 2023.
Kartik Gupta, Marios Fournarakis, Matthias Reisser, Christos Louizos, and Markus Nagel. Quantization robust federated learning for efficient inference on heterogeneous devices. preprint arXiv:2206.1884, 2022.
F. Haddadpour and M. Mahdavi. On the convergence of local descent methods in foderated learning. preprint arXiv:1910.14425 2019.
Farzin Haddadpour, Mohammad Mahdi Kamani, Aryan Mokhtari, and Mehrdad Mahdavi. Federated learn- ing with compression: Unified analysis and sharp guarantees. In International Conference on Artificial Intelligence and Statistics, pp. 2350-2358. PMLR, 2021.
Tiantian Han, Dong Li, Ji Liu, Lu Tian, and Yi Shan. Improving low-precision network quantization via bin regularization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5261 5270, 2021.
Hong Huang, Lan Zhang, Chaoyuc Sun, Ruogu Fang, Xiaoyong Yuan, and Dapeng Wu. Fodtiny: Pruned federated learning towards specialized tiny models. preprint orXiv:2212.01977, 2022.
Wei Jiang, Sifan Yang, Wenhao Yang, and Lijun Zhang. Ffficient sign-based optimization: Accelerating convergence via variance reduction. arXiv preprint arXic:2406.000 00489, 2024.
Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K Leung, and Leandros Tassiulas. Model pruning enables efficient federated learning on edge devices. IEEE Transactions Neural Networks and Learning Systems, 2022.
P. Kairouz et al. Advances and open problems in federated learning. Foundations and Trends in Machine Learning, 14(1-2):1-20,, 2019.
S. Karimireddy, Kale, M. Mohri, S. Reddi, S. Stich and A. Suresh. SCAFFOLD: Stochastic controlled averaging for on-device federated learning. In Proc. of Int. Conf. Machine Learning (ICMT.,, 2020.
14

---

Published in Transactions on Machine Learning Research (09/2025)
A. Khaled, K. Mishchenko, and P. Richtárik. First analysis of local GD on heterogeneous data, paper presented at NeurIPS Workshop on Federated Learning for Data Privacy and Confi- dentiality, 2019.
A. Khaled, K. Mishchenko, and P. Richtárik. Tighter theory for local SGD on identical and heterogencous data. In Proc. of 23rd Int. Conf. Artificial Intelligence and Statistics (AISTATS), 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Denis Kuznodclov, Eldar Kurtic, Eugenia Iofinova, Elias Frantar, Alexandra Peste, and Dan Alistarh. Ac- curale neural nelwork pruming requires rethinking sparse optimization. preprint arXiv:2308. 020600, 2023.
Yann LeCun. The MNIST database of handwritten digits. http://yann. lecun. htt//exdb/mmiisttt 1998.
X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang. On the convergence of FedAvg on non-IID data. Proc. of Inl. Conf. Learning Representations (ICLR), 2020.
G. Malinovsky, D. Kovalev, E. Gasanov, L. Condat, and P. Richtárik. From local SGD to local fixed point methods for federated learning. In Proc. of 37th Int. Conf. Machine Learning (TCML), 2020.
G. Malinovsky, K. Yi, and P. Richtárik. Variance reduced Proxskip: Algorithm, theory and application federated learning. In Proc. of Conf. Newral Information Processing Systems (NeurIPS), 2022.
A. Maranjyan, M. Safaryan, and P. Richtárik. GradSkip: coammmicattion-acceeeattt local gradient methods with better computational complexity. preprint arXiv:2210.16402, 2022.
B. McMahan, E. Moore, D. Ramage, and B. Agtera Arcas. Federated lcarning of deep networks using model averaging. preprint MxXiv:100.00562) 2016.
H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agũera y Arcas. Communication-efficenni learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
K. Mishchenko, G. Malinovsky, S. Stich, and P. Richtárik. ProxSkip: Yes! Local gradient steps provably lead to communication acceleration! Finally! In Proc. of 39th Inl. Conf. Machine Learning (ICML), 2022.
A. Mitra, R. Jaafar, G. Pappas, and H. Hassani. Linear convergence in federated learning: Tackling client heterogeneity and sparse gradients. In Proc. of Conf. Neural Information Processing Systems (NeurIPS), 2021.
P. Moritz, R. Nishihara, I. Stoica, and M. T. Jordan. SparkNet: Training deep networks Spark. In Proc. of Int. Conf. Learning Representations (ICLR), 2016.
D. Povey, X. Zhang, and S. Khudanpur. Parallel training of DNNs with natural gradient and parameter averaging. preprint arXiv:1410.7455, 2014.
Jae Hun Ro, Ananda Theertha Suresh, and Ke Wu. FedJAX: Federated learning simulation with JAX, 2020. http:ttuuu.cmmoooggggeee
URL
Shai Shalev-Shwariz and Shai Ben-David. Underslanding machine learning: from theory to algoritims. Cambridge University Press, 2014.
Juncheol Shin, Junhyuk So, Sein Park, Seungyoop Kang, Sungioo Yoo, and Eunhyeok Park. Nipq: Noise proxy-based integrated pseudo-quantizztion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognilion, pp. 3852 3861, 2023.
Tao Sun, Qingsong Wang, Dongsheng Li, and Bao Wang. Momentum ensures convergence of signsgd under weaker assumptions. In International Conference on Machine Learning, pp. 33077-33099, PMLR, 2023.
J. Wang et al. A field guide to federated optimization. preprint arXiv:2107 11612107.00917, 2021.
15

---

Published in Transactions on Machine Learning Research (09/2025)
Kai Yi, Laurent Condat, and Peter Richtárik. Explicit personalization and local training: Double commu- nication acceleration in federated learning. preprinl 0rXiv:2305.1377 2023.
Kai Yi, Nidham Gazagnadou, Peter Richtárik, and Lingjuan Lyu. Fedp3: Federated personalized and privacy-friendly network pruning under model heterogeneity. International Conference Learning Rep- resentations (ICLR), 2024.
Dun Zeng, Siqi Liang, Xiangjing Hu, Hui Wang, and Zenglin Xu. FedLab: flexible federated learning framework. Journal of Machine Learning Research, 24(100):1-7, 2023. URL http:://jmlr.orgppappep. v24/22- [] 0440 .html.
Hao Zhang, Chenglin Li, Wenrui Dai, Jumni Zou, and Hongkai Xiong. Feder: Personalized federated learning based on across-client common representation with conditional mutual information regularization. International Conference on Machine Learning, pp. 41314-41330. PMLR, 2023.
16

---

Published in Transactions on Machine Learning Research (09/2025)
Contents
1 Introduction
1
2 Related Work
2
2.1 Local Training
2
2.2 Model Compression in Federated Learning
3 Proposed Algorithm FedComLoc
4
3.1 Sparsity and Quantization
3.2 Introduction of the Algorithms
4 Experiments
5
4.1 Top-K Sparsity Ratios
7
4.2 Data Veterogennityy/Dirichle Factors
7
4.3 CNNs on FedCIFAR10
4.4 Quantization
4.5 Number of Local Iterations
8
4.6 FedComLoc Variants
4.7 FedAvg and Scaffold
9
4.8 Evaluation on Larger and Pract ical Datasets
10
5
Discussion and Future Work
Theoretical Contributions and Extensions
5.2
Evaluation on Larger and Practical Datasets
11
5.3
Computation Overhead
12
5.4
Practical Considerations
12
5.5
Comparisons with Additional Federated Learning Baselines
12
6
Conclusion
12
7 Acknowledgment
13
A Extended Related Work
19
A.1 Sign-Based Commmicationn Efficient Methods
B Experimental Details
19
B.1 Datasets and Models
19
B.2 Training Details..
19
C Complementary Experiments
20
17

---

Published in Transactions on Machine Learning Research (09/2025)
C.1 Exploring Heterogencity
20
C.1.1 Visualization of Heterogeneity
20
C.1.2 Influence of Hetcrogencity with Non-Compressed Models
................................
20
C.2 Complementary Quantization Results
21
C.2.1 Additional Quantization Results FedMNIST
................................
21
C.2.2 Quantization FedCIFAR10
................................
21
C.3 Double Compression by Sparsity and Quantization
21
C.4 Convergence Rate Analysis
22
C.5 Wall-Clock Mcasurements
23
18

---

Published in Transactions on Machine Learning Research (09/2025)
A Extended Related Work
A.1 Sign-Based Communication-Eficien Methods
Recent years have seen significant interest in sign-based gradient compression techniques for distributed and federated learning. Notable among these are SignSGD and its extensions, which use only the sign of the gradient or model update to drastically reduce communication overhead.
SignSGD with majority vote (Bernstein et al., 2018) is a pioneering approach that transmits just one bit per parameter, relying on majority vote aggregation to ensure robustness in homogeneous federated learning settings. Subsequent works, such as momentum-basse sign methods (Sun al., 2023), extend the theo- retical guarantees of SignSGD to broader conditions, including heterogeneous data distributions commonly encountered in FL. More recently, variance-reduced sign-based approackes (Jiang et al., 2024) have demon- strated further improvements in convergence speed and communication elficiency through advanced variance reduction strategies.
While these sign-based tochniques offer extreme commuicattio savings and strong thcorctical propertios in cerlain settings, they differ from the TopK sparsification and quant stralegies primarily considered this work. TopK and quantization preserve more fine-grained model information per round, which can yield improved practical accuracy, especially in highly heterogeneous or non-convex FL problems.
It is important to note that the FedConLoc framework is compatible with wide range of compressors, including sign-based methods. Integrating and empirically comparing sign-based communication within the FedComLoc paradigm is a promising avenue for future research, particularly for scenarios where communi- cation the dominant system bottleneck.
B Experimental Details
B.1 Datasets and Models
Our research primarily focuses on evaluating the effectiveness of our proposed methods and various basclines widely recognized FL datasets. These include Federated MNIST (FedMNIST) and Federated CIFAR10 (FedCTFARIO), which are benchmarks in the field. The use of the terms FedMNTST and FedCIFAR10 is intentional to distinguish our federated training approach from the centralized training methods typically used with MNIST and CIFAR10. The MNIST dataset consists of 60,000 samples distributed across 100 clients using a Dirichlet distribution. For this dataset, we employ a three-layer Multi-Layer Perceptron (MLP) as our default model. CIFAR10, also comprising 60,000 samples, is utilized in our experiments conduct various ablation studies. The default setting for our FedCIFAR10 experiments is set with 10 clients. The model chosen for CIFAR10 is Convolutional Neural Network (CNN) consisting of 2 convolutional layers and 3 fully connected layers (FCs). The network architecture is chosen in alignment with (Zeng et al., 2023).
B.2 Training Details
Our experimental setup involved the use of NVIDIA A100 or V100 GPUs, allocated based on their availability within our computing cluster. We developed our framework using PyTorch version 1.4.0 and torchvision version 0.5.0, operating within a Python 3.8 environment. The FedLab framework (Zeng et. al., 2023) was employed for the implementation of our code. For the FedMNIST dataset, we established the default number of global iterations 500, whereas for the FodCIFAR10 dataset, this number was set at 2500. We conducted a comprehensive grid search for the optimal learning rate, exploring values within the range of [0.005, 0.01, 0.05, 0. Our intention is to make the code publicly available upon the acceptance of our work.
19

---

Published in Transactions on Machine Learning Research (09/2025)
Clet
Dert
Clet
Clerk
Gest
Clest
Clert
Dent
Clert
Geet
Geet
206
400
600
300
2006
120
100
206
206
460
536
600
750
890
206
200
306
406
530
660
706
(a)
0.1
(b) a 0.3
(c) a 0.5
Clerd
Geet
Geet
deet
dver
Clird
deet
dxet
13d
2391
0xer
520
993
202
(d) 0.7
1.0
(I) 1000
Figure 10: Data distribution with different Dirichlet factors on CIFAR10 distributed over 100 clients
C Complementary Experiments
C.1 Exploring Heterogeneity
C.1.1 Visualization of Heterogeneity
The Dirichlet non-iid model serves as our primary means to simulate realistic FL scenarios. Throughout this paper, we extensively explore the effects of varying the Dirichlet factor a and examine how our algorithms perform under different degrees of data heterogeneity. In Figure 10, we present visualization of the class distríbution in the FedCIFAR10 dataset. We visualize the first 10 clients. This illustration clearly demonstrates that smaller results in greater data heterogeneity, with homogeneity. To further our investigation, we conduct thorough ablation studies using values of in the range of [0.1, 0.5, 0.7, 0.9, 1.0]. It is important to note that an a value of 1.0, while on the higher end of our test spectrum, still represents a heterogencous data distribution.
1000 approaching near-
C.1.2 Influence of Heterogeneity with Non- Con-Coepee Models
In our previous analyses, the impact of sparsified models with sparsity factor K 10% was illustrated in Figure 2, and the effects of quantized models were depicted in Figure 7. Extending this line of inquiry, we now present additional experimental results that explore the influence of data heterogeneity on models with K = 50% and Uhose without compression, as shown in Figure 11. Our findings indicate that while model compression can result in slower convergence rates, it also potentially reduces the total communication cost, thereby enhancing overall efficiency. Notably, Dirichlet factor of = 0.1 creates highly heterogencous setting, impacting both the speed of convergence and the final accuracy, with results being considerably inferior compared to other degrees of heterogeneity.
20

---

Published in Transactions on Machine Learning Research (09/2025)
3.0
3.0
2.5
2.5-
2.5
2.5
Training LosS
2.0
2.0
Training
1.5
0.5
100
1.5
0.5
10
Training
1.0
Training
1.0-
1.0
1.0
1.0-
0.5-
0.5
0.0
100
1.5
100
150
Communication rounds
Communication Bitg
Communication rounds
Communicctiin Bits Bitsle0
1.0
1.0
Testing Accuracy
0.8
Testing Accuracy
Testing Accuracy
0.8
Testing Accuracy
0.6
0.6
0.4-
0.5
0.4
0.5
0.3
0.7
0.7
0.7
0.9
0.9
0.2
0.9
0.2
0.9
50
100
150
200
0.0
0.5
1.0
1.5 2.0
0
50
100
150
0.0
0.5
1.0
1.5
Communication rounds
Communication Bitsle8
Communication rounds
Communication Be8
Exploration variations in loss and accuracy across diverse sparsity ratios, communication rounds, and communicated bits is depicted through our figures. The first set four figures on the left showcases results obtained with sparsity ratio of K 50%. In contrast, the corresponding set on the right, consisting of another four figures, represents scenarios where K 100%, indicative of scenarios without model compression.
Figure 11:
C.2 Complementary Quantization Results
C.2.1 Additional Quantization Results FedMNIST
In Figure 7, we presented the quantization results in terms of communicated bits. For completeness, we also display the results with respect communication rounds in Figure 13.
2.5-
1.0
Training Loss
2.0
0.5
2.0
0.5
0.3
Testing Accuracy
0.7
1.5
0.1
Training
0.1
1.0-
Testing
1.0
0.5
0.5
0.2
0.9
3.0
0.0
50
100
150
200
0
50
100
150
200
50
100
150
200
50
100
150
200
Communication rounds
Communication rounds
Communication rounds
Communication rounds
Figure/ 13: FedComLoc utilizing Q+(-) with fixed value of 8 (as shown in the left figure) and 16 (in the
right figure) with respected to communication rounds. We conduct ablations across various a.
C.2.2 Quantization on FedCIFAR10
Previously, Figure 6, we detailed the outcomes of applying quantization the FedMNIST dataset. This section includes an additional series of experiments conducted on the FedCIFAR10 dataset. The results of these experiments are depicted in Figure 14. Consistent with our earlier findings, observe that quantiza- tion considerably reduces communication costs with only a marginal decline in performance.
C.3 Double Compression by Sparsity and Quantization
In Sections 4.1 and 4.4, we individually investigated the effectiveness of sparsified training and quantization. Building on these findings, this section delves into the combined application of both techniques, aiming harness their synergistic potential for enhanced results. Specifically, first conduct Top-K sparsity and then
21

---

Published in Transactions on Machine Learning Research (09/2025)
3.00
3.00
2.75
32 bits
bits.
Training Loss
2.75
2.50
2.50
Training
2.25
2:25
2:00-
2.00
1.75
1.50
100
200
300
400
500
600
700
800
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
Communication rounds
Communication Bits
0.5
bits
bits
0.5
16 bits
bits
Testing Accuracy
Testing Accuracy
0.4
0.3
0.3
0.2
0.1
0.1
100
300
600
2.5
3.0
Communication rounds
Communication Bits
1e6
Figure 14: FedComLoc with Qr(-) on CIFARIO.
X-25%, 22b0s
Training Loss
2.0-
k-100%, 4bits K- 100%, 326Rts
Testing Accuracy
0.8
2.0
25%, 32bits
100%, 4btl
Testing Accuracy
0.8
100%, 32brts
1.5
1.5
Training
0.4
K-25%, 4btts
1.0
0.4
50% 16ats
0.5
0.2
K-100% 4bits
0.5
0.2
K 100%, 42tts
1-100% 32bns
100%, 32brts
Communication Bits 168
Communication 1ob 1cB
Communication Bits 108
Communication Bits 1e8
Figure 15: Comparison with double compression by sparsity and quantization.
quantize the selected Top-K weights. The outcomes of this exploration are depicted in Figure 15. The left pair of figures illustrates that applying double compression with a higher degree of compression consistently surpasses the performance rightmost. figure presents an intriguing observation: considering the communicated bits and convergence speed, there is no distinet advantage discernible between double compression and single compression when they are set to achieve the same level of compression.
lower compression degrees in terms communication bits. However, the
C.4 Convergence Rate Analysis
To assess both the efficiency and effectiveness of FedComLoc, analyze empirical convergence rates measured by the progression of test accuracy and training loss over communication rounds. As illustrated in Figures 1, 3, 6, and 9, FedComLoc and its variants consistently converge to compelitive or superior accuracy compared to the strongest baselines (FedAvg, Scaffold, and sparseFedAvg) while requiring substantially fewer communication rounds. This pattern holds across all considered datascts and heterogeneity settings. In particular, our method achieves target accuracies using up 70% less communication than uncompressod approaches, with convergence speeds that are comparable to better than those of the baseline algorithms.
22

---

Published in Transactions on Machine Learning Research (09/2025)
These results empirically validate that communication-cfficeet local training with compression does not sacrifice learning efficiency final model quality.
C.5 Wall-Clock Measurements
We systematically measured the wall-clock time per local training update to quantify the computation overhead introduced by compression (TopK sparsification and quantization). All experiments were conducted on NVIDIA A100 GPUs (40GB) with PyTorch 1.4.0 and torchvision 0.5.0 in Python 3.8 environment, following the same setup as our main experiments (see Appendix A.2).
For each dataset/mocke combination (FedMNIST, FedCIFARIO, FEMNIST, and Shakespeare), we report the average per-iteration time for three settings: No Compression (vanilla local update), TopK Com- pression, and Quantization. Measurements are averaged over 10 runs, with each run lasting at least. 100 communication rounds and batched local updates to minimize noise.
Table 5 summarizes our findings. For the default models (see Appendix A.1), the overhead from compression is modest: TopK sparsification and quantization increase the per-iteration wall-clock time by less than 2% and 1.5% respectively, relative to the baseline with no compression. We observed that the time required for forward and backward passes is dominant, and the additional time for compression is negligible practice, even for large models such those used in FEMNIST.
Table 5: Average wall-clock time per local update (in milliseconds). Overhead reported relative to No Compression.
Dataset / Model
No Compression
TopK Compression
Quantization
Overhead (%)
FedMNIST (MLP)
0.81 ms
0.82 ms
0.82 ms
<2%
FedCIFAR10 (CNN)
2.20 ms
2.23 ms
2.23 ms
<2%
FEMNIST (CNN)
3.55 ms
3.56 ms
3.57 ms
<1.5%
Shakespeare (LSTM)
2.59 ms
2.62 ms
2.62 ms
<1.5%
In summary, these results confirm that the computational cost of our compression strategies is negligible compared to the overall cost of local training, further supporting the practicality of FedComLoc real-world federated learning deployments.
23


Reviews:
[{"review_id": "LbdWJatqd6", "reviewer": "Reviewer_v34U", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n* The paper clearly defines the compression operators and provides detailed algorithmic steps.\n* The paper is well-written and clearly structured, facilitating easy understanding of methods and experiments.\n* The paper proposes a novel algorithm integrating compression techniques (TopK, Quantization) with the Scaffnew local training algorithm, providing a practical solution addressing real-world FL constraints.\n* The paper conducts thorough ablation studies that systematically evaluate the impact of various algorithmic variants.\n\n\n\n### Weaknesses\n\n* Limited theoretical contributions: The integration of compression techniques into Scaffnew is primarily empirical and heuristic, with limited theoretical analysis provided. While motivated by theoretical insights from related work, the paper does not rigorously analyze the theoretical convergence or complexity of the proposed algorithm.\n\n* Lack of evaluation with large models and complex datasets: Although the authors mention that communication cost is critical in FL particularly with large models (Section 1), experiments presented in the paper use relatively small and simple models and datasets. It remains unclear whether the algorithm maintains robust performance as model size increases, thus limiting the applicability of the results to real-world scenarios where large models truly necessitate communication efficiency. Conducting additional experiments involving large models and more complex datasets, such as CIFAR100 or ImageNet, would enhance the paper’s contribution.\n\n* While the method is validated empirically, practical considerations such as computation overhead introduced by frequent compression steps, are not discussed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents FedComLoc, an algorithm combining local training with sparsity and quantization compression techniques to reduce communication costs in federated learning (FL). Building upon the algorithm Scaffnew, FedComLoc introduces three variants tailored to different FL bottlenecks—uplink, downlink, and local resource constraints. The method is empirically validated using FL benchmarks, demonstrating improvements in communication efficiency without substantial loss in accuracy on FedMNIST and FedCIFAR10.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "### As discussed in the Weaknesses section above.\n\n### Additional Questions\n* Q: What is the Dirichlet parameter ($\\alpha$) used for the heterogeneous setting on FedCIFAR10 in Section 4.3?\n* Q: What specific models were employed for experiments on FEMNIST and Shakespeare datasets as described in Section 4.8?", "broader_impact_concerns": "I have no concerns on the ethical implications of this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "aW7eztRcsj", "reviewer": "Reviewer_D2G4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- This work addresses the communication bottleneck, a major practical issue in FL, so the research question is timely and relevant.\n- The algorithm is a practical integration of TopK and quantization into Scaffnew, which previously lacked support for biased compression.\n- It has detailed ablations on heterogeneity, compression ratios, and local update frequency.\n\nWeakness:\n- The experiments focus on small models (MLP, shallow CNN). It’s unclear if FedComLoc maintains efficiency with modern architectures (e.g., transformers).\n- The lack of convergence guarantees for biased compressors is a significant gap, as prior work (CompressedScaffnew) only handles unbiased cases. This limits FedComLoc’s theoretical foundation.\n- FedComLoc uses fixed compression ratios and learning rates. It could be helpful to dynamically adjust the compression level or learning rate based on convergence and client system capabilities. \n- While FedComLoc reduces communication, the computational overhead of local compression (e.g., Top-K selection) may be non-trivial for edge devices. A discussion on latency-energy trade-offs would strengthen the paper.\n- The paper only compares against FedAvg, Scaffold, and sparse variants. Comparisons with wider baselines can be added, such as FedProx, FedDyn, or FedNova.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces FedComLoc, an algorithm that integrates compression techniques (TopK sparsity and quantization) into the Scaffnew accelerated local training framework for federated learning (FL). FedComLoc has three communication-aware variants, including FedComLoc-Com (compresses client-to-server communication), FedComLoc-Global (compresses server-to-client updates), and FedComLoc-Local (applies compression during local training steps). The authors demonstrate good performance on MNIST, CIFAR10, FEMNIST, and Shakespeare datasets. The experiments also analyze the effect of sparsity, quantization, data heterogeneity, and the number of local updates. They show that FedComLoc can save communication by over 70% with negligible performance drop.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The following modifications can be helpful:\n- Compare with more FL baselines that target the communication issue and present their convergence & overhead tradeoffs\n- Experiment on modern architectures beyond MLP and shallow CNNs\n- Present the convergence rate of FedComLoc and discuss how it compares with the baselines\n- Add analysis on the computational burden of local compression, especially for FedComLoc-Local\n- If possible, explore adaptive TopK or quantization strategies, or at least discuss potential directions", "broader_impact_concerns": "No major ethical concerns are apparent from this paper. However, biased compressors may underrepresent clients with minority or outlier data. The impact on fairness and representativeness of global models under heterogeneous data deserves acknowledgment. A Broader Impact Statement could briefly touch on how communication reduction enables FL in low-resource or rural regions.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eqy3RDu0Al", "reviewer": "Reviewer_4inr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n1. The main strength of the paper lies in the combination of compression techniques (sparsity and quantization) with accelerated local training strategies in federated learning. This approach successfully reduces communication costs without sacrificing model accuracy.\n\n2. The paper presents a comprehensive set of experiments across multiple datasets and configurations, highlighting the effectiveness of the proposed approach.\n\n3. The FedComLoc algorithm and its three variants are described properly, making them accessible and straightforward to understand.\n\n**Weaknesses:**\n\n1. The experiments are mainly focused on small datasets like FedMNIST and FedCIFAR10. It would be beneficial if the authors could extend their experiments to larger datasets.\n\n2. The introduction to the proposed algorithm (Section 3) is somewhat brief. A more in-depth discussion of the methods would help readers better understand the proposed approach.\n\n3. Given that this paper focuses on communication-efficient FL with quantized techniques, it would be better to include a detailed discussion of sign-based methods. Specifically, [1] incorporates signSGD with majority vote and provides convergence guarantees in homogeneous settings. For more challenging heterogeneous environments, [2] introduces a momentum-based sign method with corresponding convergence guarantees. Additionally, [3] effectively improves existing methods via variance reduction, yielding strong experimental results in FL contexts.\n\n[1] J. Bernstein et al., SignSGD: Compressed optimization for non-convex problems.\n\n[2] T. Sun et al., Momentum ensures convergence of SIGNSGD under weaker assumptions.\n\n[3] W. Jiang et al., Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents FedComLoc, a novel algorithm designed to enhance communication efficiency in federated learning (FL). The authors investigate how compression techniques, such as TopK sparsification and quantization, can be integrated with local training strategies to reduce communication costs in heterogeneous environments. Extensive empirical experiments validate the effectiveness of the proposed methods, demonstrating significant communication savings while maintaining competitive model performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Extend the experiments to include larger datasets to better assess the proposed method.\n\n2. Provide a more detailed discussion of the proposed methods.\n\n3. Expand the related work section to include a detailed discussion of sign-based methods.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)