Clients Collaborate: Flexible Differentially Private Federated Learning with Guaranteed Improvement of Utility-Privacy Trade-off
Authors: Yuecheng Li
Lele Fu
Tong Wang
Jian Lou
Bin Chen
Lei Yang
Jian Shen
Zibin Zheng
Chuan Chen
Venue: ICML 2025 poster (2025)
Keywords: Federated Learning
Differential Privacy
Utility-Privacy Trade-off
Primary area: social_aspects->privacy
TL;DR: In this paper, we formulate a novel federated learning framework that achieves a guaranteed and high-quality trade-off between model utility and user privacy.

Abstract: To defend against privacy leakage of user data, differential privacy is widely used in federated learning, but it is not free. The addition of noise randomly disrupts the semantic integrity of the model and this disturbance accumulates with increased communication rounds. In this paper, we introduce a novel federated learning framework with rigorous privacy guarantees, named **FedCEO**, designed to strike a trade-off between model utility and user privacy by letting clients "***C**ollaborate with **E**ach **O**ther*". Specifically, we perform efficient tensor low-rank proximal optimization on stacked local model parameters at the server, demonstrating its capability to ***flexibly*** truncate high-frequency components in spectral space. This capability implies that our FedCEO can effectively recover the disrupted semantic information by smoothing the global semantic space for different privacy settings and continuous training processes. Moreover, we improve the SOTA utility-privacy trade-off bound by order of $\sqrt{d}$, where $d$ is the input dimension. We illustrate our theoretical results with experiments on representative datasets and observe significant performance improvements and strict privacy guarantees under different privacy settings. The **code** is available at https://github.com/6lyc/FedCEO_Collaborate-with-Each-Other.

Clients Collaborate: Flexible Differentially Private Federated Learning with Guaranteed Improvement of Utility-Privacy Trade-off
Yuecheng Li
Lele Fu'
Tong Wang²
Jian Lou
Bin Chen Chuan Chen
Lei Yang'
Jian Shen
Zibin Zheng'
Abstract
1. Introduction
Federated learning (FL) (McMahan et al., 2017), a privacy- preserving distributed machine learning paradigm, enables multiple parties to jointly learn a model under global scheduling while keeping the data from leaving the local client. Nevertheless, recent work has shown that at- tacker (i.e., the server or a particular client) can steal raw training data (Zhu et al., 2019; Jeon et al., 2021) and even specific private information (Fowl et al., 2022) by inverting parameters (gradients) uploaded from other clients. To fur- ther enhance privacy safeguards, differential privacy (DP) (Dwork, 2006; 2010) has become the prevailing standard in privacy- privacy-preserving machine learning (Jain et al., 2018; 8; Levy et al., 2021). This privacy computing technique has proven effective in federated learning, guarding against client (user) privacy breaches by introducing random noise to uploaded updates (McMahan et al., 2018; Jain et al., 2021; Malekmo- hammadi et al., 2024). Unfortunately. randomized mecha- nisms like DP may result in sacrifice of model utility, es- pecially the number of communication rounds increases (Yuan et al., 2023). Overall, the key issue is how to achieve an improved utility-privacy trade-off in differentially private federated learning (DPFL), which constitutes a novel and challenging research direction (Bietti et al., 2022; Cheng et al., 2022: Shen et al., 2023; Tsoy al., 2024).
To defend against privacy leakage of user data, differential privacy is widely used in federated learning, but it is not free. The addition of noise randomly disrupts the semantic integrity of the model and this disturbance accumulates with increased communication rounds. In this paper, we introduce novel federated learning framework with rigorous privacy guarantees, named FedCEO, designed to strike a trade-off between model utility and user privacy by letting clients "Collaborate with Each Other". Specifically, we perform efficient tensor low-rank proximal optimization
stacked local model parameters at the server, demonstrating its capability to flexibly truncate high-frequency components in spectral space. This capability implies that our FedCEO can effectively recover the disrupted semantic information by smoothing the global semantic space for different privacy settings and continuous training processes. Moreover, we improve the SOTA utility-privacy trade-off bound by order of vd, where d is the input dimension. We illustrate our theoretical results with experiments on representative datasets and observe significant performance improvements and strict privacy guarantees under different privacy settings. The code available https://githut. com/ wwt.e//mmmee/eeeeeeeeeeee FFddCEO Coll Collaborate-iitt--hchh---h---
Considering the randomness of the introduced noise for differential privacy, the specific semantic information that gets corrupted can differ across individual clients. Mean- while, there exists certain similarity between the data distributions of each client, leading to the correlation in their semantic spaces. Therefore, we propose mitigate the impact of DP noise on the utility of the global model in federated learning by exploiting the semantic complemen- tarity between the noisy parameters of different clients. To substantiate our motivation, we visualize the smoothness of the global semantic space at different stages of training (see the heat map in Figure 1). Specifically, we compute Laplacian regularization values for the last lincar layer of the backbone along the client-side direction, which serves as metric for assessing the smoothness of the global semantic space (Yin et al., 2015; Pang & Cheung, 2017). Furthermore,
'Sun Yat-sen University, Guangzhou, China Teexa A&M Uni- versity, Texas, USA 'Harbin Institute of Technology, Shenzhen, China "Zhejiang Sci-Tech University, China. Correspondence to: Chuan Chen <chenchuan@ sys..e.yu.............
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

FedCEO: Flexible Differentially Private Federated Learning
Class
Class
Class
Class 4
Class 5
Class
Class
Cless 8
Class
Class
clent
57%
clent
client44 4
clicnt
33%
28%
25%
33%
clent 7
client 8
client
Local Train
Local Train +DP
Local Train DP + Ours
Figure 1. The heat map illustrates the smoothness of the global semantic space for ten clients at different training stages, where the color gradient from red to blue signifies an increase smoothness. The bar chart illustrates the testing accuracy on the tenth class each client during different training stages, where clients experiencing significant degradation in the semantic understanding of the tenth class are highlighted in pink, while others are marked in green. Due to the randomness of DP noise, we observe significant degradation in the local semantic representation of 10-th class for clients 3, and among the ten clients (ACC significantly reduced), while the impact other clients is relatively minor. Consequently, the corresponding blocks the tenth row of the heat map matrix turn red, indicating a decrease in the smoothness of the global semantic space. In contrast, evidenced by the recovery of the blue color in the tenth row of the heat map matrix. Simultaneously. the local semantic representation of class for clients and 6 is restored (ACC improved), based the collaboration among all clients.
approach enhances the smoothness of the global semantic space,
we present the testing accuracy for each client concerning the 10-th class on the CIFAR-10 10 dataset with two-layer multi-layer perceptron (MLP) (see the bar chart Figure 1). Taking the 10-th class in CIFAR- 10 as an illustrative exam- ple, we observe that the introduction of differential privacy disrupts the smoothness of the global semantic space for the tenth class (as evidenced by the reddening of the tenth row). Simultancously, there is significant decline in testing accuracy for this class across clients 1, 3, and 6. Following proposed low-rank processing, ment in the smoothness of the global semantic space for the tenth class (as evidenced by the bluing of the tenth row), concomitant with an increase accuracy for clients 1, 3, and Moreover, a similar phenomenon is noted for the seventh class (as observed in the seventh row of the heat map). This emphasizes that smoothing the global semantic space based the semantic complementarity of noise pa- rameters between clients is key to addressing the declining utility of models in the DPFL framework. While previous works have also focused on improving utility in DPFL., they are fundamentally based on restricting the local updates (Cheng et al.. 2022; Shen al., 2023) without considering the collaborative relationship across different clients. In perspective the
an inter-client semantic collaboration approach.
Contributions. In this work, we propose optimization algorithm in the server based on tensor low-rank techniques, which offers a flexible approach to smoothing the global semantic space for different privacy settings and continuous training processes. We provide rigorous theoretical anal- ysis and extensive empirical evidence to substantiate our proposed framework. Specifically, our contributions summarized as follows:
observe enhance-
We introduce novel federated learning framework, FedCEO, characterized by stringent privacy guaran- tees and enhanced model utility. We establish its equiv- alence the spectral space to the truncated tensor sin- gular value decomposition (T-SSV) algorithm (Kilmer Martin, 2011), showing its ability achieve smooth- ness by truncating high-frequency components in the global semantic space. Benefiting from the T-SVV, we can flexibly control the degree of semantic comple- mentarity between clients according to noise levels.
2. We thcoretically prove a new utility-privacy tradeoff bound for our FedCEO yielding a notable improve- ment O(vd) over previous SOTA results due to low-rankness, where is the input dimension.
other words, our work provides utility-privacy or in federated learning by exploring
2

---

FedCEO: Flexible Differentially Private Federated Learning
3. We empirically demonstrate that our model utility out- performs previous work under various model archi- tectures and privacy settings, Furthermore, we also employ a common gradient inversion algorithm named DLG (Zhu et al., 2019) to attack our framework, val- idating its robust preecy-pres performance. In summary, we demonstrate that our approach achieves the best trade-off between utility and privacy within the framework of DPFL.
formulation of the optimization problem:
[tw)= min WER
f(w)
1 K
Σ Su (wr) K.S.
(1())
(I)
where N is the total number of clients, with a selection of a subset S. of size in each round. Let w denote the parameters of the global model, and D2 denote the local dataset of client k. Let fk == ED-D, [F(w; bj] denote the empirical risk for client k. where b represents randomly sampled mini-batch from the local dataset Dr.
1.1. Related Work
To achieve the goal of collaborative training without ex- posing local data, federated learning employs parameter transmission followed by aggregation. However, FI. intro- duces challenges not encountered in centralized learning, such as gradient conflicts arising from data heterogeneity and privacy risks due to gradient leakage. Our paper primar- ily focuses on privacy concerns in FL and explores further utility guarantees.
To ensure formal privacy guarantees, federated learning with differential privacy has undergone extensive investigation (Kim et al., 2021; Naseri et al., 2022). Recent efforts have been concentrated on user-level differential privacy, aim- ing to enhance model utility (MeMahan al., 2018; Wei et al., 2021; Shen al., 2023). In alignment with these endeavors, we also adopt user-level differential privacy and employ it locally to safeguard against potential adversarial threats to the server (Fowl et al., 2022; Hu et al., 2024). Moreover, to further enhance the model utility of DPFL, current methods mainly investigate techniques such as reg- ularization or personalization, fundamentally constraining the size of uploaded updates. (Cheng et al., 2022) proposes two regularization techniques: bounded local update regu- larization and local update sparsification. These methods enforce constraints to reduce the norm of local updates. In a more natural paradigm, PPSGD (Bietti et al., 2022) intro- duces a personalized optimization algorithm designed for training additive mod- els with user-level differential privacy. It decomposes the model into the sum of local and global learning components, selectively sharing only the global part. However, extending this method to more complex personalized models proves challenging. Work closely related to ours are (Jain et al., 2021) and CENTAUR (Shen et al., 2023), who perform sin- gular value decomposition (SVD) on noisy representation matrices of clients individually at the server. They verify that handling such issues in the spectral space is promising. However, their methods independently explore spectral in- formation. In contrast, we stack the noisy models into a higher-order tensor, capitalizing on the semantic comple- mentarity among clients to further enhance model utility in DPFL.. Consequently, we improve the utility-privacy trade- off from their 5d) and (dd to our (Vd)).
2.2. Differential Privacy
Differential Privacy (DP) is privacy computing technique with formal mathematical guarantees. We commence by presenting the definition of base DP.
Definition 2.1 (Differential Privacy (Dwork, 2006; Abadi et al., 2016)). randomized mechanism M : :D - with data domain D and output range R gives (c. -ddifferrenta
privacy if for any two adjacent datasets D, D' 'D that differ by one record (add or remove) and all outputs S C R it holds that
stochastic gradient
Pr(M(D) E Pr((()) E 6
where e and ö denote the privacy budget and the relaxation level, respectively. It indicates the hardness of obtaining information about one record in the dataset by observing the output of the algorithm M, especially when the privacy budget e is smaller.
User-level DP: In this paper, we employ user-level dif- ferential privacy, which naturally suits federated learning, shifting the protected scale to individual user (Dwork, 2010; MeMahan et al., 2018). In other words, one record in Defi- nition 2.1 consists of all data belonging to single client.
2.3. Differential Privacy in Federated Learning
2. Preliminaries
User-level differential privacy provides formal privacy guar- antees for individual clients and has found widespread appli- cations in federated learning. (McMahan et al., 2018) first introduces Av- and De-FedSGD, formally incorpo- rating user- DP into the FL framework. These methods involve clipping per- user updates at the client side, followed by aggregation and the addition of Gaussian noise at the
2.1. Federated Learning
Federated learning (FL) (McMahan et al., 2017) is a multi- round protocol involving a central server and a set of local global model. In the federated learning, we address the following
clients collaboratively training a
3

---

FedCEO: Flexible Differentially Private Federated Learning
Algorithm 1 UDP-FedAvg
izes the correlation between elements in rows columns, finding extensive applications in areas such image denois- ing (Ren et al., 2022) and data compression (Idelbayev & Carreira-Perpinál 2020). This paper focuses primarily on the low-rank property of third-order tensors, employing ten- sor nuclear norm (TNN) (Yang et al., 2016; Lu et al., 2016) for characterization and the tensor singular value decom- position (tSVD) (Kilmer Martin, 2011: ;; Lu et al., 2019) algorithm for modeling. The following sections provide detailed introduction to these relevant concepts.
Input: K: number of participating clients each round, T: communication rounds, C: clipping threshold,
a: noise multiplier, y/- learning rate, E: local epochs. Output: w/ wT)): global model.
1: Initialize global model wo( randomly
2: for t= to T do
3: Take a random subset St of K clients (Total N)
4: for all clients & in parallel do
5: (t) ClientDPUpdate (w' (t k)
6: end for
The Discrete Fourier Transform (DFT) is implicated in sev- eral related concepts introduced later in this paper. We de- note discrete Fourier transform as DFT(-), and more details are deferred the Appendix A.1.
7: (1()) R Ekes, w/6(0 (1)
8: return w/ (t)
9: end for
10: return w' (T)
For each third-order tensor W, we perform the discrete Fourier transform along its third dimension, denoted as
ClientDPUpdate(uo,. A:)
W = DFT(WW 33), where W represents the i-th frontal slice of W. Similarly, we have W = IDFT(W, 3), indi- cating the inverse discrete Fourier transform along the third dimension.
1: Initialize wh (i.e. w' (t- 1))
2: for e to E do
3: Take random split B from local dataset DR (i.e. mini-batches with size B)
4: for batch E B do
5: w w= 7 ((ann)e V (w; x, 8)
2.4.1. TENSOR NUCLEAR NORM
6: end for
The Tensor Nuclear Norm (TNN) is often employed in opti- mization problems as a metric for the low-rank property of tensor. A smaller nuclear norm indicates a lower rank for the tensor, implying stronger smoothness among its slices.
7: end for
8: A w Wp
9: Á GradientClip(A)
10:
wo + 7 Á + NO, (0209KK)
Definition 2.2 (Tensor Nuclear Norm (Lu et al., 2016)), For each third-order tensor W E R" Xnig X it defines the tensor nuclear norm, denoted as || ||-|l,, as follows:
11: return w/
GradientClip(A)
1 ng
1: Á A/ max (1, |AA| ( )
W = 73
2: return Á
It means the average of the matrix nuclear norm of all the frontal slices of W, where W is tensor after DFT on W along the third dimension.
server side, which ensures privacy protection for "large step" updates derived from user-level data. To further address po- tential malicious actions at the server (Fowl et al., 2022), we introduce user-level differential privacy (UDP) locally (Truex et al., 2020), as shown in Algorithm Although the aforementioned algorithms provide user-level privacy guar- antees in federated learning, they often result in significant degradation of model utility. In our work, we specifically focus on strategies to enhance the utility of FL models while maintaining stringent privacy guarantees.
2.4.2. T-PRODUCT AND TENSOR SVD
The tensor singular value decomposition (tSVD) can be employed to approximate low-rank tensors. To do so, we first introduce the concept of the tensor-tensor product (t- product), denoted as V (Kilmer Martin, 2011). More details are deferred Appendix A.2.
Based the t-product, we define tensor singular value decomposition (tSVD) follows.
2.4. Low-rankness
Definition 2.3 (Tensor Singular Value Decomposition (Kilmer & Martin, 2011; Lu et al., 2019)). For each third- order tensor W E Rm Xna it can be factored in
The low-rank property is crucial characteristic of both nat- ural and artificial data, offering a description of dependence relationships among different dimensions. For instance, in the case of a second-order matrix, low-rankness character-
W S HH

---

FedCEO: Flexible Differentially Private Federated Learning
Algorithm 2 FedCEO
at the server. The specific objective function is as follows:
Input: K: number of participating clients each round, T: communication rounds, C: clipping threshold, a: noise multiplier, V/= learning rate, F: local epochs, I: interval, A: initial coefficient, d: common ratio. Output: w' (T): global model.
{ (1/0+ -ww-will [IWI.... (2)
W -aag
W
Every rounds, we fold the noisy parameters uploaded by clients into a third-order tensor Wy Rdxxxk where d represents the input data dimension, fo denotes the network dimension, and K signifies the number of clients selected in each round. Subsequently, we impose constraints on its low-rank structure using the previously introduced TNN, denoted as | Furthermore, to prevent trivial solutions, we apply an offset regularization term based on the Frobe- nius norm, denoted as I Wvll? It also serves as proximal operator, ensuring the convergence of the op- timal point W (Cai et al., 2010). In the next section, we prove that the constructed optimization objective in Eq. (2) is equivalent to the T-SVV algorithm with the adaptive soft softthhresholding rule in Theorem 3.1, where the truncation threshold defined by a geometric series, denoted as z 0t.
Initialize global model ue( randomly
2: for 1 1 to 7 do
3: Take random subset St of K clients (each client k is selected with probability p }
4: for all clients & in parallel do
5: if (1- 1 = 0 then
6: (t(() = ClientDPUpdatee (u0x (t - 1). k)
7:
else
8:
w Clien(DPUpdate (w (t 1) k))
9:
end if
10:
end for
11:
if t%I = 0 then
WN fold (t), ((()),.,,,,k((()))) (t))))))
12:
W arg minw (1/01 lI Wr [W]]
13:
3.2. Analysis
(ux ()) unfold (w))
14:
elucidate the role of our low-rank proximal optimization objective the server, we introduce the following theorem.
return {20k (t)
16: else
Theorem 3.1 (Interpretability). For each T 2 0 and WN R our tensor low-rank proximal optimiza- tion objective defined algorithm 2 obeys
17:
w w'(t)= k Exes, (t)
18:
return w/ (1)
19:
end if
20:
end for
T+SVDDWW } arg min (rI - WArll + [1I1I.) W
21:
return w/ (T))
(3)
where T-tSVD(-) is a truncated tVV operator and the truncation threshold, defined as follows:
1 is
27
where U E Xn; Xnn Xns, V E Ru2XnaXn are orthogonal, i... U x l" I and V * v = I, where (-)" denotes conjugate transpose. S E IR Xna is an f-diagonal tensor, whose each frontal slice is diagonal.
T-LSVD(W,
2T == + DH
1 = 4
where D is f-diagonal tensor whose each frontal slice in the Fourier domain is D (j.j) max (i.j) #3.
j S min(d, h), 1. KK
Note that tSVD can be efficiently computed in the Fourier domain using matrix SVD, as detailed in Appendix A.3. Furthermore, by truncating smaller singular values (or by retaining the larger singular values), we can decompose the tensor into a lower-rank part, referred to as the truncated ISVD (T-SVD). We defer its algorithm Appendix A.4.
Proof. proof is given in Appendix B.1
Combined with Remark B.2 deferred to Appendix B. it indicates that our approach contributes to a smoother global semantic space by flexibly truncating the high-frequency components of the parameter tensor.
3. Main Algorithm
Specifically, our theorem elucidates that as the regulariza- tion coefficient in Eq. (3) decreases, corresponding to larger truncation threshold 27 in T-SVD, leading to smoother global semantic space. In our objective function, we set the coefficient to A/0 (9 > 1), corresponding to an adaptive threshold of 1 03. This implies that with the 2x
3.1. FedCEO
Our main algorithm, as illustrated in Algorithm 2, represents a local version the user-level differentially private feder- ated learning framework, accompanied by a tensor low-rank proximal optimization acting on stacked noisy parameters
accumulation of noise (i.e., the increase in communication
5

---

FedCEO: Flexible Differentially Private Federated Learning
rounds (), the server coordinates the gradual enhancement of semantic smoothness (collaboration) among the various clients, akin to CEO. Furthermore, we can choose ap- propriate initialization coefficients À for different privacy settings. Specifically, for stronger privacy guarantees (larger Gaussian noise), selecting a smaller À results in a smoother global semantic space. Furthermore, we derive the follow- ing corollary.
16
14-
W1
W2
Singular value
12-
Ws
W
W
FaCEO T 1/89
Proposition 3.2. Given that W RRxxixK and the reg- ularization coefficient T 20m
_ in Eq. (3), have
FedAug 01. 1/40
T-tSVD(W, Fm) with truncation threshold Om- If om larger than the highest singular value of w for i = K. the updated parameters in our FedCEO will degenerate to the low-rank approximation of the global
FOU -/-= 1(2)
0
singular value index
parameter in FedArg.
Figure 2. The visualization of the low-rank proximal optimization process, where w, is the (i 1)-th frequency component.
Formally, for all ls .,..., K we have
Im [T-tSVD(W, om)) (t) TruncatedSVD(W,
K
or (w
4. Utility-Privacy Trade-off Analysis
W
Om max 25i5n
In this section, we provide theoretical analysis of the utility and privacy guarantees for our FedCEO. We denote e, and Cp as the model utility and the privacy budget, respectively. Furthermore, we establish an improved utility-privacy trade- off bound E. Ep I O(vd/K). In comparison to recent advanced work, our approach exhibits an improvement of O(vd) over the current SOTA result (i.e. O(d/K) K) as re- ported in (Shen al., 2023)).
where W is the parameter tensor in FedCEO, W the aggerated average parameter in FedAvg and TruncatedSVD(() the truncated SVD operator for matri- ces (Defined in Appendix A.4).
Proof. A proof is given in Appendix B.2.
Hence, can infer that when the truncation threshold is appropriately chosen (allowing the parameter tensor to re- tain only the first frequency component), our method can approximately degenerate into FedAvg. Based on the above analysis, we can visualize the low-rank proximal optimiza- tion process through dynamic singular value truncation in the spectral space, as illustrated in Figure 2.
4.1. Utility Analysis
Firstly, we introduce generalized definition of utility, which encompasses the definitions in the latest works.
Definition 4.1 (Utility Loss (Zhang et al., 2022)). The model utility is characterized as the difference in perfor- mance when utilizing the protected model information sam- pled from the distribution P% compared to that drawn from the unprotected distribution Pk.
FedAvg vs. FedCEO: In contrast to our FedCEO, FedAvg implies retaining only the lowest-frequency components in the spectral space, representing coarse- santse-granedd mutual col- laboration that lacks the adaptability to different DP settings and continuous training processes. Our approach, character- ized by fiexible complementarity of semantic information due to adaptive truncation thresholds, demonstrates superior model utility in DPFL.
K E
Un(P)) - Up(P))
K
where Pk and Ps. represent, respectively, the distribu- tions of convergent models with without DP. U& (P) Ew: 1 Eses U(W,, b)] denotes the expected utility taken with respect to Wt ~ P/ or Pk: and U is any metric measuring the performance of the model.
3.3. On the Scalability
The complexity of our approach is O K min(d,dt11;
where is the number of layers, K is the number of clients and min(didf_,, d is the com- plexity of matrix-SVD for each layers. To scale to larger models, we can perform T-SVD to the last few layers only since these layers will be narrower than other layers but contain more semantic information.
In this paper, we denote Pa: as the local model distribution in FedAvg without DP, and P as the local model distribution in our FedCEO with DP. Let U represent the empirical loss of the model. We define our utility as follows by setting Ps = Wk. Pk- = w/. and = F.
6

---

FedCEO: Flexible Differentially Private Federated Learning
Definition 4.2 (Model Utility). We define the utility of the model within the differential privacy federated learning framework follows:
Theorem 4.5 (Privacy Analysis of FedCEO).. Suppose that the privacy budget cp < cIq let be the noise multiple (a tumable parameter that controls the privacy-utility trade- off), Algorithm satisfies user-level (cp. 6 with
Σ [s((W)) (la4)))((wx))) [w(w)) k=1 =s(w') I(w),
C2K /T.KTllg(1//)
K
K
k=1
Gp =
No
Proof. proof is given in Appendix B.4
where w' and w" represent, respectively, convergent global models of FedCHO or FedAvg. f& denotes local empirical risk and denotes global empirical risk.
In summary, demonstrate that our FedCEO satis- fies (<p, -)-fifeerent privacy with privacy budget fp = o2 K (1g((1))
Based on the above definitions, we present the utility guar- antee for our FedCEO as follows.
providing theoretical privacy guarantees.
No
Theorem 4.3 (Utility Analysis of FedCEO). Suppose that Assumptions B.5, B.6 and B.7 hold and let local empiri- in cal risk fi satisfies Definition B.4. Set 12 L
4.3. Guaranteed Improvement of Utility-Privacy Trade-off
On the one hand, Theorem 4.3 indicates that achieving high utility (i.e., small 6,) necessitates selecting a small noise multiplier a. On the other hand, Theorem 4.5 asserts that achieving high privacy (i.e., small 6p) necessitates selecting large noise multiplier Next, we unify the utility and privacy analyses of our FedCEO in the DPFL setting to establish the overarching utility-privace trade-off.
Eq. (3) and 0 < m < 1/p, where m =
2y²
((NNBB4) + 2La(1+3)2B* VK
Ky²
Algorithm 2 satisfies
cu = f (w') V2L1 (Vito vaVhCo) 1 2pm
Corollary 4.6. Lel Eu and Ep denote the model utility and the privacy budget, respectively. Under the Assumptions B.5 to B.7 and the conditions outlined in Theorems 4.3 and 4.5, Algorithm 2 satisfies
K
where r is the rank of the parameter tensor after our pro- cessing and is the dimension of input data.
evd.
Proof. A proof is given in Appendix B.3
In summary, we demonstrate that our FedCEO satisfies an outstanding utility bound with e, < 0( ( ( providing theoretical utility guarantees. Moreover, achieving low-rank global semantic space of the whole parameters (i.e. r & d),
where d is the inpul dimension and N is the total number of clients. è hides the constants and log terms.
Overall, our FedCEO achieves a utility-privacy trade-off bound c,. Cp S SO Va
we have 6u S <
Summary: Under the unified settings of utility (Defini- tion 4.1) and privacy (Definition 2.1), compared to exist- ing SOTA results, we improve the utility-privacy trade-off bound from (dd (Theorem 5.2 in (Jain al., 2021)) and O(d/N) (Corollary 5.1 in CENTAUR (Shen al., 2023)) O(Vd/N). In contrast, our approach leverages higher-order tensor algorithms to integrate the processes of parameter partition and semantic fusion, resulting in a notable improvement with O(vi) and providing a better guarantee for the trade-off between utility and privacy in DPFL.
4.2. Privacy Analysis
We first establish general privacy guarantee for Algorithm extending the theoretical results of data-level DP from (Abadi et al., 2016) user-level DP.
Lemma 4.4 (Privacy Analysis of UDP-FedAvg (Cheng al., 2022)). There exist constants Ci and so that given the clients sampling probability
K N and the number of communication rounds T, f.for any e < CTPTT, Algorithm
I satisfies user-level ()P for any 8 > 0 if we choose cap/Tlog(11/
5. Experiments
The above lemma proves that the model parameters up- loaded by each client have strict privacy guarantees. Fur- thermore, taking into account the tensor low-rank proximal optimization at the server, we present the privacy guarantee for our FedCEO follows.
In this section, we present empirical results on real-world federated learning datasets, validating the utility guarantees, privacy guarantees, and the advanced trade-off between the two provided by our FedCEO algorithm.

---

FedCEO: Flexible Differentially Private Federated Learning
Table 1. Testing accuracy (%) on EMNIST and CIFAR- 10 under å = 10 and various privacy settings with three common Sy. A larger
9 indicates a smaller Cp. i.. the stronger privacy guarantee. stands for the scaling factor of the parameter A.
Dataset
Model
Setting
UDP-FedAvg
PPSGD
CENTAUR
FedCEO (i==1
FedCEO (# >1)
0g 1.0
76.59%
77.01 %
77.26%
77.14%
78.05 %
EMNIST
MLP-2-Layers
gg = 1.5
69.91%
70.78%
71.86%
71.56%
72.44 %
0g = 2.0
60.32%
61.51%
62.12%
63.38%
64.20%
0g = 1.0
43.87%
49.24%
50.14%
50.09%
54.16%
CIFAR-10
LeNet-5
4 = 1.5
34.34%
47.56%
46.90%
48.89%
50.00 %
gg = 2.0
26.88%
34.61%
36.70%
37.39%
45.35
Testing accuacy (%)
45
Fedligg Privacy L.D-FFedogg Privacy (P5N8-10.33)
40
(1572--19773)
(2552-10.15)
Figure 3. Privacy protection performance of three federated Iearn- ing frameworks on 10. Both FedCEO and UDP-FedAvg demonstrate robust defense against privacy attacks with smaller Peak Signal-to- -o-isse Ratio (PSNR). while DLG successfully in- fers sensitive images from clients in FedAvg.
FedC20
CENTAUR
25
PPSGD
UDPFedAvg
0.1 (9.551
1.0 (0.89)
15 (0.48)
2.0 (0.34)
2.5 10.26)
(privacy budget Ep) with -0
Figure 4. Suility-Privacy Trade-off off for our FedCEO and other meth- ods on CIFAR-10.
5.1. Experiment Setup
We set the clipping threshold C' uniffrrlly to 1.0 and use three different privacy settings corresponding to 0g = 1.0, 1.5, and 2.0. 0g is proportional to the noise multiplier a, defined in the privacy engine from the Opacus package (Yousefpour et al., 2021). 1. We conduct experimental evalua- tions on both MLP and LeNet model architectures. reporting the testing accuracy of the global model (consistent with representative papers (Bietti et al., 2022; Shen et al., 2023) from recent years). Specific model structures and hyperpa- rameters are detailed in Appendix C.I.
5.3. Privacy Experiments
In Figure 3, our FedCEO and UDP-FedAvg exhibit simi- privacy protection performance for user data, validating the privacy analysis in Section 4.2. Specifically, we input the locally uploaded LeNet model (gradients) from three different FL frameworks into the Deep Leakage from Gra- dients (DLG) (Zhu et al., 2019) algorithm and set attack iterations to 300 with learning rate of 0.01. For our Fed- CEO, the DLG attack is conducted after the tensor low-rank proximal optimization verify that our operation does not compromise the privacy protection of DP. Then we report the inference images and their PSNR (dB, 4) values in Fig- ure 3. Refer to Appendix C.2.4 for more details.
5.2. Utility Experiments
In Table 1, our FedCEO demonstrates state-of-the-ar per- formance under different privacy settings, aligning with the utility analysis in Section 4.1. Additionally, we can adapt to various privacy settings by flexibly adjusting initial coefficient A. Specifically, larger og corresponds smaller (lower rank, i.c. smaller r). Moreover, our adaptive mech- anism increases the truncation threshold (i.e. 2A (t) as the Gaussian noise accumulates during the FL. training process, due to the geometric series with common ratio 0 > 1. Refer to Appendix C.2.1, C.2.2 and C.2.3 for experiments about model training efficiency, heterogeneous FL settings and other local model well dataset.
5.4. Improved Utility-Privacy Trade-off'
In Figure 4, we observe that our FedCEO exhibits the best utility performance across various privacy settings. More- over, compared to other privacy-preserving methods, our model shows the smallest decrease in testing accuracy as pri- vacy is enhanced (i.e. J, increases and G, decreases). This indicates that our FedCEO achieves a best utility-privacy trade-off, consistent with the analysis in Section 4.3.
8

---

Huang, S., Fu, L.,, Li, Y., Chen, C., Zheng, Z., and Dai, FedCEO: Flexible Differentially Private Federated Learning
6. Conclusion
and sparsification. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp- 10122-10131, 2022.
In this paper, we explore the concept of flexible seman- tic smoothness (collaboration) among clients, providing theoretical guarantees for both utility and privacy. Our ap- proach achieves an improved utility-privace trade-off bound of (Vvd) compared to the current SOTA results. Further- more. we conduct comprehensive experiments to validate the utility and privacy across different datasets and privacy settings, demonstrating the advanced performance consis- tent with our theoretical analysis. the future, we aim to extend tensor low-rank techniques to heterogeneous feder- ated learning, addressing challenges of gradient conflicts.
Dwork, C. Differential privacy. In International colloquium automata, languages, and programming, pp. 1-12. Springer, 2006.
Dwork, C. Differential privacy in new settings. In Proceed- ings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms, pp. 174-183. SIAM, 2010.
Fowl, L. H., Geiping, J., Czaja, W., Goldblum, M., and Goldstein, T. Robbing the fed: Directly obtaining pri- vate data federated learning with modified models. In International Conference on Learning Representations, 2022. URL https:////// / /wwp//veppeeeeepe//o//e net forum? id EwzugoOFK...
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Fu, L., Li, Y., Huang, S., Chen, C., Zhang, C., and Zheng, Z. Parameter-orientee contrastive schema and multi-level knowledge distillation for heterogeneous federated learn- ing. Information Fusion, 103:10 2025. ISSN 1566- 2025.101222.
Acknowledgements
2535. doi:
The research is supported by the National Key R&D Pro- gram of China (203YFB2703700), the National Natural Science Foundation of China (62176269), the Natural Sci- ence Foundation of Guangdong (2033A1515012026),
Hu, J., Du, J., Wang, Z., Pang. X., Zhou, Y., Sun, P., and Ren, K. Does differential privacy really protect federated learning from gradient leakage attacks? IEEE Transac- tions on Mobile Computing, 2024.
References
H.-N. cross-client coordinator in federated learning framework for conquering heterogeneity. IEEE Transac- tions on Neural Networks and Learning Systems, 36(5): 8828 8842, 2025. doi: 10. N9.10999NN.S220244499778
Abadi, M., Chu, A.. Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L. Deep learn- ing with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Com- munications Security, CCS '16, pp. 308-318, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450341394. 1976749.2978318. URL https://des http////oo.or/...//// 1145/2976749.29788
Idelbayev, Y. and Carreira-Perpinán, M. A. Low-rank com- pression of neural nets: Learning the rank of each layer. In Proceedings of the IEEE/CVE Conference on Com- puter Vision and Pattern Recognition, pp. 8049-8059, 2020.
doi: 10.1145/
Jain, P., Thakkar, O. D., and Thakurta, A. Differentially private matrix completion revisited. In International Con- ference on Machine Learning, pp. 2215-2224. PMLR, 2018.
Bietti, A., Wei, C.-Y., Dudik, M., Langford, J., and Wu, S. Personalization improves privacy-accuracy tradeoffs in federated learning. In International Conference on Machine Learning, pp. 1945- 1962. PMLR, 2022.
Jain, P., Rush, J., Smith, A., Song, S., and Guha Thakurta, A. Differentially private model personalization. Advances in Neural Information Processing Systems, 34:29723- 29735, 2021.
Cai, J.-F,, Candès, E. J., and Shen, Z. A singular value thresholding algorithm for matrix completion. SIAM Journal on optimization, 10(4):1956-11)
2010.
Caldas, S., Duddu, S. M. K., Wu, P., Li, T., Konečný, J., McMahan, H. B., Smith, V., and Talwalkar, A. Leaf: A benchmark for federated settings. arXiv preprint arXiv: 1812.00011 2018
Jeon, J., Lee, K., Oh, S., Ok, el. Gradient inversion with generative image prior. Advances in neural information processing systems, 34:29898 24-29898-299088 2021.
Kilmer, M. E. and Martin, C. D. Factorization strategies for third-order tensors. Linear Algebra and its Applications, 435(3):641-66) -658, 2011.
Cheng, A., Wang, P., Zhang, X. S., and Cheng, J. Differen- tially private federated learning with local regularization

---

FedCEO: Flexible Differentially Private Federated Learning
Kim, M., Günlü, O., and Schaefer, R. F. Federated learning with local differential privacy: Trade-offss between pri- vacy, utility, and communication. In ICASSP 2021-2021
International Conference on Multimedia, pp. 6211-6219, 2022.
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2650-2654. IEEE, 2021.
Shen, Z., Ye, J., Kang, A., Hassani, H., and Shokri, R. Share your representation only: Guaranteed improvement of the privacy-utility tradeoff in federated learning. In The Eleventh International Conference on Learning Represen- tations, 2023, URL https: openrevieww net/ forum? idw..poVc............
Levy, D., Sun, Z., Amin, K., Kale, S., Kulesza, A., Mohri, M., and Suresh, A. T. Learning with user-level privacy. Advances in Neural Information Processing Systems, 34: 12466-12479, 1021
Truex, S., Liu, L., Chow, K.-H., Gursoy, M. E., and Wei, W. Ldp-fed: Federated learning with local differential privacy. In Proceedings of the Third ACM International Workshop on Edge Systems, Analytics and Networking, pp. 61-66, 2020.
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429-450, 2020.
Lu, C., Feng, J., Chen, Y., Liu, W., Lin, Z., and Yan, S. Ten- sor robust principal component analysis: Exact recovery of corrupted low-rank tensors via convex optimization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5249-5257, 2016.
Tsoy, N., Mihalkova, A., Todorova, T. N., and Konstantinov, N. Provable mutual benefits from federated learning in Privacy-sensitivg domains. In International Conference on Artificial Intelligence and Statistics, pp. 4798-4806. PMLR, 2024.
Lu, C., Feng, J., Chen, Y., Liu, W., Lin, Z., and Yan, S. Ten- sor robust principal component analysis with a new tensor nuclear norm. IEEE transactions on pattern analysis and machine intelligence, 42(4):925-933 2019.
Wei, K., Li, J., Ding, M., Ma, C.,, Su, H., Zhang, B., and federated learn- ing: Analysis and performance optimization. IEEE Trans- actions on Mobile Computing, 21(9):3388-344 3401, 2021.
Poor, H. V. User-level
Malekmohammadi, S., Yu, Y., and Cao, Y. Noise-aware al- gorithm for heterogeneous differentially private federated learning, arXiv preprint wrXiv:2406.03519 2024.
Yang. L.,, Huang, Z.-H., Hu, S., and Han, J. An iterative algorithm for third-order tensor multi-rank minimization. Computational optimization and applications, 63:169- 202, 2016
McMahan, B., Moore, E., Ramage, D., Hampson, S., and
y Arcas, B. A. Communication-effiienn learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1282. PMLR, 2017.
Yin, M., Gao, J., and Lin, Z. Laplacian regularized low-rank representation and its applications. IEEE transactions on pattern analysis and machine intelligence, 38(3):504- 517, 2015.
McMahan, H. B., Ramage, D., Talwar, K., and Zhang, L. Learning differentially private recurrent language models. In International Conference on Learning Reprexentations, www.:/ewwwwww.eeeeeeeeeeeeeeeeee
Yousefpour, A., Shilov, I., Sablayrolles, A., Testuggine, D., Prasad, K., Malek, M., Nguyen, J., Ghosh, S., Bharadwaj. A., Zhao, J., et al. Opacus: User-friendly differential privacy library in pytorch. In NeurIPS 202/ Workshop Privacy in Machine Learning, 2021.
2018. URL https: id=BJOhF1203
Naseri, M., Hayes, J., and Cristofaro, E. D. Local and central differential privacy for robustness and privacy in feder- ated learning. In 29th Annual Network and Distributed System Security Symposium, NDSS 2022, San Diego, California, USA, April 24-28, 2022. The Internet Soci- ety, 2022. URL https: www. nnss- www.:nynpssssssssssoooooooo um. org/ ndss wwwdddspppped wwwasss-pppeaaauaaaee waapeeerrrttrrrrrreee
Yuan, X.,, Ni, W., Ding, M., Wei, K., Li, J., and Poor, H. V. Amplitude-vary varying perturbation for balancing privacy and utility in federated learning. IEEE Transactions on Infor- mation Forensics and Security, 18: 18:1884-1897, 1897, 2023.
Zhang, X., Gu, H., Fan, L., Chen, K., and Yang. Q. No free lunch theorem for security and utility in federated learning. ACM Transactions on Intelligent Systems and Technology, 14(1):1-35, 2022.
Pang, J. and Cheung, G. Graph laplacian regularization for image denoising: Analysis in the continuous domain. IEEE Transactions on Image Processing, 26(4):1770- 1785, 2017.
Zhu, L., Liu, Z., and Han, S. Deep leakage from gradients. Advances in neural information processing systems, 32, 2019.
Ren, J. Zhang, Z., Hong, R., Xu, M., Zhang, H., Zhao, M., and Wang, M. Robust low-rank convolution network for image denoising. In Proceedings of the 30th ACM
10

---

FedCEO: Flexible Differentially Private Federated Learning
A. More on Preliminaries
A.1. Discrete Fourier Transform
For each vector v E IR, defines the discrete Fourier transform, denoted as DFT(-), as follows:
DFT(() F E C", (F.)ss = ((
(4)
Here, F.. Cuxn is the DFT matrix and w e where represents the base of the natural logarithm, and i denotes the imaginary unit. Furthermore, we denote the inverse discrete Fourier transform as IDFT(-).
A.2. T-product
For each third-order tensor U R Xng and
V
1 Xma, it defines the t-product, denoted as U V, follows:
U V == fold(bcire(U4) unfold(V)) R X
(5)
where the bcire(-) is tensor matricization operator known as block circulant matricization, denoted by
U(is))
U
U
beirc(L4)=
(ns)
(ma-1)
Furthermore, we define the tensor unfolding operator as follows:
(
v(2)
unfold(V))
E
fold(unfold())) = V "2x224
(in))
where (0)) represents the i-th frontal slice of tensor V. i.e., ())
A.3. Algorithm of tSVD
For each matrix Y Re of rank it defines the singular value decomposition operator, denoted SVD(-), follows:
SVD(Y) = UVV Σ diag (Hoi)==),
(6)
where U E X V E X are orthogonal and Σ E R x is diagonal matrix, whose main diagonal elements are the singular values of the matrix Y. Next, we introduce the algorithm of singular value decomposition for third-order tensors.
Algorithm 3 Tensor singular value decomposition (tSVD)
Input: W
X
Output: u
S Ra Xm2Xng, V R X2XXna
1: W = DFT(W,3)
2: for i to n3 do
3: [ S (1)
v =VD(WD
4: end for
5: U fold
T
T (ms)
5 fold
[5() [
=fold fold (Tr )lus))?
6: 4 IDFT
U, (,,3 5
IDFT 3). IDFT(D.3)

---

FedCEO: Flexible Differentially Private Federated Learning
A.4. Algorithm of T-SSD
For each matrix Y R TruncatedVVD()). as follows:
x of rank r, it defines the truncated singular value decomposition operator, denoted as
TruncatedSVD(Y, (7) (Z) D, (2)) diag ((max (o; T, ())
(7)
where is the truncated threshold. Next, we introduce the algorithm of truncated singular value decomposition for third-order tensors.
Algorithm 4 Truncated tensor singular value decomposition (T-SVD)
Input: W
R X2gxng
Output: 4
1 D 1i
Xnzxnig
Xung
1: W = (W..
2: for
= to mi do
3: [T D
V TruncatedSVDDDWW .7)
4: end for
T (1) .... ,
[ D fold
fold (Dr.,... "...,,, )
5: U fold
6: 4 IDFT (. D IDFT IDFT (D3)) IDFT(V.3)
B. Proofs of Main Theory
B.1. Proof Theorem 3.1 (Interpretability of our optimization objective)
Proof. Based on Parseval's s theorem and the definition the tensor nuclear norm (Definition 2.2), we have
W - Wrll? ||
K
DFT(W- - WN)
||x
K
(8)
1x
where W represents the i-th frontal slice of the tensor obtained by applying the DFT along the third dimension on W. Therefore, know
11)) K
min - Wvr +| + W
min w
[[] 1
(9)
i=1
By Lemma B. we have
TruncatedSVD W. 1 arg min N:
W
W/ WWI: +
W 11
(10)
w
Now, let define W arg minw --ll-Wwlll + +| Then, based Eq. (9), Eq. (10) and algorithm A.4, we have
W = IDFT fold [[IuncatedSVDDWWW _
TruncatedSVD (K) 1
2T
N 2T
(11)
-I-SVD(WN../) )
12

---

FedCEO: Flexible Differentially Private Federated Learning
Lemma B.1 (SVT Lemma (Cai et al., 2010)). For each 0 and Y RRxx,,, the truncated singular value decomposition operator TruncatedSVD(() obeys
_ 2T
TruncatedSVDD(Y
arg min {-|| - YI + +|
Remark B.2. Due to the inherent correlation within the semantic space of clients, if we perform the Fourier transform on parameter tensor along the third dimension, the singular values of its first slice will be much greater than the later (see Fig. 5). Formally, express the assumption as follows,
> or (W
where o-(() the r-th highest singular value matrix and i 2.3...... K.
17.5
12
Singular value
10-
12.5
2.5
9.0
0.0
10
15
20
35
singular value index
singular value index
(a) EMNIST
(b) CIFAR-10
Figure 5. The singular value curves in spectral space for the original noise parameter tensors on two real datasets. w/ to 42 represent components from low high frequency.
B.2. Proof of Proposition 3.2
Next, we have the following proof,
Proof. First, consider T-tSVD(W, am) in Fourier space and have
D D max ( (j.s) om: 0}-
(12)
Since we set to be larger than the highest singular value of W ( for i = 2,3,.... K, we have
D (j.)) =
(13)
for all i .... K and
D (..) max (0 (j.j) - 0m, 0}.
(14)
has least one value be not zero by Remark B.2.
13

---

FedCEO: Flexible Differentially Private Federated Learning
Thus, when we perform the inverse Fourier transform, we will ger
$ - D v()
(T-SVDD(W,mm))
K
(1) 1 D a
(15)
U
D (1] ((
TruncatedSVD W mm)
K
From the other side, we know that
Σ
2x (k W(k)
w KW.
(16)
Therefore we obtain that
[T-tSVD(W, (. (m)]
TruncatedSVD(KW am) = TrmncatedVDD(W Fm
(17)
K
K
Moreover, we appropriately set = maxagign or }.
On the one hand, normally we know that K >> n- So, based on Eq. (17), we have
(T-SVD(W,,m.) am) TruncatedSVD(W) Im
K
(18)
W.
On the other hand, by Remark B.2, Eq. (15) and Eq. (16), we have
(T-SSDDWW.o,l om)]
1
TruneatedSVI(W mm)
K
(19)
2 w (1)
1
W.
K
B.3. Proof of Theorem 4.3 (Utility Analysis of Our FedCEO)
Let (t) denote the global model at i-th round in Algorithm 2. Let wg(t 1) denote the inexact solution (defined in B.3) of local optimization without DP by min, fu (w; w' (t) at & + 1 round, where w' (t() is the initial solution. Let M denote the DP mechanism we used. Let denote our low-rank proximal optimization at the server (i.e. adaptive T-SVD). Then we have db (t 1) F (IM (we + 1)) and define w' (t + 1) Es, [tta(t + 1)]. Moreover, we make the following definitions and assumptions.
Definition B.3 (7/+1 -niexxct solution). For a function fk (w) and 1+1 solution of min fa (w; wo) at (t + 1) round if
7 E [0, 1]. we define wx(t + 1) as a 4-1 -inexact
[V (wr (1) S o
V (wo))l
where we the initial solution in optimization. % represents the local computational load of the k-th client at (t 1)-th round.
Definition B.4 (B-local dissimilarity). The local empirical risk fa are B-locally dissimilar at w if
|²] < ["²²
Ex
E. (1 [
Moreover, we define B(w)=
for [f(ww] * 0 and then B(w) S B.
14

---

FedCEO: Flexible Differentially Private Federated Learning
Assumption B.5 (Ly-continuous). f1. .JN are all Ly-continuous:
Ve, w, ( fu (w))l S L1llu wil-
Assumption B.6 (L2-smooth). fi,".. fN are all s--smohhh:
Ve, w, fi f((wl)] - L21/ w||.
Assumption B.7 (u-strongly convex). f1.... fN all fo-strongly convex:
Va, w. (V/((v) (i))))) (v w) > م wll
We can also study the utility guarantee of our FedCEO algorithm under more general non-convex objectives if we introduce the proximal term to f& like FedProx (Li et al., 2020).
Proof. At (t + 1)-th round, considering local optimization without DP: Let us define wf + 1) arg min fi (w: u/ (t)) (i.e. exact optimal solution). So, we know
fc [wg[t 1)] O.
(20)
Then, based Assumption B.7, we have
(Vfx(v) (w) (v w) ulll w|l
(21)
Set wt + 1), = wt(t= 1), based on Eq. (20) and Definition B.3, we have
1+1
|| 1) wr (t 1)ll S
Vi (wr (t 1))II S
II (w (t))l
(22)
Similarly, we know that
V (w (+)] _ μ
+ w())ll
(23)
Combining Eq. (22) and Eq. (23), based on the triangle inequality, we have
1 + 7 1+1
[1xx((++)] + - -wo)))(()))) (1)( < [1 + 1) wr (1 + 1)ll + (1 1) - w/ (1)l S
I (w (1)]I-
(24)
Next, let define w(t 1) Ex [wv (t 1)] (an auxiliary variable). Based on the Jensen's inequality and Definition B.4, we have
+ 7
[w + 1) - w'(t)) s Eg [| + 1) w'(t)ll -
Es
[[[ fu (t)))l
11
+ kk
t+1
Es
V fx (1)II
(25)
(1 + 7k
1+1
B
B (wd(())
According to the local updates from gradient descent, we have
w + 1) = - V/ (w)(
(26)
Based on the Taylor expansion and Assumption B.6, we have
Lg
f (1el1 + 1)) S (w/ (0)) + (V (1). T (1. (1+1) - ro'(()))) -o( (1)) +
|| (1 (1+1) w (1)1
2
< (w' (t)) - - (w (t))11= L1 + 7)
IV (f))1²
(27)
24²2
(w' (f)) (1 Lg 7))
|I (wl (t))
2412
15

---

FedCEO: Flexible Differentially Private Federated Learning
where 7 m (7k 3.
In practice, we randomly select K clients (i.e. St) to participate in each round of FL training, rather than involving all clients. Therefore, we consider the following error analysis:
= Es. U/ (w (1 + 1)) f (10 (1 + 1))],
(28)
where wi(t + 1) is the actual version of wtt + 1), taking into account the randomness of client selection. Based on Assumption B.5, is easy to see that f also Tn-continuous. Therefore, we have
f (w(t 1)) - f (mit 1)) Ln ||olt + 1) -i((t 1)ll Similarly, based on the L2- smoothness of f, we have
(29)
1 < Vf (w' (f) + max (I + w (t)ll 1/w(t//(ttt + 1) w'(()))
(30)
1 Vf (w (f))l + L2 (t + w/ (()ll + 1| + - w()(t))))
Then, based Eqs. (28), (29) and (30), we have
c S Es, [h w (t + 1)) - (w (t + 1)ll
S Es, [f (w' (t)]l + L2 (1 +) w/ (t)ll + | + 1) w (t)ll) 110 + 1) w(t + 1)ll
(31)
( (w (()) Le [2 ----1 w()))l) Es, [|| + +1) m(t + 1)ll
+ L2ESs, (I) 1) w/ (()ll 11 1) at( + 1)[]-
Moreover, we know that
Es, [1 1) w/ (1)] [| - w(l
1)ll I Es, K[1lo(t + 1) の (f 1)ll + || + 1) w (t)l) [| + 1) w(t + 1)ll
(32)
||:(t + w/ (f)] Es, [1] + 1) (t 1)l Es, [luott + +1 - tt 1)
Overall, based on Eq. (31) and Eq. (32), we have
e, S ( (w/ (t)) 2L || w(t + (1t++1--10 1) (t)ll Es. [|] + 1) w0t + 1)
(33)
LgEs, [ + 1) - (t + 1)]²
Similar to (Li et al., 2020), we provide the bounded variance of wx (t + 1) as follows,
Es. ||u( + 1) - wt + +>)1²]
- Ex
1
w (t + 1) - w(t + 1)]l
K
<
2
Ex
[| (t 1) - w/ [+)
K
2 (1 + 7)
(34)
Ex [ (w (0)I
<
K
412
2 (1 + 7) B²
| (w (
<
K
112
Based on Jensen's 's inequality and Eq. (34), we have
Es. [|| + 1) w[t 1)ll -
\
Es. ||u[t + 1) w(t + 1)1l
(35)
2 (1 B)
IV (w (t)]l-
K #
16

---

FedCEO: Flexible Differentially Private Federated Learning
Combining Eq. (33) with Eqs. (25), (34), and (35), we have
Vf (wd (8)) 2L2 BB V2 (uB(1 +7) + 21 + B²)
2 (1+
IV (w' (t)) + 21 (1 B2
|V (w' (t)II
IV (w' (t))
e
V
K
K 4²2
2L + 7)
IV (w' (t))
vR
K₁²
(36)
Based on Eq. (27) and Eq. (36), we have
Es. If (w (t- 1))] S f (w/ (t))
B2 (1 + +7
(2 2L2(1 7))
2I2(1 + +2)²
Vs w (1)
(0)
(37)
24
VKμ²
p
Then, we set m =1) (1++))
2112 g² kR²
2/²²
vKμ²
+
and we have
Es. If (t+ 1))1 S (w (t) m VF w (t) (t)
(38)
Considering DP mechanism and our low-rank proximal optimization:
Based on the L--ootiinityy of f. we have
f + 1)) - (w 1)) + 4, || + 1) wit + 1)ll
(39)
Thus,
Ess [s (w/ (t. + 1})] - Es, [S (wilt + 1))] < L,,,,,
[|| (t 1) w(t + 1)ll]
= L1
Σ
11|20x (t + 1) - wx (t 1)ll]
K
(40)
KES.
V2L
Iiv i -wll
K
where VV= fold (le) (t), "xx((()))))))))) and w = fold (lw1(t), ... wx(1)()) E Rdxxix
Next, we know that
iw- w// & 1//w- - -ww| + +IWN- WlI]]
(41)
where WN = fold (1)) ... w/x (0))) E Raxhxx
Based on Parseval's theorem and Algorithm A.4, we have
|w- wy-wa -W-W
W
WN
K
W
N
diag ((max (,, (1) ((())
diag (1051 (1)
U
(42)
diag (max (a,, (T,--T,00 0]] (1) diag (a,))ニ=1) (0)
S VTTO. (as T T= T)
where (0 are the singular values of WN and the truncated threshold of Algorithm A.4.
17

---

FedCEO: Flexible Differentially Private Federated Learning
Based our DP with the Gaussian mechanism, have
I|N- - W - E/y [W VdVhCo,
(43)
where N /K) and I IRdxxx
Choice of 7 When the number of clients K is large, and the variance of local Gaussian noise N is small, we set smaller truncation threshold with the choice To/ K. This allows for the retention of more accurate semantic information from individual clients.
Combining Eq. (40) with Eqs. (41), (42), (42), we have
V2T, Es, [f (w (t + 1))] Es, If +1]++1))]
(VrTo+VaVicy
(44)
K
Overall, based on Eq. (38) and Eq. (44), we have
f (wl (t (t+111 1)) S / (wl (1)) m Vs W (1) 1|
v2h
(VrTo vavicoo.
(45)
K
Based on the p-strongly convexity of S, we have
f(w(t)) 2 (ww((t) + (f) w wt)) + =2 [[(((t) w (t)11
(46)
Now, minimize the inequity with respect to wttt and we have
f (w") ž (w'(())
(V)(w((t))
(47)
2p
where w" is the convergent global model of wt)..
Based on Eq. (45) and Eq. (47), we have
fww (t 1)) s f (wl (t)) 2pm (f (w' (t)) f (w")) + V2hn
(ViTa+ VVatho))
(48)
and thus
v2Li (ViTo vavhco)
(w (1+ 1 (w") (1 2pm) (ʃ (wo' (1)) 1 (ww))
(49)
K
Taking from 0 to 1
in Eq.
(49).
(w*) < (1 2/m)) (ʃ (w (0)) f (w")) + v2h1
f [ (1)) - (1))-
(Vito VavhCoo
(50)
K
(w' (2)) - f (w*) s (1 2/m) (f (w (1)) - f (w²)) + v2L1 (ViTo +VaVhCo)
(51)
f (w' (T)) - f (w*) s (1 - 2pm) (f (w (T 1)) - -f (w*) + v2h1 (ViTO+ vavhCo).
(52)
K
and subsequently substituting each resulting expression one by one, obtain
VLL K
(Vito+VaTTcc 1 1 (1 2pm))²
I (wl (T))
- (w") (1- 2umm)² ( (w (1)) (ww)))
(53)
2pm
When selecting a sufficiently large T and satisfying m 1/H, we have
V2L1
(Vrto + vavhCo) 1
cu (w/) (w+)
(54)
K
2pm
18

---

FedCEO: Flexible Differentially Private Federated Learning
where w' is the convergent global model of w')).
Overall, with the choice of and 7' specified in the theorem, have
cu = 0 vT v vd
(55)
K
where r is the rank of the parameter tensor after our processing and d is the dimension of input data.
Especially, When we choose an appropriate truncation threshold (regularization coefficient) such that the resulting parameter tensor is low-rank, we have
C = ( Vd
(56)
K
B.4. Proof of Theorem 4.5 (Privacy Analysis of Our FedCEO)
Proof. Let : D -> R. denote algorithm that satisfies user-level (c, 6)-DP. Based on its definition, we know for any two adjacent datasets D, D' E 'D that differ by an individual user's data and all outputs C R. holds that
Pr(M(D) E Ss S et Pr(M E S)+ 6
(57)
By theorem 3.1 and Algorithm A.4, know low-rank proximal optimization is equivalent the truncated tSVD algorithm, so it is deterministic function, denoted F : R - R'.
Fix any pair of neighboring datasets D, D' with | - D'll $ 1, and fix any output S S R'. Let Z = fz R[F(2) E S). we have
Pr(F(M(D)) E S) = Pr(M(() E 3
- exp(c) Pr[M(D') E 2
(58)
= Pe PF(F(M(D)) E 56
C. Experiments Setup and More Results
C.1. Local Model and Hyperparameters
Models. In the paper, we employ two-layer MLP for the EMNIST dataset and a LeNet-5 for the CIFAR-10 dataset. The specific network architectures are as follows.
MIP:
(1) (input layer): Linear(in featurresd, out featurese5, bias=False)
(2) (dropout layer):
Dropout(p=0.5, inplace=False)
(3) (activation layer): ReLU0
(4) (hidden layer): (inear(in.faaturesst out wu..faatres--mmmllsees bias=False)
(5) (activation layer): Softmax(dim=1)
LeNet-5:
(1) (conv I): Conv2d(3, 32, kerneL.siee=5,, 5), stride=(1, 1))
(2) (pool): MaxPooo2ddkermelsizzzz stride=2, padding=0, dilation=1, mooee=FeF)))))
(3) (conv2): Conv2d(32, 64, kernel size= 5), stride=(1, 1))
(4) (pool): MaxPool2d(kermel size=2, stride=2, padding=0, dilation=1, wet.eeeeleeeeeeeeeeee mode=False)
(5) (fel): Linear(in features=1600, out features=512, bias=True)
19

---

FedCEO: Flexible Differentially Private Federated Learning
(6) (activation layer): ReLUO
(7) (fc2): Linear(in..catuues-55] 1ea out features=512, bias=True)
(8) (activation layer): ReLUO
(9) (fc3): Linear(in features=512, out features=10, bias=True)
Hyperparameetess For each dataset, we uniformly set the global communication rounds T to 300, the total number of clients N to 100, and the sampled client number K 10, resulting a sampling rate p 0.1. Local training employs stochastic gradient descent (SGD) with 30 epochs (E), learning rate 1) of 0.1, and batch size B of 64.
Other personalized hyperparameters: initial coefficient X, common ratio v, and interval / (searched within a grid range), whose values can be qualitatively guided by practical application scenarios, and their impact on model utility is robust. For X, the search range [55, 100, 5] when og = 1.0; [0.1, 10, log] when og = 1.5; and [0.01, 1, logl when og = 2.0. For scenarios with stricter privacy requirements (larger noise), we need to choose a smaller À to achieve smoother semantic space. For 0, the search range is [1.01, 1.10, 0.01]. For I, the search range is [10, 15, 20, 25, 30J. Their specific values are listed in Table
Table 2. Our focused hyperparameters for three privacy settings EMNIST and CIFAR-IO. Hyperparameter
Dataset
Model
Setting
"g 1.0
70, / 1.08,
EMNIST
MLP-2-Layers
0g 1.5
0.5, 1.04, 20
0g 2.0
À 0.03, d = 1.06, / = 20
gg 1.0
À 85, v 1.03, 7 15
CIFAR-10
LeNet-5
Jg 1.5
À = 10, 1.07, = 10
gg 2.0
A = 0.6, O 1.04, = 10
Partial parameter analysis results follows:
Table Testing accuracy on CIFAR 10 differrnt intervals / under various privacy settings three common dg.
Dataset
Model
Setting
I = 10
I= 15
I 20
I = 25
I = 30
gg = 1.0
53.62%
54.16% %
53.01%
52.81%
52.80%
CIFAR-10 10
LetNet-5
0g 1.5
50.00%
49.71% %
48.90%
47.41%
47.98%
0g = 2.0
45.35%
44.37%
44.81%
43.75%
43.20%
C.2. More Empirical Results
C.2.1. MODEL TRAINING EFFICIENCY
To validate the efficiency of server-side low-rank proximal optimization, we conduct comparative analysis of the runtime between our FedCEO and other methods in Table 4. We observe that the training efficiency of our method significantly surpasses PPSGD and CENTAUR, approaching the efficiency of UDP-FedAvg without any utility improvement.
Table Runtime for our FedCEO and other methods on EMNIST and CIFAR-IO (One NVIDIA GeForce RTX 4090).
Time UDP-FedAvg
PPSGD
CENTAUR
FedCEO
EMNIST
5.436
24
24
5,445
CIFAR-10
3.876
> 24
> 24
3.908
C.2.2. UTILITY FOR HETEROGENEOUS FL SETTINGS
To validate the effectiveness our model in heterogeneous federated learning (Li et al., 2020; Huang et al., 2025; Fu et al., 2025), we conduct experiments using an MI.P the local model on CIFAR-10 in Table 5, We report the testing accuracy for
20

---

FedCEO: Flexible Differentially Private Federated Learning
both iid and non-iid scenarios. It can be observed that FedCEOO maintains state-of-the-art performance. Additionally, in non-iid scenarios, typically choose larger À to reduce the global semantic smoothness, preserving personalized local information.
Table 5. Testing accuracy (%) CIFAR-I0 under iid setting and non-iid setting.
Dataset
Heterogeneity
Setting
UDP-FedAvg
PPSGD
CENTAUR
FedCEO
0g 1.0
40.21%
41.34%
42.17%
42.76%
9g = 1.5
35.79%
37.28%
38.21%
39.16%
0g 2.0
31.62%
33.51%
33.86%
35.93%
CIFAR-10
gg 1.0
33.09%
34.91%
34.56%
36.10%
non-iid
0g = 1.5
28.92%
31.40%
30.87%
32.39%
Og 2.0
26.54%
28.01 %
28.11%
29.13%
C.2.3. UTILITY FOR OTHER LOCAL ARCHITECTURE DATASET
To further validate the applicability of our framework, we conduct experiments using more complex local architectures and other types of datasets, shown in Table 6. Specifically, the text dataset Sentiment140 (Sent140) (Caldas et al., 2018). It can be observed that our FedCEO still maintains SOTA performance.
AlexNet as the local model CIFAR-10 and LSTM on
Table Testing accuracy (%) CIFAR-I0 and Sent140 under 5 10 and various privacy settings with three common dg.
Dataset
Model
Setting
UDP-FedAvg
PPSGD
CENTAUR
FedCEO
Ty 1.0
50.67%
56.58%
58.44%
60.73%
CIFAR-10 AlexNet
g = 1.5
41.11%
51.07%
50.20%
55.49 %
gg = 2.0
33.38%
39.93%
43.95%
49.06 %
Jg 1.0
60.31%
61.04%
63.33%
65.70%
Sent140
LSTM
g = 1.5
57.62%
57.87%
59.05%
60.22%
9g = 2.0
50.94%
55.12%
54.88%
56.65% %
C.2.4. MORE DETAILS FOR PRIVACY EXPERIMENTS
In the three federated learning frameworks, we consider a semi-honest adversary at the server, engaging in a gradient inversion attack on the model (gradient) uploaded by a specific client in given round. This adversarial action is based on the DLG algorithm, and the detailed attack procedure is presented Figure to
'non-iid means the data among clients are not independent and identically distributed (iid).
21

---

FedCEO: Flexible Differentially Private Federated Learning
ieer 30
iter= 60
iter 90
iter= 120
iter=150
iter=180
iter= 210
iter - 240
iter 270
iter - 300
FodAvg
UDD-FedAAg
Icer=30
Iter-60
Iter=90
iter= 120
Iter-150
Iter-180
Iter-210
Iter-240
Iter-270
Iter-300
FedCEO
Figure 6. Privacy attack process on three FI. frameworks based on DL.G. We set the attack target as the image with the index 25 in CIFAR-10. For UDP-FedAvg and FedCEO, configure the local model L.Net with noise parameter o 1.0.
iter=
iter=180
iter=210
iter=300
FedAvg
iter=30
iter=
iter= 90
iter= 120
iter=150
1te
iter=210
iter= 240
iter=
iter=300
UDP-FedAvg
Iter=30
Iter=60
Iter=90
Iter=120
Iter=150
Iter=180
iter=210
Iter=240
Iter=300 Iter=270
FedCEO
Figure Privacy attack process on three frameworks based on DL.G. We set the attack target as image with the index 50 CIFAR-10. For UDP-FedAvg and our FedCEO, we configure the local model as LeNet with noise parameter of o 1.5.
Iter=60
Iter=120
iter=180
iter=240
iter=300
iter=360
iter=420
Iter=4B0
Iter=540
iter=600
FedAvg
Iter=60
Iter=120
iter=180
iter=240
iter=300
Iter=360
Iter=420
Iter=480
Iter=540
Iter=600
UDP-FedAvg
iter=60
iter= 120
iter=180
iter=240
iter-
iter-
iter=480
iter- iter 600
FodCEO
Figure 8. Privacy attack process on three FL frameworks based on DLG. We set the attack target as the image with the index 100 in CIFAR-10. For UDP-FedAvg our FedCEO, we configure the local model as LeNet noise parameter a =
22


Reviews:
[{"review_id": "hTQOthJG0d", "reviewer": "Reviewer_KHNv", "summary": "The paper introduces a new method for FL training with DP guarantees. The authors argue that the method improves on existing work in terms of the utility-privacy trade-off, supporting their algorithm with a theoretical analysis and experiments. The method is based on a tensor low-rank proximal optimization of the (stacked) local parameters at the server and intuition is provided about why this truncates high-frequency components in spectral space.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback and for taking the time to provide thorough reviews! Below are our point-by-point responses to your comments:\n\n> **Claims And Evidence (Concern 1)**  \n\nWe appreciate the reviewer's suggestion.  \n- The product form was chosen because it directly reflects the **joint tightness of the utility-privacy boundary**. This form highlights the intrinsic trade-off: improving one objective often requires relaxing the other.  \n- Secondly, we maintain the **same** joint objective formulation ($\\epsilon_u \\cdot \\epsilon_p$) as the prior SOTA method CENTAUR [1], enabling direct theoretical bound comparisons in our paper. Moreover, as shown in Theorems 4.3, 4.5, and Corollary 4.6, the product form eliminates the influence of the stochastic factor K.  \n- Compared to weighted averaging [2] or Pareto optimization [3], the product form is more **concise and interpretable**. *We will add relevant discussions and citations in the revised version*.  \n  - **Product vs. Weighted Average**: Weighted averaging (e.g., $\\alpha \\epsilon_u + (1-\\alpha)\\epsilon_{p}$) requires manual weight selection ($\\alpha$), which may introduce subjectivity. The product form avoids weights and directly captures the coupling between objectives, making it more suitable for theoretical analysis. Additionally, weighted averaging is more susceptible to scale differences.  \n  - **Product vs. Pareto Optimization**: Pareto requires simultaneous optimization of both objectives, but in practice, privacy and utility are often conflicting. The product form allows controlled compromise on one objective to achieve significant gains in the other, better aligning with practical needs.  \n\n[1] \"Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learning\" (ICLR2023)  \n\n[2] \"No free lunch theorem for security and utility in federated learning\" (TIST2022) \n\n[3] \"Optimizing privacy, utility, and efficiency in a constrained multi-objective federated learning framework\" (TIST2024)  \n\n> **Claims And Evidence (Concern 2)**  \n\nFor all compared methods in experiments, we adopted the **same DP mechanism** (user-level DP) and **unified implementation** (based on Opacus). Thus, their privacy guarantees are identical (i.e., same $\\epsilon_{p}$). In the revision, we will supplement the quantitative relationship between $\\sigma_g$ and $\\epsilon_{p}$ (e.g., adding an $\\epsilon_{p}$ column in Table 1) based on Theorem 4.5, demonstrating our method's superior utility under identical $(\\epsilon_{p}, \\delta)$.  \n\n> **Claims And Evidence (Concern 3)**  \n\nLemma 4.4 provides the privacy guarantee for the baseline algorithm UDP-FedAvg, while Theorem 4.5 proves that FedCEO's low-rank optimization (a deterministic operation without noise-dependent priors) preserves privacy (post-processing immunity). Thus, their bounds align. We will explicitly emphasize this in the revision.  \n\n> **Experimental Designs Or Analyses** \n \nThank you for the suggestion. We will add standard deviations in the revision (e.g., FedCEO 78.05%±0.2 vs. CENTAUR 77.26%±0.3 for EMNIST at $\\sigma_g=1.0$), confirming statistically significant improvements.  \n| Dataset   | Model         | Setting ($\\sigma_g$) | UDP-FedAvg       | PPSGD           | CENTAUR         | FedCEO  | FedCEO ($\\vartheta>1$)     |\n|----------|---------------|---------------|------------------|-----------------|-----------------|-----------------|------------------|\n|  |  | 1.0| 76.59% ± 0.8% | 77.01% ± 0.4% | 77.26% ± 0.3% | 77.14% ± 0.5%   | **78.05% ± 0.2%** |\n| EMNIST | MLP-2-Layers | 1.5  | 69.91% ± 0.8% | 70.78% ± 0.6% | 71.86% ± 0.2%  | 71.56% ± 1.0%   | **72.44% ± 0.8%** |\n|  | | 2.0  | 60.32% ± 1.2% | 61.51% ± 1.1% | 62.12% ± 0.9% | 63.38% ± 0.7%   | **64.20% ± 0.6%** |\n|   |    |1.0 | 43.87% ± 2.1%  | 49.24% ± 0.9% | 50.14% ± 1.4% | 50.09% ± 0.5% | **54.16% ± 0.2%** |\n|    CIFAR-10       |     LeNet-5      | 1.5 | 34.34% ± 0.7%  | 47.56% ± 1.6% | 46.90% ± 0.9% | 48.89% ± 0.6% | **50.00% ± 0.5%** |\n|       |       | 2.0 | 26.88% ± 2.8%  | 34.61% ± 0.7% | 36.70% ± 2.4% | 37.39% ± 1.1% | **45.35% ± 0.9%** |\n\n> **Others**  \n\nWe will correct the usage of `return` in Algorithm 2 (e.g., replacing it with `yield` or annotations) and move key assumptions to the main text in future versions.", "claims_and_evidence": "The paper presents DP upper bounds on the epsilon, which are a usual in this field to evaluate the privacy guarantees of an algorithm. Experiments on EMNIST and CIFAR do present improvements in terms of accuracy, for a fixed noise level. \n\nSome concerns are as follows:\n\n- The theoretical improvement of the privacy-utility trade-off is stated for a joint objective which is a product of the utility improvement and the privacy guarantee. However, the paper does not discuss why this is a good way to merge the objectives, nor does it cite another work where such a discussion can be found. Why and when may the suggested product be preferable to alternative ways to mix the objectives, e.g. weighted average, or to a Pareto treatment, under which both objectives need to be improved for a method to be considered better?\n\n- In the experiments, privacy is measured via the noise level $\\sigma$ that is used. In general, the same $\\sigma$ may lead to different levels of privacy for different learning algorithms. Why not use the guarantees provided by Theorems 4.4 and 4.5 (and similar) instead?\n\n- Related to the last point, how do the guarantees in Lemma 4.4 and Theorem 4.5 compare?", "methods_and_evaluation_criteria": "The way of evaluating the proposed methods: by theoretical upper bounds and experiments with varying noise, are to my awareness standard in this literature.", "theoretical_claims": "I have not checked the proofs, but the assumptions and type of bounds are sensible.", "experimental_designs_or_analyses": "The structure of the experiments make sense. However, on some of the experiments in Table 1, the gains compared to prior methods are relatively small. Estimates of standard deviation (under different random seeds) will be helpful here, in order to evaluate the gains from the proposed methods better.", "relation_to_broader_scientific_literature": "The paper is well-positioned with respect to prior work, as it claims a clear contribution and discusses related work in detail.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "NA", "other_comments_or_suggestions": "- Algorithm 2 pseudo code contains return commands. The current way I read this is that the algorithm will terminate after this, which is not the intended interpretation. I suggest the authors double-check that.", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JILkDRDvUx", "reviewer": "Reviewer_RVpv", "summary": "The paper introduces FedCEO, a federated learning framework that aims to balance model utility and differential privacy by applying tensor low-rank proximal optimization (via T-tSVD) on noisy client parameters. The idea of smoothing the global semantic space is interesting, though its novelty is not entirely clear compared to existing spectral methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your constructive feedback and the opportunity to improve our paper. Below are our point-by-point responses to your comments:\n\n> **W1**\n\n**Figure 4 in the paper** visualizes the robustness of our method to different noise levels on CIFAR-10. Compared to other methods, our FedCEO (orange curve) exhibits less utility degradation as noise (privacy) intensity increases, demonstrating stronger utility-privacy trade-off and robustness to privacy noise (as reflected by the **minimal slope of the orange curve**, second only to non-private FedAvg (blue curve)). In the revised version, we plan to include visualizations on additional datasets to further strengthen this conclusion.\n\n> **W2**\n\nIn addition to EMNIST and CIFAR-10 used in the main text, we have also evaluated FedCEO on **text data (Sent140)** with LSTM (see **Appendix C2.3 Table 6**), where our method consistently demonstrates superior performance.\n\n> **W3 & Other Comments/Suggestions**\n\nFedCEO is a novel federated learning parameter update framework that is **architecture-agnostic** (operating on parameter tensors) and compatible with deep architectures (e.g., AlexNet, see Table 6). Its key advantage lies in efficiency—it avoids the high computational costs of deep personalized methods like PPSGD and CENTAUR (see efficiency experiments in Appendix C2.1 Table 4).  \n\nFurthermore, we have integrated the **deep tensor method CoSTCo [1]** (*modified for low-rank approximation task*) into our framework. Experimental results show that our approach (via T-tSVD) outperforms CoSTCo on CIFAR-10, likely because CoSTCo is more suited for sparse tensors and less compatible with noisy parameter tensors. Future work will explore deep tensor methods better aligned with FedCEO.  \n| Dataset   | Model         | Setting ($\\sigma_g$)  | FedCEO w/ CoSTCo  | FedCEO w/ T-tSVD |\n|----------|---------------|---------------|------------------|-----------------|\n|   |    |1.0  | 48.31% ± 0.9% | **54.16% ± 0.2%** |\n|    CIFAR-10       |     LeNet-5      | 1.5  | 44.70% ± 1.0% | **50.00% ± 0.5%** |\n|       |       | 2.0 |  32.25% ± 0.3% | **45.35% ± 0.9%** |\n\n[1] CoSTCo: A Neural Tensor Completion Model for Sparse Tensors (KDD 2019).\n\n> **Experimental Designs/Analyses**\n\nThe initialization coefficient λ and geometric ratio ϑ are designed to **adaptively adjust the truncation threshold** $\\frac{1}{2\\lambda} \\vartheta^{\\frac{t}{I}}$ based on noise levels (Sec. 3.2). For stronger privacy guarantees (larger σg), smaller λ enhances smoothness (see Table 2), while ϑ > 1 accounts for accumulating DP noise during training. Detailed parameter selection guidelines are provided in Appendix C.1.  \n\nBeyond the robustness analysis of parameter I in Table 3, we have added the following analysis for ϑ:  \n\n| Dataset   | Model     | σ_g  | ϑ=1.00 | ϑ=1.01 | ϑ=1.02 | ϑ=1.03 | ϑ=1.04 | ϑ=1.05 | ϑ=1.06 | ϑ=1.07 | ϑ=1.08 | ϑ=1.09 | ϑ=1.10 |\n|-----------|-----------|------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|\n|   |                 | 1.0  | 50.09% | 53.62% | 53.85% | **54.16%** | 53.92% | 53.90% | 54.09% | 53.20% | 53.76% | 52.47% | 52.02% |\n| CIFAR-10  | LeNet-5    | 1.5  | 48.89% | 49.33% | 49.85% | 48.71% | 49.30% | 49.10% | 48.92% | **50.00%** | 49.62% | 48.94% | 48.28% |\n|           |           | 2.0  | 37.39% | 40.32% | 43.19% | 44.87% | **45.35%** | 44.50% | 44.81% | 41.32% | 43.90% | 39.75% | 40.27% |\n\nIt can be observed that when ϑ > 1, the model performance shows significant improvement compared to ϑ = 1, while demonstrating **strong robustness to this parameter when ϑ > 1**. We will also include a **visual analysis of mixed λ-ϑ robustness** in the revised manuscript.\n\n> **Relation to Broader Literature**\n\nFedCEO introduces a **client-collaborative tensor low-rank optimization framework** for global model updates in DPFL, explicitly leveraging inter-client semantic complementarity in spectral space. Unlike prior spectral methods (e.g., CENTAUR [Shen et al., 2023] and [Jain et al., 2021]) that apply singular value decomposition (SVD) independently to client matrices, FedCEO stacks client parameters into a **higher-order tensor** and performs truncated tensor-SVD (T-tSVD). This enables adaptive truncation of high-frequency components across clients while preserving low-rank structures, thereby improving global model utility through inter-client information fusion.", "claims_and_evidence": "The authors claim an improved utility-privacy trade-off (O(√d)) and effective adaptation to different privacy settings.", "methods_and_evaluation_criteria": "The approach leverages tensor low-rank approximation to mitigate the impact of differential privacy noise. This method is sound.", "theoretical_claims": "The paper includes comprehensive proofs for its main theoretical claims.", "experimental_designs_or_analyses": "The experimental section demonstrates that FedCEO can achieve competitive performance compared to existing methods. That said, the hyperparameter choices (such as λ and ϑ) are not thoroughly justified, leaving some questions about the robustness of the method in practical scenarios.", "relation_to_broader_scientific_literature": "The discussion does not fully differentiate FedCEO from related methods, leaving some ambiguity regarding its incremental contribution.", "essential_references_not_discussed": "There is room for a more comprehensive discussion on scalability in federated learning under differential privacy.", "other_strengths_and_weaknesses": "Strengths:\n1.The paper addresses a significant issue by proposing a way to mitigate DP noise effects using tensor methods.\n2.The combination of theoretical analysis and experimental results is a strong point.\n\nWeaknesses:\n1.The paper lacks intuitive visualizations that could better demonstrate its noise robustness.\n2. Experimental evaluation is limited to simple datasets (only two datasets), which may not fully reflect performance in real-world scenarios.\n3. FedCEO seems to be a shallow method. What are its advantages and disadvantages compared to deep methods? It would be ideal to compare it with newer deep methods to highlight the value and significance of the research in this paper.", "other_comments_or_suggestions": "Traditional tensor methods have been extensively studied, and it would be more convincing to try using them to validate the effectiveness of this method within deep learning approaches.", "questions_for_authors": "See weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uXMCfD2X3l", "reviewer": "Reviewer_kpz4", "summary": "The authors propose a novel federated learning framework with the differential privacy mechanism, focusing on improving the trade-off between model utility and user privacy. By leveraging tensor decomposition techniques, the proposed method can model the dynamic semantic relationships among different clients, thereby enhancing the performance of the global model. The authors provide comprehensive mathematical analysis and empirical evidence, demonstrating that their framework achieves a better privacy-utility trade-off.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s constructive feedback and recognition of our work. Below are our point-by-point responses:\n\n> **W1 & Comments1, 2**\n\nCondensing Background Knowledge: We agree with the suggestion. In the revised manuscript, we will streamline Section 2 (Preliminaries) by moving detailed definitions to the appendix, allowing greater emphasis on the core methodology (e.g., the optimization objective of FedCEO in Section 3 and the motivation for low-rank modeling).\n\nAbstract Experiment Description: Thank you for the correction. We will update the abstract to include results on text datasets (e.g., Sentiment140): \"...with experiments on representative image and text datasets…\".\n\n> **W2 & Comments3**\n\nWe will add a discussion of 'Provable Mutual Benefits from Federated Learning in Privacy-Sensitive Domains' in the Related Work (Section 1.1). This work proposes a theoretical framework for designing personalized privacy-preserving protocols that provably benefit all participants in privacy-sensitive domains. Compared to our approach, it does not explicitly provide quantitative utility-privacy bounds but focuses on existence proofs. Both works involve inter-client collaboration, representing two complementary approaches: **protocol design controlling client-side noise** versus **post-processing algorithms correcting inter-client semantics**.\n\n> **Q1**\n\n\"Semantic complementarity\" refers to the fact that local DP noise disrupts semantic information differently across clients. In DPFL, due to the randomness of noise introduced by each client, specific semantic information might be disrupted in some clients while remaining relatively intact in others, leading to semantic complementarity among different clients. By integrating the commonalities of parameters via tensor low-rank optimization, the server recovers global semantic smoothness. For example, in Figure 1 (CIFAR-10), the semantic space of the 10th class is corrupted (red row), but FedCEO leverages intact semantic features from other clients (e.g., client 7, 9) to restore smoothness (blue row), improving classification accuracy.\n\n\n> **Q2**\n\nConsider a DPFL framework with only two clients performing animal image classification. This collaboration mechanism enables semantic complementarity between the two clients' models, thereby improving the global model's performance in DPFL.\n\nFor one client, the noise might severely corrupt parameters responsible for recognizing facial features, while for the other client, the noise might primarily distort parameters for limb recognition. However, by performing a Fourier transform, each slice will contain information from all clients. Subsequently, applying truncated SVD to all slices can be viewed as projecting the perturbed parameters back into a smooth semantic space, leveraging knowledge from all clients.\n\nFinally, after performing the inverse Fourier transform, each slice (representing a client's parameters) will have adaptively incorporated information from other clients while retaining some of its original knowledge.\n\n> **Q3**\n\nT-tSVD is chosen for its efficiency in dynamic thresholding via Fourier-domain matrix factorization (Table 4). Future work may explore other methods (e.g.,deep tensor learning), provided they align with privacy constraints. Existing attempts can be seen in our response to **Reviewer RVpv (W3 & Other Comments/Suggestions)**.\n\n> **Q4**\n\nEquation (2) is not a denoising process, as we do not utilize any prior information about the DP noise in our modeling. Essentially, it is a controlled fusion operation, and Proposition 3.2 demonstrates that FedAvg is in fact a special case of our FedCEO method—yet FedAvg is not considered to possess denoising capabilities. Furthermore, both Theorem 4.5 and the experiments in Section 5.3 confirm that the server-side operations in Algorithm 2 do not compromise the privacy guarantees of the DPFL framework.", "claims_and_evidence": "Yes, the main claims in the paper are well supported by both experimental results and theoretical analysis. The visualization experiment in Figure 1 and Theorem 3.1 support the motivation of this work. Corollary 4.6 and the experiments in Figure 4 provide strong evidence demonstrating the effectiveness of the proposed framework in improving the privacy-utility trade-off in federated learning.", "methods_and_evaluation_criteria": "The proposed method is appropriate, as Proposition 3.2 in the paper proves that the classic federated learning algorithm FedAvg is a special case of it. The chosen datasets are suitable and comprehensive, covering general datasets of different modalities and adopting common federated learning data partitioning methods.", "theoretical_claims": "Yes, I mainly focus on the proof details of Theorem 4.3. The overall proof approach is sound and successfully demonstrates the effect of low-rank (semantic collaboration).", "experimental_designs_or_analyses": "The experiments in this work are rigorous and comprehensive, systematically designed to evaluate utility and privacy. Additionally, the study includes experiments under heterogeneous federated settings, efficiency comparisons, and tests on textual datasets.", "relation_to_broader_scientific_literature": "Previous works have considered spectral decomposition at the local client level, while this method further incorporates inter-client relationships and models them using high-order tensor singular value decomposition. Compared to (Jain et al., 2021) and CENTAUR (Shen et al., 2023), the theoretical bounds have been further optimized.", "essential_references_not_discussed": "The references in this work are generally comprehensive. However, it would be beneficial to include a discussion on recent related works, such as Provable Mutual Benefits from Federated Learning in Privacy-Sensitive Domains (AISTATS 2024).", "other_strengths_and_weaknesses": "**Strengths:**\n1. This paper primarily focuses on achieving the utility-privacy trade-off in differentially private federated learning framework. This is a crucial research topic with significant implications for the practical deployment of federated learning algorithms in industrial applications.\n2. This work follows a rigorous logical structure, where the authors naturally introduce their framework through preliminary experiments and insights from previous works.\n3. The proposed approach is novel, as it connects tensor decomposition algorithms with federated model updates and provides an equivalence proof. This offers new perspectives for designing global model update paradigms in federated learning.\n4. Theoretical analysis is thorough, and the final conclusions demonstrate the effectiveness of low-rank modeling, providing readers with a deeper understanding of the fundamental principles behind the proposed approach.\n\n**Weaknesses:**\n1. There is still room for improvement in the paper’s writing, particularly in terms of the proportion of content across different sections.\n2. The paper lacks a discussion of some recent related works, such as Provable Mutual Benefits from Federated Learning in Privacy-Sensitive Domains (AISTATS 2024).", "other_comments_or_suggestions": "1. It is recommended to compress the length of the background knowledge in Section 2 and place more emphasis on the description of the method.\n2. The description of the experiments in the abstract needs to be revised, as the paper considers both image and text datasets.\n3. In Section 1.1 (Related Work), it is suggested to discuss the differences between the proposed method and the latest techniques in terms of mechanism design and theoretical analysis. Additionally, it might be useful to add comparative baselines.", "questions_for_authors": "1. How should \"semantic complementarity\" in the introduction be understood?\n2. Can you provide a specific example of how semantic collaboration between clients is implemented?\n3. Apart from the T-tSVD algorithm used in the paper, can other more efficient tensor optimization methods be applied?\n4. Can the model optimization in Equation 2 be understood as a denoising process of the noisy model tensor? Would this compromise the privacy of the overall differential privacy federated framework? I'm slightly concerned about this.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FOnqAODu0h", "reviewer": "Reviewer_vBZu", "summary": "The paper proposes a novel federated learning framework, FedCEO, aimed at balancing model utility and user privacy through collaboration among clients. The authors introduce a compelling case study to illustrate the potential of semantic collaboration among clients in enhancing the utility of the global model, and they construct the update process as a high-order tensor low-rank optimization. Furthermore, the authors theoretically prove that their model achieves a $\\sqrt{d}$-order improvement in the utility-privacy trade-off bound. Extensive experiments, covering both utility and privacy aspects, demonstrate the effectiveness of FedCEO.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s positive feedback on our theoretical and empirical contributions. Below are our point-to-point responses to the raised concerns and suggestions.\n\n> **W1**: Lack of Depth in Technical Details: The detailed discussions on parameter selection are somewhat brief in the main text. More technical details and discussions on parameter tuning could further enhance the reproducibility and practicality of the paper.\n\nThe reviewer noted brevity in parameter selection discussions. We have detailed hyperparameter search ranges and final configurations in **Appendix C.1 (Table 2)**. For example, on CIFAR-10 with σ_g=1.5, we set λ=10, ϑ=1.07, and I=10. These parameters are selected via grid search and adaptively adjusted based on privacy requirements (e.g., smaller λ for stricter privacy). Please refer to the response to the **reviewer RVpv (Experimental Designs/Analyses) for additional experiments**.\n\n> **W2 & Q2**: Unclear Descriptions of Concepts: Some concepts or keywords in the paper, such as \"semantic collaboration\" and \"global semantic space smoothing,\" could be described more clearly and in greater detail. More precise definitions and explanations would help readers better understand these concepts.\n\n- **Global Semantic Space Smoothing**: Refers to the low-rank representation of client parameters in the spectral space after tensor decomposition, where high-frequency components are truncated to preserve semantic correlations\n\n- **Semantic Collaboration**: Achieved by integrating complementary semantic information across clients via low-rank optimization (Section 3.2). Please refer to the responses to **Reviewer kpz4 (Q2) for specific examples**.\n\n> **W3**: Breadth of Comparative Experiments: Although the paper compares with some existing methods, the scope of comparative experiments could be expanded to include more recent federated learning and differential privacy methods. This would provide a more comprehensive demonstration of FedCEO's advantages.\n\nWe appreciate your suggestion and will incorporate additional baseline comparisons in the revised manuscript to provide a more comprehensive validation of our findings.\n\n> **Q1**: Could the authors further explain why FedCEO is referred to as a \"flexible\" DPFL framework, as indicated in the title?\n\nAs illustrated in Figure 2, our FedCEO can flexibly adapt to different privacy settings and noise accumulation during continuous training by setting different initial values and employing an adaptive thresholding rule based on a geometric series. The specific advantages of this \"flexibility\" are demonstrated in the experimental results in Table 1.\n\n> **Q3**: How exactly is the non-iid data distribution set up in Table 5?\n\nApologies for the lack of details on the degree of non-iid data. For non-iid data, we used a common method to construct data heterogeneity. \n\nTaking CIFAR-10 as an example: \n\n- First, we sorted all images in the dataset by their labels. \n\n- Then, for each client, we randomly selected 250 consecutive images twice. Since CIFAR-10 has ten categories, each client has at least one category and at most four categories, verifying that our framework works even in highly heterogeneous situations.", "claims_and_evidence": "The claims made in the submission are well-supported by clear and convincing evidence. The authors provide thorough theoretical analysis and experimental validation to substantiate the significant improvements in the utility-privacy trade-off achieved by the FedCEO. Firstly, the authors derive the utility-privacy trade-off bound for FedCEO in the theoretical section and prove its improvement by a factor of $\\sqrt{d}$ over existing techniques. Secondly, extensive experiments on multiple representative datasets validate the performance enhancements and privacy-preserving capabilities of FedCEO under various privacy settings.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are highly appropriate and well-suited for the problem at hand. The experiments utilize representative datasets such as CIFAR-10, EMNIST and Sent140, along with various model architectures (e.g., MLP, LeNet and AlexNet), thoroughly validating the method's effectiveness and generality. Additionally, the authors further validate privacy protection through gradient inversion attack experiments, making the evaluation criteria comprehensive and convincing.", "theoretical_claims": "The theoretical proofs in the paper have been carefully reviewed and are generally correct and rigorous.", "experimental_designs_or_analyses": "The experimental designs and analyses in the paper have been carefully reviewed and are generally sound and valid. The experimental section covers a comprehensive evaluation of model utility, privacy protection, and the utility-privacy trade-off, with a well-designed and convincing approach. Particularly, the authors further validate the robustness of FedCEO in privacy protection through gradient inversion attack experiments, with clear and reproducible results. Overall, the experimental designs and analyses are rigorous and support the main conclusions of the paper.", "relation_to_broader_scientific_literature": "The key contributions of the paper are closely related to the broader scientific literature, particularly in the fields of DPFL. Firstly, the FedCEO addresses the utility-privacy trade-off in differentially private federated learning through semantic collaboration among clients, which differs from existing approaches that focus on regularization or personalization to improve model utility. For example, *Cheng et al.* proposed local update regularization and sparsification techniques, while *PPSGD* improved model utility through a personalized privacy-preserving stochastic gradient optimization algorithm. However, these methods primarily focus on constraining local updates and do not fully leverage the semantic complementarity among clients. FedCEO further enhances model utility through high-order tensor low-rank optimization and theoretically proves its improvement in the utility-privacy trade-off bound. Additionally, the experimental results of this paper demonstrate significant performance improvements and strict privacy guarantees compared to existing literature (e.g., *CENTAUR*). Overall, the paper introduces an innovative approach based on existing research and provides a new solution to the utility-privacy trade-off problem in federated learning.", "essential_references_not_discussed": "The references in the paper already cover most of the essential related works", "other_strengths_and_weaknesses": "Strengths:\nThe strengths of the paper lie in its originality and practical applicability. Firstly, the proposed framework addresses the utility-privacy trade-off in differentially private federated learning through semantic collaboration among clients, which is highly innovative. Secondly, the paper not only theoretically proves the improvement in the utility-privacy trade-off bound but also validates its effectiveness on real-world datasets through extensive experiments, demonstrating its potential in practical scenarios.\n \nWeaknesses:\nThe weaknesses of the paper can be outlined as follows:\n1. Lack of Depth in Technical Details: The detailed discussions on parameter selection are somewhat brief in the main text. More technical details and discussions on parameter tuning could further enhance the reproducibility and practicality of the paper.\n2. Unclear Descriptions of Concepts: Some concepts or keywords in the paper, such as \"semantic collaboration\" and \"global semantic space smoothing,\" could be described more clearly and in greater detail. More precise definitions and explanations would help readers better understand these concepts.\n3. Breadth of Comparative Experiments: Although the paper compares with some existing methods, the scope of comparative experiments could be expanded to include more recent federated learning and differential privacy methods. This would provide a more comprehensive demonstration of FedCEO's advantages.", "other_comments_or_suggestions": "Please refer to the weaknesses.", "questions_for_authors": "1. Could the authors further explain why FedCEO is referred to as a \"flexible\" DPFL framework, as indicated in the title?\n2. What is the definition of \"global semantic space\" in the paper?\n3. How exactly is the non-iid data distribution set up in Table 5?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a federated learning framework (FedCEO) that aims to improve the utility-privacy trade-off in differentially private FL settings by promoting semantic "collaboration" among clients. The method applies tensor low-rank proximal optimization on stacked local model parameters at the server. This update helps filter out high-frequency noise components introduced by differential privacy mechanisms. Theoretical results show an improved utility-privacy trade-off bound scaling with sqrt(d). The empirical results on image and text datasets validate the framework’s effectiveness. 

Several reviewers found the theoretical foundation solid. They also found the experiments well-structured. The framework's ability to outperform or match baseline methods on standard datasets such as CIFAR was also noted as a strong point by reviewers (despite these datasets have been used a lot). However, multiple reviewers raised concerns around the clarity and justification of certain methodological choices, especially regarding the definition and motivation of the utility-privacy tradeoff as a product objective (of epsilon values). There was also some concerns about insufficient discussion of hyperparameter selection, particularly the lack of experiments on more complex or large-scale datasets. In the rebuttal phase these concerns are (mostly) addressed and the reviewers generally tend toward acceptance of the paper.
Decision: Accept (poster)