Convergent Differential Privacy Analysis for General Federated Learning
Authors: Yan Sun
Qixin Zhang
Li Shen
Dacheng Tao
Venue: ICLR 2026 Poster (2026)
Keywords: Differential privacy
federated learning
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We analyze the convergent differential privacy bound for the worst case in the general federated learning.

Abstract: The powerful cooperation of federated learning (FL) and differential privacy (DP) provides a promising paradigm for the large-scale private clients. However, existing analyses in FL-DP mostly rely on the composition theorem and cannot tightly quantify the privacy leakage challenges, which is tight for a few communication rounds but yields an arbitrarily loose and divergent bound eventually. This also implies a counterintuitive judgment, suggesting that FL-DP may not provide adequate privacy support during long-term training under constant-level noisy perturbations, yielding discrepancy between the theoretical and experimental results. To further investigate the convergent privacy and reliability of the FL-DP framework, in this paper, we comprehensively evaluate the worst privacy of two classical methods under the non-convex and smooth objectives based on the $f$-DP analysis. With the aid of the shifted interpolation technique, we successfully prove that privacy in Noisy-FedAvg has a tight convergent bound. Moreover, with the regularization of the proxy term, privacy in Noisy-FedProx has a stable constant lower bound. Our analysis further demonstrates a solid theoretical foundation for the reliability of privacy in FL-DP. Meanwhile, our conclusions can also be losslessly converted to other classical DP analytical frameworks, e.g. 
$(\epsilon,\delta)$-DP and R$\'{e}$nyi-DP (RDP), to provide more fine-grained understandings for the FL-DP frameworks.

Published as a conference paper at ICLR 2026
CONVERGENT DIFFERENTIAL PRIVACY ANALYSIS FOR GENERAL FEDERATED LEARNING
Yan Sun
Qixin Zhang
School Computer Science
Generative Lab
Faculty of Engineering the University of Sydney ysun9899 sydney edu.
College of Computing and Data Science
Nanyang Technological University
qixin. zhangentu.edu edu.
Li Shen
Dacheng Tao
School of Cyber Science and Technology Shenzhen Campus Sun Yat-sen University ma mathshen] @mga 1. com
Generative AI Lab
College Computing and Data Science Nanyang Technological University da dacheng. taožntu. edu.. edu.sg
ABSTRACT
The powerful cooperation of federated learning (FL) and differential privacy (DP) provides a promising paradigm for the large-scale private clients. However, existing analyses in FL-DP mostly rely on the composition theorem and cannot tightly quan- tify the privacy leakage challenges, which is tight for a few communication rounds but yields an arbitrarily loose and divergent bound eventually. This also implies a counterintuitive judgment, suggesting that FL-DP may not provide adequate privacy support during long-term training under constant-leve noisy perturbations, yielding discrepancy between the theoretical and experimental results. To further investigate the convergent privacy and reliability of the FL-DP framework, in this paper, we comprehensively evaluate the worst privacy of two classical methods under the non-convex and smooth objectives based on the f-DP analysis. With the aid of the shifted interpolation technique, we successfully prove that privacy in Soiisy-FedAAee
has tight convergent bound. Moreover, with the regularization of the proxy term, privacy in Noisy- sy-FedP rox has a stable constant lower bound. Our analysis further demonstrates solid theoretical foundation for the reliability of privacy in FL-DP. Meanwhile, our conclusions can also be losslessly converted to other classical DP analytical frameworks, e.g. (e, 8DP and Rényi-DP (RDP), to provide more fine-grained understandings for the FL-DP frameworks.
INTRODUCTION
Since McMahan al. (2017) proposes the FedAvg method as general FL framework, has been widely developed into collaborative training standard with privacy protection attributes, which successfully avoids direct leakage of sensitive data. As research on privacy progresses, researchers have found that standard FL frameworks still face threat from indirect leakage. Attackers can potentially recover local private data through reverse inference by persistently stealing model states via model (gradient) inversion attacks (Geiping et al., 2020) distinguish whether individuals involved in the training via membership inference attacks (Nasr et al., 2019). To further strengthen the reliability of FL, DP (Dwork, 2006; Dwork et al., 2014; Abadi et al., 2016) has naturally been incorporated into the FL framework, yielding FL-DP (Wei et al., 2020). As a primary technique, the noisy perturbation is widely applied in various advanced FL methods further enhance its security.
However, the theoretical analysis of the FL-DP framework, especially in evaluating the privacy levels, currently unable provide comprchensive understanding of its proper application. Most
"quual contribution.
Corresponding author. Dr. Tao's research is partially supported by NTU RSR and Start Up Grants.

---

Published as a conference paper at ICLR 2026
Table 1: The worst privacy of the No -y-FedAvg and Neiisy-FedProx in analysis. V is the norm of clip gradient. K, T are local training interval and communication round. a is the variance of the noise. The trade-off function (c()) [o] is defined in Definition 4. μ., c and & are constants.
Convergent? Convergent? on T + 0c on K + 00
Lr [
Worst Privacy
C
Tc
2uVK v/ma
[I+aL)*+1 (1ta))FF]) [I+pL] (I+pL)RT+]
220 (20Vin( In((+)
[1+K)au++1 (1+ LKNONNT-1 (1+K)ook- (1- K))OOTT))
CD
TG
x
Noisy
/mia
FedAvg
SD
To
[2uVK Vmna /2
ID
To
22V Vmna
V
+1
FedProx a> L
Noisy
To
2V Vmaa
2a L
a 10- 7. a +(a-a=))
[a] For the trade-off function (s(s), smaller s means stronger privacy. [b] Learning rate decaying policy. C: constant learning rate; CD: cyclically decaying; SD: stage- wise decaying; ID: iteratively decaying. More details are stated in Theorem 3 4.
of the previous works are built upon the foundational lemma of privacy amplification by iteration, directly resulting in divergent privacy bound as the training communication round T becomes large. This implies an inference that contradicts intuition and empirical studics, which is, that the FL-DP framework may completely lose its privacy protection attributes T - oc. Such conclusion is almost unacceptable for FL-DP. Therefore, establishing a precise and tight analysis a crucial target.
Notably, significant progress has been made in characterizing convergent privacy in the noisy gradient descent method in RDP analysis (Chourasia et al., 2021; Ye & Shokri, 2022; Altschuler & Talwar, 2022; Altschuler et al., 2024). However, due the challenges and intricacies of the analytical techniques adopted, similar results have not yet successfully been extended to the FL.-DP. The multi- step local updates on heterogeneous datasets lead to biased local models, posing significant obstacles to the analysis. Recently, analyses based on f-DP (Dong et al., 2022) have brought promising resolution to this challenge. This information-Ee thherttiiittoterrllllll lossless definition naturally evaluates privacy by the Type I II error trade-off curve of the hypothesis testing problem about whether given individual is in the training dataset. Combined with shifted interpolation techniques (Bok et al., 2024), it successfully recovers tighter convergent privacy for strongly convex and convex objectives in noisy gradient descent methods. This may make it possible to quantify convergent privacy in FL-DP and may offer novel understandings about impacts of some key hyperparameters.
In this paper, we investigate the privacy two classic DP-FL methods, i.c. Noisy-FedRvg and Noisy-FedP rox and successfully evaluate their worst privacy in the f-DP analysis, shown in Table For the Noisy-FedAve method, we investigate four typical learning rate decay strategies and provide the coefficients corresponding to each case ensure tighter privacy lower bound. We also prove that its iterative privacy on non-convex and smooth objectives could not diverge w.r.t. the number of communication rounds T, i.e., convergent privacy. To the best of our knowledge, this contributes the first convergent privacy analysis in FL-DP methods for non-convex functions. Furthermore, by exploring the decay properties of the proximal term in No Noisy-FedPP rox, we prove that its worst privacy can converge to a general constant lower bound. Our analysis successfully challenges the long-standing belief that privacy budgets of FL-DP have to increase as training processes and provides reliable guarantees for its privacy protection ability. At the same time, the exploration from the proximal term provides a promising solution, suggesting that a well-designed local regularization term can achieve a win-win solution for both optimization and privacy in FL-DP. Our work reveals that DP maintains strong privacy protection capabilities even under complex local- global optimization dynamics. This finding provides theoretical guarantees for the native design of federated learning, demonstrating that properly structured local and global interactions can preserve rigorous privacy assurances without fundamentally compromising the learning framework.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Federated Learning. FL is a classic learning paradigm that protects local privacy. Since McMahan al. (2017) proposes the basic framework, it has been widely studied in several communities. As its foundational study, the local-SCD (Stich, 2019; Lin et al., 2019; Woodworth et al., 2020; Gorbunov et al., 2021) method fully demonstrates the efficiency of local training. Based on this, FL further considers the impacts of heterogeneous private datasets and communication bottlenecks (Wang et al., 2020; Chen al., 2021; Kairouz et al., 2021). To address these two basic issues, a series of studies have explored these processes from different perspectives. One approach involves proposing better optimization algorithms by defining concepts such as client drift (Karimireddy et al., 2020) and heterogeneity similarity (Mendieta et al., 2022), specifically targeting and resolving the additional error terms they cause. This mainly includes the natural application and expansion of variance- reduction optimizers (Jhunjhunwala al., 2022; Malinovsky et al., 2022; Li et al., 2023a), the flexible implementation of the advanced primal-dual methods (Zhang et al., 2021c; Wang et al., 2022; Sun et al., 2023b; Mishchenko et al., 2022; Grudzień et al., 2023; Acar et al.; Sun et al., 2023a), and the additional deployment of the momentum- based correction (Liu et al., 2020; Khanduri et al., 2021; Das et al., 2022: Sun et al., 2023c; 2024b;a). Upgraded optimizers allow the aggregation frequency to largely decrease while maintaining convergence. Another approach primarily focuses on sparse training and quantization reduce communication bits (Reisizadeh al., 2020; Shlezinger et al., 2020; Dai et al., 2022). Additionally, research based on data and feature domain has also made significant contributions (Yao al., 2019; Zhang al., 202 la; Xu et al.; Li et al., 2023b).
framework with theoretical foundations (Dwork et al., 2006b;a; Dwork, 2006). As one of the main algorithms for differential privacy, noise perturbation has achieved great success in deep learning (Abadi et al., 2016; Zhao et al., 2019; Arachchige et al., 2019; Wu al., 2020). Combining this, FL-DP adds noise before transmitting their variables, i.e. client-level noises (Geyer et al., 2017) and server-level noises (Wei et al., 2020). Since there is no fundamental difference between the analysis of them, in this paper, we mainly consider client-level noises. One major research direction involves conducting noise testing widely developed federated optimization algorithms (Zhu et al., 2021; Noble et al., 2022; Lowy et al., 2023; Zhang & Tang. 2022; Yang & Wu, 2023), and evaluating the performance of different methods under DP noises through convergence analysis and privacy analysis. Another research direction involves injecting noise into real-world systems to address practical challenges, which primarily focuses on personalized scenarios (Hu et al., 2020; Yang et al., 2021; 2023; Wei et al., 2023), decentralized scenarios (Witkkopp & Acker, 2020; Chen et al., 2022; Gao et al., 2023; Shi al., 2023), and adaptive asymmetric update scenarios (Girgis et al.. 2021; Wu et al., 2022; He et al., 2023). FL-DP has been extensively tested across various scales of tasks and has successfully validated its robust local privacy protection capabilities. At the same time, the theoretical analysis of FL-DP has been progressing systematically and in tandem. Based on various DP relaxations, they provide comparison of privacy performance by analyzing concepts such as privacy budgets, and further understand the specific attributes of privacy algorithms (Rodríguez-Bardt) Barroso et al., 2020; Wei et al., 2021; Kim et al., 2021; Zheng et al., 2021; Ling et al., 2024; Jiao et al., 2024). Theoretical advancements in DP have revolutionized how we could quantify and safeguard privacy, offering unprecedented precision and robustness.
FL-DP. DP is a natural privacy-preserving
PRELIMINARIES
3.1 GENERAL FL-DP FRAMEWORK
We consider the general finite-sum minimization problem in the classical federated learning:
w² E arg min [(w)
f((w),
1ET
where f.w)) = LEED, [f](w])]] denotes the local population risk. w E I denotes d-dim learnable parameters. 'D; denotes that the private dataset on client i is sampled from distribution D,, We consider the general heterogeneity, iee D; can differ from Dj if i + j. leading to f((w) + (w). More detailed notations are introduced in Appendix A.
In our analysis, we consider the FL-DP framework with the classical client-level Gaussian noises. The FL training process remains consistent with standard training procedures. The local clients
3

---

Published as a conference paper at ICLR 2026
enhance local privacy by adding isotropic Gaussian noises to the uploaded model parameters, i.e. n, ~ NV La). Then the global server aggregates the noisy parameters as the global model W1+1- Due to the page limitation, details of the algorithmic implementation deferred to the Appendix B.
Noisy-FedAve we consider that each local client performs a fundamental gradient descent as follows:
Wi,4+1,1 - Ws,k,t 1k,,
where Di,K,t (ii max
| [[( ), and V is a constant coefficient.
Noisy-FedProx: The vanilla local training in FedP rox is based on solving the following surrogate:
min fi(w) + 2 -
To generally compare with Noisy-FedRsye we consider an iterative form of gradient descent as: Wi,++1,1 = Wi,k,t Thet 19i,,,t + a(WiN,t -w,)].
3.2 DP AND f-DP
We denote heterogeneous datasets on the client by S; [Eig} (eig) and let the union of all local datasets be == {S.]. We say two unions are adjacent datasets they only differ by one data sample. For instance, there exists the union =²= = (C, (C,C") are adjacent datasets if there exists the index pair (i* (i*) such that all other data samples are the same except for E F Elz--
Definition
Definition 2 A randomized mechanism M is (c,8)-DP for any event E the the following satisfies:
P(M(C) E E) S S(P((M(C)) E E)) 8
(5)
Definition is the widely used (e, 8D, which is a lossy relaxation in the DP analysis since its probabilistic gaps. To bridge the discrepancy of precise DP definitions, statistic analysis demonstrates that DP could be naturally deduced by hypothesis- testting problems (Wasserman & Zhou, 2010; Kairouz et al., 2015). From the perspective of attackers, DP means the difficulty distinguishing and C' under the mechanism M. They can generally consider the following problem:
Given M, is the underlying union C (Ho) or C' (Hi)?
To exactly quantify the difficulty of its answer, Dong et al. (2022) propose that distinguishing these two hypotheses could be best delineated by the optimal trade-off between the possible type and type errors. Specifically, by considering rejection rules 0 I x S 1, type and type II errors can be:
E₁ EMee) Ixl,
E1 1 EMiry [x]
(6)
Here, we abuse M(C) to represent its probability distribution. To measure the fine- tieeggraiedd relation- ships between these two testing errors, f-DP is introduced.
Definition 3 (Trade-off function) For any probability distributions P and Q the trade-off function is defined as: T(P; (n()) = inf (1 IBQ Ixl Ep Ixl S 7), where the infimum is taken over all measurable rejection rules.
T(P:Q)(7) is convex, continuous, and non-increasing. For any possible rejection rules, satisfies T(P;Q)() < y. It functions as the clear boundary between the achievable and unachievable selections of type I and type errors, essentially distinguishing the difficulties between these two hypotheses. This relevant statistical property provides stricter definition of privacy, which mitigates the excessive relaxation of privacy based on composition analysis in existing approaches.
Definition (f-DP and GDP) A mechanism M is JDP if (M M(C')( 2 f(>) for all possible adjacent datasets C and C', When f measures two Gaussian distributions, namely Gaussian- DP (GDP), denoted as Te(μ)(n) T (N((, (), 1)) (7) for 0.
According to the definition, the explicit representation of GDP is To (T) ( where & denotes the standard Gaussian CDF. Any single sampling mechanism that introduces Gaussian noises be considered exact GDP, which monotonically decreases when y increases.
4

---

Published as a conference paper at ICLR 2026
WT-1
Wy = Wy
1
8
(w))
W
Data Sensitivity
W1
Wy-1 1
W #o+1
Privacy Amplification Path
('w.)
wo
wo
W
Model Sensitivity
W1
w(W,))
W
W ko
W
W W
W
Figure 1: Left: The global privacy amplification path induced by the shifted interpolation sequence. Right: Estimation the global sensitivity under local updates via an auxiliary sequence.
Threat model and Privacy. In the FL-DP framework, we primarily consider two types of privacy: (1) client-side privacy when uploading local parameters to the server, which is protected by injecting DP noise; and (2) global-model privacy when sending aggregated parameters back to the clients, ensuring that no information can be extracted or inferred from the global model. In the standard FL-DP framework, the protection of local privacy is straightforward. Our analysis therefore focuses on the second aspect: the privacy behavior associated with the global model.
4 CONVERGENT PRIVACY
In this section, we primarily demonstrate how to provide the worst privacy in FL-DP and its convergent bound. Generally, we assume that local objectives satisfy smoothness with a constant Assumption Each local objective function fz(-) satisfies L-smoothness, i.e.,
fi(w2)ll S Ll wall-
4.1 SHIFTED INTERPOLATION
To simplify presentations, denote global updates at round t on the adjacent datasets C and C' as:
C: Wx+1 <(w,) + The C' 4'44 o' (((ww) TV.
(8)
((w;) denotes the accumulation of total K steps from the initialization state wa,0.t = wy round t. Th could be considered as the averaged noise, i.e. Tú, ~ (0, la/a/m). Traditional methods require performing privacy amplification 7' times based the relationship between and w', yielding non-con non-convergent privacy as T. To avoid loose privacy amplification, we follow Bok et al. (2024) to adopt the shifted interpolation technique. Specifically, we define the following sequence:
W++1 \ (1 A++1) (w:) Th,
where lo. .... 1 By setting \r then wr wr, and we add the definition of we w/o as the beginning of interpolations. 0 < A are interpolation coefficients to be optimized. As shown in Figure (left), the interpolation sequence path enables privacy amplification analysis over 7' - to times where to is an optimizable coefficient. Therefore, we can establish the following theorem along this privacy amplification path.
Theorem Under Assumption I and corresponding updates in (8),)) after T training rounds the adjacent datasets and C', we can bound the trade-off function between wT and w/ as:
T(wT: w/r) T(wr; ((/r:w/)) Te
mm
X ( d( Σ
(10)
In addition to the influence of standard parameters, Theorem highlights the critical relationship between the privacy lower bound and the weighted sum of global sensitivity terms from to to T. Therefore, then analyze the global sensitivity term [|o(we) o' (((w)))
5

---

Published as a conference paper at ICLR 2026
Table 2: Specific formulation of pr and 7t in Theorem
Learning rate
Pt
2
(1 + (1+x
ZuV K
(1 + K) K)oub
2cV. m
In K + 1)
1+T
Noisy Feddvg
K
auh
PuV
K
c+1 I
1+1
1+I
zuL
2
2+2 1+1
1K +++1
[+1
Y78
Noisy-FedProx
non-increase
2V
T.
4.2 GLOBAL SENSITIVITY
The sensitivity term ((o(w)) - 0' (w:)))l' means the stability gaps between w; and W; after performing local training on datasets C and C' respectively. is influenced by both the model parameters and the data samples, making the analysis extremely challenging. To achieve a fine-grained analysis, propose an auxiliary sequence o' b'(wz). ). As shown in Figure I (right), the global sensitivity can be split into data sensitivity and model sensifivity. The data sensitivity measures the estimable errors obtained after training on different datasets for several steps from the same initialization. This discrepancy is solely caused by the data. The model sensitivity measures the estimable errois of the updates when two different initialized states are trained on the same dataset. Clearly, this discrepancy is directly related to the degree of similarity between the two initializations. Thus, we have:
Theorem 2 Under K local updates by E2. and (4., the global sensitivity in Noisy-FedAvg and Noi -y-FsdFFPPrrrr rox methods can be shown as:
(ló(w;) o' ('(wa)ll S
Pellw:- Well
(11)
from sonsitivity
from data sensitivity
where Pt and Yt are shown in Table 2.
Remark 2.1 The result ( (I) aligns with the intuition of designing the splitting operators. can be observed that the coefficient is consistently greater than 1, which is a typical characteristic of non-convexity. also implies that the sensitivity upper bound tends to diverge as t - co. However, in Eq. (10), the parameters 0 S X < 1 can efficiently scale the sensitivity terms. By carefully selecting the optimal Xe values, can ultimately achieve a convergent privacy lower bound.
4.3 MINIMIZATION PROBLEM ON tu AND ITS RELAXATION
According to (10) and the sensitivity bound in (1.(1 we denote the weighted accumulation of the sensitivity term as H(A, to), where xe and to are both to-be-optimized parameters. Therefore, we can provide the tight bound of the privacy by solving the minimization of the following problem:
H. min H((Ae, to) Ae,to
Σ Xb (pellwe w.ll + 2 tmmo
(12)
If lo is very small, it means that the introduced stability gap will also be very small. However, consequently, the sensitivity terms will extremely increase due the accumulation over T - rounds. Conversely, although the accumulated error is small, it remains divergent due to the unbounded global sensitivity term. To avoid this uncertain analysis, we have make a compromise. Because to an integer belonging to [0, T - 1], its optimal selection certainly exists when T is given. Therefore, we consider a relaxed and simple problem instead, i.e. under to = O.
Ho min H 0) X
Σ Xēa1 (pellw: Will 7 0=
(13)
Its advantage lies in the fact that when tn = 0, the sensitivity error is 0, avoiding its divergence. Compared to the optimal solution H., it satisfies Ho H.. More importantly, the solution of Ho

---

Published as a conference paper at ICLR 2026
eliminates the influence of to allowing to obtain an effective solution to the minimization problem by directly minimizing the X terms. The lower bound in Theorem will be replaced by:
(YmiHW
T(wr; w/1) 2 Tc
Tc
(14)
Although this is a relaxation of the privacy lower bound, our subsequent proof confirms that Ny can still achieve convergent into constant form, which means local privacy can still achieve convergence. We additionally provide a discussion of its tightness in Appendix F.
4.4 CONVERGENT PRIVACY
In this part, demonstrate convergent privacy analysis. By solving (q.(13) under corresponding Pe and Yis we provide the worst privacy for the Noisy-FedAve and Noisy Noisy-FedP rox methods. Theorem 3 Let fe(w) be I--soothh and non-covex local objective and local updates be performed as shown in (2), Under perturbations of isotropic noïses n; ~ N (0, (0,o²la). the worst privacy of the Hoisy-FedAve method achieves:
(a) under constant learning rates Uh,t = μr
2nVK Vmo
1 1 (L) + +1 (1+ pL)
T(wr; wT) 2 To
(15)
- (1 pL) - WL) KT +1
(b) under cyclically decaying Ti, = K+I*
2cV In(K + +1)
+ K)OHL + (1 + K)OLLT
T(wT: w/)) 2 To
(16)
Vmma
K) K)ouL (1 + K CCHLT
(c) under stage-wise decaying Uktt
2 1
T(wr; w/) To
Vma /2 2- T
(17)
(d) under continuously decaying Mk. = IK+k+1
22V Vinar V
1 T
T(WT: (WT)W()) > Tc
2
(18)
Remark 3.1 (General Bound.) Theorem 3 provides the worst-case privacy analysis for the Poissy-FedAvo method. Its privacy primarily affected by the clipping norm V, the local interval K, the scale m, and the noise intensity a. A larger gradient clipping norm V always results in larger gaps. The local interval K determines the sensitivity of the entire local process, which primarily influenced by the learning rate strategy. m in our proof represents the client scale; in fact, the number of data samples is also proportional to m. An increased m will largely reduce the sensitivity, yielding improvements privacy. Infinite noise can provide perfect privacy, while zero noise offers no privacy. Constant-lewe noise can still achieve convergent privacy.
Remark 3.2 (Partial Participation.) The above analysis also applies the partial participation
setting. For example, suppose there are m clients in total, and in each round clients are selected to participate. This corresponds to a sampling process, where the expected privacy each round is at least amplified by a factor of Since the analysis of local iterations is carried out on each individual client, it not affected by this change. Therefore, under partial participation, the general the number of participating nodes, and one can simply replace m
upper bound depends linearly with n. In particular, when m = nt 1, the analysis reduces the standard DP-SGD.
Theorem 4 Let (w) be L-smooth and non-conve local objective and local updates be performed as shown in Eq.(4). Let the proximal coefficient a > L and rj < a-L' under perturbations of isotropic noises n, ~ N (2)o2La), the worst privacy of the Noisy Seay--FePraae rox method achieves:
2V
2a - L
2
T(wr; w/) 2 Ta
(19)
v Mmao
L
T
+
7

---

Published as a conference paper at ICLR 2026
Table 3: Comparisons with the existing theoretical results in FL-DP. We losslessly transfer our results into (c, 6)D and RDP results. In (c,5) we compare the requirement of noise variance corresponding to achieving (e, dDD.. In eC.e)-DD we directly compare the privacy budget term )()). We mainly foeus on the privacy changes T and K. Q(-). O(-), and o(-) correspond the lower, upper bound, and not tight upper bound of the complexity, respectively.
(c.6)-DP
(C.c)-RDP
when oc
Wei et al. (2020)
0
VT mL7)
V Vylualtl
Shi ct al. (2021)
o = o
T TvK
V (vlw(()) 3)
Zhang et al. (2021b)
a == O
IT + mk
cm
V 1 27
Noble et al. (2022)
a == f
vTK
tym
+ Dc on non-convex
Vy Vyloo(i) VT
Cheng et al. (2022)
a = ?
Zhang & Tang (2022)
St
C CKTK)
V (ve+oooo(3)
et al. (2023)
a = ??
T
Turyitoly/ba(esp A
Fukami et al. (2024)
= ?
Bastianello et al. (2024)
(1 11-e-10 c-11) 2
convergent
3-strongly convex
Ours (Noisy-FedAvg)
VV (1)0 ry/
CV2 mo?
(2 b))
convergent
7
non-convex
Remark 4.1 Impacts of the Regularization Coefficient a.] Aside from the inftuence of standard coefficients, due to the correction of the regularization term, its privacy is no longer affected by the local interval K, even with a constant learning rate, which becomes a significant advantage of the Moisy-FedProm rox method. Specifically, when a: > L,, increasing significantly improves the worst privacy. As a increases, the proximal term dominates the objective to greater extent, leading to less efficient per-step updates and a longer overall training trajectory. Therefore, the selection of a is delicate trade-off between optimization and privacy. We also validate this trade-off in the subsequent experiments. An excessively large may instead hinder the training process.
Theoretical comparisons. Table 3 demonstrates the comparison between existing theoretical results and ours of the Noisy-FedAte method. Existing analyses mostly based the DP relaxations of (c, 8DP and RDP (Mironov, 2017). Apart from the lossiness in their DP definition, an important weakness is that privacy amplification on composition is entirely loose. For instance, the general amplification in (e, 8DP indicates, the composition of an (e1, 6DP and an (e2, 62)-DP leads to an (EI + E2, 81 62 -DP. Similarly, the composition of (C, RDP and (C, <2)DDPP results in (C, 61 + RDD. This simple parameter addition mechanism directly leads to a linear amplification of the privacy budget. Therefore, in previous works, when achieving specific DP guarantees. is often required that the noise intensity o² is proportional to the communication rounds T (or TK). Wei et al. (2020) prove double-noisy single-step step local training on both client and server sides is possible to achieve the privacy amplification of (T2) rate. Shi et al. (2021) further consider the local intervals K. Zhang et al. (b02lb) and Noble et al. (2022) elevate the theoretical results to O (TK). Subsequent research further indicates that the impact of the interval K can be eliminated to achieve O (T) rate via sparsified perturbation (Hu et al., 2023; Cheng et al., 2022), and algorithmic improvements (Fukami et al., 2024). However, these conclusions all indicate that the condition for achieving constant privacy guarantees is to continually increase the noise intensity. Bastianello et al. (2024) provide constant privacy under 3-strongly convex objectives.
8

---

Published as a conference paper at ICLR 2026
Table 4: Comparison of the accuracy under different experimental settings. We select the scale m from [50, 100). Each client holds 600 heterogeneous data samples of MNIST or 500 heterogeneous data samples of CIFAR-10. For each scale, we test two settings of the local interval K = 50, 100, and 200, respectively. Throughout the entire process, we fix TK 30000. "." means the training loss diverges. Each result is repeated times to compute its mean and variance.
Noisy Intensity
m 50
m 100
K =
100
K 200
K =
100
K 200
1.0
MNIST LeNet-5
a = 10-1
95,401018
95.4222015
95.21
97.321014
97.42.2018 97.50_a.11
a 10
98.334012
97.88 98.02.am.5
97.97 98.71
_0.0
97.72_m.12
σ = 10 3
9804
98.23
98.00
98,944004
98.50_a.0
98.01 10.10
0 = 1.0
CIPAR-10 ResNet-18
10
53.761025
53.49 53.38
6
61.11 61.33_o.25
a 10
70.1 10:2
66.63 29.0
10.16
72.87 74.34
20.74
10 3
+0.98
69.81 +0.00
67.98
75.38
74.44-0.12
+0.0
5 EMPIRICAL VALIDATION
Setups. We conduct experiments on MNIST (LeCun et al., 1998) and CIFAR-10 (Krizhevsky et al., 2009) with the LeNet-5 (LeCun et al., 1998) and ResNet-18 (He et al., 2016) models. We follow the widely used standard federated learning experimental sctups to introduce hetcrogeneity by the Dirichlet splitting. The heterogeneity level is set high (Dir-0.1 splitting). In the following experiments, we follow the classical studies to adopt the stage-wise decaying learning rate for training. We also provide a sensitivity studies of all learning rate schedules in Appendix C.
Accuracy. Table shows the comparison on Fed.vv.. Our theory precisely addresses this misconception and rigorously provides its privacy protection performance. It can be observed that as the number of clients increases, the impact of noise gradually diminishes. We have previously explained this principle: for the globally averaged model, the more noise involved in the averaging process, the closer it gets to the noise mean, which is akin to the situation without noise interference. When we adjust the intensity from a = 10-3 3 to 10-1, the accuracy decreases by 5.57% and 1.62% on m = 20 and 100 respectively on the MNIST and 14.19% and 11% on the CIFAR-10 10. The local interval K does not significantly affect noise, and the accuracy drops consistently. primarily affects global sensitivity and higher aggregation frequency usually means better performance.
Sensitivity in Noisy-FedAvg. We mainly study the impact from the scale m, local interval K, and clipping norm V, as shown in Fig. 2. The first figure clearly demonstrates the impact of the scale m on sensitivity, which corresponds to the worst privacy bound O
More clients generally
imply stronger global privacy. The second figure shows evident that although increasing K can raise the sensitivity during the process, it does not alter the upper bound of sensitivity after optimization converges. This is entirely consistent with our analysis, indicating that the privacy lower bound exists and is unaffected by T and K. The third figure indicates that the sensitivity will be affected by the V, which corresponds to the worst privacy bound O (V).
Sensitivity in Noisy-FedProx.
Table 5: Performance and sensitivity (T' 600).
As shown in Fig. (the fourth figure), the larger means smaller global sen- sitivity. This is consistent with our analysis, which states that the lower bound of privacy performance is given by O Ja When we select a = O, it degrades to the Noisy-FedAvg method. In fact, based on the comparison, we can see that when a is sufficiently small, i.c. a = 0.01, its global sensitivity is almost at the same level as Noisy-FedAvg. In Table 5, present comparison between them. Although the proximal term provides limited improvement in accuracy, selecting appropriate significantly reduces global sensitivity. This implies that the privacy
Accuracy
Sensitivity
Noisy-FedAvg Fox a = 0.01
60.67
31.33
Noisy-FedProx Noisy-FedProx a 0.1
60.69
30.97
60.94
18.52
Noisy-FedProx a =
56.33
6.34

---

Published as a conference paper at ICLR 2026
Dutferunt Nolky-TodAng
Different K Noley-tedAng
putferent Vn Nolay-bodAvg
Different of Noley-DedPros
V-10
communication roand
Cumomnrscaaan FOURE
commuicattion
communication round
Figure 2: Sensitivity studies on Noisy-FedAvg and Noisy Noisy-FedPros The general setups are m 20, K 5, and 10. each group, keep all other parameters fixed to ensure fairness.
performance of Noisy morisy-eeee FedF rox is far superior to that of Ncisy-FedAvg, While achieving similar performance, the regularization proxy term can significantly reduce the global sensitivity of the output model, thereby enhancing privacy. This conelusion also demonstrates the superiority on privacy of a series of FL-DP optimization methods based on training with this regularization.
CONCLUSION
To our best knowledge, this paper the first work to demonstrate convergent privacy for the general FL-DP paradigms. We comprehensively study and illustrate the fine-grainee privacy level for Noisy-FedAvg and Noisy-FedP rox methods based on f-DP analysis, an information-theoree
lossless DP definition. Moreover, we conduct comprchensive analysis with existing work other DP frameworks and highlight the long-term cognitive bias of the privacy lower bound. Our analysis fills the theoretical gap in the convergent privacy of FL-DP while further providing reliable theoretical guarantee for its privacy protection performance. Moreover, We conduct a series of experiments to verify the boundedness of global sensitivity and its influence different variables, further validating that our theoretical analysis aligns more closely with practical scenarios.
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308-318, 2016.
Durmus Alp Emre Acar, Yue Zhao, Ramon Matas, Matthew Mattina, Paul Whatmough, and Venkatesh Saligrama. Federated learning based on dynamic regularization. In International Conference on Learning Representations.
Jason Altschuler and Kunal Talwar. Privacy of noisy stochastic gradient descent: More iterations without more privacy loss. Advances in Neural Information Processing Systems, 11:3788-3800 2022.
Jason M Altschuler, Jinho Bok, and Kunal Talwar. On the privacy of noisy stochastic gradient descent for convex optimization. SIAM Journal on Computing, 53(4):969-1001, 2024.
Pathum Chamikara Mahawaga Arachchige, Peter Bertok, Ibrahim Khalil, Dongxi Liu, Seyit Camtepe, and Mohammed Atiquzzaman. Local differential privacy for deep learning. IEEE Internet of Things Journal, 7(7):5877-5842. 2019.
Caridad Arroyo Arevalo, Sayedeh Leila Noorbakhsh, Yun Dong, Yuan Hong, and Binghui Wang. Task-agnostic privacy-pres sivacc-preserving representation learning for federated learning against attribute inference attacks. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 10909-10917, 2024.
Nicola Bastianello, Changxin Liu, and Karl H Johansson. Enhancing privacy in federated learning through local training. arXiv preprint arXiv:2403.17572, 2024.
Jinho Bok, Weijie Su, and Jason M Altschuler. Shifted interpolation for differential privacy. arXiv preprint ar 2003 00278, 2024.
10

---

Published as a conference paper at ICLR 2026
Mingzhe Chen, Nir Shlezinger, H Vincent Poor, Yonina C Eldar, and Shuguang Cui. Communication- efficient federated learning. Proceedings of the National Academy of Sciences, 118(17): c2024789118, 2021.
Shuzhen Chen, Dongxiao Yu, Yifei Zou, Jiguo Yu, and Xiuzhen Cheng. Decentralized wireless federated learning with differential privacy. IEEE Transactions Industrial Informatics, 18(9): 6273-6282, 2022.
Anda Cheng, Peisong Wang, Xi Sheryl Zhang, and Jian Cheng. Differentially private federated learning with local regularization and sparsification. In Proceedings of the IEÉE/CVF conference on computer vision and pattern recognition, pp. 10122-10131, 2022.
Rishav Chourasia, Jiayuan Ye, and Reza Shokri. Differential privacy dynamics of langevin diffusion and noisy gradient descent. Advances in Neural Information Processing Systems, 34:14771-14781. 2021.
Rong Dai, Li Shen, Fengxiang He, Xinmei Tian, and Dacheng Tao. Dispfl: Towards communication- efficient personalized federated learning via decentralized sparse training. In International confer- on machine learning, pp. 4587-4604. PMLR, 2022.
Rong Dai, Xun Yang, Yan Sun, Li Shen, Xinmei Tian, Meng Wang, and Yongdong Zhang. Fedgamma: Federated learning with global sharpness-aware minimization. IEEE Transactions Neural Networks and Learning Systems, 35(12)11749--1742 2023.
Rudrajit Das, Anish Acharya, Abolfazl Hashemi, Sujay Sanghavi, Inderjit S Dhillon, and Ufuk Topcu. Faster non-convex federated learning via global and local momentum. In Uncertainty in Artificial Intelligence, pp. 496-506 506. PMLR, 2022.
Jinshuo Dong, Aaron Roth, and Weijie Su. Gaussian differential privacy. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 94(1):3-37, 2022.
Cynthia Dwork. Differential privacy. In International colloquium on automata, languages, and programming, pp. 1-12. Springer, 2006.
Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In Advances in Cryptology-EUROCRYYT 2006: 24th Annual International Conference on the Theory and Applications of Cryptographic Techniques, St. Petersburg. Russia, May 28-June 1, 2006. Proceedings 25, pp. 486-503. Springer, 2006a.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings pp. 265-284 Springer, 2006b.
Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3 933-4):211-401 2014.
Cynthia Dwork, Adam Smith, Thomas Steinke, Jonathan Ullman, and Salil Vadhan. Robust trace- ability from trace amounts. In 2015 IEEE 56th Annual Symposium Foundations of Computer Science, pp. 650-669. IEEE, 2015.
Takumi Fukami, Tomoya Murata, Kenta Niwa, and lifan Tyou. Dp-norm: Differential privacy primal-dual algorithm for decentralized federated learning. IEEE Transactions on Information Forensics and Security, 2024.
Yuanyuan Gao, Lei Zhang, Lulu Wang, preserving and reliable decentralized federated learning. IEEE Transactions Services Computing, 16(4):2879-2811, 2023.
Raymond Choo, and Rui Zhang. Privacy-
Jonas Geiping. Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. Inverting gradients-how easy is it to break privacy in federated learning? Advances in neural information processing systems, 33:169377116947 2020.
11

---

Published as a conference paper at ICLR 2026
Robin C Geyer, Tassilo Klein, and Moin Nabi. Differentially private federated learning: A client level perspective. arXiv preprint arXiv:1712.07557, 2017.
Antonious Girgis, Deepesh Data, Suhas Diggavi, Peter Kairouz, and Ananda Theertha Suresh. Shuffled model of differential privacy in federated learning. In International Conference on Artificial Intelligence and Statistics. pp. 2521-2529 PMLR, 2021.
Eduard Gorbunov, Filip Hanzely, and Peter Richtárik. Local sgd: Unified theory and new efficient methods. In International Conference on Artificial Intelligence and Statistics, pp. 3556-3564.
PMLR, R, 2021.
Michal Grudzien, Grigory Malinovsky, and Peter Richtárik. Can 5th generation local training methods support client sampling? yes! In International Conference on Artificial Intelligence and Statistics, pp. 1055-1092. PMLR, 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Zaobo He, Lintao Wang, and Zhipeng Cai. Clustered federated learning with adaptive local differential privacy on heterogeneous iot data. IEEE Internet of Things Journal, 2023.
Rui Hu, Yuanxiong Guo, Hongning Li. Qingqi Pei, and Yanmin Gong. Personalized federated learning with differential privacy. IEEE Internet of Things Journal, 7(10):550--9539 2020.
Rui Hu, Yuanxiong Guo, and Yanmin Gong. Federated learning with sparsified model perturba- tion: Improving accuracy under client-level differential privacy. IEEE Transactions on Mobile Computing, 2023.
Tiansheng Huang, Li Shen, Yan Sun, Weiwei Lin, and Dacheng Tao. Fusion of global and local knowledge for personalized federated learning. arXiv preprint arXiv:2302.1 1051, 2023.
Divyansh Jhunjhunwala, Pranay Sharma, Aushim Nagarkatti, and Gauri Joshi. Fedvarp: Tackling the variance due to partial client participation in federated learning. Uncertainty Artificial Intelligence, pp. 906-99 916. PMLR, 2022.
Sanxiu Jiao, Lecai Cai, Xinjie Wang, Kui Cheng, and Xiang Gao. differential privacy federated learning scheme based on adaptive gaussian noise. OMES-Compute Modeling in Engineering & Sciences, 138(2), 2024.
Peter Kairouz, Sewoong Oh, and Pramod Viswanath. The composition theorem for differential privacy. In International conference on machine learning, pp. 1376-1385, PMLR, 2015.
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Ad- vances and open problems in federated learning. Foundations and trends@ in machine learning, 14(1-2):1-210, 2021.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International conference machine learning, pp.
PMLR, 2020.
Prashant Khanduri, Pranay Sharma, Haibo Yang, Mingyi Hong, Jia Liu, Ketan Rajawat, and Pramod Varshney. Stem: A stochastic two-sided momentum algorithm achieving near-optimal sample and communication complexities for federated learning. Advances in Neural Information Processing Systems, 34:6050-6000 2021.
Muah Kim, Onur Günlü, and Rafael F Schaefer. Federated learning with local differential privacy: Trade-off' between privacy, utility, and communication. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2650-2654. IEEE, 2021.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
12

---

Published as a conference paper at ICLR 2026
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based based learning applied to document recognition. Proceedings of the IEEE, 86(11))2278-25 2324, 1998.
Bo Li, Mikkel N Schmidt, Tommy S Alstróm, and Sebastian U Stich. On the effectiveness of partial variance reduction in federated learning with heterogeneous data. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 9964-3973, 2023a.
Guanghao Li, Li Shen, Yan Sun, Yue Hu, Han Hu, and Dacheng Tao. Subspace based federated unlearning. arXiv preprini arXiv:2302. 12448, 2023b.
Guanghao Li, Wansen Wu, Yan Sun, Shen, Baoyuan Wu, and Dacheng Tao. Visual prompt based personalized federated learning. arXiv preprint arXiv:2303. 086678 2023c.
Tao Lin. Sebastian Urban Stich, Kumar Kshitij Patel, and Martin Jaggi. Don't use large mini- bini-batchess, use local sgd. In Proceedings of the 8th International Conference on Learning Representations, 2019.
Jie Ling, Junchang Zheng, and Jiahui Chen. Efficient federated learning privacy preservation method with heterogeneous differential privacy. Computers & Security, 109:15 2024.
Wei Liu, Li Chen, Yunfei Chen, and Wenyi Zhang. Accelerating federated learning via momentum gradient descent. IEEE Transactions on Parallel and Distributed Systems, 31(8):1754-11 1754-176,, 2020.
Yixing Liu, Yan Sun, Zhengtao Ding, Li Shen, Bo Liu, and Dacheng Tao. Enhance local consistency in federated learning: A multi-step inertial momentum approach. arXiv preprint arXiv:2302.05726, 2023.
Andrew Lowy, Ali Ghafelebashi, and Meisam Razaviyayn. Private non-convex federated learning without a trusted server. In International Conference on Artificial Intelligence and Statistics, pp. 5749-5786. PMLR, 2023.
Grigory Malinovsky, Kai Yi, and Peter Richtárik. Variance reduced proxskip: Algorithm, theory and application to federated learning. Advances in Neural Information Processing Systems, 35: 15176-15189, 2022.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas. Communication-effie learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273-1282 PMLR, 2017.
Matias Mendieta, Taojiannan Yang. Pu Wang. Minwoo Lee, Zhengming Ding, and Chen Chen. Local learning matters: Rethinking data heterogeneity in federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8397-8406, 2022.
Ilya Mironov. Rényi differential privacy. In 2017 IEEE 30th computer security foundations symposium (CSF), pp. 263-275. IEEE, 2017.
Konstantin Mishchenko, Grigory Malinovsky, Sebastian Stich, and Peter Richtárik. Proxskip: Yes! local gradient steps provably lead to communication acceleration! finally! International Conference Machine Learning, pp. 15750-15769, PMLR, 2022.
Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP), pp. 739-753. IEEE, 2019.
Maxence Noble, Aurélien Bellet, and Aymeric Dieuleveut. Differentially private federated learning on heterogeneous data. International Conference on Artificial Intelligence and Statistics, pp. 10110-10145. PMLR., 2022.
Amirhossein Reisizadeh, Aryan Mokhtari. Hamed Hassani, Ali Jadbabaie, and Ramtin Pedarsani. Fedpaq: A communication-efii efffcciinnttitttttttemee federated learning method with periodic averaging and quanti- zation. In International conference on artificial intelligence and statistics, pp. 2021 0021-2031. PMLR, 2020.
13

---

Published as a conference paper at ICLR 2026
Nuria Rodríguez-Bei Barroso, Goran Stipeich, Daniel Jiménez-López, José Antonio Ruiz-Millán Eugenio Martinez-Cámara, Gerardo González- -ecco, M Victoria Luzón, Miguel Angel Veganzones, and Francisco Herrera. Federated learning and differential privacy: Software tools analysis, the sherpa. ai fl framework and methodological guidelines for preserving data privacy. Information Fusion, (4:270-292) 2020.
Lu Shi, Jiangang Shu, Weizhe Zhang, and Yang Liu. Híi-dp: Hierarchical federated learning with differential privacy. In 2021 IEEE Global Communications Conference (GLOBECOM), pp. 1-7. IEEE, 2021.
Yifan Shi, Yingqi Liu, Kang Wei, Li Shen, Xueqian Wang, and Dacheng Tao. Make landscape flatter in differentially private federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 24552-24562, 2023.
Nir Shlezinger, Mingzhe Chen, Yonina C Eldar, Vincent Poor, and Shuguang Cui. Uveqfed: Universal vector quantization for federated learning. IEEE Transactions on Signal Processing, 69; 500-514, 2020.
Sebastian Urban Stich. Local sgd converges fast and communicates little. In ICLR 2019-nnternationl Conference Learning Representations, 2019.
Yan Sun, Li Shen, Shixiang Chen, Liang Ding, and Dacheng Tao. Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape. In International Conference on Machine Learning, pp. 32991-33013 PMLR, 2023a.
Yan Sun, Li Shen, Tiansheng Huang, Liang Ding, and Dacheng Tao. Fedspeed: Larger local interval, less communication round, and higher generalization accuracy. arXiv preprini arXiv:2302. 10429, 2023b.
Yan Sun, Li Shen, Hao Sun, Liang Ding, and Dacheng Tao. Efficient federated learning via local adaptive amended optimizer with linear speedup. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(11):14433-1446 2023c.
Yan Sun, Li Shen, and Dacheng Tao. Which mode is better for federated learning? centralized or decentralized. 2023d.
Yan Sun, Li Shen, and Dacheng Tao. A-fedpd: aligning dual-drift is all federated primal-dual learning
needs. Advances in Neural Information Processing Systems, 37:85742-85777 2024a.
Yan Sun, Li Shen, and Dacheng Tao. Understanding how consistency works in federated learning via stage-wise relaxed initialization. Advances in Neural Information Processing Systems, 36, 2024b.
Yan Sun, Li Shen, and Dacheng Tao. Towards understanding generalization and stability gaps between centralized and decentralized federated learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
Han Wang, Siddartha Marella, and James Anderson. Fedadmm: A federated primal-dual algorithm allowing partial participation. In 2022 IEEE ó1st Conference on Decision and Control (CDC), pp. 287-294. IEEE, 2022.
Yansheng Wang, Yongxin Tong, and Dingyuan Shi. Federated latent dirichlet allocation: A local differential privacy based framework. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 6283 6290, 2020.
Larry Wasserman and Shuheng Zhou. A statistical framework for differential privacy. Journal of the American Statistical Association, 105(489):375-33) 2010.
Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H Yang, Farhad Farokhi, Shi Jin, Tony QS Quek, and H Vincent Poor. Federated learning with differential privacy: Algorithms and performance analysis. IEEE transactions on information forensics and security, 15:3454-3466 2020.
Kang Wei. Jun Li. Ming Ding, Chuan Ma. Hang Su, Bo Zhang, and H Vincent Poor. User-level preneevie federated learning: Analysis and performance optimization. IEEE Transactions Mobile Computing, 21(9):3388-344 3401, 2021.
14

---

Published as a conference paper at ICLR 2026
Kang Wei, Jun Li, Chuan Ma, Ming Ding, Wen Chen, Jun Wu, Meixia Tao, and H Vincent Poor. Personalized federated learning with differential privacy and convergence guarantee. IEEE Trans- actions Information Forensics and Security, 2023.
Thorsten Wittkopp and Alexander Acker. Decentralized federated learning preserves model and data privacy. In International Conference on Service-Oriented Computing, pp. 176-187. Springer, 2020.
Blake Woodworth, Kumar Kshitij Patel, Sebastian Stich, Zhen Dai, Brian Bullins, Brendan Mcmahan, Ohad Shamir, and Nathan Srebro. Is local sgd better than minibatch sgd? In International Conference on Machine Learning, pp. 10334-10344 PMLR, 2020.
Jingfeng Wu, Wenqing Hu, Haoyi Xiong, Jun Huan, Vladimir Braverman, and Zhanxing Zhu. On the noisy gradient descent that generalizes as sgd. In International Conference on Machine Learning, pp. 10367-10376. PMLR, 2020.
Xiang Wu, Yongting Zhang, Minyu Shi, Pei Li, Ruirui Li, and Neal N Xiong. An adaptive federated learning scheme with differential privacy preserving. Future Generation Computer Systems, 127: 362-372, 2022.
Jian Xu, Xinyi Tong, and Shao-I Jun Huang. Personalized federated learning with feature alignment and classifier collaboration. In The Eleventh International Conference on Learning Representations.
Ge Yang, Shaowei Wang, and Haijie Wang. Federated learning with personalized local differential privacy. In 2021 IEEE 6th International Conference on Computer and Communication Systems (ICCCS), pp. 484 489. IEEE, 2021.
Xinyu Yang and Weisan Wu. A federated learning differential privacy algorithm for non-gaussian heterogeneous data. Scientific Reports, 13(1):5819, 2023.
Xiyuan Yang, Wenke Huang, and Mang Ye. Dynamic personalized federated learning with adaptive differential privacy. Advances in Neural Information Processing Systems, 36:72181-72192 2023.
Xin Yao, Tianchi Huang. Chenglei Wu, Ruixiao Zhang, and Lifeng Sun. Towards faster and better federated learning: A feature fusion approach. In 2019 IEEE International Conference on Image Processing (ICIP), pp. 175-179. IEEE, 2019.
Jiayuan Ye and Reza Shokri. Differentially private learning needs hidden state (or much faster convergence). Advances in Neural Information Processing Systems, 35:703-715, 2022.
Lin Zhang, Yong Luo, Yan Bai, Bo Du, and Ling- Yu Duan. Federated learning for non-iid data via unified feature learning and optimization objective alignment. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4420-4428, 2021a.
Meng Zhang, Ermin Wei, and Randall Berry, Faithful edge federated learning: Scalability and privacy. IEEE Journal on Selected Areas in Communications, 39(12):3790-388 2021b.
Xinwei Zhang, Mingyi Hong. Sairaj Dhople, Wotao Yin, and Yang Liu. Fedpd: A federated learning framework with adaptivity to non-iid data. JEEE Transactions on Signal Processing, 69:6055-6070,
2021c.
Yaling Zhang and Dongtai Tang. A differential privacy federated learning framework for accelerating convergence. In 2022 18th International Conference on Computational Intelligence and Security (CIS), pp. 122-126. IEEE, 2022.
Jingwen Zhao, Yunfang Chen, and Wei Zhang. Differential privacy preservation in deep learning: Challenges, opportunities and solutions. IEEE Access, 7:48901-488 1, 2019.
Qinqing Zheng, Shuxiao Chen, Qi Long, and Weijie Su. Federated f-differential privacy. In International conference on artificial intelligence and statistics, pp. 2251-2259, PMLR, 2021.
Linghui Zhu, Xinyi Liu, Yiming Li, Xue Yang, Shu-Tao Xia, and Rongxing Lu. A fine-grained differentially private federated learning against leakage from gradients. IEEE Internet of Things Journal, 1330:1115001111111 2021.
15

---

Published as a conference paper at ICLR 2026
Statement of Using LLMs. Large language models (LLMs) occasionally employed as writing aids in the preparation of this manuscript. Their use was restricted to detecting minor typographical emrors and refining small number of lengthy sentences for improved clarity. Beyond these limited writing-related adjustments, L.MSs played no role in the research design, implementation, or analysis.
Statement of Ethical Concerns. This paper is contributed to theoretical exploration and validation experiments conducted on publicly available datasets and models, and therefore does not involve any ethical concerns.
A NOTATIONS
In the subsequent content, we use italics for scalars and denote the integer set from by [a]. All sequences of variables are represented in subscript, e.g. Wi,KI. For arithmetic operators, unless specifically stated otherwise, the calculations are performed element-wise. Other symbols used in this paper will be explicitly defined when they are first introduced. We introduce a complete description the following Table.
Table 6: Notations descriptions.
T
Communication Round
K
Local Interval
m
Number of Clients
Noise Level
Gradient Clipping
L
Lipschitz Constant
W
Parameters
9
Gradients
n
Noise
"
Learning Rates
Proxy Coefficient
Interpolation Coefficient
Local Dataset
Client Union
M
Training Mechanism
N
Gaussian Distribution
Optimization Objective
Local Training
T(::))
Trade-off Function
To(-)
Gaussian DP Trade-off Function
Φ())
Gaussian CDF
B GENERAL FL-DP FRAMEWORK
FL framework usually allows local clients to train several iterations and then aggregates these optimized local models for global consistency guarantees. Though indirect access to the dataset significantly mitigates the risk of data leakage, vanilla gradients or parameters communicated to the server still bring privacy concerns, i.c. indirect leakage. Thus. DP techniques introduced by adding isotropic noises on local parameters before communication, to further enhance privacy protection.
16

---

Published as a conference paper at ICLR 2026
Algorithm 1 General FL-DP Framework
Input: initial parameters wo, round T. interval K
Output: global parameters Wy
1: for = 2.... '''
do
2: activate local clients and communications
3: for client E Z in parallel do
4: set the initialization Ws.0.t W/
5: for 1: 00, 1, 2, K do
6:
k+1, L-update( (.,,
7: end for
8: generate noise n, (N
9: communicate Wi,K,t + n, to the server
10: end for
11: 1t++1 = G-update((ww.. + nz²))
12: end for
In our analysis, we consider the FL-DP framework with the classical normal client-level noises, as shown in Algorithm 1. At the beginning of each communication round f, the server activates local clients and communicates necessary variables. Then local clients begin the training in parallel. We describe this process as total of K > 1 steps of L-update function updates. Depending on algorithm designs, the specific form of local update functions varies. After training, the local clients enhance local privacy by adding noise perturbations to the uploaded model parameters. Our analysis primarily considers the properties of the isotropic Gaussian noise distribution, i.e. n, ~ NO0 o² Ta). Then the global server aggregates the noisy parameters to generate the global model W++1 via the G-update function. Repeat this for T rounds and return Wy as output.
C MORE EXPERIMENTS VALIDATION
In this section, we present additional experimental validations, including larger client scales and different neural network architectures, to further substantiate our analysis.
Table 8 summarizes additional results under different noise intensities, client scales, and local iteration lengths. Overall, when the noise level very large, the model fails to converge, showing the detrimental effect of excessive perturbation. As the noise weakens, performance improves steadily, confirming the expected trade-off between privacy and accuracy. Increasing the client scale consistently leads to higher accuracy, since more participants help average out the injected noise and stabilize training. In contrast, enlarging the local iteration length tends to slightly degrade performance, especially under stronger noise, as longer updates accumulate errors. These findings further support our theoretical claims: moderate noise is essential for balancing utility and privacy, larger client populations enhance robustness, and overly strong noise inevitably causes learning to fail. These observations highlight several important insights. First, the results demonstrate that privacy-preservin noise must be carefully calibrated: too strong perturbation eliminates useful signal, while moderate levels allow training to proceed effectively. Second, the consistent benefit of larger client scales indicates that federated participation not only improves model generalization but also plays role in mitigating the variance introduced by noise. Finally, the relatively minor but noticeable impact of longer local iterations suggests a delicate balance between communication efficiency and robustness to noise. Together, these findings provide empirical evidence that supports the theoretical analysis the main text, and further confirm the scalability and stability of our proposed approach under diverse settings.
Results on ViT-Small. We conducted tests on ViT-Small, and the results show a similar trend to ResNet- 18, although the sensitivity is higher, the convergence behavior remains largely consistent. We use the m = 50, K = 5, and V = 10. To ensure satisfactory convergence the ViT-Small model, we increased the number of training steps to T 1000.
17

---

Published as a conference paper at ICLR 2026
Table 8: More experiments (hyperparameters are fixed as the same selection with Table. 4).
Noisy Intensity
m
20
m
500
K = 50
K
100
K
200
K = 50
K 100
K 200
1.0
CIFAR-10
a =
21.69 4236
41 41.94
68.42+t..
66.93-041
66.36
ResNet-18
a 10
58,9940.18
55.62 58,59
77.68
76,25_0.99
23.24
a 10
60.23
29.35
56.13
78.14
77.32-0.4
76.07 =0.17
Table 9: Sensitivity of different learning rates LeNet.
T' 100
T 200
T
300
T 400
T
500
T 600
constant
19.77
29.24
33.52
36.37
37.52
37.98
eyclically
15.36
28.44
33.37
36.24
37.82
38.15
stage-wise
15.24
24.33
26.89
27.77
28.69
29.03
iteratively
14.33
22.14
23.35
24.11
24.52
24.68
Table 7: Results ViT-Smai
= T-200
T- 400
T 600
=
1000
ResNet-18
16.94
23.13 31.37
24.04
24.11
24.17
ViT-Small
23.52
35.66
36.32
36.58
Sensitivity of ResNet-18 converges after approximately 600 steps, while that of ViT-Small converges after 800 steps. Table compares the sensitivity curves of ResNet-18 and ViT-Small under different training steps. We observe that the sensitivity of ResNet- 18 stabilizes after roughly 600 iterations, while ViT-Small requires about 800 iterations reach convergence. This indicates that larger and more expressive models generally take longer to stabilize, their higher capacity introduces additional variance the early stage of training. Nevertheless, although the convergence point delayed for ViT-Small, the eventual sensitivity magnitude does not exceed that of 18 which confirms that our theoretical stability upper bound remains unchanged regardless of model size. These results suggest that scaling up the model primarily affects the rate of convergence but not the asymptotic stability guarantee, thereby validating the robustness of our analysis in both CNN- and Transformer-basee architectures.
Our experiments follow the standard settings used in prior classical work. For complex neural networks, using constant learning rate throughout training may lead to instability, which is mainly training issue rather than the focus of this paper, as our primary contribution lies the privacy theory. In this part, we additionally train the model under four different learning-rate schedules and report the results below. As training progresses, the privacy guarantee of the model continues to exhibit strong convergence trend, which is consistent with our theoretical analysis.
Results on CIFAR-10 100 and Tinylmagenet. We follow existing works (Dai et al., 2023; Li et al., 2023c; Huang al., 2023; Liu et al., 2023: Sun al., 2023d) conduct additional noisy experiments on CIFAR- 100 and Imngnetentt and the observed behavior is consistent with the results on CIFAR- 10 and MNIST. The effect of noise strength direct, and prior work has extensively studied this phenomenon, as the introduction of noise inevitably degrades the convergence rate. This degradation stems from the optimization process itself and reflects the fundamental trade-off between privacy and utility: optimization prefers smaller noise, while privacy requires larger noise. Our analysis further improves the lower bound in privacy characterization. Previous analyses often viewed this trade-off inherently conflicting and irreconcilabl.. However, our results reveal more nuanced relationship in the federated learning setting. Specifically, given a desired privacy guarantee, the convergence speed does not decay indefinitely.
18

---

Published as a conference paper at ICLR 2026
Table 10: More experiments (hyperparameters are fixed the same selection with Table. 4).
Noisy Intensity
CIFAR-I00 100
TinyImagenet
K = 50
K
100
K 200
K_
500
K
50
K
100
K
200
K 500
10
35.42
34.18
33.25
33.21
28.52
27.39
25.94
25.65
10 2
38.42
38.11
37.36
37.03
34,99
34.36
34.10
33.84
10
38.95
38.73
38.11
37.84
35.15
34.87
34.64
34.21
Empirical Studies of Membership Inference. We conduct an empirical study using membership inference attacks to validate the stability of the privacy guarantees. We consider federated learning setup with 100 clients, among which are designated as target clients. The attacker is assumed to intercept the transmitted parameters during communication but has knowledge of their source. Using gradient inversion, the attacker attempts to identify the target clients the different training rounds. We report the attack success rate over 500 steps.
Table 11: Results of membership inference.
T 200
400
T
600
T 800
1000
0 1.0
0%
0%
10%
10%
0%
0.1
20%
30%
30%
40%
30%
0 0.01
40%
50%
60%
60%
70%
40%
80%
100%
100%
100%
can be seen that in the absence of noise, the attack success eventually stabilizes 100%, indicating complete lack of privacy protection. Under extremely strong noise, the attack performance nearly equivalent to random guessing, which indicates high level of privacy protection. Under different levels of constant noise, the attack success rate does not approach 100% even in the later stages of training. With noise level of 0.01, the success rate eventually stabilizes around 70%, while stronger noise level of 0.1 reduces it to about 30%. This demonstrates that the privacy protection does not rely on an ever-increasing noise magnitude; instead, the convergence of privacy an inherent outcome.
Different number of data samples. Since the local noise is injected at each client after completing its local training in every round, the amount of local data does not have substantial impact. We follow the previous works (Sun et al., 2025) conducted the following validation on CIFAR-10 by varying both the number of clients and the amount of data per client, and report the resulting sensitivity differences after 500 rounds of training.
Table 12: Sensitivity of different sample numbers and client numbers.
S = 100
S = 200
S = 300
S = 400
S = 500
80
17.41
17.34
17.15
17.22
17.25
m 50
24.95
24.93
24.62
24.83
24.53
20
30.12
30.38
30.45
29.89
30.08
D EMPIRICAL STUDIES OF CONVERGENT PRIVACY
We additionally conducted the following experiments on CIFAR- 10 and -IFAR-100, Specifically, we evalute the privacy performance label reconstruction, gradient inversion attack, attribute inference attack, and knowledge extraction attack to demonstrate that convergent privacy can be empirically satisfied.
Label Reconstruction. Label Reconstruction is privacy attack in which an adversary attempts to infer and reconstruct the true labels of training samples from the model's outputs gradients. We
19

---

Published as a conference paper at ICLR 2026
randomly select 1000 samples from the test set to evaluate the label reconstruction ability, which reflects the change in privacy leakage.
Table 13: Label reconstruction performance on different constant level noise.
Noise level
T
100
T 70200
= 300
T 400
7 500
CIFAR-10
= 0.1
17.7
20.4
22.0
22.5
22.8
CIFAR-10
0.01
32.5
38.3
44.2
46.6
48.2
CIFAR-10
a a=0
36.5
44.7
53.1
59.8
65.4
CIFAR-100
0.1
12.3
15.5
17.5
19.0
19.2
CIFAR-100
0.01
20.3
25.5
28.6
29.7
30.6
CIFAR-100
0=
24.4
30.1
36.5
40.5
43.3
can be observed that without noise, the reconstruction accuracy steadily increases, as the correspon- dence between gradients and labels is highly pronounced in the noise-free setting. After injecting noise with sufficient strength, the recovery accuracy gradually stabilizes at a certain level and no longer improves. This is consistent with our thcoretical analysis.
Gradient Inversion Attack. Gradient inversion attack aims to reconstruct the original input data by exploiting the gradients shared during training. Since
images as the data in this setting, the reconstruction quality is evaluated by the average L2 norm between the recovered images and the original ones. We randomly select 500 samples from the CIFAR-10 for evaluation.
Table 14: Gradient inversion attack performance different constant level noise.
Noise level
T
100
T = 200
T 300
T 400
T T= 500
CIFAR-10
0.1
20.45
18.32
17.17
16.65
16.34
CIFAR-10
0 = 0.01
15.51
12.26
10.35
8.44
8.02
CIFAR-10
0 = 0
10.23
4.55
1.72
1.34
0.97
The reconstruction results show that without noise, the recovered images become highly accurate as training progresses, since the mapping from gradients to images is very direct LeNet. contrast, when noise injected, the reconstruction quality is significantly degraded and eventually stabilizes at a constant magnitude.
Attribute Inference Attack. Attribute inference attack is privacy attack in which adversary attempts to infer hidden or sensitive attributes of training samples from the model's outputs or intermediate representations. Here follow Arevalo et al. (2024) to evaluate the attribute of "Animal or not". We select 1000 data samples to test and report the attribute inference accuracy.
Table 15: Attribute inference attack performance on different constant level noise.
Noise level
T
100
T
200
= 300
T- 400
T' 500
CIFAR-10
== 0.1
20.4
28.1
34.2
38.8
42.5
CIFAR-10
0.01
30.1
46.4
53.3
58.8
62.1
CIFAR-10
σ = 0
30.3
52.2
59.2
67.3
74.4
Attribute inference attacks share certain similarities with label reconstruction attacks, but differ that the inferred attribute is an unlabeled feature that often exhibits clustering behavior during inference. Injecting noise significantly increases the difficulty of effectively identifying this attribute. Our experiments further confirm that the privacy leakage detected by such attacks remains stable under limited noise injection: its growth does not increase indefinitely but instead stays within a gradually shrinking range and eventually converges stable level.
20

---

Published as a conference paper at ICLR 2026
Knowledge Extraction Attack. Knowledge Extraction Attack is privacy attack in which an adversary queries the target model's outputs to train a substitute model that mimics the target model's behavior. In our experiments, train the attacker model using the stolen noisy logits, and evaluate the privacy protection capability by comparing the resulting training loss. can be observed that constant noise effectively maintains stable privacy protection, which is fully consistent with theoretical predictions.
Table 16: Knowledge extraction attack performance on different constant level noise.
Noise level
Vanilla Model
Attacker
CIFAR-10
σ 0.1
74.25
60.20
CIFAR- 10
0.01
78.44
72.43
CIFAR- 10
a
82.55
80.35
CIFAR-100
0.1
32.36
15.52
CIFAR- 100
a 0.01
37.41
26.49
CIFAR- 100
45.25
38.36
E PRELIMINARY PROPERTIES OF f-DP
In our analysis, the f-DP composition is applied to the sequence of perturbed model updates, each of which is Gaussian mechanism. For Gaussian mechanisms, valid coupling is guaranteed to exist: the optimal transport coupling that aligns the two output distributions via a shared Gaussian noise source. Formally, for adjacent datasets C and C', the mechanisms can be written
M(C) Standard FI. Update(C)) Z and M(C²) Standard FL Update(C") Z,
where Standard FL Update() generally defined by the algorithm itself, e.g. FedAvg and FedProx in paper. Z is the shared Gaussian noise. This defines measurable coupling, since the pair (Standard FL. Update(C) + Z, Standard FL Update(C") + 2) is a measurable mapping of Gaussian random variable. Under this coupling, the privacy-loss random variable has the explicit closed form used f-DP analysis, and its mean shift is
[Stanaard FL Update(C") Standard FL Update (C")
2x
which is finite and fully controlled through our sensitivity bounds. Therefore, the total mean shift E, Me is well-defined and directly bounded by model-sensitivity recursion, ensuring that the f-DP composition lemma applies without additional assumptions.
In this section, mainly supplement some basic properties of f-DP, all of which are lemmas proposed by Dong et al. (2022). Specifically, Lemmas and are employed in our theoretical analysis, whereas Lemmas 3 and 4 facilitate lossless translation of our results into other standard DP frameworks for comparative purposes.
Lemma (Post-processing) If randomized mechanism M is J-DP, any post processing mecha- nism based on M is still at leust f-DP, iee. Q() 2 T(P; Q) for any post-processing mapping which leads to P > p' and Q Q'
Intuitively, yost-processing mappings bring some changes in the original distributions. However, such changes not allow the updated distributions to be much easier to discern. This lemma also widely exists in other DP relaxations and stands as one of the foundational elements current privacy analyses. In fDP, this lemma also clearly demonstrates that the difficulty of hypothesis testing problems can not be simplified with the addition of known information, which still preserves the original distinguishability.
Lemma 2 (Composition) We have series of mechanisms M; and joint serial composition mech- anism M. Let each private mechanism Mi(:, U1.... 1i-1) be f--DP for all y1 E Y....., -.i-1 E Y, is fi @ ... DDP
Y--. Then the n-fold composed mechanism M X Yi where & denotes the joint distribution. For instance, if T(P: Q) and = T(P': Q'), then
g T(P Pt: Q")
21

---

Published as a conference paper at ICLR 2026
The composition in the f-DP framework is closed and right. This is also one of the advantages of privacy representation in f-DP. Correspondingly, the advanced composition theorem for (e, 6-DDP not admit the optimal parameters to exactly capture the privacy in the composition process (Dwork et al., 2015). However, the trade-off function has an exact probabilistic interpretation and can precisely measure the composition.
Lemma 3 (GDP- (e, 6)-DP) A μ-GDP mechanism with a trade-off function Te(p) is also (e, De)) for all 2 0 where
(-=+ 2 ) -6' (-
8(c) 1
(20)
Lemma 4 (GDP + RDP) A μ-GDP mechanism with a trade-off function To(u) is also (5. =²² RDP for any Ç >
We state the transition and conversion calculations from f-D (we specifically consider the GDP) to other DP relaxations, e.g. for the (s, 5DD and RDP. These lemmas can effectively compare our theoretical results with existing ones. Our comparison primarily aims to demonstrate that the convergent privacy obtained in our analysis would directly derive bounded privacy budgets in other DP relaxations. Moreover, we will illustrate how the convergent f-DP further addresses conclusions that current FL-DP work cannot cover theoretically, which provides solid support for understanding its reliability of privacy protection.
Lemma (Accumulation in GDP.) For GDP, Tc(mn) ..... TT(mm) Ta(V
Proof. Let μ (Mi E I2 and 12 be the x 2 identity matrix. Then
(M( (2( (()(0,1) (00 (Jn; 1) (12, 1)) T(N(O, N2); T2))- Again we use the invariance of trade-off functions under invertible transformations. N(0, 12) is rotation invariant, so we can rotate N (js, I2) so that the mean (v 14 + 12z 0). ie.,
T(N(0, (2); 12)) (O, N):N( ui + 12, ()) (o, 1)) = To (())
F A DISCUSSION OF TIGHTNESS IN RELAXATION
The appropriateness of our relaxation can be justified through parallels with prior work, especially the analysis in shifted-interpolationn on DP-SGD. The essential challenge arises from the fact that obtaining the worst-case privacy loss requires minimizing the privacy term jointly over X and tu. which is an NP-hard problem. To make the computation feasible, Bok et al. assume that the optimization domain has diameter D, thereby ensuring that the global sensitivity at every to is bounded by this constant. Although this assumption simplifies the optimization, such projection is rarely aligned with practical training dynamics. If to were fixed and known constant, the assumption would indeed apply. However, when to increases with the total number of training steps T. this simplification may no longer hold. Our analysis introduces relaxed upper bound, but comparison with their results indicates that our bound differs only by a constant multiplicative term. As T/to- + co, this discrepancy shrinks rapidly and becomes negligible.
G PROOF OF MAIN THEOREMS
Proof Sketch. Since the standard interpolation technique cannot be directly applied to the federated learning setting, we introduce more sophisticated interpolation construction. In this design, the interpolation sequence is defined at the level of the global model, while its evolution is implicitly governed by the local training updates performed on each client. To quantify the effect of this interpolation on the original optimization trajectory, we conduct separate analyses of model sensitivity and data sensitivity for the local sequences, which allow us to derive upper bounds on how these sensitivities change. Using the global interpolation, characterization of the FL-DP framework, which ultimately leads to a bounded privacy guarantee. In this section, provide the main lemmas of sensitivity studies on the different cases.
then obtain the complete f-DP privacy
22

---

Published as a conference paper at ICLR 2026
G.1 PROOFS THEOREM
We consider the general updates on the adjacent datasets C and C" on round as follows:
W++1 = x(ww)) + 1 Σ
mia
iEI
(21)
w/+1 = ('(w() +
Σ iEz
where wi is the initial state. 7i,t and ni,, are two noises generated from the normal distribution (00,, o² aa). To construct the interpolated sequence, we introduce the concentration coefficients X to provide a convex combination of the updates above, which is,
#++11 >e+1 + \+11) +
Σ R4,b
(22)
fort = to to+ ,T,,,,,.IIIII,1 Furthermore, we set Ar = 1 to öग = ((WT-1)+ 1 Eiez 11 ur, and we add the definition of Wi, = w/, as the interpolation beginning. to determines the length of the interpolation sequence.
Lemma According the expansion of trade-off functions, for the general updates in Eq,( we have the following recurrence relation:
T (We+i: w(42)) 2 T (wi @ To ( - o' ((T)))))
(23)
Proof. Based on the post-pree post-processing and compositions, let z and z' be the corresponding noises above, for any constant X E [0, 1]. we have (subscripts are temporarily omitted):
())((w) (1
2 -
T
2 7 (d' (T({((()) 2;2²)
2 TW; w') T (A o(m)) 2;2²),
where z and z' are two Gaussian noises that can be considered to be sampled from N(0, o² La) (av- erage of m isotropic Gaussian noises). Therefore, the distinguishability between the first term and the second term does not exceed the mean shift of the distribution, which is y X (k(w) o' (w) ||.
By taking w = WW and X >4t1, the proofs are completed.
According to the above lemma, by expanding it from = to T and the factor T T(w,, wh,)) To),), we can prove the formulation in Eq, (10).
G.2 PROOFS OF THEOREM
Lemma 6 provides the general recursive relationship the global states along the communication round t. To obtain the lower bound of the trade-off function, we only need to solve for the gaps [(o(w) - ø' |. It is worth noting that the local update process here involves dual replacement of both the dataset (ó and ø') and the initial state (w and w). Therefore, we measure their maximum discrepancy by assessing their respective distances to the intermediate variable constructed by the cross-items:
(((()) - ('(w) S [(((w) -
-(()))(((m) (((w)] [1'(ww) (
(24)
Data Sensitivity
Model Sensitivity
The first term measures the disparity in training different datasets and the second term measures the gap in training from different initial models. One our contributions to provide their general gaps. In our paper, expand the update function (xx) by considering the multiple local iterations
23

---

Published as a conference paper at ICLR 2026
and federated cross-device settings. By simply setting the local interval to 1 and the number of clients to 1, our results can easily reproduce the original conclusion in (Bok et al., 2024). Furthermore, our comprehensive considerations have led to understanding of the impact of local updates privacy.
((w))) and o' (w:) begin from we- (wx) and d' ((te adopt the data samples e' E C'. We naturally Wik and k, to represent individual states in ((w,) and o' (W²). respectively. To avoid ambiguity, we define the states in of (wr) as Wi When i of i", since s', then wi only differs from Wi, on -" client.
ON THE No ISY FEDAVV METHOО::
Lemma (Data Sensitivity.) The data sensitivity caused by gradient descent steps can be bounded as:
K- 1
2V
((((w)) - d' (ww)))
Tht
(25)
R=0
where ne,t is the learning rate the k-th iteration of t-th communication round. Proof. By directly expanding the update functions & and o' wh, have:
[(ó(w)) 8' (wx))
K-1
=|lw= m
The V (w..b.))))
we
Thet
m
FET
K-1
The Vf Vfi(WaA...
iET k=0
K
2V K
Mh.el fi (we, fi (1r k.ts e'Il
Σ Thtt-
m
m
The last equation adopts E = E' when i + it. This completes the proofs.
Lemma (Model Sensitivity.) The model sensitivity caused by gradient descent steps can be bounded as:
(ww((w)) o' (((x))l = (1 ((K,t))) w| - l.ll.
(26)
where ((Ktt)= TJo, + 1000 rates.
EK- Th.t II;=o (1 TT.eL) is constant related the selection of learning
Proof. We first learn an individual case. On the t-th round, we assume the initial states of two sequences are we and wi Each is performed by the update function ø' for local K steps. For each step, we have:
| 1,el
[ Wi TE T V (( V Vi(((T,,, e')|l
< (1 + L1)00,.) - W.
This implies each gap when 2 can be upper bounded by:
|100,, - - Wi.keell 5 (1 + (.
8i 1.2
II (1 + Li, lwwe - W. ==0
Then we consider the recursive formulation of the stability gaps along the iterations We can directly apply (q,(22) to obtain the relationship for the differences updated from different initial states on the same dataset. By directly expanding the update function o' w, and We we have:
(lo'(w)) d0' b'(wT))l
24

---

Published as a conference paper at ICLR 2026
K
R-
= =|10,
Thet V
(i(A,, - -Wg
T V Vi (i(
m
iEZ ==0
iEI k=0
<
100
will
Ma.t (V (VS( - V - V.
k=0
K
<1100
will
+
Tk R...lli Wi.kel
m
iE
h=
K-1
h-1
2
S 1+
100
MRA II
(1 + mard)
|lus Well
k=1
==0
This completes the proofs.
We have successfully quantified the specific form of the problem as above. By solving for series of reasonable values of the auxiliary variable minimize the above problem, we obtain the tight lower bound on privacy. Before that, let's discuss the learning rate simplify this expression. Both 77(,, 1) and Σ Yher terms are highly related to the selections of learning rates. Typically, this choice determined by the optimization process. Whether it's generalization privacy analysis, both are based on the assumption that the optimization can converge properly. Therefore, we selected several different learning rate designs based various combination methods to complete the subsequent analysis. Due to the unique two-stage learning perspective federated learning, current methods for designing the learning rate generally choose between constant rate rate that decreases with local rounds or iterations. Therefore, we discuss them separately including constant learning rate, cyclically decaying learning rate, stage-wise decaying learning rate, and continuously decaying learning rate. We provide simple comparison in Figure 3.
Communication Rounds
Communication Rounds
Communication Rounds
Communication Rounds
Figure 3: Four general setups of learning rate adopted in the federated learning community. From left to right, they are: Constant learning rates, Cyclically decaying learning rates, Stage-wise decaying learning rate, and Continuously decaying learning rate.
Constant learning rates This is currently the simplest case. We consider the learning rate to always be constant, i.e. lhk = Then we have that the accumulation term =k=0 Ykkt jK. For the (,, t) term, we have:
K-1
1-1
K-1 Σ (1 (L+UL)* = 1 ((1 uL)K ((1+uL)K
(1+ n(,t) Noo Σ
The
j=0
When K selected, both of them can be considered as a constant related K. The choice μ also requires careful consideration. Although it is constant, its selection is typically related to m, K, and 7' based on the optimization process. We will discuss this point the final theorems.
Cyclically decaying learning rates Some works treat this learning process
aggregation process of several local training processes, i.e. each local client learns from better initial state (knowledge learned from other clients). And since the client pool is very large, most clients will exit after obtaining the model they desire. This setting is often used in "cross-device" scenarios (Kairouz independent learning process. In this
al., 2021). Thus, local learning be considered
25

---

Published as a conference paper at ICLR 2026
case, the learning rate is designed to decay in an inversely proportional function to achieve optimal 1 local accuracy, i.c. T7k, = K+11 and is restored to a larger initial value at the start of each round, i.e.
Voo = μ.. Then we have the accumulation term:
In(K (((K1))= k= kook+t
k+1 dk /
K
1
Σ
1 S1+ k+1
K-1
k+1 dl 1dk=++Ing In(K).
(27)
When K is large, this term is dominated by (ln(K)). Based on the fact that K is very large in federated learning, we further approximate this term to cln(K + 1) where es a scaled constant. It is easy to check that there must exist 1 S c < 1.543 for any K 2 1. Thus we have the accumulation term as Ex=0 IJk, enkknKK + 1). For the ny (, 1) term, we have its upper bound:
K-1 (K,t)= μ
K-1
pL
Su+
Σ
exp
j j+1
k
k+1
j j+1
j=0
K
Σ j 1+11 - ] j=0
h-1
K-1
= h
#
exp
1
Σ
exp (cln(k + 1))]
h ++
k ++1
1=1
k=0
K-1
K
(k + 1)oul-1 S
(k + 1)ouL-1
dk = cL
((1 + Kyouk - 1).
k=
k=0
The first inequality adopts 1 + < e" and the last adopts the concavity. Actually, we still can learn its general lower bound by a scaling constant. By adopting a scaling b, we can have + x 2 ehz, which is equal to S 1n(2+1) It is also easy to check 0.693 < b < 1 when 0 < x S 1. Thus we have:
n(Kt)) - +
K-
1
+
uL
2
K
Σ
exp
ubL
k
k +
j j+1
k=1
3=0
k=1
j=
ubb
K-1
Σ
j 3111
1
K-1 Σ
|
lexp (cIn(k 1))
exp
k+1
k +
k=1
k=0
K-1
K-1 (k + 1)oubl-1 dk
Σ
(ks + 1)ouht-11
Keubt.
#
cbt.
The last inequality also adopts concavity. Through this simple scaling, we learn the general bounds for the learning rate function 1) (K, f) as:
1 K cubL < <n(Ktt) S cL ((1+ -1).
(28)
ebL
where 1 S c < 1.543, 0.693 < b < 1 and μ S + (this condition is almost universally satisfied in cannot precisely find the tight bound of this function
current optimization theories). Although 7(K,,) t), we can still treat it as a form based on constants to complete the subsequent analysis, i.c. could be approximated as a larger upper bound 1
I ((1 + Ky - 1). More importantly, we have determined that this learning rate function still diverges as K increases.
Stage- wise decaying learning rates This is one of the most common selections of learning rate in the current federated community, which commonly applied in "cross-silo" scenarios (Kairouz et al., 2021). When the client pool is not very large, clients who participate in the training often aim to establish long-term cooperation to continuously improve their models. Therefore, each client will contribute to the entire training process over long period. From learning perspective, local training is more like exploring the path to a local optimum rather than actually achieving the local optimum. Therefore, each local training will adopt constant learning rate and perform several update steps, i.e. Tht = Mt. At each communication round, the learning rate decays once and continues to the next stage, i.e. ?p- = # Based on the analysis of the constant learning rate, the accumulation term is
26

---

Published as a conference paper at ICLR 2026
k-1 pk -k=0 7h,t" 1+I-
For the n(K,t t) term, we have:
K-1
k-1
1+ mL (41)
n(K())
+
j=0
K
- + (1+
(1+
k
can be seen that the analysis of this function is more challenging because the learning rate function (K,, t) is decided by t. which introduces complexity to the subsequent analysis. We will explain this in detail in the subsequent discussion.
Continuously decaying learning rates This is common selection of learning rate in the federated community, involving dual learning rate decay along both local training and global training. This can almost be applied to all methods to adapt to the final training. including both the cross-silo and cross-device cases. At the same time, its analysis is also more challenging because the learning rate is coupled with communication rounds and local iterations, yielding new upper and lower bounds. Therefore, the accumulation term can be bounded as:
We consider the general case I]k,
H EK+
K
1
[K + K +1 K+KK+
dk In
tK +
k
k=0
tK + +1
tK +1
K-
K-1
1
1
tK + K tK +
dk
In
tK +K+ k=
KK 1K+1
k= tK ++++1
tK +1
Similarly, when K is large enough, this term is dominated by O (In
(tt2)) For simplicity in the to include the term at 0.
subsequent proof, we follow the process above and let it be x In
1+2 8+1
It is also easy to check that > 1 is a constant for any K > 1. And a is also constant. It means we can always select the lower bound as its representation. Therefore, for the learning rate function n (K, t), we have:
K
a
(K,1))
1
tk +
tK
tK +J+1)
1=0
K-1
h-1
#
exp Σ
1
tK +1
tK + k+1
tk + j + 1
1-n
K-1 Σ
p
[KK + (
)
+
exp
a1n
IK+1
IK + k +1
k=1
K
(tK + k + 1)su1-1
(tK + 1) cub
k=0
[KK K + )1) zuL
(IK + k 1)zuk-1
dk
-11
(tK + 1) T
ZT.
IK IK+1
Similarly, we introduce the coefficient b to provide the lower bound as:
T(Ktt) t)
1
K-1
1
nL
1
tK + +j+1
tK + 1
tK +
k
k=1
1=0
abL
K-1
h-1
Σ 1=0
exp
tK +1 1
tk
+ k 11
tk + +j+1 +1
k=1
27

---

Published as a conference paper at ICLR 2026
1
K-
tk [ -
Hob
exp
tK
+1
tK
+ 1
tk +
k=
K-1
H
(tK + k + 1)snbt
(IK
1)=ubL
k=
k-1
(IK + k + 1)smbl-11
dk
1
((* (
tK + +K)
subL
tk
SHbL
(tK
+ 1)=
2hT.
IK +1
1
IK + K
ZaiL
2bL
tk ttK+1 +1
Through the sample scaling, we learn the general bounds for the learning rate function 17( t) as:
tk K
Spal
-1 -1) ((, z7
(+K
tK + K
Sue
-1)-
(29)
zbL
tk +1
tK
where < 2, 0.693 < < and 1 S 1. Obviously, when K is large enough, the learning rate term
is still dominated by O specific form of the learning rate function based on the constant scaling as T.
1+2
zpL
-1) Therefore, to lear the general cases, we can consider the
#pL
-1) As
L+2
t increases, this function will approach zero.
ON THE NOTSY-FEDPREN METHOD:
In this part, we will address the differential privacy analysis of noisy version of another classical federated learning optimization method, i.c. the Noisy-FedP rox method. The vanilla FedP rox method is an optimization algorithm designed for cross-silo federated learning, particularly to address the challenges caused by data heterogeneity across different clients. Unlike traditional federated learning algorithms like FedAvg, which can struggle with variations in data distribution, it introduces proximal term to the objective function. This helps stabilize the training process and improve convergence. Specifically, it adopts the consistency as the penalized term to correct the local objective:
min fi(w) + - [| w- w
(30)
The proximal term is a very common regularization term in federated learning and has been widely used both federated learning and personalized federated learning approaches. It introduces an additional penalty to the local objective, ensuring that local updates optimized towards the local optimal solution while being subject to an extra global constraint, i.c. each local update does not stray too far from the initialization point. In fact, there many optimization methods that apply such regularization terms. For example, various federated primal-dual methods based on the ADMM approach construct local Lagrangian functions, and in personalized federated learning, local privatization regularization terms are introduced to differentiate from the vanilla consistency objective. The analysis of the above methods is fundamentally based on a correct understanding of the advantages and significance of the proximal term in stability eтror. In this paper, to achieve coos-comparison while maintaining generality, we consider the optimization process of local training as total K -tep updates:
K-1 1
((w)) = Wi
Σ VEI k=0
The (V + a (wikkt w.)).
(31)
Here, we also employ the proofs mentioned in the previous section, and our study of the difference term is based on both data sensitivity and model sensitivity perspectives. We provide these two main lemmas as follows.
28

---

method at Published as a conference paper at ICLR 2026
Lemma 9 (Data Sensitivity.) The local data sensitivity of the Noisy-FedProx method t-th communication round can be upper bounded as:
2V
[(((w)) d' d'(w))ll
(32)
ma
Proof. We first consider single step in Eq.(31)
Wi,++11t = w... Th.t ( (((LLLL)))) + a(wike -w)))
The proximal term brings more opportunities to enhance the analysis of local updates. We can split the proximal term and subtract the we term both sides, resulting in recursive formula for the cumulative update term:
Wi,k+1,1 We (1 (11-77,,40 77, (wi,k,t w2) T fe(oE....))).
The above equation indicates that reduction factor 1 can limit the scale of local updates. This is a very good property, allowing us to shift the analysis of the data sensitivity to their relationship of local updates. According the above, we can upper bound the gups between {wi and (o sequences as:
[[ - we) (0i,+I,, - -w+)ll
(1 - (),, [(wikht - Wt) (iii, - W[)] Thtt (Vi( V (i $ (1 m),(aa) || (Wy,,, Wi) (Wig - wt) + TTke V ((W.k.C))))
V (((L...).))))
$ (1 (a, || (W.,k.t - wi) -(Wi,,, - w)) + 2
Different from proofs in Lemma 7, the term T],, can further decrease the stability gap during accumulation. By summing form k = 0 to K 1, we can obtain:
[| (wi,KK -w)) - (ı0, wa)l
K
K
K
< (1- mm) [1.00,00 we) -(00,0, -0))l
+
Σ
II j=++1
(1 - nHaaa)
2nk.
k=
k=
K-1
K K-1
= 2V
(1-1baa) j=
(1
- 1ixaa)
The
k=0
Here, we provide a simple proof using a constant learning rate demonstrate that its upper bound can be independent of K. By considering nh,t J.. we have;
K-1
K-1 II (1- n5aaa j=k-1
K
K
1- - = #
Σ K=O
Ma.t
Σ K=U
(1 ua) j=++1
II
In fact, when the learning rate decays with k. it can still be easily proven to have a constant upper bound. Therefore, in the subsequent proofs, directly use the form this constant upper bound as the result of data sensitivity in the Noi soisy-FedFPl rox method. Based on the definition of ((w), we have:
[lo(wr) - o' ('(wi)l (ō(w:) -we) (o'(w,) - we) =
[W.LK. wi) - -(u w:)]
TET
1
2V
(Wir, wt) (wi K.t we)
ma
This completes the proofs.
Lemma 10 (Model Sensitivity.) The local model sensitivity of the t-th comnunication round can be upper bounded as:
((0((w)) 6 (wwx)) <
| Wall
(33)
al
29

---

Published as a conference paper at ICLR 2026
Proof. We also adopt the splitting above. Since both sequences trained on the same dataset, the gradient difference can be measured by the parameter difference. Therefore, we directly consider the form of the parameter difference:
|00 1,, 1,,
[1(1 1)x, (W,, (i,,.() ne, (V - V.((( =')) n0 1i,1
- (1 a (.,,t - Ws,k.ll + TT, I. - Wi,k,el + I, - -Well
Weseel + 1TS,, Well;
where az - constant. Here, consider L. When a L, its upper bound can not be guaranteed to be reduced. When a > L, it can restore the property of decayed stability. By summinn from K we can obtain:
|ū, - kr
K-
K-1
K-1
n, m,n -
Σ
II
na.aellure -w.l
k=0
k=U
K-1 (1 11
K-1
K-1
II
+
Σ
II (1
(1-7,,aa))
11we 11we- well-
km0
jm=++
Similarly, we learn the upper bound from simple constant learning rate. By select 17st = p. we have:
-
K-1
K
II (1
+
Σ
II (1 m.rrr)))
ma
K=
A=
j=h+1
K-1
K-1
II (1 (az)
II (1 (1-Han maa)) na
k=0
k=
s=k+1
1
(1==aa)k
(1 - pars) K +
02
L(I )
a
a2
0.L
aL
The same, it can also be checked that the general upper bound of the stability gaps is constant even if the learning rate is selected to be decayed along iteration k. Therefore, in the subsequent proofs, we directly use the form of this constant upper bound the result of model sensifivity in the Noisy-FedP rox method. Based on the definition of ((w), we have:
[lo' - ('( (w.))
1
Σ SEI
(w.,K, W.,K..) 11
Wi. Wi. S C
||w; Well
1ET
This completes the proofs.
G.3 SOLUTION OF EQ. (13)
According the recurrence relation in Lemma 6, we can confine the privacy amplification process to finite number of steps with the aid of an interpolation sequence, yielding to the convergent bound. Therefore, we have:
(WT; w/y) = T (WT; why)
vm (rllo(wr-1) - -'(
> W (Wr-1: @ Ta
Vm
2 T (w:: why) 8 ... 1 T To
> ('(wT-1)))
30

---

Published as a conference paper at ICLR 2026
T-1
vm
T (w/ who) &Tc
2 (lll()(()))))))))))))))) - o'(
t=o
Vm
Tc
X2 (Pellec Well + ro)
t=to
Although the above form appears promising, an inappropriate selection of the key parameters will still cause divergence due to the recurrence term coefficient 1 + n(K,t)L 1, leading it to approach infinity as increases. For instance, small ln will result in a significantly increased and the bound will be closed to the stability gap ||wT w/y ll. and large to will result in long accumulation of the stability gaps, which also unsatisfied. At the same time, it is also crucial to choose appropriate to ensure that the stability accumulation can be reasonably diluted. Therefore, we also need to thoroughly investigate how significant the stability gap caused by the interpolation points is. According to (21( and (22), we have:
||02+1 - W S (1-A4+1) (p:lee, W.ll +1))
The above relationship further constrains the stability of the interpolation sequence. worth noting that the upper bound of the final step is independent of the choice of X. At the same time, since all terms are positive, given a group of specific A, taking the upper bound at each possible t will result in the maximum error accumulation. This is also the worst-case privacy we have constructed. Therefore, solving the worst privacy could be considered as solving the following problem:
T-1 Σ
min (Ar11) (1lmn - oel
max
N (pellu: u.ll + 7x))²
(34)
worst privacy
right privacy lower bound
s.t. || w || < (1 Xe+1) (pel) Will +%).
Based on the above analysis, this problem can be directly transformed into a privacy minimization problem when the interpolation sequence reaches the maximum stability error. Therefore, we just need to solve the following problem:
T-1
min Σ (1111),00
Ait1 (pellur ū. + 7
(35)
s.t. [c++ W || = (1 \441) (pill well + 71)
It is important note that this upper bound condition is usually loose because the probability that the interpolation terms simultaneously reach their maximum deviation is very low. This merely the theoretical worst-case privacy scenario.
Then we solve the minimization problem. By considering the worst stability conditions, we can provide the relationship between the gaps and coefficients A as:
|| [ W = pellur well + To det1 (Pellur well + 76)-
Expanding it from fo to we have:
0 = =[[WT= Wrl
II Pt
||
Wioll +
Σ t=
II
Pj [7e
Aeti (pellws W + 7))
Due to the term de+1 (pe ||,, (/w.--Wil] + being part of the analytical form of the minimization objective, we preserve the integrity of this algebraic form and only split it from the perspectives of coefficients de, Pt and According to the definition wh = w/o then we have:
T-1
T-1
T-1
T-1
Σ
II Pj -=++1
Aet1 (pellu, Well + m)) =
II 1=to
|lwe,
+
II /=
PJ
% (36)
t=to
1=to
31

---

Published as a conference paper at ICLR 2026
The above equation presents the summation of the term A4+1 (pille, ū7 + 7) accompanied by a scaling coefficient (IT p3) > 1. It naturally transforms the summation form into an initial stability gap and a constant term achieved through a combination of learning rates, To solve it, we can directly adopt the Cauchy- -Shhwaz inequality to separate the terms and construct a constant term based on the form of the scaling coefficient to find its achievable lower bound:
Σ 1 (Pellece - Will + 7x)²
))
T-1
T
T
Aeti (pelless - Well
+ (pellu--@ell++++ well+ -1))
Σ
Pj
t=f
j=(+1
g=++1
2
II
|| w//,
Σ I=ta
II
Σ
Pt t=fo
Pj
Pj
t=ta
Although the original problem requires solving the Ac+1, here we can know one possible minimum form of the problem no longer includes this parameter. In fact, this parameter has been transformed into the optimality condition of the Cauchy-Schwar inequality.
Therefore, we only need optimize it w.r.t the parameter tu- Unfortunately, this part highly correlates with the stability gaps l/w,, - w4, ||. Current research progress indicates that in non-convex optimization, this term diverges as the number of training rounds increases. This makes difficult for us to accurately quantify its specific impact on the privacy bound. If la is very small, it means that the introduced stability gap will also be very small. However, consequently, the coefficients of the Pt and " terms will increase due to the accumulation over T to rounds. To detail this, we have to make certain compromises. Because to is an integer belonging to [0. T 1]. we denote its optimal selection by t" (it certainly exists when T is given). Therefore, the privacy lower bound under other choices of to will certainly be more relaxed, i.e. Privacy,, to I Privacy, (privacy is weak at other selection of to). This allows us look for other asymptotic solutions instead of finding the optimal solution. Although we cannot ultimately achieve the form of the optimal solution, we can still provide a stable privacy lower bound. To eliminate the impact of stability error, we directly choose to yielding the following bound:
2
2
-1
T-1 II
T-1
T-1
T
H. S Ho
Pt
w/
+
Pj
Yt
Σ
Pj
4==0
j=t+1
T
T-1
T
Pj
1=
j=++1
By substituting the values of Pr. and % under different cases, then we can prove the main theorems in this paper.
32


Reviews:
[{"review_id": "qf5PAIugDk", "reviewer": "Reviewer_2qBX", "summary": "This paper addresses a problem of the arbitrarily divergent privacy bound in FL-DP. The authors provide a convergent privacy analysis for FL-DP by leveraging the f-DP framework combined with a shifted interpolation technique. The analysis focuses on two classical algorithms, Noisy-FedAvg and Noisy-FedProx, under the non-convex and smooth objective settings. The results establish a solid theoretical foundation for the long-term privacy reliability of FL-DP.", "questions": "1. Could the authors provide more insights about how assuming **$t_0 = 0$** affects the tightness of the privacy bound?\n2. Which of the four learning rate strategies analyzed in Theorem 3 was actually used to generate the experimental results?\n3. What are the new techniques introduced in this paper to eliminate the need for the convexity assumption, compared to the literature?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. With the aid of the recently developed shifted interpolation technique for f-DP, this paper proves that privacy in Noisy-FedAvg has a convergent bound, and privacy in Noisy-FedProx has a constant lower bound.\n2. The results for the shifted interpolation technique are extended to the non-convex and smooth objectives, which advance the literature.", "weaknesses": "1. The threat model is not very clear. Does this paper consider the privacy of each client or the overall privacy of all the clients?\n2. The readability could also be improved if the differences between the proposed proof techniques and the existing shifted interpolation technique in Bok et al. (2024) are clearly demonstrated and highlighted. \n3. The authors derive convergent privacy bounds for Noisy-FedAvg under four different LR schedules. However, the experimental section omits any validation or comparison of these strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h8X0LXsvxL", "reviewer": "Reviewer_sKHr", "summary": "In this paper, the authors investigate the privacy of two classic DP-FL methods, namely Noisy-FedAvg and Noisy-FedProx, and their worst-case privacy. For the Noisy-FedAvg method, four typical learning rate decay strategies have been studied, providing the coefficients corresponding to each case for a tighter privacy lower bound. The nondivergence of iterative privacy with respect to the number of communication rounds has been studied for non-convex and smooth objectives. The decay properties of the proximal term in Noisy-FedProx have also been studied, which has the worst privacy. The authors claim to challenge the belief that the privacy budgets of FL-DP must increase as training processes progress and provide reliable guarantees for their ability to protect privacy.", "questions": "1.  The f-DP composition lemma requires control over the total mean shift under a valid coupling; the paper may show the coupling exists and is measurable.\n\n2. If some steps use non-measurable randomness or side information not included in the coupling, can the post-processing inequality be applicable?\n\n3. State a lemma or short proof explaining why per-round mean shifts combine into the squared sum inside the square root (or provide bounds if exact orthogonality does not hold).", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The use of f-DP (a lossless, information-theoretic DP definition) represents a rigorous advancement over traditional additive privacy accounting.\n2. The introduction of shifted interpolation for analysing privacy amplification paths is elegant and well-motivated.\n3. Addresses a long-standing gap", "weaknesses": "1. Proofs need better explanation. The paper is hard to follow.\n2. Only two datasets (MNIST, CIFAR-10) are used. More experimentation is required.\n3. Poor notation use makes the paper even more difficult to follow.\n4. Lack of empirical ε, δ estimates.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "smz6VFCAxa", "reviewer": "Reviewer_13SH", "summary": "Many prior FL-DP studies build on the foundational lemma of privacy amplification by iteration, which typically motivates choosing a certainly small number of communication rounds T (since as $T \\to \\infty$, privacy guarantees can deteriorate). This tension is counterintuitive, so a precise and tight analysis of FL under DP is meaningful. This paper provides a worst-privacy analysis for two classic DP-FL methods—Noisy-FedAvg and Noisy-FedProx—under non-convex, $L$-smooth objectives. As summarized in Table 1, for Noisy-FedAvg (across several learning-rate schedulers) and for Noisy-FedProx (under the condition $\\alpha > L$), the worst-privacy bounds are shown to be convergent even as $T \\to \\infty$.", "questions": "1.\tIn Table 1, explicitly labeling Noisy-FedProx ($\\alpha > L$) may be better. \n\n2.\tMaybe typo at l.357: Regularization Coefficient $\\alpha$", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**(S1) Clear presentation of major contributions (Table 1).**\nThe high-level claims and settings are well organized and easy to follow.\n\n**(S2) Sufficient theoretical contribution.**\nBy using shifted interpolation, the analysis avoids loose privacy amplification and attains tighter bounds. The key sensitivity term is decomposed into model sensitivity + data sensitivity (as in Eq.  (11)), and with an appropriate choice of interpolation coefficients $\\lambda_t$, the paper derives convergent worst-privacy bounds. Concretely, for Noisy-FedAvg the results cover four practical learning-rate schedulers, and for Noisy-FedProx the analysis holds under $\\alpha > L$. In both cases, the worst-privacy bound remains finite-convergent as $T \\to \\infty$.", "weaknesses": "**(W1) Coverage beyond non-convex, L-smooth objectives.**\nTable 1 neatly covers the non-convex, $L$-smooth case. However, are there worst-privacy results for strongly convex or (general) convex objectives? Have techniques akin to those proposed here already been applied to those regimes? Lines 77–80 and Table 3 briefly allude to related settings, but the status remains unclear.\n\n**(W2) Noisy-FedProx when $\\alpha \\le L$.**\nIn Lines. 360-361, “When $\\alpha \\le L$, under a decayed learning rate, its worst privacy bound remains convergent, consistent with Noisy-FedAvg.” Where is this proved (including the appendix)?\nAlso, since Noisy-FedProx reduces to Noisy-FedAvg at $\\alpha=0$, do your results recover the Noisy-FedAvg guarantees in that limit? A short corollary or pointer would help. \n\n**(W3) “Constant-level noise can still achieve convergent privacy.” (l.341).**\nWhat is the quantitative range for “constant-level” here? In Table 4, it appears that $\\sigma=1.0$ may not work as expected.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ibpaZ6jMLY", "reviewer": "Reviewer_WtWo", "summary": "The paper presents a novel federated privacy analysis framework based on f-DP. By applying a drift interpolation technique from a global perspective and introducing a new interpolation sequence that separates data and model sensitivities at the local level, the authors provide the first convergence-privacy analysis for two general FL-DP methods. This contribution offers a theoretically grounded and technically elegant approach to bridging the gap between privacy guarantees and convergence behavior in federated learning.", "questions": "- Does this result also extend to other fundamental methods? In addition, it would be helpful if the authors could provide a theoretical comparison with DP-SGD to better illustrate the advantages of their approach.\n- Can the authors provide results of gradient inversion or membership inference attacks to validate the claimed bounded convergence?\n- In DP-FedProx, how can one determine the optimal choice of the parameter $\\alpha$?\n- What would be the impact if the number of clients is increased but the amount of data per client is reduced?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The overall theoretical development is clear and complete, and the conclusions are sound. The paper provides the first convergent privacy bound under the FL-DP framework and offers an extensive discussion on how various hyperparameters. This analysis represents a valuable contribution to the federated learning community.\n- The comparative study between FedProx and FedAvg presented in this paper is particularly meaningful. By jointly analyzing stability and sensitivity, the authors provide an insightful discussion of how different mainstream optimization methods exhibit distinct privacy behaviors. This analysis can further guide the development of new and more effective privacy-optimized federated learning algorithms.\n- The paper is very well written and provides a thorough review of prior theoretical work. It extends convergence-based privacy guarantees in federated learning from strongly convex objectives to the more general non-convex setting. \n- The authors conduct extensive experiments on standard datasets and tasks, providing solid empirical evidence that strongly supports and validates the theoretical findings.", "weaknesses": "- It would be helpful for the authors to clarify whether the proposed analytical framework can be extended to cover the fundamental DP-SGD approach to better highlight the privacy advantages of federated learning. Including such a discussion would make the paper more comprehensive and strengthen its contribution.\n- It would further strengthen the theoretical contribution if the authors could include some experiments demonstrating DP convergence. Such empirical results would provide more direct evidence supporting the theoretical claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)