Federated Learning of Quantile Inference under Local Differential Privacy
Authors: Leheng Cai
Qirui Hu
Shuyuan Wu
Venue: ICLR 2026 Poster (2026)
Keywords: Confidence interval; Federated learning; Local differential privacy; Quantile; Self-normalization
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We consider the statistical inference of federated learning for quantile loss via local stochastic gradient decent under local differential privacy

Abstract: In this paper, we investigate federated learning for quantile inference under local differential privacy (LDP). We propose an estimator based on local stochastic gradient descent (SGD), whose local gradients are perturbed via a randomized mechanism with global parameters, making the procedure tolerant of communication and storage constraints without compromising statistical efficiency. Although the quantile loss and its corresponding gradient do not satisfy standard smoothness conditions typically assumed in existing literature, we establish asymptotic normality for our estimator as well as a functional central limit theorem. The proposed method accommodates data heterogeneity and allows each server to operate with an individual privacy budget. Furthermore, we construct confidence intervals for the target value through a self‐normalization approach, thereby circumventing the need to estimate additional nuisance parameters. Extensive numerical experiments and real data application validate the theoretical guarantees of the proposed methodology.

Published as a conference paper at ICLR 2026
FEDERATED LEARNING OF QUANTILE INFERENCE UN- DER LOCAL DIFFERENTIAL PRIVACY
Leheng Cai
Qirui Hu
Department of Statistics and Data Science Tsinghua University
School of Statistics and Data Science, Shanghai University of Finance and Economics Institute of Big Data Research,
Beijing 100084, China cailh22 @mails t tsinghua. edu on
Shanghai University of Finance and Economies Shanghai 200443, China
huqirui @mmaimmallll shufe enu.cn on
Shuyuan Wu
School of Statistics and Data Science,
Shanghai University of Finance and Economics Shanghai 200433, China
wushuyuan@mail. wuf..edu.ccmmmm cn
ABSTRACT
In this paper, we investigate federated learning for quantile inference under lo- cal differential privacy (LDP). We propose an estimator based on local stochas- tic gradient descent (SGD), whose local gradients are perturbed via a randomized mechanism with global parameters, making the procedure tolerant of communica- tion and storage constraints without compromising statistical efficiency. Although the quantile loss and its corresponding gradient do not satisfy standard smooth- ness conditions typically assumed in existing literature, establish asymptotic normality for our estimator as well as functional central limit theorem. The pro- posed method accommodates data heterogeneity and allows each server operate with an individual privacy budget. Furthermore, we construct confidence intervals for the target value through nelfiiormalization approach, thereby circumventing the need to estimate additional nuisance parameters. Extensive numerical experi- ments and real data application validate the theoretical guarantees of the proposed methodology.
INTRODUCTION
Modern data ecosystems increasingly require distributional guarantees rather than simple aver- ages. For instance, a national hospital network may track the 0.9 quantile of emergency waiting times across sites to ensure that "nine out of ten patients are seen within T' minutes" (Yadlowsky) et al. 2025), while financial institutions assess tail risk via value-at-risk
expected shortfall un- der strict confidentiality constraints (Barbaglia et al., 2023] Chen 2008, |Wang al. 2012)). In both settings, the target quantile of heterogeneous, possibly heavy-tailed distribution. and the scientific moals-comparing tail performance, checking compliance, or detecting post-intervention shifts-require full inferential tools, not just point estimates.
Quantile methods naturally support these tasks, revealing heterogeneity and tail behavior often in-
visible to means (Angrist et al. 2006 Kallus al. 2202 Chermozhukov & Fernández Vall 2011 Chernozhukov & Hansen] 20055] He et al. 2023 Hu et al. [0022] Chen et al. 2023). Yet the data increasingly distributed: hospitals, banks, and user-facing services each hold their own records, and centralizing raw data is often infeasible due to communication, storage, privacy, and regulatory barriers.
required for such inference
"ooressonndig author
The authors listed alphabetical order with equal contribution

---

Published as a conference paper at ICLR 2026
These challenges motivate federated learning (McMahan al. 2017 Konečný et al. 2016 Liu al. 2020 Tian et al., 2023), where a server coordinates updates from many clients without collecting their raw data. Local SGD can be optimal under i..d. data (Stich, 2018, 2022). but realistic federated environments are heterogeneous
2018: Li al. 2020 Chen al.
distributions, sample sizes, and even target quantiles, complicating optimization and inference and prompting methods based regularization, momentum, and worst-case analysis (Hu et al. 2024 Li al. 2020; 2025 2022).
At the same time, protections limited servers silos are no longer considered sufficient. Breaches of medical and financial records show that server-side DP alone (Dwork et al. 2006) cannot prevent
disclosure when the custodian is compromised. Local differential privacy (LDP) mitigates this risk
by randomizing each individual record before transmission (Duchi et al., 2013); deployed systems
such as Google's RAPPOR, Apple's telemetry, and Windows diagnostics demonstrate its practicality
Erlingsson et al. 2014 Ding al.. 2017
Differentially private federated learning merges these ideas and has gained substantial interest (Agar- wal et al. 201 8 (Shi et al., 2022] Ma et al. 2022, Liu et al. (023a).. As emphasized by Lowy & Razaviyayn (202)), privacy guarantees vary by trust assumptions; LDP corresponds to the most conservative regime in which individuals trust neither server nor silo. Recent work studies LDP in distributed settings for generalized linear models, mean estimation, and related tasks (Zhao et al,, Shen et al. 2023 Jiang et al. 2022). but primarily from an estimation perspective. For quan- tiles, existing LDP methods are either single-machine procedures that ignore client heterogeneity
2020
(Huang et al. 2021
Liu et al. 2023b) or provide point estimators without general inferential guar-
antees.
In contrast, our focus is quantile inference under LDP federated, heterogeneous environment. Concretely, we consider a setting in which many clients, each holding local data drawn from poten- tially different distributions and targeting possibly different quantile levels, collaborate estimate and infer global quantile while ensuring that every message they send is locally private.
We illustrate with simple example. Consider collaboratively estimating the national median annual income using state-level data from the United States, where each state is treated as client. First, income distributions typically vary across states (see Figure Ii). Second, privacy preferences can differ across states due to cultural norms and development levels (Milberg et al. Belllma Bellman et al.) 2004]. Figure Mii) shows how the released information can vary under different privacy budgets. Due to such heterogeneity, direct application of the divide-and-conquer method, which combines local LDP estimators from Liu et al. (2023b) designed for a single client, can yield estimation with significant bias and invalid inference.
NOT REPORTED
105k
106k
25k
8skk-66k
(i) Data heterogeneity
(ii) Privacy heterogeneity
Figure 1: Illustration of client heterogeneity. Income data source: U.S. Census Bureau https: //data. wwwsusenn.c..ee http://p/ww////./////////2
01 01111001000000000001100000000001 0XX000U54000000, Panel (i) shows median annual income by state. Panel (ii) shows three income disclosure schemes un- der different privacy budgets: (a) each individual release true income; (b) each individual release an income interval; and (c) withholding release.
The key question is: Can design federated procedure for quantile estimation under LDP that admits valid confidence intervals and hypothesis tests, while accommodating client heterogeneity and non-smooth loss functions?
2

---

Published as a conference paper at ICLR 2026
Answering this question is challenging for at least three reasons. First, inference requires not only a limiting distribution for the estimator, but also a consistent estimator of its asymptotic variance. For SGD-based procedures, such variance estimators typically involve the Hessian of smooth loss (Chen et al. 2020 yet quantile losses are non-smooth and heterogeneous across clients. Second, the IDP regime, only privatized gradients are observed, naive variance estimation may sume additional privacy budget or demand data-splitting. Third, federated quantile algorithms must remain robust to heterogeneous loss functions and client-level privacy parameters, which further complicates both optimization and asymptotic analysis.
In this paper, our contributions can be summarized as follows:
1. We propose novel federated learning algorithm for quantile inference under LDP. Our method accommodates client-level heterogeneity in quantile targets, privacy budgets, and data distributions, thereby enhancing the applicability of quantile inference in realistic fed- erated environments.
2. We first design an LDP mechanism that effectively reduces the federated quantile estima- tion problem to equivalent non-private setting. Exploiting this reduction, establish the estimator's 's asymptotic normality and derive a functional central limit theorem without average-smoothnes: condition on the loss function. To the best of our knowledge, this con- stitutes the first weak-conv convergence result for local SGD when the loss does not satisfy the usual average-smootiee smoothness condition (Li et al., 2022 Xie al.) 2024 Zhu et al., 2024).
3. Building on these non-privare non-private asymptotic results, we develop an LDP inference procedure for federated quantile estimation. By employing self nofff-oomlizationn technique, we avoid direct estimation of the asymptotic variance, instead constructing confidence intervals that automatically eliminate the unknown variance term. These procedures can be also con- ducted in non-DP case.
The remainder of the paper organized as follows. Section introduces the formal problem setup, background and notation, and presents our LDP mechanism and federated quantile estimator, Sec- tion 3 develops the asymptotic theory, including weak convergence and self-normalized inference. Section 4 reports numerical experiments and real-data application illustrating the practical per- formance and robustness our method. Notations can refer Table S.. Appendix A..I All technical proofs and additional simulation results are deferred to the Appendices Baad
METHODOLOGIES
First, recall the definitions of central and local differential privacy. We then describe our problem setting and algorithmic details.
Definition (Central Differential Privacy, CDP (Dwork et al. 2006)). A randomized algorithm A operating dataset S is (c, diffferrreyyll private if. for any pair of datasets S and S' differing in single record and for any measurable set E,
Pr(A(S) E E) S E)( +6.
e
When A is called e-DP.
Definition (Local Differential Privacy. LDP (Joseph al. 2019)) family of randomized map- pings R X + Y is (c, -locally differentially private for every pair of inputs x' E X and every measurable subset E Y,
Pr(B(z) E E) se Pr(B(z') +
Under CDP, trusted curator collects data and adds noise before release, simplifying algorithm design and typically incurring only an O(1nn) loss in accuracy (Cai et al. 2021). so the privatized estimator ØCDP often satisfies 0CDP D Op(n~ shares the same v si-scaled asymptotic law as the non non-private estimator, and permits recovery of its asymptotic variance at modest additional privacy cost. In contrast, under the non-adaptive LDP model (Cheu et al. 2019) Definitions 2.3 and 2.6), each user with private value X; applies a fixed randomized mechanism R, satisfying (c, 5DP and reports only the perturbed output, so inference must rely solely locally privatized
3

---

Published as a conference paper at ICLR 2026
data; the typical error rate deteriorates OLDP noUDP Op(n (/12), which alters the limiting distribution, inflates the asymptotic variance, and generally makes consistent variance estimation from data collected solely for point estimation infeasible.
We consider a federated learning framework involving K clients, each independently holding a local dataset i.i.d. drawn from unknown distribution Pk with cumulative distribution function (CDF) Fk and density function f& (Li et al.] 2022). For Ps: weight assigned to client k. global quantile level (0, 1), the goal is collaboratively estimate the global quantile Q", which satisfies that Σ (QT) = т. Our objective Q can be obtained by solving the following optimization problem associated with the global loss, which is defined as weighted sum of the local losses:
K
Q' arg min ((Q) QEB
arg min
Σ PkCre (Q)
def arg min > QEB
ps PrE ( (IX. Q)}. (2.1)
QEB
Here, the function E, (x, Q) represents the check loss function defined as:
(x, = (x Q) (Te - I(x < Q)),
(2.2)
where I(.) the indicator function. Besides, J& is the sample generated from Pk. and T E (0, 1) is the local quantile level satisfying ビ PI: = Therefore, our framework also covers federated quantile loss optimization problems in which each local loss corresponds to level Tke and the global objective level т aggregates these local target levels through Σ Рą Note that we require only the knowledge of т, rather than each individual Tk. In the following. denote (Q*))) Qk. and considers the parameter space è is bounded; see geeGuu & Chen (202)).
As noted in the introduction, improve communication efficiency we consider local- SGD--GGGG estimator. Let I = fto ti...., tr for to < t < ... < ty denote the ordered set of communication iterations. At each E I, the global server receives the local updates and broadcasts the aggregated value to all K clients; otherwise, the updates are performed locally on each client. Specifically, for k .......... K,
gt {I(x gi) - T E 1 E Pm lat" mh [[xT] < -
t I,
LE
Here 77, is the predetermined learning rate, and x is observation from the local dataset held by client & at iteration 6. The final estimator is of Polyak-Ruppert type, obtained by averaging the aggregated historical iterates:
@T T ΣΣ>
The communication and statistical efficiency are determined by the interval length Em tm tm-1 for m E N+ If E,,. 1, the local clients must communicate with the global server at every iteration. In this scenario, the approach reduces to parallel SGID, which, noted by Li et al.) (2022). may achieve the Cramér-Rao lower bound and thus serve as an efficient estimator for certain smooth loss functions. Conversely, if communication is performed only once the final iteration (i.c., [Z] 1), then the estimator degenerates to a divide-and-conquee (DC) estimator. In this case, minimizing the loss function 2.1) becomes distributed learning problem. However, as pointed out by Gu & Chen 2023). the divide-and-comee and-conquer estimator may still be statistically inefficient for certain weight choices. Therefore, careful balance must be struck between communication and statistical efficiency. For a general positive interval Fm, > O, the local SGD method allows us to find an appropriate choice of F.,. that can ensure an optimal trade-off between these efficiencies.
On the other hand, the data collected from each client may be subject to privacy protection policies, particularly in surveys involving sensitive information such as income or health status. For the local quantile loss function (2.2 we observe that the structure of its gradient resembles binary response. This motivates us to incorporate an LDP mechanism based on randomized response and permutation, following the framework of Liu et al. (2023b), with truthful response rate r E (0, 1). Specifically, the mechanism allows each local client to either return a true gradient with probability rkk synthetic Bemoulli random variable with probability 1 1-rg- This iterative mechanism ensures
4

---

Published as a conference paper at ICLR 2026
CR -P,, where the privacy parameter is given by <; =ogg(++r)) -logg-1))) established in Liu
jet al. (2023b). Combined with local SGD, the complete procedure is summarized Algorithm
and we denote the resulting estimator as Qr. By direct composition properties, Algorithm enjoys
( cky 0)-LDP guarantees.
Algorithm 1: Federated quantile estimation with local SGD under LDP
Input: step sizes {mmm I target quantile т E (0, 1), truthful response rates {rw (r*)-=1> communication set I = (to, f1....,TT] ).).
Initialization: set 9 = 90 ~O, 1) for all - K, let Qo t for m to do for k = 1 to K (distributedly) do for mm-1 + to tm do
Local updates
of Bernoulli(0.3) = Mx xh-1) (T- = 1) + ot Tuu = =0)
Bernoulli(rg))
1- Tk + 2TTK
1 1 2TT
o ai +
7m 11s} =
11(s) 0)
Mm
2T
2r
end for
end for
Ti
Ek=1 Photi g.m + am for all < ks S K.
b Aggregation and synchronization.
Qm -i(m 1)Qm qt am 1/m.
end for
Return: Qr-
In Algorithm [ each iteration integrates global information (the global quantile 7) with client- specific data zi and correpsonding privacy budget (rk), thereby correcting bias arising from the aggregation of heterogeneous local LDP mechanisms and loss functions.
Theorem 2.1. Denote Rx = TET + +1-r+)) For privacy budget e = log(1 1 log(1 TR), there exists a datasel consisting of i..dd samples drawn from some distribution Pk: S S K, such that solving the federated loss (2.1) with LD-LPP using data drawn from Pk. for client k. is equivalent to solving the following non-private problem:
K Pk
Q* arg argmin min ((Q) arg min Z(Q) def arg min
ax P,, flix (Fk: (Fx:1)}-
(2.3)
Tk
By Theorem 2.1 federated quantile estimation via 2.1 with LDP data drawn from {Px k=1 can be reformulated federated quantile estimation with non-DP data drawn from modified distributions (P&) and shifted quantile levels, in (2.3) Consequently, the target value Q* can be treated as the minimizer of Z(Q), a non-DP objective. The main challenge reduces to analyzing the statis- tical properties of the resulting non-DP estimator (under the ideal data from (Px K particularly the presence of the non-smooth quantile loss function. Once such properties are established, the LDP inference theory be translated directly from the relation between ((Pa) ((Pa)K 1
and
ASYMPTOTIC ANALYSIS
In this section, we focus the asymptotic analysis of the proposed LDP estimator and the prac- tical construction confidence intervals. Due to the space limit, the complete assumptions and corresponding comments provided in Appendix A.2
Theorem 3.1. Under Assumptions enjoys
5.7 S.3
as T
DR, the proposed LDP federated estimator
Ex=
Pk f (2Qu - 1)2]
0, vtr(Qr - QQ) $ 0,0
K I px
5

---

Published as a conference paper at ICLR 2026
Theorem 3.1 establishes the asymptotic normality of the estimator Qr. It follows that the conver- gence rate of Qr is of order (mini Fh tr-)1//2 Recall that the proposed algorithm satisfies (maxiga<K (1r(())))(()-T))) 0)-LDP. which implies that the overall rate is determined by the client with the largest privacy budget. Smaller values of Fa correspond to stronger privacy protection but inevitably yield poorer estimation accuracy. If there exists any and the estimator becomes inconsistent. In contrast, all rk 1, the result recovers the classical non-DP asymptotic normality. This highlights the fundamental privacy-accurea trade-off inherent the proposed estimator.
0, the variance diverges
Theorem 3.1 allows for the theoretical construction of a confidence interval for Q*. However, the construction involves unknown quantities, such as the individual quantiles Qu and the density val- ues (Q"). Even in cases where Qs is known, the estimation of fx(Q" remains challenging. In particular, it is difficult to recover these density values using only the perturbed gradients available from Algorithm [ Moreover, in SGD-SGGDae based methods, consistent variance estimation typically relies on the Hessian matrix, which is well-defined only for smooth loss functions, as previously dis- cussed. Therefore, although Theorem 3.1 provides theoretically valid basis for confidence interval construction, it is not practically implementable due to these limitations.
To address this issue, establish functional central limit theorem (FCLT) in Theorem 3.2 which serves as the foundation for constructing pivotal statistics via self-normalization (Liu et al. 2023b). This allows us to conduct inference without estimating nuisance parameters, thereby providing direct motivation for strengthening the asymptotic result.
Theorem 3.2. Under Assumptions
S.3
S.7 S.3 as T + DC, we have
@r(s) = vtr T
M(aTT)
QQ") 4 E == P { (2Qx - 1)2} (
B(s),
m=1
where tr = =Em=b T- Exm, Imm
E Prok... B()) is standard Brownian motion on [0, 1]. and
h(s,T) max
Z>o
Σ
T
1
1
for s E (0. 1].
Em
Em
Theorem 3.2 establishes a FCLT for Qr(s) over & E (0, 1], showing that converges weakly the
(0, (the space of bounded real-valued functions) to Brownian motion, which is our another major theoretical contribution. Note that the sample quantile loss doesn't satisfy the common L- average smooth conditions for weakly convergence result, such in (Li al. 2022 Xie et al. [0224] Zhu et al. 2024). leading to extra challenge in deriving the almost sure and L convergence rates of It... which are essential for handling the asymptotically negligible terms. Theorem 3.1 arises
special case of Theorem 3.2 when s = 1. Building on Theorem 3.2 we proceed to construct a self-
normalized test statistic and derive its asymptotic pivotal distribution via the continuous mapping
theorem.
Define ro and, for m 2 1, I'm EE 1/E.) (Et 1/E.)"
which ensures that
Vtr (q. Q*), and in particular, 2r(1)
@r(mm)
Σ (a. Q*).
T
T
Following the arguments in (Shao 2015), once a functional central limit theorem such as Theo- rem 3.2 is established, one can construct a self-normalized statistic that asymptotically enjoys a pivotal distribution. Specifically, define
VT Σ (rm rm-1) (r(mm) T er(1)3²
(3.1)
Corollary 3.1. Suppose Assumptions S.I S.3 hold and g(rm) x m//T for some continuous function g [0, 1). Then, T 20,
1r(1) d VVT
B(1)
So (B(r) - - dr

---

Published as a conference paper at ICLR 2026
presents the asymptotic distribution of the celf-normalized statistic Qr(1)/Vr, which distribution free. As a result, there is no need allocate additional DP budget to estimate nuisance parameters when constructing confidence intervals.
Corollary 3.1
The selection the self-normalizer is not unique, and an appropriate norm of the Gaussian pro- cess (r) (() yield similar results to those in Corollary 3.1 For example, using the supremum norm and the L1 norm, one can define alternative self-normalizers as follows:
V
sup ISMST
@r(mm) @r(1),
V
Σ m=1
(Tm Tm- @r(mm)) (r(1))
which related to the processes suPosrs1 [B(r) g B(1) and Jo B(r) ((r) dr,
spectively. However, the self-normalizer defined in equation (3.1) enjoys greater computational efficiency, as the L2 norm can be computed an online manner, as described in Algorithm 2 Let
Fr denote the estimator of the self-normalizer (3.1 and let va/2.g be the (1 - a/2) quantile of
random variable B(1)/ J6' (B(r) - (1)) dr 1/2 The following corollary ensures the asymptotic validity of the constructed LDP confidence interval.
The following corollary ensures the asymptotic validity of the constructed LDP confidence interval. Corollary 3.2. Suppose the same conditions in Theorem 3.2 hold, as T + oc, one has that
- Up Vo
Vo
Vy sas s θr + v + +1-6 -a
Algorithm 2: Online Inference
Input: step sizes {m 1T m= target quantile E (0, 1), truthful response rates {maJkm1>
communication set I = (to, .1....,77 lT).
Initialization: set gb ~ (0, 1) for all k. let Vi 0. V + O. V/ < O, Vo < and Qu 0.
for m to do
Obtain Qm from Algorithm
1
Va f Vm + /nnn Em-
2bb + Vth +m² m²2 m Em.
Em = tm tm-1
Vs + V
D m m-1 11/Em
t- Dp m-1 + m Em--
m
m=Ve (v% 2V Q + 1 ?
Online inference.
end for
- /i,, Qr V
Return: Confidence interval
EXPERIMENTS
4.1 SIMULATION SETUP
We first evaluate our proposed method through extensive simulation studies using synthetic data. In all experiments, we fixed ps = 1/K for < k < K, the number of clients is fixed at K = 10. The quantile levels examined range from 0.3 0.8, and the truthful response rates vary between 0.25 and 0.9. We focus on the following four scenarios of heterogeneity:
heterogeneous quantile levels: We investigate two distinct scenarios: (1) Case Tow lower quan- tile levels, where each client assigned unique quantile level тı ranging uniformly from 0.3 to 0.5; and (2) Case Thigh: higher quantile levels, where ranges uniformly from 0.5 to 0.8.
heterogeneous response rates. Each client has unique truthful response rate Tg. ranging uni- formly from 0.25 0.9.
heterogeneous locations (Hete L). Data for each client & are independently generated from N(k,, 1) C(uR, 1), where P4 (00 1).
7

---

Published as a conference paper at ICLR 2026
heterogeneous distribution families (Hete D). Data are generated independently across ten clients, with three drawing from. (O 1), three from the uniform distribution 21( ((11, 1), and four from a standard Cauchy distribution C(O, 1).
We set the step size I' as: "Im = 20 + 100), with T'm = Em and F = K- - Tk- Following Li et al. (2022),
implement warm-up phase, setting the communication interval Em = 1 for the first 5% of iterations. After the warm-up period, we redefine the interval sequence {Em} based on a new sequence m. specifically: Em mm005-577 We examine three different
interval strategies for F (1) C1: Fmm = 1 (equivalent parallel SGD). (2) C5: F 5, and (3) Log: F = [log2 (m + 1)]- The initial parameter estimates are set to % = qo ~ (0, 1) for all clients k. All experimental settings are replicated R 1, 000 times. The simulations are conducted on computational resources comprising 36 Intel Xeon Gold 6271 CPUs, with a total of 128GB RAM and 500GB storage.
4.2 SIMULATION RESULTS
We first illustrate the performance of our proposed method by presenting sample iteration trajec- tories for estimation and inference. Specifically, we randomly select one simulation run and plot the resulting estimates and corresponding confidence intervals against ty (Figure [S.Ip. The results demonstrate that our approach accurately captures the true quantile value and provides reliable in- ference. Subsequently,
fix ly at 10, 000 and 50, 000 and evaluate the finite sample performance under different settings. Let Qf/" denote the quantile estimator and CI represent the correspond- ing 95% confidence interval obtained from Algorithm 2 in the r-th simulation. We consider two R
metries: the mean absolute error (MAE), defined as R " QQ/² Q" and the empirical cov- erage probability (ECP), defined as R-1 Σ T(Q" cT ). For comparison, we also consider two alternative methods: (1) the SGD with DP updates (Song et al. 2013) (DP-SGD), which adds noise directly to the gradients instead of introducing DP through randomized response. To align with the original paper's setup, we focus on the case with C' = 1. In this regime, the gradient-descon update in Algorithm| I becomes
df + + + 1h-1 (Tull(e I
<i-1) (1-
Th- Ma < ai-1) + z
where Z is drawn from Laplace distribution. A simple calculation shows that Z has mean zero and scale parameter 1/ log {(1 + r1)/(1 - -ra)]- (2) the divide-and-conque (DC) method, which corresponds to the special case F... Lr. Here we use step size 1): = 27 277(11051 + 100) (Goyal al., 2017]. (3) the single-machine LDP quantile inference method (Single) (Liu et al.] 2023b), where all data hold on a single device and LDP-SGD is performed without any federated communication. The numerical results for all methods under the normal distribution setting are reported in Tables and 2 while the corresponding results under the Cauchy distribution setting are summarized in Tables S and
From the results, we observe that our method consistently achieves ECP close to or exceeding the nominal 95% level across all scenarios, and that the C1 strategy (parallel SGD) performs essen- tially comparably to the Single haseline. As either the total sample size by or the truthful response rate increases, the MAE decreases, which aligns with our theoretical results. Comparing the three interval strategies, we find that the Cl strategy yields the smallest MAE, as it has the highest com- munication frequency. Comparing with the two competing methods, we find that the DC approach results in the largest errors. Notably, in certain heterogeneous cases, such as Hete L with r 0.8, the DC estimator exhibits significant bias and an ECP far below the nominal 95% level. In contrast, our proposed estimators successfully achieve approximately 95% empirical coverage in these cases. Moreover, while DP-SGD attains empirical coverage probabilities close to or even exceeding 95% in most settings, its MAE remain uniformly larger than those of our method.
To further illustrate the efffciiecy-accuracy trade-off of our method, we conduct a quantitative anal- ysis in which we record the wall-clock time; the results are shown in Figure S.2 in Appendix C. We also consider scenarios with a fixed number of communication rounds T, with the corresponding results summarized in Tables S.4 and S.5 in Appendix C.I We observe that our proposed method continues to provide valid inference. Additionally, under fixed communication rounds, the Log strategy generally achieves the best performance, yielding the smallest MAE.
8

---

Published as a conference paper at ICLR 2026
Quantile (F)
Rate (r)
C1
C5
Log
DP:SGD (CI)
DC
Single
lltte
10000
0.3
0.25
0.958(0.0184)
0.981(0.0511)
0.990x0.0452)
0.942(0.0260)
0.965(0.3066)
0.950X0.0222)
0.3
betero
0.949r0.0096)
0.982(0.0150)
0.94710.0142) 0.903)
0.89800.1300
1995400.0995
0.3
0.9
1,0000 0029)
1,0000, (066)
1.0000.0100)
0.98
0,21500, (273)
0.962/0 0055)
0.5
0.25
0.9500.01665
0.9846000315)
0.068)
0.95310. 0224)
1.00000
0.95000.0199
0.5
hetero
0,952(0.0085
(1115))1155)
0.998/0.0221)
0,5550000
19)
(.00000025
0.952/0 0083)
0.5
0.9
0.996(0.0025)
0.999(0.0078))
1.0000.01200
(98440000441)
0.000000100)
0.952/0.0051 I)
0.8
0.25
0.950005122)
0.995)
03320000338)
D. 6152)
0.972))00277)
0.8
hetero
0 0.962(0.0112) 22)
0.995(0.0227)
0.996))0347)
0.943/0.0186)
D. 0.709(0.2684
0,958(0.01 14)
0.8
0.9
0.9900.0042)
1.0000.0116)
1.0050.0185)
0.96810.0055)
0.049(020098)
0.966(0.0067)
Herel
500100
0.3
0.25
0.937(0.0089)
0.981(001111)
0.95))001165)
0.916(00115)
0.949(0.1328)
0.948/0.0095)
0.3
hotero
0.981(0.0056 0.911(0.0056)
0.907)0.0080)
0.885(0.00833
0.09300.1283
0.958/0.0033
0.3
0.9
0.977(0.0034)
1,0000000000 0019)
1.0000.008 0030)
00041)
0.00000 1290)
0.958(0.0024)
0.5
0.25
0.958(0.0066)
0. 0.988/0 (098))
0.9957).0147)
(.94990.0099)
1 1.0000.0609)
0,94000.0088
0.5
hetero
0,964(0 0035)
0.99400 0048)
0.996000000
0,95710 0052)
0.997(0.0445)
0.95800.0034
0.5
0.9
10000.0010)
..000.0010
1.0000000601
0.993(0.0018)
0.9790.0143)
0,95000.0022)
0.8
0.25
0.991(.00144) 0.956(0.0102)
0.9360000266
(93110.01000
0.7990.2823 (8299
0.954(0.011)
0.8
hetero
0.992/0.0072) 0.95000.0055)
0.99700.0122 (2)
0.923(0.0099)
0.014(0.2034
0.952(0.0046)
0.8
0.9
1. 1.00X0.00133
1.0000.0053)
0.999)
0006)
0.9000.1929)
0.950X0.0027)
Hete D
by = 10000
0.5
0.25
0.949(0.0132)
0.985(.02243)
0.986(00035)
0.953(00018)
0.93600.01339 0.904().2466
0.5
hetero
0.966(0.0069)
0.991X0.0117)
0.989(0.0172)
0.955(0.0099)
D. 0.9990.0488)
0.958(0.0057)
0.5
0.9
1,003000002)
1.00X0.0074)
1.0000.0117)
0.99110.0035)
1 1.0000000633
0,946(0.0037)
Hete
by 500000
0,5
0,25
0.958(0.005))
0.98000.0082
0.99340.1227)
0.98110.0589 0,94310.0081)
0,94000.0055
0.5
hetero
0.966(0.0033)
0.988/0 0046)
0.998(0.0073)
0.95010.0044 00400
I 1.0000.0111)
0,95800.0025
0.5
0.9
(.909().0008)
..000.0022)
1.0000.0052)
0.990(0.0014)
I 1.0000.0037)
0.950X0.0016)
Table 1: Empirical coverage probabilities at the 95% nominal level (mean absolute errors 4) under heterogencous distributions for different tr. The number of clients K is fixed at 10. In Hete L, data for each client & are independently generated from N (Ux, 1), where μl ~ ~V 1). In Hete D, data are generated from (0, 1), 21( ((11, 1), and C(O, 1) across different clients. "hetero" indicates client-specific truthful response rates ryK range uniformly from [0.25, 0.9]
Quantile (r)
Rate (r)
C1
CS
Log
DP-SGD (CI)
DC
Single
by 10000
0.5
0,25
0.949(0.0133)
(.244))
0.992/0.0360)
0.939(0.2503) 0.949(0.0191)
0.934x0.0141)
0.5
hetero
0. 0.989(001112) 0.963(00007))
0.997(.016)
0.955(0.0100)
1. 1.00000.0997)
0.956(0.0059)
0.5
0.9
0.995(0.0023)
1.0000.00544
1.000(00002)
1.88000.0066
1 1.0010001588
0.950X0.0039)
Tow
0.25
0.947(0.0136)
0.922000253))
0.990(0.0369
0.940(0.0200)
D. 0.969/0.2116
0,950(0.0141)
Too
hetero
0.962(0.0072)
0.993(0.0113)
0,997(0.0162)
0,949r0.0100 0105)
0.9990005300
0.946(0.0063)
Towr
0.9
0.909(000020)
1.0000.0055)
1.000X0.0083)
0.98500.0uS8 0036)
1 1.0000.0062)
0,942(0.00H0)
Thigh
0,25
0.939(000145)
0 (1967000268)
0.95210.02 0,9860003999
0.9840.2771)
0.930X0.0152)
Thigh
hetero
0.968(0.0076)
0.987(0.0126)
0.956(0.0111) 0.999(0.0182)
1 (0016.0516)
0.972(0.0061)
Thigh
0.9
0.996(0.0023)
1.09000006)
1.0000.0102)
0.980/0.0038)
..000.0172)
0.962(0.0339)
50000
0,5
0,25
0.95((0.0056
0.982/0.00 008 1)
0,996
0.949y0.000 (0000.081)
0. 0.988000571)
0.952(0.0055
0.5
hetero
0.966(000032)
0.97900 0044)
0.992(0.0066)
0.950/0.0008 0046)
0,96000.0025 0.000.01155)
0.5
0.9
0. 0.988(0.0027) 1.00X0.0018)
0.99000.00366
(.983)0,0021)
I 1.0000.0038)
0.95000.0016)
Tow
0.25
(.957().006))
0.981 0083)
0.994(0.0125)
0.944:0.009) 1)
0.9930.0594)
0.944(0.0059)
Tow
hetero
0.953(0.0036)
0. (0.014)
O. 0.990X0.0066)
(934(0.0054)
D. 0.9911).0121)
O. 0.962(0.0026)
Tow
0.9
000000.0019)
(026)
0.989)0.0038)
0.98810
0057)
0.95%(000117)
Thigh
0.25
0 0.958(0.0059)
0. 0.986000000
0.997(0.0133)
0.94600.0088 0089)
D. 0.9090000220
0.964K0.0061
Thigh
betero
0.99))0.0045) 0.953(0.0032)
0.995/0.0065)
0.993(0.0154) 0.952(0.014))
0.948(0.0028)
Thigh
0.9
0.998(0.0010)
0.909)
1.0000.00344)
0.977(0.0016)
0.9380000122)
0.958(0.0017)
Table 2: Empirical coverage probabilities at the 95% nominal level (mean absolute errors 4) under varying quantile levels and response rates, with different &r and fixed K = 10 clients and data generated from (O, 1). In Case Nlw. each client uses a unique quantile level тą ranging uniformly from [0.3, 0.5]: in Case Thigh. Tk is ranging from [0.5, 0.8). "hetero" indicates client-specific truthful response rates ry range uniformly from [0.25, 0.9].
Finally, to further strengthen our simulation study, we conducted a set of additional experiments, including: (I) consider partial client participation to mimic more realistic federated environments; (2) sensitivity analyses the truthful response rate and step-size schedule to assess the robustness of the method; (3) a comparison between the resulting confidence intervals and oracle normal- based intervals to illustrate the conservativeness of the self-nommalized inference procedure; and (4) an initial exploration of extending our method to a fully decentralized federated learning setting; see Appendix for details.

---

Published as a conference paper at ICLR 2026
4.3 REAL DATA
In this subsection, we empirically evaluate the effectiveness of our proposed method using rep- resentative real-world dataset widely employed in privacy research: Government Salary Dataset (Plečko et al. (2224) This dataset sourced from the 2018 American Community Survey con- ducted by the U.S. Census Bureau and contains over 200,000 records, with annual salary (in USD) as the response variable. Since annual salary represents sensitive financial information (Gillenwater al. 202T), we treat it as requiring privacy protection. To incorporate the dataset's inherent ge- ographic structure, we partition the sample according to the feature "economic region." The three smallest regions are merged into a single "Others" category, yielding seven regions in total, each regarded as one client. Because region-level sample sizes vary, we apply oversampling to balance the data, resulting in tr = 53, 960 observations per client. All other hyperparameters follow the settings in Section 4.1. For analysis, apply log transformation to the response variable and subsequently toccktransform it.
{0.3, 0.5, 0.8} and consider response rates T {0.6, hetero, 0.9). where "hetero" indicates client-specific truthful response rates r% range uniformly from 0.6 to 0.9. For reference, also compute the full-sample quantiles without LDP. The resulting estimators and confidence-interval lengths are summarized in Table As shown, higher response rates r and more communication rounds generally produce shorter confidence intervals, consistent with our simulation findings. In most cases, the empirical quantiles fall within our reported intervals, highlighting the practical utility of our method for real data.
We target quantile levels Tk
Quantile (T)
Rate (r)
C1
C5
Log
Empirical
0.3
0.6
33367 (1742)
33184 (6697)
33030 (12093)
hetero
33418 (1424)
33229 (5291)
33140 (9788)
34000
0.3
0.9
33547 (1548)
33403 (4443)
33239 (7828)
0.5
0.6
48454 (2255)
48212 (6315)
47951 (11361)
0.5
hetero
48462 (1435)
48290 (4973)
48091 (9025)
50000
0.5
0.9
48610 (1454)
48494 (3851)
48311 (6863)
0.8
0.6
78586 (2066)
78168 (6646)
77995 (13144)
0.8
hetero
78390 (1291)
78054 (5862)
77722 (11101)
80000
0.8
0.9
78657 (1138)
78300 (4677)
78084 (8928)
Table 3: Estimation results (interval lengths) on the real dataset across varying quantile levels and response rates. "Empirical" denotes the full-sample quantile estimator without LDP. "hetero" indi- cates client-specific truthful response rates range uniformly from 0.6 to 0.9.
CONCLUDING REMARK
We propose federated-leamdnn algorithm for quantile inference under LDP that flexibly accom- modates client-level heterogeneity in quantile targets, privacy budgets, and data distributions. In addition, one innovation that should be emphasized is that our developed theoretical results of local SGD quantile estimator. We first design an LDP mechanism that can transform the LDP federated quantile estimation into the non-DP case, and then derive the asymptotic normality and functional central limit theorem of the proposed estimator under non-DP cases. It is first weak-convergence result for local SGD without the usual average-smoothness assumption in existing literature. Build- ing on these non- private asymptotic results, we develop a self-normalle constructs valid confidence intervals under LDP without requiring direct estimation of variance.
inference procedure that
Despite these advances, our method has several limitations. First, it relies on additional regularity assumptions to handle arbitrary client-level data heterogeneity. Second, as noted (Shao) 2015). self-normalization yields heavier-tailed limit distributions than the Gaussian, which can produce conservative confidence intervals or reduced power in hypothesis testing. In addition, theoretically understanding how asynchronous partial client participation affects estimation and inference is also an important direction for future work. Finally, our framework depends on a central server for aggregation and synchronization. which may not be available in fully decentralized environments. Addressing these challenges and extending the algorithm to decentralized settings remain important directions for future research.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The authors sincerely thank the anonymous reviewers, AC, and PCs for their valuable suggestions that have greatly improved the quality of our work. This work was supported by the Shanghai Engineering Research Center of Finance Intelligence (Grant No,19DZ2254600) Leheng Cai would thank to the funding supported by China Association for Science and Technology and National Natural Science Foundation of China (No. 12171269). Shuyuan Wu's research is partially supported by National Natural Science Foundation of China (No. 12401392) and China Postdoctoral Science Foundation (No. 2024M751929, No. 2024T170540).
REPRODUCIBILITY STATEMENT
All numerical experiments and real-data analyses fully reproducible via the code included in the supplementary materials.
REFERENCES
Naman Agarwal, Ananda Theertha Suresh, Felix Xinnan X Yu, Sanjiv Kumar, and Brendan McMa- han. cpsgd: Communication-efficiie and differentially-prriate distributed sgd. Advances in Neural Information Processing Systems, 31, 2018.
Joshua Angrist, Victor Chernozhukov, and Iván Fernández- Val. Quantile regression under misspec- ification, with an application to the us wage structure. Econometrica, 74(2):539-563, 2006.
Luca Barbaglia, Sergio Consoli, and Sehastiano Manzan. Forecasting with economic news. Journal of Business & Economic Statistics, 41(3):708-719) 2023.
Steven Bellman, Eric J Johnson, Stephen Kobrin, and Gerald Lohse. International differences in information privacy concerns: A global survey of consumers. The Information Society, 20(5): 313-324, 2004.
T Cai, Yichen Wang, and Linjun Zhang. The cost of privacy: Optimal rates of convergence for parameter estimation with differential privacy. The Annals of Statisties, 49(5):2825-2850, 2021.
Likai Chen, Georg Keilbar, and Wei Biao Wu. Recursive quantile estimation: Non-asymptotic confidence bounds. Journal of Machine Learning Research, 24(91):1-25, 2023.
Song Xi Chen. Nonparametric estimation of expected shortfall. Journal of financial econometrics, 6(1):87-107, 2008
Xi Chen, Jason D Lee, Xin T Tong, and Yichen Zhang. Statistical inference for model parameters in stochastic gradient descent. The Annals of Statistics, 48(1):251-273 2020.
Xi Chen, Weidong Liu, and Yichen Zhang. First-order newton-type estimator for distributed es- timation and inference. Journal of the American Statistical Association, 1888540):1858-1888 2022.
Victor Chernozhukov and Iván Fernández- Val. Inference for extremal conditional quantile models, with an application market and birthweight risks. The Review of Economic Studies, 78(2): 559-589, 2011.
Victor Chemoohukoy and Christian Hansen. An iv model of quantile treatment effects. Economet- rica, 73(1):245-26) 2005.
Albert Cheu, Adam Smith, Jonathan Ullman, David Zeber, and Maxim Zhilyaev. Distributed dif- ferential privacy via shuffling. In Advances in CespologgyEUURRCHE termational Conference on the Theory and Applications of Cryptographic Techniques, Darmstadt, Germany, May 19-23, 2019, Proceedings, Part 38. pp. 375-403 Springer, 2019.
2019: 38th Annual In-
Bolin Ding, Janardhan Kulkarni, and Sergey Yekhanin. Collecting telemetry data privately. Ad- vances Neural Information Processing Systems, 30, 2017.

---

Published as a conference paper at ICLR 2026
John C Duchi, Michael Jordan, and Martin J Wainwright. Local privacy and statistical minimax rates. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science, pp. IEEE, 2013.
Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In Anmual International Conference on the Theory and Applications of Cryptographic Techniques, pp. 486-503. Springer, 2006.
Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. Rappor: Randomized aggregatable privacy- presseving ordinal response. In Proceedings of the 2014 ACM SIGSAC conference on computer and communications security, pp. 1054-1067, 2014.
Jennifer Gillenwater, Matthew Joseph, and Alex Kulesza. Differentially private quantiles. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 3713-3722. PMLR, 18-24 Jul 2021.
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, An- drew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in hour. arXiv preprint 17Xiv:170.006677, 2017.
Jia Gu and Song Xi Chen. Distributed statistical inference under heterogeneity. Journal of Machine Learning Research, 24(387):1-57, 2023.
Peter Hall and Christopher C Heyde. Martingale limit theory and its application. Academic press, 2014.
Xuming He, Xiaoou Pan, Kean Ming Tan, and Wen-Xin Zhou. Smoothed quantile regression with large-scale inference. Journal of Econometrics, 232(2)3367-388, 2023.
Jiaqiao Hu, Yijie Peng, Gongbo Zhang, and Qi Zhang. A stochastic approximation method for simulation-based quantile optimization. INFORMS Journal on Computing, 34(6):2889-2077, 2022.
Jie Hu, Yi-Ting Ma, and Do-no Eun. Does worst-performing agent lead the pack? analyzing agent dynamics in unified distributed sgd. Advances in Neural Information Processing Systems, 74.77754-22789 2024.
Ziyue Huang, Yuting Liang, and Ke Yi. Instance-optimal mean estimation under differential privacy. Advances in Neural Information Processing Systems, 34:25993-26000 2021.
Xue liang, Xuebing Zhou, and Jens Grossklags. Signds-fl: Local differentially private federated learning with sign-based dimension selection. ACM Transactions on Intelligent Systems and Technology (T7ST), 1355)2122, 2022.
Matthew Joseph, Jieming Mao, Seth Neel, and Aaron Roth. The role of interactivity in local dif- ferential privacy. In 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS), pp. 94-105, 2019. doi: 109/FOSS 2019.00015.
Nathan Kallus, Xiaojie Mao, and Masatoshi Uehara. Localized debiased machine learning: Efficient inference on quantile treatment effects and beyond. Journal of Machine Learning Research, 25 (16):1-59, 2024.
Jakub Konečný, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint ariiv: 2016.
Boyuan Li, Shaohui Zhang, and Qiuying Han. Federated learning with joint server-client momen- tum. Scientific Reports, 15(1):15626, 15626, 2025.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and sys- tems, 45429-450, 2020.
12

---

Published as a conference paper at ICLR 2026
Xiang Li, Jiadong Liang, Xiangyu Chang, and Zhihua Zhang. Statistical estimation and online inference via local sgd. In Conference on Learning Theory, pp. 1613-1661. PMLR, 2022.
Chun Liu, Youliang Tian, Jinchuan Tang. Shuping Dang, and Gaojie Chen. A novel local differential privacy federated learning under multi-privacy regimes. Expert systems with applications, 227: 120266, 2023a.
Wei Liu, Li Chen, Yunfei Chen, and Wenyi Zhang. Accelerating federated learning via momentum gradient descent. IEEE Transactions on Parallel and Distributed Systems, 31 (8)):1754-11 2020.
Yi Liu, Qirui Hu. Lei Ding, and Linglong Kong. Online local differential private quantile inference via seff-normalization. In International Conference Machine Learning, pp. 21698-21714. PMLR, 2023b.
Andrew Lowy and Meisam Razaviyayn. Private federated learning without trusted server: Optimal algorithms for convex losses. In The Eleventh International Conference on Learning Representa- lions, 2023.
Xu Ma, Xiaoqian Sun, Yuduo Wu, Zheli Liu, Xiaofeng Chen, and Changyu Dong. Differentially pri- vate byzantine-robust federated learning. IEEE Transactions on Parallel and Distributed Systems, (3(12):360037001 2022.
Horia Mania, Xinghao Pan, Dimitris Papailiopoulos, Benjamin Recht, Kannan Ramchandran, and Michael I Jordan. Perturbed iterate analysis for asynchronous stochastic optimization. SIAM Journal on Optimization, (7(4)2 22299, 2017.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-effiienn learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273-1282. PMLR, 2017.
Sandra J Milberg, H Jeff Smith, and Sandra J Burke. Information privacy: Corporate management and national regulation. Organization science, 11(1):35-57, 2000.
Drago Plečko, Nicolas Bennett, and Nicolai Meinshausen, fairadapt: Causal reasoning for fair data preprocessing. Journal of Statistical Software, 1 110:1-35, 2024.
Herbert Robbins and David Siegmund. A convergence theorem for non negative almost super- martingales and some applications. In Optimizing methods in statistics, pp. 233-257. Elsevier, 1971.
Xiaofeng Shao. Self-normalization for time series: review of recent developments. Journal of the American Statistical Association, 1106512)1199--1817, 2015.
Xiaoying Shen, Hang Jiang, Yange Chen, Baocang Wang, and Gao. Pldp-fl: Federated learning with personalized local differential privacy. Entropy, 25(3):485, 2023.
Siping Shi, Chuang Hu, Dan Wang, Yifei Zhu, and Zhu Han. Distributionally robust federated learn- ing for differentially private data. 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS), pp. 842- 852. IEFF, 2022.
Shuang Song, Kamalika Chaudhuri, and Anand D Sarwate. Stochastic gradient descent with differ- entially private updates. In 2013 IEEE global conference on signal and information processing, pp. 245-248 248. IEEE, 2013.
Sebastian U Stich. Local sgd converges fast and communicates little. arXiv:1805.00 097777770077.,....0 2018.
arXiv preprint
Weijie Su and Yuancheng Zhu. Uncertainty quantification for online learning and stochastic ap- proximation via hierarchical incremental gradient descent. arXiv preprint arXiv: arXiv:1802 2018.
Ye Tian, Haolei Weng, and Yang Feng. Unsupervised federated learning: A federated gradient algorithm for heterogeneous mixture models with robustness against adversarial attacks. arXiv preprint 2...23300..3330 2023.
13

---

Published as a conference paper at ICLR 2026
Huixia Judy Wang, Deyuan Li, and Xuming He. Estimation of high conditional quantiles for heavy- tailed distributions. Journal of the American Statistical Association, 107(500):1453- 14633,14442404 2012.
Chuhan Xie, Kaicheng Jin, Jiadong Liang, and Zhihua Zhang. Asymptotic time-uniform inference for parameters in averaged stochastic approximation. arXiv preprint arXiv:2410.15057, 2024.
Steve Yadlowsky, Scott Fleming, Nigam Shah, Emma Brunskill, and Stefan Wager. Evaluating treatment prioritization rules via rank bank-weighted average treatment effects. Journal of the Ameri- can Statistical Association, 120(549):38-51,, 2025.
Yang Zhao, Jun Zhao, Mengmeng Yang. Teng Wang, Ning Wang, Lingjuan Lyu, Dusit Niyato, and Kwok-Yan Lam. Local differential privacy-based federated learning for internet of things. IEEE 2020.
Internet of Things Journal, 8(1 (11):8836-8853 ((1))88)66
Wanrong Zhu, Zhipeng Lou, Ziyang Wei, and Wei Biao Wu. High confidence level inference is almost free using parallel stochastic optimization. arXiv preprint arXiv:2401.09344 2024.
A NOTATIONS AND COMPLETE ASSUMPTIONS
A.I NOTATIONS
Notations
Meanings
L
Global Loss function
T
Global quantile
Cre
Local loss function for client with local quantile T the weight assigned to client k:
Pk
Distribution for client & CDF for client k
FR
Jr.
Density function for client Communication iteration sets Communication iteration time Interval length
I
Im
Em
Q
Variable in loss function True target value Value at F5 (Q")
Q'
Qk
Iterations at step for client & Response rate for for client & Privacy budget for client
LDP federated quantile estimator LDP federated quantile self-normalizer Brownian motion
Dr
B(s)
Va/2.g
(1 a/2) quantile of self-normalization distribution
Table S.1: Notations table.
A.2 COMPLETE ASSUMPTIONS
We introduce the following necessary assumptions.
Assumption S.1. For some constant C, > 0, fa()), 1kkk K, the density function for client k; uniformly bounded by Cy, and mingskgK fx(Q*)
Assumption S.2. Define the effective step = Tm Em. which is non-increasing in m and satisfies
that E 2 < oc, Em= Yim = 00, and (7m 7mm+1\/mm o(mm))
Assumption S.3. The sequence {Em)mzı satisfies that
(a) m21 is either uniformly bounded non-decreasing.
14

---

Published as a conference paper at ICLR 2026
(b) There exist some å > 0 and v 2 1 such that
T-1
T-1 Σ Fm mm00
T-
lim sup T-t00
Σ mm0
El+ó
Σ
< 00, lim
Σ mm0
Em
= v.
T
T-000 T
mm=0
(c) Denote ty
T-1 En satisfying
vl T
vlT 1 =0 T
lim T-000
= 0,
lim T-S
V/T
m==
Assumption S.1 is mild and regular condition concerning the uniform boundedness of density functions. Assumptions S.2 and [S.3] require that the effective step sizes decay slowly and the com- munication intervals increase slowly; see also Li al. 2022)
B TECHNIQUE PROOFS
Proof of Theorem
2.1 Note that the following recursive equation
Th + 2TTk
1 rs
2TT
oi 9-11
7mm
16( =1)
7m
1166 0),
2rk
2rk
asymptotically equivalent
1 1
Tkk + +TTk
I(T, S I
di = -i-1 + 17m-
2
where
P( ,/ Tk:
P(Z oo) = P(7 = =x) (1 (1-r)/
Observe that the above SGD update is designed to solve the following non-DP loss function
arg min IE
my-D, fri Cr, Qx))
Qx
Inspired by this, we denote
arg min
P
A2NPA foē Q)}
arg min
Pk B
p.
[[TA - -Q) (Tx- T(xr S Q)))
Ikk
xx
In the following, we desire to verify that Q* Q* By the definition of the minimizer of the objective function,
Pk
(P(*) < @*) [r 0.
Tk
Since F = TxT + (1 r2,/2, and
PP < ===))==r-kee = r<FFi("
+ (1-11r)) (1-- rw)/2,
one has
Σ ( P& rk)/((rr))
Σ (0a(1-ra)(22r) PKT).
Subtracting Ek (px(1 (2rx) from both sides of the equation, we obtain
K
K
Σ
Pa F2(
Σ PAT
15

---

Published as a conference paper at ICLR 2026
Recall that the definition of Ek= (Q*))) we show that
K
Σ
Ph PM
Σ Ph Fx(
k=1
which implies Q*.
The proof is now completed.
Proof of Theorem 3.1:
Theorem 3.1 is direct consequence of Theorem 3.2
Proof of Theorem 3.2
We follows the perturbed iterate framework that is derived by Mania et al. (2017) and also used in
Li
2022 Define the sequence O in the following way:
K
Pade
Define s== = (x UK, VKyT with
P(U =1
P(U = = 1-11-r
P(VV = 1) P(V* = = 1/2.
For & ,...., K. let Up and V be i.dd. copies of U and V*. respectively. Denote c
(xt U. V() Define
G+(
+ Tk 2F&T
11 S gi uu +1- v(l
2rk
(B.1)
1 Tk + TT
11x > > - (
2r
Elementary calculations show that
9k (g) = BGxlg, (t) = Fk(() - T.
Define
s(1) = Σ
(agk(1)
k=1
Denote
(x) = Gklg, ct 9g((g)-
Besides,
E (( 1- 12FF 1) 112
4r
By definition, for tm S t < fm+1 - 1, we have
de+1 = Tt.
1mm Σ Gulal. ct).
Define S. qtm Q", and recall that Em tm+1 tm and Ym Mm Elementary Iteration
from 1. = mm to 1m+11 yields
8m41-1 K
Σ Σ DeaGrlgi-nst = 8m - Vmm
8m++1 8m
17m
t=
k=
in which
Am+1
Dm
E
t=tm
16

---

Published as a conference paper at ICLR 2026
We define
Fix 1-1 K
hm =
Σ
Gr (Ttmi (t),
Em
and further decompose that
Vm G& + (g (4tm Gom) + (hm (Qtm)) + (Vm - hm))
= Gsmm + I' + Em + 8m:
where G= -K= Pk (Q*)) is the Hessian at Q*. It then follows that
8m+1 (1 G) 8m 7' (rm Em + 6m) == Bm Vm Vm
(B.2)
where Bm : 7m G and Um = Im + Em + 8m for short. Recurring (3.2 gives
Sm++1
II /=0
B
B; (=++1
II
1=0
Here, we use the convention that 1==++1 B, = 1 for any m 2 0. Recall the definition that
1
h(r; T)
max
E
E
Em
m=1
m=1
Hence,
h(r.T)
h(r.T)
vt
8m+1
Σ
II /=0
B
80
Σ
II
B
T
T
m==
m=0
==
+=+1
h(r,T)
MrrT)) N(r.T)
VlT
VTT
Σ
II )=0
B
80
Σ
Σ
II imj+1
B;
YsUg.
T
T
m==
1=0
m=j
For any n 2 j. define AB
A7
II i=j+1
B
With the notation of Ay. we can rewrite that
Mr.T)
M(r,T)
vt
8mt1=
T7o
VtT
Au
M(r,T)
Bo80
Vtr T
Σ
Ab(r.T)
Um.
m==
Since Um = T' + Em + Sm then
h(r:T")
h(r:T)
vtr h(r:T)
VT
8mt1
Vtr
G G-
cm
VTr
A0
h(r:T)
Boso
Ah(rTT m (rm + 8m)
T
T
Tro
T
カ=
m==
m=
M(r.T)
vtT
(AT - -G-1) Em
T
m==
M(r.T)
Vtr
(Ah(T.T) ATT Em
Too
Ti
Ts.
To complete the proof, we first investigate the partial-sum asymptotic behavior of
h(r.T)
Vtr
G
Ems
T
m=
17

---

Published as a conference paper at ICLR 2026
and then show that the four separate terms: sup/e[o.1] Toll supp/ [Ta]], suprejo.1] ||Tall, and
1 |Ta|| are (r(1), respectively.
We aim to follow the proof of Theorem 4.2 in Li et al. (022 However, we find that the average smoothness condition their Assumption 3.1 is not satisfied. Due to the presence of the indicator function in (B.I. we only have
v (E(GG(T, - Guly. | ------ yl
(B.3)
Upon close examination of their proof, we find that this condition crucial in the proof of their key Lemma B.2.
In the following, we re-establish the proof of
E It.. Q7 7m,
4tm
Q-
under the condition given in B.3). Consider that
E (10f+1 Teml
|l 1m
Σ Gulat stt) 1Ftm
i=
IE ( (155)1
7m
(1+ Fcml 1 + It. - Q71),
where the last inequality holds by the following fact
E (Gt((tts))))) = ([aaC (+) 9n(gtt)² (F,) + (g
(IGA(u ..) -gx(x")) )² 1F:
+ 2lgu (1.)) gn (x()))+++ +20)(Q")
2 [1 + (Q")) + lutt Q71
+ of @tm 1 + It.. Q.P².
Define
V, Σ P&E (1gt Ii. (F
k=1
Hence,
V4+1 7m Σ (1+ Tr...
+2*+++))
which further implies that (since Vim. =)
Am41-1
1 fmi1-2
1mt1-2
Σ V
Σ
V4+1 S
Mm
Σ
18tm - +11++v)
Em
Em
Em
t=
t=
1=
i=fm
fmt1-2
7/m
Σ
(tm-1 -t 11 (1+ l - Q*1
Em
t=tm
tm41-2
S 1m
(1 17tm QQ"I+ Vi)
1 (mi1 1-1
E 7m Em
| - Q'++ +
Σ
Vi
Em
t=tm
18

---

Published as a conference paper at ICLR 2026
Denote by 7/ 1m It follows that
Em E
V mm (1+ (t.... Q*1)-
(B.4)
Let GR(+)) denote an antiderivative of 9u (:), and G()) = EE= (..). Let A. = - The equation (17) in Li et al. (2022) shows that for some constant L >>20
E {9(( F
1 g(@t..) - (V ) + LE Fi
+ (7m/2++ 7m L) E( Fr...;
where
fm+1 -1 K
bm+1 -1 K
hm
Pk V 6m = Em
Pk VG
Em
t=
t=
Lemma B.9 of Li et al. (2022) obtains that
C1
C2
E S (T..)...)) -p+
+
Tt - QPP.
Fm
Fm
Notice that
(Gx(0 G (Fim } 2 E2 (1,t" Tim 1itm)))
E
Thus,
fm+1 11
Σ
Σ
P (Gx(UE-1:(t))
Gk (Tt. ((t) FFim))
Em
k=1
tm41
Σ Ba (lvē
Qrm ((tm))
E
2 mm (1 1 Q*1).
where the last inequality holds by (B.4). Therefore, we obtain that
E[Am+ ((a)) Am - 7m / )1 7 L |V9(0m.))) )1++
Ci Fm
C2 Fm
1t... 1 Q41
+
+ (7m/² + 2 m/ (C1 + C2 - 1)1)
< (1 cm + 02 Am + (c3 + ca A 7.m
Since we assume that the parameter space is uniformly bounded, it entails that Am is also uniformly bounded. Thus, we have
B(A F S (1 7m + C2? Am + (e3 cs)
mm-
Apply Robbins Robbins-Siegmund theorem in Robbins & Siegmund (971) to obtain T... + Q" almost surely. Lemma A. in Su Zhu 2018) states that any positive constants C₁ C2. if 7m o(1), 7m 1/7m = + o(7m), ), and B,, is positive sequence, satisfying
Bmn 7m- (1 - 61 7m Bm-1 + 027/m=
Ym
then supm Bm Using this lemma, immediately obtain that for some positive constant C' > 0,
EAm sup m21 7m-1
< <C,
which entails that
Elgt... 2-12 < RAAm 7m-1 S Tm (1 o(Tm)) S mm-
19

---

Published as a conference paper at ICLR 2026
To demonstrate that our setting satisfies Assumption 3.2 of Li et al. (2022). define
S = (e)((") 1 1 12 132
1- 1-r 12( 112
4
4r
Hence,
IE (5 (0
F(-)) E=i( ~ |4h-1 QQ%
satisfying Assumption 3.2 in Li et al. (0022)
Assumptions 3.3 and 3.4 of Li ct al. (2022) are the
same as our Assumptions 3-4.
By closely examining the proof of Li ct al. 20022 we find that their Assumption 3.1 is used exclusively establish the L convergence rate and the almost sure convergence result in their Lemma B.2. The appearance of Assumption 1 in the conditions of their Lemma B.3, which derivs the functional weak convergence, is solely for enabling the application of Lemma B.2.
In our analysis, however, the key bound E ft... - Q¹12 2 YYm is proved directly under equation (B.3) This bound fully replaces the role played by Lemma B.2 in Li et al. (2022). Consequently, Assump- tion 3.1 of Li et al. (2022) is not needed in our theoretical development beyond this step, and the functional central limit theorem follows without invoking that assumption.
Recall that
fmil
Σ Pk { (T:[Tem: (t) g((t..)) )
8m
Fu
t=
k=
Define U3
=
E
F
and
(rU7 - U77)1111
Er(r) UT!
9m
U² -U
U? SrU <1+t1
m=1
Then,
Vt
h(r.T")
VT T
UhirTT U?
Em
Ur
T
m=1
Following Lemma B.16 in Li et al. (2022). one obtains that
Vtr
UR(OT))
VtT
sup rE[0.1]
T
Er
U7
T
Ur (r)
5..
(B.5)
By evaluating the conditions in Lemma B.13 of [i al. (2022). the invariance principles in the martingale CLT yileds that Er(r) 4, B(r) in C(O, 1), which is followed by
EK == p fri² (2Qx - 1]) B(r).
VT UTE(()) 4 T
2 k pnfx(Q")
Hus, Theorem A.2 of Hall & Heyde
2014 implies that (VITU)E((//) is tight in C(O. 1]- we could obtain the tightness of
Using the tightness of [VirUrEr (-) T21 and (B5
VtT U
U2
UZ
T21
following the arguments in the proof of Lemma B.16 of Li et al.
(022). Therefore, we could follow
the arguments in the proof of Theorem 4.2 in Li al. CLT.
(2022) complete the proof the functional
C ADDITIONAL EXPERIMENTS AND DISCUSSIONS
In this section, we report additional simulation results that complement the main experiments in Section 4. Unless otherwise specified, all settings are identical to those in Section 4.1.
20

---

Published as a conference paper at ICLR 2026
Log
Est bounds
0,450
0,425
0.400
0.375
0.350
100
200
300
100
200
300
100
200
300
(x1000)
(x1000)
Cy (x1000)
C1
Log
-0.10
Est
0.15
bounds
True Q
0.20
0.25
-033
-033
-0.40
100
200
300
0
100 200
300
100
200
300
(x1000)
ty (x1000)
(y (x1000)
Figure S.1: Sample trajectories of the iterative estimator and corresponding confidence intervals under heterogeneous distributions (Hete I., with тą 0.9 and 0.5. upper panel) and heteroge- neous quantile levels (Now. with heterogeneous response rates, lower pancl). The horizontal dotted line indicates the quantile value Q
Quantile (7)
Rate (r)
Log
DP-SGD (CI)
DC
Single
10000
0.5
0.25
0.950X0.0160)
0.979(0.0250)
0.978(0.0324)
0.952(0.0242)
0.802(0.3199)
0.926(0.0179)
0.5
hetern
0.953(0.0086)
0.994(0.0124)
0.993(0.0168)
0.956(0.01255
1.999(0.0678))
0.942(0.0075)
1.000X0.0027)
1.000(0.0057)
1.00000.0097)
0.987(0.0044)
1.000(00216)
0.954(0.0004))
Tlw
0.25
0.929(0.0196)
0.976(0.0270)
0.978(0.0349)
0.911(0.039)
0.740(0. (3003)
0.930(0.0194)
hetern
0.923(0.01 18)
0.983(0.011))
0.983(0.0174)
0.896(0.0181)
0.999(0.0735)
0.960(0.0083)
Tlw
0.969(0.0055)
0.999(0.0047)
1.00000000 1)
0.922(0.0074)
1.0000000311)
0.954(0.0054)
Taigh
(.972(000110) 0.956(0.02115)
0.971(0.0399)
0.944(0.0313)
0.729(0.
0.944(00.013)
Thigh
hetero
0.956(0.0109)
0.986(0.0145)
0.988(0.0198)
0.959(0.0160)
0.997(0.0839)
0.954(0.0089
0.9
0.999(0.0044)
0.997(0.0046)
1.000000068)
0 0.989(0.0059)
1.00((00059)
0.974(0.0059)
fr
50000
0.960X0.0070)
0,9870.0111)
(0.014)))
0.955(0.0104)
0.977(00787))
0.958(0.0071)
0.5
hetero
0.975(0.0037)
0.986(0.0052)
0.994x0.0077
(0055)
1.000(0.0156)
0.970(0.0030)
0.5
0.9
1.00000.0110
1.000(0.0016)
1.0000.0029)
0.995(0.0019)
1.00000.000 0051)
0.950(0.0020)
Tho
0.973(0.0119 0.913(0.0104)
0.988(0.0154)
0.878(0.017D)
0.973(0.0863)
0.946(0.0079)
Tow
hetero
(8196000076)
0,942(0.0070 0070)
0.977(0.0884)
0.786/0.01
0,99910.0168 0166))
0.952(0.0034)
0.9
0.987(0.0042)
0.988(0.0018)
O. (9022)
0.854(0.0054)
1.0000.0000 0054)
0.962(0.0021)
Taigh
0.25
0.951(000107)
0.86(0.0163)
0.9100001835 0.9000.2500
0.976(0.0983)
0.950X0.0899)
Taigh
0.995(0.0103) 0.967(0.00)7)
0.988(0.0164)
0.924(0.01
1,0000.0189)
0.946(0.0039)
7aich
1.000X0.001)
1.000(0.0433)
1.000000083)
0.995(0.0027)
1.00000.0599
0.946(0.002))
Table S.2: Empirical coverage probabilities at the 95% nominal level (mean absolute errors L) under varying quantile levels and response rates, with different tr and fixed K 10 clients and data generated from C(O, 1). In Case Tlw, each client uses unique quantile level ranging uniformly from [0.3, 0.5]; in Case Thigh: Ti ranging from (0.5, 0.8). "hetero" indicates client-specific truthful response rates T'k range uniformly from [0.25, 0.9).
21

---

Published as a conference paper at ICLR 2026
Quantile (r)
Rate (r)
C1
C5
Log
DP-SGD (CI)
DC
Single
fr 10000
0.3
0.25
0.947(0.0271)
(.9800))400)
(.967().0545)
0.925(0.0223)
0.546(0.61 11)
0.946(0.0339)
0.3
hetero
0.957(0.0138)
0.994(0.0231)
0.992(0.0311)
0.942(0.0223)
0.948(0.0137) 0.682(0.346)
0.9
0.000000039)
1.000r0.0170)
1.0000002244
0.994(0.0066)
(.07000.3044)
0.972(0.0083)
0.25
0.942(0.0225)
0.983(0.0298)
0.977(00389)
0.935(0.0314)
0.936(0.0264) 0,968(0.3335)
0.5
hetero
0.946(0.0132)
0.970(0.0155)
0,951(000170) 0.981(00019)
1.0000.0757)
0.942(0.0109)
0.5
0.99
0.994(0.0053) (
0.998(0.0644)
0.9530001000
0.968(0.0071) 0.961(000119)
01958(007299 0.935(0.0450)
0.96000.0933)
0.938(000677)
0.08701 0TO0)
0.950(0.0552)
0.8
hetero
0.956(0.0232)
0.963()03370)
0.971(0.0457)
0.963(0.0343)
0.968(0.0210) 0.557(
0.8
0.9
0.985(0.0171)
0.836(0.0269)
0.933(0.0267)
0.971(0.0165)
0. (..440))
0.982(0.0122)
GT 50000
0.25
0.902(0.0151)
0.979(0.0165)
(88(0.0225)
0.863(0.0253)
0.958(0.0137) (..9(0)))56)
0.3
hetero
0.835(0.0102)
0.975(00088)
O. (.99(000108))
0.023(0.30266 0.831(0.0164)
0.948(0.0059)
0.3
0.9
0.984(0.0085 00566)
1.000(0.0034)
1. ..0000.0079)
0.869(0.0072)
0. 0.000(0.295))
0.938(0.0037)
0.25
0,935(0.0100)
0,983(0.0127)
0.986(0.01 80)
0.926(0.0144)
1.000(0.0910)
0.958(0.0108)
0.5
hetero
0.942(0.0063
0.995(0.0003) 0.981(0.0044)
0,947(0.0081)
0.853(0.0496)
0.954(0.0045)
0.5
0.9
0.998(0.0049)
0.999(0.0017)
1.0000.00339)
0.959(0.0052)
0.948(0.0029) 0.477(..0481)
0.8
0.25
0.926(0.0247)
0.991(0.0344)
0.986(0.0514)
0.952(0.0202) 0.522(0.5824) 0.887(0.0437)
hetero
0.925(0.0149)
0.990(0.0231)
0.990(0.0371)
0.890(0.0278)
0.079(0.4077)
0.950(0.0086)
0.8
0.9
1. 1000(0.0038)
1.0000.0107)
1.00X0.0197)
0.979(0.0077)
0.001(0.3822)
0.968(0.0049)
Table S.3: Empirical coverage probabilities at the 95% nominal level (mean absolute errors L) under heterogeneous distributions for different êT. The number of clients K is fixed 10. Data for each client k are independently generated from C(pz, 1), where Pa ~ NN00, 1). "hetero" indicates client- specific truthful response rates riK range uniformly from [0.25, 0.9].
Quantile (T)
Rate (r)
C1
C5
Log
T = 5000
0.5
0.25
0,954(0.0189)
0.974(0.0129)
0.986(0.0112)
0.5
hetero
0.959(0.0103)
0.976(0.0065)
0.995(0.0052)
0.5
0.9
0.999(0.0033)
1.000(0.0040)
1.00((0.0026)
Now
0.25
0.957(0.0200)
0.974(0.0137)
0.991(0.0116)
Now
hetero
0.957(0.0108)
0.977(0.0067)
0.993(0.0053)
Now
0.9
1.000(0.0033)
1.000(0.0040)
1.000(0.002))
Thigh
0.25
0.956(0.0212)
0.975(0.0128)
0.993(0.0123)
Thigh
hetero
0.961(0.0112)
0.984(0.0062)
0.996(0.0056)
Thigh
0.9
0.998(0.0037)
0.997(0.0028)
1.000(0.003))
10000
0.5
0.25
0.949(0.0133)
0.968(0.0078)
0.987(0.0061)
0.5
hetero
0.963(0.0071)
0.978(0.0037)
(.991(0.0030)
0.5
0.9
0.995(0.0023)
0.999(0.0020)
0.999(0.0014)
Now
0.25
0.947(0.0136)
0.972(0.0078)
0.984(0.0064)
Now
hetero
0.962(0.0072)
0.985(0.0038)
0.983(0.0033)
Tlw
0.9
0.999(0.0020)
1.000(0.0016)
0.967(0.0018)
Thigh
0.25
0.939(0.0145)
0.974(0.0086)
0.985(0.0066)
Thigh
hetero
0.968(0.0076)
0.988(0.0043)
0.985(0.0032)
Thigh
0.9
0.996(0.0023)
0.999(0.0031)
0.996(0.0014)
Table S.4: Empirical coverage probabilities at the 95% nominal level (mean absolute errors L) under varying quantile levels and response rates, with different T and fixed K 10 clients and data generated from A A/(0, 1). In Case nlw. each client uses a unique quantile level тą ranging uniformly from [0.3, 0.5); in Case Thigh: Tk ranging from (0.5, 0.8). "hetero" indicates client-specific truthful response rates r&k range uniformly from [0.25, 0.9).
22

---

Published as a conference paper at ICLR 2026
Quantile (T)
Rate (r)
C1
C5
Log
Hete I. - 7 5000
0.3
0.25
0.942(0.0271)
0.960(0.0168)
0.975(0.0151)
0.3
hetero
0.96((0.0131)
0.966(0.0086)
0.987(0.0067)
0.3
0.9
0.998(0.0043)
0.959(0.0063)
1.000(0.0033)
0.5
0.25
0.954(0.0254)
0.973(0.0154)
0.990(0.0153)
0.5
hetero
0.981(0.0072) 0.963(0.0120)
0.991(0.0071)
0.5
0.9
0.992(0.0042)
0.998(0.0032)
1.000(0.0034)
0.8
0.25
0.982(0.0242) (.954().0375)
0.998(0.0248)
0.8
hetero 0.9
0.968(0.0181)
0.988(0.0109)
0.998(0.0116)
0.8
0.985(0.0108)
0.999(0.0070)
0.982(0.0094)
Hete L L- T 10000
0.3
0.25
0.958(0.0184)
0.966(0.0102)
0.981(0.0083)
0.3
hetero
0.949(0.0096)
0.965(0.0050)
0.979(0.0040)
0.3
0.9
1.00((00009)
0.979(0.0022)
0.867(0.0036)
0.5
0.25
0,950(0.0165)
0.974(0.0094)
0.985
0.5
hetero
0,952(0.0085)
0.976(0.0045)
0.991(0.0039)
0.5
0.9
0.996(0.0025)
0.985(0.0018)
1.000(0.006))
0,8
0.25
0,966(0.0237)
0.983(0.0163)
0.990(0.0149)
0.8
hetero
0.962(0.0122)
0.988(0.0088)
0.974(0.0090)
0.8
0.9
0.990(0.0042)
0.997(0.0087)
(.645))
Hete D) T 5000
0.5
0.25
0.954(0.0195)
0.974(0.0129)
0.987(0.0109)
0.5
hetero 0.9
0,965(0.0098)
0.974(0.0075)
0.993(0.0049)
0.5
1.000(0.0037)
0.989(0.0000)
1.000(0.0026)
Hete D-T 10000
0.5
0.25
0,949(0.0132)
0.963(0.0078)
0.982(0.0064)
0.5
hetero 0.9
0.966(0.0069) 1.000(0.0023) 0.999(0.0014) 0.966(0.0023)
0.973(0.0039)
0.972(0.0034)
0.5
Table S.5: Empirical coverage probabilities at the 95% nominal level (mean absolute errors J) under heterogeneous distributions for different T. The number clients K fixed at 10. Hete L, data for each client l are independently generated from N ((He, 1), where μ4 NO,, 1). In Hete D, data are generated from (0, 1). (1111, 1), and C(O, 1) across different clients. "hetero" indicates client-specific truthful response rates vı range uniformly from [0.25, 0.9).
C.1 OTHER RESULTS IN SECTION 4
Efficiency-Accurac Trade-off. We first quantitatively study the computation and communica- tion time of the proposed method. Specifically, for each simulation run, record the wall-clock time, including both computation and communication components, and then average the results over 1,000 repetitions. We consider different communication strategies for the proposed estimators as well as the competing baselines. Figure S.2 reports the results for representative setting with quantile level 7ą normal distribution. and total sample size tr = 10,000. We obtain the following interesting findings. First, for the proposed methods, as the communication frequency increases, the MAE decreases, but both computation time and communication cost naturally grow, demonstrating a clear efficiency- accuracy trade-off. In addition, comparing across different methods, the proposed method attains communication cost comparable to DP-SGD while requiring noticeably less computation time. The DC method incurs the lowest overall cost because it performs only a single aggregation step; however, it also exhibits the largest MAE and substantial under-coverage in several heterogeneous settings, as reported in Section 4.
0.5. truthful response rate rk r == 0.5, data generated from a standard
Partial Participation. We further examine partial client participation to evaluate the proposed method in more realistic federated environments. In this setting, at each communication round, only out of the 10 clients are randomly selected to participate, while all other configurations remain identical to those in Section& 4.1. The results are reported in Tables S.6land S.7 We observe that
23

---

Published as a conference paper at ICLR 2026
Computation Time
Communication Time
1.2
140
120
Time (s)
0.8
80
60
0.4
40
0.2
20
0.0
C1
C5
Log
DP-SGD
DC
C1
C5
Log
DP-SGD
DC
Figure S.2: Computation time and communication time for different methods. We fix K 10 and tr 10,000.
the empirical coverage probabilities remain close the nominal 95% level, and the MAE increases only mildly compared with the full-participation case. These findings indicate that the proposed estimator is robust to partial and asynchronous client participation.
C.2 SENSITIVITY ANALYSIS
We begin by examining how the proposed method responds different truthful response rates and step-size schedules. Throughout this subsection, we adopt the normal design with X ~ N/ 1) and target quantile level ты = 0.5, and consider Ey, 5 and tr {10,000, 50,000}. All other settings follow Section 4.1.
First, conduct sensitivity analysis the truthful response rate. The results are reported in Figure SS.3] We observe that both the MAE and the average interval length steadily decrease as r increases, while the ECP remains close to above the nominal 95% level across wide range of response rates.
Second, investigate the effect of the step-size schedule. We adopt
20 F
7m.
m + 100*
where F denotes the average truthful response rate and a > controls the decay speed. Figure S.4 summarizes the results. Under the same experimental setting, the MAE, ECP, and Avg Len (average confidence-interval lengths) remain comparable across different values of a. This indicates that the proposed estimator is stable with respect to the choice of the step-size schedule.
C.3 CONSERVATIVENESS OF THE SELF-NORMALIZED INFERENCE PROCEDURE
To assess the efficiency and conservativeness of the resulting confidence intervals, we compare the resulting confidence intervals with oracle normal-bases intervals that use the true asymptotic vari- ance derived in Theorem 3.1 Tables S.8.S.9 S.9 report the empirical coverage probabilities (ECP) and average interval lengths (Avg Len) for both types of intervals under normal and Cauchy data- generating distributions. We observe that compared with the oracle intervals, the nelf-norma intervals are slightly conservative: they achieve coverage at or above the nominal 95% level but produce moderately wider intervals. This mild conservativeness arises because self-normalization induces heavier-taile tailed limiting distributions compared with the standard normal approximation. is important to note, however, that estimating the oracle variance in practice typically requires additional procedures that may consume further privacy budget under LDP. In contrast, the self- normalized construction avoids any variance estimation and remains fully privacy-preserving
24

---

is replaced by the Published as a conference paper at ICLR 2026
C.4 EXTENSION DECENTRALIZED FEDERATED LEARNING
In this subsection, we provide an initial exploration of extending our method to a fully decentralized federated learning setting, where central server is available and each client communicates only with its neighbors. Let the K clients be connected through an undirected communication graph represented by an adjacency matrix A (ak,k)) k where k2 1 if client Y can exchange messages with client k2 and 02 = 0 otherwise. Here we assume that every client is connected to itself, i.e., Okk 1, k 1...., K. Define the degree da, = Ex, 4ky, and construct the row-stochastic weight matrix W (Wkyks) as
Wkik2
d&
which encodes the network topology and performs neighbor averaging.
During communication iterations E I, the aggregation step in Algorithm following weighted decentralized averaging:
Wky Pj Vim.
I,...,.. K.
The local SGD update for t &f I remains unchanged, yielding fully decentralized variant of our estimator.
As preliminary empirical study of this extension, we evaluate its finite- sample performance under a ring network topology, where each client communicates only with itself and its two immediate neighbors. All other experimental settings follow Section 4.1. The results in Table [S.10) show that the decentralized estimator exhibits qualitatively similar behavior to federated counterpart, suggesting that method can naturally extend decentralized learning. A complete theoretical analysis of this decentralized variant is left for future work.
0.014
f 10k
0.10
0.375
0.0 0.010
0.008
0.925
Avg
0.06
0.006
0.900
0.04
0.02
0.850
0.5
Figure S.3: Mean absolute errors (MAE J), empirical coverage probabilities at the 95% nominal level (ECP), and averaged confidence interval lengths (Avg Len () under varying response rates. We fix TA = 0.5 and K 10, and report results for different values of br with data generated from N(001).
25

---

Published as a conference paper at ICLR 2026
Scenario Quantile (r)
Rate (r)
CI
C5
Log
t 10000
0.5
0.25
0.950(0.0186)
0.971(0.0292)
0.979(0.0417)
0.5
hetero
0.942(0.0102)
0.976(0.0140)
0.993(0.0194)
0.5
0.9
0.955(0.0047)
0.988(0.0062)
0,998(0.00)) (0880)))87)
Tlow
0.25
0.949(0.0190)
0.975(0.0305)
0.984(0.0437)
Homo
Tlow
hetero
0.953(0.0097)
0.981(0.0146)
0.994(0.0203)
Tlow
0.9
0.985(0.0032)
0.999(0.0044)
1.000(0.0091)
Thigh
0.25
0.947(0.0208)
0.971(0.0316)
0.982(0.046))
Thigh
hetero
0.928(0.0126)
0.984(0.0155)
0.993(0.0219)
Thigh
0.9
0.973(0.0058)
0.997(0.0062)
1.00((000000)
0.3
0.25
0.942(0.0255)
0.971(0.0401)
0.972(0.0565)
0.3
hetero
0.968(0.0198) (0.51()))1
0.983 (33008258)
0.3
0.9
0.946(0.0054)
0.961(00009))
0.990(0.0119)
0.5
0.25
0.955(0.0225)
0.970(0.0375)
0,984(0.0531)
Hete L
0.5
hetero
0.946(0.0116)
0.98100.0180)
0.997(0.0247)
0,5
0.9
0.978(0.0044)
0.993(0.0078)
0.995(0.0119)
0.8
0.25
0.955(0.0339)
0.975(0.0614)
0.986(0.0890)
0.8
hetero 0.9
0.964(0.0171)
0.987(0.0279)
0.992(0.0387)
0.8
0.998(0.0050)
0.988(0.0147)
0.999(0.0197)
f = 50000
0.5
0.25
0.962(0.0081)
0.984(0.0106)
0. 0.998()
0.5
hetero
0.988(0.0055) 0.962(0.0040)
0.996(0.0088)
0.5
0.9
1.000(0.0012)
1.000(0.0017)
1.000(0.0028
Tlow
0.25
0.945(0.0090)
0.986(0.0110)
0.994(0.0163)
Homo
Too
hetero
0.970(0.0047)
0.989(0.0058)
0.995(0.0086)
Now
0.9
1.000(000020)
1.0000000000)
1.000(0.0030)
Thigh
0.25
0.950(0.0088)
0.978(0.0118)
0.998(0.0176)
Thigh
hetero
0.942(0.0044)
0.983(0.0062)
0.997(0.0090)
Thigh
0.9
0.984(0.0017)
0.998(0.0036)
0.999(0.0056)
0.3
0.25
0.943()
0.974(0.0155)
O. 0.990(0.0228)
0.3
hetero
0.937(0.0067)
0.982(0.007))
0.994
0.3
0.9
0.992(0.0022)
0.991(0.0029)
0.997(0.0045)
0.5
0.25
0.952(0.0099)
0.981(0.0136)
0.994(0.0202)
Hete L
0.5
hetero
0.966(0.0051)
0.974(0.0065)
(092(0.0093)
0,5
0.9
0.978(0.0021)
0.974(0.0028)
0.993(0.0040)
0.8
0.25
0.948(0.0152)
0.984(0.0207)
0.990(0.0300)
0.8
hetero
0.949(0.0074)
0.962(0.0102)
0.995(0.0139)
0.8
0.9
0.902(0.0035)
0.954(0.0084)
0.991(0.0097)
Table S.6: Empirical coverage probabilities at the 95% nominal level (mean absolute errors L) under varying quantile levels and truthful response rates, with different values of tr and fixed number of K = 10 clients. Only 5 out of the 10 clients are randomly selected to participate in each commu- nication round. In Homo, data for each client i are independently drawn from (000, 1). In Hete L., client-specific data are independently drawn from A N(H,,) 1) with 14 ~ [o00, 1). The label "hetero" indicates heterogeneous truthful response with 7% ranging uniformly from 0.25 to 0.9.
26

---

Published as a conference paper at ICLR 2026
Scenario Quantile (r)
Rate (r)
C1
C5
Log
f 10
0.5
0.25
0.950(0.0232)
(.973().0353)
0.968(0.0473)
0.5
hetero
0.966(0.0114)
0.991(0.0169)
0.991(0.0229)
0.5
0.9
1.000(0.0034)
1.000(0.0062)
0.999(0.0100)
Tlow
0.25
0.936(0.0279)
0.974(0.0393)
0.967(0.0509)
Homo
Tlow
hetero
0.928(0.0150)
0.989(0.0187)
0.988(0.0241)
Tlow
0.9
0.997(0.0048)
1.000(0.0068)
1.000(0.0105)
Thigh
0.25
0.940(0.0333)
0.965(0.0466)
0.961(0.0621)
Thigh
hetero
0.951(0.0184)
0.985(0.0246)
0.980(0.0338)
Thigh
0.9
1.000(0.0045))
1 1.000(0.0088)
0.999(0.0132)
0.3
0.25
0.968(0.0588) 0.947(0.0396)
0.952(0.0763)
0.3
hetero
0.956(0.0199)
0.984(0.0306)
0.99300.0397)
0.3
0.9
0.999(0.0069)
0.999(000257)
1.00000.3550
0.5
0.25
0,940(0.0309)
0.963(0.0432)
0,968(0.0561)
Hete L
0.5
hetero
0.965(0.0148)
0.989(00220))
0.980(0.0284)
0,5
0.9
0.999(0.0046)
0.998(0.0076)
0.998(0.0112)
0.8
0.25
0.935(0.0743)
0.956(0.1039)
0.956(0.1376)
0.8
hetero 0.9
0.947(0.0423)
0.975(0.0539)
0.972(0.0687)
0.8
(.993(0.0114)
0.996(0.0145)
0.990(0.0192)
f = 50000
0.5
0.25
0.943(0.0106)
0,985(0.0148)
0.0011)
0.5
hetero
0.970(0.0053)
0.986(0.0087)
0.9890001221)
0.5
0.9
1.000(0.0023)
1.000(0.0050)
0.997(0.0072)
Tlow
0.25
0.896(0.0147)
0.988(0.0178)
0.986(0.0248)
Homo
Too
hetero
0.902(0.0087)
0.980(0.0092)
0.991(0.0124)
Now
0.9
1.0000.0002))
1.000(.0002))
0.999(0.0044
Thigh
0.25
0.982(0.0293) 0.893(0.0205)
0.971(0.0431)
Thigh
hetero
0.930(0.0130)
0.990(0.0212)
0.985(0.0316)
Thigh
0.9
1.000(0.0046)
1. 1.000(0.0104)
1.00((000661)
0.3
0.25
0.895(0.0232)
0.970(0.0272)
0. 0.989(0.0377)
0.3
hetero
0.875(0.0138)
0.983(0.0140)
0.987(0.0188)
0.3
0.9
1.000(0.0046)
1.000(0.0047)
0.990(0.0089)
0.5
0.25
0.957(0.0136)
0.984(0.0193)
0.985(0.0274)
Hete L
0.5
hetero
0.988(0.0068)
000910000100)
0.991(0.0154)
0,5
0.9
1.000(0.0031)
1.000(0.0045)
0.990(0.0090)
0.8
0.25
0.972(0.0659) (.923(0.0422)
0.943(0.0969)
0.8
hetero
0.932(0.0284)
0.987(0.0444)
0.966(0.0678)
0.8
0.9
1.000(0.0089)
1.000(0.0182)
0.999(0.0299)
Table S.7: Empirical coverage probabilities at the 95% nominal level (mean absolute errors L) under varying quantile levels and truthful response rates, with different values of tr and fixed number of K = 10 clients. Only 5 out of the 10 clients are randomly selected to participate in each commu- nication round. In Homo, data for each client i independently drawn from C(O, 1). In Hete L., client-specific data are independently drawn from C(4 1) with b4 ~ (0, 1). The label "hetero" indicates heterogeneous truthful response with 7% ranging uniformly from 0.25 to 0.9.
27

---

Published as a conference paper at ICLR 2026
mm0.25
meetter
0m0.9
0.016
1.00
0.12
0.30
0.012
Aug Len
0.008
0.007
0.05
0.99
0.000
0.003
0.97
Avg
0.00
0.004
0.96
0.003
0.55
0.60
0.65
0.70
0.55
0.60
0.65
0.70
0.55
0.60
0.05
0.70
Figure S.4: Mean absolute errors (MAE J). empirical coverage probabilities the 95% nominal level (ECP), and averaged confidence interval lengths (Avg Len J.) under varying step-size parameter a. We fix 76 = r = 0.5 and K 10, with data generated from W(0, 1). Three response-rate scenarios considered, where "hetero" indicates heterogeneous client-specific truthful response rates тæ ranging uniformly from 0.25 to 0.9. The upper panels correspond to tr = 10,000, and the lower panels to fr 50,000.
Normal
SN Avg Len (10-2)
Scenario
Quantile (r)
Rate (r)
ECP
Avg Len (10-?)
ECP
0.5
0.25
0.930
6.215
0.949
8.792
0.5
hetero
0.946
3.452
0.963
4.566
0.5
0.9
0.996
1.726
0.995
1.909
Now
0.25
0.928
6.411
0.947
9.135
Homo
Tlow
hetero
0.947
3.550
0.962
4.709
Too
0.9
0.999
1.753
0.999
2.093
Thigh
0.25
0.944
6.681
0.939
9.535
Thigh
hetero
0,942
3.686
0.968
4,914
Thigh
0.9
0.996
1.791
0.996
2,232
0.3
0.25
0.930
8.504
0.958
12,090
0.3
hetero
0.951
4.613
0.949
6,404
0.3
0.9
0.996
2.053
1.000
3,228
0.5
0.25
0.972
9.439
0.950
10.891
Hete
0.5
hetero
0.987
5.099
0.952
5.604
0.5
0.9
1,000
2.200
0.996
2,429
0.8
0.25
0.997
19.021
0.966
17.014
0.8
hetero 0.9
1,000
10.029
0.962
8.792
0.8
0.999
3,488
0.990
4.759
0.5
0.25
0.938
6.246
0.949
8,848
Hete
0.5
hetero 0.9
0.958
3,469
0.966
4.788
0.5
0.998
1.735
1.000
2.524
Table S.8: Empirical coverage probabilities at 95% nominal level (ECP) and average confidence- interval lengths (Avg Len 4) for the self-normalized and oracle cormal-based confidence intervals. We fix K 10, Ex, = 1, and tr = 10,000. Three heterogeneity scenarios are considered. Homo, data for each client & are independently generated from N (0, 1). In Hete L, data for each client & are independently generated from N (u 1). where μ4 ~ A 1). Hete D, data are generated from (0 1), 44-11 1), and C(O. 1) across different clients. "hetero" indicates client-specife truthful response rates гı range uniformly from [0.25, 0.9].
28

---

Published as a conference paper at ICLR 2026
Normal
SN
Scenario
Quantile (r)
Rate (r)
ECP
Avg Len (10-2)
ECP
Avg Len (10-3)
0.5
0.25
0.940
7.792
0.950
10.398
0.5
hetero
0.953
4.326
0.953
5,906
0.5
0.9
0.999
2.164
1.000
3,290
Tlow
0.25
0.921
8.657
0.929
11.436
Homo
Now
hetero
0.897
4.788
0.923
6,215
7low
0.9
0.983
2.360
0.969
3,222
Thigh
0.25
0.946
9.811
0.956
13.120
Thigh
hetero
0.955
5.400
0.956
7,578
Thigh
0.9
0.995
2.614
0.999
4.119
0.3
0.25
0.923
12.044
0.947
17.008
0.3
hetero
0.947
6.600
0.957
9.670
0.3
0.9
0.999
3.105
1.000
5.241
0.5
0.25
0.975
12.495
0.942
13.759
Hete L
0.5
hetero
0.971
6.835
0.946
8.276
0.5
0.9
0.952
3.179
0.976
5.085
0.8
0.25
0.999
38.893
0.935
28.048
0.8
hetero
1,000
20.716
0.956
17.456
0.8
0.9
0.998
8.240
0.985
10.558
Table S.9: Empirical coverage probabilities at the 95% (ECP) and average confidence-interval lengths (Avg Len 4) for the self-normalized and oracle normal-based confidence intervals. We fix K = 10, En = 1, and ty 10,000. In Homo, data for each client k are independently generated from C(O, 1). Hete I., data for each client & are independently generated from C(UR, 1), where 1* (0, "hetero" indicates client-specific truthful response rates range uniformly from [0.25, 0.9].
Scenario
Quantile (T)
Rate (r)
CI
CS
Log
tr = 10000
0.3
hetero
0.937(0.0178)
0.969(0.0271)
0.970(0.0352)
Hete I.
0.5
hetero
0.920(0.0142)
0.964(0.0206)
0.974(0.0260)
0.8
hetero
0.943(0.0261)
0.975(0.0395)
0.975(0.0515)
0.5
0.25
0.941(0.0141)
0.977(0.0246)
0.988(0.0362)
Hete D
0.5
hetero 0.90
0.956(0.0073)
0.981(0.0124)
0.992(0.0172)
0.5
0.942(0.0039)
0.994(0.0067)
0.999(0.0097)
by = 50000
0.3
hetero
0.918(0.0093)
0.950(0.0131)
0.962(0.0178)
Hete L
0.5
hetero
(0901().0073)
0.959(0.0101)
0.972(0.0136)
0.8
hetero
0.908(0.0142)
0.968(0.0191)
0.978(0.0270)
0.5
0.25
(.941(0.0061)
0.977(0.0082)
0.997(0.0127)
Hete D
0.5
hetero
0.940(0.0034)
0.977(0.0043)
0.997(0.0063)
0.5
0.90
0.958(0.0016)
0.978(0.0023)
0.998(0.0035)
Table S.10: Empirical coverage probabilities at the 95% nominal level (mean absolute errors 4) un- der varying quantile levels and truthful response rates, evaluated under a decentralized ring topology with different values of tr and fixed number of K 10 clients. In Hete client-specific data are independently drawn from C(pR, 1) with ~ A000, 1). In Hete D, data are generated from (O, 1). 1), and C(O, 1) across different clients. The label "hetero" indicates heterogeneous truthful response rates with F% ranging uniformly from 0.25 to 0.9.
29


Reviews:
[{"review_id": "usa9krXNNq", "reviewer": "Reviewer_Betv", "summary": "This paper considers the statistical inference of federated learning for quantile loss via local stochastic gradient decent under local differential privacy. It proposes a local-SGD algorithm with randomized response on the binary quantile “gradient” supports heterogeneous clients, and outputs a Polyak–Ruppert averaged estimator. Then, its theoretical properties were explored and experimentally verified.", "questions": "See W1-W5.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "S1. This work designs an LDP mechanism that can transform the LDP federated quantile estimation into the non-DP case, and then derive the asymptotic normality and functional central limit theorem of the proposed estimator under non-DP cases.\n\nS2. It is the first weak-convergence result for local SGD without the usual average-smoothness assumption in existing literature", "weaknesses": "My comments focus on presentation and experimental design issues.\n\nW1. The introduction section reads like an extended related work. Missing the importance and/or motivation of the question. I suggest giving 1–2 concrete use cases, or moving a simplified version of Figure 1 to the intro to anchor the problem.\n\nW2. The Methodology section lacks early notation and problem formulation. Even Q is not defined.\n\nW3. In the Experiment section, placing synthetic results in the appendix is a bad idea. There is no direct comparison against prior work (e.g., Liu et al, 2023b). Table 1 lacks guidance on “what good looks like”, nominal 95% coverage with the shortest average CI length? Why not visually highlight when empirical quantiles fall inside/outside CIs?\n\nW4. For the truthful response rate, I am unsure who sets $r_k$, whether it is policy-driven, user-chosen, or purely synthetic. The phrase \"range from 0.6 to 0.9\" in real data is missing a clear definition. I suggest adding the sensitivity of the experiment regarding the truthful response rate.\n\nW5. In Asymptotic Analysis, are the theorems and algorithms (partially) verified in the experimental section? If not, is verification possible?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UGxPlc4ke1", "reviewer": "Reviewer_eKT7", "summary": "This paper propose to use federated learning (FL) for quantile inference under local differential privacy (LDP). The core idea of this paper is to design an LDP mechanism that can transform the LDP federated quantile estimation into the non-DP case. In addition, via constructing valid confidence intervals under LDP, the authors develop a self-normalized inference procedure without requiring direct estimation of the asymptotic\nvariance. The extensive experiments evaluate the effectiveness of the proposed method. Overall, this paper considers a new problem in this area. However, I have some concerns as follows: 1) I cannot see any baselines in the experiments. Although the authors claim that this is the first inference framework for federated quantile estimation, related works are existing and also can be adopted as baselines; 2) The theoretical analysis is not complete, especially for DP. The authors claim that they transform the LDP federated quantile estimation into the non-DP case. Can this transformation maintain DP guarantee? 3) Existing results in simulation cannot evaluate the proposed methods well. Some more experimental should be added.", "questions": "1) I cannot see any baselines in the experiments. Although the authors claim that this is the first inference framework for federated quantile estimation, related works are existing and also can be adopted as baselines; \n2) The theoretical analysis is not complete, especially for DP. The authors claim that they transform the LDP federated quantile estimation into the non-DP case. Can this transformation maintain DP guarantee? \n3) Existing results in simulation cannot evaluate the proposed methods well. Some more experimental should be added.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper propose to use federated learning (FL) for quantile inference under local differential privacy (LDP). The core idea of this paper is to design an LDP mechanism that can transform the LDP federated quantile estimation into the non-DP case. In addition, via constructing valid confidence intervals under LDP, the authors develop a self-normalized inference procedure without requiring direct estimation of the asymptotic\nvariance. The extensive experiments evaluate the effectiveness of the proposed method. Overall, this paper considers a new problem in this area and well written.", "weaknesses": "However, I have some concerns as follows: 1) I cannot see any baselines in the experiments. Although the authors claim that this is the first inference framework for federated quantile estimation, related works are existing and also can be adopted as baselines; 2) The theoretical analysis is not complete, especially for DP. The authors claim that they transform the LDP federated quantile estimation into the non-DP case. Can this transformation maintain DP guarantee? 3) Existing results in simulation cannot evaluate the proposed methods well. Some more experimental should be added.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ieh77h1Ra6", "reviewer": "Reviewer_t17M", "summary": "This paper addresses a significant and challenging problem at the intersection of federated learning, statistical inference, and privacy preservation: performing quantile inference under local differential privacy (LDP). The authors propose a novel federated learning algorithm based on local Stochastic Gradient Descent (SGD), where client gradients are perturbed using an LDP mechanism. The key innovation is a theoretical analysis that establishes the asymptotic normality and a functional central limit theorem (FCLT) for their estimator, even though the quantile loss function is non-smooth. Leveraging this FCLT, they develop a self-normalized inference procedure that constructs valid confidence intervals without needing to estimate the asymptotic variance directly, thus avoiding additional privacy costs. Extensive simulations and a real-data application on government salary data demonstrate the method's effectiveness under various heterogeneity settings.", "questions": "1.  Have you quantified the degree of conservatism introduced by the self-normalization approach in your experiments? Are there potential adjustments or alternative techniques to mitigate this while remaining within the LDP constraint?\n2.  Can your theoretical framework be extended to more complex models, such as high-dimensional or conditional quantile regression? What would be the main challenges?\n3.  How would the algorithm perform under more realistic, asynchronous client participation patterns (e.g., clients joining at random times)? Does the theory accommodate such scenarios?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.  Important and Timely Problem: The combination of federated quantile estimation with the strong privacy guarantee of LDP is highly relevant. Quantile inference is crucial for understanding data heterogeneity (e.g., in healthcare or economics), and LDP is the gold standard for privacy in distributed settings where a central server is not trusted. Tackling inference under these constraints is a substantial contribution.\n2.  Significant Theoretical Contribution: The paper's primary strength is its theoretical analysis. Proving a functional central limit theorem for a local SGD-based estimator with a non-smooth loss function is a non-trivial and valuable advancement. This provides a solid foundation for inference in a setting where standard smoothness assumptions fail.\n3.  Practical and Elegant Inference Solution: The use of self-normalization to construct confidence intervals is a clever and highly practical solution. It bypasses the notoriously difficult problem of estimating the asymptotic variance under LDP, which would typically require additional privacy budget or complex techniques. The resulting method is computationally efficient and integrates seamlessly with the federated learning workflow.\n4.  Comprehensive Experimental Evaluation: The paper validates its theoretical claims with extensive simulations covering a wide range of heterogeneity scenarios (in quantile levels, privacy budgets, and data distributions). The inclusion of a real-world dataset (government salaries) strengthens the practical relevance of the findings. Comparisons against appropriate baselines like DP-SGD and a divide-and-conquer method are well-executed.", "weaknesses": "1.  Reliance on Strong Assumptions: The theoretical analysis relies on assumptions such as the boundedness of the parameter space and the uniform boundedness of the client density functions. While common in theoretical proofs, the practical implications of these assumptions could be discussed further (e.g., how to ensure parameter space boundedness in practice).\n2.  Inherent Conservatism of Self-Normalization: The authors correctly note that self-normalization can lead to heavier-tailed limiting distributions than the standard normal. This often results in conservative confidence intervals (wider than necessary), which can reduce statistical power in hypothesis testing. This is a well-known trade-off of the method but is an important limitation to highlight.\n3.  Centralized Server Dependency: The proposed framework requires a central server for aggregation and synchronization. The algorithm's applicability to fully decentralized or peer-to-peer federated learning environments, an active area of research, is not explored.\n4.  Limited Analysis of Communication/Computation Trade-offs: While different communication strategies (C1, C5, Log) are compared, a more detailed analysis of the total communication cost and wall-clock time (computation + communication) compared to baseline methods would be beneficial for assessing real-world efficiency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4RfhnBgXu5", "reviewer": "Reviewer_6bXx", "summary": "The paper proposes a federated algorithm for quantile estimation and inference under\nLocal Differential Privacy (LDP) using local stochastic gradient descent (Local-SGD). The\nmethod perturbs local gradients via a randomised-response mechanism that ensures client-\nlevel privacy while maintaining statistical efficiency. Key theoretical contributions include:\n(i) an equivalence theorem reducing the LDP problem to a non-private one with adjusted\nquantile levels, (ii) asymptotic normality and functional central limit theorem (FCLT) for the\nnon-smooth quantile loss, and (iii) a self-normalised confidence interval method that avoids\nestimating nuisance parameters. Experiments and a real-data application validate the\ntheoretical results.", "questions": "• Provide a proof sketch for Theorem 2.1 explicitly showing how the randomized response\ninduces the r_k scaling and τ̃_k shift.\n• Expand on the removal of Assumption 1 from Li et al. (2022) — clarify exactly which steps\nare substituted by the new bound E|q̄−Q*|² ≲ γ_m.\n• Include empirical analysis of conservative confidence intervals — e.g., actual vs nominal\ncoverage rates.\n• Test robustness under heavy-tailed client data (e.g., Cauchy distributions) to validate\nbounded-density assumptions.\n• Discuss communication-computation trade-offs more quantitatively under different local\nupdate frequencies E_m.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "• Novel formulation combining federated learning, quantile inference, and local differential\nprivacy — a previously unexplored combination.\n• Theoretical depth: establishes asymptotic normality and FCLT for non-smooth quantile\nloss without the average-smoothness assumption.\n• Elegant reduction (Theorem 2.1) that transforms the privatised problem into an\nequivalent non-private version, enabling classical analysis.\n• Self-normalised inference avoids density estimation, conserving privacy budget and\nimproving practical applicability.\n• Comprehensive experiments demonstrating good accuracy and confidence interval\ncoverage across various privacy levels and heterogeneity settings.\n• Clear discussion of limitations and directions for future research.", "weaknesses": "• The theoretical results depend on several strong regularity assumptions, particularly\nbounded densities and step-size control, which may limit practical generality.\n• The self-normalized approach yields heavier-tailed distributions, producing conservative\nconfidence intervals and lower test power.\n• Dependence on a central aggregation server restricts use in fully decentralized federated\nsystems.\n• Empirical validation is limited to tabular, one-dimensional quantiles — no high-\ndimensional or complex FL tasks are explored.\n\n• Certain proof steps (e.g., adaptation of Li et al., 2022 assumptions) rely on claims without\ncomplete derivation; these could be more explicit for reproducibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YhsZ18Vpg6", "reviewer": "Reviewer_2i31", "summary": "The authors consider the problem of estimating a quantile in a federated learning setting with a local differential privacy constraint.  They present a new algorithm for computing this central quantile and extensive asymptotic analysis that can be used for statistical inference.  They highlight how estimating the quantile in this LDP setting is equivalent to estimating a quantile in a non private setting, but with the distribution shifted in a known way (i.e. solving one gives a computable solution to the other).  To get around estimating nuisance parameters, the authors also present a self-normalized approach based on a functional central limit theorem.", "questions": "Why in Theorem 3.2 do you use $\\ell^\\infty$ instead of $D[0,1]$ (the Skorohod space)?  My understanding is that a partial sum process like this is not tight in $\\ell^\\infty$ so you have to work in $D[0,1]$.  Or if you want to work in $\\ell^\\infty$ you can linearly interpolate the partial sum process so that you have continuous paths (and then you are essentially working in C[0,1]).  But as stated, I'm not sure Thm 3.2 is actually correct since a lack of tightness implies you don't have a functional CLT (in the given space).  But let me know if I'm missing something as it's been a while since I worked on functional CLTs.\n\nI would argue that (2.1) is not the goal, but that the goal is outlined in lines 147-149.  It just so happens that the minimizer of (2.1) gives you the desired parameter,  i.e. the goal is to estimate the quantile of some target population.\n\nThe discussion around Thm 2.1 is quite weak and should be expanded.  \n\nThe discussion/motivation for Thm 3.2 could be strengthened before being presented.  Just saying \"it is necessary to strengthen\" it, isn't really motivating the result.  \n\nLine 48: \"Guarantees convergence\" is a bit vague.  Statistically or computationally?  \n\nLine 61: Nasr should be in parentheses.  In general, the paper's handling of references is really all over the place.  References out of the sentence should be in parentheses, those that are part of the sentence shouldn't be, and you shouldn't have double parentheses like …(e.g.(Zhao…  I suggest the authors go through and clean these up.\n\nLn 124: When talking about estimation accuracy, I think the privacy parameters should be in the rate as well.  The choice of $\\epsilon$ or $\\delta$ play a large practical role.\n\nLn 130: The author should note the $\\theta$ here is the one computed from the data, as opposed to the population level quantity.\n\nLn 151-161: The setup for the estimator is not very clear.  I suppose the authors are saying that at each step each client either updates locally or uses the current estimates from all of the other clients. But it is presented in a confusing fashion.  E.g. \"$x_k^t$ is an independent realization of P_k\", yes, but it is also just one of the datapoints held by a client.  \"$E_m = n$\" is divide an conquer?  So all clients have the same sample sizes?  I suppose $\\mathcal I$ is ordered.  I would just add more details for the reader.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall the paper tackles an interesting problem and presents extensive mathematical results.", "weaknesses": "The organization and writing of the paper is interesting, but feels terse or missing in details at times.  See questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)