Personalized Federated Conformal Prediction with Localization
Authors: Yinjie Min
Chuchen Zhang
Liuhua Peng
Changliang Zou
Venue: NeurIPS 2025 poster ()
Keywords: Personalized federated learning; Conformal prediction; Localization; Conditional coverage
Primary area: probabilistic_methods

Abstract: Personalized federated learning addresses data heterogeneity across distributed agents but lacks uncertainty quantification that is both agent-specific and instance-specific, which is a critical requirement for risk-sensitive applications. We propose personalized federated conformal prediction (PFCP), a novel framework that combines personalized federated learning with conformal prediction to provide statistically valid agent-personalized prediction sets with instance-localization. By leveraging privacy-preserving knowledge transfer from other source agents, PFCP ensures marginal coverage guarantees for target agents while significantly improving conditional coverage performance on individual test instances, which has been validated by extensive experiments.

Personalized Federated Conformal Prediction with Localization
Yinjie Min School of Statistics and Data Science
Chuchen Zhang
School of Statistics and Data Science
Nankai University
Nankai University
nk. a www.yyiiiggmm...om
chuchenz@mail nankain edu,
Liuhua Peng
School of Mathematics Statistics The University of Melbourne liuhua. peng@unimelb edu, am
Changliang Zou t
School of Statistics and Data Science, LPMC and KLMDASR and LEBPS
Nankai University
nk. mkk.chlzo@@@@malccomm
Abstract
Personalized federated learning addresses data heterogeneity across distributed agents but lacks uncertainty quantification that is both agent-specific and instance- specific, which is a critical requirement for risk-sensitive applications. We pro- pose personalized federated conformal prediction (PFCP), a novel framework that combines personalized federated learning with conformal prediction to provide statistically valid egent-personalize prediction sets with lostance-ocaalzztionn. By leveraging
knowledge transfer from other source agents, PFCP ensures marginal coverage guarantees for target agents while significantly improv- ing conditional coverage performance on individual test instances, which has been validated by extensive experiments.
Introduction
The rise of distributed intelligent systems presents a fundamental challenge: heterogeneous data distributions across agents necessitate personalized modeling and analysis to capture agent-specific characteristics. In domains such as medical diagnostics 4T and autonomous driving ( agent- specific feature-label relationships can differ substantially. These discrepancies often render global models ineffective, motivating the development of personalized federated learning (PFL.) [37]. Unlike conventional federated learning, which prioritizes a unified global model, PFL explicitly balances shared knowledge transfer with agent-specific adaptation, enabling personalized models that respect data heterogeneity while preserving privacy.
In risk-sensitive domains, agent-specifie modeling is insufficient. Safety-critical decisions, such as those made by autonomous vehicles under sensor noise or clinical models diagnosing rare patient subgroups, require reliable uncertainty quantification to ensure safety [19]. Conformal prediction (CP) offers statistically valid approach, providing set- set-valued outputs with rigorous marginal coverage
Corresponniigg author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

guarantees. In this paper, we incorporate conformal prediction into the PFL framework for uncertainty quantification to ensure prediction sets remain valid at the individual agent level.
Given a calibration dataset {(X,, Y=) from a joint distribution P, conformal prediction constructs set C. (Xn+1) for test instance Xn+1 at a desired confidence level 1 E (0, 1) such that
P(Y++1 E (X,+1)) 21aa
providing distribution-free coverage guarantees without assumptions on P [34]. Split conformal prediction (SCP) [39], widely used variant of conformal prediction, achieves marginal coverage by leveraging pretrained models and calibration data to compute conformity scores.
Marginal coverage ( ensures average validity over Xn+1 but can mask severe miscoverage for individual instances. In medical diagnosis, undercoverage for patients with high-risk disease features, such as rare genetic markers or specific symptom combinations [19], can lead to delayed diagnoses and life-threatening consequences. A more meaningful guarantee is the test-conditional coverage:
F(Y.+1 (X.+1) Xx+1 x) 2 1- 1-a
for all x, which aligns prediction sets with local uncertainty. This motivates the design of conformal prediction methods with instance-specific adaptation, which conventional CP and SCP do not address.
However, achieving such conditional coverage is generally impossible without strong distributional as- sumptions [2[]. Several conformal prediction methods have been developed that aim for approximate or asymptotic test-conditional coverage by introducing localization and instance-specific adaptation. These approaches either modify the calibration step [25] 26] 20 13 12] or employ alternative score functions (32 3 [4] 6). Several of these approaches rely on estimating local distributional features, such as conditional quantiles [12] or conditional distributions [3], which requires more calibration data to efficiently estimate local properties for instance-specific adaptation.
Problem formulation. Suppose there are K + 1 agents, each with a dataset D3 = {Z = (XR,i, ) containing ng independent samples from distribution PA supported X' y. Without loss of generality, we designate agent K + 1 as the target agent. For simplicity, we omit the subscript K + 1 and denote Pk+1 P and DK+1 =: D Der U Deal, where D.r { (X,, H and Deal = (Z;- (X,, Yo) are training and calibration sets of equal size n. drawn from P Px x Prxx. Our goal is to construct prediction set C2 (Xn+1) for test input X,+1 from the target agent under the collaboration of the other K agents without sharing raw data. Due to heterogeneity across agents (P;s may differ) and variation in the conditional distribution of Y given X within agents, the prediction set should account for both inter-agent and intra-agent heterogeneity.
A natural solution is to apply distributional [ or localized conformal prediction [3] using only the target agent's data. While this ensures marginal coverage and allows for mnstance-specife localization, it overlooks valuable information from other agents and may lead to unreliable prediction sets with poor local or instance-specific performance, especially in regions with limited or sparse calibration data, where estimating local distributional properties is challenging.
Federated learning (FL) enables collaborative model training across agents without sharing raw data, but conventional FL often produces a global model, neglecting agent-specific adaptation. Conformal prediction adds further challenges under FL due to privacy constraints and distributional heterogeneity, which violate conformal prediction's 's exchangeability assumption. Existing federated conformal prediction methods usually assume the test input X,+1 is from mixture distribution
Pw Ex= =k=1 W&P' with Wkk & (n4 + 1). misaligned with agent-specific prediction and lacking marginal guarantees for the target agent. To bridge this gap, we consider constructing conformal prediction sets for a target agent using all agents' data in a grivacy-preserving way, ensuring marginal coverage while achieving conditional coverage closer to the nominal level 1 and producing narrower prediction sets compared to those constructed without using other agents.
Contributions. In this paper, we propose Personalized Federated Conformal Prediction (PFCP), a novel framework that constructs prediction sets with both agent-specific and instance-specific adaptation. The contributions of this paper are:
We formalize generalized localized conformal prediction (GLCP), a novel extension of localized conformal prediction [13 that adopts any conditional distribution estimator and thus accommodates

---

any data type. GLCP enjoys marginal coverage guarantee and localization due to the transformed score that contains conditional property.
We propose personalized federated conformal prediction (PFCP) framework with localization, which achieves marginal coverage for target agent under heterogeneity across agents. More importantly PFCP incorporates data from other source agents to improve the local performance of the prediction sets under the federated learning framework without sharing raw data.
We establish finite sample marginal coverage guarantee PFCP, and theoretical non-asymptoti test-conditional coverage improvement of PFCP through cross-agent knowledge transfer compared with GLCP based solely on target agent data.
We validate PFCP performance through synthetic and
dataset experiments, showing PFCP simultaneously maintains marginal coverage, improves test-conditional coverage perfor- mance significantly and reduces prediction set size compared to GLCPP, which relies solely on scarce calibration and training data from the target agent.
Related work. Lu and Kalpathy-Crrann -ramer 22 first explore conformal prediction in the FL, proposing an aggregation algorithm for prediction set construction, albeit without formal coverage guarantees. Humbert et al. [16[1]] [7] introduce FedCP-QQ. one-shot federated conformal prediction. It computes the a' quantile locally and the 1 - 3 quantile of the K local quantiles on a central server, providing - a marginal coverage with specified a' and 3 under the i.i.d. assumption across agents.
Another line of work, including Lu al. [23), assumes test data from the mixture distribution Ps. = Σ ugPk. In their FedCP method, the central server aggregates sketch of scores from local agents to determine the quantile of conformal scores, ensuring marginal coverage under the FL. exchangeability assumption. Wen
(40 extend this to a decentralized setting with limited calibration data and local communication, while Kang et al. [18] propose robust version of FedCP under Byzantine attacks. However, these methods assume test data originate from a mixture distribution, making them unsuitable for personalized settings. In our PFL setting where test data come from a specific agent, the FL exchangeability [23] assumption is violated, and marginal coverage guarantees may no longer hold for FedCP and its variants.
The most closely related works to ours are Plassier et al. [27 (28). whose methods, referred as CPlab and CPhet respectively, consider agent heterogeneity arising from covariate or label shift. They estimate global and target-agent densities to compute a density ratio, then apply weighted conformal prediction 38 to construct the prediction set. However, their marginal coverage depends heavily on accurate density ratio estimation, limiting robustness. In contrast, our PFCP method guarantees marginal coverage irrespective of estimator quality. Notably, all existing federated conformal prediction approaches have not considered localization or improvements conditional coverage and size of prediction sets.
Methodology
Let s(x, y) and {su(2, k> be pretrained conformity score functions for target agent and source agents. They measure how well label aligns with the predicted value at z, with smaller values indicating higher conformity. Let S4., s& (Xp,s, Yk.)) for i E [nu] be scores from source agent k. and let S. s(X,, Y2) and S
sXX, YY) for i E [n] be scores from target agent. For test point X Px from the target agent and a trial label y. define SM S; for i E [m]. and Sy s(xn+1,10) v)-
Let fxx,) and Sa (z, 8) denote the joint density functions of (X1, S1) and (XR,1, 1r respectively, Denote the marginal covariate density under the target distribution P by f(x) and under Ps by fa(x) for & E [K]. Let f(s z) = f(r fz)((=) denote the conditional PDF of S₁ given X1 = x, and F(s z) the corresponding conditional CDF.
2.1 Generalized localized conformal prediction based solely on target agent data
In this section, we propose the Generalized Localized Conformal Prediction (GLCP) framework, which relies solely on target agent data. To account for heterogeneity in Pyix and capture local information at a given point z, we estimate the conditional distribution of the score S given X1
3

---

using dataset Di. {XX; - Let F(s x) = A( )) be the estimator of F(s Ix), where A(-) represents generie conditional distribution estimation algorithm, e.g., engression Shen and Meinshausen 35 Define V = =(S X;) for i E [n + 1]. The GLCP set is defined as
ConcP(Xnn11) = {u: V @(1 a: (n +) (I ōVp x 51)) 61))3
(2)
where de denotes the point mass at v and Q(1 a; denotes the (1 a) qquantile of the distribution the second argument. The following lemma follows directly from the basic validity guarantees of conformal prediction theory [39].
Lemma 2.1. data pairs (X Vi)...., (X,, [x V2+1) are i.d. from P. and F(s x)) is trained using dataset Der {(X,, (A))E we have IP P(Yn+1 E CGLCP (Xn+1)) 2 -0.
Actually, GLCP is designed to generalize and unify existing approaches. If we choose F(s Ix) to be the Nadaraya- Watson estimator [15,, GLCP reduces to the localized conformal prediction in Guan [3]. If the score function is specified s(z y) y. GLCP becomes equivalent to distributional conformal prediction in Cheroozuukvv et al. [5] Notably, GLCP accommodates any conditional distribution estimator and thus accommodates any data type, providing greater flexibility. For instance, one may adopt estimators tailored to high-dimensional covariates, thereby alleviating the curse of dimensionality inherent to the Nadaraya- Badaraya-Watson Watson estimator used in Guan [1].
GLCP naturally supports localization and instance-specific adaptation, emerging seamlessly from its formulation. Given score function s(.; )), the oracle conditional conformal prediction set is defined as {y : sz, y) I. Q(1 - a, F(s r))}. which is equivalent to {y : F(s(x, y) x) S 1 a}. The transformed score V." serves
estimator of F(S X;), and it converges in distribution U'(0, 1]
if ((s) x) is consistent estimator of F(s x). In this case, the empirical quantile of {V?))*+
converges to 1 - a, which ensures that the GLCP set a DGGLP(Xx++1 in (2) approximates the oracle conditional prediction set. Therefore, the GLCP inherits the asymptotic conditional property.
However, accurately estimating F ((s x) at each x E X' requires a large sample size, and the estimation complexity grows with the dimensionality of the covariates. When the target agent has limited data, GLCP may produce unreliable or excessively large prediction sets in certain regions.
2.2 Personalized federated conformal prediction
In federated learning, directly pooling data across agents to construct prediction sets is infeasible due to privacy constraints and data heterogeneity, which violates exchangeability required by conformal prediction. Existing federated conformal prediction methods either assume no heterogeneity among agents or the test data are from a mixture distribution over all agents, which incompatible with our personalized setting. Moreover, these approaches do not account for localization or instance-specific adaptation. To bridge this gap, we propose Personalized Federated Conformal Prediction (PFCP).
Define weight T&k = nx/ Ei= ne for k [K] and the mixture density as fmix (x, 8) EK= fz s) with corresponding marginal covariate density fu)x(x)). Let fuix(( | x)) Smix (2, 8)/
(0)/fmxx(a fmix(x)), with corresponding CDF Fmix x). Define the density ratios as r(x) f(x) (x)) 5(1,5)= f(x f0)/mixxxz (I, 8) and r(s x)) f(s x)/ (s x).. The key idea
of PFCP is to derive an enhanced estimator of the conditional distribution F(s x) that will be utilized in the transformed score V." by strategically incorporating auxiliary data from other agents. Regarding the following equivalent reformulations of F(s) z):
F(s x) E5-F(1/=) SS _ f(t ' Smixet x)r(( x)dt
EE (slan) ( [1(S S
s)r(S s
Therefore, we can derive alternative estimator for F(s x)) using estimators of Fiix((s x)) and corresponding density ratio r(s x:) via (3. While F(s x)) is estimated locally using algorithm A(.)
in Section 2.1 estimating Fmix (s æ)) requires datasets from multiple source agents under federated setting. This achieved using algorithm A(.), which integrates local applications of algorithm A(-) on each source agent locally with federated learning algorithm (c.g., federated averaging (24) as

---

detailed in Section A.1.100 of the Appendix. On the other hand, estimating r(s x) typically requires computing joint and marginal density ratios first, and taking their quotient afterwards, which may introduce errors. To circumvent this, we propose using a weight function we(x) to aggregate the two (s 2)) estimators { bypassing r(s ar) estimation while using an estimator of r(x, 8) to formulate the prediction set. The estimation of r(2, s) is derived using algorithm Aar(-), which is the federated implementation of density ratio estimation algorithm as detailed in Section A.1.2 Finally, we introduce Aagg (+) as the aggregation algorithm based on (5) that combines two estimators of the conditional distribution Fs z) and the estimator of the density ratio r(z, s).
Due to space constraints, we present details of A(-) (uses local data at target agent) and AA.)) (uses data from K source agents) in Section A.I.I. of the Appendix. The
density ratio
estimation algorithm (-r(-) uses data from both target and source agents is detailed in Section A.1.2 of the Appendix. The aggregation algorithm A.ge(-) used to derive the enhanced estimator of the
conditional distribution F(s x)) is detailed in Section 2.3 Let Dr-
th for l E [K]. {(Xx, R,ú, Sk,))
The PFCP procedure is summarized as below:
Step 1. Train F(s x) = A(D,r) as an estimator of F(s x) using Dir in the target agent. In addition, train Fmix(( x) A(DI,... Dx) estimator of Fiix(s x) using Di,..., Dx from source agents under the FL setting.
Step 2. Train f(x, = Aar (Der, DI.... DK) as an estimator of r(z s) under the FL setup using Dur: DI...., Dx from both target and source agents.
Step 3. Use an aggregation algorithm that combines F(s xx), Fini(( | æ)) and F(z, to produce Fagg (s z²) Aags (F z), (mix f), s)) as enhanced estimator of F(s (z).
Step 4. Define transformed scores on ceal U ((X 8)} as VV The PFCP set is then defined as
agg,s Fage(Sy (Sy X2)) for E [n+ 1].
T2
CPFCP (Xa+1) = {u: V ngg.n+1 n+ S (1(1-cr; + 1)- Σ
8y Vas. + 51 b1))}
(m
The algorithm following these steps provided in Algorithm 5 in Section A.1.4 of the Appendix.
In only data from the target agent is used for calibration, which guarantees marginal coverage regardless of the data heterogeneity across agents. Furthermore, while F(s z)) trained on limited target data often performs poorly, the aggregated estimator (a | x:) typically achieves higher accuracy. By substituting [(s I) with F (a (s xr). we obtain superior estimates that simultaneously improve conditional coverage accuracy and yield tighter prediction sets.
Notably, A(-) and (-) are federated learning algorithms involving data communication among agents, while A(-) and Auga(-) execute locally. Specifically, A(-) operates locally on the target agent's 's data. In contrast, A(-) and (-r(-) collaboratively train models across source agents using FedAvg, exchanging only model updates-not raw data. Finally, Augs (-) ensures privacy by relying solely pre-computed estimates of conditional distributions and density ratios, requiring no access to raw data during aggregation.
x)) can be performed using either all available source agents or a selected subset of agents that are most relevant to the target domain. The criteria and implementation details for source agent selection are discussed in Section A.3 of the Appendix.
Remark 2.2. The estimation of Fmix(( |
2.3 Aggregate Conditional Distribution Estimators
This section formalizes a general framework for obtaining the aggregate conditional distribution estimator Fage(s i)) given E Fmix(( x). F(z, and F(s x). Without loss of generality, we assume [(s .r = 0 for all O. We further define F(x) B (sla) ( and F(s x) F(I, s))(()). Then, both E. Eg (S < s) and Es (1(S S s)F(S x)) can serve as estimators of F(s z). Introducing weight function (x)), aggregate these estimators through
5

---

the following combination:
Fw Agg. 1) (1 (1 1(S I ()
s- (1 S s)F(S) x
In framework, only f(z, s) is available, and computing f(s x) requires estimating r(z). potentially introducing additional error. To avoid this, we adopt the simplification w(x) = f((x)
leading to the following estimator:
Fagg(s 2) (1 f(x)} P(s L(S S + Pm {1(S S ( (5)
In practice, the scalar term 1/11+ F(x)) need not be estimated explicitly, the estimator is obtained by weighting samples from the two distributions and computing the proportion below the threshold 8.
When integrated with the aggregation scheme based on engression
[35 developed in Section A.1.3
of the Appendix, the estimator Fagg( (s x)) in (5) admits a simple form.
Remark 2.3. Other choices of (x) are possible, but setting w(z) = f(z) circumvents the need to explicitly estimate r(x) and has been found perform well in numerical experiments.
2.4 Theoretical Analysis
In this section, we establish marginal coverage of our proposed PFCP and its test-conditional coverage property compared to GL.PP. Let Vagg,i = F Fazz(* (*,,, Y2) X:) for E [n+1], where Vagg,i = V agg,i
for E [n] and Vagg.n+1 = V "ag..++-- The next theorem offers the marginal coverage guarantee.
Theorem 2.4 (Marginal Validity). Assume (X1,Y1), ...,, (Xns Yn), (Xn+1, Yn+1) are i.i.d. f.i.d.from P. ()) is pre-trained using training data, which is independent of the calibration dataset. and Faza(s (
Then P(Yn+1 E CPECP (Xa+1))
In addition, if ties between Vagg,1...., Vogg,n+1 Occur
with probability zero, then
P(Y C.PCP (X,+1)) E (8|
Remark 2.5. Theorem 2.4 al4 holds if F Fagg(s |x) relies on calibration and test data but is invariant
the permutation the combination of calibration and test data. However, in personalized federated setting, this will incur great data communication.
Define 81 (x; F) = drv (F( 1). provides upper bound the test-conditional miscoverage error of GLCP.
x)). where drv denotes the total variation. The next lemma
Lemma 2.6. Under the conditions of Theorem 2.4 let Fv be the CDF of (s(, Yi) X1). which is supported on [0, 1] and assumed to be continuous. Let = [(1 a)(n + 1)1 1)//n. Then, for any
5 E 10,
[P(Y++1 & GGLCP (Xx+1) Xn+1 = =x)- a
<max [11 - a (g <) + 6),11 - -a- -- Fy 813 + (x;
where = (22) In
The result of APFCP (Xn+1) can be derived by replacing F(s x)) with Fagg(s 2). and Fy with the CDF of Fage (s( (X1, Yi) X1). While the first term captures the global calibration error, the second term quantifies localized distributional estimation accuracy. The superiority of ČPFCP (Xn+1) lies in its reduction of (2;; F compared to 51 (z; F), achieved through its federated learning architecture that uses f(xr, s). This comparison will be formally established in Theorem 2.8
Assumption 1. For each æ E X, the score s(x, Y1) on the target agent is bounded in [0, Mx)), and [( x) 2 (/L)(1)) for (0, (x)), where M(x) < co and Lx(x) < sc.
Assumption For each x E X, there exist constants inf (x)) K mIf((z)} 2 (/Lx(x); (ii) for any s, there exists A E [K] satisfies f(s
Lo(x), Lx (x) < xc, such that: (i)
r)/fx(8 x) < Lz()..
Assumption 3. The support of s under F(xx, s) is bounded in [0, (z)). For any xr E X, there exists E [0, M2 (z)) such that F(x, 8) > and f(x,,) 8) is Lipschitz continuous in s on its support with constant zF,

---

Remark 2.7. If the distribution of Y1 conditional on X1 = x is bounded with density bounded away from zero, and the conformity score function s(z, y) = --(2)), where the pretrained predictor p(x) estimates B(Y, x1= 11=2 x)) and lies within the support of Y, then Assumption I holds. Assumption has two key implications. First, each source's 's covariate distribution contributes nontrivially to the overall mixture (non-negligibh mixture weight). Second, the scores for given x in the target agent are also observable in at least one source agent. Intuitively, this requires the source agents to possess richer more informative data than the target agent, or more precisely, what we truly require is that the mixture of source distributions contains sufficient information about the target, meaning the Σk=1 TK(s [() x)) has a positive lower bound, which is a necessary condition for the source data meaningfully assist the target. Finally, Assumption 3, which requires the estimated density ratio f(z,8) to have bounded support and be Lipschitz continuous, can be enforced in practice by truncating extreme score and ratio values, a straightforward step when employing continuous neural networks or predictors with bounded parameters.
Define ((z; f) = Jo (misx(| (s1 2)) F2,, s) - r(z,, s)sss, and 82(2; Frni) as the Ly-dinnance between the density of Fmii(e x)) and Fmix(- 2), and (2(z)) - U00 ((s x))d) -1/2,
Theorem 2.8. Under the conditions of Theorem 2.4 Assumptions 1-3, we have L2(x;;) < dc and
Fagg(s (s| I) in Section 2.3 satisfies that 26 (z; F Fagz) is bounded by
28 (1; Fagx) S (1+ 22 (2:; F) 8( + (F(x) - r(()) L2(2;
Ta(o)) (1;; (orr::Pmii Fmix)]
Theorem 2.8 implies that 6, (a; Fagz) < 8, (z; F) holds when (2 ()2 Fmixx
+ + +((; +))
r(x)] 2f(z)) (z; F). In practice, this condition is attainable under mild settings. With access to sufficient data from source agents, the estimation error 52(2; Fmix) becomes asymptotically negligible. Moreover, density ratio estimation can be framed as a binary classification task, which is often easier to learn than conditional distribution estimation under similar conditions. Taken together, these observations suggest that Fage(( (s 1r) is likely to yield improved estimation of F(s x)) over F(s æt), and thus PFCP is expected to outperform GLCP.
On the other hand, if the density ratio is poorly estimated, the federated conditional distribution estimator may not more accurate, i.e. 81 (z; Fagx)) 2 8, (-r; F). then the aggregation may not result in any improvement. Furthermore, if the source distribution significantly differs from the target distribution, for instance, when the support of P and any Ps do not intersect, the information from source agents cannot help improve the conformal set the target agent. Additionally, when the data on the target agent is extremely limited, insufficient to yield a reliable estimate of the conditional distribution F(s z), both GLCP and PFCP may fail to deliver conformal sets with satisfying test-conditional coverage, though PFCP may still outperform GLCP in this regard.
3 Experiments
We conduct experiments on synthetic data and five real datasets demonstrate marginal coverage validity of our proposed PFCP compared to FedCP (33, FedCP-QQ [6), CPlab [27 and CPhet (8
and its local coverage improvement relative to GLCP. The nominal coverage level is set to 1
90%. For a prediction set (X,+1)) the following criteria are used for comparison: (1) marginal
coverage PY E (X((x)+)): (2) test-conditional miscoverage error E [ E (X (xx+1)
X (1 - a)] } and (3) average size of the prediction set. All simulation results are based on 100 replications. The codes are available in the repository httpp::// //t::////tthbtt. wwmm/OsiinMin/PPe
3.1 Synthetic Data
In our synthetic experiments, the regression model for target agent data is Y; = (x(,)) + c(X,). where u(z)= E(Y; X; 2) is the conditional mean function and the residual c c(X,) may depend on X,, For agent E [K] the model is Y2, = (4x( + (X..)) We set ((x) = Di= x, and (z) 2/3 Ei= xy for (#1,..., ata) E Rd. The covariates X; and XR.; are sampled from the d-dimensional standard normal distribution (2 Ia). We consider two setting of heterogeneity across agents:
7

---

S1: c(x) ~ No. E /al/vvd /2d (x(x) NO 7/ E lol/VVd 2dd (Ein x4) ).
S2: (x) ~ NV cos (Ei= (). (x) N
(0. 7M cos
Here, Nhe controls the level of heterogeneity across agents. The target agent dataset is divided into four parts: predictor training set (I), a conditional distribution training (II), calibration dataset (IID, and test dataset (IV), where the first three datasets share the same data volume n. To ensure each agent has the same volume of available data, the source agent data set divided into two parts (I) and (II), each with data volume 3n/2.
We set 100 in this experiment. The pretrained predictors Ti(-) and (T)())) obtained using neural networks with same hidden layers 30 -> 30. The score function defined as the residue s(r, y) ly ](x:) and (,, y) ly (.)]. We report the experimental results evaluating the performance of PFCP across different covariate dimensions, with d E {10, 15, 20, 25, 30). Additional simulations exploring different source agent heterogeneity, quantities of source agents, and required coverage level and source agent selection methods can be found in Section A.4 in the Appendix. We consider two different sampling schemes for generating Vk- applied under settings S1 and S2. Specifically, in scenarios S1.1 and $2.1, we sample Y ~ (0,8, 1], while in S1.2 and S2.2, we use Tkk U [[1, 1.4). We compare two density ratio estimation methods within the PFCP framework. PFCP1 uses logistic regression, while PFCP2 employs neural network with a 30-unit hidden layer, followed by Platt calibration. case PFCP2, the training data are randomly split 9:1 ratio for classifier training and Platt calibration.
Results. The top row of Figure presents the marginal coverage rates of all methods across four configurations. It shows that both GLCP and PFCP achieve valid marginal coverage, whereas FedCP and FedCP-QQ fail to meet this guarantee. Consequently, the bottom row of Figure focuses exclusively on GLCP and PFCP for test-conditional miscoverage error comparison. The results demonstrate that PFCP consistently outperforms GLCP in terms of conditional coverage, with the performance gap getting larger as the covariate dimension d increases, highlighting the robustness of PFCP in high-dimensional settings. Additionally, PFCP2, which employs a calibrated classifier via Platt scaling after neural feature extraction, generally exhibits slightly lower test-conditional miscoverage eтror than PFCPI, which uses the logistic regression-bee based density ratio estimation method, especially in high- sish-immnssioal settings.
GLCP
PFCP1
PFCP2
.... FedCP
FedCP.00 QQ
CPlab
-k
CPhet
51.1
S1.2
S2.1
$2.2
Marginal Coverage
0.95
0.95
0.95
0.85
0.85
0.85
0.85-
0.80
0.80
0.80
0.80
0.75
20
25
30
0.75
25
0.75
0.75
S1.1
$1.2
52.1
52.2
Miscoverage
0.09
0.09
0.07
0.07
0.07
0.07
0.05
0.05
0.05
10
15
20
25
30
10
15
20
25
30
10
15
20
25
30
10
15
20
25
30
Figure 1: Marginal coverage (top row) and lest-conditional miscoverage error (bottom row) under scenarios S1.1, S1.2, 12.1, and S2.2.
3.2 Real data analysis
We evaluate our proposed PFCP method on five public-domaie regression datasets also considered by
32 33 16]: physicochemical properties of protein tertiary structure (BIO) [30, bike sharing
(BIKE)
8 communities and crimes (CRIME) (31. Tennessee's student teacher achievement ratio
(STAR) D. concrete compressive strength (CONCRETE) 43), and derma image classification
dataset (DERMA) 42
8

---

We first partition each dataset into multiple agents, with the partitioning strategy detailed in Section A.5 in the Appendix. We set n 50 for the CONCRETE dataset and 100 for other four datasets. In each repeated experiment, the number of source agents (excluding the target) varies across benchmark datasets: 20 for BIO, 12 for BIKE, 4 for CRIME, for STAR, 5 for CONCRETE and for DERMA. To ensure fair comparison, all agents use neural networks with identical architectures for point estimation, with hidden layers of size 30 + 30, and the engressor [35] is trained using the same network structure with hidden layers of 100 + 100, all trained for the same number of epochs and learning rate. For regression tasks, the score function still uses the residue score. For classification problem, the point estimate and fa output a probability vector, and the score is denoted as s y) = 1 {f(z))y, 8k (I, y) = (Px(z)) where the subscript indicates the y-th element of the probability vector.
The Marginal Coverage part of Table summarizes the marginal coverage rates of four methods across 100 repeats. Both GL.PP and PFCP achieve reliable marginal coverage close to the required a = 90% level, while FedCP, FedCP-QQ. CPlab and CPhet deviate significantly from the target in most scenarios. This highlights substantial data heterogeneity across agents in real-world settings, where achieving nominal marginal coverage relies on rare coincidental alignment-a high-risk strategy lacking robustness. Results violating marginal validity are boldfaced and marked with x. The conditional miscoverage error requires the calculation of conditional coverage rates, with the computation method detailed in Section A.55 in the Appendix. Additionally, the Miscoverage and Size part of reports the conditional miscoverage errors and mean set size of GLCP and PFCP. Across all scenarios, PFCP consistently outperforms GLCP, with improvement percentages shown in parentheses.
Results:
Table 1: Marginal Coverage, miscoverage rate, and size of prediction sets.
Dataset
GLCP
PFCP
FedCP
FedCP-QQ
CPlab
CPhet
Marginal
BIO
0.902
0.903
0.993
0.970
0.978
0.981
BIKE
0.900
0.900
0.885 x
0.890
0.885 x
0.883x
CRIME
0.900
0,896
0.866x
0,852x
0.862
0,864x
STAR
0.898
0.899
0.897
0.892
0.897
0.898
CONCRETE
0.903
0.905
0.947
0.963
0.949 x
0.946 x
DERMA
0.895
0.899
0.824
0.809
0.880
0.868 x
Miscoverage
BIO 0.0315
0.0199
0.0941
0.0753
0.0822
0.0844
BIKE
0.0234
0.0193
0.0678
0.0647
0.0681
0.0690
CRIME
0.0387
0.0268
0.0426
0.0495
0.0439
0.0429
STAR
0.0392
0.0244
0.0502
0.0507
0.0498
0.0493
CONCRETE
0.0366
0.0238
0.0582
0.0675
0.0600
0.0580
DERMA
0.0300
0.0230
0.0884
0.0976
0.0616
0.0659
Size
BIO
0.5141
0.5032
0.9991
0.7087
0.8054
0.8271
BIKE
4.0968
3,9645
4.0044
4.0910
3.9959
3.9671
CRIME
5.1830
4.4961
3.9176
3.7724
3,8802
3.8855
STAR
48.4987
43.2931
42.7556
42.1185
42.8081
43.0090
CONCRETE
34.4859
28.3797
31.7115
38.9746
35.15-15
34.6905
DERMA
2.4926
2.4430
1.2490
1.1943
1.5649
1.4810
To intuitively demonstrate PFCP's improvement in conditional coverage over GLCP, Figure visualizes the conditional miscoverage errors across subspaces for the CONCRETE dataset (results for other datasets are in the Appendix, see Figure [2 16). The figure shows that GLCP exhibits highly heterogeneous coverage across subspaces, while PFCP achieves more stable and better controlled miscoverage rates closer to the target
To further investigate how the number of source agents impacts target agent performance, we hypothesize that increasing the number of source agents with distributional alignment to the target should enhance task performance, while using an optimal subset of source agents becomes critical when distribution shifts are significant. Figure demonstrates this phenomenon on the CRIME dataset with a = 0.1 (with n = 75 adjusted to accommodate more source agents), where target performance improves steadily as the number of source agents grows. Results for other datasets are provided in the Appendix (see Figure 17 to 2Tp, reinforcing that task task-specific distributional alignment-rather than simply maximizing source agents-drives drives reliable performance gains.

---

Required
GLCP
PFCP
0.30
Miscoverage of GLCP 0.037
Miscoverage of PFCP 0.024
0.25
coverage
0.20
0.10
0.05
0.00
1
3
4
6
8
9
10
Subset Group Index
Figure 2: Conditional coverage all subspace for CONCRETE dataset.
a= 0.1)
PFCP
GLCP
PFCP
FedCP
FedCP-QQ
CPlab
CPhet
0.032
scsoveaaee GLCP 0.050
Mean Size of GLOP 7.742
7.0
Marginal Coverage
0.90
Miscoverage
0.028
6.5
0.88
0.026
0.86
0.024
0.022
Agents
Number Agents
Agents
Figure 3: Impact of source agent quantity on target task performance for CRIME dataset.
Conclusion
We present Personalized Federated Conformal Prediction (PFCP), novel framework that addresses both agent-specific and instance-specific uncertainty quantification in federated learning. By in- tegrating localized conformal prediction with personalized federated models, PFCP guarantees marginal coverage for target agents while significantly improving test-conding cost-oonitional coverage through prreey-prsserving.. knowledge transfer from source agents. Theoretical analysis and empirical results demonstrate that PFCP achieves tighter prediction sets and better conditional reliability compared to methods relying solely on local data. This work bridges the critical gap between federated learning's privacy constraints and risk-sensitive applications' demand for instance-aware uncertainty quantifica- tion. The framework's 's robustness to distribution shifts and its compatibility with arbitrary data types position PFCP principled solution for safety-critical federated systems requiring personalized reliability guarantees.
Acknowledgments and Disclosure of Funding
Peng was supported by ARC (Grant No. LP240100101). Zou was supported by the National Key R&D Program of China (Grant No. 2022YFA and the National Natural Science Foundation of China (Grant Nos. 12231011 and 12531011).
References
[1] Charles M. Achilles, Helen Pate Bain, Fred Bellott, Jayne Boyd- Jeremy Finn, John Folger, John Johnston, and Elizabeth Word. Tennessee's student teacher achievement ratio (star) project. Harvard Dataverse, 1:2008, 2008
[2] Eric Bazan, Petr Dokládal, and Eva Dokladalova. Quantitative analysis of similarity measures of distributions. In British Machine Vision Conference (BMVC), 2019.
10

---

[3] Christopher Briggs, Zhong Fan, and Peter Andras. Federated learning with hierarchical cluster- ing of local updates improve training on non-iid data. In 2020 international joint conference on neural networks (IJCNN), pages 1-9. IEFE, 2020.
[4] Sung-Hyuk Cha. Comprehensive survey on distance/similarity measures between probability density functions. City, 1(2):1, 2007.
[5] Victor Chernozhukov, Kaspar Wüthrich, and Yinchu Zhu, Distributional conformal prediction. In Proceedings of the National Academy of Sciences, volume 118. National Acad Sciences, 2021.
[6] Tiffany Ding, Anastasios Angelopoulos, Stephen Bates, Michael Jordan, and Ryan Tibshirani. Class-conditional conformal prediction with many classes. In Advances in Neural Information Processing Systems, volume 36, 2024.
[7] Fernanda Famá, Charalampos Kalalas, Sandra Lagen, and Paolo Dini. Measuring data similarity for efficient federated learning: a feasibility study. In 2024 International Conference on Computing, Networking and Communications (ICNC), pages 394-400. TEFE, 2024.
[8] Hadi T and Joao Gama. Event labeling combining ensemble detectors and background knowledge. Progress in Artificial Intelligence, 2:11 2:113-127, 2014.
[9] Lidia Fantauzzo, Eros Fani, Debora Caldarola, Antonio Tavera, Fabio Cermelli, Marco Ciccone, and Barbara Caputo. Feddrive: Generalizing federated learning to semantic segmentation in autonomous driving. In 2022 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 1504-11511. IEEE, 2022.
[10] Yann Fraboni, Richard Vidal, Laetitia Kameni, and Marco Lorenzi. Clustered sampling: Low-variance and improved representativity for clients selection in federated learning. In International Conference on Machine Learning, pages 3407-3416. PMLR, 2021.
[11] Swier Garst and Marcel Reinders. Federated k-means clustering. International Conference on Pattern Recognition, pages 107-122. Springer, 2024.
[12] Isaac Gibbs, John J Cherian, and Emmanuel J Candès. Conformal prediction with conditional guarantees. arXiv preprint arXiv:2305. /2616, 2023.
[13] Leying Guan. Localized conformal prediction: A generalized inference framework for confor- mal prediction. Biometrika, 110(1):33-50, 2023.
[14] Chirag Gupta, Arun K Kuchibhotla, and Aaditya Ramdas. Nested conformal prediction and quantile out-of-bag ensemble methods. Pattern Recognition, 127:10 2022.
[15] Peter Hall, Rodney CL Wolff, and Qiwei Yao. Methods for estimating conditional distribution function. Journal of the American Statistical association, 94(445):154-163 1999.
[16] Pierre Humbert, Batiste Le Bars, Aurélien Bellet, and Sylvain Arlot. One-shot federated conformal prediction. In International Conference on Machine Learning, pages 14153-14177.
PMLR, 2023.
[17] Pierre Humbert, Batiste Le Bars, Aurélien Bellet, and Sylvain Arlot. Marginal and training- conditional guarantees in one-shot federated conformal prediction, 2024.
[18] Mintong Kang, Zhen Lin, Jimeng Sun, Cao Xiao, and Bo Li. Certifiably Byzantine- Federated Conformal Prediction. In Proceedings of the 41st International Conference on Machine Learning, pages
PMLR, 2024.
[19] Benjamin Kompa, Jasper Snoek, and Andrew Beam. Second opinion needed: communicating uncertainty in medical machine learning. NPJ Digital Medicine, 4(1):4, 2021.
[20] Jing Lei and Larry Wasserman. Distribution- free prediction bands for non-parametric regression. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76(1):71-96, 2014.
[21] Jing Lei, James Robins, and Larry Wasserman. Distribution-free prediction sets. Journal of the American Statistical Association, 108(501):278-258) 2013.

---

[22] Charles Lu and Jayasheree Calpathy-Crames Distribution-Free Federated Learning with Conformal Predictions, 2022.
[23] Charles Lu, Yaodong Yu, Sai Praneeth Karimireddy, Michael Jordan, and Ramesh Raskar, Federated conformal predictors for distributed uncertainty quantification. In International Conference on Machine Learning, pages 22942-22966 PMLR, 2023.
[24] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. conmunication-efficcee learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017.
[25] Harris Papadopoulos, Alex Gammerman, and Volodya Vovk. Normalized nonconfornity measures for regression conformal prediction. In Proceedings of the IASTED International Conference on Artificial Intelligence and Applications, pages 64-69, 2008.
[26] Harris Papadopoulos, Vladimir Vovk, and Alex Gammerman. Regression conformal prediction with nearest neighbours. Journal of Artificial Intelligence Research, 40:815-840, 2011.
[27] Vincent Plassier, Mehdi Makni, Aleksandr Rubashevskii, Eric Moulines, and Maxim Panov, Conformal prediction for federated uncertainty quantification under label shift. International Conference Machine Learning, pages 27907-279477. PMLR, 2023.
[28] Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, and Maxim Panov. Efficient conformal prediction under data heterogeneity. In International Conference on Artificial Intelligence and Statistics, pages 4879-4887. PMLR, 2024.
[29] John Platt al. Probabilistic outputs for support vector machines and comparisons regularized likelihood methods. Advances in large margin classifiers, 10(3):61-74, 1999.
[30] Prashant S Rana. Physicochemical properties of protein tertiary structure data set. UCI Machine Learning Repository, 2013.
[31] Michael Redmond. Communities and Crime Unnormalized. UCI Machine Learning Repository, 2009. DOI: http://4o..og//102433//CCCCC
[32] Yaniv Romano, Evan Patterson, and Emmanuel Candès. Conformalized quantile regression. In Advances in Neural Information Processing Systems, volume 32, 2019.
[33] Matteo Sesia and Yaniv Romano. Conformal prediction using conditional histograms. Advances in Neural Information Processing Systems, 34:6304-6315, 2021.
[34] Glenn Shafer and Vladimir Vovk. A tutorial on conformal prediction. Journal of Machine Learning Research, 9(3), 2008.
[35] Xinwei Shen and Nicolai Meinshausen. Engression: extrapolation through the lens of distribu- tional regression. Journal of the Royal Statistical Society Series B: Statistical Methodology, page qkae 2024.
[36] Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation in machine learning. Cambridge University Press, 2012.
[37] Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. Towards personalized federated learning. IEEE transactions on neural networks and learning systems, 34(12):9587-966 2022.
[38] Ryan Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal prediction under covariate shift. Advances in neural information processing systems, 32, 2019.
[39] Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world. New York: Springer Science & Business Media, 2005.
[40] Haifeng Wen, Hong Xing, and Osvaldo Simeone, Distributed Conformal Prediction via Message Passing, 2025.
12

---

[41] Jie Xu, Benjamin S Glicksberg, Chang Su, Peter Walker, Jiang Bian, and Fei Wang. Federated learning for healthcare informatics. Journal of healthcare informatics research, 5:1-19, 2021.
[42] Jiancheng Yang, Rui Shi, and Bingbing Ni. Medmnist classification decathlon: A lightweight automl benchmark for medical image analysis. In IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 191-195, 2021.
[43] I-C Yeh. Modeling of strength of high-performance concrete using artificial neural networks. Cement and Concrete research, 28( 1)) 1998.
[44] Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 694-699, 2002.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The formulation of GLCP can be found in section 2. and PFCP can be found
in section 2.2 Theoretical analysis can be found in section 2.4 and experiments can be found section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of our proposed method are discussed section A.6
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [Yes]
Justification: All assumptions are represented in section 2.4 and all proofs can be found in section
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed descriptions and formulations required reproduce the main experimental results, including information on the model, datasets, baselines, and evaluation metrics. Additionally, we present the algorithms and implementation details in section A. to offer clearer implementation guidance.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
The datasets utilized in this study publicly accessible via the cited refer- ences, and the implementation code is available upon request.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co.
sai.iiooisiiiiycc for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7 //nips. www.cc/pwwppppccccccceoeooooeeee htte://wiisss/eeee www.eeeuwisssoonnooooooooooooeee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:The experimental settings detailed in Section and additional experiments are presented in Section A.4 to provide further insights into the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: For experimental results both synthetic and real-world datasets include sigma error bars in the coverage rate and prediction interval size plots to illustrate variability. Due computational constraints, we omit error bars of miscoverage plots which would require extensive repeated experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Experiments Compute Resources are clearly stated in Section 3
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps:/nneurips http://////euippsspppppii/////// hctpuuliic/thhcssuue.eee
Answer: [Yes]
Justification: We have carefully reviewed the NeurIPS Code of Ethics and have ensured that all aspects our research fully comply with its guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The socictal impacts of our proposed method are discussed in Section A.7
17

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We believe that our work poses no foresceable risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets used in our experiments are properly cited, and source links are provided for reference.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.com//aaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
Our code is well documented with the documentation provided.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our work does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: We do not use LLM to impact the core methodology, scientific rigorousness, originality of the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurl)) htte::///earrrppccc/oooofrrrere/ /c/////oeeeeeeeee/222////.. for what should or should not be described.
20

---

A Technical Appendices and Supplementary Material
A.1 Implementation Details
A.1.1 Engression as Conditional Distribution Estimator
Engression consider the problem of estimating the conditional distribution Y X via a generative approach. Let X X C CR, Y y S R and E & C Rd be the input, output variables and random noise, respectively. Engression constructs a generator G(z,e; ()) X' x ع + y that maps covariate and random noise variable Pe to a sample from the conditional distribution P(Y = x), where Pe is pre-defined distribution and A E θ is the parameter space. The randomness of enables the generator to produce diverse outputs for a fixed z, thereby approximating the full conditional distribution [35].
To evaluate the quality of the generated distribution, engression the energy score, strictly proper scoring rule. For a distribution P and an observation the energy score is defined as
ES(P, |
E Zz Z'll - -E-p v11
where || denotes the Euclidean norm. A scoring rule S(P.y) is strictly proper if satisfies
Ey [S(Po. Y)1 < Ey [S(Q,Y))
for any candidate distribution Q + Po. where Po the true data-generating distribution. This property ensures that the oracle objective ming Exx,y) [S(Q, Y)] is uniquely minimized at Q Po- The energy score is strictly proper scoring rule that can be employed identify the true underlying distribution.
Local Training Procedure A(-): Given n observed data pairs {X,,, Y)) engression defines an empirical loss function to approximate the oracle objective. For each X; we generate m independent noise samples
{
1m with Ei, Pe. and compute the loss (Enx()) as
Eug
Y ((K,, 2m(m 1) IIG(X, G(X,,
The first term measures the alignment between generated samples and the observed data, while the second term regularizes the dispersion of the generated distribution by encouraging pairwise differences between samples. Minimizing this loss jointly optimizes both fidelity and diversity the estimated conditional distribution. The pseudocode of A(-) is provided in Algorithm 1
Algorithm Local Training Procedure of Engression
Input:
Data pairs {(,, ? initial parameter Gg training steps T, learning rate Co. noise distribution P., number of noise samples m
For = 1 to T' do
Li 1: Generate m i.i.d. samples {y with Ex,i,j ~ Pa for each [n]]
2: Replace Ei by Et,i,j in (6) of [Eng( (0c-1), and take gradient of at 0,-1 as V (O
3: Update parameters by 0, 0,-1 V Cos
4: return Ør.
Federated Training Procedure A(-): We extend the engression algorithm to a federated version. Given K groups of observed data pairs {(XR,, YR.s) )n h: 1, 1,..., K, generate independent noise samples {ER,,g) (I with Ei, ~ P.. Define L (0) as the engression loss function for agent kt obtained by replacing all instances of X,, Y,, Eij with X i, Yk,s Ekadd and the sample size
(Fng(()
21

---

with in (0):
ne
1 m
(0)
m
2=1
(G()))) G(X (AAP;
77)
2m(m
Thus the federated empirical loss can be represented as
K
Bug (0)
L k
Eug (0).
Tkk
To minimize loss (0), each agent computes the average gradient over its local dataset and
participates in the aggregation of K gradients to iteratively update the model G(:;; 0) [24). The parameter updating procedure is similar to that Section A.1.2 The pseudocode of A(-) is provided
in Algorithm?
Algorithm 2 Federated Training Procedure of Engression
Input: Data pairs {(XR,s, K" k [K], initial parameter p. training steps T, learning rate Ço- noise distribution Pe, number of noise samples m
For & = 1 to T do
For k = 1 to do Generate i.i.d. samples (et,k.i. 1 with Et,k,i. Pe for each i [mx]] Replace Ek.ig by Et.knig in (7) of Eng (0t-1), and take gradient of 0 at 0,- as VaCEns (0,-1)
1:
2:
3: Communicate and aggregate K gradients with weights proportion to mi,
TKK the
aggregated gradient is denoted as (Σ Ex=1 m4)-1 Σ n& VoLk
Eng
(0--1
4:
Update parameters by 0,-1 - Co (o n&)) E n& V
Eng (0 (0,-1)
5:
return Ōr.
A.1.2 Federated Density Ratio Estimation
The density ratio r(z, s) = fxx, fmix s), which quantifies the discrepancy between the target distribution P and the mixture Σk_ TT can reduced to binary classification problem 336 Specifically, we first train classifier to distinguish samples from the target versus mixture distributions, then apply calibration methods [44), i.e., Platt-calibration (2 to convert classifier outputs into well-calibrated probability estimates. This framework naturally extends to federated settings: the classification component implemented through standard FedAvg optimization across agents, while Platt calibration calibration-perforreed locally using only prediction scores- requires no exchange of private data.
We formulate a binary classification task where samples from the target distribution (X, sX. Y)) fxx 8) are assigned label 1 1, while those from the mixture distribution (Xe,-) 8% (XR,,, Yi.-)) fuix (z, s) receive label = 0. A global probabilistic classifier f(z, s; where E Fa is the model neural network, is trained collaboratively across all K agents (one target and K auxiliary) using federated averaging [24]. During each communication round, local updates are computed by minimizing the cross-entropy loss:
parameter, implemented
(G) -
Σ (x,y)ED
log { f(x, (x(2))):))
(8)
(6( -n
log [a, (t,
(9)
(
22

---

The federated loss can be written as
K
(n) = (=)))))))))))))))
Σ nk)
( n
c (n)
Model parameters updated with gradient
where
K
K
V (T = V ( (1-1)
n& v Lx (7-11))
After T training epochs, the final parameter is denoted by 7 Fr.
To calibrate the classifier outputs into reliable density ratio estimates, apply Platt scaling hold-out subset Phold from the target agent and other source agents. This involves fitting a logistic regression model to adjust the raw predictions by f(z,,) - o(a f(x, s; 7) + b), where a(-) is the sigmoid function and (a. b) are calibration parameters optimized on Dhola- The density ratio is then estimated as F(z, s) = fzz s)/ f(z,s)) s)}
Data privacy is preserved by ensuring raw covariates X and responses Y never leave their originating agents - only logits f(X, s( (X, Y): and model gradients are shared. This federated implementa- tion adheres to data isolation constraints while mitigating distributional shifts through collaborative learning.
Algorithm 3 Federated Density Ratio Estimation Procedure
Input: Datasets Da = DDr and (Da k= initial parameter To. training steps T, learning rate Co- pretrained score function -).-)) and (sx(t,) )) hold-out ratio 1)
1: Compute conformity scores s(x, y) for all (z, y) Do
2: Split dataset D, = Do,1 Do,2, where Do,1 n Do,2 = 0 and (oo22 = [n/Doo]
For k to K do Compute conformity scores se(x, y) for all (z. y) E D
3:
4:
Split dataset Dr Dx,1 U DR,,, where Dx,1 n Dx. 0 and [Dx,2l [n]Px]] For t= 1 to do
For k = 1 to K do
5:
Calculate the gradients of (+) with respect to y at h--1 and denote the gradients as v
using dataset DR,1-
6:
Communicate and aggregate K + gradients, the aggregated gradient is denoted as (..r(((.-1)) = (V (ft-)) .) (ER m44='(E n& V, 4 (f-1)))
7:
Update parameters by 70 h V (6-1))
8: Construct raw predictor fax,, 8; 7T)
9: Locally calculate and construct logits data pairs (f(z, s(a, fT), 1) for (x, E Do.2
For A: = to K do
10: Locally calculate (f(z, sx(2, y): Pr), 0) for (x, y) E Dx.2
11: Communicate the K + logits data pairs and fit logistic regression f(z, o(a flz, s;
12: return Density ratio estimation f(zz s) = f(z, s)/ f(x,,))
A.1.3 Aggregate Engression- Based Conditional Distribution Estimators
Let G(n, 0)) and G(x, E; 0xx denote the empirical estimators of the conditional distributions corresponding densities s) and fmix(a,,), s), respectively. The true underlying parameters Og and 0x are defined such that, for any fixed æ, the random variables G(x, e; 0b) and G(x, e; 0x) follow distributions with conditional densities [( æ)) and fmix(s a:) respectively. Given covariate x, reweight samples from C(Z,E;0p 0x) with weight proportion to r(z,s) s) and we get samples with density f(s z).
Km = "KK.,..,,KK..a be sampled i.i.d. from (Pa) and (Pa) ym, We propose estimating P(s(X.Y) & X = x) by aggregating G(z, 00) and
Let Eo,ma =
23

---

G(I,e;0k) with f(-.-) by:
ma (a Elu, Ek.i (x) (C)) S 8)
3r (8; CK (I,, E0,m, 3kmm)
Σ
k=0,K
(10)
where Ck(x,E0, kk,m) Ex= Σj (a G(a, skiji 0k))} -1 and 8o 1
Therefore, the conformal score on calibration can be written as V. Pp. (s(X,, Yi);X,, 2), for , and Yu+1 y.
Algorithm 4 Conditional Distribution Estimator Aggregation Procedure
Input: Two conditional distribution estimators G(z,, s; 0o) and Giæ,,; C(x,e;0k) density ratio estimator FK = F. noise distribution Pe, number of noise samples mo, mK
1: Sample independent noise vectors T0mm = (fo.1....,,,,,o))) &Kmx = (EK,1)-.... EK ,KKmx) from (P's)ro tho and (Pa)
2: Calculate 07,0.,5x (5; in (10) as an aggregated estimator of P(s(X,Y) S 8 X ==
3: return 37,00.0x (s)
A.1.4 PFCP Algorithm
provide the algorithm that follows PFCP procedure in Algorithm 5
Algorithm 5 Personalized Federated Conformal Prediction (PFCP)
Input: Training and calibration datasets from target agent Du. Deals datasets from K source agents D....., DR, pretrained score functions s(:-)) and (sx(;-)) algorithms A, A, Aar and Aags. test point X nominal coverage rate o..
Preparing Step:
Calculate conformity scores {S {5 and (Sk,) ) for k E [K].
2: Obtain datasets Du, = {(X,, S and D = {(x... Sk. for k E [K].
3: Train ((s x) A(D..)) and Fmix (s xx) = A(D1. ..... Dk).
4: Train F(x,, 8) Aar Dtr: Di,.... Dx)-
Conformal Prediction Step:
5: Obtain aggregate estimator Fage (s 2)) Auge (F(s) [s] x)) Fmix(s x)) f(x, s)).
6: Calculate transformed scores V F Fage (S X2) for [1+
7: Construct PFCP set GPFCP (Xn+1) using (4)
8: return GPFCP (Xn+1)-
A.2 Proofs
A.2.1
Proof of Theorem 2.4
For notation simplicity, we omit the superscript Ys+1 of V.Y.1,, that is, the transformed scores Due to the i.i.d. data pairs (1, (Xn+1,, Y.+1) drawn from P. and their independence from Drr and Dr, k = L...,,K K, the transformed scores (V)) are exchangeable. Let Vin,n) denote the k-thh smallest value in V By definition, as long [[1 (n + 1)]
V, F (s Yt)
1, have:
o @ 1 a: (n 1
Σ 8v. + 51 61)
)) = V(ra (1(0=o)(++)].m))
(11)

---

Therefore,
P P(V++1 S V(ra-a)0++11,.))
(
(Va+1= S V ((10-0)(o-111a4)))
P (V. S (f]1)-a)(==],,-ii)))
x+11
{ s ([( ((((-a)
(68) [(1 () (m+
n+1
Here, (i) is because V S V(K.a) if and only if Vn+1 V(R,++1). Indeed, if Vn+1 V(k,))-
then 1( is the larger of V(k-1,n) and Va+1; therefore, V(k.+1)) 2 Vn+1- Conversely, Vn+ S 11), then also Va41 ViK,.) because V(k,++1) S V(K,n)- (ii) follows from the exchangeability of (Vin=1 and (iii) follows from the no-tie assumption.
1 Then by the fact - I [(1-a)=-1]] < (1-a)n+11+11 =1 a a+ we achieve the second a+1
n-1
result. the no-tie assumption does not hold, the equality in (iii) can be replaced by an inequality "z" based on the definition V(k,++1)- Consequently, we still obtain marginal validity of CPFCP (X,+1)
A.2.2 Proof of Lemma 2.6
Recall that
GLCPP (Xn+1)= =
{v: =:+1))=(::00 V S 5Q(1 a; 11 <n Svy + 61 })
1=1
the probability of test-conditional miscoverage error can be written as
IP (Y+11 & CGLCP (Xn+1) |x+1= =x)
-P Vn+i
PVn111 > ar (n 1)-1
Σ Sv: +0i)) |n+1 xz).
(12)
From equation [I] we let
1)-1 (n Vg = @0 (n +
Σ 8v, +d))) = V[[ng])))
where 9 [[1 (n + 1)]/n. Notably, Vú is the q-th quantile of empirical distribution Fy nn dv. DKW inequality gives that for any E (0,
P sup Pz(z) - Fr(x)) -F(()) >))= 1o,
where (1(2))-1((2/88)) -1/2, The definition of quantile gives
1/2,
FV < < 1
Denote A_ (sup (f(()) Fv(x)) < c). On A. we have
q-ES P- (P,) <e+a+- 1 n
( < (
25

---

Define Un+1 F(s(Xn+1, Yx+1) | Xx+1) with Xn+1 x, such that Unt1 ~ U10, 1]. Therefore, the total variation between the distribution of Vn+1 and Un+1 is d1 (z; F). We have
2 61 + P Un+i (n 1) Σ gv + +o1)) Xn+1 = =x)
(n
()( + Ae, Un+1 > Vy Xn+1 z))
<b fb(1;)) + 8 P Unti > (g - c) Xu+1 = xz
= (x; F) + 8 + (4 (
On the other hand, we can similarly derive that
12 2 P (Unn+1 - (n (1 Σ gv +81)) Xn+1 = =x) - -51(2; F)
1=1
2 P Ae, Unt1 > Vy Xn+1 = x) - (2;
2 Ac, Ant11 >
Xn+1 =x) -21 F)
2 Unii Fr (E - q + Xn+1 Xn+1=I) =x)
-P(A.) (x; F)
2 Fv (e + ( 61 (1; -
Therefore,
[((Y++1 $ CGLCP (Xn+1) Xn+1 = aa
S max
[|1 - a- - Fv (g + 81, Fv' + -81 + (2:
A.2.3 Proof of Theorem 2.8
Part
For any (x, 8), based on the first inequality of Assumption 2, we have
fmix(s 2) E Tefu(Es) ΣK Tafa(z)Ja(( x)
K
EkI 2 L Σ fa(s x))
k=
Moreover, based on the second property of Assumption 2, there exists & E [K] such that fa(( x) 2 f(s T.g Therefore, considering Assumption 1, we have
fmix(s (s x) 2 (LoLx)) f(s x) 2 ([)(z))ooxx)1
Based on Assumption we denote
arg max F(z,8). sE(o.M,(z))
Based on the Lipschitz property we can derive that
10 f(z,t)dt >
(. f(x,t)dt
L(f,)) di (f(z, Sx (F))
(F,
Therefore
F(x, 8)
2L (z) L
F(s x))
Jo smix (1 2))((,, t)dt
(7)
which does not rely on and (z; Fmix) 00.
26

---

Part 2.
In the following proof assume the conditional distribution estimators F(s ax) and mi x)) are continuous with respect to 8 and differentiable. The derivative with respect to is denoted as conditional density estimators f(s 2) and fmix(s 2). We rewrite the aggregated conditional distribution estimator as follows:
Fagz (s x)) - (1+ ! fa F(11)) + mmix(1 x))(( ())())
which is derived by
F(s (2) 2) x1= = f(tlz)dt= I'" fmix(t x)r(t x)dt,
and weight two estimators with and f(z).
Notably, the total variation can be written as the integration of the abstract difference of densities:
1
/
81 F)
2
If(s (x) f(s x))ds,,
2
8(1; Funix Fmix))
1
/
20
[ruix(s xx) (
- fmix(( x)) ds.
2
The derivative of (s x)) with respect to 8
Saee(s x)) = (1 + F(x)) [J(s | x) + Pmix(s) x)F(s x))(x))
Thus the 6, (1; F FaRR) can be calculated as
8, (2;; Payx)= = 1 [ines(s) xx) - -5(0) x))ds
-1 i1 10 [((11)) {(s x))
$ (1+
F(x) / |imix(s x)F(s x) - f(s xs)dds}
= -[++ { 25 (z; M) + f(()) /00 [mmie(s) x)F( i(s ws)
= (1+ 22 (x;
1 [Iuix(( x) 8) - Smix(( x))(2, 8) F(xx)
ds
r(x:)
s {1+ 26; (2;; F) + 1- [mixx(s
x))(x,)) ( - fmix(( (x)((, 8) ds
f [fmii(s s)) fmix(s f 8) f(x:) ds
r(x)
$ (1+ (1+f(z)) 20 (or; F) 10- S(s ())r()())11111))))))()))))))))) -(())/((()))l
/ [mix(s | (F(( r(z, s))
/ (), xx) -mix x)} ds
$ (1+ 281 (x:; F) + |r(x) - F(x)] + 7)x;
[// (I, 1o (mix(( x)) - fmix(s 22) 1/22
(1 + [())) -1 28 (o;; F) 8(,; + [((x) - r(x) + 5z(2; Pmnx) (x;; 77]
27

---

A.3 One-Shot Agent Similarity Matching
In real-world federated learning scenarios where the number of source agents can scale to thousands (e.g,, smart device ecosystems with thousands of user-level agents), two critical challenges arise: (1) the computational infcasibility of utilizing all agents, and (2) the potential presence of distributional outliers that degrade model performance. Our framework addresses both issues through efficient agent selection first reducing computational costs by prioritizing top-K' agents via lightweight distributional alignment metrics. We test our similarity metric in Section A.4.4
A.3.1 Federated Data Similarity: A Survey
In federated learning for classification tasks, common approach measures agent similarity by computing class label distributions (or marginal response distributions in regression settings) as feature vectors, then comparing these vectors across agents [7]. However, this method oversimplifies distributional alignment by solely considering label/response statistics, failing to account for the joint distributional relationships between inputs X and outputs Y. Such marginal matching proves insufficient when covariate shifts or feature-label dependency mismatches exist across agents.
awwar clustering. Here, clients are grouped by the similarity of their model weight updates compared to the global joint model 3 or representative gradients [T0], defined as the difference between their locally updated model parameters and the global model. These gradients implicitly capture the direction and magnitude of client-specific updates, reflecting underlying data distributions. A hierarchical clustering algorithm partitions clients into clusters such that those within a cluster share similar gradient profiles, ensuring that each cluster represents distinct data pattern.
Briggs et al. 3]. Fraboni et al. [10] proposed
Existing gradient-based methods for measuring similarity among federated agents require partic- ipation in global task training, which imposes substantial computational overhead. However, in our framework, we propose a lightweight alternative that directly screens source agents during preliminary analysis to accelerate similarity quantification. This paradigm shift enables rapid identi- fication of inter-agent correlations without full model deployment, permitting selectively utilizing subset of source agents with higher similarity for model training. We subsequently introduce an efficient Kmeans-driven similarity metric that operates on parameter distributions rather than gradient interactions, achieving comparable discriminative power while significantly reducing time complexity.
A.3.2 K-means-Driven Similarity Metric
classical approach for measuring distributional similarity 4 between two datasets D2 and D2 involves binning the data into M intervals. This produces histogram vectors II; (#11..... ... R1M) and II, = (.. r2.). where Tim represents the proportion of data points from D; falling into the m-th bin. Standard bin-to-bin similarity measures can then be applied quantify distribution alignment based on these frequency vectors. We list several classical measurements here:
Total Variation: This index measures the L, distance of two input bins:
dII, 1l2)
Thi
T2:
2. Histogram Intersection: This index is expressed by min function to get the overlap of two input bins:
M
dinter 112) 1122 1
Σ
min( m22)
3. Histogram Correlation: This index measures the correlation of two input bins:
ΣM
(TH (T)( - T2)
door (1I, (I1,,
ΣM Di (mi )1 1>
( T2)
28

---

Existing approaches for dataset similarity assessment in federated learning often rely on raw data interactions, posing privacy risks and communication bottlenecks. We address this by developing prracccpreserving binning strategy: instead of exchanging original data samples, all agents collabo- ratively construct global codebook through K-means clustering on aggregated data distributions, partitioning the feature space into M non-overlapping subspaces where each point belongs to its nearest cluster center. By converting local datasets into cluster-indexed histograms through simple counting operations, agents can efficiently compute distribution similarities using histogram distances.
Following the established federated clustering paradigm
[ each souree agent {I...., K}
(M)) through localized K-means clustering initially identifies Mo local cluster centers {C ...., Cx
on its dataset DE. while recording corresponding sample counts (N Ny.MM)). These locally optimized cluster centers and their associated cardinalities are securely transmitted to the central server, which aggregates all x Mg centers and performs weighted K K-means clustering using the transmitted sample counts importance weights to derive M global cluster centroids. Upon receiving these consensus cluster centroids, each source agent constructs a distribution histogram by mapping its local samples to the nearest cluster centroids. The histogram of souree agent & is denoted by II = (Tkl...., TkM,)) Therefore we measure the distance of Dj and Dj by
dkm (Di, (D, = drv [I); I1;).
A.4
Additional Experiments: Synthetic Data
A.4.1 Experiments on Different Extent of Source Agent Data Heterogeneity
Our second experiment quantifies how varying degrees of source agent heterogeneity (7k distribution shifts) affect target task performance. We design two regimes: S1.3 S2.3: 7/k ~ U[L c1 and S1.4 & $2.4: "/k [1, + e2]. where с₁ [0, 1] and e2 [0, 5] control heterogeneity intensity (larger values indicate stronger distributional mismatch). Density ratio estimation is fixed to the calibrated classifier approach to isolate heterogeneity effects. This setup systematically probes how under-provisioned (ci -driven) versus over provisioovii (c2-driven) source agent distributions influence federated conformal prediction robustness.
Figure demonstrates that PFCP exhibits notable robustness to distributional heterogeneity, achieving consistent performance improvements compared to baseline methods across broad spectrum of 7 variation magnitudes. This empirically verifies PFCP's capability to leverage useful source agents while automatically discounting those with severely mismatched distributions. Furthermore, the performance degradation remains gradual even under extreme hetcrogeneity, highlighting practical reliability in real-world federated scenarios where precise prior knowledge of agent distributions is often unavailable.
GLCP
PFCP
FedCP
FedCP-QQ
CPlab
CPhet
$1.3
$1.4
$2.3
$2.4
1.00
1.00
1.00
1.00
Marginal Caverage
0.95
0.95
0.95
0.95
0.90
0.90
0.90
0.90
0.85
0.85
0.85
0.85
0.80
0.80
0.80
0.80
0.75
0.2
0.4
0.8
1.0
0.75
6
8
0.75
0.2
0.4
0.8
1.0
0.75
6
C2
$1.3
$1.4
$2.3
$2.4
0.10
0.10
0.10
0.10
844*0
Mecovarage
0.06
0.06
0.06
0.06
0.04
0.04:
0.04
0.04
0.02
0.02
0.00
0.00
0.00
0.00
0.2
0.8
1.0
6
0.2
0.4
0.8
1.0
Figure 4: Results of synthetic data with varying source agent distribution.
29

---

A.4.2 Experiments on Different Source Agents Quantities
Central to PFCP is leveraging auxiliary data from source agents to enhance data-scarce target tasks. Follow the setting of S1 and S2 and denote the setting with Y U as S1.5 and S2.5, we systematically investigate how performance scales with increasing numbers of source agents (K {2, 4, 20)). Each agent, including the target, maintains identical data volume (3n samples: for the source agents n for point estimation, 2n for conditional distribution calibration). This controlled experiment isolates the effect of collaborative scale, revealing empirical scaling law where PFCP's prediction interval width decreases sublinearly with K, achieving improvements in sample efficiency over single-agent baselines.
Figure 5 demonstrates that even minimal collaboration (2 source agents) delivers substantial improvements-reddccing conditional miscoverage by 19.7% (S1.5) and 25.6% (S2.5), and prediction interval widths by 10.5% (S1.5) and 3.9% (S2.5) compared to isolated learning. The enhancements scale sublinearly with agent count K. This quantifies PFCP's data-efficiency in leveraging sparse yet distributed knowledge sources.
GLCP
PFCP
$1.5
$1.5
S1.5
Marginal Coverage age
5.5
0.95
Miscoveaaee
0.08
0.90
0.07
4.5
0.85
0.06
4.0
0.80
0.05
15
3.5
K
$2.5
$2.5
$2.5
Marginal Coverage
0.95
Miscoverage
0.08
5.5
0.90
0.07
Size
0.85
0.06
4.5-
0.05
10
15
20
K
Figure 5: Results of synthetic data with varying source agent quantities.
A.4.3 Experiments on Different Required Coverage Levels
To rigorously assess PFCP's robustness beyond the conventional 0.1 setting, systematically evaluate its performance across a E [0.05, 0.95]. We still follow the setting of S1 and S2 and denote the varying setting $1.6 and S2.6, where set % U/0.6. 1].
Figere that GLCP exhibits slight over-coverage tendencies at extreme values (a 2 0.9), due to instability in prediction set construction under sparse calibration scenarios. In contrast, PFCP maintains coverage rates tightly clustered around the target a level across the entire spectrum. Also Figure 6 reveals that PFCP maintains superior reliability over GLCP throughout this parametric spectrum: while both methods achieve nominal marginal coverage, PFCP reduces conditional miscoverage across extreme quantiles (a < 0.3 or 0.8) through adaptive density ratio reweighting. Simultancously, the prediction interval size analysis demonstrates that while both methods show monotonically increasing interval lengths with higher achieves systematically tighter intervals.
requirements, PFCP
This systematic evaluation demonstrates PFCP's unique capability to preserve both marginal validity and conditional calibration across diverse coverage requirements while achieving dual improvements: exact coverage maintenance and statistically efficient interval growth. By adaptively balancing reliability and precision, PFCP enhances statistical efficiency without compromising robustness-a critical advantage for real-world applications demanding flexible yet rigorous error rate control.
30

---

-*- GLCP
PFCP
51.6
S1.6
$1.6
1.0
Marginal coverage
0.8
0.08
Miscoverage
0.6
0.06
Size
0.4
0.04
0.2
0.02
0.2
0.6
0.8
1 a
1
52.6
$2.6
52.6
1.0
Marginal Coverage
0.8
Miccoverage
0.08
0.6
0.06
0.4
0.04
0.2
0.02
0.2
0.4
0.6
0.2
0.4
0.6
0.8
0.8
Figure 6: Results of synthetic data with varying required coverage.
A.4.4 Efficiency of K-means-Driven Similarity Metric: A Toy Example
In this toy example, we consider 40 source agents with Poor Ponttoontoleeed corruption levels, where each agent's is distribution similarity parameter 7 follows the mixture model 7/k Po, 0.2] (1 - ( 1]. Following and S2, two experimental scenarios (S1.7 & S2.7) are designed to simulate varying proportions of corrupted agents exhibiting distributional divergence from the target domain (y = 1). The ranking efficacy of similarity metric dkm is quantified by verifying whether the sorted agent sequence maintains decreasing "ik values monotonic ordering indicates that d successfully prioritizes agents with higher distributional alignment to the target.
For each repeat, we denote the order statistics of "...... "KK by 7(1) 2 2 ((K)- The corresponding similarity parameter of the sorted agent sequence is denoted by ?1...... "/- First we compare Ercy..... (Y( and ..,,..., EÝK in Figure 7 The experimental results demonstrate that our similarity metric achieves effective distributional discrimination, as evidenced by the monotonic ranking of source agents those prioritized by the method exhibit significantly closer alignment to the target distribution (> = 1). Notably, the curve reveals a steep decline phase where our selective screening coincides perfectly with the oracle selection, indicating that prior knowledge of the corruption ratio Poor enables near-perfect identification of distributionally divergent agents through this approach.
When utilizing filtered source agents as an aggregated composite distribution, next we focus on evaluating the collective alignment through average similarity parameters, Specifically, upon selecting the top K' agents ranked by dkm, we quantify their ensemble relevance by computing Ýк*
k==1 Tt/ K'. We compare 7x. with optimal 7x-- = Ek'n T(k) K' in Figure
A.4.5 Efficiency of K-means-Driven Similarity Metric
We test the performance PFCP with similarity agent matching using setting S1.7 and S2.7. We consider 40 source agents and select top Tsel to serve as the auxiliary agents. We test Peor 0.1, 0.3, 0.5, 0.7, 0.9 and niel = 10, 20, 30 respectively.
Figure demonstrates the performance across four methods under fixed selected agent numbers Agell where the marginal coverage of FedCP and FedCP-QQ progressively degrade as the corrupted agent ratio Peor increases. These methods exhibit increasing susceptibility to distributionally divergent agents at higher corruption levels.
Figure 10 illustrates the conditional miscoverage dynamics of PFCP under varying corrupted ra- tios p. demonstrating that our similarity-awant matching mechanism effectively desensitizes algorithm to distributional outliers. The density ratio estimation intrinsically mitigates adverse
31

---

Oracle Mean
Selected Mean
S1.7 Pro
$1.7 Peor 0.3
S1.7 Poor 0.5
S1.7 Peor 0.7
51.7 Pcor 0.9
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.0
S2.7 Peor
0.1
52.7 Peor
0.3
S2.7 Peer
0.5
S2.7 Peer
0.7
$2.7 Peer 0.9
1.0
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
0.8
menn
0.6
0.6
0.6
0.6
0.6
0.4
0.4
0.2
0.0
20
40
20
40
20
40
20
40
20
K"
K²
K'
Figure 7: Similarity of the k-th selected agent.
Oracle Mean
Selected Mean Y
$1.7 0.1
S1.7 Peor 0.3
$1.7
0.5
$1.7
0.7
$1.7 0.9
meen
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.2
0.0
0.0
0.0
0.0
0.0
40
40
20
0
S2.7 Pear
0.1
S2.7 Poor
0.3
S2.7 Peer
0.5
S2.7 Peor
0.7
$2.7 Deo 0.9
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
0.8
mean
0.6
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.2
0.0
0.0
0.0
Figure 8: Average similarity of the top h selected agent.
impacts from corrupted agents by reweighting samples according to their distributional congruence with the target domain. Complementing these findings, Figure IT confirms PFCP's robustness in high-corruption regimes even under severe data erosion, incremental improvements in predictive reliability emerge through expanded dataset aggregation, though such gains remain statistically marginal due to fundamental limitations in information recovery from corrupted sources.
A.5 Additional Results of Real Data
This section presents the agent partitioning strategy and supplementary visual analyses omitted from the main text, including conditional coverage uniformity across feature subspaces for the BIO, BIKE, CRIME, and STAR datasets, and the scaling laws of transfer efficiency with respect the number of source agents on BIO, BIKE, CRIME, STAR, and CONCRETE datasets, quantifying the diminishing returns of agent aggregation under domain heterogeneity.
Agent Partitioning Strategy: For the BIO dataset (feature dimension d 9, response: Size of the residue) and the CONCRETE dataset (d = 8, response: Concrete compressive strength), features themselves not exhibit clear distinctions for agent partitioning. However, domain- specific scenarios may lead to systematic variations for instance, certain protein studies may involve residues with atypically large sizes, while some concrete manufacturers may produce materials with lower compressive strength due to varying equipment processes. To capture such distributional
32

---

+ GLCP
PFCP
FedCP
FedCP-QQ
CHlab
-
CPhet
Marginal Coverage
0.9
0.85
0.8
0.8
0.7
0.75
0.7
0.6
0.70
0.2
0.4
0.6
0.8
0.4
0.6
0.8
0.2
0.4
0.6
0.8
Por
Prer
Pow
$2.7
10
52.7
20
S2.7
30
Marginal Coverage
0.90
0.9
0.85
0.8
0.8
0.80
0.7
Por
Poo
Por
Figure Marginal coverage with varying Poor
GLCP
PFCP
$1.7 Pgg =
$1.7 ngg 20
S1.7 n, 30
0.08
0.08
0.08
Miscoverge
0.06
0.06
0.06
0.04
0.04
0.04
0.02
0.02
0.02
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
$2.7 Nael
10
S2.7 Bud
$2.7 Auel
0.08
0.08
Niccoverage
0.04
0.02
0.02
0.2
Peor
Poor
Poor
Figure 10: Conditional miscoverage with varying Peor-
differences, we adopt response-basee partitioning scheme. Given a dataset { V/),.... (Enz Vn.) }. we first compute the au- Ru-quintile yo of the response variable. Each sample (xt, 3/) is then assigned weight proportional to ((1ys- / oo). where K(-) is the Gaussian kernel function and on scales the deviation. Finally, we perform probability-pre proplittopoooooinnaall sampling without replacement to construct the agent-specific dataset. To induce meaningful distribution shifts across agents, we set the quantile parameter dg to 0.9 for the Protein dataset and 0.3 for the Concrete dataset. The scale parameter do defined the difference between the (ao 0.1) and (ac- 0.1) quantiles of the response variable, ensuring adaptive bandwidth for kernel weighting.
For the BIKE dataset (feature dimension = 9, response: user count per record), we partition the data by month since it recorded chronologically, using the last four months as the target agent and each remaining month as separate agent. For the CRIME dataset (d = 20, response: violent crimes per 100K population per district), where existing research primarily focuses on densely populated metropolitan areas leaving less-populated regions understudied, we designate the district with the smallest population as target agent and randomly partition the others. the STAR dataset (d 20, response: ACT score), given that many studies examine the impact of carly education
33

---

PFCP
Pow
Poo
Poor
Poo =
Pror
0.054
0.054
0.054
0.056
0.060
Miscoverag
0.052
0.052
0.052
0.054
0.058
0.050
0.050
0.050
0.052
0.048
0.048
0.050
0.054
10
20
30
20
30
10
20
30
10
30
10
20
Reel
Rewl
Real
Aee
S2.7
0.1
S2.7
0.3
S2.7
0.5
52.7
0.7
S2.7
0.9
0.052
0.052
0.052
0.060
Misc.verage
00.050
0.050
0.048
0.048
0.050
0.056
0.046
0.054
0.046
0.048
30
Paal
Rial
Figure 11: Conditional miscoverage with varying Vsel
future achievement particularly the negative effects of resource deprivation we define students from rural schools in their early age as the target agent and randomly partition the remaining students.
For the classification dataset DERMA, the input image format for each sample 28 28 3. There are a total of 7 classes, and the class distribution is highly imbalanced, with some classes accounting for very small proportion (e.g., some specific symptoms disease susceptibility for particular high-risk group, where such group contains relatively higher proportion of rare classes, we sample from all classes as uniformly as possible form the target data pool, while the remaining samples constitute the auxiliary data pool. When partitioning the auxiliary data, is divided evenly according the number of agents.
rare). To simulate the prediction of
Computing conditional coverage for prediction sets requires knowing the exact conditional distribution of Y given X, which is unavailable in real-world experi- ments. For each test sample (X, Y) and prediction set (X), we evaluate coverage via the indicator 1(Y E Ca (X)). We evaluate our method using relaxed conditional coverage. We partition the feature space into N, disjoint subspaces, ensuring each test sample belongs to exactly 100 trials, we compute the average coverage rate within each subspace as the conditional coverage metric. Let . PN. denote the conditional coverage rates and w1,..., ........ WN, the proportions of samples in each subspace. The conditional miscoverage then defined as EN= wtlp. (1 - a)-- The partitioning strategy in this work first applies K-means clustering to the target agent's 's data to identify N₂ centroids, then divides the spatial domain into N₂ disjoint sub-regions based on these centroids, ensuring spatially coherent and interpretable partitions while maintaining alignment with the underlying data distribution.
Conditional Coverage Calculation:
subspace. Over
For classification datasets, the test-condiig cott-onditional coverage is directly partitioned based on the class labels of the samples. Assume there exist N,, classes for the dataset, and let PI.....PA PN, denote the conditional coverage rates and wi...., wN,, the proportions of samples in each class. The gap between the coverage and a computed separately for each class, and then weighted by the proportion of each class to obtain the conditional miscoverage N w; lp. a)].
A.6 Limitations
The method is designed to enhance conformal prediction federated setting by leveraging auxiliary data from multiple source agents to support conformal prediction on the target agent. While effective under certain conditions, this strategy assumes that the data distributions of the source and target agents are relatively aligned. If the distributional shift between source and target is extremely significant, the effectiveness of the auxiliary data may degrade. The estimated F can be approximately 0 the support of source agent data and makes source agent data uscless.
34

---

Required
GLCP
Miscoverage of GLCP 0.033
Miscoverage PFCP 0.020
0.4
coverage
0.3
0.1
0.0
101112131415166 01111111191222 Subset Group Index
Figure 12: Conditional coverage over all subspace for BIO dataset.
Required
GLCP
PFCP
Miscoverage of GLCP 0.023
0.20
Miscoverage of PFCP 0.019
1- coverage
0.15
0.10
0.05
0.00
10 11
12
13
14
15
16
18
19
20
Subset Group Index
Figure 13: Conditional coverage over all subspace for BIKE dataset.
Required
GLCP
PFCP
0.30
Miscoverage of GLCP 0.039
Miscoverage of PFCP 0.027
0.25
coverage
0.20
0.15
0.10
0.05
0.00
2
3
4
5
6
7
8
10
11
12
13
14
15
16
17
Subset Group Index
Figure 14: Conditional coverage over all subspace for CRIME dataset.
35

---

Required
GLCP
PFCP
Miscoverage of GLCP 0.039
Miscoverage of PFCP 0.024
coverage
0.1
0.0
7
10 11
12
13
14
15
16
17
18
19
20
Subset Group Index
Figure 15: Conditional coverage over all subspace for STAR dataset.
Required
GLCP
PFCP
Miscoverage of GLCP 0.030
0.3
Miscoverage of PFCP 0.023
coverage
0.2
0.1
3
5
Subset Group Index
Figure 16: Conditional coverage over all subspace for DERMA dataset.
BIO
100,
GLCP
PFCP
GLCP
PFCP
.
FedCP
FedCP-QQ
CPlab
CPhet
0.022
scoverage
0.032
0.54
1.00
Micooerage
0.020
0.52
Marginal Coverage
0.98
0.50
0.96
0.018
0.016
0.46
0.92
0.44
15
25
30 Agents
35
40
10
25
35
Agents
Figure 17: Impact of source agent quantity on target task performance for BIO dataset.
BIKE (n= a
GLCP
PFCP
GLCP
PFCP
FedCP
FeaCP-QQ
CPlab
CPhet
0.92
0.022
scoverage GLCP 0.023
Mean Size of GLOP 4.097
4,2
Marginal Carginal Coverage
Micoverage
0.020
4.0
0.88
0.018
3.8
0.016
16 Agents
1'4
18
Number Agents
Agents
Figure 18: Impact of source agent quantity on target task performance for BIKE dataset.
36

---

STAR
100, 0.1)
GLCP
PFCP
GLCP
PFCP
FedCP
FeaCP-QQ
CPlab
CPhe
0.032
Miscoverage
0.030
Marginal Coverage
0.905
Misooerage
0.028
46
0.900
0.026
0.895
0.024
42
0.890
0.885
3
2
3
7
8
3
7
Agents
Agents
Agents
Figure 19: Impact of source agent quantity on target task performance for STAR dataset.
CONCRETE (n - 36, a 0.1)
FedCP-QQ
0.030
scooeraage of GLCP 0,045
Mean Size of GLCP 39.625
0.028
Marginal Coverage
scooeeaag
0.026
0.94
32
30
0.92
0.022
28
0.90
0.020
26
2
4
6
3
5
3
5
Number of Agents
Number of Agents
Number Agents
Figure 20: Impact of source agent quantity target task performance for CONCRETE dataset.
Moreover, incorporating a large volume of auxiliary data from multiple sources introduces significant computational overhead. This includes communication costs in federated settings and increased putational requirements for estimating Fagg. Future work could address these issues by developing adaptive strategies that weigh source contributions based on distributional similarity or by designing more efficient protocols for federated uncertainty quantification.
A.7 Societal Impact
The methods proposed have implications that can be interpreted as having both positive and negative socictal consequences. On the positive side, the proposed conformal prediction methods enable more reliable uncertainty quantification in complex predictive tasks. This has the potential to significantly enhance decision-makinn making high-stakes domains such as healthcare, finance, and environmental mod- eling, where accurate assessments of predictive uncertainty erucial. Furthermore, by improving the reliability and fairness of predictions, these methods may help foster greater trust Al systems, especially when applied in sensitive settings involving diverse population groups.
DERMA (n = 200, = 0.1)
GLCP
FedCP
FeaCP-QQ
0.030
Miscoverage of GLCP 0.030
2.7
Mean Size of GLOP 2.484
0.900
0.028
Marginal Coverage
60.875
Miccoverage
2.5
0.825
0.024
0.800
0.022
2.3
0.020
2.2
0.750
5
2
Number of Agents
Number of Agents
Number of Agents
Figure 21: Impact of source agent quantity on target task performance for DERMA dataset.
37

---

On the other hand, there are potential negative impacts to consider, A key concern is that overreliance on conformal prediction cehhiiuue-paaticulaate without sufficient domain expertise-mat -ay lead to overly simplistic or misinformed interpretations of the uncertainty information provided. This could result suboptimal or even harmful decisions if the limitations of the method are not fully understood or communicated. As such, while the work contributes valuable tools for uncertainty quantification, its responsible deployment requires careful consideration of context, assumptions, and the broader social environment in which such tools are used.
38


Reviews:
[{"review_id": "O42qNn7shS", "reviewer": "Reviewer_kS3y", "summary": "Authors propose a personalization framework for  (conditional) conformal prediction in federated settings using engression and federation of a learnable conditional density estimator.", "questions": "* Why the proposed algorithm do not operate on classification datasets used in [1]. This limits the credibility of the algortihm as the FedCP is not tested on these datasets as well. \n* Some very closest works mentioned [1,2] , but not included in the baselines. What is the exact reason? Some of the datasets in this study has been used \n* What distribution shift do you consider in the proposed algorithm?\n\n\n[1]  Charles Lu, Yaodong Yu, Sai Praneeth Karimireddy, Michael Jordan, and Ramesh Raskar. Federated conformal predictors for distributed uncertainty quantification. In International Conference on Machine Learning, pages 22942–22964. PMLR, 2023.\n\n[2]Vincent Plassier, Alexander Fishkov, Maxim Panov, and Eric Moulines. Conditionally valid probabilistic conformal prediction. stat, 1050:1, 2024.\n\n[3] Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, and Maxim Panov. Efficient conformal prediction under data heterogeneity. In International Conference on Artificial Intelligence and Statistics, pages 4879–4887. PMLR, 2024", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n* The personalization problem of federated conformal prediction is novel and interesting. \n* Leveraging a federated conditional distribution estimator to leverage PFCP is novel. \n\n**Weaknesses**\n* Evaluation with unknown datasets compared to the baselines[1].\n* Some missing baselines [2,3]\n* GLCP is not clear. Is it also federated, or is there no communication between the clients in this case?\n* As long as coverages are greater then 1 - $\\alpha$, the coverage results are fine. However, the effect of personalization over the set size as opposed to [1] is not clear to me.\n* The personalization part is not clear throughout the steps in lines 172-180. Is personalization done over the aggregation level , or not sharing some specific model?\n* Notation is overall hard to follow as opposed to previous works in federated CP. \n\n[1]  Charles Lu, Yaodong Yu, Sai Praneeth Karimireddy, Michael Jordan, and Ramesh Raskar. Federated conformal predictors for distributed uncertainty quantification. In International Conference on Machine Learning, pages 22942–22964. PMLR, 2023.\n\n[2]Vincent Plassier, Alexander Fishkov, Maxim Panov, and Eric Moulines. Conditionally valid probabilistic conformal prediction. stat, 1050:1, 2024.\n\n[3] Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, and Maxim Panov. Efficient conformal prediction under data heterogeneity. In International Conference on Artificial Intelligence and Statistics, pages 4879–4887. PMLR, 2024", "was_revised": true, "final_justification": "The authors provided reasonable arguments to my responses.", "rebuttal": "Thank you for your positive assessment recognizing both the novelty of the personalization problem in federated conformal prediction and the methodological innovation of our federated conditional distribution estimator for PFCP. We sincerely thank the reviewer for the constructive and thoughtful feedback. Below, we address each concern in detail and provide clarifications and additional numerical results.\n\n**W1\\&Q1\\&W2\\&Q2:**\n- First, we would like to clarify a typo in our citation in line 96. We mistakenly cited [3], and the correct references should be [2] (for label shift) and [4] (for covariate shift). We will correct this in the revised version.\n- For the classification experiments (W1\\&Q1), we have now implemented the code for classification task using the score function $s(x,y)=1-[f(x)]_y$ as in [1]. We conduct additional experiments on DermaMNIST as a representative classification dataset (denoted as DERMA), and will include these new results in our revised manuscript.\n- For the additional baselines (W2\\&Q2), we sincerely appreciate the reviewer's suggestion to include additional baselines. While [2] (CPlab) and [4] (CPhet) share conceptual similarities with our approach, we note they were originally designed for different problem settings. We have implemented both methods for fair comparison.\n - The real data results are shown in Table below. The experimental results demonstrate that both GLCP and PFCP consistently achieve valid marginal coverage on the target agent, while the other baseline methods frequently fail to maintain this fundamental guarantee. Furthermore, PFCP exhibits significantly lower test-conditional miscoverage compared to GLCP, with both proposed methods substantially outperforming all remaining baselines.\n\n||Dataset|GLCP|PFCP|FedCP|FedCP-QQ|CPhet|CPlab|\n|-|-|-|-|-|-|-|-|\n|Marginal|BIO|0.902|0.903|**0.993**$\\times$|**0.970**$\\times$|**0.981**$\\times$|**0.978**$\\times$|\n||BIKE|0.900|0.900|**0.885**$\\times$|**0.890**$\\times$|**0.883**$\\times$|**0.885**$\\times$|\n||CRIME|0.900|0.896|**0.866**$\\times$|**0.852**$\\times$|**0.864**$\\times$|**0.863**$\\times$|\n||STAR|0.898|0.899|0.897|**0.910**$\\times$|0.899|0.897|\n||CONCRETE|0.903|0.905|**0.947**$\\times$|**0.963**$\\times$|**0.946**$\\times$|**0.949**$\\times$|\n||DERMA|0.895|0.895|**0.830**$\\times$|**0.819**$\\times$|**0.849**$\\times$|**0.867**$\\times$|\n|Miscoverage|BIO|0.0315|**0.0199**(36.8%)|0.0941|0.0753|0.0844|0.0822|\n||BIKE|0.0234|**0.0184**(21.4%)|0.0678|0.0647|0.0690|0.0681|\n||CRIME|0.0387|**0.0268**(30.7%)|0.0426|0.0495|0.0429|0.0439|\n||STAR|0.0392|**0.0243**(38.0%)|0.0500|0.0492|0.0496|0.0495|\n||CONCRETE|0.0366|**0.0238**(35.0%)|0.0582|0.0675|0.0580|0.0600|\n||DERMA|0.0307|**0.0264**(14.0%)|0.0939|0.1004|0.0802|0.0701|\n|Size|BIO|0.514|**0.503**(2.1%)|0.999|0.709|0.827|0.805|\n||BIKE|4.097|**3.988**(2.7%)|4.004|4.091|3.967|3.996|\n||CRIME|5.183|**4.496**(13.3%)|3.918|3.772|3.886|3.880|\n||STAR|48.50|**43.02**(11.3%)|42.80|45.30|43.08|42.85|\n||CONCRETE|34.49|**29.38**(17.7%)|34.71|38.97|34.69|35.15|\n||DERMA|2.409|**2.327**(3.4%)|1.233|1.199|1.331|1.426|\n\n**W3:**\nAs stated in line 116 of our paper, GLCP relies solely on the target agent's data, meaning that it does not involve any communication or data sharing with other agents. As we mentioned in line 139, this reliance on limited target data can lead to unreliable or excessively large prediction sets. This observation motivates our proposed PFCP method, which leverages data from other agents to improve efficiency.\n\n**W4:**\nWe sincerely thank the reviewer for raising the important question regarding the coverage and prediction set size in the personalized setting. \n- [1] consider a setting where the test data is drawn from a mixture of source agents’ distributions—implicitly assuming a certain relationship between the source and target distributions. In contrast, we are considering the scenario when the test data is from a specific (target) agent. As a result, the method of [1] does not guarantee marginal coverage under our setting, which is also confirmed empirically in Figure 1 and Table 1. \n- The results in the above Table reveal an important trade-off: when baseline methods achieve marginal coverage exceeding the nominal level (0.9 in our experiments), their prediction sets become substantially larger than those produced by PFCP. Conversely, when these baselines fail to maintain the required coverage (falling below 0.9), their prediction sets shrink to unrealistically small sizes. However, such under-coverage scenarios render the size comparisons meaningless, as the resulting prediction sets lack the fundamental validity guarantee that forms the basis of conformal prediction.\n- In our proposed PFCP framework, personalization is achieved by leveraging information from source agents through density-ratio weighting, tailored for a specific target agent. This allows for a more accurate estimation of $F(s\\mid x)$ compared to GLCP, which relies solely on target agent data. Consequently, PFCP leads to more efficient prediction intervals (i.e., smaller set sizes), as shown in Table 1.\n\n**W5:**\nWe sincerely thank the reviewer for pointing out the lack of clarity in our explanation. \n- The personalization in our method is primarily achieved through the calibration step (Step 4) is performed on the target agent's data to guarantee marginal coverage on the target distribution. \n- The incorporation of source data is solely for improving the CDF estimation accuracy. The density ratio estimation emerges as a necessary component precisely because we introduce source data to enhance the conditional distribution estimation while maintaining the personalized calibration. \n- This design ensures that while we leverage additional data from source agents to improve the quality of our conformal prediction sets, the marginal coverage on the target agent is strictly guaranteed. \n\n**W6:**\nWe sincerely appreciate this feedback and apologize for any confusion caused by our notation. \n- The complexity arises from the need to precisely represent multiple algorithmic components in our generalized framework. We apologize if the complexity of the notation caused any confusion. \n- To better address your concerns, could you kindly specify which particular notational aspects you found challenging to follow? This would enable us to provide more targeted explanations or improvements to the relevant notations.\n\n**Q3:**\nWe sincerely thank the reviewer for highlighting this important aspect. \n- Our algorithm is designed for any type of heterogeneous local data distributions across agents. \n- In our synthetic data experiments, we primarily focus on concept shift, i.e., changes in $P_{Y\\mid X}$ across agents, as described in Section 3.1. \n- For the real-world datasets, both covariate and concept heterogeneity exist among agents. \n\nWe are more than happy to answer any further questions during the Discussion.\n\n**Reference**\n\n[1] Charles Lu, Yaodong Yu, Sai Praneeth Karimireddy, Michael Jordan, and Ramesh Raskar. Federated conformal predictors for distributed uncertainty quantification. In International Conference on Machine Learning, pages 22942–22964. PMLR, 2023.\n\n[2] Vincent Plassier, Mehdi Makni, Aleksandr Rubashevskii, Eric Moulines, and Maxim Panov. Conformal prediction for federated uncertainty quantification under label shift. In International Conference on Machine Learning, pages 27907–27947. PMLR, 2023.\n\n[3] Vincent Plassier, Alexander Fishkov, Maxim Panov, and Eric Moulines. Conditionally valid probabilistic conformal prediction. stat, 1050:1, 2024.\n\n[4] Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, and Maxim Panov. Efficient conformal prediction under data heterogeneity. In International Conference on Artificial Intelligence and Statistics, pages 4879–4887. PMLR, 2024."}, {"review_id": "7jviSZsv5l", "reviewer": "Reviewer_Tp6A", "summary": "The authors propose PFCP (Personalized Federated Conformal Prediction), which aims to combine personalized federated learning with conformal prediction. This can provide uncertainty sets that are both agent-specific and instance-specific. The authors also introduce GLCP (Generalized Localized Conformal Prediction) an extension of personalized federated learning that can accommodate any conditional distribution estimator. The proposed method PFCP shows how to leverage data from source agents to improve prediction sets --compared to GLCP which can be excessively large-- for a target agent while maintaining privacy and marginal coverage guarantees.", "questions": "Please address Weaknesses W1, W2, and W3.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1.  Combining both personalized federated learning and conformal prediction addresses an important field.\n2.  The paper provides formal marginal coverage guarantees for their method.\n3. The paper is well written and easy to follow.\n\nWeaknesses:\n1. The authors mention that the works of Plassier et al. [27, 28] are the most closely related works. However, they do not compare against these important baslines. Could the authors highlight the key differences of why they are not in the comparison table? Otherwise, could they add the results of these methods in Table 1?\n2. Limited discussion of when the method might fail. Could the authors expand on this?\n3. I am not sure if \"personalized\" federated learning is the proper term to be used in this context. This method only improves the prediction sets for the target agent satisfying Assumption 2. Could the authors reflect on this?\n\n[27] Vincent Plassier, Alexander Fishkov, Maxim Panov, and Eric Moulines. Conditionally valid probabilistic conformal prediction. stat, 1050:1, 2024.\n[28] Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, and Maxim Panov. Efficient conformal prediction under data heterogeneity. In International Conference on Artificial Intelligence and Statistics, pages 4879–4887. PMLR, 2024.", "was_revised": true, "final_justification": "The authors addressed my concerns about the lacking 2 baselines, I appreciate their experimental setup which seems modular enough to run things easily during the rebuttal phase. I think this is technically strong and I don't have other remaining pressing concerns.\nThat said, I don't think this paper would have a high impact on the field of conformal prediction nor a moderate impact on multiple areas of AI, hence I didn't recommend 5.", "rebuttal": "We are grateful for your positive perspective recognizing the importance of combining personalized federated learning with conformal prediction, as well as our method's formal marginal coverage guarantees and clear presentation. We are grateful to the reviewer for the insightful and constructive feedback. These comments have significantly contributed to improving the quality and presentation of our work. Below we provide detailed responses and clarifications to address the raised concerns.\n\n**W1:**\nWe appreciate this important suggestion. \n- First, we would like to clarify a typo in our citation in line 96. We mistakenly cited [1,2], but it should have been [1] and [3], which correspond to their approaches under label shift and covariate shift settings, respectively. We denote these two methods as CPhet [3] and CPlab [1]. They share similarities with our approach but are designed for very different settings. \n- They are not compared because they cannot guarantee marginal coverage under our setup when the test data is from the target agent distribution, which is different from the source agents'.\n- We have now included these baselines in our experiments. Real data experiment results are shown in the table below:\n\n||Dataset|GLCP|PFCP|FedCP|FedCP-QQ|CPhet|CPlab|\n|-|-|-|-|-|-|-|-|\n|Marginal|BIO|0.902|0.903|**0.993**$\\times$|**0.970**$\\times$|**0.981**$\\times$|**0.978**$\\times$|\n||BIKE|0.900|0.900|**0.885**$\\times$|**0.890**$\\times$|**0.883**$\\times$|**0.885**$\\times$|\n||CRIME|0.900|0.896|**0.866**$\\times$|**0.852**$\\times$|**0.864**$\\times$|**0.863**$\\times$|\n||STAR|0.898|0.899|0.897|**0.910**$\\times$|0.899|0.897|\n||CONCRETE|0.903|0.905|**0.947**$\\times$|**0.963**$\\times$|**0.946**$\\times$|**0.949**$\\times$|\n||DERMA|0.895|0.895|**0.830**$\\times$|**0.819**$\\times$|**0.849**$\\times$|**0.867**$\\times$|\n|Miscoverage|BIO|0.0315|**0.0199**(36.8%)|0.0941|0.0753|0.0844|0.0822|\n||BIKE|0.0234|**0.0184**(21.4%)|0.0678|0.0647|0.0690|0.0681|\n||CRIME|0.0387|**0.0268**(30.7%)|0.0426|0.0495|0.0429|0.0439|\n||STAR|0.0392|**0.0243**(38.0%)|0.0500|0.0492|0.0496|0.0495|\n||CONCRETE|0.0366|**0.0238**(35.0%)|0.0582|0.0675|0.0580|0.0600|\n||DERMA|0.0307|**0.0264**(14.0%)|0.0939|0.1004|0.0802|0.0701|\n|Size|BIO|0.514|**0.503**(2.1%)|0.999|0.709|0.827|0.805|\n||BIKE|4.097|**3.988**(2.7%)|4.004|4.091|3.967|3.996|\n||CRIME|5.183|**4.496**(13.3%)|3.918|3.772|3.886|3.880|\n||STAR|48.50|**43.02**(11.3%)|42.80|45.30|43.08|42.85|\n||CONCRETE|34.49|**29.38**(17.7%)|34.71|38.97|34.69|35.15|\n||DERMA|2.409|**2.327**(3.4%)|1.233|1.199|1.331|1.426|\n\n\n\n**W2:**\nThank you for highlighting this. \n- Under our setup with test data from the target agent distribution, both GLCP and PFCP consistently guarantee the desired marginal coverage regardless of the distribution of target and source agents. \n- In contrast, FedCP, FedCP-QQ, CPhet and CPlab do not maintain such guarantees when the target distribution differs from the source agents’. \n- For the performance of test-conditional coverage, we outline the limitations of PFCP as follow:\n  - As shown in Theorem 2.8, achieving better conditional coverage than GLCP requires a moderately well-trained federated conditional density estimator, such that $\\delta_1(x,\\hat{F}_{\\rm agg})<\\delta_1(x,\\hat{F})$, thereby leading to improved conditional coverage. However, if the density ratio is poorly estimated, the aggregation may not result in any improvement.\n  - If the source distribution significantly differs from the target distribution, for instance the support of $P$ and any $P_k$ do not intersect, the information from source agents cannot help improve the conformal set on the target agent.\n  - When the data on the target agent is extremely limited, insufficient to yield a reliable estimate of the conditional distribution $F(s\\mid x)$ — both GLCP and PFCP may not deliver conformal sets with satisfying test-conditional coverage, though PFCP may still outperform GLCP in the sense of test-conditional coverage. Nonetheless, the marginal coverage remains valid in both cases.\n\nWe'll add these discussions in the revised version.\n\n\n**W3:**\nWe thank the reviewer for raising this important point. \n- Personalized setting refers to scenarios where agent-specific feature-label relationships differ substantially. The key characteristic of our problem is that test samples originate from a single designated target agent, and our primary objective is to guarantee the validity of conformal prediction specifically for this target agent. Our coverage guarantees are explicitly personalized to the target agent's distribution and simultaneously improving local (instance-specific) predictive performance. \n- Indeed, Assumption 2 serves as a condition to ensure the transferability of information from source to target domains. The fundamental requirement is that the conditional distribution $f(s\\mid x)$ in the target domain can be effectively reconstructed through weighted combinations of source conditional distributions $\\sum_k\\pi_kf_k(s\\mid x)$. \n- While Assumption 2 assumes a uniform lower bound on the density ratio $\\frac{f(s\\mid x)}{f_k(s\\mid x)}$ for all $(s,x)$ pairs and some source distribution $P_k$, the actual necessary condition is substantially weaker. What we truly require is that the mixture of source distributions contains sufficient information about the target, meaning the corresponding density ratio $\\sum_k\\pi_kf_k(s\\mid x)/f(s\\mid x)$ has lower bounded (equation between line 560-561). This much milder condition would still guarantee that the target conditional distribution can be adequately approximated through our federated estimation framework, while being more realistic in practical applications. We will add more discussion on Assumption 2 in the revised version.\n\nWe are more than happy to answer any further questions during the Discussion.\n\n **Reference**\n\n[1] Vincent Plassier, Mehdi Makni, Aleksandr Rubashevskii, Eric Moulines, and Maxim Panov. Conformal prediction for federated uncertainty quantification under label shift. In International Conference on Machine Learning, pages 27907–27947. PMLR, 2023.\n\n[2] Vincent Plassier, Alexander Fishkov, Maxim Panov, and Eric Moulines. Conditionally valid probabilistic conformal prediction. stat, 1050:1, 2024.\n\n[3] Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, and Maxim Panov. Efficient conformal prediction under data heterogeneity. In International Conference on Artificial Intelligence and Statistics, pages 4879–4887. PMLR, 2024."}, {"review_id": "LGYcfmFc98", "reviewer": "Reviewer_P7Ti", "summary": "This paper introduces a conformal prediction method to quantify the uncertainty of algorithms in a personalized Federated Learning (FL) setting. An important distinction from prior works is that each agent has its own prediction function, a characteristic of the personalized FL setting. The authors claim to propose a method that 1) constructs conformal prediction sets for a target agent using data from all agents in a privacy-preserving manner 2) ensures marginal coverage and asymptotic conditional coverage. Finally, they show the efficiency of their method through various experiments.", "questions": "1\\ As already said, Theorem 2.4 only depends on $n$ and not on all the data points. Hence, how do you theoretically know that your method uses all the points for the coverage? \n\n2\\ Since the article's contribution is mainly a new score function, what are the particularities of the techniques used for the proofs compared to other CP articles that don't deal with federated learning?\n\n3\\ It is say that the method is private but the privacy is never quantify. How do you know that your method is private? And are you able to give theoretical results about that?\n\n4\\ In line 186, it is said that « we obtain superior estimates that simultaneously improve conditional coverage accuracy and yield tighter prediction sets. » How do you know that since no theoretical results are given about the size of the sets ?\n\n5\\ In experiments, which score function is used for FedCP and FedCPQQ ?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 1, "strengths_and_weaknesses": "------- Strengths -------\n\n1. The paper is well-written, and the subject is of significant interest to the conformal prediction community. \n\n2. The method is clearly explained. \n\n3. The related works section is thoroughly detailed.\n\n\n------- Weaknesses -------\n\n1. The article is really just an implementation of a new score function adapted to the federated learning framework.\n\n2. The bound in Theorem 2.4 only depends on $n$, the number of points for the $K+1$-th agent. In contrast, in Theorem 2.2 of [1] for instance, the bound depends on $N$ the total number of points (i.e. the summation of the number of points of each agent).\n\n3. Overall, it seems to me that the main contribution of this article is more about how to build CDF estimators in FL than how to perform conformal prediction in a personalized federated learning context (in a sens that the conformal part used only standard results).\n\n4. (minor) The size of returned sets must always be indicated. For instance, this is not the case in Figure 1.\n\n[1] Conformal Prediction for Federated Uncertainty Quantification Under Label Shift, Plassier et. al", "was_revised": true, "final_justification": "The authors made a great effort in their responses to my questions.\nAfter the rebuttal and based on the other reviews, I raised my score to 4.", "rebuttal": "We thank the reviewer for the positive feedback regarding the paper's clarity, methodological presentation, and comprehensive literature review, as well as thoughtful and valuable suggestions. We address the key concerns below:\n\n**W1\\&Q2\\&W3:**\nWe respectfully clarify that our contribution is *not* limited to introducing a new score function and disagree with the claim that our contribution is mainly about constructing CDF estimators. \n- First, we consider conformal prediction under the PFCP setting, where test data comes from the target agent, not the mixture of all source agents. Our framework not only guarantees marginal coverage on the target agent regardless of the source data distribution, but also improves test-conditional coverage for the target agent, which is an objective that existing federated conformal prediction methods have not addressed. This novel formulation, as noted by Reviewer kS3y, distinguishes our work from other FCP approaches.\n- Second, the fundamental motivation behind PFCP is to simultaneously guarantee marginal coverage while enhancing test-conditional coverage on the target agent. The key innovation resides in effectively transferring information from source agents to improve the conditional coverage performance on the target agent. This strategic information transfer, achieved through our carefully designed framework, is novel as noted by Reviewer kS3y.\n- Third, while the core technical contribution of our work lies in the construction of novel conformal scores, these scores are not simple function mappings. They are carefully designed fusion estimators that incorporate information from other source agents while strictly preserving the marginal coverage guarantee.\n- Our theoretical results go beyond standard CP analyses. In particular, Lemma 2.6 establishes a crucial theoretical connection between the CDF estimator and the test-conditional coverage of conformal prediction, which serves as the fundamental basis for analyzing the improvement. Further, Theorem 2.8 provides a non-asymptotic bound demonstrating that PFCP improves conditional coverage over using only local (target-agent) data (GLCP), thus justifying the benefit of leveraging source information through our reweighting scheme. This theoretical analysis proves PFCP attains superiority over GLCP and prior works, as noted by Reviewer 2MYP.\n\n**W2\\&Q1:**\nThanks for the question regarding the difference in sample size dependence between our work and [2]. The distinction stems from fundamentally different problem settings and objectives. Our PFCP framework has several key differences compared with [2]: \n- In PFCP setting, the test sample comes from the target agent, with distribution different from source agents, and we make minimal distributional assumptions: we only require the target agent's n samples to be i.i.d., without imposing any restrictive requirements between source and target agents, nor assuming known density ratios of any form. This is because we operate in a more challenging environment where source and target agents have different distributions (not limited to only label shift or covariate shift); \n- [2] focuses specifically on label shift scenarios where $X\\mid Y$ distributions are identical across domains. Their Theorems 2.1 and 2.2 show that with *known* density ratios of $Y$, weighted conformal prediction (WCP) can leverage all $N$ samples. In practice (as noted in their Theorem 2.4), when density ratios must be estimated, marginal coverage error includes an additional term $R$ from estimation error, and this breaks the exact marginal coverage guarantee in our PFCP setting. \n- Our method provides strict finite-sample marginal coverage guarantees regardless of source agent data quality because we only rely on the target agent's $n$ samples for coverage calibration and source data only assists in improving conditional coverage. This is formalized in Theorem 2.8, which provides a non-asymptotic bound showing that our approach (PFCP) can improve conditional coverage compared to methods using only local data. The estimation of the density ratio $\\hat{r}$, which is crucial for this improvement, depends on data from all source agents.\n\n**W4:**\nThank you for pointing this out. We have now included the size of the prediction sets in the Table below. Since the other baseline methods fail to consistently achieve the marginal coverage requirement, we omit detailed size comparisons for these approaches. Our results demonstrate that the density ratio estimation using calibrated neural networks (PFCP2) yields significantly more efficient prediction sets compared to the logistic regression approach (PFCP1) and GLCP.\n\n|d|10|15|20|25|30|10|15|20|25|30|\n|-|-|-|-|-|-|-|-|-|-|-|\n||||S1.1|||||S1.2|||\n|GLCP|**3.60**|**4.42**|**5.08**|**5.91**|**6.84**|**3.60**|**4.42**|**5.08**|**5.91**|**6.84**|\n|PFCP1|**3.76**|**3.97**|**4.28**|**4.88**|**5.90**|**4.23**|**4.12**|**4.32**|**5.02**|**5.76**|\n||-4.34%|10.01%|15.73%|17.49%|13.79%|-17.52%|6.74%|14.95%|15.08%|15.75%|\n|PFCP2|**2.97**|**3.20**|**3.69**|**4.59**|**5.57**|**2.97**|**3.21**|**3.62**|**4.46**|**5.39**|\n||17.38%|27.49%|27.28%|22.36%|18.57%|17.62%|27.22%|28.75%|24.54%|21.27%|\n||||S1.1|||||S1.1||\n|GLCP|**4.28**|**4.88**|**5.29**|**6.16**|**7.15**|**4.28**|**4.88**|**5.29**|**6.16**|**7.15**|\n|PFCP1|**4.12**|**4.44**|**4.61**|**5.20**|**6.29**|**4.25**|**4.48**|**4.65**|**5.23**|**6.24**|\n||3.64%|8.86%|12.87%|15.60%|11.94%|0.69%|8.13%|12.06%|15.14%|12.76%|\n|PFCP2|**3.61**|**3.90**|**4.17**|**5.05**|**6.08**|**3.57**|**3.87**|**4.07**|**4.84**|**5.90**|\n||15.54%|20.03%|21.15%|18.02%|14.95%|16.45%|20.67%|23.08%|21.56%|17.41%|\n\n**Q3:**\nThanks for raising this important point. \n- In our framework, privacy is maintained in the following ways: (i) no agent accesses other agents’ raw local data; (ii) information exchange only occurs during the estimation of $\\widehat{F}_{\\text{mix}}$ and the density ratio, both estimation optimized via the FedAvg algorithm; and (iii) the estimated CDF involves only conditional distribution of conformity scores which contains no invertible mapping to raw features or labels.\n- We acknowledge that although FedAvg is widely used, it may still carry risks of privacy leakage. However, we can directly replace FedAvg with existing privacy-preserving federated optimization techniques, such as differentially private mechanisms [3] or secure aggregation protocols [4], to provide formal privacy guarantees. \n\nWe will add more discussion in the revised version and consider this a direction for future work.\n\n**Q4:**\nWe appreciate the reviewer’s insightful question. It touches upon an important aspect of our method’s performance. \n- As stated in Theorem 2.8, our work provides a theoretical guarantee on the improvement of conditional coverage. However, we acknowledge that a formal theoretical result on the reduction of prediction set size is not currently provided. \n- Instead, we substantiate this claim through extensive empirical evidence from both simulation studies and real-world datasets, where PFCP consistently outperforms GLCP in producing smaller prediction sets. Please refer to Table 1 in the original manuscript, Figure 3 and Figures 16--19 in the appendix, as well as Table below and Table above provided in our response to W4, for detailed results supporting this observation.\n\n||Dataset|GLCP|PFCP|FedCP|FedCP-QQ|CPhet|CPlab|\n|-|-|-|-|-|-|-|-|\n|Marginal|BIO|0.902|0.903|**0.993**$\\times$|**0.970**$\\times$|**0.981**$\\times$|**0.978**$\\times$|\n||BIKE|0.900|0.900|**0.885**$\\times$|**0.890**$\\times$|**0.883**$\\times$|**0.885**$\\times$|\n||CRIME|0.900|0.896|**0.866**$\\times$|**0.852**$\\times$|**0.864**$\\times$|**0.863**$\\times$|\n||STAR|0.898|0.899|0.897|**0.910**$\\times$|0.899|0.897|\n||CONCRETE|0.903|0.905|**0.947**$\\times$|**0.963**$\\times$|**0.946**$\\times$|**0.949**$\\times$|\n||DERMA|0.895|0.895|**0.830**$\\times$|**0.819**$\\times$|**0.849**$\\times$|**0.867**$\\times$|\n|Miscoverage|BIO|0.0315|**0.0199**(36.8%)|0.0941|0.0753|0.0844|0.0822|\n||BIKE|0.0234|**0.0184**(21.4%)|0.0678|0.0647|0.0690|0.0681|\n||CRIME|0.0387|**0.0268**(30.7%)|0.0426|0.0495|0.0429|0.0439|\n||STAR|0.0392|**0.0243**(38.0%)|0.0500|0.0492|0.0496|0.0495|\n||CONCRETE|0.0366|**0.0238**(35.0%)|0.0582|0.0675|0.0580|0.0600|\n||DERMA|0.0307|**0.0264**(14.0%)|0.0939|0.1004|0.0802|0.0701|\n|Size|BIO|0.514|**0.503**(2.1%)|0.999|0.709|0.827|0.805|\n||BIKE|4.097|**3.988**(2.7%)|4.004|4.091|3.967|3.996|\n||CRIME|5.183|**4.496**(13.3%)|3.918|3.772|3.886|3.880|\n||STAR|48.50|**43.02**(11.3%)|42.80|45.30|43.08|42.85|\n||CONCRETE|34.49|**29.38**(17.7%)|34.71|38.97|34.69|35.15|\n||DERMA|2.409|**2.327**(3.4%)|1.233|1.199|1.331|1.426|\n\n**Q5:**\n- Since all the datasets used in our main experiments are regression tasks, all methods (including FedCP and FedCPQQ) adopt the standard residual score function, defined as $s(x,y) = |y - f(x)|$, where $f$ is the pre-trained regression model. \n- For the classification task (DermaMNIST dataset) mentioned by Reviewer kS3y, we use the common classification score function $s(x,y) = 1 - [f(x)]_y$, where $f(x)$ returns the predicted class probabilities and $[f(x)]_y$ denotes the predicted probability of the true label $y$.\n\nWe are more than happy to answer any further questions during the Discussion.\n\n**Reference**\n\n[1] Charles Lu, et al. (2023) Federated conformal predictors for distributed uncertainty quantification.\n\n[2] Vincent Plassier, et al. (2023) Conformal prediction for federated uncertainty quantification under label shift.\n\n[3] McMahan, H. B., et al. (2017). Learning differentially private recurrent language models.\n\n[4] Bonawitz, Keith, et al. (2017). Practical secure aggregation for privacy-preserving machine learning."}, {"review_id": "K8OCfEU8Af", "reviewer": "Reviewer_2MYP", "summary": "This paper considers the setup of federated learning, where the data comes from multiple agents, and the goal is to quantify the uncertainty of a test agent. The challenge is that the source and target agents have different distributions. This work proposes “personalized federated conformal prediction”, a method that constructs prediction sets that contain the responses of the test agent at a user-specified probability. This method builds on conformal prediction and localized conformal prediction. The theoretical analysis indicates that the proposed aggregation of conditional distribution estimators should attain better conditional coverage compared to the baseline Generalized Localized Conformal Prediction. Experiments on synthetic and real datasets reveal that the proposed PFCP performs better than prior work.", "questions": "## Questions\n1.\tIn line 200, what does “valid” estimator mean?\n2.\tThe informativeness of the bounds in Lemma 2.6 and Theorem 2.8 is not clear. Is it possible that the bounds might get trivial in practice?\n3.\tIn line 249, $\\tilde{\\delta}$ is defined as an integral over $x$ or $s$?\n4.\tIn line 266, over what the randomness is taken? Random data splits?\n5.\tHow is the test-conditional miscoverage computed in the experiments?\n\n## Minor comments\n1.\tThe proof for Lemma 2.1 follows directly from the theory of conformal prediction, as in [1], but it is not stated in the paper. Since there is no proof for this Lemma, it should be at least noted that it follows from general results on conformal inference.  \n2.\tThe weight $\\pi$ is defined in line 111, but it is only used in line 147. It should be defined close to its first use for clarity.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n1.\tThe proposed PFCP attains better performance compared to the baseline GLCP and prior work.\n2.\tThis paper provides a theoretical analysis for the worst-case conditional coverage based on the accuracy of the density estimation. While the assumptions are complex, the authors provide an interpretation for each assumption.\n3.\tThe PFCP and GLCP methods have a valid marginal coverage guarantee, by building on conformal prediction.\n4.\tThe paper is written clearly, presented well, and easy to follow.\n\n\n## Weaknesses\n1.\tThe proposed GLCP is very similar to the localized conformal prediction [1] and to distributional conformal prediction [2]. In contrast to the statement in line, 67, in my opinion, this extension is marginal and not novel.\n2.\tThe experiments do not include the test-conditional miscoverage error of FedCP and FedCP-QQ.\n3.\tThe experiments in [3] show that FedCP-QQ achieves a marginal coverage rate that is very close to $1-\\alpha=90\\%$ on the same real datasets. In contrast, the experiments in this paper show that the coverage rate attained by this method is far from the nominal level. This discrepancy between the two results is not addressed by the authors.\n4.\tThe real-dataset experiments do not include a dataset that is suitable for federated learning setup, such as MedMNIST used in [4].\n\n\n\n## References\n[1] Leying Guan. Localized conformal prediction: A generalized inference framework for conformal prediction. Biometrika, 110(1):33–50, 2023\n\n[2] Victor Chernozhukov, Kaspar Wüthrich, and Yinchu Zhu. Distributional conformal prediction. In Proceedings of the National Academy of Sciences, volume 118. National Acad Sciences, 2021.\n\n[3] Pierre Humbert, Batiste Le Bars, Aurélien Bellet, and Sylvain Arlot. One-shot federated conformal prediction. In International Conference on Machine Learning, pages 14153–14177. PMLR, 2023\n\n[4] Charles Lu and Jayasheree Kalpathy-Cramer. Distribution-Free Federated Learning with Conformal Predictions, 2022.", "was_revised": true, "final_justification": "The authors conducted additional experiments that highlight the advantages of their proposed approach.", "rebuttal": "We sincerely thank the reviewer for your thoughtful and encouraging assessment of our work. We are particularly grateful for your recognition of PFCP's theoretical and empirical advantages over existing methods, and the marginal guarantee of our methods under personalized federated setting, as well as your appreciation of our efforts to maintain clear presentation while tackling technically complex problems. Their insightful comments have helped us further improve the manuscript. Below we provide detailed responses to all points raised.\n\n**W1:**\nWe appreciate the reviewer's insightful observation regarding the relationship between GLCP and existing localized conformal prediction methods. \n- We would like to clarify that GLCP was indeed designed to *generalize* and *unify* existing approaches. \n- When using kernel conditional CDF estimation (NW estimator), GLCP reduces to Localized Conformal Prediction (LCP). With identity score mapping, GLCP becomes equivalent to Distributional Conformal Prediction (DCP). \n\nWe will revise and tone down our statement on GLCP accordingly in the revised version.\n\n**W2\\&W4:**\nThanks for the valuable feedback regarding reporting test-conditional miscoverage error of FedCP and FedCP-QQ, and including a dataset that is suitable for federated learning setup. \n- In response, we have supplemented table below with new real data results comparing test-conditional miscoverage rates, including evaluations on: (1) the DermaMNIST classification dataset from MedMNIST that is suitable for federated learning setup, and (2) two additional baselines (CPhet [1] and CPlab [2]).\n||Dataset|GLCP|PFCP|FedCP|FedCP-QQ|CPhet|CPlab|\n|-|-|-|-|-|-|-|-|\n|Marginal|BIO|0.902|0.903|**0.993**$\\times$|**0.970**$\\times$|**0.981**$\\times$|**0.978**$\\times$|\n||BIKE|0.900|0.900|**0.885**$\\times$|**0.890**$\\times$|**0.883**$\\times$|**0.885**$\\times$|\n||CRIME|0.900|0.896|**0.866**$\\times$|**0.852**$\\times$|**0.864**$\\times$|**0.863**$\\times$|\n||STAR|0.898|0.899|0.897|**0.910**$\\times$|0.899|0.897|\n||CONCRETE|0.903|0.905|**0.947**$\\times$|**0.963**$\\times$|**0.946**$\\times$|**0.949**$\\times$|\n||DERMA|0.895|0.895|**0.830**$\\times$|**0.819**$\\times$|**0.849**$\\times$|**0.867**$\\times$|\n|Miscoverage|BIO|0.0315|**0.0199**(36.8%)|0.0941|0.0753|0.0844|0.0822|\n||BIKE|0.0234|**0.0184**(21.4%)|0.0678|0.0647|0.0690|0.0681|\n||CRIME|0.0387|**0.0268**(30.7%)|0.0426|0.0495|0.0429|0.0439|\n||STAR|0.0392|**0.0243**(38.0%)|0.0500|0.0492|0.0496|0.0495|\n||CONCRETE|0.0366|**0.0238**(35.0%)|0.0582|0.0675|0.0580|0.0600|\n||DERMA|0.0307|**0.0264**(14.0%)|0.0939|0.1004|0.0802|0.0701|\n|Size|BIO|0.514|**0.503**(2.1%)|0.999|0.709|0.827|0.805|\n||BIKE|4.097|**3.988**(2.7%)|4.004|4.091|3.967|3.996|\n||CRIME|5.183|**4.496**(13.3%)|3.918|3.772|3.886|3.880|\n||STAR|48.50|**43.02**(11.3%)|42.80|45.30|43.08|42.85|\n||CONCRETE|34.49|**29.38**(17.7%)|34.71|38.97|34.69|35.15|\n||DERMA|2.409|**2.327**(3.4%)|1.233|1.199|1.331|1.426|\n\n- Moreover, we would like to clarify that in our personalized federated learning setting with test data from the target agent distribution, ensuring valid marginal coverage is the fundamental requirement to be satisfied first, as it forms the basis for a reliable conformal prediction set. FedCP, FedCP-QQ, CPhet, and CPlab cannot guarantee this crucial marginal coverage property in our setting. \n- Although FedCP, FedCP-QQ, CPhet, and CPlab cannot guarantee marginal coverage, they may occasionally achieve the desired coverage level in certain scenarios (e.g., the STAR dataset). Even in these cases, GLCP and PFCP outperform the other methods in test-conditional miscoverage error.\n\n**W3:**\nWe sincerely appreciate the detailed comment regarding this experimental detail. \n- We would like to clarify that FedCP-QQ requires all agents' data to be i.i.d. to achieve marginal coverage, which fundamentally contradicts our PFCP setting where the target agent and source agents follow different data distributions and the test point is from the target agent distribution. \n- In our experimental design, we artificially partition agents such that they follow distinct distributions to reflect the personalized federated learning scenario - while FedCP-QQ employs random sampling from the entire dataset for each agent in their experiments to reflect the i.i.d. case. \n- Thus, under our experiment setup, the distributional heterogeneity between target and source agents inherently violates FedCP-QQ's i.i.d. assumption and consequently undermines its marginal coverage guarantees.\n\n**Q1:**\n- The sentence in line 200 was originally intended to describe the existence of two different approaches to estimating $F(s\\mid x)$. However, the use of the word \"valid\" may have been somewhat misleading. We meant they are both consistent estimators of $F(s\\mid x)$ under certain conditions theoretically. \n- In the revised version, to avoid confusion, we will rephrase it as:\n\n\"both $\\mathbb{E}\\_{S\\sim\\widehat{F}}\\mathbb{1}(S\\leq s)$ and $\\mathbb{E}\\_{S\\sim\\widehat{F}\\_{\\rm mix}(s\\mid x)}\\\\{\\mathbb{1}(S\\leq s)\\widehat{r}(S\\mid x)\\\\}$ can serve as estimators of $F(s\\mid x)$\".\n\n\n**Q2:**\nWe appreciate this insightful question on our theoretical results. \n- In Lemma 2.6, the first term in the right-hand side of the inequality represents the global calibration error, which is typically small and scales with the target agent's sample size $n$. \nWhen $\\widehat{F}$ is the empirical distribution estimator, this term approximately equals $\\epsilon + \\delta + 1/n$. \n- The second term quantifies the estimation accuracy of the conditional distribution estimator, indicating that pointwise test-conditional coverage is governed by the local precision of the conditional distribution estimation. \n- Theorem 2.8 further establishes that the aggregated conditional distribution estimation is controlled by both the individual conditional distribution estimation accuracies and the precision of the density ratio estimation. In practice, source data is often abundant, enabling accurate estimation of its conditional distribution, in which case the overall precision becomes primarily determined by the density ratio estimation error. When sufficient relevance exists between source and target domains, this density ratio estimation can be performed with higher precision. \n- Even though it is unclear whether the bounds in Lemma 2.6 and Theorem 2.8 are sharp,  they are sufficient to provide theoretical justification for PFCP's improved performance compared to GLCP, which is the main focus of our theoretical study.\n\n\n**Q3:**\nThank you for pointing out this typo. The integral should indeed be taken with respect to the score variable $s$ rather than the input variable $x$.\n\n**Q4:**\n- For synthetic data, the randomness is introduced through different random seeds when sampling from the given distributions. \n- For real-world datasets, the stochasticity arises from different random seeds used to partition the data into training, calibration, and test sets.\n\n**Q5:**\n- For synthetic data, we generate 100 prediction sets $C_1(x_i),...,C_{100}(x_i)$ for each test point $x_i$, compute the corresponding coverage using its true distribution we get $p_1(x_i),...,p_{100}(x_i)$, and estimate the conditional coverage at $x_i$ by averaging these values. The test-conditional miscoverage is then obtained by averaging across all test points. \n- For real-world data, we first partition the feature space into $k$ clusters using $k$-means. During each experimental repeat, we compute the empirical coverage within each cluster, then average these values across $100$ repeats to obtain the test-conditional coverage estimates. The final test-conditional miscoverage metric is calculated as a weighted average across all clusters, with weights proportional to the number of test samples in each cluster.\n\n**Minor1:**\nWe sincerely thank the reviewer for identifying this omission. Lemma 2.1 follows directly from the basic validity guarantees of conformal prediction theory. In the revised version, we will add this citation.\n\n**Minor2:**\nThank you for the suggestion to improve the clarity of our notation. In the revised manuscript, we will move the definition of the weight $\\pi$ closer to its first usage in line 147.\n\nWe are more than happy to answer any further questions during the Discussion.\n\n**Reference**\n\n[1] Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, and Maxim Panov. Efficient conformal prediction under data heterogeneity. In International Conference on Artificial Intelligence and Statistics, pages 4879–4887. PMLR, 2024.\n\n[2] Vincent Plassier, Mehdi Makni, Aleksandr Rubashevskii, Eric Moulines, and Maxim Panov. Conformal prediction for federated uncertainty quantification under label shift. In International Conference on Machine Learning, pages 27907–27947. PMLR, 2023."}]

Author rebuttal:

Meta-review: This paper presents a personalized federated conformal prediction method which can provide agent-specific and instance-localized uncertainty quantification. The work comes with finite-sample marginal coverage guarantees, as well as extensive experiments across synthetic and real-world datasets. 

After initial reviews and discussions, the four reviewers unanimously praised the novelty and significance of this work in addressing an interesting problem of reliable uncertainty estimates in heterogeneous federated environments. Some minor concerns regarding presentation clarity, privacy claims, and experimental setups were effectively resolved in rebuttals and reviewer-author discussions. So the resulting sentiment is that this is a novel and technically sound FL paper worthy to be accepted to NeurIPS 2025. The authors are encouraged to address the issues raised in the reviews as much as possible in the revised paper.
Decision: Accept (poster)