Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
Authors: Chenyin Gao
Peter B. Gilbert
Larry Han
Venue: ICML 2025 poster (2025)
Keywords: Conformal inference
Individualized prediction
Clustering
Surrogate outcomes
Group fairness
Primary area: general_machine_learning->evaluation
TL;DR: We propose a novel surrogate-assisted clustered conformal inference framework that empowers the construction of efficient prediction sets by pooling the protected groups into larger clusters and leveraging the surrogates.

Abstract: Standard conformal prediction ensures marginal coverage but consistently undercovers underrepresented groups, limiting its reliability for fair uncertainty quantification. Group fairness requires prediction sets to achieve a user-specified coverage level within each protected group. While group-wise conformal inference meets this requirement, it often produces excessively wide prediction sets due to limited sample sizes in underrepresented groups, highlighting a fundamental tradeoff between fairness and efficiency. To bridge this gap, we introduce Surrogate-Assisted Group-Clustered Conformal Inference (SAGCCI), a framework that improves efficiency through two key innovations: (1) clustering protected groups with similar conformal score distributions to enhance precision while maintaining fairness, and (2) deriving an efficient influence function that optimally integrates surrogate outcomes to construct tighter prediction sets. Theoretically, SAGCCI guarantees approximate group-conditional coverage in a doubly robust manner under mild convergence conditions, enabling flexible nuisance model estimation. Empirically, through simulations and an analysis of the phase 3 Moderna COVE COVID-19 vaccine trial, we demonstrate that SAGCCI outperforms existing methods, producing narrower prediction sets while maintaining valid group-conditional coverage, effectively balancing fairness and efficiency in uncertainty quantification.

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
Chenyin Gao
Peter B. Gilbert
Larry Han 23
Abstract
1. Introduction
Personalized medicine calls for precise individualized pre- dictions guide patient-level decision-making. Equally critical is robust uncertainty quantification for these predic- tions, particularly in contexts like vaccine safety or efficacy studies, where personalized vaccine recommendations can influence critical individual-level decisions. Examples in- clude tailoring guidance for vulnerable populations, such as immunocompromised individuals, or for individuals evalu- ated based antibody levels against specific pathogens.
Standard conformal prediction ensures marginal coverage but consistently undercovers underrep- resented groups, limiting its reliability for fair un- certainty quantification. Group fairness requires prediction sets to achieve user-specified cover- age level within each protected group. While group-wise conformal inference meets this re- quirement, it often produces excessively wide prediction sets due to limited sample sizes in un- derrepresented groups, highlighting fundamen- tal tradeoff' between fairness and efficiency. To bridge this gap, we introduce Marrogate-Assister Croup-CCustered Conformal Inference (SAGCCI), framework that improves efficiency through two key innovations: (1) clustering protected groups with similar conformal score distributions to en- hance precision while maintaining fairesss, and (2) deriving an efficient influence function that op- timally integrates surrogate outcomes to construct tighter prediction sets. Theoretically, SAGCCI guarantees approximate group-conditions cover- age in doubly robust manner under mild con- vergence conditions, enabling flexible nuisance model estimation. Empirically, through simula- tions and an analysis of the phase 3 Moderna COVE COVID-19 vaccine trial, we demonstrate that SAGCCI outperforms existing methods, pro- ducing narrower prediction sets while maintain- ing valid group-conditional coverage, effectively balancing faimess and efficiency in uncertainty quantification.
An ideal prediction set for uncertainty quantification must meet two essential criteria: (1) valid coverage in finite sam- ples without relying on stringent distributional assumptions, and (2) sufficiently short prediction set lengths to ensure practical utility. Conformal prediction is powerful method for uncertainty quantification, offering distribution-fii cov- erage guarantees alongside reasonable prediction set lengths (Vovk et al., 2009; Lei et al., 2013). However, standard conformal prediction methods ensure marginal coverage only, i.e., that prediction sets contain the true labels on av- erage across the population. While effective in providing distribution-fer guarantees, these methods often fail to pro- vide adequate coverage for sub- tro-grouss which may result in disparities and lead ethical or legal implications. Fair coverage is crucial in socially consequential domains, where prediction intervals can help guide eecision-making related to access to resources, opportunities, or fair treatment. For example, if a mortgage lender's prediction intervals sys- tematically under-cover minority borrowers, they may face inflated interest rates or higher denial rates. In healthcare, if limited supplies of critical antibioties demand triaging decisions, under-coverage for certain demographics could result in denying them lifesaving medications.
Efforts to design conformal inference for fair predictions, such as the approach proposed by Vovk (2012), recommend performing conformal inference separately for each sub- group ensure group-conditional coverage. However, this strategy often produces excessively wide prediction inter- vals due limited sample sizes within smaller uub-groups, reducing their practical value. There is growing need to develop methods that design more efficient prediction sets for sub-groupe particularly contexts where faimesss and
Harvard University, Department of Biostatistics, Boston, MA, USA VVaccine and Infectious Disease and Public Health Sci- ences Divisions, Fred Hutch Cancer Center. Seattle. WA, USA "Northeastern University, Department of Public Health and Health Sciences, Boston, MA, USA. Correspondence to: Larry Han Starhann@northesttmm.cdd.
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
precision are critical. Two promising avenues address this challenge are clustering and the use of surrogate outcomes. Clustered conformal inference, introduced by Ding et al. (2024) for suttcome-dependen inference, provides a more efficient solution for achieving group-conditional coverage. Instead of performing group-wise conformal inference, this method clusters groups with similar conformalized score distributions into larger clusters, allowing for more efficient individualized predictions while maintaining fairess. Addi- tionally, surrogate outcomes- such as biological markers or machine-learned noisy predictions - offer another powerful approach to improving efficiency in uncertainty quantifica- tion. These surrogates, which are often easier to observe or collect than the primary outcomes of interest, enhance effi cieney, especially in scenarios where primary outcomes are challenging to measure or largely unavailable. For instance, in vaccine studies, immune correlates like neutralizing anti- body titers have been shown to serve as effective surrogates for predicting vaceine efficacy (Gilbert et al., 2022; 2024).
Since then, numerous surrogate validation methods have been developed to assess surrogate outcomes, which Conlon et al. (2017) broadly categorized into two main approaches: causal effects (Gilbert & Hudgens, 2008) and causal associ- ation frameworks (Li et al., 2010; Alonso et al., 2016). For a detailed review of these methods and recent advancements, see Elliott (2023). Surrogate validation aims to identify out- comes that can replace primary endpoints in clinical trials, thereby facilitating the expedited approval of treatments, particularly for critical illnesses. Regulatory agencies like the FDA have relied on surrogate endpoints to accelerate drug approvals (FDA, 1992; 2021). However, surrogate out- comes are not always reliable, as illustrated by cases where treatments approved based on surrogate markers, such as arrhythmia suppression, ultimmateyy led to adverse outcomes like increased mortality in follow-up studies (Fleming & DeMets, 1996). To address these challenges, surrogate val- idation methods often impose stringent assumptions, such as the strong statistical surrogate condition (Prentice, 1989), which can be difficult to verify in practice. For example, Athey et al. (2019) relied on such assumptions to integrate experimental and observational datasets.
Our contributions In this paper, we propose a novel Conformal Inference (SAGCCI) framework that achieves group-conditional cov- erage while improving the efficiency of uncertainty quantifi- cation through two key innovations:
Surrogate-Assisted
In contrast, our work treats surrogates auxiliary informa- tion or "helper covariates" (Xia & Wainwright, 2024) rather than as substitutes for primary outcomes. This perspective aligns with recent work that leverages surrogate outcomes to improve the efficiency of estimating long-term or primary treatment effects. For instance, Athey et al. (2020) proposed combining experimental data (containing only surrogates) with observational data (containing both surrogates and pri- mary outcomes) to estimate long-term treatment effects. Chen & Ritzwoller (2023) considered this framework and derived semiparametric efficiency bounds. Other studies, in- cluding Cheng et al. (2021) and Kallus & Mao (2024), have explored efficient estimation methods that rely on a limited number of primary outcomes alongside abundant surrogates, without requiring strong statistical surrogacy. Furthermore, Imbens et al. (2024) examined sequentially structured sur- rogates and addressed challenges in identifying average treatment effects when unmeasured confounders are present. Gao et al. (2024) leveraged surrogates to improve the ef- ficiency of prediction sets for individual treatment effects, showing improved efficiency compared to existing methods (Lei & Candès, 2021; Yang et al., 2024).
1. Clustering:
By pooling groups with similar score dis- tributions, we leverage clustering algorithms deal with sub-groups with small sample sizes.
2. Surrogate-assisted efficient influence function (EIF): We develop an ElF-based approach for constructing conformalized prediction sets, effectively incorporat- ing surrogate outcomes to maximize efficiency.
Theoretically, we demonstrate that our method guarantees approximate group-conditionn coverage, provided the clus- tering algorithm pools groups accurately with minimal er- rors and the model estimation satisfics standard convergence rate conditions, which are common in flexible machine learn- ing models (Chernozhukov et al., 2018). We validate the practical effectiveness of SAGCCI through extensive syn- thetic simulations and real-data analysis of the phase 3 Moderna COVE COVID-19 vaccine, showcasing its abil- ity to improve uncertainty quantification for individualized predictions. Our proposal advances statistical theory and machine learning applications in personalized medicine, en- abling fairer and more efficient uncertainty quantification of predictions. Our code is publicly available at ht https: // gi github.cc , com /atpc/aaccccn/cccaaaaeaa/aaaaaae wttpr//////rrooooaoalll..
3. Preliminaries
3.1. Notations
Let X' C Rdx, 2 21, (M) and y' denote the space of features, protected groups (or sensitive attributes, e.g., race/gender). and the primary outcomes, respectively. De- fine X, Z, and Y as random variables taking values in X, 2, and V' respectively. Additionally, suppose we have access
2. Related work in surrogates
The concept of statistical surrogacy was introduced by Pren- tice (1989), who defined criteria for surrogate evaluation.
2

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
to surrogate outcomes S E 8 C Rds for each individual. These surrogates, such biomarkers (e.g., neutralizing tibody titers), may serve as predictors of the outcome Y.
However, exact covariate-conditional coverage is known to be generally unattainable, shown in Foygel Barber et al. (2021), Some efforts have been made to achieve approximate covariate-conditinn better scores (Romano et al., 2019; 2020) or by modifying the conformal procedure (Guan, 2023; Gibbs et al., 2023).
coverage by designing
We distinguish between two types of data: smaller source data of size DDI, where the primary outcome Y is observed, and larger target data of size rip where Y is missing. This setup motivated by real-world situations where labeled source data may be costly or require lengthy follow-up periods, while large amounts of unlabeled target data can be cheaply obtained from existing databases such electronic health records (Cheng et al., 2021). Let D E {0, }} indicate the data origin, with D = representing source data and D = 0 representing target data. In summary, we observe a source dataset {(X; Z,, St, Y, D. = 1))=al
3.3. Fairness notion in coverage
Despite the impossibility of achieving exact covariate- conditional coverage, weaker coverage guarantees can be obtained by conditioning on a set of W rather than limit- ing W specific value w. This approach, referred as group-conditional coverage, aligns closely with the concept of group faireess (Dwork et al., 2012), demographic par- ity (Kusner et al., 2017). Group faimess requires certain statistical metrics to be equalized across different levels of a protected attribute, such as race or gender (Makhlouf et al., 2021). For further discussion on related group faireess no- tions, including equalized odds and equal opportunity, see Hardt et al. (2016).
aD1 and target
dataset {(X,, Z,, St, Y; NA, D; 0) f=nnm++ where NA means "not available" (missing), and N = RD1 +npo
is the total sample size.
In our framework, we treat the baseline covariates X and the surrogate outcomes S separately, as the distributional dif- ferences in surrogates across the source and target datasets may be fully explained by the covariates (e.g.. Assumption 4.2(a)). This enables to leverage surrogates from both datasets to improve the estimation of primary outcomes, conditional on the covariates (Theorem 4.4).
Next, we claim that the group-specific prediction set C(W; satisfies group fairesss, such that
3.2. Conformalized prediction
PV~Po (y C(W; = 2)21-0 a,
(2)
Let W () W = X S be set of predic- tors, and let Y) W x y + R denote the non- conformity score function (Vovk et al., 2005). For ex- ample, common choices of R(W, Y) include the regres- sion residual R) = YY E(Y W) (Lei et al., 2018) or the conformalized quantile residual R(W.Y) = Y (W)),, where Ta/2(-) and Gi-a/2() are estimated conditional quantiles (Romano et al. 2019). Given independent and identically distributed data V, (W,, Z,, Yt) PD, drawn from the source dis- tribution PD, for = 1,.... ....... RDD1, the prediction sets can be constructed based the non-conformity scores R(W, Y) such that C(W)- = ra) = {v y) S ra}. given a threshold ra for a user-specified miscoverage level a (0, 1).
where rā is the threshold for group 2. Therefore, the predic- tion sets defined in (2) are fair across all protected groups in terms of their coverage, irrespective of group size.
Remark 3.1. a given prediction model achieves fair cov- erage, it does not necessarily imply fairness in point predic- tions, especially when the lengths of the prediction sets are heterogeneous, Therefore, fair coverage can be viewed as fairness metric for interval predictions, as it accounts for the uncertainty in point predictions.
max (W))
To achieve the desired coverage across all levels of the pro- tected variable as defined in (2), it is natural to estimate the thresholds ră separately for each protected group (Vovk, 2012), and further account for the covariate shifts across the protected groups as in Gibbs et al. (2023). However, this group-wise strategy can be overly conservative when some protected groups are small in size, leading to predic- tion sets that are excessively large and of limited practical use in finite samples. Conversely, if the threshold ra is esti- mated using the entire dataset without splitting by group, the standard conformal inference procedure cannot guarantee group-conde coup-cooditional coverage. In such cases, certain groups may experience substantial under-coverage, while others may be significantly over-covered.
The threshold ra should be chosen such that, for any data == (W, Z,Y) Y) ~ PD, drawn from the target distribution, class label Y has least a probabiliiy of being included in the prediction set C(W;ra). that is,
PV~Pos (Y E C(W:ra)) 1-a.
(1)
where the probability is taken over the marginal distribution of the source data and the future observation from the target data V. The marginal coverage guarantee (1) does not imply conditional coverage, Pv~Po., (y E C(W:rā)
W = w) 2 1 a, which is more desirable in practice.
3

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
4. Surrogate-assisted group-clustered conformal inference (SAGCCI)
who used clustering to improve outcome-dependent cov- erage in settings with categorical outcomes across many classes
To overcome the limitations of standard conformal infer- ence and group-condition condition conformal inference and to bridge fairness and efficiency, we propose SAGCCI, which leverages (i) clustering of protected groups based on non- conformity scores to overcome the issue of limited sample sizes in some groups and (ii) an Eiseb estimator for constructing the conformalized prediction sets, which uses surrogates to boost the efficiency of the prediction sets.
First, we embed the empirical distribution for the group- specific scores using a vector constituted by their quantiles evaluated at discrete levels T = {0.5.- , .0.9 U {1 - ax. Various embedding methods, such as spectral embeddings or other non-linear embeddings, can be employed, where a greater distance between any two embeddings indicates larger disparity in their score distributions.
Once these embeddings are constructed, we can efficiently implement any clustering algorithm, such as k-means, to produce the desired clusters; other cluster mappings, such as fuzzy k-means or overlapping k-means, can be applied, to obtain overlapping clusters. For a pre-specified number of clusters K, we apply the clustering algorithm to partition the embeddings accordingly. Let I = {i Z = 2, represent the indices of subjects in the protected group 2. However, for subjects with unknown protected groups, or for protected groups with extremely small sizes (e.g., |T < (1/a) 1), the empirical (1 a-)-hh quantile of the scores for these groups does not exist. To address this issue, these groups are assigned to null cluster. We define the cluster mapping from protected groups to cluster labels as ft, where h Z + (l...., K} u (null).
4.1. Assumptions
We require the following assumptions to ensure the iden- tification of the threshold for the non-conformity scores, which are an essential ingredient in the construction of our ElF-based estimator.
Assumption 4.1. There exists some constant 0 < cu < 1/2 such that co < P(D 1 | x) 1 co for any x such that f(x) > 0.
.N I,....N
Assumption 4.1 states that each individual should have a probability of at least co of being included in the target data, conditional on any covariates with a positive likeli- hood of occurrence. This overlap assumption is common in the causal inference and missing data literature (Imbens & Rubin, 2015).
If the score distributions of groups assigned to the same cluster by h are sufficiently similar, approximate group- conditional coverage guarantees can be established. Similar- ity of the scores is measured using the Kolmogorov- -miino (KS) distance, which quantifies the maximum discrepancy between the cumulative distribution functions of two ran- dom variables X and Y: KS(X, Y) = supxeR IP S
Assumption 4.2. (a) D S| | X, and (b) D DILM Y | S.
Assumption 4.2(a) states that the data origin indicator D) is conditionally independent of the surrogates S given the observed covariates X. This implies that surrogates from the combined dataset (if available) can be utilized to improve efficiency.
X) - P(Y S A))-
Furthermore, Assumption 4.2(b) states that the data origin indicator D is conditionally independent of the primary outcome Y given the observed covariates X and the surro- gates S. Since D effectively indicates whether Y is missing, Assumption 4.2(b) corresponds to the missing at random (MAR) assumption in the missing data literature (Little & Rubin, 2019). This implies that any missingness in the pri- mary outcome fully explained by the observed covariates and surrogates. As a result, covariates and surrogates from the source data can be used to infer information about the missing primary outcomes in the target data.
Lemma 4.3. Let R = (R(W, Y,) : Z, - =} be the non- conformity scores for group z. Suppose the clustering map h satisfies KS( ,R ) S e for every pair (z, 2²) such that T((2) = h((²) = k. Then, for any protected group a with T((2) = k, the following holds:
PV~Po, (y E C(W:ra) I 2)
= PPVPP (R(W. y) S ST == a
where rt is the threshold for cluster k.
Lemma 4.3 demonstrates that when the score distributions of two protected groups assigned the same cluster are sufficiently similar, measured by their KS distance, the cluster-specific thresholds r can be used to construct pre- diction sets for the associated groups with guaranteed cover- age. This approach enhances the precision of the prediction sets, as the thresholds are estimated using larger, pooled clusters, thereby benefiting from increased data size. The
4.2. Clustered conformal prediction
In this section, we introduce clustering approach for pro- tected groups based on the similarity of their conformal score distributions. By grouping smaller protected groups into larger clusters, this method improves the efficiency of prediction sets, particularly for groups with limited data. This strategy builds on similar ideas from Ding et al. (2024),
4

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
number of clusters K should be pre-specified, and we pro- vide simple heuristic in the Appendix for selecting this parameter in practice.
Corollary 4.5 quantifies the efficiency gain achieved by incorporating surrogates in terms of the semiparametric effi- ciency lower bound for estimating the threshold ra. Specif- ically, this efficiency gain depends on how predictive the surrogates are for the primary outcomes. The predictive- ness is captured by the variance var ( W) X) = var P(R S vo | (W) | X). which reflects the additional variability in predicting the scores R that is explained by the surrogates S beyond the covariates X. A higher variance in- dicates that the surrogates contribute substantial information about the primary outcomes, resulting in greater efficiency gains in estimating Ta. Moreover, the efficiency gain is influenced by the proportion of source data, as determined by the propensity scores ex()). When en(X) X) is large, the proportion of the labeled source data is high, which intuitively means there is less missingness in the primary outcomes to address. As result, the potential efficiency gain from leveraging surrogates is reduced in such cases.
4.3. Efficient influence function
We now derive the EIF for To (or ra for the k-tth cluster), which is functional derivative that characterizes how sen- sitive the estimand is to changes in the data generation distributions PD, and PD. The EIF, also known as the canonical gradient (Van der Laan et al., 2011). is a funda- mental tool for achieving statistical efficiency. Leveraging the EIF, we construct estimators for ra that incorporate sur- rogate outcomes, enabling the estimator achieve local semiparametric efficiency.
Theorem 4.4. Under Assumptions 4.1 and 4.2, the EIF v r without surrogates and the EIF v. (V:ra) with surrogates for aa is, up to proportionality constant,
4. (V:Vee, ra) = (1 - (m)(raa( (1-a)) + DTD((X)1( S ra) X)),
4.4. Implementation
To construct the semiparametric efficient estimator Ph for cluster h 1, K, we adopt the split conformal in- ference strategy (Lei & Wasserman, 2014). This method involves randomly spliting the combined data into two folds, I1 and Zz. The first fold, I1, is used to train the learning algorithms (e.g., the nuisance functions cD, m, and m)) The second fold, I2 is used to construct prediction intervals based on the non-conformity scores.
and
No (V; = (ViED,
+ [XTn(X) - (1- en (x) (m((m)) X) m(rr. W)}.
where eD(X) P(D =1 X) is the propen- sity score of being the source data, TD(X) {1- (D(X))/eD(X) is the inverse odds of observing pri- mary outcomes, vn(ra,, ()) the conditional cumulative distribution function (CDF) of the non-conformity scores incorporating surrogates, and m((a..) = P(R S Ta | X) is the conditional CDF with- out surrogates, both evaluated at Ta-
P(R S ra W) is
For concreteness, we describe the split conformal inference for Y using conformalized quantile residuals (CQR) on the target data. Let (-)) and --))/((-) denote the quantile models for the primary outcomes Y. trained on the first fold L. Additionally, the propensity score ēD is estimated using Zi To mitigate the computational complexity of estimating the full conditional distributions m() and mr,r W) for infinitely many r, we adopt the localized debiased machine learning approach (Kallus et al., 2024). Under this approach, the first fold, I, is further split into two sub-folds, In and Z12. In the first sub-fold, I11, we construct an initial estima- tor pinit for ra- A natural choice is the weighted split-CQR estimator from Lei & Candès (2021). In the second sub- fold, I12, the nuisance models m X) and m (Fanit, W) are estimated using any machine learning binary algorithm given the single initial estimator pinit
We refer to ep. m, and m as nuisance functions since they are not of our interest but are unknown and required to evaluate the EIFs for Ta- Theorem 4.4 establishes that the surrogate- assiistd EIF for ra extends the EIF without sur- rogates, (V; eD, m, T.), by incorporating an additional augmentation term. This augmentation term has a condi- tional expectation of zero, ensuring that it introduces no bias. The purpose to leverage the residual information provided by the surrogates S in both datasets, thereby boost- ing the estimation efficiency. The efficiency gain achieved through the inclusion of surrogates is formally quantified in Corollary 4.5.
Next, we compute the non-conformity scores R for the second fold, Iz, using the estimated quantile mod- els 9./2(-) and @i-a/2(-). The scores are defined as R(W,, Y:) max (W:)) Y,, Y, /2 (W:)) From these scores, the score distributions for each protected group are embedded using their group-specific quantiles. We then apply h-means clustering to map the M protected groups into K clusters, learning the cluster mapping h based
Corollary 4.5. Under Assumptions 4.1 and 4.2, the effi- ciency gain of leveraging the surrogates is given by
V
Vest
F [X) -xx)))(lle(() W) [x]],
where V = varlet Van = var(va((:raa),
5

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
on the score embeddings. Finally, within Iz, we identify the cluster-specific efficient estimators P for the threshold. Let ((k) = {i : ((.)) = k i E ]2} represent the indices of sub- jects who are assigned to cluster & by the mapping h. These estimators are defined as the smallest values satisfying the condition: ΣiEInnI Va ED; m. m, Ft) 2 O, where the nuisance models m, m. and ēD are estimated from I1. Full implementation details are provided in Algorithm 1.
O(N-1/2), results from bounding the empirical mean of (V; m, m, fa) in finite samples. This error decreases as the sample size increases, given a fixed number of clusters and similar sizes for the data folds I and In used in the split inference strategy.
3. Product bias: The third term reflects the bias in- duced by estimation errors for the nuisance func- tions. The bias is negligible if either [[ēb(X) en(X)] - sup, [[ X) m((r, X)Il = o(1) and [x² - ep (x)l sup, || [ W) (n(r W)l = (11) This property, known as rate double robustness (Chernozhukov et al., 2018), implies that small pertur- bations in the nuisance functions affect the coverage error only in second-order terms.
4.5. Theoretical properties
Next, we establish the theoretical properties of the estima- tor F& for cluster k. To do so, we first list the regularity conditions for the nuisance functions:
(AI) The estimated nuisance functions (X). mirr X) and m(r, W) are bounded. Specifically, there exist constants Tu. my and my such that [xb(X)] s To. [m) S mo. and [m(r, W)I S mo-
As a result, the prediction sets C(W:rā) constructed us- ing SAGCCI approximately achieve the desired group- conditional coverage level, up a negligible term that vanishes with high probability, that is, Pv~Po, (y C(Wrr)) Z 2) erty is commonly referred to as Probably Approximately Correct (PAC) coverage in the statistical literature (Krish-
(A2) The estimators m(r, X) and f(r; W) are non- decreasing with respect to r.
= 1 a (1) as N + co. This prop-
Using the split conformal inference strategy outlined in Sec- tion 4.4, the desired asymptotic group-conditional coverage of the prediction sets is guaranteed in Theorem 4.6.
namoorthy & Mathew, 2009).
5. Simulation studies
Theorem 4.6. Under Assumptions 4.1 and 4.2, along with regularity conditions (AI) and (A2), there exist constants Co and Cn such that, for any group satisfying T((z) = k; the following holds with probability at least 1 - 8 with 8 > 0:
We evaluate the performance of our proposed method through a series of numerical experiments. Empirical re- sults show that SAGCCI effectively reduces disparities in grodp-conditional coverage while significantly shrinking prediction set sizes relative to existing methods.
PV~Poo (y E :(W;r 2 = z)
= -Pv-PDo (R(W,)) y) S & 7 = 2) 21-a-
5.1. Experimental setup
log(1/6) +1
-Co(TO +mo+ + )
To simulate challenging scenario, we assume that the source data (with observed primary outcomes) is signifi- cantly smaller than the target data. Specifically, we ran- domly select a small proportion, P(D = 1) = N-1/10 of the total population as the source data (D = 1), with the re- maining data constituting the target data (D = 0). The total sample sizes considered are N 1000, 3000, 5000, 10000. We first generate the baseline covariates X R from
[Zg n NZ()
-ci (o(118(x) - (((((X))ee(A))))
exx()]l suup [r, X)- - m(,
- (oo(x))e((A))) sup |Í,r, W) m(r, (
Theorem 4.6 ensures a group-conditional coverage guaran- tee of approximately 1 a a for the cluster-specific prediction sets of Y from the target population. The slack in the cover- age guarantee is the sum of three components:
multivariate normal distribution, X ~ (0, I2), for the entire population.
Next, we generate the protected group variable Z with M = 3 levels from a multinomial distribution. Specif- ically, P(Z
1. Score distribution discrepancy: The first term, e, arises due to differences in the score distributions of pro- tected groups within the same cluster, which usually diminishes to zero for large sample sizes (c.g., the KS h-mma clustering (Zhu et al., 2021)).
| X) x exp -az,1 -, X,/2). P(Z 2 x x exp az2 X1 X2/ and P(Z 3 X) x exp (-az,3 X1/2 - X2). where M = 3 and az = (az,1, az,,, az,3) is adaptively chosen to ensure the group proportions are approximately 0.5, 0.3, and 0.2, re- spectively; additional experiments with larger number of groups (e.g., M 10, 20) are presented in Appendix B.
2. Finite sample error: The second term, obtained from empirical process theory, is proportional to
6

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
Algorithm I Surrogate-Assisted Group Clustered Conformal Inference (SAGCCI)
Input:
Source data D, [(W, Zi, Y, D; 1) target data Do {(W, Zi, NA, D; = =]] iEnDI++ and miscoverage level E (0,1)
Preparation
Randomly split the data D2 U Du into training and calibration folds Z and Iz.
Training
Split 11 into sub-folds Z11 and Z12-
Fit score function R(W.Y) on I 0 11
< e.g. quantile random forest e.g. SuperLearner for binary outcomes
Fit propensity function ( on Zz
Obtain the initial threshold Fluit on In n nDD, via the WCQR Algorithm from Lei & Candès (2021). Fit conditional CDF (Finii W) and m( X) on I12 n21 < eegg SuperLearner for continuous outcomes
Calibration
Compute R2 R(W, Yi) for i E I2 D1.
Determine clustering number K by assessing R2 across Z,.
4 e.g., Elbow method or other herustics
Obtain the clustering mapping Tí(Z) on I2 nD),
e.g., k-means
Compute the clustering indices ((k] = {i T((z)) = k, e for k 1, I.... K. Compute the cluster-specific threshold Fà as the smallest values satisfying Σieianzus tha (V; m, m, r 2 0.
4 Prediction
For a new data point 2W, PD, from the target population, compute C(W, where k = h(2). Output: prediction set C(W,
The surrogate outcomes S E R² are modeled as S (12,, 12/22, where d3- = 1. 3.5. The variance o² reflects the extent to which the surrogates S explain the variability in the primary outcome Y, after adjusting for the baseline covariates X. larger n indicates higher predictiveness of the surrogates S for the primary outcomes Y.
as in Gibbs et al. (2023); 3) surrogate-assisted standard con- formal inference without accounting for protected groups (SURRO + STANDARD); 4) weighted conformal quantile regression (WCQR) from Lei & Candès (2021); and 5) con- formalized fair quantile regression (CFQR) from Liu et al. (2022).
Finally, the primary outcomes Y generated multinomial variable with five levels. Specifically, for = 5, the conditional probabilities are given -3 ) where y 1 serves the reference level, and ay adap- tively chosen to ensure marginal probabilities of P(Y k) = (0.1, 0.2, 0.4, 0.15, 0.15) for y = mory=l...... 1.....5
5.2. Evaluation metrics
Denote the second data fold Ie as the validation dataset: ((W, Zg, Y)) The empirical group-conditions cov- erage for group is defined as: 2 = Eiezonz, 1(Y; C(W)))/IZ= n I2l. where I O represents the subset of validation data points belonging to group z, and ITa n Iz! is the number of such points.
by: P(y=][X,s] exp -i Y
The non-conformity scores for the categorical outcomes are derived using the nested prediction sets approach from Kuchibhotla & Berk (2023); more details are presented in the Appendix. Following Sesia & Candès (2020), we split 75% of the data into the first fold I₁ for model training, and use the remaining 25% as the second fold Iz construct pre- diction sets. We compare our proposed method (SAGCCI) with K = 2 clusters against four alternative approaches: 1) clustered conformal inference without leveraging surrogates (NOSURRO + CLUSTER): 2) surrogate-assisted grse conformal inference (SURRO GROUP), which accounts for the covariate shifts to obtain the group-specific thresholds
To evaluate the quality of the prediction sets while incor- porating considerations of group fairness, we use two pri- mary metrics: 1) Average Size (AvgSize) of the prediction sets, defined as AvgSize IC(W.) the size of the prediction set for individual i; and 2) Average Group Coverage Gap (CovGap). defined as CovGap = Σ.ez C.2 (1 a)M, where M the total number of groups
Erez, IC(W))l/Izl, where
|Z| is
The AvgSize metric assesses the precision of the prediction sets, where smaller values indicate more precise intervals. Meanwhile, the CovGap metrie evaluates how closely the
7

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
group-conditional coverage aligns with the desired level 1 a. A smaller CovGap reflects better adherence to fair- ness goals across groups, Additionally, we evaluate the fraction of under-covered and over-covered groups in each experiment and provide the results in Appendix B; similar conclusions can be drawn from these results.
the smallest AvgSize across all settings by utilizing the en- tire dataset without accounting for protected groups. How- ever, expected, its CovGap does not decrease to zero, as it lacks guarantees for group- coopp-onniiional coverage. In contrast, the SURRO + GROUP method fairly ensures the desired group-conditional coverage but with a significantly larger AvgSize, indicating less precise prediction sets. The CFQR method focuses on learning a fair quantile function to construct the non-conformity score used to produce fair prediction sets. However, this approach may not generalize well to scores for categorical outcomes, as evidenced by its unsatisfactory performance under our problem setup. In contrast, our proposed SAGCCI method takes a balanced approach by leveraging the similarity in score distributions across protected groups to form larger clusters and esti- mate cluster-specific thresholds, which is more general and can be applied to any type of non-conformity scores. This strategy maintains high-quality and precise prediction sets comparable to SURRO + STANDARD in terms of AvgSize, while effectively addressing disparities in group- croup-conditional coverage for protected groups with a diminishing CovGap, similar to SURRO GROUP.
5.3. Results
First, we evaluate the benefits of leveraging surrogates to construct prediction sets. Figure summarizes the perfor- mance metries for the proposed clustered conformal infer- ence methods, both with and without surrogates, across 500 Monte Carlo simulations for Øg 1,3,5, 5, representing low, medium, and high surrogate effects, respectively. The WCQR method with surrogates produces more precise pre- diction sets compared to NOSURRO + CLUSTER when sur- rogates are highly predictive of the primary outcomes (i.e., as as increases). However, it fails to ensure valid group- conditional coverage, indicated by its non-decreasing CovGap with increasing sample size. In contrast, SAGCCI consistently achieves the smallest average prediction set sizes while maintaining valid group-conditional coverage, evidenced by its empirically decreasing AvgSize and Cov- Gap. Its efficiency gain over NOSURRO + CLUSTER be- comes more pronounced as og increases. With larger as, surrogates explain greater proportion of the variability in the primary outcomes, increasing their predictiveness and improving the construction of prediction sets. The results align with the theoretical results presented in Corollary 4.5, where surrogates are leveraged in the most efficient way by the EIF to estimate the thresholds.
For example, when σş 3, the proposed SAGCCI reduces AvgSize by over 60% average compared to SURRO + GROUP at N = 1000, while increasing the CovGap by only 7%. Compared to SURRO + STANDARD, the AvgSizes are similar, but SAGCCI noticeably reduces the CovGap by over 80% at N = 3000. Even when surrogate outcomes are not available (i.e., σg = 0), cluster mapping remains benefi- cial, seen by comparing the performance of NOSURRO + CLUSTER against NOSURRO STANDARD and NOSURRO GROUP in Appendix B.
SAGCCI NoS unro Cluster
WCQR
surrogate
medium surropate effects
high surrogate effects
SAGCCI Surr Group
Suno Standdrd
CFOR
Σ 1 K
low surmogate effects
medium surmggtt effects
high surrogate effects
3.0
4.0
2.5
3.5
3.0
3-
CoySa8
2.5
7.5
T
5.0
1000 3000 5000 10000
1000 3000
5000 10000
1000 3000 5000
10000
Sample size
2.5
0.0
1000
3000
5000
10000
1000 3000
5000 10000
1000 3000 5000 10000
Sample size
Figure 1. Comparison of AvgSize and CovGap for the considered methods. The error bar plot denote - the standard errors.
Figure 2. Comparison of AvgSize and CovGap for the considered methods. The error bar plot denote + the standard errors.
Next, we investigate the impact of cluster mapping on em- pirical group-conditional coverages, with results presented in Figure 2. The SURRO STANDARD method achieves
8

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
6. Real-data application
substantially improves efficiency over existing group-wise conformal methods by producing tighter prediction sets while maintaining valid group-conditional coverage, mak- ing it well-suited for high-stakes applications such as clin- ical risk assessment and policy evaluation. A promising direction for future research is to extend the generalizability and transportability of these conformal inference methods across multiple data sources, such as multi-site randomized clinical trials or observational studies. This could involve relaxing the MAR assumption and addressing potential dis- tributional shifts, including conditional shifts in outcomes and surrogates (Liu et al., 2024). Such extensions would be particularly valuable in settings where privacy constraints preclude the sharing of individual-level data, a common challenge in federated causal inference (Han et al., 2023; 2024; 2025; Guo et al., 2025; Xiong al., 2023).
We analyzed data from the Moderna COVE phase 3 COVID- 19 vaccine efficacy trial, which randomized adults to receive two doses of mRNA-1273 or placebo Days and 29. Our per-protocol analysis included participants who received both injections without specified protocol violations. Par- ticipants were randomly sampled into the immunogenicity subcohort using stratified Bernoulli random sampling, with antibody markers measured at Days 1,, and 57. Our analy- sis focused on cohort of 1418 individuals, including 46.0% underrepresented minorities (n- 652), defined as Blacks or African Americans, Hispanies Latinos, American Indians or Alaska Natives, Native Hawaiian, and Pacific Islanders. The source population of non- non-minorities (n -666) included all other races (e.g.. White, Asian) and non-Hispanic ethnic- ity. Confounders included a standardized COVID-19 risk score built using machine learning and baseline covariates (e.g., sex, BMI, enrollment time), with faimess evaluations conducted across age-comoriidity stratification groups: (i) 65 or older, (ii) younger than 65 with comorbidities, and (iii) younger than 65 without comorbidities. The surrogate outcome was the Day 29 10 nAb-ID50 titer, while the pri- mary outcome was a discrete variable representing quintiles of the binding spike antibody level at Day 57.
Software and Data
Our R code, complete with illustrative examples, is available at
com/Gaochenyin/
SurrConformalDe 1DR.
Acknowledgements
We set the miscoverage level a 0.05 and employed a 75- 25 train-test split, repeating the procedure 100 times to sum- marize results. Table 1 presents the results. Methods lever- aging surrogate outcomes (SURRO+ + STANDARD, SURRO + GROUP, and SAGCCI) consistently outperformed those that did not, producing lower coverage gaps and shorter pre- diction interval sizes. Among these, our proposed SAGCCI achieves the best overall performance, with the shortest pre- diction intervals across all three age-comoryidity subgroups, and valid group-conditional coverage. This highlights the ef- fectiveness of combining surrogate information with cluster- ing to achieve both precision and fairness in individualized vaccine efficacy estimation.
We thank the anonymous (eevaa-seviewwers of ICML 2025 for their helpful comments. We thank the COVE study par- ticipants and study team, other COVPN colleagues, and Moderna colleagues. This research was supported in part by the Administration for Strategic Preparedness and Response, Biomedical Advanced Research and Development Author- ity Contracts No. 75A50120C00034 (P3001 study) and No. A50 22C00013 (Immune Assays) and the National In- stitute of Allergy and Infectious Diseases (NIAID) of the National Institutes of Health through grant
The findings and conclusions herein are those of the authors and do not necessarily represent the views of the Depart- ment of Health and Human Services or its components. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Department of Health and Human Services.
Table Comparison of methods: AvgSize, CovGap, empirical coverage, and width for age-comorbidity stratification groups
Method
AvgSize
CorGap
Coverage
Width
NOSURRO- STANDARD
1.16
1.39
(0.97, 0.87, 0.92)
(..79,4.200,4 4.39)
SURRO + STANDARD
3,83
1.09
(0.95, 8.93, 0.96)
(3882, 3.76)
Impact Statement
NOSUBRO + GROUP
4.63
2:22
(0.97, 0.90, 0.95)
(4811 4,64)
NOSURRO- + CLUSTER
SUXBO + GROUP
4.48
0.14
3.28
(0.94, 0.87. 0.94)
0.95)
(3.80, 4.06. 3.94)
(4.62, 4,27, 4,53)
The proposed approach improves the efficiency of uncer- tainty quantification by leveraging surrogates, thereby re- ducing implementation costs in practice. Furthermore, it incorporates fairness to prevent models from discriminating against certain protected groups, aligning predictions with ethical and regulatory standards. Therefore, this approach builds trust and accountability in individualized decision- making across critical fields, ensuring compliance with so- cictal and legal regulations.
WOOR
5:24
(0.98,0.82,0058
3,52,
SAGCCI (ours)
3.61
0.18
(0.95, 1.95, 0.95)
(363, 3.55, 3.66)
7. Diseussion
We have introduced a novel surrogate- assiitte group- clustered conformal inference (SAGCCI) framework for constructing fair and efficient prediction sets. SAGCCI

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
References
Fleming. T. R. and DeMets, D. L. Surrogate end points in clinical trials: are we being misled? Annals of Internal Medicine, 125:605-613, 1996.
Alonso, A., Van der Elst, W., Molenberghs, G., Buyse, M., and Burzykowski, T. An iitoommationthhoorettee approach for the evaluation of surrogate endpoints based on causal inference. Biometrics, 677,, 2016.
Foygel Barber, R., Candes, E. J., Ramdas, A., and Tibshirani, J. The limits of distribution free conditional predictive inference. Information and Inference: A Journal of the IMA, 10(2):455 482, 2021.
Athey, S., Chetty, R., Imbens, G. W., and Kang, H. The sur- rogate index: Combining short-term proxies to estimate long-term treatment effects more rapidly and precisely. Technical report, National Bureau of Economic Research, 2019.
Gao, C., Gilbert, B., and Han, L. On the role of surrogates in conformal inference of individual causal effects. arXiv preprint arXiv:2412.1 12365, 2024.
Athey, S., Chetty, R., and Imbens, G. Combining experimen- tal and observational data to estimate treatment effects long term outcomes. arXiv Preprint arXiv:2006.0940 09676, 2020.
Gibbs, I., Cherian, J. J., and Candès, E. J. Conformal prediction with conditional guarantees. arXiv preprint 2tti:23005/26//6 2023.
Gilbert, P. B. and Hudgens, M. G. Evaluating candidate principal surrogate endpoints. Biometrics, 64: 18:1146-111 1154, 2008.
Chen, J. and Ritzwoller, D. M. Semiparametric estimation of long-term treatment effects. Journal of Econometrics, 237:105545, 2023.
Gilbert, P. B., Montefiori, D. C., McDermott, A. B., Fong, Y., Benkeser, D., Deng, W., Zhou, H., Houchens, C. R., Martins, K., Jayashankar, L., et al. Immune correlates analysis of the mrna- 1273 covid-19 vaccine efficacy clin- ical trial. Science, 375:43-50, 2022.
Cheng, D., Ananthakrishnan, A. N., and Cai, T. Robust and efficient semi semi-supervised estimation of average treatment effects with application to electronic health records data.
Biometrics, 77:413-425 2021.
Cherozhukkvv, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. Double/debiased machine learning for treatment and structural parameters, 2018.
Gilbert, P. B., Peng, J., Han, L., Lange, T., Lu, Y., Nic, L., Shih, M.-C., Waddy, S. P., Wiley, K., Yann, M., al. A surrogate endpoint based provisional approval causal roadmap. arXiv Preprint arXi::2407.06330, 2024.
Conlon, A., Taylor, J., Li, Y., Diaz Ordaz, K., and Elliott, M. Links between causal effects and causal association for surrogacy evaluation in a gaussian setting. Statistics in Medicine, 36:4243 4265, 2017.
Guan, L. Localized conformal prediction: A generalized in- ference framework for conformal prediction. Biometrika, 110(1):33 50, 2023.
Guo, Z., Li, X., Han, L., and Cai, T. Robust inference for federated meta-learning Journal of the American Statistical Association, pp. 1-16, 2025.
Ding, T., Angelopoulos, A., Bates, S., Jordan, M., and Tib- shirani, R. Class-conditionall conformal prediction with many classes, Advances in Neural Information Process- ing Systems, 36, 2024.
Han, I.., Shen, Z., and Zubizarreta, J. Multiply robust federated estimation of targeted average treatment effects. Advances in Neural Information Processing Systems, 36: 70453-70483 2023.
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. Fairness through awareness. Proceedings of the 3rd innovations in theoretical computer science conference, pp. 214 226, 2012.
Han, L., Li, Y., Niknam, B., and Zubizarreta, J. R. Privacy- preserving, caaamunication-effiiicii and target-flexible hospital quality measurement. The Annals of Applied Statistics, 18(2): 1337-13337335, 2024.
Elliott, M. R. Surrogate endpoints in clinical trials. Annual Review of Statistics and its Application, 10:75-96, 2023.
FDA. Accelerated approval of new drugs for serious or life- threatening illnesses. Federal Register, 57:58942-58960, 1992.
Han, L., Hou, J., Cho, K., Duan, R., and Cai, T. Federated adaptive causal estimation (face) of target treatment ef- fects. Journal of the American Statistical Association, (just-accepted):1 2025.
FDA. Accelerated approval program. https://www fda. haddlcoo//rrugg/ddd--a-----a---p accelerateed 2021. Ac- cessed: 2024-10-31.
Hardt, M., Price, E., and Srebro, N. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
10

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clusteres Approach
Imbens, G., Kallus, N., Mao, X., and Wang. Y. Long-term causal inference under persistent confounding via data combination. Journal of the Royal Statistical Society Series B: Statistical Methodology, pp. qkac095, 2024.
Liu, M., Ding, L., Yu, D., Liu, W., Kong. L.,, and Jiang, B. Conformalized faimesss via quantile regression. Advances in Neural Information Processing Systems, 35:1 1561- 11572, 2022.
Imbens, G. W. and Rubin, D. B. Causal inference in statis- tics, social, and biomedical sciences. Cambridge Univer- sity Press, 2015.
Liu, Y., Levis, A., Normand, S.-L., and Han, L. Multi- source conformal inference under distribution shift. In Proceedings of the 41st International Conference on Ma- chine Learning, volume 235, pp. 31344-31 31382. PMLR, 2024.
Kallus, N. and Mao, X. On the role of surrogates in the efficient estimation of treatment effects with limited out- come data. Journal of the Royal Statistical Society Series B: Statistical Methodology, pp. qkae099, 2024.
Makhlouf, K., Zhioua, S., and Palamidessi, C. Machine learning fairness notions: Bridging the gap with real- world applications. Information Processing & Manage- ment, 58(5):02244, 2021.
Kallus, N., Mao, X., and Uehara, M. Localized debiased machine learning: Efficient inference on quantile treat- ment effects and beyond. Journal of Machine Learning Research, 25:1-54 19111-9,000 2024.
Prentice, R. L. Surrogate endpoints in clinical trials: defi- nition and operational criteria. Statistics in Medicine, 8: 431 440, 1989.
Krishnamoorthy, K. and Mathew, T. Statistical tolerance regions: theory, applications, and computation. John Wiley & Sons, 2009.
Romano, Y., Patterson, E., and Candes, E. Conformalized quantile regression. Advances in Neural Information Processing Systems, 32, 2019.
Kuchibhotla, A. K. and Berk, R. A. Nested conformal prediction sets for classification with applications to pro- bation data. The Annals of Applied Statistics, 17:761-785, 2023.
Romano, Y., Sesia, M., and Candes, E. Classification with valid and adaptive coverage. Advances in Neural Infor- mation Processing Systems, 33:3581-3505 2020.
Sesia, M. and Candès, E. J. A comparison of some confor- mal quantile regression methods. Stat, 2,220,,2020. 2020.
Kusner, M. J., Loftus, J., Russell, C., and Silva, R. Counter- factual fairness. Advances in neural information process- ing systems, 30, 2017.
Van der Laan, M. J., Polley, E. C., and Hubbard, A. E. Super learner. Statistical applications in genetics and molecular biology, 6(1), 2007.
Lei, J. and Wasserman,
L.. Distribution-fre prediction bands for non-parametric regression. Journal of the Royal Statis- fical Society Series B: Statistical Methodology, 76:71 1199,, 2014.
Van der Laan, M. J., Rose, S., et al. Targeted learning: causal inference for observational and experimental data, volume 4. Springer, 2011.
Lei, J., Robins, J., and Wasserman, L. Distribution- prediction sets. Journal of the American Statistical Asso- ciation, 108(501):278- 28778828772211 2013.
Van der Vaart, A. W. Asymptotic Statistics, volume 3. Cam- bridge University Press, 2000.
Vovk, V. Conditional validity of inductive conformal pre- dictors. In Asian conference on machine learning, pp. 490- PMLR, 2012.
Lei, J., G'Sell, M., Rinaldo, A., Tibshirani, R. J., and
Wasserman, L. Distribution predictive inference for regression. Journal of the American Statistical Associa- tion, 113:1094-111,, 2018.
Vovk, V., Gammerman, A., and Shafer, G. Algorithmic learning in a random world, volume 29. Springer, 2005.
Lei, I.. and Candès, Conformal inference of counter- factuals and individual treatment effects. Journal of the Royal Statistical Society Series B: Statistical Methodol- ogy, 83:911 13-88, 2021.
Vovk, V., Nouretdinov, I., and Gammerman, A. On-line predictive linear regression. The Annals of Statistics, pp. 1566-1550 1590, 2009.
Xia, E. and Wainwright, M. J. Prediction aided by surrogate training. arXiv preprint arXiv:2412.09364 2024.
Li, Y. Taylor, J. M., and Elliott, M. R. A bayesian approach to surrogacy assessment using principal stratification in clinical trials. Biometrics, 66:523-531, 0311 2010.
Xiong, R., Koenecke, A., Powell, M., Shen, Z., Vogelstein, J. T., and Athey, S. Federated causal inference in hetero- gencous observational data. Statistics in Medicine, 42 (24):44884439, 2023.
Little, R. J. and Rubin, D. B. Statistical analysis with missing data, volume 793. John Wiley & Sons, 2019.

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Clustered Approach
Yang, Y., Kuchibhotla, A. K., and Tchetgen Tchetgen, E. Doubly robust calibration of prediction sets under covari- ate shift. Journal of the Royal Statistical Society Series B: Statistical Methodology, pp. qkae009, 2024.
Zhu, Y., Deng, Q. Huang, D., Jing, B., and Zhang, B. Clustering based on soimogorov-smirnoy statistic with application to bank card transaction data. Journal of the Royal Statistical Society Series C: Applied Statistics, 70 (3):558-578, 2021.
12

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
A. Proofs
A.1. Proof of Theorem 4.4
In this section, we derive the EIF for the threshold Ta- Since the threshold can be group-(or cluster-)specific, the protected group Z is excluded from the observed data in the derivation. Let denote the index for the parametric submodels fc(O) of the observed data (X, D, with true density function evaluated i i.e. f(O) Jt- (O), where
f(O) = f(x) x (L)()( x S(S) D, J(Y D,),
the observed score function is derived by the pathwise derivatives of f(() with respect to t, given by s,(O) sY D,D,) + si(S D) + si(D | X) + s((X). The tangent space corresponding to this model A x @b DiI @ AS D.X @ Ayjs.D.X, where Ax, ADJX. A.s.. and Dy are the mean square closures of the score vector of the submodels:
xx= (T()) / (( =)) 0).
ASID,-= = D, / [T(S, ((s, D, D, 0),
AYsS.D.X (T(Y,,, : fI... D, 0},
for any two arbitrary square-integrable measurable functions a(X) and 6(X).
Let u(R, ra) 1(R S Fa D 0) (1 a)), and r(t) be the corresponding (1 quaantil for the scores of the target population R D under the parametric submodels, which satisfies E, [u R2, m (f) = O with r(t") Ta- Next, we can show that
i (alu(Rrr(())))) 0 Ōt
8 _ B(( Or
2 - Er. [rr
0r(1)
Øt -=
which leads
Ör(()
0
[1 Øt
i Fu(u(R,
Fr. (u(
(alu(R
t=t-
Ör
at
Under Assumptions 4.1 and 4.2, we have
0 E Ed - PR ra D= ===0 0)
0 Ex.(LE,,(Br;:d <ra,1 D= 1,,, XX 0]k
01
8
Fx.1((1 Ds (P. <r. D 1,5, x) XH]e
0t
0
Ex, 0 (P, <r. D= L,S, XHI|
0t
= 7 + + T4,
which is constituted by five terms Ti, T2, T3, and T₁. We analyze these four terms separately,
Ti Ex, - X)m(r, x)Hl (x X) x) - (1 D) -))(1-))ss(X),,
Ot Ô
T2 Ex 0 | x)m(r - X)],
at &
T3 Ex[P(D= 0| (m() W) | B(P(D= (m(r, W) m(r, X)) s(S D,X)),
0t
13

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Croup-Clustered Approach
and
Fx P = 01 (),s < D | 1D= XHIl-.-
0t
s(Y S. x x)) E (DTD(() (X) {1(B S ra) (0,, s(Y S,D,
(P(D= - 0 X)
D S ra) mm(r
Ex
Ot
eD(X)
Putting these terms together, we can show that
Ōr(()
① Felu(R, a. DXX))
Øt t--- ot
and
va(V; m, ra) = = (m X) (1 - a}}
+ {1 - (m)))
W) - - x){
DTT(()(1 < ra) mi(ra. W)}
(1 (m))mr X) (1- + DI r.) m((o, X)}
(X) - (1-=
mi(ra. W)},
where (1 D)m(ra, X) (1- a)} Ax, {1- e W) m(ra. Aspx, DTD(X){ Ta) -m(ra, W)} Avjs, Thus, it completes the proof of Theorem 4.4 by the definition of efficient influence function.
A.2. Proof of Theorem 4.6
In this section, we establish asymptotic properties for the estimator of Tax which is important for constructing prediction set with the user-defined coverage level 1 First, we establish the connection between the asymptotic coverage probability P(R < r. D = 0) and the EIF for in Lemma A.I.
Under Lemma A.I, we can show that for the empirically estimated Pa using the second data fold I2, we have
P D= 00 - (1-a) 12,, (VV:rr()
P(D P(D=0) =0)
P(D(V:ra)) - Pz, (V((: P(ū(V;ra) (ra.11 W)} P(D 0)
0 T1 Iz.
P(D 0)
Here, use the fact that P2,, ( (v(;)r)) > 0 by definition. The term I is negligible if (VV;ra) belongs to Donsker class (Van der Vaart, 2000). Even if the Donsker condition is not met, the sample-splitinn procedure in (Chernozhukov et al., 2018) can be used to assure that / is negligible, where the first data fold Z₁ is used to estimate ēp. mn and m, and the second data fold Ig is used to compute ra: see Lemma A.2 for more details on the bound of IT.
The term T2 the second-order remainder term, which is bounded by the product of estimation error for the nuisance functions:
- -H
m(rr
+P [1 [1-))(()))m(r,)) W) x)r + (X) W) mrr w)}]
= =P((1ee((x)]0000 m(rrX)
+P (1 w) m(r, + - m(r,
+F (xx(X))x(x))(((())))) W))
=P((E((X) -
m [|
+P (1
Ex(X) (x)(())(((()))))))))))) W) - mm wo}]
x ((x((X) -x -
sup m
m(r, x)1l (x)(X) - (x) sup [m, w) -m(r, W)ll,
14

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
where the last inequality follows from the Cauchy-Schward inequality. Combining the bounds for and 12 gives us
PRR Fa = 0) 2 (1 ax)
log( (0g(1/6) +
Corooso + ma co ma) (
I2
1oo ex(x)] -e)(x) sup [[ X) sup mm(r, + ([eb(X) - =x(X)]l sup |n(r w) m(r, (o],
which completes the proof.
A.3. Proof of Corollary 4.5
By straightforward algebra, the EIF for Ta without surrogates is & (V;rra) = (1 - ()m(ra; - (1 - a)} +
DTp (x) (I S ra) - m(ra: X)}. The variance for estimating Ta without surrogates is Vas = (V;r))
V + V VJ + 2 2V1s 2V23, where
VT" = var [[1 - ( - -1 (1- a)}],
V2 var (DTD(X)I(R S ra)l, Vi = var (Dzp(X)m(raxx)); X))
VT2 cov [(1 -1 (t
V cov [(1 - D))mmaa (1 =0
V23 = cov
< DTD
X)] V/.
Similarly, can show that the variance for estimating To with surrogates is Vat var
2V + 213- 2V14 + 2/23 - 21 2V3 where
Vi = var [(1 (m((a. X) a)))
1/2 = var [[1 - ([)-)e(X)))(tz W) m(ra.c,))H
V/ = var (DxD(X)I(R & rra), Va2 = var (DTD(X)m(rr,W)))
V12 = Via = 11==144- 00
1/2 V24
cov [[1 -eD(X)) W) - (X))) W)], 1/4 cov (D ra), D W)] V4.
Therefore, verify that
Voit
Vor V4 V V2
= (DTD(X))((a W)} var (DTD(X)mo(tactt)) var (xx]eee(X))mitpe W) me(ra,
<E)D (rai w)} E (D E - (x( W) -mc(r::))3 (L-e)(X)} (m (ra. W) - m X)} - [1_ W) (ra: X, x)].
ex(
= E [x)( -
where V = Vi and V = V3. Hence, it completes the proof of Corollary 4.5.
A.4. Additional technical details
A.4.1. PROOF OF LEMMA A.1
Lemma A.1. Under Assumptions 4.1 and 4.2, the following holds for any EIF (V;rra):
E(e(V:ra) P(D P)
P(R < Ta D= 0) 11
15

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clusterie Clustered Approach
On the hand, can show that
D = 0) = ( ra) D = D= P ra
(3)
On the other hand, we can prove that
((M(V)r)) = F Pm (1--()1)))))) -
EE((DD = 5x PDD=0)1--0)
=E((1
P(D ra
0) aa)
(r())))) | P(D 0)(1 a.).
D 0) = =1 =1 = 0). Combine it with (3), have P(R S ra
A.4.2. PROOF OF LEMMA A.2
Lemma A.2. Under the regularity conditions (Al) and (A2), there exists constant Co such that for any ô > 0,
CoEoled mo + 5 -1 mo) mm) P(D=1
log (og(1/8)) +1
P
I 21
N
The proof of Lemma A.2 is adapted from Theorem in Yang al. (2024). First, expand the numerator of I into the following four parts,
(4)
L
ITI2
+
(m (x) (xb(X) - D} m1(ra1,))
(5)
+
(X; - -1 -
((( W,) - P(D - {1
W)
(6)
IEI2
(1 LL- D.)- P(D P
0) 00) (1 aa)
iEIa
= Rem, (Fa) + Remz(ra) + Remz(Fa) + Rem4.
Bound on supr. (ra)
We define class of functions Fi by
F (fr : fo = DTD ra))
One can observe that vʃ, E Fi, we have |/r| At (X)l S To. Therefore, is an envelope function of Fi. Let ||=||= denote
the supermum norm of z over a class of function F, defined by || = sup [=(f). For O = (X;, A,, S₁, Y, D.) and any function f, we define
GN
Σ [f(0)) - -(oo))(((())..
Applying Lemma A.3 with sa,, w) D and h(w, v) R produces El[G S S. with Cj being a universal constant. By McDiarnid's inequality, can show that
2t
2t2
t2
P(IGN Is. ElGGN 2 S exp
exp
exxp
ok
MN 2 22/N
4m
16

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted
Approach
where
1 G S sup sup VN (a <() - (e <r) o,,0;
2mp VN
Substituting the expectation bound to the inequality gives
|G 2 C2ROg/ 1 + log (7)) S 6,
(8)
for constant C2.
Bound supr, [Remz(ra)) Similar the bound supr. Remm (ra)l, class of function F2 is defined by F2 ( f. (ēn(X) D} m( X), VrJ, where
f. (ēb(X) - D} m)
(ēb(X) D} / mo
1fm(ra: 2 du
mo
(eD(X) D) 1fra 2
where the first equality holds by mi(a, X) Jomo 1fm X) 2 uJu and the second equality holds as fn(r X) is monotonously increase in ra- By Lemma A.3 and McDiarmid's inequality, we have
G -
du S mo. P(| BlIGNl 20) S exp
(-am²)
Amg
Substituting the expectation bound back gives us
P [GaN|l5 Camon/ 1 log
1 111 θ
S
(9)
for some constant C3.
Bound on supr,, [Rema(ra) Similarly, we first define the class of function F3 as
Fs { (i)((X {1 - (x(Xi m(r W)
The supermum norm of G. F3 and the bound of |G be obtained by Lemma A.3 and McDiarmid's inequality, respectively:
GNG $ {1mm
mo
12
Todu S Tomo,
P(IGnlla EllGn 2 S exp
4
which yields
P ||GN|15 2 Canomoy log (
S 8,
(10)
for some constant C4.
Bound on |Remal
The random variables (1 A,) is i...d. and Hoeffding inequality gives us
N Σ 1 - D.) P(D -0) 2t - <xxp (- 212
N
P
(- N
17

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Croup-Clustered Approach
which leads
log ( )?)==
P
Rema 2 \
/
sd.
(11)
2N
Combining (8), (9), (10), and (11) together with the help of union bound, we can show that with probability larger than 6,
log(1/6)
sup [emm (ra) + Rema(ra) + (ema(ra) + Remal S (mo + mo + )
[2p
which completes the proof for the bound of J--
A.4.3. ADDITIONAL LEMMAS AND THEOREMS
Lemma A.3 (Lemma 8, Yang et al. (2024)). There exists universal constant C such that for any functions s(a, d. w) [-Ko;K0] Ko] and h(w,y) y).
E sup GN/ss///saaaaa (w) s sril] < CRa-
A.5. Proof of Lemma 4.3
Denote Z(k) = {z E Z h(2)= = k3 be the set of protected groups that the clustering algorithm 7 assign to cluster k. Let R(K) be the non-conformity scores for cluster l: and R be the non-conformity scores for group 2. Since we assume that the KS distance between the score distribution for each pair of the cluster k: bounded by e, it follows that KS (R" ((() S by the triangle inequality as (k)) follows a mixture distribution of the group-specifis R such that 2 E 2
By the definition of KS distance, it implies |PRRW, v) S r.t Z 2) P(R(W, v) r Z ( c, where r is the cluster-specific threshold for cluster k. Since performing conformal inference separately for each cluster guarantees the coverage, we have
PPR(W, 9) 1 ra 7 =) Ply E C(W; Z 5
Sc.
where Ply e C(W;r&) | z E 2 > a. Therefore, we have P(R(W.y) y) s r Z 2) 2 1 a e, which completes the proof by integrating (W, Z, Y) over the target population.
B. Additional simulation details
Guidance on selecting the number of clusters The number of clusters K should be pre-specified, and we provide a simple heuristic for selecting this parameter in practice. On the one hand, each cluster should have a sufficiently large sample size to ensure efficient estimation of the thresholds. On the other hand, should not be too large; otherwise this will reduce the method group-wise conformal inference.
In particular, recommend targeting an average of 100 subjects per cluster in our study, while Ding et al. (2024) advocated for 150 subjects per cluster in their practical applications. Therefore, K is chosen as the smallest number of clusters such that each cluster has at least 100 subjects, which is adopted throughout our simulations and real-data analyses. Other human-in-the-logg strategies can be adopted to avoid overly clustering on non-clusterable groups, such as the Elbow method or other information criterion (c.g., AIC, BIC) in a cross- validation setting.
Details of nonconformity scores for categorical primary outcomes Let F(W:t) : W) t) denote nested sequence of sets constructed using the first data fold, I,. The corresponding non-conformity scores R(W, for the source data are defined as R(W,Y) Y) = inf estimated using multinomial logistic regression from the NNET package. To construct the scores for the target data, where the primary outcomes are unobserved, we fit a model on the scores R(W,, Y2) from the source data = 1,... nDi on W., and predict the scores for the target data. Finally, the semiparametric efficient estimator çk for cluster l is computed as the smallest value satisfying the condition Eie 2), (V; c. M, m. Pb) 2 O, where the nuisance models m. m, and ē are estimated from Z₁ The models are all fitted using SUPERLEARNER (Van der Laan et al., 2007), with Random Forests and generalized linear models as base learners.
E F(W; t)), where the conditional probabilities Py W) are
18

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustered Approach
Fraction of under-coverage and over-coverage The fraction of protected groups that under-covered and over-covered evaluated well. We define the fraction of under-covered (FracUnderCov) and over-covered (FracOverCov) groups by: FracOverCov = Dzez 1 - a + 0.01)/M and FracUnderCov Excz 1 ( < 1 0.01)/M, where the value 0.01 is included to account for kinite-saple randomness and č₂ is the empirical group group-conditional conditional coverage for group 2.
SAGCCI
Surro + Group
Surro + Standard
low surrogate effects
medium surrogate effects
high surrogate effects
0.5
0.4
FracOverCov
0.3
0.2
0.1
0.0
0.4
FracUnderCow
0.2
0.0
1000
3000
5000
10000
1000
3000
5000
10000
1000
3000
5000
10000
Sample size
Figure 3. Comparison of FracOverCov and FracUnderCov for the considered surrogate-assisted methods. The error bar plot denotes + the standard errors.
Figure illustrates the results for the considered surrogate- assiste methods. can be observed that both SURRO + GROUP and SAGCCI exhibit well-controlled fractions of under-coverage and over-coverage the sample size increases. However, SURRO GROUP achieves the low FracOverCov and FracUnderCov by producing larger prediction sets average, which can be observed in Figure 2. Moreover, SURRO + STANDARD continues to exhibit non-negligible fractions of under-coverage or over-coverage, even with large sample sizes, as is unable to guarantee group-conditional coverage.
Larger number of protected groups and clusters To evaluate the impact of a larger number of protected groups, we generate the protected group variable by randomly sampling from {1.... ..), where the number of groups M is set 10 20. The baseline covariates X and the surrogate outcomes are generated in the same way as described in the main simulation studies. Next, the primary outcomes are generated as a multinomial variable with five levels, where the conditional probabilities are given by: P(Y=y|X,Z.5) exp (-ay 25
Z X2
1= St with the same marginal
probabilities P(Y = k) 0.2, 0.4, 0.15, 0.15) in the main paper.
Figure presents the results for fixed number of clusters, K 5. As the number groups increases, the performance of SURRO+GROUP noticeably degrades in both metries (AvgSize and CovGap), even when N 2 5000 for M = 20. This is because each group contains fewer samples, making harder to estimate non-conformity scores accurately. In while
contrast, SAGCCI adopts more balanced approach, achieving AvgSize comparable SURRO+STANDAN also maintaining smaller CovGap the sample size increases. Furthermore, in the absence of surrogate outcomes, VOSURRO+CLUSTER the aon-surrogate-asisteee version of SAGCCI, still delivers strong performance by effectively balancing AvgSize and CovGap.
19

---

Bridging Fairness and Efficiency in Conformal Inference: A Surrogate-Assisted Group-Clustern Clustered Approach
SAGCCI
Surro + Group
Surro + Standard
Number of G roup 10
Number of Group =0
high surrogate effects
high surrogate effects
5
4
AvgSize
3
2
6
CovGap
T
2
1000
3000
5000
1000
3000
5000
NoSurro Standard
NoSurro Group
NoSurro Cluster
Number of Group = 10
Number of roup 20
no surrogate effects
no surrogate effects
4.75
4.50
AvgSize
4.25
4.00
3.75
6
T
CovGap
4
2
F
0
1000
3000
5000
1000
3000
5000
Sample size
Figure 4. Comparison of AvgSize and CovGap for the considered methods for larger number of groups and clusters. The error bar plot denotes + the standard errors.
20


Reviews:
[{"review_id": "lo9coFdggx", "reviewer": "Reviewer_BCfr", "summary": "This paper introduces a conformal inference algorithm (SAGCCI) aimed to produce prediction sets that satisfy group-conditional coverage guarantees. The algorithm is intended to be robust to missing information settings, where sometimes the primary outcome (around which we are trying to produce prediction sets) is missing in the training data, but assuming the existence of a surrogate outcome that may be correlated and easier to collect data for. It also achieves small mis-coverage rates even on groups of small size with little data to extrapolate from. \n\nTo achieve this, SAGCCI does two main things: (1) clustering together groups with similar non-conformity score distributions to more accurately estimate the $(1-\\alpha)$-quantile for each of them, by having larger amounts of data, and (2) using an efficient influence function to use the surrogate outcomes to estimate non-conformity scores (the appropriate threshold to satisfy the desired coverage level) even when primary outcomes are absent and the distribution may be different. \n\nThe main contribution is combining the clustering approach (allowing you to extrapolate about low data groups using other groups with similar score distributions) with using EIF for a more efficient estimator of the non-conformity threshold (to produce tighter prediction sets).", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the careful review and nice comments! We here provide detailed responses to your comments.\n\n**Experimental Designs Or Analyses** & **Other Comments Or Suggestions**\n\n1. The **size of the groups was $M=3$**. We have conducted **additional experiments with a larger number of groups (e.g., $K=10, 20$), where similar conclusions can be drawn; the results are presented [here](https://anonymous.4open.science/r/ICML2025-5158/fig1.png).** The considered group-conditional conformal algorithm (i.e., Surro+Group) is extended from **Vovk (2012)**, and is similar to **Gibbs et al (2023)**, where we further account for the covariate shifts across the groups.  However, this method *neglects the fairness condition and may have inflated CovGap, as seen in Figure 3*. We will emphasize this point in the paper.\n\n**Other Strengths And Weaknesses** & **Relation To Broader Scientific Literature**\n\n1. Thanks for bringing up the question about overlapping protected groups! **Our method is actually applicable to protected groups that overlap with a relatively minor extension.** Let $G_{fair}$ represent the set of groups (potentially overlapping) across which we aim to impose fairness, which is essentially a function of the protected groups $Z$ (see Assumption 2.1 in \"Fairness with Overlapping Groups\"). Then our method could be used on this new set of groups $G_{fair}$. Furthermore, our method could incorporate other clustering mappings instead of k-means, such as *Fuzzy k-means* or *overlapping k-means*, which allows for overlapping clustering as well. Thank you for providing more papers on conformal inference for achieving group-conditional guarantees. We will check them and reference them in the revised paper. \n\n2. Although the clustering idea is introduced in Ding et al, its **application to fairness-related problems is new, as mentioned by Reviewers G4EX (\"Relation to Broader Scientific Literature\") and uHDb (\"Claims And Evidence\").** Furthermore, the derivation of the EIF with surrogates is an interesting theoretical result in its own right. We further quantify the efficiency gain of leveraging surrogates in Corollary 4.5 and establish the coverage guarantee in Theorem 4.6. See the JRSS-B paper \"On the role of surrogates in the efficient estimation of treatment effects with limited outcome data\" for related theory, which develops the EIF with surrogates to estimate average treatment effects. Our work extends this theory to the conformal prediction setting. This point will be mentioned in the paper.\n\n**Questions For Authors**\n\n1. We noticed that we were not clear in explaining the split conformal inference strategy in Section 4.4. We have provided a new pseudo-code to illustrate this algorithm. To your question specifically, these two datasets are combined and split into training and calibration folds as $I_1$ and $I_2$. The intersection of the training and source data (i.e., $I_1\\cap D_1$) is used to fit the non-conformity scores (e.g., the quantile functions), and the full training data $I_1$ is used to fit the density ratio $\\pi_D(X)$. The reason for the random splitting of the combined data is to ensure that the density ratio $\\pi_D(X)$ remains exchangeable across folds $I_1$ and $I_2$. Next, these fitted models are used in the EIF evaluated on the calibration set $I_2$ to obtain the estimator for the cluster-specific threshold $r_{\\alpha}^k$ over the target data. The **full algorithm is provided [here](https://anonymous.4open.science/r/ICML2025-5158/alg1.png).**\n\n---\n\n**References**\n\n1. Vovk, V. Conditional validity of inductive conformal predictors. In Asian conference on machine learning, pp. 475–490. PMLR, 2012.\n\n2. Gibbs, I., Cherian, J. J., and Cand `es, E. J. Conformal prediction with conditional guarantees. arXiv preprint arXiv:2305.12616, 2023.", "claims_and_evidence": "The theoretical claims seem sound, and the experiments bear out that the paper’s algorithm achieves smaller prediction sets than existing methods.", "methods_and_evaluation_criteria": "Yes, it seems fine.", "theoretical_claims": "I did not check the proof for Theorem 4.4 (in the appendix), but the remaining proofs look good.", "experimental_designs_or_analyses": "The experiments looked good. Since the role of clustering is to provide better coverage guarantees for smaller groups, I would have liked to see the size of the groups over which we are trying to achieve coverage, not sure if this is mentioned somewhere but I couldn’t find it. Also, it is not clear what group-conditional conformal inference algorithm the authors are using as a baseline to compare against – is it the approach of calibrating thresholds separately for each group as in Vovk 2012? The authors cite “Conformal Prediction with Conditional Guarantees” (Gibbs et al 2023) as having algorithms for group-conditional guarantees but I don’t think they compare against this.", "relation_to_broader_scientific_literature": "The closest connection in terms of the clustering idea is “Class-Conditional Conformal Prediction with Many Classes” (Ding et al) as the authors cite. This work does not extend the ideas of the original paper – the clustering is applied to abstractly defined disjoint groups here, rather than groups defined implicitly by class label. \n\nThere is existing work on conformal inference for achieving group-conditional guarantees (Jung et al 2022, Gibbs et al 2023) in distributional settings as well as in adversarial settings (Bastani et al 2022), which may have connections as the adversarial case can handle settings where primary outcomes may be unavailable. These algorithms may not be directly comparable to SAGCCI as they also allow for overlapping groups.", "essential_references_not_discussed": "Not to my knowledge.", "other_strengths_and_weaknesses": "Strengths:\n\nThe setting for achieving group-conditional coverage even in missing information settings is interesting. The experiments show the efficacy of this approach with smaller prediction set sizes.\n\nWeaknesses:\n\nVia the given algorithm, group-wise coverage can only be achieved on disjoint groups on the dataset; if one wanted to achieve fairness in terms of coverage over protected groups that overlap, this algorithm cannot be used. I also find the paper a bit lacking in originality. The authors claim two innovations – using clustering, and the surrogate-assisted EIF. As far as clustering goes, the process seems identical to that in Ding et al (2024), applied to general disjoint groups rather than class label groups. I am not very familiar with derivations of EIF, so am not sure whether the derivation here is standard or not – perhaps other reviewers could weigh in or maybe the authors could clarify this.", "other_comments_or_suggestions": "I would have liked to see the size of the groups defined for group-conditional coverage in each of the experiments.", "questions_for_authors": "1.\tI am quite unclear in which places the source data (where primary outcomes are observable) and target data are used together, and when they are separated. In Section 3.2, it seems to be that the source data is used to estimate a non-conformity threshold over the target data (which may come from a different distribution). But in the implementation section (Section 4.4), it says that the data is combined and then split into folds, and in the experiments, they seem to be used together as well. Could the authors clarify this point?\n\n2.\tIf the fold $I_1$ includes some target data (Section 4.4), how are the quantile models trained when some of the primary outcome data is missing?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "volii7xDnS", "reviewer": "Reviewer_G4EX", "summary": "This work studies how to provide (conformal) prediction intervals for individual outcomes when additional information, in the form of surrogate outcomes, is available for both source and target datasets. The goal is to provide tight (short) prediction intervals that also achieve nominal coverage for all subgroups. The proposed method achieves this by clustering the groups based on their conformal score distributions, then leveraging the surrogates to provide per-cluster thresholds for the conformal scores (i.e. to construct prediction intervals). The authors provide theoretical results about the efficacy of their method and give experiments on both simulated and real data.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed comments and provide itemized responses to your questions.\n\n**Theoretical Claims** & **Questions For Authors**\n\n1. The asymptotic coverage in Theorem 4.6 should be interpreted as $P_{V\\sim P_{D_0}}(y \\in C(W;r_{\\alpha}^k)\\mid Z = z) \\geq 1-\\alpha - o(1)$ as $N\\rightarrow \\infty$ since the **constants do not depend on the sample size $N$**. Therefore, the $(\\alpha, \\delta)$-PAC guarantee can be **approximately** met with negligible errors; see the last paragraph of Section 3.1 in the paper by Yang et al. titled \"Doubly Robust Calibration of Prediction Sets under Covariate Shift\" published in JRSS-B. **The constants are a result of the use of concentration inequalities**. We will update our statements to be more precise in the revised paper.\n\n2. Thanks for pointing out that the protected groups may not always be clusterable. We feel this question would fall into a more general concern as \"clustering on non-clusterable groups\". We have provided a **simple heuristic in Appendix B to choose the number of clusters,** inspired by Appendix B.4 in \"Class-Conditional Conformal Prediction with Many Classes\". We could further resort to human-in-the-loop strategies to avoid overly clustering on non-clusterable groups, such as the Elbow method and other information criterion (e.g., AIC, BIC) in a cross-validation setting. This point will be added to Appendix B of our paper.\n\n**Experimental Designs Or Analyses** & **Questions For Authors**\n\n1.  The number of protected groups $M$ for the presented experiments is $3$ and the number of clusters is $K=2$ as expected. To achieve the tradeoff between efficiency and fairness, $K$ should be no larger than $M$ but larger than $1$ (i.e., $1<K\\leq M)$. \n\n2. The reason we only show a subset of these considered methods is to avoid presenting too much information in one figure and to avoid clutter. Figure 2 focuses on comparing SAGCCI with NoSurro+Cluster and WCQR (with surrogates) to show that we have leveraged surrogates in the most efficient way by leveraging the EIF. Figure 3 focuses on comparing SAGCCI with Surro+Group and Surro+Standard to showcase the advantages of using group-clustered conformal inference to achieve a good balance between efficiency and fairness.\n\n3. For the presented experiments, SAGCCI and Surro+Group achieve similar performance when $N\\geq 3000$. When we increase the number of groups $M$ to $10$ and $20$ while fixing $K=5$, Surro+Group has noticeably worse performance, even if $N\\geq 5000$, when $M=20$ since each group would have fewer samples to fit the non-conformity scores, which leads to deteriorated performance in both AvgSize and CovGap. These **new results are presented [here](https://anonymous.4open.science/r/ICML2025-5158/fig1.png).** This point will be made clear in the revised paper.\n\n**Relation To Broader Scientific Literature**\n\n1. When the surrogate outcomes are not unavailable, the benefits of clustering can be shown by comparing the performance of NoSurro+Cluster with NoSurro+Standard and NoSurro+Group, with **new results shown [here](https://anonymous.4open.science/r/ICML2025-5158/fig3.png)**, where $\\sigma_S = 0$ indicates there are no surrogate effects. Full comparisons will be added to the Appendix for completeness. \n\n**Other Comments Or Suggestions**\n\n1. We agree that Figure 1 can be confusing to readers and therefore **replace it with a detailed pseudo-code algorithm, which can be found [here](https://anonymous.4open.science/r/ICML2025-5158/alg1.png).**", "claims_and_evidence": "* See below. Overall yes, though there are limitations to the main theoretical result as written; unclear if the experimental setup captures everything that the work hopes to.", "methods_and_evaluation_criteria": "* Synthetic and real datasets make sense. See below for comments on experiments.", "theoretical_claims": "* I did not check proofs in detail; the statements and high-level structure of the arguments are consistent with what I expect. \n* I am guessing that the constants in Theorem 4.6, if they had been worked out, might be very large; thus, though the result is technically given as a finite-sample bound, the interpretation of it should probably be more qualitative (“we should more or less expect this to work”) rather than precisely quantitative (“we need X samples to guarantee $(\\alpha+\\epsilon)$-valid intervals with probability $1-\\delta$”). That said, I think this is fine given the experiments seem to suggest that the method can do well with reasonably small $n$.\n* A detail that seems like it might be important, but is mostly glossed over, is that it's important for (a) the score distributions to be clusterable and (b) the clustering method to work well, since this error shows up additively. It's not actually obvious to me that (a) is actually always possible.", "experimental_designs_or_analyses": "* Based on the exposition in section 4, I thought that $M$ might be very large and that $K << M$. I was thus a bit confused/surprised to see $M = 3$; and I couldn’t find what $K$ was set to (although for it to be anything nontrivial, I’m guessing it must be 2). Unless the $M$ groups are actually split up into $K >> M$ clusters so that cluster $k$ is only a subset of the samples from group $m$? And then the clustering approach also has the added benefit of adding structure to intra-group variability? But in that case I feel like section 4 should be written differently to reflect that. \n* The benchmarks/ablations all make sense, though I’m not sure why each plot only shows a subset of them. \n* Based on Figure 3, it seems like for $n \\geq 3000$ that SACGCI and surro+group are essentially identical?", "relation_to_broader_scientific_literature": "* This work utilizes tools and ideas from causal inference (surrogates), and combines them with a clustering idea (which, though has been proposed in other fairness-related works, I haven’t seen in this context), in order to address a well-established problem in the conformal prediction community. \n* It would be nice to understand whether, for problem settings where surrogate outcomes are unavailable, clustering might still be beneficial.", "essential_references_not_discussed": "* N/A to my knowledge", "other_strengths_and_weaknesses": "* N/A", "other_comments_or_suggestions": "* It would be nice in terms of presentation to have a pseudocode algorithm box to summarize the procedure. I think Figure 1 is supposed to illutrate this but it’s honestly very confusing to me.", "questions_for_authors": "* Can you comment on the relationship between $K$ and $M$? Experimentally, what did you set as $K$? Did you try experiments for large $M$, or are there barriers to doing so with this method? \n* Can you comment on the extent to which you might generally expect score distributions to actually be clusterable?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R7qw1pr8S0", "reviewer": "Reviewer_uU3r", "summary": "The authors introduce a new strategy for achieving equitable group coverage in conformal inference. The approach consists of two components: (1) rather than using raw sensitive groups, they cluster groups with similar conformal score distributions via K-means and (2) they leverage surrogate information correlated with the outcome to obtain tighter prediction sets. This method is supported by theoretical guarantees. To evaluate its effectiveness, the authors conduct experiments on a synthetic dataset and a real-world dataset, analyzing the impact of each component as well as the overall performance.\n\n## Updated after rebuttal\n\nI sincerely appreciate the authors’ detailed response and the additional experimental results. These new findings seem promising. However, I remain concerned about the **empirical evaluation**, which I find to be quite **limited** for a venue like ICML. I believe that the theory needs to be justified with more rigorous experiments and examples, highlighting the benefits and limitations of the approach. In its current form, the evaluation does not, in my view, provide sufficient evidence to support strong conclusions regarding the effectiveness of the proposed approach. The experimental analysis focuses primarily on a toy problem (which could be constructed to highlight the strengths of the proposed method) and only a single real-world dataset. This narrow evaluation makes it challenging to fully assess the practical benefits and general applicability of the approach. Moreover, the comparison to SOTA methods is rather limited, which further hinders the understanding of the method’s relative performance.\n\nFinally, since the proposed approach appears to build upon the method by Ding et al., with additional features intended to improve fairness, it would be particularly valuable to include a direct comparison to Ding et al. in the empirical analysis. Such a comparison would help to clearly demonstrate the impact of the proposed enhancements.\n\nTherefore, in principle, I will mantain my score.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the comprehensive review. We now provide detailed responses to your concerns.\n\n**Theoretical Claims**\n1. $V_{eff}^r$ and $V_{eff}$ are referred to as the **variances** of the estimators. Therefore, the more predictive the surrogates are, the smaller $V_{eff}$ will be and the larger (more positive) the quantity $V_{eff}^r - V_{eff}$ will be.\n\n2. The quantile functions are trained over the source data $D_1$ and can be used on the target data $D_0$ under Assumption 4.2 (b) $D \\perp Y \\mid S, X$. \n\n3. Thanks for catching this; we have updated Assumption 4.1 to be $c_0 < P(D=1\\mid x)$ instead.\n\n**Experimental Designs Or Analyses**\n1. The improved efficiency of SAGCCI is achieved by approximately guaranteeing group-conditional coverage, **a tradeoff between efficiency and fairness**. The intention is to produce prediction sets that have good group-conditional coverage AND are not too large to be useful. We will remove overstatements of the superiority of our method and add: *\"the choice of an appropriate conformal prediction method should be assessed based on the problem setting at hand and the objectives of the analysis.\"*\n\n2. For the presented experiments, SAGCCI and Surro+Group have similar performance when $N\\geq 3000$. When we increase the number of groups $M$ to 10 and 20, while fixing $K=5$, Surro+Group performs much worse, even if $N\\geq 5000$, since each group would have fewer samples to fit the non-conformity scores, which leads to deteriorated performance in both AvgSize and CovGap. The **new results are presented [here](https://anonymous.4open.science/r/ICML2025-5158/fig1.png).** This point will be made clear in the paper.\n\n3. When $\\sigma_S^2$ increases, surrogate information will be **less** representative across the groups, since each group tends to have more different surrogates in finite samples. Therefore, the **Surro+Group will have larger AvgSize in finite samples** since it is influenced more by the surrogates which have larger variability.\n\n4. A more precise illustration is now presented: when $\\sigma_S=3$, the proposed **SAGCCI reduces the size of prediction sets by over 60% on average compared to Surro+Group at $N=1000$ but increases the CovGap by only 7%**. **Compared to Surro+Standard, the sizes of the prediction sets are similar, but SAGCCI reduces the CovGap by over 80% at $N=3000$.**\n\n5. Age-morbidity groups are of **clinical relevance** and the fairness evaluation is of real world concern -- we will update the paper to motivate the sensitive groups based on these subgroups rather than on race.\n\n**Essential References Not Discussed**\n1. We now provide more discussion of the proposed method and state-of-the-art methods. Existing fair conformalized predictions (Liu et. al., 2022, Vadlamani et. al. 2025) focus on learning a fair quantile function (termed **ConformalFairQuantl**) to construct the non-conformity score to deliver reliable fair prediction sets. However, **our method is more general and applicable to any non-conformity score**, and is not restricted to only conformalized quantile scores. **Their methods do not account for the distributional shifts in covariates**, and cannot be used in our setting, where outcomes from the target data are missing; **see new results [here](https://anonymous.4open.science/r/ICML2025-5158/fig2.png), which show that the performance of ConformalFairQuantl is not satisfactory under our problem setup.** The review paper and other fair conformalized predictions will be referenced in the paper. \n\n**Answer to Questions For Authors**\n1. We realized that we were not clear about the implementation of the proposed method. **A full algorithm is now presented [here](https://anonymous.4open.science/r/ICML2025-5158/alg1.png).** Most of the models (e.g., quantiles and clustering) are fitted on $I_1\\cap D_1$, while the density ratio is fitted on $I_1$ with both source and target data.\n\n2. Although the clustering idea is introduced in Ding et al, its **application to fairness-related problems is new, as mentioned by Reviewers G4EX (\"Relation to Broader Scientific Literature\") and uHDb (\"Claims And Evidence\").** \n\n3. **New results for a larger number of protected groups are presented [here](https://anonymous.4open.science/r/ICML2025-5158/fig1.png).**\n\n4. If one's sensitive information is missing, we could assume these subjects with missing sensitive information belong to one cluster and proceed with our approach.\n\n5. The cluster number is now chosen **automatically** by the simple heuristic described in Appendix B. To select the optimal cluster number, we could also use human-in-the-loop strategies, such as the Elbow method and other information criterion (e.g., AIC, BIC). This point will be added to Appendix B.\n\n6. Since $S$ is included in the generation of $Y$, higher variance in $S$ indicates that $S$ is more predictive of $Y$. Therefore, there is no upper limit on the maximum value of efficiency gain as long as $S$ is included in the data generation.", "claims_and_evidence": "Most claims are supported by the analysis conducted in the paper. However, some statements raise concerns:\n\n (1) The authors describe their experiments as \"extensive synthetic simulations,\" yet only a single synthetic problem is considered. Referring to this as extensive seems like an overstatement of the empirical analysis provided.\n\n(2) The term \"uncertainty quantification\" appears in both the abstract and the final paragraph of the introduction. However, this term is broad, as uncertainty can stem from multiple sources. The paper does not further discuss uncertainty estimation or quantification, nor does it provide specific theoretical or empirical analysis on the topic. To avoid unsupported claims, the authors should either use a more precise term or clarify their intended meaning on this matter.", "methods_and_evaluation_criteria": "The synthetic and real-world datasets chosen are well-suited for evaluating the proposed approach. Additionally, analyzing the individual contributions of clustering and surrogate information through ablation studies is highly valuable for understanding their respective effects and whether their combination leads to the best performance.\n\nTo further strengthen the empirical evaluation, it would be beneficial to include comparisons with existing state-of-the-art methods (e.g., Ding et al. (2024), Gao et al. (2024)). This would provide a clearer assessment of the method's impact and contribution within the broader literature. Especially since the text often lacks clarity on how the proposed approach aligns with or differs from the referenced works in the literature.", "theoretical_claims": "I did not check the correctness of the proofs in detail. \n\nHowever I have several concerns:\n\n(1) You define efficiency gain as $V_{err}^r - V_{err}$, however the former term refer to the EIF without surrogates and the latter with surrogates. Therefore the bigger the EIF of the surrogate information the more negative this quantity will be? In order to refer to a gain, wouldn't it be $V_{err} - V_{err}^r$?\n\n(2) Throughout the text, starting from equation (1), you estimate a probability based on the quantity Y over the target set $D_0$ where, according to your definition,  Y is unknown... can you make clarifications on this matter?\n\n(3) In Assumption 4.1. you employ the term $P(D=1|x)$, but then you claim that the assumption \"states that each individual should have probability of at least $c_0$ of being included in the target data (...)\", but D=1 refers to source data. Could you clarify this matter?", "experimental_designs_or_analyses": "Some results could benefit from a more nuanced discussion to avoid potential overestimation of their implications.\n\nFor instance, in Figure 3, the primary difference between SURRO+GROUP and SAGCCI emerges when the number of instances is N=1000; for other values, their performance is quite similar across both considered metrics. At N=1000, SURRO+GROUP achieves better coverage but results in larger prediction sets. Given this trade-off, it is challenging to conclude that SAGCCI is definitively superior. Moreover, the performance gap appears to widen as the surrogate information becomes more representative, which is somewhat unexpected since both methods leverage surrogate information. Why does this difference arise given the fact that the only difference between both approaches is that one employs the clusters and the other the raw group information?\nA deeper discussion on why these differences arise would strengthen the analysis. Additionally, when evaluating multiple objectives, it is important to avoid suggesting a dominance relationship unless it is well-supported. Since the two objectives are measured on different scales, a difference of 1 in one metric may not correspond to an equivalent change in the other, further complicating direct comparisons.\n\nSimilarly, in Section 6, the claim that \"SAGCCI achieves the best overall performance\" may be somewhat misleading. The evaluation is multi-objective, and while SAGCCI performs best on some metrics, it ranks second on others. Since it does not consistently outperform across all criteria, asserting overall superiority requires additional justification. Given that the metrics are on different scales, it is also unclear whether a simple aggregation would confirm this claim.\n\nClarifying these aspects and being precise about comparative statements would enhance the rigor of the evaluation and make the findings more transparent.\n\nMoreover, building on the previous comments, the claim at the beginning of Section 5—\"Empirical results show (...) while significantly shrinking prediction set sizes relative to existing methods\"—appears to be somewhat overstated and could benefit from a more precise formulation. The phrase \"significantly shrinking\" is subjective and lacks quantitative support. It would be more informative to specify the extent of the reduction in numerical terms, such as stating that the prediction set size is reduced by a certain percentage (e.g., 50%). Additionally, the assertion that the method always produces the smallest prediction set by a large margin wrt other methods is not entirely accurate. It would be helpful to clarify under what conditions the method outperforms others and acknowledge any cases where it does not. This refinement would improve the clarity and credibility of the statement.\n\nOn the other hand, I also have an additional concern regarding the experiments in Section 6 (real data). While the study describes different sensitive groups based on race, the fairness evaluation is conducted in terms of age-morbidity groups.", "relation_to_broader_scientific_literature": "The authors propose a combination of two existing techniques within the literature of conformal inference that are the clustering of the groups (e.g., Ding et al. (2024)) for a fair coverage, and employing surrogate information correlated with the true class label to construct tighter prediction sets (e.g., Gao et al. (2024)).", "essential_references_not_discussed": "The related works section focuses exclusively on the use of surrogates. Expanding it to include a discussion on conformal inference and its intersection with fairness guarantees would provide a more comprehensive overview of the field. This addition would help readers better understand the broader context and position the contribution within existing research.\n\nAdditionally, the paper does not clearly discuss the similarities and differences between the proposed approach and state-of-the-art methods. Providing such a comparison would highlight the specific advantages of the proposed method relative to existing works.\n\nFurthermore, in Section 3.3, when discussing group fairness notions, the authors cite Hardt et al. (2016). However, this work specifically addresses a post-processing method for enforcing equality of opportunity and equalized odds, rather than general group fairness metrics. A more appropriate reference, such as [1], would better align with the discussion on group fairness notions.\n\n[1] Pessach, D., & Shmueli, E. (2022). A review on fairness in machine learning. ACM Computing Surveys (CSUR), 55(3), 1-44.", "other_strengths_and_weaknesses": "This is a very important problem, and the authors' approach to analyzing it offers a particularly compelling perspective.", "other_comments_or_suggestions": "The section titles do not adhere to the conference's recommended style, which requires capitalization of the words.", "questions_for_authors": "Q1 - The role of each dataset is not entirely clear. Do you use $D_1$ in a specific step of the training process (e.g., clustering) and $D_0$ in another? It seems that the entire training process requires access to Y, suggesting that the approach is exclusively trained on $D_1$. Is this true? Or does $D_0$ have a significant role in training, or is its contribution primarily in another stage?\n\nQ2 - Which are the main differences between your work and the work by Ding et al. (2024)? \n\nQ3 - In real-world applications the number of sensitive groups can be high, specially when intersectionality is considered. How does your method behave for an increasing number of sensitive groups? \n\nQ4 - Another common real-world scenario is having missing sensitive information. Could this method be extended to handle such scenarios?\n\nQ5 - It is known that when employing K-means for clustering the subsequent results can vary considerably according to the value of K. Since the clustering is a key point of the proposed approach, I would like to ask how sensitive are your results for different values of K.\n\nQ6 - Is it always the case that a higher variance on the surrogate variable indicates higher predictiveness of the primary outcomes Y? Or is there any upper limitation on the maximum value that this variance can take (above which this informativeness is lost or at most not improved)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9yUntP8c2I", "reviewer": "Reviewer_uHDb", "summary": "- This paper presents a novel approach to fair conformal inference.\n- Specifically, the proposed method is based on an group-conditional conformal inference.\n- The proposed method is based on two keys: (1) clustering score values and (2) the use of an influence function for prediction sets.\n- Experimental results suggest that the proposed method is competitive to three alternative approaches and one existing approach.\n\n## update after rebuttal\n\n- I thank the authors for their clarifications. Based on their response, which includes additional experiments, I will raise my score from 2 to 3.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful review. We provide responses to your questions:\n\n**Claims And Evidence**\n\n1. **Fair coverage is crucial in socially consequential domains**, where prediction intervals can help guide decision-making related to access to resources, opportunities, or fair treatment. For example, if a mortgage lender’s prediction intervals systematically under-cover minority borrowers, they may face inflated interest rates or higher denial rates. In healthcare, if limited supplies of critical antibiotics demand triaging decisions, under-coverage for certain demographics could result in denying them lifesaving medications. We will include these examples in the paper.\n\n2. If a given prediction model achieves fair coverage in terms of conformalized prediction sets, it does not necessarily lead to fairness in point predictions. If the length of the prediction sets are the same and the prediction sets are symmetric about the point predictions, fair coverage in prediction sets could imply fairness in point predictions. Therefore, fair coverage in prediction sets could be considered as an extension to group fairness (e.g., demographic parity) in point predictions, as coverage in prediction sets additionally accounts for the uncertainty of point predictions (**as our primary motivation**). Even if a given model does not achieve group fairness in terms of the point predictions, it can achieve fair coverage in the prediction sets, and one could conclude that its prediction is fair with $(1-\\alpha)$ confidence given a user-specified miscoverage level $\\alpha$. This point will be emphasized in our paper.\n\n**Methods And Evaluation Criteria**\n1. The baseline WCQR from Lei & Candes (2021) was designed for counterfactuals (or individual treatment effects), which essentially aims to account for covariate shifts between groups with observed outcomes and missing outcomes. Therefore, it is **directly applicable in our setting** to account for covariate shifts between source data (with outcomes observed) and the target data (with outcomes missing). Therefore, **WCQR is an appropriate baseline comparator method for SAGCCI.** **We highlight the additional efficiency SAGCCI gains vs WCQR by estimating the thresholds via the EIF.**\n\n**Experimental Designs Or Analyses**\n1.  For the presented experiments with the number of groups $M=3$ and the number of clusters $K=2$, SAGCCI and Surro+Group have similar performances when $N\\geq 3000$. When we increase $M$ to $10$ and $20$, while fixing $K=5$, Surro+Group would have much worse performance even if $N\\geq 5000$, since each group would have fewer samples to fit the non-conformity scores, which leads to deteriorated performance in both AvgSize and CovGap. The **new results are presented [here](https://anonymous.4open.science/r/ICML2025-5158/fig1.png).** \n\n2. All the considered conformal inference methods are implemented by the split conformal inference strategy -- therefore, their running times are similar. For example, **for $N=1000$, the running time of these methods are all around 1 second** when 25% of the data has been used for calibration and to construct the conformalized prediction sets. This point will be made clear in the paper. \n\n**Essential References Not Discussed**\n1. Liu et. al., (2022) and Vadlamani et. al. (2025) focus on learning a fair quantile function (termed **ConformalFairQuantl**) to construct the non-conformity score to deliver reliable fair prediction sets. However, **our method is more general and applicable to any non-conformity score**, and is not restricted to these conformalized quantile scores. **Their methods do not account for the distributional shifts in covariates**, and is therefore not directly suitable to our setting, where the outcomes from the target data are missing; **see the updated results [here](https://anonymous.4open.science/r/ICML2025-5158/fig2.png),** where the performance of **ConformalFairQuantl** is not satisfactory under our problem setup.\n\n**Other Comments Or Suggestions** & **Questions For Authors**\n\n1. We have fixed the typos and remove the subscript in $V_f$ to avoid any confusion. Now, we have a new data set $V= (W,Z,Y)\\sim P_{D_0}$. We use the quantile random forest to fit the quantiles $q_{\\alpha/2}$ and $q_{1-\\alpha/2}$ and the non-conformity scores are constructed by the conformalized quantile residual by $R(W, Y)= \\max(q_{\\alpha/2}(W)-Y, Y-q_{1-\\alpha/2}(W))$.\n\n---\n**References**\n\n1. Lei, L. and Candes, E. J. Conformal inference of counterfactuals and individual treatment effects. Journal of the Royal Statistical Society Series B: Statistical Methodology, 83:911–938, 2021.\n\n2. Liu, Meichen, et al. Conformalized fairness via quantile regression. Advances in Neural Information Processing Systems 35 (2022): 11561-11572.\n\n3. Vadlamani, Aditya T., et al. \"A Generic Framework for Conformal Fairness.\" (2025) The Thirteenth International Conference on Learning Representations.", "claims_and_evidence": "- The group fairness notion in coverage, which is introduced in Section 3.3, could be clarified with concrete justifications or examples. Why is fair coverage particularly important in social aspects? Introducing examples of undesirable consequences due to unfair coverages would be helpful.\n- Furthermore, if a given prediction model does not achieve demographic parity but achieves fair coverage in conformal inference, does this imply that decision-making based on this model is fair?\n- This paper focuses on protected group-wise fairness, while Ding et al. (2023) focused on the label class-wise conformal prediction. That is, the concept of \"group-conditional\" conformal inference is not entirely novel within the broader machine learning research, though its application to algorithmic fairness is new.", "methods_and_evaluation_criteria": "- The experiment part would be more convincing if the authors could explain details about the baseline method WCQR from Lei & Candes (2021), as WCQR is originally designed for counterfactuals and individual treatment effects. Is WCQR an appropriate baseline for SAGCCI?", "theoretical_claims": "- I have briefly checked the proofs of Lemma 4.3 and Theorem 4.4.", "experimental_designs_or_analyses": "- The authors claim that SAGCCI outperforms existing methods, however, several experimental results (e.g., Figure 3, Table 1, Figure 4) show that a baseline (Surro + Group) is competitive to or sometimes outperforms SAGCCI.\n- As Section 4.4 presents that SAGCCI requires additional computation for split conformal inference and non-conformity scores, a comparison of running time between SAGCCI and baseline methods would enrich the experiment parts.", "relation_to_broader_scientific_literature": "- This work would contribute to emphasize a need of uncertainty quantification technique for algorithmic fairness.", "essential_references_not_discussed": "- I believe that the authors could compare (conceptually at least) the proposed algorithm with existing methods in conformal prediction and algorithmic fairness.\n   - Examples:\n      -  https://papers.neurips.cc/paper_files/paper/2022/file/4b52b3c50110fc10f6a1a86055682ea2-Paper-Conference.pdf \n      - https://openreview.net/pdf?id=xiQNfYl33p", "other_strengths_and_weaknesses": "- N/A", "other_comments_or_suggestions": "- (typo) line 39: estaimaled -> estimated", "questions_for_authors": "- Could the authors clarify the meaning of the subscript $f$ in several notations such as $V_{f}, Y_{f}, W_{f}$ in eq. (1)?\n- What model was used in practice for estimating $\\hat{E}$ in the definition of $R(W,Y)$ in Section 3.2?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces SAGCCI, an innovative algorithm for group-conditional conformal inference, leveraging clustering and surrogate outcomes to enhance fairness and efficiency in prediction intervals. The theoretical foundations are sound and well-aligned with the broader literature, offering an interesting and novel perspective on conformal inference. However, the empirical evaluation raises some concerns, which limit the ability to fully assess the method's practical contributions.

The primary issue lies in the scope and depth of the empirical evaluation. While the authors conduct experiments on a synthetic dataset and a single real-world dataset, these settings may not be sufficient to establish the generalizability and robustness of the proposed approach. Reviewers expressed a desire for additional benchmarks against state-of-the-art methods, as well as a more detailed exploration of key factors, such as the variability of group sizes and the trade-offs between fairness and efficiency metrics. These elements are critical to understanding the practical implications of the method and are not fully addressed in the current work.

Although the authors provided further experiments and clarifications in their rebuttal, some aspects remain unclear. For instance, the reported improvements in prediction set sizes are accompanied by increases in coverage gaps, but the trade-offs are not analyzed in sufficient depth. Additionally, the definition and interpretation of the efficiency gain metric, a key contribution of the paper, could be more transparent and grounded in practical scenarios.

Overall, while the paper offers a promising theoretical framework and addresses an important problem, the current empirical evaluation does not fully substantiate its claims. Expanding the experimental analysis to include a broader range of datasets, comparisons with existing methods, and a deeper exploration of the trade-offs would significantly strengthen the paper’s impact and clarity.
Decision: Accept (poster)