Conformalized Decision Risk Assessment
Authors: Wenbin Zhou
Agni Orfanoudaki
Shixiang Zhu
Venue: ICLR 2026 Poster (2026)
Keywords: Conformal prediction
inverse optimization
risk assessment
decision making under uncertainty
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: A framework for quantifying the probability that a candidate decision is suboptimal in an optimization setting

Abstract: High-stakes decisions in healthcare, energy, and public policy have long depended on human expertise and heuristics, but are now increasingly supported by predictive and optimization-based tools. A prevailing paradigm in operations research is predict-then-optimize, where predictive models estimate uncertain inputs and optimization models recommend decisions. However, such approaches often sideline human judgment, creating a disconnect between algorithmic outputs and expert intuition that undermines trust and adoption in practice.
To bridge this gap, we propose CREDO, a framework that, for any candidate decision proposed by human experts, provides a distribution-free upper bound on the probability of suboptimality---informed by both the optimization structure and the data distribution. By combining inverse optimization geometry with conformal generative prediction, CREDO delivers statistically rigorous yet practically interpretable risk certificates. This framework allows human decision-makers to audit and validate their decisions under uncertainty, strengthening the alignment between algorithmic tools and human intuition.

Published as a conference paper at ICLR 2026
CONFORMALIZED DECISION RISK ASSESSMENT
Wenbin Zhou', Agni Orfanoudaki & Shixiang Zhu' CCamegie Mellon University ?University of Oxford (wenbinz2, shixianz @andrew. cmu. edu ogri..oraaouuakieee ox. ac.uk
ABSTRACT
High-stakes decisions in healthcare, energy, and public policy have long depended on human expertise and heuristics, but are now increasingly supported by predic- tive and optimization- based tools. A prevailing paradigm in operations research is predict-then teapiittthen-ppimize, where predictive models estimate uncertain inputs and optimization models recommend decisions. However, such approaches often side- line human judgment, creating disconnect between algorithmic outputs and expert intuition that undermines trust and adoption in practice. To bridge this gap, propose CREDO, framework that, for any candidate decision proposed by human experts, provides a distribution- suboptimality informed by both the optimization structure and the data distri- bution. By combining inverse optimization geometry with conformal generative prediction, CREDO delivers statistically rigorous risk certificates. This framework allows human decision-makers to audit and validate their decisions under uncer- tainty, strengthening the alignment between algorithmic tools and human intuition.
upper bound on the probability
innfore
INTRODUCTION
Decision-making under uncertainty in domains with significant societal and economic consequences requires not only the identification of nominally optimal solutions, but also rigorous quantification of the probability that any candidate decision maintains optimality when uncertain parameters
are realized (Zhu et al. 2022). The prevailing predict-heen-opiimizee (PTO) paradigm addresses uncertainty by first estimating unknown parameters through predictive models (e.g., future demand or patient outcomes), then solving an optimization problem to recommend actions (Bertsimas & Kallus 2020 Elmachtoub Grigas 2022). This pipeline has become the foundation many data-driven decision systems across wide spectrum of applications (Bertsimas al. 202 Tian et al. 2023).
Despite its ubiquity, the PTO approach has two fundamental shortcomings high-stakes contexts. First, these systems often function as black boxes that preseribe decisions without revealing their robustness or sensitivity to uncertainty. As a result, they do not provide any transparency as to whether alternative decisions might perform comparably better under different parameter realiza- tions. This opacity makes it difficult for human decision-makers to gauge confidence algorithmic recommendations or determine when override them with their own expertise (Li & Zhu 2024] Zhang et al. 2025). ). Second, such pipelines are inherently limited to point predictions that cannot capture distributional complexity. For example, under multi-modal parameter distributions, these methods recommend decisions optimized for expected values that fall between modes, where the true parameters may have negligible probability (Sim al., 2024). In practice, this means PTO may not only fail to guide human judgment effectively but also recommend harmful misleading actions.
These shortcomings are especially problematic given that high-stakes real-world decision-making rarely relies solely on algorithmic prescriptions. Experienced practitioners often propose alternatives based on domain knowledge that extends beyond available data, such as insights about rare events, operational constraints, risk factors not captured in historical records. Yet, current optimization frameworks offer no principled way to evaluate these generatdd decisions, creating a disconnect between algorithmic tools and practitioner expertise. Decision-makers need methods to rigorously assess any candidate solution, enabling them to compare algorithmic prescriptions with alternatives derived from human judgment.

---

Published as a conference paper at ICLR 2026
To bridge this gap, propose the complementary decide-then-assess paradigm. Rather than replacing human expertise with model prescriptions, our goal is to support human judgment by auditing candidate decisions in data driven manner. Specifically, we ask: For user-specified decision, how likely is to remain optimal under the true, unknown realization of uncertainty? This perspective enables rigorous risk assessment for any candidate decision, regardless of whether it. originates from optimization algorithms, expert judgment, or external constraints.
Building on this principle, we introduce CREDO-Cooformmaized Risk Estimation for Decision Optimization-a framework that quantifies, for any candidate decision, a distribution-free upper bound on the probability of suboptimality (Figure IT. Unlike scenario-based plamning, which evaluates decisions against discrete set of possible futures, CREDO quantifies optimality probability over the entire parameter distribution with rigorous statistical guarantees. The framework rests on two key insights: (i) in broad class of optimization problems, the optimal solution is deterministic function of the objective parameters (Chan al., 2025), which allows us to invert this mapping and characterize the set of outcomes under which decision remains optimal; and (ii) by combining conformal prediction (Shafer & Vovk 2008) with generative modeling, we can estimate the probability mass of this set, yielding valid, data-driven upper bounds decision risk. The resulting tool provides informative and computationally efficient risk certificates, empowering human decision- makers to audit and validate their choices under uncertainty, while systematically exposing discrepancies between human intuition and empirical data.
Our contributions can be summarized as follows:
Problem Formulation: We formalize decision risk as- sessment as computing distribution-free bounds the probability that a candidate decision remains optimal under parameter uncertainty, establishing its connection to inverse optimization and conformal prediction.
Random outcome Y e-g.
c.g. Profit maximization
Decision rule
y1s 32.
Assess the risk of candidate decision
egg, Investment plans
CREDO Framework: We propose CREDO, distribution- free framework that upper bounds the probability that a given decision is suboptimal. We further develop an efficient closed-form estimator for linear programs that enables practical implementation in real-world decision contexts and establish marginal conservativeness guar- antees on our risk estimates, characterizing the accuracy and variance properties of our estimator.
21,
Given the data, how likely is optimal?
Illustrative example of the problem setting of CREDO. For given candidate decision and relevant data, CREDO estimates the risk of the likeli- hood for the decision to be optimal.
Figure 1:
Empirical Validation: Through controlled synthetic ex- periments and real-world power grid planning appli- cation, we verify CREDO's theoretical guarantees and demonstrate its practical advantages in risk assessment accuracy, highlighting its calibration, efficiency, and ability to distinguish between robust and fragile decisions under uncertainty.
Decision makinng under uncertainty is a fundamental challenge in operations research and machine learning. A large body of work addresses this through robust optimization (RO), which ensures decisions perform well under worst-case realizations within an uncertainty
Related Works
(Bertsimas & Thiele 2006). Distributionally robust optimization (DRO) generalizes this by
accounting for ambiguity in the underlying distribution (Delage & Ye 2010 Duchi & Namkoong
202T
Levy et al. 2020 Rahimian & Mehrotra 2022) More recently, the PTO paradigm has
gained traction as practical framework for data- dataedriven
moisiinn-mking (Bertsimas & Kallus 2020
Elmachtoub et al.
2020
Lepenioti et al.) 2020). It involves two-stage procedure: first predicting
unknown parameters via machine learning, then solving deterministic optimization problem using these estimates. This has inspired decision-focuseeee learning (DFL.) approaches, where model training directly targets decision quality by differentiating through the optimization layer (Amos & Kolter 2017 Chen et al. 22025: Mandi et al.) 2024); (Shah et al., 2022. Wang et al., 22025) Wilder et al. 2019) While these frameworks may account for risks through their modeling assumptions, they do not explicitly quantify the level of risk associated with each decision. Our work addresses this gap directly.
Methodologically, our work builds upon the literature on conformal prediction (CP), a general statis- tical framework that enables distribution-free uncertainty quantification by constructing calibrated
2

---

Published as a conference paper at ICLR 2026
Probubility distribution
Deciston space
Output space
(0, 1)
Decision risk
assessmentt
(1, 0)
Fensible region (())
(1.0)
[0. 1)
(0,0) 0)
(x)
Decision
Y1
(0.0)
(0,
(1,0)
Inverse feasible region
Conformulized rish estimation
Figure 2:
The overall architecture of the proposed framework. It contains two main steps: (i) Map
the given candidate decision (e.g. (0.0)) to its inverse feasible region æ (z); (6i) Conservatively assess the risk via conformalized risk estimation over (2) using collected data of X and Y.
prediction sets under mild assumptions (Papadopoulos et al. 2002; Shafer Vovk 2008; Vovk et al.
2005). Two strands of recent research are particularly relevant to approach. First, growing body of work explores replacing traditional point prediction models in CP with generative models, enabling better modeling of stochastic outputs and yielding tighter prediction sets (Wang al. 2023 Zheng & Vhu 2024 Zhou et al., 2024). Our methodology adopts this generative modeling perspective key component improve the accuracy of risk estimates. A parallel line of research also investigates the "inverted" use of conformal prediction, where the goal is to estimate the miscoverage rate corresponding fixed prediction set (Prinster al. 2022 2023 Singh al. 2024 Gauthier
et al. 2025a). The technique we use is most similar to the concurrent work of (Gauthier et al. 2025a).
leveraging recent advancements e-value conformal prediction (Vovk 2025 Balinsky Balinsky
2024 Gauthier et al., 2025b). However, our framework departs from their work by focusing on the
decision risk assessment setting rather than being pure conformal prediction task.
Integration of CP and optimization has also been explored in recent works for decision-making, including RO (Kiyani et al.) [2025] | Andrews & Chen [225 Patel et al. [2024] CCenreddy & Delage 2024; Lin et al. [2024] Chan al. [2024). DFL (Cortes-Gomez et al., (2024). and human-related contexts (Hullman et al. (025).. Specifically, Kiyani et al. (2025) shows that prediction sets (e.g.,
via conformal prediction) naturally define robust optimization uncertainty sets whose miscoverage level encodes risk tolerance under VaR, Andrews & Chen (2025) demonstrate that ambiguity-averr decisions can be framed minimax choices certificates. In comparison, our work differs in both role and objective: instead of using conformal sets as inputs produce optimal decisions, invert conformal prediction quantify how likely a given decision is optimal, making our framework descriptive (auditing decisions) rather than prescriptive (recommending them), and thus complementary these prior approaches.
confidence sets, yielding high-probability loss
Finally, our study contributes to a growing body of research on human-Al interaction in decision- making. Recent work has investigated how machine learning algorithms can serve as advisory tools to support human decisions (Chen et al. Orfanoudaki et al. 2022). A parallel line of research has explored how generative models can be leveraged to present more diverse set of decisions, thereby encouraging exploration and aiding the development improved decision strategies (Ajay et al. 2022) Krishnamoorthy et al. 2023 Li & Zhu 2024). Our work advances further in quantilying the risk associated with each candidate decision, enriching the informational quality of decision support part--paattllllyyy when multiple dimensions of decision quality must be traded off (Li & Zhu 2024
& Pauphilet 2024 Hullman et al. 2025
Masin & Bukchin 2008).
2 PROBLEM SETUP
Let E denote observed covariates, and let E y be random outcome variable represent- ing uncertain objective parameters. The decision-making task is formalized as solving a general constrained optimization problem, whose solution set is given by:
(Y: arg min 92, Y,0) C Z(0).
(1)

---

Published as a conference paper at ICLR 2026
Here, 9 denotes known parameters of the objective function g and the feasible region 2(0), with the latter assumed to be a compact subset of 2. Our goal is to develop a distribution-free data-driven method for estimating the probability that prescribed decision is suboptimal. Specifically, given candidate decision 2, we aim to develop a risk measure a(2) that satisfies:
f E (Y) 2 1 1--alp a()), E,
where omit the dependency of a(z) x for notation simplicity. The randomness the left-hand side is induced by the o); which is dependent on the random variable Y. The objective in equation2lis well-defined for any optimization problem and candidate decision satisfying P( E <Y;; 8)} O. Otherwise, one can take 6(2) = 1 as the trivial solution
3 CREDO: CONFORMALIZED DECISION RISK ASSESSMENT
We propose a framework for quantifying decision risk a(z) ) by casting this problem as a structured uncertainty quantification task over the decision space, as illustrated by Figure 2 The key idea is to project the decision's 's optimality condition to an inverse feasible region (Chan et al., 2025 Tavashoglu et al. 2018) in the outcome space, and then construct inner geometric approximations of this region using conformal calibrated regions (Shafer & Vovk 2008 Wang et al., 2023). where their miscovcrage rates will be finally used as the risk estimates. We detail this procedure in two steps.
For any given realization of Y, decision 2 is optimal and only if it achieves the smallest objective value among all feasible decisions:
Step I: Reformulation with Inverse Feasible Region
2 E (y:
g(2, y: S g(2' g): V2' E Z(0).
Therefore, the inverse feasible region, which is defined as the set of outcomes for which is an optimal decision, can be defined as
(2;
n Eye y g(2 0) s g(z', 8))
2'E2(0)
The definition equation Jlllow the objective to be reformulated as summarized in Proposition D Proposition (Reformulation). Let (2; 0) be defined in equation then the objective defined in equation 2 can be equivalently expressed as:
E 1 0)) = P
(4)
The reformulated objective in equation 4 separates the random variable Y from the mapping т. As a result, the original problem reduces to a standardized uncertainty quantification task, where the goal is to estimate the probability that Y lies within π (2; 0).
Step 2: Risk Estimation via Generative Conformal Prediction To estimate the reformulated objective in equation 4 we propose to do so by constructing calibrated inner approximation sets of π (2; Specifically, we require two core conditions to be satisfied by the constructed set: (a) It is fully contained within π (2; f); ;; (b) Its coverage probability of Y is known or can be quantified. This allows us to use this constructed set as a surrogate to bound the reformulated objective by:
(a) 2 P(Y E c(X;a)) 2 1
(b)
(5)
Here, we denote a( as the constructed set given input X, and a is the coverage probability, which should also serve as an estimate for (x()) the objective defined equation? 2
Additionally, the estimate a should not be overly conservative. This requires constructing approx- imation sets that not only fully exploit the space within π (2; 0) but also concentrate around the high-density regions of Y. ensuring that both inequalities (a) and (b) in equation are tight enough. A possible idea is to employ generative models as base predictors to capture the underlying distribution,
Another option is to generalize the objective so that the near optimal candidate decisions are also considered solution. This can he achieved by relaxing the optimal objective value by prespecified small margin, see Andrews & Chen (2055: Kiyani al. 20255
4

---

Published as a conference paper at ICLR 2026
Point prediction model
Generative model
Baseline model
K
(x)
x (2)
g
20%
60%
g/3
Estimated confidence: 20%
0% 60%) 20%
(0% + 0% 60% + 40%) 25%6
Figure 3: Visualization of decision risk estimation under different modeling choices. The figure compares point prediction model with generative models (K = 3 and K = 4) as the base prediction model, respectively. The black lines denote the boundary of the inverse feasible region æ (=). The blue shade represents the true conditional distribution Y X, and the black dots indicate generated predictions ((*). The green ball indicates the conformalized sets ((R) (x; 6(2)).
and then generate multiple and diverse conformalized sets as the approximation sets, which should empirically satisfy both conditions when the generative models are well-trained. We refer to this as the generative conformal prediction procedure, and is formally detailed as follows.
We begin with training (conditional) generative model f X -> on a training dataset to approximate the conditional distribution of Y|X and allow samples to be drawn from it. For test 1) and construct the conformalized as lg ball
input + we draw prediction In+1 fxx
centered at 1
C(E+11a)) = EyEVill - in+1lla < Ma)} if else 0,
(6)
where R(a) denotes the conformalized radius, which is calibrated using dataset {(,,, w)) w)) to guarantee that C ())+1;a) achieves valid (1 - a) coverage of the distribution of Y. For instance, one possible configuration of R(a)) is:
R(a) E.= 00: y:112
ann 1
The coverage level a is determined by solving the following optimization problem, which finds the smallest a such that the conformalized set entirely contained within the inverse feasible region:
á(=) aε(1/(m-11,1 1).1]
min
{a C(z.+1:a) S (z;00)
(8)
The procedure is repeated K times to obtain the collection of estimates (*) (=)] which then averaged to yield the final estimator:
- (0)0((())))) + /K.
The full pseudocode of the algorithm is provided in Appendix A
Remark 1. We further clarify the role of generative models by emphasizing their advantages over point prediction models (c.g.. regressors). As shown in Figure 3 point prediction models can produce outputs that lie near the boundary of. or even outside, T2; 0), leading to overly conservative risk estimates (i.e., risk equal to one). In contrast, generative models allow multiple draws, and increasing raises the chance that at least one prediction falls within the inverse feasible region, resulting more accurate (less than one) risk estimate. This idea will later be formalized through the notion of the true positive rate in our theoretical analysis.
4 THEORETICAL ANALYSIS
This section presents three key theoretical analyses of our method. First, we prove conservativeness, showing that the estimator provides valid upper bound of the decision risk. Second, provide
5

---

Published as a conference paper at ICLR 2026
unifying view the proposed risk estimator with Monte Carlo estimators. Finally, we show that the method supports hial-quality decision decision-making, with its true positive rate increasing as the number of generated predictions grows. In our analyses, set the conformalized radius equation 7
Our first theoretical result on conservativeness relies on the assumption that the calibration data are exchangeable. This requirement is milder than the i.i.d. assumption, as the latter implies the former, and is therefore commonly adopted in the conformal prediction literature (Angelopoulos & Bates 2021 Barber al. 2023 Papadopoulos et al. 2002).
m++1 is exchangeable if. for any permutation Assumption 1 (Exchangeabillty). A dataset {( yo) y)) of the index set (I...., 1} and all measurable sets A E (X' x D)a+1, there
P ((11, 31). (In+1, Mn+1) E AA = P (((((())))))) Bo(1). (Z (p(++))) E A}.
Under this assumption, we can derive Theorem I which states that our estimator yields a valid upper bound on the decision risk in expectation. Notably, this result does not require any assumption on how well the generative model captures the underlying distribution, which underscores the robustness of algorithm against potential model misspecification, as well the trustworthiness of the algorithm for applications where conservativeness critical.
Theorem 1 (Conservatism), Under Assumption
7 the estimator a(z) defined in equation 9 satisfies: P piz a(Yn+1: 0)} 1- - E[ā(2)].
Theorem T shows that the proposed estimator á is, in expectation, a conservative estimate of the decision risk. The success of the proof relies the post-hoe validity property of e-value conformal prediction (Vovk) 2025 [alinsky&& & Balinsky, [024] Gauthier et al.] (2025b). which motivates our design of the conformalized radius in equation 7 The detailed proof, along with discussion of the first-order Taylor approximation procedure used therein, is provided in Appendix B
Next, highlight an important theoretical insight: the proposed estimator be interpreted as weighted Monte Carlo (MC) probability estimator based on generated predictions.
Proposition 2. The estimator in equation 9
can be re-written as:
W (H, 1 ( Dati Ex=1
=(z)=1-
(10)
Here, the conformalized weights ()(*) (2,1x)1) E [0, 1] are determined by our conformalization procedure, ensuring the conservatism guarantee established in Theorem [ The unifying view in Proposition 2 hints potential extensions: by setting the conformalized weights to 1, we get a more radical variant of the estimator with more accurate risk estimation but may fail the conservatism guarantee. We will examine this ablation variant in our subsequent synthetic experiments. The proof of Proposition 2 provided in Appendix C
Finally, we study the estimate's 's true positive rate (TPR), defined as the ratio of total decisions that are correctly estimated to have risks smaller than out of all decisions with risks smaller than one:
E [# (2 E 2 a<=) < and á(=) <1]
TPR
(11)
#(2 Z(0) a(2) 1}
where with some abuse of notation, we denote (2)) Ff2 & 5 =(Y++1:0) {)} as the actual risk. TPR indirectly reflects the discrepancy in final decisions led by different outputted risk estimates: assuming choose to rule out decisions z with á(=) = 1, then for those "false-posse poolsv--ssstve"
that decision-maken rule-outs, the maake would risk losing profits as they' ve omitted decisions that are possible to be optimal. Therefore, TPR scales positively with the quality the decision assessment.
Proposition 3 (True positive rate). TPR (equation
17
monotonically increases as K increases.
Proposition 3 specifically highlights the generative model's important role avoiding overconser- vatism, confirming the intuition built by Figure 3 when K is small (e.g., K 1), or when the generative model is replaced by a deterministic predictor, the estimator becomes more susceptible to such erroneous exclusions, potentially leading to a lower TPR. Therefore, the generative approach of drawing diverse predictions would help mitigate this issue and support high-quality decision making... These insights also tested in our numerical experiments. Its proof is detailed in Appendix D
Assuming K is large and the predictive model f(Y|X)) well-trained.

---

Published as a conference paper at ICLR 2026
5 EFFICIENT COMPUTATION FOR LINEAR PROGRAMS
Recall that the generic CREDO algorithm described in Section B makes no assumptions about the form of the objective function g(2, 0; or the feasible region ( enabling its application to general constrained optimization problems. In particular, under a linear programming setting, we can further show that the algorithm highly computationally efficient it admits closed-form solution, thereby eliminating the need for iterative or approximation-basee procedures.
Suppose v 2 C Rdd linear programming (LP) problem is defined as:
TLP (Y; =;; arg min (Y,z), ( = (= E b),
(12)
=EZ(0)
where A _ddm and E R are known parameters belonging to 0. As one of the most funndamental optimization structures, LPs arise naturally in wide range of real-world applications, including supply chain management, energy dispatch, and transportation (Charnes & Cooper 1957). Moreover, LPs are often employed to approximate more complex optimization problems, with many relaxations proving both useful and widely applied in practice (Lovász) 1975). The closed-form expression of our estimator in this setting follows directly as corollary to Equation (T0
Corollary 1. Suppose the oplimization problem is defined in equation then:
K Σ a(=) =1 K
1
(DD())
-E 1bb vill2
+
II VEV(O)
{10 100t112-00 -) s00
(13)
(n + 1)D())
where V(A) denotes the set of vertices of 2()), and D(K) is the distance of Dutt to the boundary of (2; with the expression (( min (Mat1; w)1/11=- u112-
In Corollary computing vertices of a polytope ((0 can be efficiently executed with well- established algorithms such as the double description method (uukuda & Prodon 1995 Motzkin al., 1953). Once the vertices are precomputed, the estimator a(z) can be computed via a single pass over equation [3 entirely avoiding set enumerations and numerical optimizations. This renders it highly efficient for large-scale LP problems, with a computational complexity of O(K n (V01) that is independent of the number of iterations/epochs. Moreover, one prove that when V(0). the indicator term turns out to be zero (Tavashoglu et al. 2018). As a result, the algorithm can be further accelerated by automatically outputting risk one if it detects that the input decision is not on the vertex of the feasible region, making our proposed algorithm computationally efficient and implenmenaaiion-rriendde The proof of Corollary is provided in Appendix C
We emphasize that although closed-form characterizations are available only for LPs, the algorithm described in Section 3 more broadly computationally feasible general convex optimization
problems. In particular, following Proposition 2 one can construct the risk estimator by: (i) solving
forward optimization problem determine whether e((4)] lies in the inverse feasible region associated with decision z, and (iv) computing the conformalized weight (2) In+1) as defined in Appendix C
6 EXPERIMENTS
In this section, we evaluate CREDO on both synthetic and real-world optimization problems. We show that: (i) The proposed risk estimator is empirically conservative, adding to Theorem T (it) The generative approach improves the accuracy of CREDO in risk estimates, adding Proposition 3 (Mi)
How the key parameters, such as the number of generated samples K" and variance scales a, impact the performance of CREDO; (iv) CREDO produces high quality decisions under various settings.
Our experiment settings include: ()) Two synthetic settings that are referred to as Setting I and Setting II, illustrated in Figure Setting features triangular feasible region with three vertices and can be interpreted as a basic profit-maximizattte scenario with an octagonal feasible region comprising five vertices. (ii) A real-world infrastructure planning problem, where we study a budget-constraine substation upgrade problem. formulated as knapsack optimization, that aims minimize the expected overflow of solar panel installations
problem. In contrast, Setting II presents a more stylized
7

---

Published as a conference paper at ICLR 2026
Setting!
Serting II
Dociioon spece
Outocme spact
Dossion spass
CHECC SOUKS
01. 5g
010
0.4
012
-05
(v
-1.0
2.03
-25
Figure 4: Illustration of settings I and II. The gray region represents the feasible region in the decision space, and the cones to the right are the corresponding inverse feasible regions in the outcome space. The blue shade denotes the density mass of Y.
Conservativeness
TPR vx.
Aec vs.
Toee poodre 10le
Bdlao AKCERS
0,9
Pkir Modk
Setting
CREDO
90
0.00
0.0
Pole
100
10-1
30-2
10
(Misrrne comgooent) vetaree
1.0
CREDO
The proltre 33to
13.
Setting
0.65
044
0.4
Poire Mark!
0.0
0.0
Poae
20
100
Staniaation Jbpn NC
(Maator (omgreaaa) vaziasec
Figure 5: Selected results from the ablation comparison. Each column shows (from left to right): Conservativeness of different ablation models; True positive rate (TPR) versus generative sample size K: Relative accuracy versus variance scale a. Experiments are repeated across 20 independent trials. The default configuration is K 100 and a unless otherwise specified.
in a power distribution grid (Zhou et al. 2024). We use real data of solar panel installation records collected 2010-2024 from utility situated in Indiana, U.S. More details can be found in Appendix E
In our experiments, we specify the generative model as a three-component Gaussian Mixture model, fitted using the EM algorithm for 1 x 102 epochs. This modeling choice balances its capability of accurately capturing multi-modality in the data, without the need for a large amount of training data as required by deep learning architectures. The training and calibration data randomly and
equally split following the split conformal prediction framework (Papadopoulos et al. 2002). Across all experiments, we adopt the standard p-value conformalized radius (Singh et al. 2024) as the
conformalized radius R(a), which has been demonstrated by prior works show strong empirical accuracy and conservativeness properties. More details can be found in Appendix E
Conservativeness vs. Accuracy Tradeoff In the first set of experiments, we evaluate the properties of CREDO through controlled component analysis under the synthetic settings. We focus on testing for two hypotheses: it can consistently produce conservative risk estimates, and achieves superior accuracy, meaning that it can closely estimate the ground-true risk.
We adopt three metricy (i) Conservativeness rate: the percentage of trials where the risk estimate satisfies the conservativeness guarantee in equation for all feasible decisions; (ii) True positive rate (TPR): the percentage actions where the method correctly identifies them to have risks of less than one, averaged across trials (equation TT): (Hi) Relative accuracy (Acc): the negative normalized sum Note that the latter two both aim at assessing the accuracy.
8

---

Published as a conference paper at ICLR 2026
of absolute differences between the estimated and true risks across all decisions, averaged across trials.
We compare the performance of CREDO these metrics with two direct ablation variants: (i) Point: a variant that uses a point prediction (the conditional mean) instead of sampling from a full probabilistic model (Singh et al. 2024).
(it) NS: the naive sampling estimator that sets the conformalized weights to one in equation 10 taking a Monte Carlo probability estimator form.
We present some selected insightful results in Figure and and leave the complete results to Appendix F It can be observed from the first column that both CREDO and Point consistently achieve 100% conservative estimates, whereas NS attains only around 50%. These results align with Theorem highlighting the significance of the conformalized weighting term and its effectiveness in downscaling the estimate satisfy the conservativeness guarantee. Due to the deficiency of NS in conservativeness, which is the key property that we study for, it has been withdrawn from subsequent comparisons.
In the second column of Figure 5 we observe that as the sample size K increases, CREDO exhibits a significant increase in true positive rate. In contrast, the curve for Point remains flat. This indicates that, while maintaining a similar level of conservativeness, CREDO can identify a greater number of potentially suboptimal actions, whereas Pe i nt may overlook viable alternatives that the decksion-maker could consider. This serves as strong numerical evidence supporting Proposition 3
In the third column of Figure 5 we observe that in Setting I, as the data variance scale increases, the performance of CREDO crosses over and eventually surpasses that of Point. In Setting II, CREDO consistently achieves higher relative accuracy across all variance levels. These results suggest that CREDO is particularly effective when the outcome variable is highly stochastic and challenging to approximate with point estimators. This further validates the underlying motivation of incorporating generative components instead of point-prediction models.
Decision Quality Evaluation In the second experiment, we assess decision quality based on risk estimates produced by CREDO across both synthetic and real-world settings. We show that CREDO can be used to select decisions with consistently higher confidence, demonstrating the effectiveness of CREDO's risk estimates in guiding practical decision-making.
We adopt empirical confidence ranking as our primary evaluation metric: given decision policy π and test dataset {z, y:) i-1. we apply π to each input æ; to generate predicted decisions )m We then compute the score Σi= h(zz), where h maps each prediction to a discrete rank based on its frequency among the ground-truth optimal decisions {ar m in the test set. This metric is l==1 designed to capture method's 's tendency to select decisions that are most likely to be optimal. It is centacc-indeeendeett and would discourage decisions that are rare the optimal set of actions the test set, which is consistent with what a decision-make would desire for prescribed decision to achieve when operating under ex-ante uncertainty. Furthermore, the metric is unit-independent and robust to extreme outliers, which renders it more appropriate for one-shot decision- makinn scheme considered our paper, compared to the standard notions such as regret.
We define the decision policy for CREDO selecting the action with the lowest estimated risk. It is compared with four decision-making baselines: predict-then-optimizz (PTC) (Bertsimas & Kallus 2020), robust optimization (RO) (Bertsimas & Thiele (0066), smart PTO (SPO+) (Elmachtoub & Grigas 2022), and reciision-fousser learning (DFL.) (Amos & Kolter 2017). These baselines are
chosen for their popularity and their alignment with the prescriptive, risk-averse decision-making paradigm underlying our decision risk assessment setup.
Table 1: Evaluated empirical confidence ranking (L) for different methods across three datasets.
Setting
Setting II
Real Data
Method
0 0.1
0 0.1
10
PTO
1.00
:: 0.00
2.76
+ 0.59
3.55
1 0.50
3.36 + 0.48
2.04 + 1.65
1.75 4: 1.69
RD
1.00
1:00+0.00 0.00
2.98
+ 0.14
3.00 + 0.00
4.99
0.10
6.00
0.00
3.98
0.80
3.00 + 1.29
SPO+
1.00
0.000
2.68
0.65
2.02 0.82
3.95
1.20
4.67
1.56
3.56
1.50
2.67
1.43
DFL
0.64
1.83
2.79
3.60
+ 1.52
3.96
2.07
3.66
1.92
1.04
CREDO
0.75
1.61 -
1.48 0.52
1.05 = 0.22
1.00 +: 0.00
2.03 + 0.96
1.75 0.92

---

Published as a conference paper at ICLR 2026
Table I presents the comparison results. It can be that CREDO achieves the smallest ranking metric value across most datasets, on average selecting the top two most likely decisions across all datasets. Though it might seem concerning that the in Setting I (ar 0.1), PTO, RO, and SPO+ all achieve better performance than CREDO, this is because when a is small, the data becomes highly concentrated around the mean, rendering the problem nearly deterministic and can be best dealt with point-prediction basclines. This also mirrors the behavior observed in the Accuracy vs. plot for Setting in Figure These results highlight the capability of our method for high-quality decision-make makinn, especially under highly uncertain environments.
CONCLUSION
We proposed CREDO, distribution-fie framework for decision risk assessment that combines inverse optimization with conformal prediction to estimate the probability that a candidate decision is suboptimal. Our method provides statistically valid risk certificates by characterizing inverse optimality regions and using generative models to construct calibrated inner approximations, Em- pirical results on synthetic and real-world datasets demonstrate that CREDO offers conservative yet informative risk estimates, enabling more robust and interpretable decision support under uncertainty,
From practical standpoint, the conformalized radius in CREDO is restricted the e-value variant discussed in this work. The e-value approach offers rigorous post-hoc validity guarantees, but alternatives such as the p-value variant provide tighter risk estimates the cost of weakened validity. Selecting appropriate radius thus requires balancing the trade-off between validity and informativeness, ensuring that decision support remains both safe and actionable.
We also highlight potential concern regarding selection bias. When the risk estimate produced by CREDO is used select decision, re-evaluating the risk of that selected decision may invalidate the original guarantee. This arises because our current framework implicitly assumes that conditioning the calibration data. Once this assumption
on the input decision doesn't 't violate the exchangeability broken, the theoretical validity of the risk estimate can longer be ensured. A simple mitigation strategy is to use data splitting, separating the data used for decision selection from that used for risk evaluation. Additionally, we encourage future work address this selection bias without relying sample splitting, potentially by developing principled relaxations or adjustments of the risk guarantees that explicitly account for data-dependent decision selection.
Ethics Statement This work does not involve human subjects, sensitive personal data, or appli- cations that pose direct risks of harm. All datasets used in this paper, along with anonymized codebase, are provided in the supplementary material, with any potentially sensitive information removed from the real data. We ensured that our methods were applied in way that does not introduce or amplify unfair bias, discrimination, or privacy risks. The research was conducted in accordance with the ICLR Code of Ethics.
Reproducibility Statement We have taken steps to ensure the reproducibility of our results. Exper- imental details, including dataset descriptions, preprocessing, model architectures, hyperparameters. and evaluation metrics, are documented in the main text and the appendix. We have included clear explanations of any assumptions and complete proofs of the theoretical results in both the main paper and the appendix. An anonymized codebase with implementation and notebooks for reproducing our experiments is included in the supplementary material.
REFERENCES
Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. Is con- ditional generative modeling all you need for decinion arXiv preprint arXiv:2211.1565) 2022.
Brandon Amos and J Zico Kolter. Optnet: Differentiable optimization as layer in neural networks. In International conference on machine learning, pp. 136-145. PMLR, 2017.
Isaiah Andrews and Jiafeng Chen. Certified decisions. arXiv preprint 2rXiv:2502.178.. 2025.
Anastasios N Angelopoulos and Stephen Bates. A gentle introduction to conformal prediction and distribution-fres uncertainty quantification. arXiv preprint arXiv:2107.075// 2021.
10

---

Published as a conference paper at ICLR 2026
Alexander A Balinsky and Alexander David Balinsky. Enhancing conformal prediction using e-test statistics. In The 13th Symposium on Conformal and Probabilistic Prediction with Applications, pp. 65-72. PMLR, 2024.
Rina Foygel Barber, Emmanuel Candes, Aaditya Ramdas, and Ryan Tibshirami. Conformal prediction beyond exchangeability. The Annals of Statistics, 51(2):816-88) 845, 2023.
Dimitris Bertsimas and Nathan Kallus. From predictive to prescriptive analytics. Management Science, 06(3):1025-10444 2020.
Dimitris Bertsimas and Aurélie Thiele. Robust and data-driven optimization: modern decision making under uncertainty. In Models, methods, and applications for innovative decision making, pp. 122. INFORMS, 2006.
Dimitris Bertsimas, Leonard Boussioux, Ryan Cory-Wright Arthur Delarue, Vassilis Digalakis, Alexandre Jacquillat, Driss Lahlou Kitane, Galit Lukin, Michael Li, Luca Mingardi, et al. From predictions prescriptions: A data-driven response to covid-19. Health care management science, 24:253-272, 2021.
Timothy Chan, Erick Delage, and Bo Lin. Conformal inverse optimization for adherence-aware prescriptive analyties. Available at SSRN, 2024.
Timothy CY Chan, Rafid Mahmood, and Ian Yihang Zhu. Inverse optimization: Theory and applications. Operations Research, 73(2):1046-1044 2025.
Abraham Charnes and William Wager Cooper. Management models and industrial applications of linear programming. Management science, 4(1):38 91, 1957.
Guanting Chen, Xiaocheng Li, Chunlin Sun, and Hanzhao Wang. Learning make adherence-aware advice. In The Twelfth International Conference on Learning Representations.
Shuyi Chen, Ferdinando Fioretto, Feng Qiu, and Shixiang Zhu. Global-becisson-foouueed neural odes for proactive grid resilience management. arXiv preprint arX1v:2502.18332, 2025.
Abhilash Reddy Chenreddy and Erick Delage. End-to-end conditional robust optimization. In Uncertainty in Artificial Intelligence, pp. 736-748. PMLR, 2024.
Santiago Cortes-Gomez, Carlos Patiño, Yewon Byun, Steven Wu. Eric Horvitz, and Bryan Wilder. Decision-focused uncertainty quantification. arXiv preprint arXiv:2410.01767 2024.
Erick Delage and Yinyu Ye. Distributionally robust optimization under moment uncertainty with application to data-driven problems. Operations research, 58(3):595-612, 2010.
John Duchi and Hongseok Namkoong. Learning models with uniform performance via distribu- tionally robust optimization. The Annals of Statistics, 49(3):137--1400) 2021.
Adam N Elmachtoub and Paul Grigas. Smart "predict, then optimize". Management Science, 68(1): 9-26, 2022.
Adam N Elmachtoub, Jason Cheuk Nam Liang, and Ryan McNellis. Decision trees for decision- making under the predict-thennoptimize framework. In International conference on machine learning, pp. 2858-2867. PMLR, 2020.
Komei Fukuda. cdd/cdd+ reference manual. Institute for Operations Research, ETH-Zentrum, pp- 91-111, 1997.
Komei Fukuda and Alain Prodon. Double description method revisited. In Franco-Japanese and Franco-Chinese conference on combinatorics and computer science, pp. 91-111. Springer, 1995.
Etienne Gauthier, Francis Bach, and Michael Jordan. Backward conformal prediction. arXiv preprint arXiv:2505. 13732, 2025a.
Etienne Gauthier, Francis Bach, and Michael I Jordan. E-values expand the scope of conformal prediction. arXiv preprint MXXx:25033330500 2025b.

---

Published as a conference paper at ICLR 2026
Julien Crand-Clément and Jean Pauphilet. The best decisions are not the best advice: Making adherence- aware recommendations. Management Science, 2024.
Jessica Hullman, Yifan Wu, Dawei Xie, Ziyang Guo, and Andrew Gelman. Conformal prediction and human decision making. arXiv preprint arXiv:2503. 17709, 2025.
Shayan Kiyani, George Pappas, Aaron Roth, and Hamed Hassani. Decision theoretic foundations for conformal prediction: Optimal uncertainty quantification for risk-averse agents. arXiv preprint arXiv:2502.02561, 2025.
Mark Krein and David Milman. On extreme points of regular convex sets. Studia Mathematica, 9: 133-138, 1940.
Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Diffusion models for black- box optimization. In International Conference on Machine Learning, pp. 17842-17857. PMLR, 2023.
Katerina Lepenioti, Alexandros Bousdekis, Dimitris Apostolou, and Gregoris Mentzas. Prescrip- tive analyties: Literature review and research challenges. International Journal of Information Management, 50:57-70, 2020.
Daniel Levy. Yair Carmon, John C Duchi, and Aaron Sidford. Large-scale methods for distributionally robust optimization. Advances in Neural Information Processing Systems, 33:8847-8860, 2020.
Michael Lingzhi Li and Shixiang Zhu. Balancing optimality and diversity: Human-centered decision making through generative curation. arXiv preprint 1xXx:22409.1153: 2024.
Bo Lin, Erick Delage, and Timothy Chan. Conformal inverse optimization. Advances in Neural Information Processing Systems, 37:63534-63564 2024.
László Lovász. On the ratio of optimal integral and fractional covers. Discrete mathematics, 13(4): 383-390, 1975.
Jayanta Mandi, James Kotary, Senne Berden, Maxime Mulamba, Victor Bucarey, Tias Guns, and Ferdinando Fioretto. Decision-focused learning: Foundations, state of the art, benchmark and future opportunities. Journal of Artificial Intelligence Research, 80: 10:1623-1701, 2024.
Michael Masin and Yossi Bukchin. Diversity maximization approach for multiobjective optimization. Operations Research, 56(2):411 1-44,, 2008
Theodore S Motzkin, Howard Raiffa, Gerald L Thompson, and Robert M Thrall. The double description method. Contributions the Theory of Games, 2(28):51-73, 1953.
Agni Orfanoudaki, Soroush Saghafian, Karen Song, Harini A Chakkera, and Curtiss Cook. Algorithm, human, the centaur: How to enhance clinical care? 2022.
Harris Papadopoulos, Kostas Proedrou, Volodya Vovk, and Alex Gammerman. Inductive confidence machines for regression. In Machine learning: ECML 2002: 13th European conference machine learning Helsinki, Finland, August 19-23, 2002 proceedings 13, pp. 345-356. Springer, 2002.
Yash P Patel, Sahana Rayan, and Ambuj Tewari. Conformal contextual robust optimization. In International Conference on Artificial Intelligence and Statistics, pp. 2485-2443 2493.. PMLR, 2024.
Fabian Pedregosa, Gaël Varoquaux. Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, al. Scikit-learm Machine learning in python. the Journal of machine Learning research, 12:2825-2830, 2011 1.
Drew Prinster, Anqi Liu, and Suchi Saria. Jaws: Auditing predictive uncertainty under covariate shift. Advances in Neural Information Processing Systems, 35:35907-35920, 2022.
Drew Prinster, Suchi Saria, and Angi Liu. Jaws-x: addressing efficiency bottlenecks of conformal prediction under standard and feedback covariate shift. In International Conference on Machine Learning, pp. 28167-28199 28190. PMLR, 2023.
12

---

Published as a conference paper at ICLR 2026
Hamed Rahimian and Sanjay Mehrotra. Frameworks and results in distributionally robust optimiza- tion. Open Journal of Mathematical Optimization, 3:1-85, 2022.
Glenn Shafer and Vladimir Vovk. A tutorial on conformal prediction. Journal of Machine Learning Research, 9(3), 2008
Sanket Shah, Kai Wang, Bryan Wilder, Andrew Perrault, and Milind Tambe. Decision-focusee learning without Decision-making: Learning locally optimized decision losses. Advances in Neural Information Processing Systems, 35:1320-1333 2022.
Melvyn Sim, Qinshen Tang, Minglong Zhou, and Taozeng Zhu. The analytics of robust satisficing: predict, optimize, satisfice, then fortify. Operations Research, 2024.
Sukrita Singh, Neeraj Sama, Yuanyuan Li, Yang Lin, Agni Orfanoudaki, and Michael Berger. Distribution-free risk assessment of regression- based machine learning algorithms. In The 13th Symposium on Conformal and Probabilistic Prediction with Applications. pp. 44-64. PMLR, 2024.
Onur Tavashooglu, Taewoo Lee, Silviya Valeva, and Andrew Schaefer. On the structure of the inverse-feasible region of a linear program. Operations Research Letters, 46(1):614-11555 2018.
Xuecheng Tian, Ran Yan, Yannick Liu, and Shuaian Wang. A smart predict-then-optimzze method for targeted and cost-effective maritime transportation. Transportation Research Part B: Method- ological, 172:32-55 52, 2023.
Vladimir Vovk. Conformal e-prediction. Pattern Recognition, pp. 111674, 2025.
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning random world, volume 29, Springer, 2005.
Prince Zizhuang Wang, Jinhao Liang, Shuyi Chen, Ferdinando Fioretto, and Shixiang Zhu. Gen-dfl: Decision- Decision-focused focused generative learning for robust decision making. arXiv preprint arXiv:2502.05468, 2025.
Zhendong Wang, Ruijiang Gao. Mingzhang Yin, Mingyuan Zhou, and David Blei. Probabilistic conformal prediction using conditional random samples. In International Conference on Artificial Intelligence and Statistics, pp. 8814-8836. PMLR, 2023.
Bryan Wilder, Bistra Dilkina, and Milind Tambe. Melding the data-decisions pipeline: Decision- focused learning for combinatorial optimization. In Proceedingy of the AAAI Conference on Artificial Intelligence, volume 33, pp. 1658-1665, 2019.
Zhiqiang Zhang, Pengyi Shi, and Amy Ward. Admission decisions under imperfect classification: An application in criminal justice. Available at SSRN 5214197, 2025.
Minxing Zheng and Shixiang Zhu. Generative conformal prediction with vectorized non-conformity scores. arXiv preprint arXiv:2410.13735. 2024.
Wenbin Zhou, Shixiang Zhu, Feng Qiu, and Xuan Wu. Hierarchical spatio-temporal uncertainty quantification for distributed energy adoption. arXiv preprint arXiv:2411.12193, 2024.
Shixiang Zhu, He Wang, and Yao Xie. Data-driven optimization for atlanta police-zone design. INFORMS Journal Applied Analytics, 52(5):412 4-22, 2022.
13

---

Published as a conference paper at ICLR 2026
APPENDIX OVERVIEW
In the Appendix, we provide comprehensive supporting material for the main manuscript. We include a detailed description of the algorithm pseudo code (Appendix A), detailed proof and derivation for all theoretical results (Appendices) D) detailed experiment setups Appendix and additional
experiment results Appendix F
Remark Throughou the theoretical derivations, we introduce slightly modified versions of some definitions of the algorithmic components in Section 3 In particular, redefine the conformalized set (equation conformalized radius (equation and estimated risk (equation respectivly as
C(En+1;0) = y E y 11 1-0+t1l22 < R(al].
to0
ifa [0, -
R((a) = Ei Mill a(n+l)-1
ifa E I+ 1),
if =
a(2) = min {a C(E++1:a) S a (2;0)} ac|0.1]
Compared to the original definitions, these modified versions more convenient for mathematical derivations, though slightly less interpretable. It is straightfrrwal to verify that adopting these modifications does not affect the CREDO algorithm its associated theoretical guarantees.
A ADDITIONAL ALGORITHM DETAILS
The algorithm pseudo-code of the conformalized decision risk assessment described in the main text summarized in Algorithm [
(2) in Step 9 with the algorithm pscudo-code summarized in Algorithm 2 Specifically, we note that the user needs to specify two
Additionally, we propose heuristic algorithm for solving (z)
lists of finite points 5 y and Z C (0) priori, which serve as approximations for their original space counterparts. Some examples that can be considered for D include:
When | < +00, one can trivially take y =
When bounded metric space, one can take y as the e-net of y (a.k.a. the vertices of some finite grid discretization).
When y an unbounded metric space, one can set ) as the c-net of the level set of the marginal distribution of Y. Namely, for some B 2 O, this level set is denoted as: {y y Py (v) B). It indicates the regions where Y has a density mass of at least / 2
For 2(0), since we have already assumed it be compact space, then the first two options can be similarly used when constructing 2.
B PROOF OF THEOREM
Conditioned on the modified definitions introduced in Remark 2 we first present the following
lemma:
Lemma (E-value post-hoc validity). Under Assumption 7 then the estimator à(z) satisfies
P (YY+I E c(k) (X)+1;;;) 2 1 E[á], Vk = 1....,
where à can prohabilistically depend on ((X,, 1+) in arbitrary ways.
Proof of Lemma When < 1/(n + 1) or = 1, by the definition of R(a)), the statement holds
trivially. So we only need to consider the case when > à 1/(n + 1). For any k 1,...; K,
14

---

sup E Published as a conference paper at ICLR 2026
Algorithm I Conformalized Decision Risk Assessment (CREDO)
Require: Fitted generative model f: Calibration dataset {(T,, F=))==> Sample size K; Optimization parameter 0: Decision 2; Test covariate In+1-
1: π (2;0)) < Compute the inverse feasible region defined in equation 3
2: Initialize nonconformity score & 0:
3: for i E fl..... n do
4: 0r ~ f(a): rs + llys
úill:: & & (ri):
5: end for
6: for k == 1..... K do
7: Datl ~ f(ant1);
8: C( k Construct conformal set given & and gn via equation 6
9: (2) + Solve for the k-th decision risk via equation & (refer to Algorithm
10: end for
11: return á(z) + 1/K Ek (z).
Algorithm 2 Heuristic algorithm for risk estimation
Require: Proposal points Y and 2; Required inputs of Algorithm
1: #े (2; N=EEZ {u ) v; v)
2: ( & min geR. (2:00) 11 Untilla 112.
3: (2) 1500+ Din 19-22
(m ()
4: return Approximated risk estimator a (2).
begin by expanding the left-hand side:
P ( & (())(((;()))))) (x;a)
P 11 (R)
V112 â(n + 1)
Σ 10: Yill2
P
à(n + 1)1lY- - - -84+1l22
10: Yilla + 110.02)
1112
E 1== 110: Yill2 + |10 Yl12
P
à
(n + 1) 1b((
Y112
(n + 1) (k) Y1122
P
>
Σ 11 Yill2 + || 1112
à
Denote the following random variables.
(n + 1)llú: Y2ll2
F.
Vi I,...,
110: Yill2 [10 (R) Yll2
Di
(n + ()) Y112
(k)
Pat1
ΣL || Y2 + 11 Y112
be seen that the following two conditions hold:
(a) E + F, F Fn4
(b) E
E(F.l = F(a++))
where (b) holds by exchangeability (Assumption D}. Therefore, there is
E[F++]=
(14)
Using this result, it can be derived that
PP P( Fn+1 > 1/5)
ā E
sup E
E F =1,
ā
à
15

---

Published as a conference paper at ICLR 2026
where the inequality follows from Markov's inequality. Therefore, for any & which may depend on the data {X,, 1i there is:
P (P 1/a ④))
P(Fn+1 > 1/ā)
E
S sup E á
<
Ô
á
Using first-order Taylor expansion the left-hand side of the inequality above, there is
P( 1/a a)
E [P 1/a a)]
E
(15)
á
E@a
We assume that this approximation is exact, so the sign can be replaced with sign (detailed discussion in Appendix B.I.. Consequently, by combining the two equations above, we get
E [P( 1/a a)] S B[a] = P P( ( (Xn-1; 2 1
where á can probabilistically depend on {(X, K)) in arbitrary ways. This finishes the proof.
Proof of Theorem ! We begin by observing that
= (P 2 K
P Ep(reci)) c(k) (x:a
(16)
The first equality is due to the problem reformulation equation 4 The second inequality holds due to the definition of & (z). which guarantees that the k-th generated conformal prediction region is always contained in --1(2), (2). Since by Lemma]) we have proved that:
p {(r E c/ E
(x)) 21- a (k) (x)
Therefore, combining this with equation 16 we obtain:
P ( E (T:
21 E
(x)
=1 E [x(=))
K
k=1
We conclude the entire proof for Theorem [
B.1 DISCUSSION ON THE APPROXIMATION ERROR IN EQUATION 15
In this section, comment the approximation error of the first-order Taylor approximation in equation] T5] This approximation trick has been adopted in prior works (Gauthier et al. 20255), which has been argued that its error is small when the estimator á is well concentrated around mean. Empirically, this condition is usually satisfied in our setting. For example. when CREDO is deployed human-algorithm collaboration setting, the candidate decisions provided from the decision maker would be expected to be near optimal and should already enjoy a relatively small ground truth risk. This makes à have relatively small variance and well concentrated around its mean.
Even when this condition does not hold, we can resort to an alternative way to account for the approx- imation error during risk assessment. This can be done by theoretically deriving the approximation error and then manually offsetting the error in our risk estimator to achieve exact conservativeness guarantee. Specifically, let h())= = EEP((( > 1/a &)], there is:
((a)
E[h(a)] E
+ + Var(h((a) Var (
E
B[((a)]
E
E(ā
The first term is the Jensen gap, and the second term is by the Stuchy-Schward inequality. Assuming [8, 1] almost surely, then
Var(a). and
Var
S 53
Var (6).
E
33
16

---

Published as a conference paper at ICLR 2026
Then, plugging them into the previous equation, we get
((())))
[[[(a]]
1
I 53
Var (ā) + 20a
Var(ā).
&
E[á]
Since for random variables bounded within [6. there is the following trivial upper bound:
Var(á) S
Σ
Cov (Jx. [2) S 4'
K²
therefore, we can conclude that:
(P F > 1/a a)
E P 1/6 [a]]
E
+
ā
[[
48
46
Plugging this result back into the proof of Theorem
we get
P( =T;=m(YY)) 2 1 B[á(z)) - 1/4(0 48-2)
Therefore, one can take the final estimator as
min(å(z) + 1/4(0-3 +,
(17)
so that exact conservativeness is achieved.
Note that in the derivation above, we have assumed that E [6, almost surely and ő is known. A trivial value that the user can take for is 1/(n+ + 1), which is guaranteed by the design of the CREDO algorithm. We can also manually tune the value of d by modifying the conformalized radius as
+00, Ei= lb 1))
if a E [0, 8), 1) if a [8, 1), o
R((a)
+
to achieve tighter bound (i.e., smaller offset). One can prove that as long as 5 is chosen such that 1/(n + 1), all theorems presented in the main text remain valid, and the bound (i.e., offset) becomes tighter as 8 increases. Then, one can take equation 17 the final estimator.
In the meantime, we note that the above definition of the conformalized radius is equivalent to truncating the lower part of d(2) at 6, ie., setting max(á(2), (({2),)) 8) the risk estimator, and then taking equation 17 as the final estimator.
C PROOF OF PROPOSITION 2 AND COROLLARY I
This section consists of three parts: we first prove the weighted Monte Carlo estimator form pro- vided in equation TO in Proposition 2 Then, we derive the indicator term; Finally, we derive the conformalized weighting term. The latter two all appear in Corollary [
Please note that all derivations use the modified definitions introduced in Remark 2
Proof of equation 10 Note that by definition of the conformalized radius ?(a) and (*), when the k-th prediction falls outside of the inverse feasible region, then (x) would be conservatively to one. Therefore, we can make the following decomposition:
(2) (2)
{ (k) Bati E (2; min {a
((*) C (x;;0))) + yon $ x (2:0))
aE(0,1)
(k) { |ot1 E (2;0)) +1 Unti E (x;
(b))
+ 1 I (K))
Unt1 E = (x:00]
min
{a c( I = -1) (K)
(E(0.1)
=1 (1- min
{a (2 (=
(x=1(==))
1 (/) E x x
Unfi
aE(u,1)
- (k) (2, (2,1)) 1
(k))
Vati E En 2
17

---

Published as a conference paper at ICLR 2026
where in the last equation, we denote the conformalized weight as:
w ( (z, In+1)
min aε(U,1)
a
cli (In+1:a) - - (2;
The final estimator an average over all K of these estimators:
K
a(z) - K
Σ
(ohts E (x;0))
1=1
which concludes this part of the derivation.
Derivation of the indicator term. Under the linear programming assumption, there is:
(y E U (2; = n E g(2. y: S g(=',):00)) 0)
#'E2(0)
n
{y E y 9(2,1;0) g( v: 0) s 0)
xEEZ(0)
N
{y E y (v, - 2) s 0}
2'E2(8)
0
{y E y (y, 2)) S 0)
2(E((0)
{y E y
sup z'2((0)
(y, 2¹) < 0).
We rewrite the condition in the set as:
sup (()
(y, 2²) (y,
inf (y, z'). (8)
Since 2(0) is a compact set, by the
theorem Krein Milman 1940), there is
inf (j,2') inf 2'E2(0)
v).
vEV(O)
where we denote V(0) as the collection of extreme points of Z(0). Plugging in the original equation, we get
VVEyIVEN Ey E y sup (y.2 8) 0}
n
{v E (y, v) s 0).
vEV(O)
vEV(0)
Therefore, there is:
(f Unit E E (2:0))
= II
1 (00ff1: o) 50}
vEV(O)
This concludes this part of the derivation.
Derivation of the conformalized weight. Recall that the conformalized weight is defined as
(k) (z, In+1) = = min (0)
{ a c(
(xn+1:a) C T (=;
By the definition of the conformalized set, there is the following equivalence:
cI ( C -z; + A((a) S inf
[1 - yll2,
year (=:00)
where the right-hand side represents the closest distance to the boundary of a π - (): Under linear programming assumption, the inverse feasible region can be written as
(2; = n {u E Rdd 1 -
n VE
{u E Td
T (z v) s s0}. 1
z(EZ(0)
18

---

Published as a conference paper at ICLR 2026
Therefore, - f) is polyhedral cone, which is group of intersecting hyperplanes. Therefore, the closest distance to the boundary this cone can be derived as:
Datl (z
inf
10
ull
min vEV(M(=
D(K)
v00x
(2;0)
||2 - 2112
We denote the right-hand side quantity ((A) Therefore, by the definition of R(a,,, there is
(()) Σ 11: -vil2 (n D)
R(aa) S D(k)
and 1. n2 n+1
Therefore, the solution to the optimization problem yields closed-fore forsed-fomm solution as:
min {a (1 S - (=;00 min {
DD - >. 1 vill2
aE(0.1)
(n + 1)D(R)
Therefore, plugging this back the conformalized weight:
w (k) (2, (2,1))1) = (T.((k ΣE 11: Voll2
(n + 1)k())
where (.)+ is the positive part operator. This concludes the proof.
D PROOF OF PROPOSITION 3
Proof. For notation simplicity, denote the sets in the numerator and the denominator of TPR defined
in equation 11 as:
A (2 E (() P[ (Y:0)) and &(z) 1}, B = fa 2(0) Pf & (Y: < 1}.
Note that A B. Without loss of generality, we assume that (therefore A) is finite set, i.e. there is only finite set of decisions that have ground-truth risk smaller than one")
Since B is constant irrelevant to K, we begin by expanding the following expression:
E
[ E
Σ
(á(=))
<1>
LIEB
(1
(k) Mot1 $ π (x;0)]
>o}))
E
>
LEBB
k=1
K
E
[
E
II A
LEB k=1
Here, the second equality results from the observation that when: (i) at least one of the model predictions falls within -2; f), and (ii) its conformalized weight zero, then the estimated risk after averaging would be less than one. In the last equality, we denote
A = Muti n (x;
(2,
Note that A E {0, 1) and has a nonzero probability taking ond Since at the right hand side, the term I k= A& monotonically decreases with K almost surely, then E # A] monotonically increase
with K. Since TPR E A| / /E B), we know that TPR monotonically increases with K. This
concludes the proof.
"This proof naturally extends the infinite case by replacing the counting measure "H" with continuous measures, such as Lebesgue measure defined within the decision space Z.
Under some regularity condition on the distribution of Y, e.g., is supported on the whole space Without this condition, the statement also holds by changing "monotonically increasing" to "non-decreasing".
19

---

Published as a conference paper at ICLR 2026
E EXPERIMENT SETTING
This section is organized follows: Appendix E.1 shows the computation resourees used in our experiments. Appendix F.2 and Appendix E..3 presents the detailed configurations for the two synthetic settings and the real-world case study, respectively. Appendix F.4 describes the detailed configurations CREDO, the baselines, and the procedures for the two experiments (Conservativeness versus Accuracy Tradeoff and Decision Quality Evaluation).
E.1 COMPUTATION RESOURCES
Table 2: Computation resources specfications
Operating System
CPU
RAM
GPU
Windows 11
sett-ggneration Intel Core i7, 16 cores
16GB
Not used
The computational resources used in experiments detailed in Table Our code is implemented in Python, with key dependencies including Scikit-learn (Pedregosa et al. 201 T) for solving linear programs and working with Gaussian mixture models (fitting and sampling), and CDD (Fukuda) 1997) for computing the vertices of polytopes. A complete list of dependencies and their version numbers available in our codebase.
In terms of execution time, each single experiment (consisting of multiple independent trials) in the paper takes less than 30 minutes run. All models can be executed almost instantly for single trial in our experiment, except SPO+ (Elmachtoub & Grigas (002).. The main computational bottleneck of SPC+ (Elmachtoub Grigas) 2022) arises from the need solve two linear programming problems for each iteration of parameter update, which can be easily addressed by using more efficient solvers.
E.2 SYNTHETIC SETTING
We denote as the component variance scale (a by default), and we denote Ig the two- dimensional identity matrix.
Setting I linear programming problem featuring a triangular feasible region with three vertex decisions, defined as:
Y1 Y22
N (x((()).o11))
max (Vi21 ZER
22
21
22
O, 20)
This optimization problem can be interpreted as profit maximization task, where manufacturer chooses the optimal production quantities 2 and z2 under budget constraint. The Gaussian random revenues Yi and Y2 have negative expected values but may exhibit some variance, capturing risky market scenario that could still yield profit under favorable conditions.
The feasible region in this problem can be more compactly denoted by its constraint matrix A and constraint vector as
b
0
Setting II A linear programming problem that employs more complex octagonal feasible region with five vertices and multimodal objective uncertainty, allowing assess performance in scenarios with multiple potentially optimal decisions. Specifically. the optimization problem set up as the canonical form max { Ty= z A < b]. where the constraint matrices A and constraint vector
20

---

Published as a conference paper at ICLR 2026
b are defined
0.5
0
0.5
1
1
0.5
5
A
b
-1
10
1
0
5.5
2.5
-1
0
-1
The random vector Y E R2 drawn from three-component Gaussian mixture distribution
p(z)
Σ
Wk
N (x
Pkx x. 22
k=1
where the mixture weights are = (0.3, 0.4, 0.3), the component means
0.0 (-o.s)
-0.5
0.8
H2 =
0.25
Ms =
-0.1)
and the component variances are
of = (0.013?
of (0.03)?, o3 = (0.02)?.
E.3 REAL-WORDD SETTING
As described in the main paper, we consider real-world power grid investment decision-making problem (Zhou al. (024).. We begin by introducing the application background, followed by formalization of the problem knapsack optimization. Finally, we present a linear programming relaxation this problem, which can be directly used in CREDO.
Background A utility company based in Indianapolis, Indiana, has compiled detailed records of over 1,700 solar panel installations between 2010 and 2024, including the installation dates and full scale, the management
affiliated grid components. With the renewable energy sector team anticipates a steady and significant monthly increase solar adoption in the downtown area. In preparation for the incoming demand, they are planning targeted upgrades to grid-level inverters at four selected substations (we refer to them as Substation A to D) under a limited budget. The utility company would like consult CREDO's suggestion for a list of candidate upgrade plans that would most likely be optimal.
The data available in codebase in spatio-temporally aggregated format (monthly, by substa- tion), with substation names anonymized, However, the granular solar panel installation data used in this study cannot be shared publicly, as is proprictary the utility company.
Mathematical Formulation This problem can be mathematically formulated as a penalized knap- sack problem, whose goal is minimize the total penalty from capacity violations while ensuring that the total upgrade cost does not exceed the budget. Formally, define as the total number of substations, and define parameters as in Table the optimization problem can written as
min (0(0,))
1 2 T.) a:)4 Σ a.c <
(18)
In our experiment, configure these parameters follows: we assume the cost and loss I; for all substations equals one unit. The capacity threshold т, set be the historical average solar panel monthly increment. The budget b is set to half the cost of upgrading all substations, allowing at most two out of four substations can be upgraded. Note that these parameters require additional information from the company, which is not available the time of writing this paper.
21

---

Published as a conference paper at ICLR 2026
Table 3: Parameter definitions for the real-world optimization problem
Parameter
Definition
Binary decision variable; a, = 1 indicates a substation up- grade, and a; = 0 indicates upgrade.
Y
Random variable representing the monthly increase in solar panel installations.
Cost associated with upgrading substation i. Capacity threshold-tie the maximum number of allowable lar connections at substation i. Penalty incurred the threshold тÚ exceeded without an upgrade.
Ti
Total budget.
The knapsack problem defined in equation can relaxed into the following linear programming problem:
Relaxed Formulation
min #e (
Σ
-0 a.)
Σ aa S
Σe
c
(19)
where 3 is an arbitrary smoothing parameter that we set 0.5. Note that in the equation above, two relaxations have been applied:
The binary random variable (Y; 2 Te) has been relaxed to a continuous random variable over [0, 1] using the sigmoid function, where the smoothing parameter 3 controls the sharpness of the approximation.
The decision space of a, has been relaxed from the discrete set 0, 1d the continuous unit hypereube [0, 11 This relaxation does not introduce approximation bias, as the optimal solutions the original and relaxed problems are almost surely the same both attained at one of the hypercube's 's vertices.
Denote the vectors
Y 1+
C1
1 a1 ... On
(20)
Cn
-+e-A(Y--r)
we can compactly write the optimization problem above as
min
{ TyT2 -c <b 1 0 0s
(21)
zER
This can be written as the canonical form of linear programming min (YT2) 2 A $ b) by defining
b 1
L
1a
0a
where Il denotes the d-dimensional identity matrix, and 1 and 0g denote the d-dimensional all-ones and all-zeros vectors, respectively. These parameters can be directly input to the CREDO framework using the data Y obtained via sigmoid transformation defined in equation 20 to obtain decision risk estimates.
Therefore, with the above reformulation in equation 21 the problem is now linear programming problem, where the closed-for risk estimator derived Corollary can be directly applicd.
E.4 BENCHMARKING BASELINES DESCRIPTION
In this part, deseribe the p-value conformalized radius, baselines, metrics, well the detailed evaluation procedure for each experiment that we conducted in the main paper. For both experiments, the following (Table 4 default values of hyperparameters used for CREDO.
22

---

Published as a conference paper at ICLR 2026
The p-value conformalized radius (Singh al. 2024) originates from the standard conformal prediction literature, defined as:
R(o) = R(a)= (n (Innt
1)(1 a)l
where Q(-) denotes the empirical quantile function of the calibrated nonconformity scores {ll will) with y; being generated prediction from (xx;
Table Hyperparameters in the Convservativeness Accuracy Tradeoff experiment.
Hyperparameter
Description
Default value
K
Simulation size
1 x 102
(Component) variance scale
1
71
Calibration dataset size
1 x 10a
771
Training dataset size
1 x 102
Convservativeness vs. Accuracy Tradeoff Table 5 summarizes the used baslines, which are two ablation models mainly target replacing step 2 of CREDO with different methods that can evaluate the probability P {Y π (2; }) We note that we implement Point nt by setting CREDO with K = 1 and configure the probabilistic model to collapse to a point mass centered at the mean.
Table 5: Summary table of baselines used in the Convservativeness vs. Accuracy Tradeoff experiment
Model Name
Description
Additional Remark
Point
A deterministic variant of CREDO, where the gen- erative model is replaced by a deterministic point prediction model. The prediction model captures the of the distribution of Y. Note that this ablation model is adopted from previous literature (Singh et al. 2024), where it was originally applied to assessing the risk of given prediction intervals.
The design of Point limits its ability to capture multi- modal heteroskedastic un- certainty in the objective coef- ficient space.
A naive sampling estimator computed as NS
The design of NS contrasts the proposed estimator, which be rewritten as equation [0 by dropping the conformal- ized weighting term. This lim- its its ability to meet the con- servativeness guarantee.
(NS(2)
Σ
1
Unti E '=;0)}
where (1)) Pat1:... Pat1 are independent samples drawn from (x)+1). This estimator directly ap- proximates the probability mass over the inverse feasible region using samples from the generative model.
Table 6 summarizes the evaluation metrics used in this experiment. We use T to denote the total number of repeated independent trials, where we T 20 across all of our experiments. For each metric, the eиror is reported as the standard error of the mean (SEM), calculated as the standard deviation of the metric values across trials divided by VT. Note that while most metrics require summing over all feasible decisions, we can simplify to summing all the decisions defined on the vertices of the feasible region, as we have justified in the main text.
Decision Quality Evaluation Table 7 presents the four selected baselines used the Decision Quality Evaluation experiment. The baselines have been selected to satisfy the following criteria: (i) the ability to produce a decision that maximizes a (linear) objective function; and (ii) the capacity to handle randomness in the underlying optimization problem.
We set (()) 0 and Q(1)
complete the definition.
23

---

Published as a conference paper at ICLR 2026
Table 6: Summary table of metries used in the Convservativeness vs. Accuracy Tradeoff experiment.
Metric Name
Description
Mathematical Formula
Conservativeness rate
The percentage of decisions where the risk estimate satis- fies the conservativeness guar- antee in equation 2 avveraged across all trails.
Σ [(((z) S a()))
T [V(0)] =CV(0)
True positive rate
The pereentage of decisions where the method correctly identifies them to have risk of less than one, averaged across all trials.
Σ
[ā((2) < 0) - <
T V1O)
=EV(8)
Relative accuracy
negative absolute differ- ences hetween the estimated and true risks across all de- cisions and trials, normalized [0, 1] across different base- lines.
Σ [ā(z) - a(=)]
T T- [[ =EV(0)
Table 7: Summary table of baselines used in the Decision Quality Evaluation experiment
Model Name
Description
Additional remarks
PTO
The standard two-stage predict-then-optimizz ap- proach (Bertsimas & Kallus 2020 which first predicts parameters and then solves the resulting optimization problem;
Since is omitted in our setting, PTO degenerates to stochastic optimization, and Y can be simply specified as the empirical average estimator.
min g(z, Y), sE2(0)
where Y is a point estimate of [[][X].
RO
Robust optimization defined as
The uncertainty set con- structed in data-driven man- ner by applying naive confor- mal prediction to Y, using the fx norm residual as the non- conformity score (Zhou et al., 2024).
min max g(z, =0) yell
where U y is the uncertainty set of Y.
SPO+
A predict-then-opimizz method where the predic- tion model is trained using the surrogate smart the-opiite loss (Elmachtoub & Grigas 2022
For each trial, is trained for 1 102 epochs using a learn- ing rate of 10
DFL
Decision-focused learning. where the prediction model is trained by directly optimizing the down- stream objective through end-to-end differentiation of the optimization layer (Amos & Kolter 2017).
For each trial, it is trained for 1 102 epochs using a learn ing rate of x 10-1
As for the evaluation metric, Algorithm outlines the generic procedure for computing empirical confidence rankings in single trial. In the synthetic setting, this procedure is repeated over T 1 x 102 independent trials with L the evaluation is performed rolling over T 12 periods, spanning from 2010 to 2022. Each trial uses a two-year window (24 months) of data, which sequentially split into training, calibration, and testing sets in an 8:8:8 ratio. The reported error is the standard error the mean (SEM), computed the standard deviation of the metric across trials divided by vT.
10 test data points. Meanwhile, in the real-world setting,
24

---

Published as a conference paper at ICLR 2026
Algorithm 3 Empirical Confidence Ranking
Require: Original optimization πT" ) + 2; Evaluated policy π X - 2; Test dataset
(( (()::)))==
1: 2" =0
2: for i (l,...,e] do
3:
2 x'
4: Z' ++ -=-
5: end for
6: for E V(0) do
7: h1(2) f- the total occurence of decision [il 2 =
8: end for
9: for (L...., f do
10: & - T.;)).
11: rank; ranking of estimated decision {z' E V(0) h(z²) 2 h(4)
12: end for
13: return Empirical confidence ranking 1/6 Σ rank,,
ADDITIONAL EXPERIMENT RESULTS
This section shows additional experiments from the experiments (Conservativeness vs. Accuracy Tradeoff, Decision Quality Evaluation) that were shown in the main paper, as well as an additional experiment that investigates the effect of misspecified models on the risk estimation under the CREDO framework.
Conservativesness vs. Accuracy Tradcoff In this section, we first present three additional insights based on the results in Figure 5 followed by the complete set of experimental results that were only partially shown in the main paper. These experiments aim to investigate the sensitivity of method's ' accuracy different hyperparameter settings. At high level, we find that CREDO consistently outperforms Poi nt, with the performance gap becoming more pronounced at larger variance scales and higher simulation sizes K.
From Figure 5 we draw the following observations in addition to those discussed in the main paper:
A general trend observed in both settings the decline in relative efficiency. This is expected: conformal prediction must adopt more conservative approach to preserve its validity guarantees as the data becomes more stochastic. While this ensures coverage, also results in wider prediction sets and, consequently, lower relative efficiency.
In the first example, Po PointModel achieves the best performance when the variance small. This is because the generated distribution is tightly concentrated around the origin (0, 0), rendering the problem effectively deterministic-aan ideal scenario for point estimation. In contrast, the second example features mixture components that inherently dispersed across different inverse optimality regions. As result, the data remains non non-deterministic even at low variance levels, and CREDO consistently outperforms the basclines by capturing this underlying distributional uncertainty.
The performance gap between NS and CREDO leads to an alternative explanation of our pro- posed method: the weighting term takes value from to 1, and "safeguards" the estimation from being overestimation by downscaling the contribution of each sample that would fall within region (z). The value of this weight is determined through a data-driven, confor- malized procedure, ensuring conservativeness is achieved. This proves the significance of our proposed algorithm.
Figure 6
Figure 7 Figure Figure 9 illustrate the full results evaluating the accuracy of three methods under three hyperparameter selections. We make two remarks:
Observe that NS achieves the highest accuracy metrics across nearly all plots. This outcome is expected, as NS is explicitly designed to maximize accuracy. However, fails to satisfy the conservativeness guarantee, rendering its performance substantially less reliable in
25

---

Published as a conference paper at ICLR 2026
practice. Therefore, exclude from the result demonstration in experiment result comparisons.
Observe that CREDO consistently achieves higher accuracy metrics than Point, highlight- ing the advantage of incorporating generative models in CREDO over relying solely point predictions.
Additionally, Figure 10 and Figure I1 present the raw estimated risks for all decisions across the three methods under Setting I and Setting II. These plots provide more fundamental perspective underlying the summarized metries shown previously. Each boxplot is computed over 20 independent trials using the default hyperparameter settings. CREDO and Point provide confidence estimates that fall below it, NS may overshoot to the red region. This indicates the conservativeness guarantee provided by the conformallieed procedure in our framework.
1.00
True poséive tate
0.75
postive
CREDO
CREDO
0.50
Point Model
Point Model
NS
NS
0.25
0.25
mm
0.00
20
40
100
10
10-2
10P
10
Simulation size
Variance scale
Figure 6: True positive versus simulation size K (left) and varaince scale (right) in Setting I.
0.0
CREDO
CREDO
Relative accurncy
-0.1
Point Model
Relative aceurney
Point Model
NS
-0.1
NS
-0.2
-0.2
-0.3
10-3
10-2
10a
10a
Sinmlation
size K
Variance scale
Figure 7: Relative accuracy versus simulation size K (left) and varaince scale (right) in Setting I.
26

---

Published as a conference paper at ICLR 2026
1.0
posétive rate
0.8
True prsétive rate
0.6
CREDO
0.6
Model
0.4
0.4
CREDO
0.2
0.2
Point Model
NS
0.0
20
40
60
80
100
10
10
10
10a
10
Simmlation size K
Variance scale of
Figure 8: True positive rate versus simulation size K (left) and varaince scale (right) in Setting II.
-0.05
-0.025
Relative accuracy
Relative aecunec
-0.050
CREDO
0.0075
-0.15
Point Model
00.00
NS
CREDO
-0.20
Point Model
0.125
NS
-0.25
0.150
0
20
40
60
80
100
10
10-2
10
10P
10
Simmlation
K
Variance
Figure 9: Relative accuracy versus simulation size K (left) and varaince scale a (right) in Setting II.
Decision Quality Evaluation Figure 12 shows the frequency that the decisions get picked for each baseline of Setting I and Setting II. In both settings, the output of CREDO closely aligns with the mimull l on benchmark computed from the ground-truth distribution, whereas most other baselines tend to select different decisions. This highlights two key points: (i) the maximum likelihood decision rule can lead to substantially different decisions compared to methods that optimize the objective without accounting for probabilistic structure; and (ii) in such case, CREDO provides a reliable first-stage identification of the magnitude risk for each provided decision.
Figure 1 presents the top four candidate upgrade plans identified by CREDO. The lowest-risk, and thus most confident, recommendation to upgrade substations A and C. This highlights CREDO's prioritization of these two substations for immediate upgrades in future operations. While the second- to fourth-ranked combinations may not be the most likely to be optimal under the model's risk assessment, they may still be valuable in practice. These alternatives might offer advantages along other dimensions not captured by the knapsack formulation, such as geographic equity, policy constraints, or logistical feasibility. We envision CREDO as decision support tool that complements, rather than replaces, human judgment. Its ranked recommendations provide a diverse set of plausible solutions, enabling informed and context-an aware dacision-making instead of enforcing single deterministic outcome.
Aditionaa experiment: Misspecified Model We conduct an additional experiment to evaluate the conservativenss-accrrayy tradeoff of various models under setting where the prediction or generative model is entirely misspecified. Our results show that, despite the misspecification. CREDO consistently maintains conservative estimates while achieving relatively high efficiency compared to its point prediction variant.
Specifically, the data generation distribution and model distribution are assumed take the following form
GMreal W.
((-00
-0.5
(o
0
+ - -
((05)
a
0
0
0
σ
0
((-0.5)
o
0)
0
(0.5)
0
G.Mmoeel = 0.66 WN
+ 0.4 WN
0
27

---

Published as a conference paper at ICLR 2026
Decision
Decision
Decision
Estimated confideuce
0.6
0.4
0.2
0.0
CRÉDO
Point
CREDO
Point
CREDO
Method
Method
Method
Figure 10: Estimated confidence level (1 a) versus different baseline models in Setting The red dashed line indicates actual confidence computed from the ground-truth distribution.
Decision
Decision 2
Decision 3
Decision
1.0
Eetimated confedence
0.8
CREDO NS
Point
CREDO
NS
Point
CREDO
NS
Point
CREDO
NS
Point
Method
Mothod
Method
Method
Decision
Decislon 6
Decision 7
Deciston
1.0
Extimated confidence
0.8
0.6
0.4
0.2
CREDO
NS Method
Point
CREDO
NS
Point
CREDO
NS
Point
CREDO
N8
Point
Method
Method
Method
Figure 11: Estimated confidence level (1 a) versus different baseline models in Setting II. The dashed line indicates actual confidence computed from the ground-truth distribution.
Figure 14
provides the raw view of estimated risk versus the different specifications of calibration dataset size under Setting I. Figure [55 highlights the inverse feasible region correspondence from each decision in the feasible region.
Figure 16 presents the main results of this experiment, illustrating the estimated confidence levels of different models across various decisions. Due to model misspecification, NS notably overestimates the confidence for Decision 2. In contrast, both CREDO and Point avoid such overestimation, align- ing with their conservativeness guarantees. Furthermore, across all decisions, CREDO consistently yields higher confidence than Point, indicating superior accuracy.
28

---

Published as a conference paper at ICLR 2026
Setting I
Setting II
100
100
Rvequesiy
Proquenzyy
30
50
25
25
[t. 0.]
[0. 1.] Decisions
[0. 0.]
[1. 1.5]
[1.
0.5
[2. 0.]
[5. 0.)
(5.5
55.5
2
2.25
[5. 2.5]
[3. 2.5)
Decisions
Simalatioo (Oracle)
CREDO
PTO
RO
SPO+
DFL
Figure 12: Frequency of each selected decision over 100 repeated trials, compared across all baseline methods. The left panel corresponds to Setting I, and the right to Setting II.
Stacolon
Upgrade
Not upgrade
Figure 13: Top four candidate upgrade decisions with the lowest estimated risks (left right), as recommended by CREDO. Each shaded region represents the span of a substation network.
Deciston
Decision
2
Decision
1.0
Estimated confidence
0.8
0.6
0.4
0.2
0.0
CRÉDO NS
Point
CREDO
NS
Point
CREDO
NS
Point
Method
Figure 16: Estimated confidence level (1 a) versus different baseline models in the misspecified model setting. The red dashed line indicates actual confidence computed from the ground-truth distribution.
29

---

Published as a conference paper at ICLR 2026
spacn
Real data distribution
Model distcibation
Deceou
Dec
Secoal timeiion 1t
0.00
Scoued darcias
Sccual disreiine
Vistioor le
0.009
BUL22
0.012
Dodiinn
Docisio
Docision
Deciiio 31
0.9%
0
2
Fink dimusioon
Fine dimombion
Fins dinsetionn
Figure 14: Setting visualization of the misspecified model experiment setting
Decision space
Outcome space
Decision
Docision
Decision
Decision A
Decision
De E
2
Feasible region
Decision
Decision
-2
Decision
Decision C
-2
-2
-2
0
V1
Figure 15: Decisions space and outcome space correspondence through the inverse feasible mapping in our misspecified experiment problem setting.
30


Reviews:
[{"review_id": "HGCdMfDnDx", "reviewer": "Reviewer_yHqA", "summary": "This paper proposes a method, to provide a statistically valid estimates of the probability of sub-optimality for any candidate decision proposed by a human expert. They do this through rewriting this problem in terms of an inverse optimization problem of the outcome and then construct a conformal set that is contained in the inverse space. They also provide a computationally efficient implfememntation of this method.", "questions": ".", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The problem of designing ML-powered, distribution-free valid risk certificates is a timely and impactful problem. The authors make a meaningful step by connecting this problem to running CP on an inverse optimization problem.\n- Their work has a very good balance of theory and experiment. The theoretical guarantees are of the interest of practice, and they showcase that it actually works in real world datasets.", "weaknesses": "My first concern is regarding \"selection bias\". Selection bias is well-known phenomena is statistics, which points toward the scenario where a decision maker wants to use an estimation to inform their decisions. The bias arises, when the estimation is performed without the knowledge of the down stream decision problem, and this can potentially disrupt the statistical guarantees of the original estimation. In the context of the problem that is studied in this paper, it shows itself as follows: say a decision maker want to pick a decision such that the risk certificates is larger than \\tau. Now what they could do is to run your algorithm, get the certificate, and then filter out the decisions that the corresponding certificate is less than \\tau, and pick one of the remaining. Now the issue is, from the viewpoint of the decision maker, what actually matters is that the risk certificate be statistically valid conditioned on the certificates larger than \\tau. If this doesn't hold, then the algorithm suffers from \"selection bias\". The question is, whether this algorithm suffers from such a situation (my guess is yes), and if yes, to what extend (this could be certified in experiments), and if the bias is significant, how to fix it (this could at least be discussed\\reported in the future works and limitations).\n\nThe second concern is regarding a proper discussion regarding the two recent works of [1] and [2]. Although the scope of those papers, particularly [1], is to \"find\" a low-risk action, rather than \"certify\" the low risk ones, however, one can use their method to derive risk certificates too. The idea is, you can run your favorite CP method, to get a set of labels C(x). Then if you run the max-min rule defined in these papers, you get both a certificate (as the max-min value), and an action (the argmax-min). Alternatively, you can ignore the outer max, and for each action candidate of your interest, you can just solve the inner min of their method, and that would give you a risk certificate that holds with high probability. There needs to be a discussion to distinguish this simple method with yours, in terms of scope, practicality, and use cases.\n\nI like this paper, and if the authors provide a satisfying answer to these concerns I would be happy to vote for acceptance.\n\n\n[1]: Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents, Kiyani et. al.\n\n[2]: Certified Decisions, Andrews et. al.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5ruzAQKd8z", "reviewer": "Reviewer_XFwE", "summary": "CREDO provides a distribution-free upper bound on the probability that a candidate decision is suboptimal, using inverse optimization geometry and conformal prediction with generative models. It enables practitioners to audit both algorithmic and expert-proposed decisions, offering statistically valid risk certificates. Theoretical guarantees and empirical results on synthetic and real-world tasks demonstrate that CREDO delivers conservative, interpretable, and actionable risk estimates, improving trust and decision quality compared to standard PTO and robust optimization approaches.", "questions": "1. For general nonlinear or combinatorial optimization problems, how is the inverse feasible region (\\pi^{-1}(z;\\theta)) practically characterized? Are there efficient relaxations that maintain the validity of the risk certificate, or does the method require exact computation?\n2. The framework uses conformal prediction with generative models to construct inner approximations of the inverse feasible region. How sensitive is the risk estimate to the choice of conformal set (e.g., L2 balls vs. other shapes)?\n3. What are the theoretical or empirical sample complexity requirements for the calibration set to ensure valid and non-trivial risk certificates, especially as the dimension of Y increases? How does the method perform with limited calibration data?\n4. Can the CREDO framework be extended to settings where decisions are made sequentially or in multiple stages, with uncertainty revealed over time? What are the main challenges or limitations in such extensions?\n5. Beyond generative models, have the authors empirically compared CREDO with approaches using quantile regression, Bayesian models, or ensemble methods for conditional uncertainty estimation?\n6. Can the authors provide a more detailed analysis of the computational complexity of CREDO for both the linear and general cases, including the cost of generating samples, constructing conformal sets, and evaluating the inverse feasible region?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper addresses an under explored research area about how to provide rigorous, interpretable risk certificates for candidate decisions in high stakes, uncertain environments.\n2. The \"decide-then-assess\" paradigm is a reasonable variation from the standard \"predict-then-optimize\" pipeline, and is well-motivated by practical needs for human-AI collaboration.\n3. The use of inverse optimization geometry to characterize the optimality region for a decision is well done.\n4. The integration of conformal prediction with generative models for risk estimation and the corresponding theoretical guarantees are clearly stated and proved.\n5. The closed form solution for linear programs makes the method practical for large scale problems.\nThe experiments are well designed, covering both synthetic and real world settings (e.g., power grid planning).", "weaknesses": "1. While the method is general, the closed form efficiency is only for linear programs. For nonlinear or combinatorial problems, the computational cost of characterizing the inverse feasible region may be significant.\n2. The approach assumes access to a well calibrated conditional model for the uncertain parameters. While the paper uses generative models to estimate the conditional distribution, in practice, any model that can accurately capture and sample from P(Y∣X) would suffice, including parametric or non-parametric approaches. I think it is limiting to claim the importance of generative models in this use case.\n3. The method is conservative by design, but this can lead to loose risk estimates in some settings. The paper discusses this tradeoff, but more empirical analysis of the \"tightness\" of the certificates would strengthen the work.\n4. The paper is motivated by human AI collaboration, but there is little discussion or experimentation on how practitioners actually use or interpret the risk certificates. A user study or qualitative feedback would be valuable.\n5. The paper positions itself relative to robust optimization, DRO, and conformal prediction, but could more deeply discuss how CREDO compares to recent advances in human-in-the-loop optimization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9JVD1ZAHFH", "reviewer": "Reviewer_TYnS", "summary": "The paper introduces a framework that provides distribution-free upper bounds on the probability that a given decision is suboptimal. The authors use inverse optimization space of the outcome and then construct a conformal set that is contained in the inverse space to produce the upper bound on the probability of suboptimal decision. The authors give a computational efficient algorithm when the objective function is a linear combination of decision and outcome. The authors validate the framework by experiments on synthetic and power-grid planning datasets.", "questions": "- What are the intuitions on the generative model $\\hat{f}$? How does it impact the quality of $\\alpha$? Is there any guideline for choosing $\\hat{f}$?\n- Why the radius version of conformal set is taken instead of the more general one?\n- What does this repeat K times do? What is the random variable here?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper's motivation is practical. The interpretable risk assessment in high-stakes domains is widely applicable and important.\n- The theorems on conservativeness and the Monte Carlo interpretation demonstrate sound reasoning and attention to statistical guarantees.\n- The idea of using inverse optimal space of outcome to find an upper bound on distribution-free probability is refreshing and can lead to potentially stronger results.", "weaknesses": "- The computational efficiency of the framework is not discussed. Finding the inverse space of the outcome where a given decision is optimal can be NP-hard for any objective function. It is also NP-hard to check whether the conformal set is included by the inverse space. The radius assumption here is still not enough since the inverse space can be non-convex.\n- The upper bound that is found by the framework could be arbitrarily bad. That says there is no result on the lower bound of the probability of decision being suboptimal. I'm having this worry especially because the conformal set algorithm constructs the candidate space as a naive radius space. It is very easy to construct a case where the radius is arbitrarily small such that $\\alpha$ is arbitrarily large.\n- The linear form of objective function is not a very general form. A lot of utility functions in decision-making such as brier score cannot be converted to linear form.\n- I found the paper is a little hard to follow. Some properties are not discussed. See more details in questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rSflRQ7yme", "reviewer": "Reviewer_pzmF", "summary": "This paper proposes a new framework that can evaluate the probability of suboptimality for any decision with strong statistical guarantees. The authors reformulate the probability of suboptimality as the probability of the outcome variable belonging to an inverse feasible region. This probability can then be estimated using conformal prediction sets of varying levels of marginal coverage produced from $K$ samples from a generative model. The method is validated on two synthetic datasets and a real-world infrastructure planning problem.", "questions": "* In Figure 5 Column 2, why isn’t the Point Model just a flat line? It shouldn’t be changing with $K$ increasing.\n* Why isn’t NS included in Columns 2 and 3 of Figure 5?\n* Column 3 of Figure 5 appears to be inaccurately interpreted (Lines 432-437). While high accuracy, in the typical sense, indicates better performance, “accuracy”, as defined by the authors, seems to be like a loss (absolute difference between true and estimated risk). So, shouldn’t the method with lower “accuracy” be better?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written and well-organized. The theoretical contributions are meaningful in validating the proposed method. For example, Proposition 3 justified the use of generative models very well. This framework is a novel approach to handling risk in decision-making by quantifying it directly rather than being robust to it during optimization.", "weaknesses": "The authors compare their method with other robust decision-making methods using empirical confidence ranking. However, I am not convinced that this is the best metric to evaluate decision quality. How is the predicted decision’s rank in terms of its frequency in the ground truth optimal decision set informative of decision quality? This metric is instance-dependent, and so choosing an action $z$ that is optimal (i.e., $z \\in \\pi(Y; \\theta)$), but rare among the optimal set of actions in the test set, would be discouraged by this metric. That doesn’t seem like a fair assessment of decision quality. An experiment evaluating decision quality seems crucial in building a case for viewing robust decision-making through a different lens. If the authors can clearly and strongly justify this metric or reproduce this experiment with a metric more indicative of decision quality, I will reconsider my score. \n\nAdditionally, I believe that Kiyani et al. (2025) seems quite relevant to this line of work; however, it wasn’t included in the experiments. I believe adding this baseline can strengthen the paper.\n\n_References_\n* Kiyani et al. (2025), Decision theoretic foundations for conformal prediction: Optimal uncertainty quantification for risk-averse agents. https://arxiv.org/pdf/2502.02561.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)