Conformal Prediction for Causal Effects of Continuous Treatments
Authors: Maresa Schröder
Dennis Frauen
Jonas Schweisthal
Konstantin Hess
Valentyn Melnychuk
Stefan Feuerriegel
Venue: NeurIPS 2025 poster ()
Keywords: causality
dosage response curves
conformal prediction
uncertainty quantification
Primary area: probabilistic_methods

Abstract: Uncertainty quantification of causal effects is crucial for safety-critical applications such as personalized medicine. A powerful approach for this is conformal prediction, which has several practical benefits due to model-agnostic finite-sample guarantees. Yet, existing methods for conformal prediction of causal effects are limited to binary/discrete treatments and make highly restrictive assumptions, such as known propensity scores. In this work, we provide a novel conformal prediction method for potential outcomes of continuous treatments. We account for the additional uncertainty introduced through propensity estimation so that our conformal prediction intervals are valid even if the propensity score is unknown. Our contributions are three-fold: (1) We derive finite-sample validity guarantees for prediction intervals of potential outcomes of continuous treatments. (2) We provide an algorithm for calculating the derived intervals. (3) We demonstrate the effectiveness of the conformal prediction intervals in experiments on synthetic and real-world datasets. To the best of our knowledge, we are the first to propose conformal prediction for continuous treatments when the propensity score is unknown and must be estimated from data.

Conformal prediction for causal effects of continuous treatments
Maresa Schröder
Dennis Frauen
LMU Munich
LMU Munich
Munich Center for Machine Learning
Munich Center for Machine Learning
maresa. wmw.oeder@luu.ccoommmm
frauen0lmu de
Jonas Schweisthal
Konstantin Hess
LMU Munich
LMU Munich
Munich Center for Machine Learning jonas. scwweisthaleeecee @1mu de
Munich Center for Machine Learning k. ww.hes@lmm.de de
Valentyn Melnychuk
Stefan Feuerriegel
LMU Munich
LMU Munich
Munich Center for Machine Learning melnychuk@lmu.co
Munich Center for Machine Learning feuerriegel @lmu. de
Abstract
Uncertainty quantification of causal effects is crucial for safety-critical applica- tions such personalized medicine. A powerful approach for this is conformal prediction, which has several practical benefits due to model-agnostic simite-sample guarantees. Yet, existing methods for conformal prediction of causal effects are limited to binary/discrete treatments and make highly restrictive assumptions, such known propensity scores. In this work, provide novel conformal predie- tion method for potential outcomes of continuous treatments. We account for the additional uncertainty introduced through propensity estimation that conformal prediction intervals are valid even the propensity score unknown. Our contributions are three-fold: (1) We derive finite-sample validity guarantees for prediction intervals of potential outcomes of continuous treatments. (2) We provide an algorithm for calculating the derived intervals. (3) We demonstrate the effectiveness of the conformal prediction intervals in experiments on synthetic and real-world datasets. To the best of our knowledge, are the first propose con- formal prediction for continuous treatments when the propensity score is unknown and must be estimated from data.
Introduction
Machine learning (ML) for estimating causal quantities such as causal effects and the potential outcomes of treatments is nowadays widely used in real-world applications such personalized medicine [2]. However, existing methods from causal ML typically focus on point estimates [e.g..
38 _0], which means that the uncertainty in the predictions is neglected and hinders the use of causal ML in safety-critical applications
[12 31]. As the following example shows, uncertainty quantification (UQ) of causal quantities is crucial for reliable decision-making.
Motivating example: Let us consider doctor who seeks to determine the dosage of chemotherapy in cancer care. This requires estimating the tumor size in response to the dosage for specific patient
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

profile. A point estimate will predict the average size of the tumor post-treatment, but it will neglect that chemotherapy is ineffective for some patients. In contrast, UQ will give a range of the tumor size that is to be expected post-treatment, so that doctors can assess the probability that the patients will actually benefit from treatment. This helps to understand the risk of a treatment being ineffective and can guide doctors to choose treatments that are effective with large probability.
A powerful method for UQ is conformal pre- diction (CP) [35, 39, 51I. CP provides model- saite- va- lidity guarantees for quantifying uncertainty. CP has been widely used for traditional, predictive ML [e.g., A 6 18). where it has been shown to yield reliable prediction intervals in finite- sample settings (see Fig. [ Recently, there have been works that adapt CP for estimating causal quantities (see Fig. 2 for an overview).
guararivee:
Ug Che/a)wih
agnostic and distribution free,
Conforrel intervel Car(a)
Potact el coanage 9sp
Cinlel) 2
frits sumples
PYY Choeel <1
Figure 1:
CP intervals on finite-sampl data. UQ methods with asymptotic guarantees might suffer from under-coverage and are often not faithful.
Thus, we aim at CP with finite-sample guarantees. discrete treatments [e.g.. 2 27 36), but not continuous treatments, which is our novelty.
Yet, existing methods for CP focus on binary or
Adapting CP to causal quantities non-trivial for two main reasons. Challenge @ Intervening on the treatment induces a shift in the covariate distribution, specifically in the propensity score. As result, the so-called exchangeability assumption, which is inherent to CP [5]], violated between the observational and interventional distributior(] and because of this, standard CP intervals are not valid. Thus, we must later account for the distribution shift and derive preamennt-condiiinee guarantees. Challenge O: Assessing the aforementioned shift in the distribution requires information about the propensity score; yet, the propensity score is typically unknown. Hence, estimating the propensity score introduces additional uncertainty. However, incorporating the additional uncertainty in the overall CP intervals cannot be done in a simple plug-in manner, and is highly non-trivial.
Unique to CP for effects of continuous treatments is a third challenge c data points with the same treatment value are rarely observed. Thus, we later employ smoothing model the propensity shift.
In this paper, we develop a CP method for causal quantities, such as potential outcomes, of continuous treatments. Our method is designed to account for the additional uncertainty introduced during propensity estimation and is thus applicable to settings where the propensity score is unknown.
Our contributions (1) We propose novel method for CP of causal quantities such as potential outcomes or treatment effects of continuous treatments. For this, we mathematically derive finite- sample prediction intervals for potential outcomes under known and unknown propensity functions. (2) We provide an algorithm for efficiently calculating the derived intervals. (3) We demonstrate the effectiveness of the derived CP intervals in experiments multiple datasets.
2 Related Work
UQ for causal effects: Existing methods for UQ of causal quantities are often based on Bayesian methods [c.g., [ 21 22 24). However, Bayesian methods require the specification of a prior distribution based on domain knowledge and are thus neither robust to model misspecification nor generalizable model-agnostic machine learning models. A common ad hoc method for computing uncertainty intervals is Monte Carlo (MC) dropout [4]. However, MC dropout yields approximations of the posterior distribution, which are not faithful [34].
Conformal prediction: CP [35 39 51 has recently received large attention for finite-sample UQ. For a prediction model trained dataset Dr (X,, Y:) and new test sample Xx. CP aims to construct a prediction interval C(X2)) such that P(Y, E C(X+)) 2 1 - a for some significance level a. We refer to [4] for in-depth overview. Due to its strong finite sample validity guarantees, CP is widely used for traditional, predictive ML with widespread applications such as in medical settings [57 drug discovery 3 10
By interventional we refer to the distribution after the intervention "Code and data are available at our public GitHub repository: ContinnousCausalCCP

---

Several extensions have been developed for CP. One literature stream focuses on CP with marginal cover- age under distribution shifts between training and test
Causal conformal prediction
Unknown progemaity
Finte cample cxect guararbocs
Cantinoose bestmentt
data le.g.. , [
16 15
18
48 55].
Whing wia 1P10
X
X
Our setting later also involves a distribution shift due to the intervention on the treatment but differs from the latter in that the true distribution shift is unknown. Another literature stream constructs intervals condi- tional on the variables following the shifted distribu- tion. Since, in general, exact conditional coverage has been proven impossible [35] [50]. the works in
X
Jankara (2504)
Leland Candle (2001)
X
Figure 2: Key works causal CP.
this literature stream have two key limitations: (1) they only guarantee approximate conditional (2) they restricted to specific data structures such binary variables [e.g., 35 50] Because of that, none of the existing methods for marginal and conditional
coverage [e.g.. 5 7 35 43
coverage be applied derive prediction intervals with finite-sample validity guarantees for causal quantities of continuous treatments.
Conformal prediction for causal quantities: Only a few works focus on CP for causal quantities (see
Fig. 2 Examples are methods aimed at off-po learning [47 58] conformal sensitivity analysis [56], meta-learners for the conditional average treatment effect (CATE) 2 27 36 52]. However, there are crucial differences to our setting: First, the existing works (a) assume that the propensity is known and thus achieve umite-sample coverage guarantees, the existing works (b) focus the easier task of giving asymptotic guarantees but then might suffer from under-coverage because of which the intervals are not faithful. Only Lei and Candès [36] provides finite-sample coverage guarantees under estimated propensity scores. However, all existing CP methods are designed for binary or discrete treatments. Applying such methods discretized continuous treatments leads to ill-defined causal estimands. Therefore, none of the existing methods are applicable to our continuous treatment setting. We offer detailed discussion in Supplement F
Research gap: To the best of our knowledge, no work has provided prediction intervals with finite-sample validity guarantees for causal quantities of continuous treatments.
3 Problem formulation
Notation:
We denote random variables by capital letters X with realizations x. Let Px be the probability distribution over X. We omit the subscript whenever it is obvious from the context. For discrete X, we denote the probability mass function by P(x) = P(X = x) and the conditional probability mass functions by Pwv x) P(Y = y X x) for a discrete random variable Y. For continuous X, p(z) is the probability density function w.r.t. the Lebesgue measure.
Setting: Let the data (X,, A,, Yi)==..... consisting of observed confounders X E X' continuous treatment A A, and outcome Y y be drawn exchangeably from the joint distribution P. Additionally, let new sample of confounders X,+1 be drawn independently from the marginal distribution Px- Throughout our work, we split the dataset into proper training dataset Dr = (X,, A,, Ya)==I..... and a calibration dataset De (X,, A,, k))==m+ Furthermore, let [a I) define the generalized propensity score for treatment A = a given X =
Throughout this work, we build upon the potential outcomes framework 44 We denote the potential outcomes of hard intervention a" by Y(a*) and of soft intervention A by (x)) We make three standard identifiability assumptions for causal effect estimation: positivity, consistency, and unconfoundedness [c.g.. 217 Finally, we consider arbitrary machine learning model é to predict the potential outcomes. Hence, we define the outcome prediction function á X' x A + IR, o(X, A) + Y. We assume the dose-response curve to be sufficiently Hölder-smooth. This is common in settings with continuous treatments [e.g., 40 45
T
PA-IX==
IIteevvnntioss are characterized by two classes: hard (structural) and soft (parametric) interventions. Hard interventions directly affect the treatment by setting it specific value and removing the edge the graph (as in the co-operator). Soft interventions of the treatment given the confounders. All interventions affect the propensity score, but the mathematical consequences are different. For soft interventions, the new (interventional) propensity is a function of the original propensity. For hard interventions, the new propensity is given by the Dirac-delta function.
change the structure the graph affect the conditional distribution
3

---

Our objective: In this work, we aim to derive conformal prediction intervals C(X++1. Q) for the prediction of a potential outcome (0) of a new data point under either hard, O a², or soft intervention, O A*(X n+) ~ #(a Xx+1). The derived intervals are called valid for any new exchangeable sample Xn+1 with non-exchangeable intervention O i.e., for & (a" (*
and significance level a E (0,1)
P(Y,+1(0) E C(X.+1, 0))
Of note, our CP method can be used with an arbitrary ML model ςi predict the potential outcomes.
In CP, the interval C/ is constructed based on so-called non-confornity scores [51, which capture the performance of the prediction model 6. For example, common choice for the non-conformity score is the residual of the fitted model s(X, A, Y) = Y ((X, A)), which we will use throughout our work. For ease of notation, define S; == (X,, A,, Y,).
Why CP for causal quantities non-trivial? There are two main reasons. First, coverage guarantees of CP intervals essentially rely the exchangeability of the non-conformity scores. However, intervening on treatment A shifts the propensity function and, therefore, induces a shift in the Formally, we have a propensity shift in which the intervention shifts the propensity function x x) to either Dirac-delta distribution of the hard intervention, (a), or to the distribution of the soft intervention, a(a I)), without affecting the outcome function 6(2, a). As a result, the test data sample under & does not follow the same distribution as the train and calibration data, i.c., the exchangeability assumption is violated.
covariates (X, A) (- Challenge
Second, the propensity score is commonly unknown in observational data and, therefore, must be estimated, which introduces additional uncertainty that one must account for when constructing CP intervals (-> Challenge b). Crucially, existing coverage guarantees [e.g., 48 do not hold in our setting. Instead, we must derive new intervals with valid coverage under propensity shift.
In the following, we address the above propensity shift by performing calibration conditional on the shift induced by the intervention, which allows us then to yield valid prediction intervals with significance level (i 1 a:) for potential outcomes of specific hard or soft intervention. We emphasize that the extension to intervals for causal effects is straightforward, in that one combines the intervals for each potential outcome under a certain treatment and without treatment, so that eventually arrives at CP intervals for the individual treatment effect (ITE). Details are in Supplement A
CP intervals for potential outcomes of continuous treatments
Recall that intervening on test data breaks the necessary exchangeability assumption, i.e., the guaranteed coverage at least (1 - a). Therefore, we now construct CP intervals where we account for (potentially unknown) propensity shift in the test data induced by the intervention.
Scenarios: In our derivation, we distinguish two different scenarios(see Fig.
Known
Same
propensity score
1
Known propensity score (see Section 4.1 If the propensity score in the observational data is known, it means that the treatment policy is known. Then, aim to update the policy by increasing/decrrasing the treatment by value A. i.e., A (X) A + A- Example: A doctor preseribes a medication to new patient. Instead of prescribing the same dosage as he would have prescribed a similar patient in the past, the doctor is interested in the potential health outcome when increasing (or decreasing) the original dosage by an amount A
Challenge: )
physician
f(*)
Unknown propensity score
Challenge: @
physician
6, f())
Figure 3: Use cases of the two scenarios: ① The new assignment function of the original pol- icy (i.e., soft intervention). 2 The policy in the dataset unknown. The new assignment cannot be expressed as function of the original policy (i.e., hard intervention).
Unknown propensity score (see Section
In observational data, the propensity score is unknown. Therefore, we usually assess the effect of hard interventions, i.c., a". Here, we face
4.2

---

additional uncertainty from propensity score estimation (++ Challenge b Example: In our running example, patient sees doctor who has never prescribed the respective medication and thus will base the decision on observational data (electronic health records), which was collected under different, unknown treatment policy (e-g., from another physician). Thus, the prescribed intervention (dosage) cannot be expressed in terms of the policy in the observational data.
In our derivations, we make use of the following two mathematical tools. First, we define the propensity shift. Formally, is the shift between the observational and interventional distributions, P and P, in terms of the tilting the propensity function by a non-negative function f. Hence, we have
f(a, x)
#(a x) EP]S(A,
T(a x
for some f with Erlff(X, A)] 0 and a A
Second, our CP method will build upon ideas from so-called split conformal prediction 39 51 yet with crucial differences. In our methods, the calibration step differs from the standard procedure in that we conditionally calibrate the non-conformity scores depending on the tilting function f to achieve marginal coverage for the interventional and thus shifted data.
Our derivation in Sections 4.1 4.2 proceed as follows. Following 18 we reformulate split conformal prediction as an augmented quantile regression. Let S; represent the I...., n of the calibration dataset and Sa+1 = S an imputed value for the unknown score of the new sample. We define
High-level outline:
non-conformity score of the sample (X,, A, Y;) for
(0, + t.(0,5)) (a((0))) +ta(.)))
0g = arg min
Σ
(3)
PER
n - m
i==++1
where
La (0, (a 1 (ooss -5 -S) ((= a) 0), if 0, a( 5). if
(4)
8.
Of note, Ós is estimator of the (1- -u-antiiet of the non-conformity scores 32 46 Using 0s. we construct the CP interval with the desired coverage (1 - a). However, the interval is only valid for exchangeable data. Quantile regression might yield non-unique solutions that depend on the indices of the scores [8] so we later restrict the analysis to solvers invariant to the data ordering
4.1 Scenario I: Known propensity score
We first consider scenario ① with known propensity scores. Here, existing CP intervals not directly applicable due to the shift from old to new propensity (-> Challenge @ For our derivation, need the following lemma building upon and generalizing the intuition presented above.
Lemma 4.1 (18)). Let FF define finite-dimessions function class that includes the function f characterizing the shift in the (potentially unknown) propensity function π (see Eq, (2). Define the distribution-shiff shiiriiio-iiifi-cbrrtteee (1 a) a--qunnteee of the non-conformity scores
ğs(N (X arg min g
Σ (==++1
La (a(((X)) + + (o(o((x+1),
(5)
for imputed guess S of the (n 1)-th non-conformity score S.t1. The prediction interval
C(Xu+1) = tv Sn+1(y) S 9s (w) (Xn+1)
(6)
for the true Sn+1 given a realization of Yn+1 = satisfies the desired coverage guarantee under all distribution shifts f E F, i.e.,
(Y E C(( 21
'Throughout main paper, focus the setting of hard interventions. In cases, might also be of interest perform soft interventions on the estimated propensity score. We provide derivations for this setting Supplement A
We note that commonly used solvers, such as interior point solvers, are invariant the data ordering.
5

---

Building upon Lemma we derive our first main result in Theorem 4.2 We define the finite- dimensional function class of interest F = {0 (0*(o+AAl)) (x)
|0 E +R+ It is easy to verify that all f E F represent the desired propensity shift to (a z = T(a A z) as defined in Eq. 2
<(aa=)
However, note that the optimization problem in Eq. 5 requires knowledge about the true or optimal imputed scores Sn+1- Directly solving the problem this form would require running for all possible imputed values S E IR, i.e., an infinite amount of times. As a remedy, we exploit the strong duality property and present our results in terms of dual problem formulation.
Theorem 4.2 (Conformal prediction intervals for known baseline policy), Consider new dat- apoint with Xa+1 In+1. An+1 On+1, and A* (X a" an+1 + DA Let
T = = + I=--" be the optimal solution to
T(a +
AA 2i)
a(a" In-1) S 0 (an+1 In+1)
min
Σ i=++1
Th S
0
+ m++1
π <( x4)
S S
Vi m
T
(8)
for an imputed unknown Sn41 = S. Furthermore, let S be defined as the maximum S s.t. nhtt 1 - a. Then, the prediction interval
<
C(Int1.a") == E8 Sx+1 S S*}
(9)
satisfies the desired coverage guarantee
(( E
21-a
(10)
Proof. We provide full proof Supplement D.2 Here, we briefly outline the underlying idea of the proof. First, we show that the function class F indeed satisfies Eq. equation 2 for the intervention A" (X) = A+ A,, and we then rewrite Eq. equation 5| convex optimization problem. Next, we exploit the strong duality property. We optimize over the corresponding dual problem to receive dual prediction set with equal coverage probability. Finally, we derive S* from the dual prediction set to construct Ca+1 and prove the overall coverage guarantee.
4.2 Scenario 2: Unknown treatment policy
If the underlying treatment policy is unknown, the only possible intervention is hard intervention a". As described above, measuring the induced propensity shift is non-trivial due to two reasons: (i) The propensity model needs be estimated, which introduces additional uncertainty affecting the validity of the intervals (-+ Challenge D). ). (ii) The density function corresponding to a hard intervention is
given by the Dirac delta function
0, for a a a" 6- (a) =
(1I)
oo, for = a".
which hinders direct adaptation of Theorem 4.2 due to the inherent discontinuity of the improper function. Hence, we make the following assumption on the propensity estimator.
Assumption 1. The estimation error of the propensity function functiooon n(a x) is bounded in the sense that, for all I,.... + 1, there exists M > 0 such that
A(a x:)
Ca. =
E [[M.
(12)
T(a x3)
Under Assumption 1, the distribution shift induced by the intervention is then defined as
6-- (a) #(a x) i(a | x)) = a(a x) Ta x)) ((a x)) = ca (a x)
sa (a)
f(a. x)
TTa x)) Er(f(A, x)
a (t),
(13)
for suitable function f. We further formulate 5,- (a) in terms of a Gaussian function as
(a a²)² (a--a))
8 (a)
lim --
exp
(14)
2
202
This motivates the following lemma. Therein, we specify the class F of tilting functions that represent the distribution shift induced by the hard intervention a".

---

Lemma 4.3. For > O, we define
exp( (a-e")
f(a,z):
Co
20
(15)
V2TT
#(a
(a)
with (r[s(A, Furthermore, we define the finite-dimensioal function class
exp( (a-a-)²
F
Co
20²
0 0<a, M sca Sc.sM..
(16)
V2TO
a(a
Then, [(a, x) E F for all Ca E [,,, M] and o + As result, the distribution shift
exp( (a-a")²
Ca
T(a x)
lim a-0
T x)
(17)
V
7a
r)
can be represented in terms of Eq. equation 2 through functions f E F.
Proof. See Supplement D.1
Following the motivation in scenario 1. we thus aim estimate the (1 (1- quaantile of the non-
conformity scores under the distribution shift in Lemma 4.1 We can reformulate this problem as
min
(1 aju; + avu
0>0, i
SM
i=++1
(a: (a;-a")²
(18)
s.t. S
Co
exp
20
+ U2 =0,
0,
Vi m + 1..., n +1,
a(;
Ug D2 2
Vi m 1.....n+
for the imputed score Sn+1 S. As the score is unknown, computing the CP interval would require solving equation 18 for all S E, yet which is computationally infeasible. Before, exploited properties of the dual optimization problem and the Lagrange multipliers of the convex problem in Theorem 4.2 to efficiently compute the CP intervals. However, the present non-convex problem does not automatically allow for the same simplifications. Instead, we now present remedy for efficient computation of the CP intervals in the following lemma. We prove Lemma 4.4 in Supplement [
Lemma 4.4. The problem equation 78 is Type-I invex and satisfies the linear independence constraint qualification (LJCQ).
Lemma 4.4 allows to derive properties of the present non-convex optimization problem in terms of the Karush-Kull Kuhn-Tucker (KKT) conditions. For this, we note that the fulfillment of the LICQ serves sufficient regularity condition for the KKT hold at any (local) optimum of equation [8 Combined with the Type-I invexity of the objective function and the constraints, the KKT conditions are not only necessary but also sufficient for global optimum. As a result, we can employ the KKT conditions at the optimal v o² and e to derive coverage guarantees of our CP interval in similar fashion as Theorem 4.2 We thus arrive at the following theorem to provide CP intervals for the scenario with unknown propensity scores.
Theorem 4.5 (Conformal prediction intervals for unknown propensity scores), Let u (unti....,.e " E Ra-m 8 e E RR be the optimal solution to S
Su. " "u
n+1 (1-a)u +02
Σ i=++1
min 0>0 SC.SM
(a--n")²
(19)
s.t. S
Co
exp
2n
Uc + 0,
Vi
11
2To
A(a;
Mi. 05
0. Vi +
'We provide an interpretation of the optimal values in Supplement G Therein, we further discuss the implications of the proposed kemel smoothing of 8.- (a).
7

---

for an imputed unknown Su+1 = S. Let S* be defined as the maximun S s.t. 1 > 0. Then,
C(Xn+1,a")= = tu Sn+1ly) S 8
(20)
satisfies the desired coverage guarantee
P(Y(a") E (()
(21)
Proof. See Supplement D.3
In certain applications, it might be beneficial to fix a to a small value dg to approximate da (a) though soft intervention and only construct the CP interval through optimizing over Ca. We present an alternative theorem for this case Supplement[A]
We now use Thm. 4.5 to present algorithm for computing CP intervals of potential outcomes from continuous treatment variables under unknown propensities Alg. | for scenario 1 with known propensities and discuss the computational complexity in Supplement B
We present a similar algorithm
5 Experiments
Baselines: As we have discussed above, there are no baselines that directly compute prediction intervals with finite-sample guarantees for potential outcomes of continuous treatments. Therefore, we compare our method against MC dropout [14] and deep ensemble methods [33]. Yet, we again emphasize that MC dropout is an ad hoc method with poor approximations of the posterior, which is known to give unfaithful intervals [34]. Furthermore, we report the empirical coverage achieved by intervals from a Gaussian process regression (GP). By doing so, we consider method that assesses the underlying aleatoric uncertainty. Additionally, we compare our method against the naive vanilla CP and the method by 36 for binary treatments in Supplements CennntttCCnndd Cadd
Implementation: All methods are implemented with ø as a multi-layer perceptron (MLP) and an MC dropout regularization of rate 0.1. Crucially, we the identical MI.P for both our CP method and MC dropout. Hence, all performance gains must be attributed to the coverage guarantees of conformal method. In the MC dropout baseline, the uncertainty intervals computed via Monte Carlo sampling. In scenario 2 we perform conditional density estimation by conditional normalizing flows 49] Implementation and training details are in Supplement G
We evaluate the methods terms of whether the prediction intervals are faithful (1....21.. That is, compute whether the empirical coverage of the prediction intervals surpasses the threshold - a for different significance levels a E {0.05, 0.1, 0.2}. Additionally, report the width of the resulting intervals in Supplement H
Performance metries:
5.1 Datasets
Synthetic datasets: We follow common practice and evaluate our methods using synthetic datasets (e.g..2125) 25). Due to the fundamental problem of causal inference, counterfactual outcomes are never observable in real-world datasets. Synthetic datasets enable us to access counterfactual outcomes and thus to benchmark methods in terms of whether the computed intervals are faithful. Additionally, we perform experiments on the semi-synthic TCGA dataset in Supplement We hereby show the applicability of our method to high dish-niiensionall real-world data in controlled environment.
Medical dataset: We demonstrate the applicability of our CP method to medical datasets on the MIMIC-III dataset (26). MIMIC-III contains de- identified health records from patients admitted to critical care units at large hospitals. Our goal is to predict patient outcomes in terms of blood pressure when treated with different duration of mechanical ventilation. We use 8 confounders from medical practice (e.g., respiratory rate, hematocrit). Overall, we consider 14,719 patients, split into train (60%), ), validation (10%), calibration (20%), and test (10%) sets. Details are in Supplement G
8

---

5.2 Results for synthetic datasets
0.05
0.1.1
0. 0.2
We consider two synthetic datasets with different propensity scores and outcome functions. Dataset uses a step-wise propensity function and concave outcome function. Dataset 2 is more complex and uses a Gaus- sian propensity function and oscillat- ing outcome functions. Both datasets contain a single discrete confounder, a continuous treatment, and a contin- uous outcome.
Ensemble
MC diopout
Ours
0.0
0.5 Coverage
1.0 0.0
0.5
1.0 0.0
0.5
1.0
Goverage
Coverage
Figure 4: Comparison of faithfulnesss on dataset across 50 runs. Larger values better. For each a, the plots show how often the empirical intervals contain the true outcome. Intervals should ideally yield a coverage of I a (red line).
0.05
By choosing low-dimensional data- sets, we later render it possible to plot the treatment-response -epssee curves
Enswmble
MC
so that one can inspect the predic- tion intervals visually. (We later also show that our method scales to high- dimensional settings as part of the real-world dataset.) Details about the
10
Coverage
Coverage
Coverage
Figure 5: Comparison of faithfulness dataset 2 across 50 runs. Larger values are better.
data generation are in Supplement [
We evaluate the faithfulness of our CP intervals. On each dataset, we analyze the performance of the intervals in the presence of various soft interventions A. E 5, 10) and hard interventions a" E {7x, 10x} for each X = I. We average the empirical coverage across 50 runs with random
seeds. The results are in Fig. 4 and Fig. 5 Additionally, we report the empirical coverage of the GP.
We make the following observations. First, our CP intervals comply with the targeted signifi- cance level a and are therefore faithful. Second, both MC dropout and the deep ensemble method have a considerably lower coverage, implying that the intervals
Coverage
Data
Intervention
ar 0.05
0.1
0.2
10ar
1.00 10.28
0.91 /0.23
0.11
7x
1.09/0002
0.94/0.02
0..50000
faithful. This is in line with the literature, where MC dropout is found produce poor approximations of the poste- rior (34). In particular, the ensemble method con-
a"' 10x
1.00 0.08
0.84 0.00
0.83/ 0.07
Table I: Coverage of the intervals from our CP method/ / MC dropout for various hard interven- tions a" and significance levels a. Intervals with coverage 2 1 a are considered faithful.
highly unfaithful. Thus, we will sider this baseline in all the following experi- ments. Third, our method has only a small variability in terms of empirical coverage, whereas the empirical coverage of MC dropout varies greatly. This corroborates the robustness of our method. Fourth, the results are consistent for both datasets. Fifth, the GP only able to cap- ture the true potential outcome in the prediction intervals for small distribution shifts (A = 1) on dataset 2. However, the empirical coverage is extremely low: a 0.05: 0.125; = 0.1: 0. 0.125; a 0.2: 0.0833. This aligns with our expectations, the aleatoric uncertainty in our experiments is low. Therefore, the GP intervals are small (average width of 0. 1293) and barely valid after the intervention. sum, this demon- strates our CP method's efffcctiveness.
Table presents the empirical coverage of the intervals from our CP method vs. MC dropout across different a and hard interventions a". We observe that our CP intervals are effective and achieve the intended coverage. In contrast, MC dropout does not provide faithful intervals. Our findings are again line with the literature, where MC dropout is found to produce poor
45
Figure 6: CP intervals for multiple significance levels for Dataset 1 with intervention A 5.

---

approximations of the posterior and thus might provide poor coverage [3 We present further results in Supplement[F H
Insights: We plot the intervals across different levels a and covariates X (Fig. 6 allowing us to inspect the intervals visually. We observe that the intervals behave as expected: they become sharper with increasing significance level a. We further see that our CP intervals are slightly wider (see details in Supplement H). yet this is intended as it ensures that the intervals are faithful. Our CP intervals (blue) generally include the true outcome. In contrast, the intervals from MC dropout (orange) often do not include the true outcome (e.g.. see the bottom row Fig. 6 and are thus not faithful.
5.3 Results for the MIMIC dataset
Hira
Boot paomere
Recall that numerically evaluating causal inference meth- ods real-world data is not possible due the fundamen- tal problem of causal inference. Therefore. we provide insights on the MIMIC dataset in Fig. 7 We compare the CP intervals of two male and two female patients of differing ages when treated with increasing duration of mechanical ventilation. Our intervals show higher uncer- tainty in treatment regions rarely included in the training data (high medium to high treatments). The intervals given by MC-dropout are narrower, which suggests lower cover- age, confirming the effectiveness of our proposed method. This finding aligns with our observation from the synthetic datasets.
SMC
Farila
02
Dusaton
Figure 7: CP intervals for potential out- comes of increasing duration of mechan- ical ventilation for 4 exemplary patients.
6 Discussion
Limitations: As with any other method, our UQ method has limitations that offer opportunities for future research. Our method relies the quality of the propensity estimator. Although incorporate estimation errors when constructing intervals, poorly estimated propensities could potentially lead to wide prediction intervals. We acknowledge that our intervals conservative for point interventions and for segments of the output space with limited calibration data, implying that representative calibration dataset is essential for the performance of method. As for all CP methods, the use of sample splitting may reduce data efficiency. Furthermore, we note that the optimization procedure can be computationally expensive for large CATE vectors.
Broader impact:
Our method makes a significant step toward UQ for potential outcomes and, thus, toward reliable decision- decision-making. We provided strong theoretical and empirical evidence that prediction intervals are valid. To this end, our method fills an important demand for using causal ML in medical practice and other cafety-critical applications with limited data.
Considerations for practical application: First, our method is designed for single continuous treatments with a univariate outcome, which is common in dosing (e.g., determining the dosage of wawnnnttrapp/iislliiyyytttttttte
drugs). randomized controlled trials (when the propensity directly apply Theorem 4.2 on top of the trained outcome prediction model
score is known), to construct our CP intervals at target coverage level a. In observational studies, when the propensity score is unknown, we require the practitioner to have sufficient prior knowledge to set bound M on the propensity estimation error. The practitioner should choose this bound with care and rather in conservative way prevent undercoverage. Then, the CP intervals be derived based on Theorem 4.5) We emphasize again that our intervals do not suffer from undercoverage due to limited data, as CP guarantees are valid for any sample size and our method is kept fully model agnostic, enabling the practitioner to make reliable judgements based on limited data and an MI. model of choice.
We presented a novel conformal prediction method for potential outcomes of continuous treatments with sinite-sample guarantees. Our method extends naturally to treatment effects. A key strength of our method is that the intervals are valid under distribution shifts introduced by the treatment assignment, even if the propensity score is unknown and has to be estimated.
Conclusion:
10

---

Acknowledgements
This paper is supported by the DAAD program Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry of Education and Research.
References
[1] A. Alaa and M. van der Schaar. Bayesian inference of individualized treatment effects using multi-task Gaussian processes. In Conference on Neural Information Processing Systems (NeurIPS), 2017.
[2] A. Alaa, Z. Ahmad, and M. van der Laan. Conformal meta-learners for predictive inference of individual treatment effects. In Conference Neural Information Processing Systems (NeurIPS), 2023.
[3] Alvarsson, S. Arvidsson McShane, U. Norinder, and Spjuth. Predicting with confidence: Using conformal prediction in drug discovery. Journal of Pharmaceutical Sciences, 110(1):
42-49, 2021.
[4] A. N. Angelopoulos, S. Bates, A. Fisch, L. Lei, and T. Schuster. Conformal risk control. In International Conference on Learning Representations (ICLR), 2024.
[5] R. F. Barber, E. J. Candès, A. Ramdas, and R. J. Tibshirani. The limits of distribution- conditional predictive inference. Information and Inference: A Journal of the IMA, 10(2): 455 482, 2021.
[6] R. Barber, E. J. Candès, A. Ramdas, and J. Tibshirani. Conformal prediction beyond exchangeability. The Annuls of Statistics, (1),), 2023.
[7] T. T. Cai, M. Low, and Z. Ma. Adaptive confidence bands for nonparametric regression functions. Journal of the Royal Statistical Society Series B: Statistical Methodology, 109: 1054-1070, 2014.
[8] M. Cauchois, S. Gupta, A. Ali, and J. C. Duchi. Robust validation: Confident predictions when distributions shift. Journal of the American Statistical Association, 1 119(54))30033330 3043-3044, 2024.
[9] Z. Chen, R. Guo, J.-F. Ton, and Y. Liu. Conformal counterfactual inference under hidden confounding. In Conference on Knowledge Discovery and Data Mining (KDD), 2024.
[10] M. Eklund, U. Norinder, S. Boyer, and L. Carlsson. The application of conformal prediction to the drug discovery process. Annals of Mathematics and Artificial Intelligence, 14(1-2):117-132
2015.
[11] C. Fannjiang, S. Bates, A. N. Angelopoulos, J. Listgarten, and M. I. Jordan. Conformal prediction under feedback covariate shift for biomolecular design. Proceedings of the National Academy of Sciences of the United States of America, 1 12244)))22045661 19, 2022.
[12] S. Feuerriegel, D. Frauen, V. Melnychuk, J. Schweisthal, K. Hess, A. Curth, S. Bauer, N. Kil- bertus, I. S. Kohane, and M. van der Schaar. Causal machine learning for predicting treatment outcomes, Nature Medicine, 30(4):958-966 2024.
[13] D. Frauen, F. Imrie, A. Curth, V. Melnychuk, S. Feuerriegel, and M. van der Schaar. A neural framework for generalized causal sensitivity analysis. In International Conference on Learning Representations (ICLR), 2024.
[14] Y. Gal and Z. Ghahramani. Dropout as a bayesian approximation: Representing model uncer- tainty in deep learning. In International Conference Machine Learning (ICMI.), 2016.
[15) A. Gendler, T.-W. Weng, L. Daniel, and Y. Romano. Adversally robust conformal prediction. In International Conference on Learning Representations (ICLR), 2022.
[16] S. Ghosh, Y. Shi. Belkhouja, Y. Yan, J. Doppa, and B. Jones. Probabilistically robust conformal prediction. In Conference on Uncertainty in Artificial Intelligence (UAI), 2023.
[17] I. Gibbs and E. J. Candès. Adaptive conformal inference under distribution shift. In Conference on Neural Information Processing Systems (NeurIPS), 2021.
[18] I. Gibbs, J. J. Cherian, and E. J. Candès. Conformal prediction with conditional guarantees. Journal of the Royal Statistical Society Series B: Statistical Methodology, 87(4): 100 1126, 2025.
11

---

[19] L. Guan. Localized conformal prediction: generalized inference framework for conformal prediction. Biometrika, 110(1):33-50, 2023.
[20] M. A. Hanson and Mond. Necessary and sufficient conditions in constrained optimization. Mathematical Programming, 37(1):51-58, 1987.
[21] K. Hess, Melnychuk, D. Frauen, and Feuerriegel. Bayesian neural controlled differ- ential equations for treatment effect estimation. In International Conference Learning Representations (ICLR), 2024.
[22] L. Hill. Bayesian nonparametrie modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1): 2011.
[23] G. W. Imbens. The role of the propensity score in estimating dose-response functions. Biometrika, 87(3):706- 2000.
[24] A. Jesson, S. Mindermann, U. Shalit, and Y. Gal. Identifying causal-effect inference failure with uncertainty-aware models. In Conference on Neural Information Processing Systems (NeurIPS), 2020.
[25] Y. Jin, Z. Ren, and E. J. Candès. Sensitivity analysis of individual treatment effects: A robust conformal inference approach. Proceedings of the National Academy of Sciences of the United States of America, (20(6))c221488))
2023.
[26] A. E. W. Johnson, T. J. Pollard, L. Shen, L.-W. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits. L. A. Celi, and R. G. Mark. MIMIC-III, freelly accessible critical care database. Scientific Data, 3(1):1-9, 2016.
[27] Jonkers, J. Verhaeghe, van Wallendael, L.. Duchateau, and S. van Hoecke. Conformal Monte Carlo meta-learners for predictive inference of individual treatment effects. arXiv preprint, arXiv:2402.04900 2024.
[28] N. Kallus and A. Zhou. Confounding-robust policy improvement. In Conference on Neural Information Processing Systems (NeurIPS), 2018.
[29] E. H. Kemnedy. Nonparametric causal effects based on incremental propensity score interven- tions. Journal of the American Statistical Association, I 114(526):44--556 2019.
[30] D. Kivaranovic, R. Ristl, M. Posch, and H. Leeb. Conformal prediction intervals for the individual treatment effect. arXiv preprint, arXiv:2006.0144 2020.
[31] T. Kneib, A. Silbersdorff, and B. Säfken. Rage against the mean: A review of distributional regression approaches. Econometrics and Statistics, 26(C):99-123, 2023.
[32] R. Koenker and G. Bassett. Regression quantiles. Econometrica, 46(1):33-50, 1978.
[33] B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Conference on Neural Information Processing Systems (NeurIPS), 2017.
[34] L. Le Folgoc, V. Baltatzis, S. Desai, A. Devaraj, S. Ellis, O. E. M. Manzanera, A. Nair, H. Qiu, J. Schnabel, and B. Glocker. Is MC dropout Bayesian? arXiv preprint, 1rXiv:21 2021.
[35] Lei and L. Wasserman. Distribution-fre prediction bands for nonparametric regression. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76(1):71-96, 2014.
[36] L. Lei and E. J. Candès, Conformal inference of counterfactuals and individual treatment effects. Journal of the Royal Statistical Society Series B: Statistical Methodology, 83(5)-9119938 2021.
[37] L. Nagalapatti, A. Iyer, Abir, and S. Sarawagi. Continuous treatment effect estimation using gradient interpolation and kernel smoothing. In Conference on Artificial Intelligence (AAAI), 2024.
[38] L. Nie, M. Ye, Q. Liu, and D. Nicolae. VCNet and functional targeted regularization for learning causal effects of continuous treatments. In International Conference on Learning Representations (ICLR), 2021.
[39] H. Papadopoulos. Inductive confidence machines for regression. In European Conference on Machine Learning (ECMI.), 2002.
[40] Patrick Schwab, Lorenz Linhardt, Stefan Bauer, Joachim M. Buhmann, and Walter Karlen. Learning counterfactual representations for estimating individual dose-response curves. In Conference Artificial Intelligence (AAAI), 2020.
12

---

[41] A. Podkopaev and A. Ramdas. Distribution-free uncertainty quantification for classification under label shift. Conference on Uncertainty in Artificial Intelligence (UAI), 2021.
[42] Y. Romano, E. Patterson, and E. Candès. Conformalized quantile regression. In Conference on Neural Information Processing Systems (NeurIPS), 2019.
[43] Y. Romano, M. Sesia, and E. J. Candès. Classification with valid and adaptive coverage. In Conference Neural Information Processing Systems (NeurIPS), 2020.
[44] D. B. Rubin. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469):322-331 2005.
[45] Schweisthal, D. Frauen, V. Melnychuk, and S. Feuerriegel. Reliable off-policy learning for dosage combinations. In Conference on Neural Information Processing Systems (NeurIPS), 2023.
[46] I. Steinwart and A. Christmann. Estimating conditional quantiles with the help of the pinball loss. Bernoulli, 17(1):211-225, 2011.
[47] M. F. Taufiq, J.-F. Ton, R. Cornish, Y. W. Teh, and A. Doucet. Conformal off-policy prediction in contextual bandits. In Conference Neural Information Processing Systems (NeurIPS), 2022.
[48] R. Tibshirani, covariate shift. In Conference on Neural Information Processing Systems (NeurIPS), 2019,
Barber, E. Candès, and A. Ramdas. Conformal prediction under
[49] B. L. Trippe and R. E. Turner. Conditional density estimation with bayesian normalising flows arXiv preprint, 18X2:1802 2018.
[50] V. Vovk. Conditional validity of inductive conformal predictors. In Asian Conference Machine Learning (ACMI.), 2012.
[51] V. Vovk, A. Gammerman, and G. Shafer. On-line compression modeling I: conformal prediction. In: Algorithmic Learning in a Random World. Springer, Boston, 2005.
[52] B. Wang, Li, and M. Yu. Conformal causal inference for cluster randomized trials: model- robust inference without asymptotic approximations. arXiv preprint, arXiv:2401 1019777 2024.
[53] S. Wang. M. B. A. McDermott, G. Chauhan, M. Ghassemi, M. C. Hughes, and T. Naumann. MIMIC-Extract. In Conference on Health, Inference, and Learning (CHIL.,, 2020.
[54] N. Weinstein, E. A. Collisson, G. B. Mills, K. R. Mills Shaw,
Ozenberger, K. Ellrott, I. Shmulevich, C. Sander, and J. M. Stuart. The cancer genome atlas pan pan-cancer analysis project. Nature Genetics, 45(10):1113-1122 2013.
[55] Y. Yang, A. K. Kuchibhotla, and E. T. Tehetgen. Doubly robust calibration of prediction sets under covariate shift. Journal of the Royal Statistical Society Series B: Statistical Methodology, 86(4):943-98 965,, 2024.
[56] M. Yin, C. Shi, Y. Wang, and D. M. Blei. Conformal sensitivity analysis for individual treatment effects. Journal of the American Statistical Association, 1196545): 2024.
[57] X. Zhan, Z. Wang, M. Yang. Z. Luo, Wang, and G. Li. An electronic nose-based assistive diagnostic prototype for lung cancer detection with conformal prediction. Measurement, 158: 107588, 2020.
[58] Y. Zhang, Shi, and S. Luo. Conformal off-policy prediction. In Conference on Artificial Intelligence and Statistics (AISTATS), 2023.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction are later stated as theorems and proofed in the Appendix.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper discusses the limitations in the end of the main paper well as the Appendix.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer:
[Yes]
Justification:
All results are proven in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The paper discusses the implementation details in 'the Appendix. Furthermore, the paper includes a link to the code for reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
15

---

reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper provides link to an anonymized GitHub repository containing code necessary for reproducing the results in the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nnis.co cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/
/ooeeeee//ooSeSommaiio ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: A short section on the experimental setup is provided in the main paper. More details can be found in the Appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: The paper reports the mean and standard deviation for the empirical coverage.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The computational complexity and compute resources are stated in the Ap- pendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses the broader societal impact of the contribution in the end of the main paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not pose the risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All original owners of code and (real-world) datasets stated in the paper as well as the GitHub repository.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, datasets.paperssiii wwwppppppttwhhoooo thcode www.coo//aaaaoo datasets
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The usage of LLMs not a standard component of the core methods this рaper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
20

---

A Additional theoretical results
A.1 Calculating prediction intervals for further causal quantities and differences
We presented a method for calculating conformal prediction intervals for potential outcomes of continuous treatments. In the following, show how the intervals can be combined to yicld valid prediction intervals for further causal quantities, such as the individual treatment effect (ITE) 7 of treatment
(aa)= := Yi(a) - Y((0).
(22)
Here, we consider the setting in which the non-conformity score chosen to be the absolute residual. Lemma A.I. Let St and S denote the optimal imputed non-conformity scores Sn+1 for treatment a and no trealment at significance level
3 for a (0, 1), respectively. Furthermore, let C+ := o a) + S -
(z4, 0) + S
(23)
C := 6(z;, a) S - 6 (2 0) S
(24)
Then the interval (Xi, = , contains the ITE with probability 1 a.
Proof. Let (a) be the estimation error of the potential outcome, i.e.
<:(a) Y;(a) ((I;; a).
(25)
We can rewrite the coverage guarantee of the original conformal prediction intervals for the potential outcome (a) as
P(Yi(a) E P(eedl)
= P(le((a)) S ( 2
(26)
Now observe that
= P(x((a) E P(((()))))))())))))))))))))))))) Y,(O)) C,(2n,))
(27)
=P((X((a) 2 C + ((()) (Y,(a) S S + +Y(()))
(28)
=P((e((a) 2 E;( (S) + +5)) A (E((a) s (0) + +S + ()))
(29)
=P([c,(a) - -0) S S +
(30)
2P((=;(a)] + (o) s S + +g).
(31)
Thus, it follows directly that
P(n((a) E C,(Xi,a)) 2 a.
(32)
A.2
Alternative scenario 2: Fixing approximation of 5.. (a)
In Section we formulated the unknown propensity shift in terms
(a - aa" a²)²
8 (a) lim
exp
(33)
0 0
/ 2
o2
and minimized over a and Ca in Theorem 4.5 construct the CP intervals. However, in certain
applications, it might be beneficial to control the spread of the approximation of 8a. (a) through fixing
exp (- 12-0
a to a small value do and performing the soft intervention a(a | x)) /2ndo case, the resulting optimization problem is convex problem similar Theorem alternative optimization problem below.
In this
We present the
Theorem A.2 (Alternative for Theorem
4.5 Conformal prediction intervals for unknown propen- sity scores). Let a new datapoint be given with Xa+1 = En+1 and Ant1 = anti- im ..... n E 1=*+1 be the optimal solution
Let
(0=-0")2)
min m1 ar Sea
max Tis
Σ n
S
Ca
exp
o
+ M/+1 (s
Co
2
i(a;
v 2 # a(a; Int1)
Tm
S.L.
a S 1i S a,
Vi 1,..., n +1,
(34)
21

---

for an imputed unknown Su+1 = S. Furthermore, let S be defined as the maximum S s.t. Bh <
1 a. Then, the prediction interval
C(Ent1,a") == {v (n+ 55*)
(35)
satisfies the desired coverage guarantee
(a) E
(36)
where with a slight abuse of notation (a)) denotes the potential outcome under the soft intervention
ग. above.
Proof.
The statement follows from Theorem 4.5
A.3
Soft-interventions on estimated propensities
In the main paper, we presented algorithms for constructing prediction intervals for soft interventions unknown. These are arguably
the propensity function known and hard interventions the most common scenarios in practice. However, in some cases, one might also be interested in the effect of soft interventions on estimated propensity scores [e.g.. ? J. Therefore, we present an alternative theorem for calculating conformal prediction intervals under soft interventions with estimated propensity scores below.
Theorem A.3 (Conformal prediction intervals for soft interventions with unknown propensity scores). Let a new datapoint be given with X Int1 and An+1 an++1. Furthermore, let & denote the estimated propensity score with estimation error bounded by [ M). M > The soft intervention
represented by the shift given through A E R. Let 17 = {m th+ E B 1 solution to
be the optimal
Cañ((a + A x31)
Ca Cat(ant1 + A In+1)
min Se.
8
+ Mht1
S
S.
A(a; 174)
# (ant1 In+1)
1=1
S.f.
13 S 1 a, Vi 1,
(37)
for an imputed unknown Sat1 = Furthermore, let S be defined as the maximum S s.s. n <
1 a. Then, the prediction interval
C(In+1, = tyr Sn+1(y) S S*}
(38)
satisfies the desired coverage guarantee
(aY C(Xx+1,a)) 1-
(39)
where with slight abuse of notation Y (a') denotes the potential outcome under the soft intervention represented by A.
Proof. The statement follows from Theorem 4.2 and Theorem 4.5
22

---

B
Algorithm
We now use Thm. 4.5 to present an algorithm for computing CP intervals of potential outcomes from continuous treatment variables under unknown propensities in Alg. | We present a similar algorithm for scenario ! with known propensities and discuss the computational complexity in below.
We make the following comments: In our algorithm, optimization solver is used to calculate + according to Theorem 4.5, The specific choice of the solver is left to the user. In our experiments in Section | perform the optimization via interior point methods. Further, the overall goal of our algorithm is to find the optimal imputed non-conformity score 5 such that the coverage guarantees hold. It can be implemented through suitable iterative search algorithms.
Algorithm 1: Algorithm for computing CP intervals of potential outcomes of continuous interven-
tions under unknown propensities.
Input: Calibration data (X,, A, Yi)ke (m.+.......) new sample Xn+1 and intervention a", significance level a, prediction model Φ. propensity estimator ât, assumed bound M, error tolerance s, optimization solver
Output: CP interval Ca+1 for a new test sample
Sup <- max (aax;==1===1,... St, 13; Stow t- min (miil) S., 1};
2.so 2 up t solver(o, (Xe, A., (.)i Xmf1, M, Sup):
/* Calculate un+1
2e41 t solver((Xi,, As, Yi) (he((m)), (m+1 Xati,a', a, M,, Slow)
Iterative search for S
4 while unt1 > do
Sup < 2Sup; F++1 solver(o), (X., Ai, ()) Xu41, M.S)..
Sup);
7 end
8 while Jow = 0 do
Stow f 0.5.yowi
10
low cit t solver(d, n,, (Xt, Ar,
a', M, Stow):
end
S* <- SuptSrom
2
while Sap Stow > E do
14
e < solver(d) n, (Xe, Y.)ig(mtt X.411 Xut1, a, M,S*);
if 5' > then
15
Slow + S.p
16
2
end
18
S.
Sup + Ston
20
end
S*
Sup Shaw
2
22
end
/* Compute C(Xnt1,a))
return C(Xn+1,a") = [v] (y) SS3
Algorithm explanation: Theorem 4.5 requires knowledge of the optimal imputed non-conformity score S*. Since is unknown beforehand, we implement Algorithm[[e Tas a binary search algorithm. We find suitable upper and lower bounds for S* (until line 8). To validate that the bounds are indeed smaller/larger than the optimal non-conformity score, we observe the corresponding dual value # (Theorem 4.5 After finding valid bounds, we start searching for S* via standard binary search, continuously ly increasing the lower and decreasing the upper bound. When the difference between the bounds is less than specified error tolerance e, we stop and take S" to be the mean of the interval. In every iteration (when calling 'solver'), ), we make use of the optimization Theorem 4.3 and check whether the dual value fulfills the requirement in the Theorem.
Below, we state a second algorithm that is applicable the propensity score is known. In this case, convex optimization solver can be used.
23

---

Algorithm 2: Algorithm for computing CP intervals of potential outcomes of continuous interventions under known propensities.
Input: Calihration data (X,, A,, (m))(( new sample Xa+1 and soft intervention A (Xn+1).
significance level a, prediction model @, error tolerance optimization solver
Output: CP interval Cu+1 for new test sample
Sup <- { mmxx
St. 1); Stow t- min (uint=m+1,.... S., -1]:
/* Calculate 1h Ma41, where o is the optimal solution Eq.
M641 t solver(Ø,, ft (Xi, A., (m) X041, A" a, low
3 Mn41 <- solver( As, (mmtt........ X (X++1)) a, Stow);
/* Iterative search for S
while 1/
do
5
Sup <- 2Sup;
6
Mo f solver(o, n. (Xi, A., (n) Xmt1, (Xn41), Sup):
up
end
while WArT >= do
Stow + 0.5Showi
Th solver( (Xi, As, x) a, Stow):
11 end
S* f- SuptSuew
12
2
while Sup Stow > & do
Mh solver((), (X,, As, x)
if mh+ < a then
Slow f- +.por
17
end else
18
19
Sup <
Sup Stow 2
end
S
Sup + +.ow
2
end
/+ Compute C(X,+1, (Xm+1))
return C(X,+1, A (Xn+1)) [wl (y) ST)
Computational complexity: The complexity of running our algorithms depends heavily on the em- ployed optimization solver with complexity a, (n..) (e.g., polynomial complexity for suitable convex solvers) and the size of the calibration dataset ne- This might become costly for large-scale calibration datasets in practice. The outer algorithm has time complexity of at most Oloog( SSp 1).
Overall, our algorithm has a fixed complexity of (log( Sup- Stow ( (ne)). The complexity of deriv- ing intervals through MC-dropout ensemble methods depends, however, samples or models, respectively. The latter thus scales with the precision of the intervals, which might be difficult to control. Furthermore, therefore not directly comparable. While the theoretical runtime of CP may exhibit more complex scaling behavior (e.g., cubic or non-linear), our empirical results demonstrate that CP scales well in practice. our work, we use optimization as a tool to provide conformal prediction intervals. Future research should focus developing more efficient optimization algorithms for this task.
the number of MC
emphasize that MC intervals are generally not faithful and
Note on the stability of the optimization algorithm: Scenario 1, the only source of potential instability can be very low propensity score in low-overlap regions of the covariate space. This, however, only a problem if (a]x)) << T + A|r). We can thus consider this unlikely in practice. For example, consider a patient who would be treated with dosage of 10mg of some medication, which is prescribed in the range from to 50mg. practitioner is likely to be interested the effect of increase of the dosage to 15mg (i.e., A 5) in contrast to increase to 50mg. Furthermore, reasonable to assume the propensity function to be locally smooth. Therefore, the instability of a (alar) << (a + A|z) is unlikely to occur.
A suitable solver refers solver designed to handle the constrained convex problem in our theorem
24

---

In Scenario could additionally face underflow of (=p(-"===a)) or blow-up of the pre- factor (or(x;)) As remedy, can reparameterize the problem work in the log domain and only exponentiate after subtracting a stable (maximum) constant across all datapoints. This additionally prevents the Jacobian/Hessian of the constraints from becoming nearly singular or wildly varying, causing Newton-type steps to blow up or stall.
25
25

---

C Semi-synthetic experiments
To underline the effectiveness of our method, we perform additional experiments on the semi-synthetic TCGA dataset. The Cancer Genome Atlas (TCGA) dataset [54] consists of a comprehensive and diverse collection of gene expression data. The data was collected from patients with different cancer types. In our experiment, we consider the gene expression measurements of the 4,000 genes with the highest variability, which we employ as our features X. The study cohort consisted of a total of 9659 patients. We model continuous treatment based on the sum of the 10 covariates with the highest variance and assign a treatment effect that is constant the sum of the covariates. Specifically, we model the treatment to follow normal distribution centered at 100 of the 10 covariate values, and the outcome to follow a normal distribution centered at the sum of the treatment and the covariate sum times 100.
As in the main paper, we construct CP intervals for different interventions and confidence levels a. We state the empirical coverage of our method in Table 2 as well as the coverage of the intervals returned by the ensemble method and MC dropout below. The prediction performance of the trained model on the hold-out test dataset is reported. We find that our method is highly effective.
Table 2: Coverage of the intervals from our CP method as well the ensemble method and MC dropout on the TCGA dataset. We report the mean followed by the standard deviation in apprentices.
Confidence level
Intervention
Method
a = 0.05
a 0.1
a = 0.2
Ensemble
0.0640 (0.0445)
0.0600 (0.0379)
0.0480 (0.0412)
A = 0.5
MC
0.8280 (0.0795)
0.8160 (0.0783)
0.8040 (0.0741)
Ours
0.9680 (0.0324)
0.8920 (0.0391)
0.8040 (0.0741)
Ensemble
0,0880 (0.0483)
0.0680 (0.0371)
0.0520 (0.0348)
A 1.0
MC
0.8560 (0.0612)
0.8520 (0.0614)
0.8400 (0.0657)
Ours
0.9733 (0.0377)
0.9500 (0.0500)
0.7667 (0.0618)
Ensemble
0.0720 (0.0411)
0.0680 (0.0412)
0.0640 (0.0389)
A = 1.5
MC
0,7880 (0.0815)
0,8040 (0.0925)
0,8280 (0.0786)
Ours
0.9400 (0.0438)
0.8920 (0.0699)
0,8200 (0.0619)
We observe that our method consistently achieves the desired coverage. To evaluate the usefulness of intervals, we also report the interval width in Table 3 below. The range of the outcomes was 2.0.
Table 3: Width of the intervals from our CP method on the TCGA dataset. We report the mean followed by the standard deviation in apprentices.
Confidence level
Intervention
a = 0.05
a = 0.1
a = 0.2
A 0.5
0.1003 (0.0331)
0.0843 (0.0252)
0.0683 (0.0169)
A 1.0
0.1017 (0.0420)
0.0877 (0.0349)
0.0642 (0.0200)
A 1.5
0.0930 (0.0272)
0.0822 (0.0260)
0.0674 (0.0180)
Comparison the CP methods for binary treatments 36
To investigate whether simple weighted CP methods for causal tasks on binary treatments sufficiently address the non-exchangeability due the distribution shift induced by the intervention, we compare our method with the method by Lei and Candès [36]. Of note, both methods fulfill the coverage guarantees. However, when comparing the interval width, we see that our method is superior: Our intervals have an average width of 0.6255 (sd 0.1714). In contrast, the intervals on the binarized treatment obtained through the method by Lei and Candès (36] have an average width of 3.2876 (sd 0.5587). Hence, the intervals by our method are by far more informative.
26

---

D Proofs
D.1 Proofs of the supporting lemmas
In the following, we prove Lemma 4.3 and Lemma 4.4 from our main paper.
Proof of Lemma 4.3 Recall the definition of the hard intervention
(a)
a(a
xr) (a
a(a x)) = d.- (a)
x)
(40)
a(a x)
((a
xr)
where
exp (- (a
aa-a"²²
(a) = lim
(41)
a v 2TT
σ²
Under Assumption 1, have
a x)) <(a x
1
Ca
.M]
(42)
M
for some M > 0 and all a, x. Then
exp (a-a")2
(a-a")a
1
exp
lim 0-0 2TT
M S (1a)
lim
M.
(43)
a(a xx) Therefore, the distribution shift induced by the hard intervention can be represented as
ia z)
0-0
V
(a-a*)
(a-a")²
exp
exp
0 << 0. E E
f(, = lim 0
#(a ( EF =
M]
12To
2
#(a
x)
M
(44)
Proof of Lemma 4.4
We first prove that the problem equation 18 fulfills the linear independence
constraint qualifications. For all i = m + 1,..., n + 1, we denote the constraints of problem equation 18
hu U, Ca. o) = S us + Us
Ca
exp
2a
(45)
2
*(a;,
The gradient of h2 is given by
[1, U, Ca.
Oum+1
Ohi
U1 U. O)
0om+1
ah,
u, U, Car σ)
o)
Du;
(46)
Oh;
u, U. Ca.
Thi
exp( (a
Ica
(u, U, Ca. 0)
V2na
#(a;,=))
12
Ohy
exp(
Go -a
u, U, Cas 0)
V2
2/2
do
Therefore, with Vh (Vhm41..... Vhn+1) and gn+1 we obtain
Vh == + A= 2=0ER
(47)
27

---

As result, the constraints are linearly independent. This property suffices for the KKT conditions to hold at any (local) optimum of equation To furthermore show that the KKT conditions are also sufficient for glohal optimum, we show that equation [8] is Type-I invex. An optimization problem
with objective function f(x) and constraints g(x) <= 0 with E Ra+1 is Type-I invex at zo,
there exists (0) E IR++1 such that
f(x) f(00) 2 v(x, 20) Vf(oo)
(48)
and
-g(0)) 2 u(x ro) VVg(o))))))
(49)
20) In problem equation| 18 the gradients of the objective function and of each constraint h; for all j = m ...... n + 1 at Zo are given by
Bobj uo, vo, Cao: (o) 1 at,
8obj( un, vo, Cao aT) az
(50)
Ou, θobj ug. Uos Coo: 00)
Ov,
dobj( uo, co. Cao oo)
0
(51)
dca
do
Vi = m + 1,...,... 12 + 1 and
0h
-1, for i = j,
Oh,
{1, for i = j. (0 else,
(52)
Ju.
40,00,000 d0
0,
else,
ao. ww. d0
(o, "2
0h
cxp
200
(53)
0ca
20. do
VTTO
* *(ai, xi:)
(a: (a:-a")
Oh,
exp
200
Cap
Cao (ai a²)3
(54)
A(a;; xz4)
V2ma
2v 2/2
For
n(uu, Ca. .), (uo, to, Cags oo)) (- uo,
.........
UO0+1 U0
VO.+1 Can 0)²
(55)
the definition of Type-I invexity holds for equation 18 for global optimum.
Thus, the KKT conditions are also sufficient
D.2 Proof of Theorem
three steps: (i) We show that function class (0 ( | R+} <(al=)
We prove Theorem indeed satisfies Eq. equation 2 for the intervention a" = a + A A and rewrite Eq. equation as convex optimization problem. (ii) We retrieve the corresponding dual problem. derive dual prediction set, and show the equality of the coverage guarantee of the dual and the primal prediction construct Cu+1 and prove the overall coverage
sets. (iii) We derive S from the dual prediction guarantee. For further theoretical background on the idea of the proof, we refer to Gibbs et al. [8]
Justification of the distribution shift Observe that (A, 0 for all f F
(0 ((m) 0 E t.. Therefore, Eq. equation simplifies to
x(a(=)
#(a, x) T(a + A x)) T(a x) =<(a + AA
(56)
(a x))
Thus, FF satisfies the propensity shift from Eq. equation for the soft intervention a" + AA- Following Lemma4.1 we thus aim to find
Ās arg 1
T(a, +
AA xz)
a(a" 2n+1) 83) 33) (57) St) 10
Σ
La
0>0 n m
n (ai
x.))
n (an+1 Ønt1)
28

---

Dual problem formulation First, we rewrite the primal problem as
n+1
min 820
Σ +
(1 - a-u; a02
S. T Taa; + A Ii)
(58)
s.t.
- + D4 0, vi= m -
n n n
11,1 = S
(a; Zi
U1 U 2 0,
Vi m 1....
For reference,
18
The Lagrangian of the primal problem states
n++
n+1
T(a AA xz)
n+1
L a)a any
Σ
Ti
S
T(a xz)
Σ
(n, u4 + 12.
1=++1
1=++1
1=m+1
(59)
Setting derivative of L w.r.t UG and Us 0 results in
0C
(1 a)) Th. 71 0,
Vi ..........
(60)
0u,
0C
(1 - 77 72. 1 0,
Vi = m + 1.....
(61)
Dc,
Since n, 72, 2 0 Vi, it follows for all = + 1..... that
(1 a) 7/ 00 and 13
= -
S mi S a
(62)
Therefore, the dual problem is formulated as
Σ Thi S 0 ((a + AA xz)
5 Taa" In+1)
min 850
7/+
(a; xi))
ग. (ax+1 In+1)
i=m-1
s.t.
171 az,
vi 1 ,
(63)
Coverage guarantee Recall from Lemma 4.1 that, for
la T(a + Aa 2.) xt (a; x:)
x(a" In41) 7(an4 #n+1)
@ (y) arg min
S:) la
(64)
0>0
m
$=++1
we can construct Cn+1 tu i S QS... (1)) to achieve the desired coverage guarantee
Pa(Y(a") E C(x++1, A' ('(x)+1)) 2 --a a.
(65)
It is infeasible to calculate $s (v) directly. Therefore, we optimize the dual problem to receive C(Xn+i, a²) = tu (1) S*}
(66)
with S* the maximum S, s.t. for T maximizing the dual problem, 7 < -1 Hence, is left show that replacing Ūs..) (v) by S" in C does not change to coverage guarantee.
To do so, we fix some / > 0 to obtain a specific f(a, = n primal optimal solution. Recall the Lagrangian
Let ğ(a, E F denote the *(alz)-
n+1
n+1
n+1
(1- a)w + 00% +
msS flai, us vs)
Σ i=m
(m. + 32.0))
(67)
i=m+1
1=m+1
Deriving wrt. yields the stationarity condition
0 Σ
TH
2.)
(68)
$=
Σ
2a))
Σ
n f(ai
Σ
n f(s,
(69)
S.sg(a..z))
8,>3(a,.2.)
8,=0(ar,))
29

---

The complementary slackness
Kuhn-Tucker conditions yield
if S: < Iz); if S; ((a
o
L al
(70)
1-aa az
S; > T.),
Therefore, we can rewrite the equation from above as
0
Σ
af(a
Σ (1 (1))))
E
Ti f(a,z))
(71)
S.=((anth))
= Σ af(a,,z))= - Σ (1 (1- [1=,))(a;;)))
Σ
(a (a,,
(72)
of <1-a
=1--a
<1-a;
S.=8(ar,)))
n+1
Σ (a (()
Σ <I-a. S,=8(a,.))))
(a (
(73)
1=++1
Before deriving the coverage guarantee from the stationarity condition, we state the following lemma to underline the definition of S*
Lemma D.1 (Gibbs al. 18 maximizing
The mapping nhtt is non-decreasing in S for all tht1
n
min ne. n+ gE
max
(s - gg(a;, 7m---1 1S g(n+ In+1))
(74)
1// $ Vi= 1, ,....,
for non-negative function classes F.
To prove final coverage guarantee, we observe that
(1 (1))
(75)
= [l (( ( 2off(on+1:2+++)(ee -1((()=))))))
(76)
(77)
With the definition of S as the maximum optimizer nht with n < and Lemma D.1 follows that
[lf(ant1, In (n - [(()) = E((a - 1 a
(78)
and, by exchangeability of (f(a;, x;) (.,, S.), that 2;), n m
Bl(a- 1 ((a
(a 1 -1 -1)
(79)
imm+1
E
Σ
(a + mh (f((
(80)
n
<11-a S,=0(a,,2))
Since is positive and n) E [-a, a), it follows
11 1.0)] - -(1 20
(81)
and thus
(a") CCn2 A (Xn+z2)) >
(82)
30

---

D.3 Proof of Theorem 4.5
We follow the same outline as in the proof of Theorem 4.2 Section D.2 In Lemma 4.3 we
motivated the functional class of distribution shifts. Therefore, it is left to prove the coverage guarantee of C++1-
Key to our proof is the following lemma.
Lemma D.2. The mapping S - + is non-increasing Sor all g¹ (x, a) minimizing
u
n+1
min
Σ
(1 11-a)a a)u; 0.0;
SEF
(83)
1=m+1
s.l.
(.) - - Vi = m +1 m
for non-negative function classes F and imputed Sn+1 S stemming from non-negative conformity score function (e.g, the residual of the prediction).
Proof. Assume for contradiction that there exists > S such that S.$ n+1 > u Then
(5 5)
1 O.
(84)
We observe that
S(S 9d (In+1 , (n+1) S v S(5 و (m+1)) un+1 + unt1)
(85)
Reformulating the equation above yields
- (1 1)
(86)
(In+1, Suăt1 Sun+ + (In+1, ( + Su R11 Se
(87)
< S(u unti + g (IntI1 an+1) 8 (In+1, an41) (v m
(88)
S(S-5)) 5).
(89)
This is equivalent to
(S - () v < S(S S)
(90)
u unt1 > S220 0,
(91)
which contradicts the assumption that nn+11 v
Coverage guarantees. As in D.2 fix some
0 and Ca [
M] to obtain specific
f(a,z) == 2
2=2
We further denote ~(a, x) E F the optimal solution given by the
A(a; les)
optimal values ā and Ca.
With the definition of S* as the minimum S such that 1 = 0 and Lemma D.2 we now can state
1,0
(1 -1-o0 a))] ([f 2, 1) (6 So] (1 aa)]]
(92)
= [B]f(ant, In+1) =0))1
(93)
and, by exchangeability of (f(a;, Ii), âs (a,, S.), that
(afti, (a 1 1055_ up]
=
E
((ant1, 1105
(94)
nt
m
i=m+1
1
E
Σ
x)) ( a))((,
(95)
71 m
oo >0
=f=0
F
Σ
af(a).z))
Σ
- fass,
71 m
5,20(a.2.)
(96)
31

---

Deriving the Lagrangian above wrt. f yields the stationarity condition
Th x()
(97)
i=++1
Th (ai, +
Σ
Th f(a, xz)
E
Th (ai,
(98)
The complementary slackness KauuuhhKKun-Tuune conditions yield
if S: < (();, if S; T.), if S; > ((),
a - a]. 1-0, a2
(99)
Therefore, receive
H001aa 102,,
E
Σ af(a;,
Σ )1 f)(()))))))
(100)
m²=1-a
IF
Σ (a+m))
(101)
m
"i ki-1, 8:=0(amm)))
Since f is positive and E [-a, 1 a]. it follows
Blfaa.1...::1 - [1-a)]] - 20
(102)
and thus
Pa(Y(a") C(Xnt1,a')) 21-a.
(103)
L
32

---

E
Additional background
E.1 Extended literature review
Uncertainty quantification for causal quantities
There exist various methods for uncertainty quantification of causal quantities. These are often based on Bayesian methods Ve.g..(22 22 (24). However, Bayesian methods require the specification of a prior distribution based on domain knowledge and are thus neither robust model misspecification nor generalizable to model model-agnostie aggnostic machine learning models. Other methods only provide asymptotic guarantees [e.g.. 25.225] 27). The strength of conformal prediction, however, is to provide unite-sample uncertainty guarantees.
In the following, we present related work on CP for causal quantities in more detail.
Recently, Alaa al. 2 provided predictive intervals for CATE meta-learners under the assumption of full knowledge of the propensity score. As an extension, Jonkers et al. [27] proposed Monte- Carlo sampling approach to receive less conservative intervals. Chen al. provide prediction intervals for counterfactual outcomes. However, the proposed method requires access to additional interventional data and thus not applicable to real-world applications on observational data. All methods are restricted to binary treatments.
Other works focus on prediction intervals for off-policy prediction 47 58 and conformal sensitivity analysis [56], thus neglecting estimation errors arising from propensity or weight estimation or for randomized control trials B0) Wang et al. 52 constructed intervals with treatment- conniiiiono coverage of discrete treatments. Aiming for couppconniiional coverage, Wang et al. [52] adapted CP to cluster randomized trials. Nevertheless, the method only applies to a finite number of treatments and thus is not applicable to continuous treatments.
Lei and Candès (36 consider the estimated propensity by incorporating the estimation error TV-distance term in the coverage guarantees. However, for large TV-distances (close to 1), the proposed method can only construct intervals with a very limited coverage a E (0, 1 TV)).. Hence, the method is not suitable for applications in medical practice. Our method, however, can also construct intervals with high coverage guarantees for high estimation errors. An increased error will widen the prediction intervals instead of reducing the coverage guarantee. We consider our approach more suitable for medical practice, as one can visually inspect the intervals and decide on the suitability of the task at hand.
Overall, no method can provide exact intervals for continuous treatments. Especially, no method considers the error arising from propensity estimation in the analysis.
Conformal prediction under covariate shift Multiple works CP with marginal coverage under distribution shifts between training and test data have been introduced in the literature [c.g.. [ [5]
[6] [7 18
36 41 48 (55.. Our setting also involves distribution shift due to the intervention on the treatment but differs from the latter in that the true distribution shift is unknown.
19
Gibbs et al. 18 introduced an approach to derive CP intervals under unknown distribution shifts. It proves valid finite-samin sample prediction intervals for all distribution shifts in a finite-dimenas dimensionn function class. However, the approach does not directly apply to causal inference settings. Nevertheless, our framework builds upon the work by Gibbs et al. [88 in that we re-frame the proposed approach to apply to the distribution shift induced through the intervention in causal effect estimation. In this setting, the distribution shift is captured by the shift of the propensity function. Adapting Gibbs et al. [18 to causal inference setting requires carefully addressing the underlying challenges that come from computing CP intervals in a causal inference setting (c.g., propensity score estimation, hard/soft interventions). which we regard as our main novelty and which is of immediate practical relevance (e.g., in personalized medicine).
E.2
The need for exchangeability in CP
Coverage guarantees of existing CP intervals essentially rely on the exchangeability of the non- conformity scores. Exchangeability assures that the nonconformity score of the test point n + equally likely to fall anywhere among the calibration scores, its rank is uniform, and that uniformity
33

---

is exactly what yields the distribution-Fi free coverage guarantee, Without exchangeability, the rank is guaranteed be uniform, and the marginal coverage bound can fail.
However, intervening on treatment A shifts the propensity function and, therefore, induces a shift in the covariates between calibration and test data, specifically in treatment A. Therefore, exchangeabil- ity is not fulfilled, and the coverage guarantees might fail.
As remedy, we present novel and powerful remedy in our work: The overall distribution of the confounders X is assumed to stay constant between train, calibration, and test data (as standard in ML problems). This is completely orthogonal to constructing intervals for different (e.g., young or old) patients. Note that CP intervals are constructed for only one sample/patient at a time. This means that different intervals are constructed for patients with different features X. In other words, the intervals are constructed conditionally on X, but the coverage guarantee is marginal across the complete population of X. Overall, the shift from patient another does pose any challenges for CP methods.
34

---

F Extended discussion
E.1 Discussion on the tightness of our CP intervals:
Our method builds upon the idea of CP to provide finite-sample coverage guarantees. Notably, standard CP does not provide intervals that are proven to be sharp. To our knowledge, there is no method that provides sharp/the tightest possible intervals for potential outcomes. Exploring the tightness of our CP intervals is an interesting and important direction for future research.
In practice, it is possible to observe the width, and thus the informativeness for decision-making, of the intervals. However, coverage guarantees cannot be observed. Therefore, we help the decision- maker by providing valid intervals. The decision-make maker has to decide, case by case, if the returned intervals are beneficial for the problem at hand. Note that this aspect of the informativeness of intervals holds true for any uncertainty quantification method (including those to be proven to be sharp).
F.2 A note on challenges and difficulties in CP for causal effects of continuous treatments
Existing works on conformal prediction for binary or (low-dimensional) discrete treatment are commonly based on (a) weighted conformal prediction 48 or (b) conformal prediction local coverage guarantees [35]. The first approach provides marginal coverage under a distribution shift through reweighting. It requires computing the weights based on the probability of treatment A However, for continuous treatments, this is always zero. Although applicable to binary or low-dimensional discrete treatments [e.g.. 36). this weighting approach cannot be extended similarly to continuous treatments. Furthermore, the propensity of a continuous treatment given by the Dirac delta function 5a would require us restrict the calibration to data samples of the specific treatment, which are extremely rare or even might be missing. Therefore, the calibration step cannot be employed in our setting. The second approach provides treatment group-conditionns coverage. Although again possible for binary or low- oow-dimensional treatments, this approach does not apply to continuous treatments as no treatment groups can be defined. Instead, we propose novel method for conformal predictions that circumvents the above problems and is carefully tailored to continuous treatments.
F33 Causal effects of continuous treatments & kernel smoothing
Causal inference becomes challenging with continuous treatments primarily due to the infinite number of potential outcomes per sample, from which only one outcome observed. Continuous treatments thus result in causal effects that are generally represented by curves (called dose-response curves) This is unlike binary treatment, where the causal effects are represented by a single discrete value.
29
For continuous treatments, the dose-response curves are typically assumed to fulfill some smoothness criterion [e.g., 40 45 Hence, when estimating treatment effects, interpolation and kernel smoothing of the outcome function are commonly employed [e.g., 29 37).
Underlying causal estimation with continuous treatments the generalized propensity score [23]. is defined as the conditional probability of receiving treatment a" given the covariates X under the following regularity conditions: (i) For each i, Y:(a), æi, A, are defined on a common probability space; (ii) A, is continuously distributed with respect to the Lebesgue measure; and (iii) Y; Y((,)) is a well-defined random variable.
Approximating the density da- (a) of the hard intervention a" through a Gaussian kernel follows directly from the definition of 8 (a) as the limit of such kernel. This is also common in the literature [e.g..28] Importantly, we note that we do not directly approximate the potential outcome Y(a") )) (but only the propensity scores). Thus, we do not have variance trade-off of the estimated outcome. Due to the smoothness of the dose-response curve, it is now valid to employ observed samples within a treatment region of a² defined by a to construct the intervals. We note that the importance of the samples is weighted by the inverse distance of the sample to a" in treatment space. We give further intuition on the relationship between a, the importance of observational samples, and the prediction interval width in the following.
35

---

F.4 Interpretation of optimal parameters
To obtain CP intervals under an unknown distribution shift, we approximate the Dirac-delta distribu- tion representing the hard intervention by Gaussian function as
exp (- (a-aa)² 2x²
oe- (a) = lim a v 2
(104)
thus optimize over distribution shit-calibrated non-conformity scores. The optimal parameter o² represents a trade-off between the uncertainty in the prediction and the uncertainty in the interval construction: small o" resembles the propensity of the hard intervention best. Thus. with sufficient or even infinite data close to a", we could construct the narrowest CP interval. However, the smaller a, the less data close to a" will be available to calculate the prediction interval in practice. As a result, many calibration data samples will be strongly perturbed during the calculation, which increases the uncertainty and, thus, the interval size.
In Theorem 4.5
and Co [4 .M] obtain the a1- -quantileee of the
The parameter ea allows us incorporate the estimation error in the propensity score. represents a weighting of the propensity shift such that the (1 -equntleeeee of the non-conformity scores is increased with higher estimation error.
E,5 Interpretation of parameter M
Our optimization requires the specification of a parameter M, denoting a bound on the propensity estimation error. One view the parameter M as type of sensitivity parameter. Therefore, follow former work in causal inference and propose to incorporate domain knowledge to specify the parameter M [e.g.. 13? Another way of making intervals change for varying M. This indicates how much effect the propensity misspecification has on the prediction interval and can help in making reliable decisions. A third option to calibrate M is to employ measures for epistemic uncertainty on top of the propensity estimate when there is no domain knowledge for specifying M.
of the parameter M is observe how the
F.6 A note the stability of method
In our experiments, one can observe some instability for certain privacy budgets and intervention combinations. This is likely due the fact that the CP coverage guarantees only marginal. Therefore, we might experience under- or over-coverage. However, we note that across all runs, our method, average, achieves the desired coverage. These instabilities only occur in single settings. Furthermore, the variance in coverage of our method is much lower than the coverage variance of the baselines, A more in-depth analysis of the stability of the proposed method is left for future work.
36

---

G Experimentation details
G.1 Synthetic dataset generation
We consider two different propensity and outcome functions. In each setting, we assign two types of interventions: known propensity shift of Á 1,5, 10, i.e., three soft interventions a* + A, and the point interventions a" E (1x, 5x, 10, given the confounder X Z.
We generate synthetic datasets for each setting. Specifically, we draw each 2000 train, 1000 calibration, and each 1000 test samples per intervention from the following structural equations. Dataset is given by
X Uniform[1, [1,4] (integer)
~ p Uniform[0. 5X) + (1 p) Uniform||5x 15,440 p P~ Bernoulli(O.3)
Y ~ sin
6 (0.1A - + Normal(O,0... 0.1),
and dataset by
~ Uniform(1, [1.4] (integer)
A (5x,
Y sin (" (0.1A - -0.x)) + N0rmal(0,0.1),
G.2 Medical dataset
We use the MIMIC-III dataset 26 which includes electronic health records (EHRs) from patients admitted intensive care units. From this dataset, we extract confounders (heart rate, sodium, blood pressure, glucose, hematocrit, respiratory rate, age, gender) and continuous treatment (mechanical ventilation) using an open-source preprocessing pipeline [53] From each patient trajectory in the EHRs, we sample random time points and average the value of each variable over the ten hours before the sampled time point. We define the variable blood pressure after treatment as the outcome, for which we additionally apply transformation to be more dependent on the treatment and less on the blood pressure before treatment. We remove all patients (samples) with missing values and outliers from the dataset. Outliers defined samples with values smaller than the 0.1th percentile or larger than the 99,9th percentile the corresponding variable. The final datasct contains 14719 samples, which we split into train (60%), val (10%), calibration (20%), and test (10%) sets.
G.3 Implementation details
Our experiments are implemented in PyTorch Lightning. We provide our code in our GitHub repository. All experiments cores and 32GB RAM.
AMD Ryzen 7 PRO 6850U 2.70 GHz CPU with eight
We limited the experiments to standard multi-layer perception (MLP) regression models, consisting of three layers of width 16 with ReLu activation function and MC dropout at rate of 0. optimized via Adam. We did not perform hyperparameter optimization, our method aimed to provide an agnostic prediction interval applicable to any prediction model. All models were trained for 300 epochs with batch size 32.
Our algorithm requires solving (non-com con-cco optimization problems through mathematical optimiza- tion. We chose to employ two interior-pon solvers in our experiments: For the experiments with soft interventions that pose convex optimization problems, we use the solver MOSEK. For the hard interventions, which included non-convex problems, we used the solver IPOPT. Both solvers were run with default parameters.
G.4 Selection of the interventions in our experiments
The treatment (in the complete dataset) is modeled to lie in the range [0,40]. Therefore, the treat- ments/interventions can also only fall into this range. All samples that would have achieved treatment outside this range through the interventions were neglected in our analysis. To guarantec
37

---

that a sufficient number of samples were included in our experiments, we chose the maximal soft treatment as an increase of 10.
We sampled the covariates X in the range from 1 to 4. To again perform interventions that fall inside the range of [0,40], decided to set the intervention 7 X and 10XX. Other choices of interventions would also have been possible. Reassuringly, there was no systematic selection of interventions in our experiments besides the considerations above.
For soft interventions, we do not see different effects of the interventions on the two datasets. For observe a difference. Recall that in dataset the treatment was sampled uniformly from [0,40] (with a dependence on X), whereas in dataset 2, it was sampled from normal distribution with mean of 5.X. Therefore, the intervention 10XX is far in the tail of the distribution. As a result, we observe a slightly lower coverage for this intervention on dataset compared to dataset 1.
the hard interventions, however,
38

---

H
Further results
We present further results from our experiments Section Specifically, we state the prediction performance of the underlying models o, discuss the scalability of our approach, and show the prediction intervals per covariate for various significance levels a and soft interventions A of our synthetic experiments on dataset 1 and dataset 2.
Performance: We first report the performance of the underlying prediction models & for the synthetic dataset achieved a mean
datasets across 50 runs in Table The prediction model on the
squared error loss of 1.2373.
Dataset
Dataset 2
MIMIC
0.0216 (0.0056)
0.9029 (0.3908)
0.0141 (0.0057)
Ens.
0.0094 (2.1169e=))
0.0130 (0.0003)
Table 4: Mean and standard deviation of MSE loss of prediction models & across 50 runs.
Width of the prediction intervals: We further report the width of the prediction intervals in our synthetic experiments in T The width is important to assess the usefulness of the resulting prediction intervals. As the performance of the ensemble method is not comparable with the coverage of MC-Dropout and our CP method, we only compare the latter two methods with regard to the interval width.
Dataset 1
Dataset 2
Delta
Ours
MC-Dropout
Ours
MC-Dropout
0.3647 (0. (2.4
0. 1939 (0. 170)
0.4051 (0. (0.1036)
0.2897 (0. 1480)
5
0.4024 (0. (0.2285)
0. 1615 103)
0,4610 (0.2479)
0.3036 (455)
10
0.4301 (0.2610)
0.1639 (0.1080)
0.6711 (0.8520)
0.3235 ( (0. 1445)
Table 5: Mean and standard deviation of the resulting prediction intervals.
Comparison to the vanilla CP baseline: We compare our method to the naive vanilla CP (V-CP), i... a CP method that does not account for the distribution shift. We observe that V-CP does not achieve any valid prediction interval across all distribution shifts and confidence levels. This can be explained by the good prediction performance of the underlying model. Thus, V-CP intervals extremely small (average width of 0.0003) and can never cover the true potential outcome after the intervention. Overall, the results confirm the importance of accounting for the distribution shift induced by the intervention.
X+4
Scalability: Calculating the prediction intervals requires an iterative search for an optimal value S* Therefore, the underlying optimization problem must be fitted multiple times through- out the algorithm, potentially posing scalability problems. In our empirical studies, however, we did not encounter scalability issues. Importantly, found that the average runtime of our algo- rithm on a standard desktop CPU only 16.43 seconds. On the MIMIC dataset, computing CP intervals takes roughly 10 times longer than empha-
computing MC intervals. However, size that MC intervals are generally not faithful and therefore directly comparable.
Tradkard
Figure 8: Prediction intervals for multiple signif- icance levels a for the synthetic dataset with intervention A = 1.
Prediction intervals: In Figures and 10 present the prediction bands given by our
39

---

method and MC dropout on dataset 1. In particular, for confounder X = 1, our method shows large increase in the uncertainty in the potential outcomes of treatments affected by the intervention. MC dropout does not capture this uncertainty.
0004*
1Em
Figure 9: Prediction intervals for multiple signif- Figure 10: Prediction intervals for multiple sig- icance levels for the synthetic dataset with intervention A
nificance levels a for the synthetic dataset I with intervention 10.
In Figures 11 and 12 we present the prediction bands given by our method and MC dropout on dataset 2 for the soft interventions A and A 10 (the results for 5 were presented in the main paper). We observe that the prediction intervals for A = 10 become extremely wide for high treatments. This aligns with our expectation, as data for high treatments combination with low confounders is rare even absent in the dataset. Thus, the expected uncertainty is very high.
X+2
1240
Calcee
Thammer
The Thammet
Teireet
Figure 11: Prediction intervals for multiple sig- nificance levels a for the synthetic dataset 2 with intervention A-
Figure 12: Prediction intervals for multiple sig- nificance levels for the synthetic dataset 2 with intervention = 10.
40


Reviews:
[{"review_id": "ehQm8vUiyW", "reviewer": "Reviewer_v4YF", "summary": "This paper focuses on the task of conformal prediction of causal effects from continuous-valued treatments. The novelty in this paper stems from handling the cause of continuous-valued treatment, while previous literature focuses on binary or discrete-valued treatments.\n\nThe paper starts by tackling the causal effect estimation setting, making the standard assumptions of positivity, consistency, and unconfoundedness. They also assume access to a ML model to predict potential outcomes. The paper tackles producing CP intervals under either discretized hard or soft interventions, which cause the resulting distribution to violate standard exchangeability assumptions.\nTo address this shift in distribution, the authors perform calibration (conditioned on the propensity shift). They define this shift in Equation (2) and use split conformal prediction with a calibration step based on the shifting function $f$. \n\nThe authors empirically compare their approach with MC-dropout (with the same underlying MLP) and deep ensembles, given that there are no baselines that handle the setting of continuous treatments. They provide results on synthetic data, finding that the proposed approach has much higher coverage.", "questions": "Empirically, how does the runtime compare to the alternatives on the MIMIC dataset?\n\nAlso, see other weaknesses above.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- Tackles a new (and well-motivated) setting of CP for causal effect estimation in the setting of continuous-valued treatments\n- Technical novelty in deriving a new theoretically sound approach (i.e., with proper coverage guarantees), adapting the approach from split CP\n- Develops technical methodology to handle both cases where propensity scores are known and unknown\n\nWeaknesses\n- No comparison to a naive CP implementation as a baseline – this would be another valid and interesting baseline to understand the importance of the proposed approach\n- The method seems computationally intensive; it requires calls to the optimization solver during each iteration of a binary search. If I’m not mistaken, shouldn’t the algorithm’s fixed complexity be $O(\\log(\\frac{S_{up} - S_{low}}{\\epsilon} * \\sigma_s(n_c)) (lines 601-602)? \n- Minor formatting issues (e.g., the wrapped figures on Page 8)", "was_revised": true, "final_justification": "I remain borderline as the method is computationally intensive and the authors have not empirically demonstrated the runtime comparisons. These would be good improvements to keep in mind for future versions of the work.", "rebuttal": "Thank you for your feedback and the positive evaluation of our manuscript. Below, we provide answers to all the questions and concerns you raised. We will add the corresponding improvements labeled as “**Action**” to our revised manuscript.\n\n&nbsp;\n\n**Comparison to naive CP baseline:**\n\nWe followed your suggestion and added a comparison to the naive vanilla CP (V-CP), i.e., CP, which does not account for the distribution shift. This may be of interest to the reader to understand the importance of the proposed approach. In our experiments, we observed that V-CP does **not** provide any valid prediction interval, _across all distribution shifts and confidence levels_. This can be explained by the good prediction performance of the underlying model (see Supplement G). Thus, V-CP intervals are extremely small (average width of 0.0003) and can never cover the true potential outcome after the intervention. Overall, the results confirm the importance of accounting for the distribution shift induced by the intervention.\n\nTo investigate whether simple weighted CP methods for causal tasks on binary treatments sufficiently address this challenge, we compare our method with the method by Lei and Candés. Of note, both methods fulfill the coverage guarantees. However, when comparing the interval width, we see that **our method is superior**: Our intervals have an average width of 0.6255 (sd = 0.1714). In contrast, the intervals on the binarized treatment via the Lei-Candés method have an average width of 3.2876 (sd = 0.5587). Hence, the intervals by our method are – by far – more informative.\n\n**Action:** We will include our observations on the performance of both methods in the camera-ready version of our manuscript.\n\n&nbsp;\n\n**Computational complexity:**\n\nYou are correct that the overall complexity is $O(\\log(\\frac{S_{up} - S_{low}}{\\epsilon} \\cdot \\sigma_s(n_c))$ following from the argumentation in the derivation. Please excuse the typo. \n\nWe acknowledge that our method is computationally rather complex, which we also stated when discussing the limitations of our method. However, this is not unique to our method, but also inherent to the baselines. As a case in point, the complexity of deriving intervals through the baselines (i.e., MC-dropout or ensemble methods) depends on the number of MC samples or models, respectively, both of which must be rather large to offer similar computational performance. In particular, both baselines do not offer valid prediction intervals, which is the contribution of our method. On top of that, the latter baseline (ensemble methods) scales with the precision of the intervals, which might be difficult to control and is thus impractical in real-world applications. In our work, we use optimization as a tool to provide conformal prediction intervals. Future research should focus on developing more efficient optimization algorithms for this task.\n\n**Action:** We will correct the typo in the camera-ready version of our manuscript.\n\n&nbsp;\n\n**Runtime on MIMIC dataset:**\n\nThis is an interesting question. Due to the optimization problem that needs to be solved in our approach, the time needed for computing our CP intervals is naturally higher than the time needed to calculate the MC intervals. While the theoretical runtime of CP may exhibit more complex scaling behavior (e.g., cubic or non-linear), our empirical results demonstrate that CP scales well in practice. Specifically, computing CP intervals takes roughly 10 times longer than computing MC intervals. However, we emphasize that MC intervals are generally **not** faithful and therefore **not** directly comparable. To better understand the scaling behavior of our CP method in practice, we examine computation time relative to the average interval width. Under this lens, both methods exhibit a similar time-to-width ratio of approximately one. In other words, our experiments do not indicate that CP intervals scale poorly—in fact, _we observe good scaling properties in practice_.\n\n\n**Action:** We will include a full runtime comparison on the MIMIC dataset in the camera-ready version of our paper."}, {"review_id": "SQQ4Q8j7O4", "reviewer": "Reviewer_aQE1", "summary": "This paper develops a conformal prediction framework for estimating uncertainty intervals of potential outcomes under continuous treatment settings in causal inference, which inherently break exchangeability. In this paper, authors propose CP method to  remain valid in terms of exchangeable assumption under known or unknown propensity scores, when interventions shift the treatment assignment mechanism. Synthetic and real-world experiments confirm reliable coverage and adaptive interval widths under various intervention scenarios and significance level.", "questions": "(1) The motivation of uncertainty quantification is not clear. As claim in the paper, the point estimate will neglect that certain treatment is ineffective for some patients profile. however, some causal estimands, e.g., CATE, can also fill this gap.\n\n(2) one of the challenges of conformal prediction for causal quantities lies on the non-exchangeability due to the intervention. But why the challenge is non-trivial? The manuscript is unclear for thin point. Why CP needs exchangeability assumption?\n\n(3) in eq. (3) and (4), what does the function $l_{\\alpha}$ represent? this part lacks important details about the quantile regression.\n\n(4) in line 208, the vector \\eta^S has n+1 elements? it is not correct.\n\n(5) it's very confusing that why one needs to estimate the weight $\\eta$ in eq.(8), instead of directly using $(1-\\alpha)$ and $\\alpha$? what's the difference between eq,(9) and (3), (4)?\n\n(6) and it also lacks of specific explanation why use max $\\eta$ in the max-min objective in eq.(8).\n\n(7) how does the Eq.(Ps) is reformulated? what does u_i and v_i represent?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n(1) The paper addresses a clear gap in the conformal prediction literature: providing finite-sample valid prediction intervals for counterfactual outcomes under continuous treatments\n\n(2) The paper provides finite-sample marginal coverage guarantees for the constructed intervals in both known and unknown propensity settings, which is rare in causal CP work\n\n(3) The method is carefully grounded in the potential outcomes framework, and the authors precisely handle the violation of exchangeability under interventions.\n\nWeakness:\n(1) In the unknown propensity case, the method uses estimated scores but doesn't model or quantify uncertainty in their estimation. This may degrade validity if the estimation is poor.\n\n(2) The experiments do not include baseline methods for conformal prediction under causal settings (e.g., DR-CP, CPC for binary treatments). Though not directly comparable, some kind of ablation or prior method could serve as a sanity check.\n\n(3) Some strong assumptions about the variables and function form, such as $\\delta$ in terms of Gaussian function in Eq. (14).", "was_revised": true, "final_justification": "Most papers on causal inference, specifically treatment effect estimation or counterfactual estimation, focus on point estimates. This paper nicely fills the gap between effect estimation and uncertainty quantification, and its theoretical framework is relatively clear. However, since my expertise lies primarily in causality, and I am less familiar with conformal prediction, there may be areas where I lack sufficient domain knowledge. Therefore, I am unable to provide a more definitive recommendation, but I lean toward a borderline accept.", "rebuttal": "Thank you for your constructive feedback and the positive evaluation of our manuscript. Below, we provide answers to all questions. We will add the corresponding improvements labeled as “**Action**” to our revised manuscript.\n\n&nbsp;\n\n**Answers to weaknesses:**\n\n&nbsp;\n\n**(1) Uncertainty quantification of propensity estimation:**\n\nThank you for raising this question. Accounting for the propensity estimation error is exactly one of the strengths of our method (see Section 4.2) and differentiates our framework from former work in this field (see Section 2). Under Assumption 1 (by stating a user-specified upper bound on the estimation error), our intervals are valid for all misspecified propensity estimations with estimation error smaller than or equal to the bound. \n \n&nbsp;\n\n**(2) CP Baselines:**\n\nWe followed your suggestion and added a comparison to the naive vanilla CP (V-CP), i.e., CP, which does not account for the distribution shift. Specifically, we now report the performance of (i) the naive vanilla CP (V-CP), i.e., CP, which does not account for the distribution shift, and (ii) the method proposed by Lei and Candés for binary treatments (for which we binarize our treatment).\n\n(i) We observe that V-CP does **not** achieve any valid prediction interval, _across all distribution shifts and confidence levels_. This can be explained by the good prediction performance of the underlying model (see Supplement G). Thus, V-CP intervals are extremely small (average width of 0.0003) and can never cover the true potential outcome after the intervention. Overall, the results confirm the importance of accounting for the distribution shift induced by the intervention.\n\n(ii) We compare our method and the method by Lei and Candés on the semi-synthetic TCGA dataset, where both methods fulfill the coverage guarantees. However, when comparing the interval width, we see that **our method is superior*: Our intervals have an average width of 0.6255 (sd = 0.1714). In contrast, the intervals on the binarized treatment via the Lei-Candés method have an average width of 3.2876 (sd = 0.5587). Hence, the intervals by our method are – by far – more informative.\n\n**Action:** We will include our performance results of the V-CP and the binary method by Lei and Candés in the camera-ready version of our manuscript.\n\n&nbsp;\n\n**(3) Functional form of $\\delta$:**\n\nWe are happy to explain our choice of representation of the Dirac delta function. Note that it is a common way to represent the Dirac delta as the limit of a family of probability densities that become infinitely concentrated at zero. One such family is the Gaussian density. Other options are, e.g., the representation through Lorentz spikes, rectangular pulses, or sinc kernels. For background on the representation, we refer the reviewer to, e.g., [1]. Please let us know if there are other functional forms in our paper that should be explained in more detail.\n\n\n&nbsp;\n\n**Answers to questions:**\n\n&nbsp;\n\n**(1) Motivation for UQ:**\n\nThe reviewer is correct that heterogeneous effects, such as CATE, are designed to capture variability in the treatment effect estimate; however, this only refers to the variability based on the heterogeneity in effects for varying covariates. It does **not** include the variability in the effects due to noise in the data or modeling error (which is our objective). Therefore, uncertainty quantification (as in our paper) is necessary in causal inference settings in the same way as for other ML applications.\n\n&nbsp;\n\n**(2) Exchangeability assumption:**\n\nCoverage guarantees of existing CP intervals essentially rely on the exchangeability of the non-conformity scores. Exchangeability assures that the nonconformity score of the test point $n+1$ is equally likely to fall anywhere among the calibration scores, its rank is uniform, and that uniformity is exactly what yields the distribution‑free coverage guarantee. Without exchangeability, the rank is not guaranteed to be uniform, and the marginal coverage bound can fail.\n\nHowever, intervening on treatment $A$ shifts the propensity function and, therefore, induces a shift in the covariates between calibration and test data, specifically in treatment $A$. Therefore, exchangeability is not fulfilled, and the coverage guarantees might fail.\n\nAs a remedy, we present a novel and powerful remedy in our work: our calibration step differs from the standard procedure in that we conditionally calibrate the non-conformity scores depending on the tilting function $f$ to achieve marginal coverage for the interventional and thus shifted data. Thereby, we ensure that the exchangability assumption holds.\n\n&nbsp;\n\n**(3) Quantile regression Eq. 3 & 4:**\n\nEquations 3 & 4 represent the general form of a quantile regression as a minimization problem based on the so-called _pinnball-loss_ $l_{\\alpha}$, a standard result in the statistical regression literature (e.g. [2],[3],[4],[5]). \n\nObserve, that $l_{\\alpha}$ can equivalently be represented as $l_{\\alpha}(\\theta, S) = (\\alpha - \\mathbb{1}_\\{[\\theta-S < 0]\\})(\\theta-S)$. This asymmetric linear loss penalizes under-prediction $(\\theta-S) > 0$ proportionally to $\\alpha$ and over-prediction $(\\theta-S) < 0$ proportionally to $1-\\alpha$, which results in the fitted function (see Eq. 3) targeting the $\\alpha$-th conditional quantile.\n\n**Action:** We will include more background information on the reformulation of quantile regression in the camera-ready version of our manuscript.\n\n&nbsp;\n\n**(4) Dimension of vector in line 208:**\n\nThank you for noting the typo! Initially, we referred to the sizes of both the training and calibration datasets as $n$, resulting in the dimension stated in line 208; however, we later adapted our formulation to enhance the reader’s understanding. The correct dimension of $\\eta$ is $n-m$. \n\n\n**Action:** We will correct the typo in the camera-ready version of our paper.\n\n&nbsp;\n\n **(5), (6) & (7) Dual optimization:**\n\nWe are happy to explain the necessity and the derivations of the **dual optimization** problem in **Theorem 4.2** from the **primal optimization** problem in **Lemma 4.1**. \n\nFirst, observe that the objective function and loss formulation in Eq. 3 & 4 denote a general quantile regression, which does _not_ directly correspond to our CP problem. The precise (primal) optimization problem for our CP setting is given in Lemma 4.1 (Eq. 5). \n\nNote that directly **solving the problem given in Lemma 4.1 is not feasible**, as it would require solving Eq. 5 for all possible imputed values $S \\in \\mathbb{R}$ of $S_{n+1}$. It is unclear how to feasibly find the imputation of the “true” $S_{n+1}$ needed for the interval construction in Eq. 6. We thus follow former results in the literature (e.g., [6]) and exploit the efficient computation through dual optimization.\n\n\nWe can rewrite the primal problem in Lemma 4.1 as\n\n$\n\\min_{\\theta > 0} \\quad \\sum_{i=m+1}^{n+1} (1-\\alpha)u_i + \\alpha v_i\\\\\n\\qquad\n\\text{ s.t. } S_i - \\theta \\frac{\\pi(a_i + \\Delta_A \\mid x_i)}{\\pi(a_i \\mid x_i)}) -u_i + v_i = 0, \\quad \\forall i=m+1\\ldots,n+1, S_{n+1} = S,\n\\quad\nu_i, v_i \\geq 0,  \\quad \\forall i=m+1\\ldots,n+1.\n$\n\nNote that this is precisely the formulation of the respective primal problem stated in **Eq. P_S** as well as Theorem 4.5, where **$u_i$ and $v_i$ represent the primal optimization variables.**\n\n\nThe **Lagrangian** of the primal problem with Lagrangian multipliers $\\eta, \\gamma_1, \\gamma_2$ states\n$\n\\mathcal{L} = \\sum_{i=m+1}^{n+1} (1-\\alpha) u_i + \\alpha v_i + \\sum_{i=m+1}^{n+1} \\eta_i \\left( S_i - \\theta \\frac{\\pi(a_i + \\Delta_A \\mid x_i)}{\\pi(a_i \\mid x_i)} - u_i+v_i \\right) - \\sum_{i=m+1}^{n+1} (\\gamma_{1_i}u_i + \\gamma_{2_i}v_i).\n$\n\nleading to the **dual problem** \n\n$\n\\max_{\\eta_i, i=m+1,\\ldots,n+1} \\min_{\\theta > 0} \\quad \\sum_{i=m+1}^{n} \\eta_i \\left(S_i - \\theta \\frac{\\pi(a_i + \\Delta_A \\mid x_i)}{\\pi(a_i \\mid x_i)} \\right) + \\eta_{n+1} \\left( S - \\theta \\frac{\\pi(a^{\\ast} \\mid x_{n+1})}{\\pi(a_{n+1} \\mid x_{n+1})} \\right) \\\\\n\\qquad \\text{ s.t.} -\\alpha \\leq \\eta_i \\leq 1-\\alpha, \\quad \\forall i=m+1,\\ldots,n+1, \n$\n\nwith **dual optimization variables $\\eta_{m+1},...,\\eta{n+1}$.** \n\nFor more background on **dual (convex) optimization theory**, we refer the reviewer to, e.g., [7], [8], due to space constraints in the rebuttal. Specific details on our derivations can be found in the **proofs of Theorems 4.2 and 4.5 in Appendix D**.\n\n\nFor more details on the **coverage guarantees/reformulation from Eq. 6 to Eq. 9**, we refer the reviewer to the respective part of the **proof of Thm. 4.2** in lines 663 and following in our manuscript.\n\n**Action:** We will provide more background and explain the intuition for the dual optimization step in more detail in the camera-ready version of our manuscript.\n\n&nbsp;\n\n[1] Gel’fand, I. M. and Shilov, G. E. . Generalized Functions, Vol. 1: Properties and Operations (Academic Press, 1964)\n\n[2] Koenker, R., and Bassett Jr., G. \"Regression quantiles.\" Econometrica: journal of the Econometric Society (1978): 33-50.\n\n[3] Koenker, R. Quantile regression. Vol. 38. Cambridge University Press, 2005.\n\n[4] Yu, K., and Moyeed, R. A. \"Bayesian quantile regression.\" Statistics & Probability Letters 54.4 (2001): 437-447.\n\n[5] Belloni, A., and Chernozhukov, V. \"ℓ 1-penalized quantile regression in high-dimensional sparse models.\" (2011): 82-130.\n\n[6] Gibbs, I. et al. Conformal prediction with conditional guarantees. arXiv preprint, 2023.\n\n[7] Boyd, S. P., and Vandenberghe. L.. Convex optimization. Cambridge University Press, 2004.\n\n[8] Luenberger, D. G., and Ye, Y. Linear and nonlinear programming. Vol. 2. Reading, MA: Addison-Wesley, 1984."}, {"review_id": "PHC5IK5DZs", "reviewer": "Reviewer_K7Nc", "summary": "This paper proposes a CP framework for constructing finite-sample valid prediction intervals for counterfactual outcomes, under both hard and soft interventions on a continuous treatment. It addresses a key limitation of standard conformal prediction—the violation of exchangeability between calibration and test points due to distribution shifts induced by interventions. To restore valid coverage, the authors introduce a tilting function based on the ratio of interventional to observational treatment densities (i.e., propensity scores). The method is empirically validated on both synthetic data and the MIMIC-III ICU dataset.", "questions": "1. No comparisons with causal UQ baselines. Additionally, it would be helpful to include a related work section discussing recent progress on causal CP and causal UQ methods.\n2. The method assumes that outcome exchangeability is violated. Is this violation primarily due to treatment shift (e.g., a patient receiving a different treatment than observed), or due to covariate distribution shift (e.g. a young patient and an old patient)? How does the proposed approach handle these two types of shift, especially when they co-occur?\n3. This may be slightly tangential, but in line 276 you mention aleatoric uncertainty. Could you elaborate on how aleatoric and epistemic uncertainties are defined in this context, and whether (or how) the proposed method addresses each type?", "limitations": "See above.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "1. Strong motivation as the authors clearly identify three challenges and address them under different scenarios. They also provide strong theoretical results with finite-sample validity in sections 4.1 and 4.2.\n\n2. I quite like the idea of causal conformal prediction—the use of a tilting function based on the treatment distribution is clever. However, this also leads to my another concern: the method in this paper does not guarantee causal identifiability in the traditional sense. Instead,  it is more like assumes the predictive model (and, in some cases, the propensity model) is already \"good enough\" and focuses on valid uncertainty quantification under distribution shifts induced by hard or soft interventions. It lies at the intersection of predictive modeling, distribution shift correction / importance weighting, and finite-sample uncertainty quantification, and touches causal inference only to the extent that it predicts under interventions, but does not address identification or estimation of causal effects directly.\n\n3. The authors provide extensive experimental results to illustrate the superior performance of their method. However, the baseline methods are limited to MC dropout and deep ensembles. Later, they find that ensembles are highly unfaithful and remove them from comparison. This raises a concern about fairness: MC dropout and ensembles are not designed to handle counterfactual or distributional shift elements (they rely on the exchangeability assumption), so comparing them to the proposed method is like comparing a bicycle to a jet engine. This might explain why the figure 4 and 5 look too good to be true. It would be more convincing to compare the proposed method against baselines that incorporate causal elements, such as Bayesian causal forests or causal GPs, which are widely used and provide credible intervals for potential outcomes or treatment effects.\n\n4. As mentioned above, the proposed methods heavily rely on the correctness of the predictive model. It would be helpful to include some ablation analysis to show how performance depends on the accuracy of propensity score estimation, model misspecification, or unmeasured confounding.", "was_revised": true, "final_justification": "I quite like the idea of using a causal conformal score. However, the paper does not stand out sufficiently from existing methods due to a weak empirical evaluation and some ambiguity in distinguishing causal interventions from general distributional shifts or conditional prediction problems. Therefore, I have decided on a borderline accept.", "rebuttal": "Thank you for your constructive feedback and the positive evaluation of our manuscript. Below, we provide answers to all questions. We will add the corresponding improvements labeled as “**Action**” to our revised manuscript.\n\n&nbsp;\n\n**Answers to weaknesses:**\n\n&nbsp;\n\n**Causal identifiability:**\n\nThroughout our work, we make the **three standard assumptions** in causal inference: **positivity, consistency, and unconfoundedness** (see Section 3). These assumptions are standard in the literature (e.g., [1],[2],[3],[4]). Under these assumptions, the conditional average **treatment effect** $\\tau(x) = \\mathbb{E}[Y(a)-Y(a^{‘})| X=x]$ is **identified** as $\\tau(x) = \\mathbb{E}[Y|A=a, X=x] -\\mathbb{E}[Y|A=a^{‘},X=x]$. The potential outcome estimation thus simplifies to estimating the conditional expectation; in our case, estimated through the ML model $\\phi$ (see **Section 3**). In the main paper, we provide conformal intervals for potential outcomes. We derive **CP intervals for the treatment** effect in **Appendix A**. Given that these assumptions are standard and generally satisfied by design (e.g., in clinical trials) (e.g., [4],[5],[6]), we do not share the concern raised by the reviewer. However, we are happy to clarify further aspects if needed.\n\n&nbsp;\n\n**Causal UQ baselines:**\n\nTo foster the evaluation of our method, we additionally provide coverage results on two further causal UQ baselines: (i) causal Gaussian processes, and (ii) the causal CP method for binary treatments by Lei & Candés.\n\n(i) The GP is only able to capture the true potential outcome in the prediction intervals for small distribution shifts (Δ = 1) on dataset 2. However, the empirical coverage is extremely low: $\\alpha = 0.05: 0.125; \\alpha = 0.1: 0.125; \\alpha = 0.2: 0.0833$. This result aligns with our expectations, as the aleatoric uncertainty in our experiments is low. Therefore, the GP intervals will be small (average width of 0.1293) and barely provide valid intervals for the potential outcomes after the intervention.\n\n(ii) We compare our method and the method by Lei and Candés on the semi-synthetic TCGA dataset. However, when comparing the interval width, we see that **our method is superior**: Our intervals have an average width of 0.6255 (sd = 0.1714). In contrast, the intervals on the binarized treatment via the Lei-Candés method have an average width of 3.2876 (sd = 0.5587). Hence, the intervals by our method are – by far – more informative.\n\n&nbsp;\n\n**Dependence on propensity score estimation, model misspecification, or unmeasured confounding:**\n\n*Propensity estimation:* Our intervals depend on the quality of propensity estimation through the parameter $M$. Larger $M$ represents more uncertainty in the propensity estimation and thus wider intervals. We conducted an ablation study on the TCGA dataset: the results confirm this pattern, as expected.\n\n*Model misspecification:* Our CP intervals are exactly designed to account for the bias and uncertainty stemming from model misspecification. Misspecified models lead, on average, to higher non-conformity scores on the calibration dataset and, thus, to wider intervals.\n\n*Unmeasured confounding:* We highlight again that we make the three standard assumptions in causal inference, including unconfoundedness. In the presence of unmeasured confounders, the potential outcomes are not (point-)identified. Our method is not designed for this setting. For future work, extending our method to partial identification of potential outcomes based on sensitivity analysis could be of interest. \n\n\n**Action**: We will add the above discussion to the camera-ready version of our paper, including the results of the new ablation study.\n\n\n&nbsp;\n\n**Answers to questions:**\n\n&nbsp;\n\n**(1) Causal UQ baselines and related work:**\n\nFor your question on causal UQ baselines, please see our answer above.\n\nThere exists a variety of work for uncertainty quantification (UQ) in causal effect estimation for continuous treatments. Existing methods for UQ of causal quantities are often based on Bayesian methods (e.g., [1],[2],[7]). However, Bayesian methods **require the specification of a prior distribution** based on domain knowledge and are thus neither robust to model misspecification nor generalizable to model-agnostic machine learning models. Other methods **only provide asymptotic guarantees** (e.g. [3],[8]). The strength of conformal prediction, however, is to provide finite-sample uncertainty guarantees. Overall, **none of the methods** tackles the problem of distribution-free uncertainty quantification for causal effect estimation of continuous treatments in finite sample settings.\n\nFor an extensive discussion of related work on CP for causal quantities, we refer the reviewer to our extensive **extended literature review** in **Appendix E**.\n\n&nbsp;\n\n**(2) Exchangeability violation:**\n\nCoverage guarantees of existing CP intervals essentially rely on the exchangeability of the non-conformity scores. Exchangeability assures that the nonconformity score of the test point $n+1$ is equally likely to fall anywhere among the calibration scores, its rank is uniform, and that uniformity is exactly what yields the distribution‑free coverage guarantee. Without exchangeability, the rank is not guaranteed to be uniform, and the marginal coverage bound can fail.\n\nHowever, **intervening on treatment** $A$ shifts the propensity function and, therefore, induces a shift in the covariates between calibration and test data, specifically in the treatment $A$. Therefore, **exchangeability is not fulfilled**, and the coverage guarantees might fail (at least for existing methods).\n\nAs a remedy, we present a novel and powerful remedy in our work: The overall distribution of the *confounders* $X$ is assumed to stay constant between train, calibration, and test data (as standard in ML problems). This is completely orthogonal to constructing intervals for different (e.g., young or old) patients. Note that CP intervals are constructed for only one sample/patient at a time. This means that different intervals are constructed for patients with different features $X$. In other words, the **intervals** are constructed **conditionally** on $X$, but the **coverage guarantee** is **marginal** across the complete population of $X$. Overall, the shift from one patient to another does not pose any challenges for CP methods.\n\n\n**Action:** We will include a discussion on the role of the exchangeability assumption and the limitations it poses for existing causal CP methods in the camera-ready version of our paper. We will clarify how our proposed method circumvents these limitations, offering a more robust and practical approach under realistic causal settings.\n\n&nbsp;\n\n**(3) Aleatoric and epistemic uncertainty:**\n\n\nThis is an interesting question! First, we like to highlight that CP guarantees marginal (total) coverage, _agnostic_ to the split between aleatoric and epistemic error.\n\nAleatoric uncertainty denotes the irreducible noise inherent in the data–generation process, such as measurement error or intra‑subject variability. Epistemic uncertainty is present due to the limited data available and due to model misspecification. This uncertainty is, in principle, reducible by gathering more data or improving the model.\n\n&nbsp;\n\n[1] A. Alaa and M. van der Schaar. Bayesian inference of individualized treatment effects using multi-task Gaussian processes. In Conference on Neural Information Processing Systems (NeurIPS), 2017\n\n[2] K. Hess et. al. Bayesian neural controlled differential equations for treatment effect estimation. In International Conference on Learning Representations (ICLR), 2024.\n\n[3] J. Jonkers et.al. Conformal Monte Carlo meta-learners for predictive inference of individual treatment effects. arXiv preprint, 2024. \n\n[4] P. R. Rosenbaum and D. B. Rubin.The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41–55, 1983\n\n[5] S. Feuerriegel et. al. Causal machine learning for predicting treatment outcomes. Nature Medicine, 30(4):958–968, 2024.\n\n[6] P. Sanchez et al. Causal machine learning for healthcare and precision medicine. Royal Society Open Science 9(8), 2022\n\n[7]\tA. Jesson et.al. Identifying causal-effect inference failure with uncertainty-aware models. In Conference on Neural Information Processing Systems (NeurIPS), 2020. \n\n[8]\tY. Jin et.al. Sensitivity analysis of individual treatment effects: A robust conformal inference approach. Proceedings of the National Academy of Sciences of the United States of America, 120(6), 2023."}, {"review_id": "48JRNbhTNx", "reviewer": "Reviewer_6PUM", "summary": "This paper proposes a conformal prediction method for the estimation of potential outcomes under continuous treatments and known/unknown propensity scores. The authors provide finite-sample validity guarantees and validate their method empirically on synthetic and real-world data.", "questions": "Can you elaborate on the numerical stability of your method, particularly in Scenario 2 where the optimization is non-convex and involves small propensity estimates and Gaussian tilting?", "limitations": "Weak (practically relevant) empirical validation and uncritical/difficult to interpret theoretical motivation. If I was a physician I wish the potential challenges potentially invalidating the method were better stressed through the paper: e.g., I have limited data, I have a prior on how to bound M, (when) can I use your method?\n\nSee weaknesses and questions.", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**: \n- *Novelty*: First to provide conformal prediction intervals for continuous treatments with unknown propensities.\n- *Structured*: The methodological contributions are well-structured (e.g., known propensity->soft intervention, unknown propensity-> hard intervention) and carefully motivated (e.g., challenges a,b,c discussions).\n\n**Weaknesses**:\n- *Weak Empirical Evidence*: \n    - Despite the consistent results on the Synthetic Dataset 1-2, such DGP with a single covariate is over-simplistic to relate with real-world applications. \n    - I really appreciate the attempt to complement such experiments with semi-real and real-world data. However, the experiments on TGCA dataset (semi-synthetic) are poorly documented (unclear outcome of interest, treatment assignment, evaluation procedure, ground truth) and lack comparison with other approaches. \n    - Similarly, the experiment on MIMIC dataset (real-data) looks incomplete: Figure 7 only shows the CP-estimated intervals (compared with MC dropout), which a reader can only evaluate by their qualitative width but not their coverage (how to establish that yours is out-performing MC-dropout?). Also unclear if the counterfactual estimation on this dataset isn't just trivial linear interpolation  (also what are the dotted-line in Figure 7?). Finally, I don't understand why showing the intervals per individual if the conditional (over the covariates) coverage is never discussed.\n- *Section 4*: Despite the great high-level structure, I struggled to both follow and interpret the two theoretical results (also not very familiar with the related works though). The math sounds correct (despite I may have missed some details), but I found difficult to interpret both variables (f, l, g, ...) and formulations. In practice, missing interpretations makes difficult to understand the limitations and practical validity of the method. Are the propensity score estimation and calibration data size challenges the only challenges of this approach (as reported in Section 6) to get exact finte sample guarantees?", "was_revised": true, "final_justification": "The authors have addressed several of my concerns in a detailed and constructive way. They improved the documentation of the TCGA setup and added comparisons to baselines, which partially strengthens the empirical section. However, the experimental validation remains limited overall: the TCGA experiment is still fully synthetic, and the MIMIC evaluation is qualitative, lacking ground truth, coverage assessment, or clarity.\n\nTheoretical sections are clearer after the response, especially the reformulation via quantile regression and dual optimization, though the notation would still benefit from more intuitive interpretation. The explanation of numerical stability is sound and technically competent.\n\nOverall, I appreciate the conceptual and theoretical contributions, but remain disappointed by the very limited empirical evidence, which may fall short of expectations for practitioners interested in applying these methods on real-world applications.", "rebuttal": "Thank you for your constructive feedback. We will add the corresponding improvements labeled as “**Action**” to our revised manuscript. \n\n&nbsp;\n\n**Answer to weaknesses**\n\n&nbsp;\n\n**Empirical evaluation:**\n\nWe are happy to discuss our empirical evaluation and give more background on the TCGA and MIMIC experiments below.\n\n**(i) TCGA:**\nWe synthetically model a continuous treatment based on the sum of the 10 covariates with the highest variance and assign a treatment effect that is constant in the sum of the covariates. Specifically, we model the treatment to follow a normal distribution centered at 100*sum of the 10 covariate values, and the outcome to follow a normal distribution centered at the sum of the treatment and the covariate sum times 100.\n\nWe now also obtained prediction intervals from the other baselines (MC dropout and the ensemble method) and report the coverage results below. The results are in line with our findings on datasets 1 & 2.\n\n| Intervention | α = 0.05          | α = 0.1           | α = 0.2           |\n|-----------------|--------------------|--------------------|--------------------|\n| Δ = 0.5;   MC:  | 0.8280 (0.0795)   | 0.8160 (0.0783)   | 0.8040 (0.0741)   |\n|      Δ = 0.5; E:      | 0.0640 (0.0445)  | 0.0600 (0.0379)  | 0.0480 (0.0412) |\n| Δ = 1.0;  MC:  | 0.8560 (0.0612)   | 0.8520 (0.0614)   | 0.8400 (0.0657)   |\n|    Δ = 1.0; E:   | 0.0880 (0.0483)   | 0.0680 (0.0371) |  0.0520 (0.0348) |\n| Δ = 1.5;    MC:  | 0.7880 (0.0815)   | 0.8040 (0.0925)   | 0.8280 (0.0786)   |\n| Δ = 1.5;    E:  | 0.0720 (0.0411)    | 0.0680 (0.0412)  | 0.0640 (0.0389) |\n\n\n\n**(ii) MIMIC:**\nNote that numerically evaluating causal inference methods on real-world data is not possible due to the fundamental problem of causal inference, i.e., only one potential outcome (for continuous treatments out of infinitely many) is observed for each individual. Therefore, we chose to provide “insights plots” (Fig. 7) for this dataset, as (i) assessing overall coverage across the test dataset is impossible, and (ii) aggregating prediction intervals is uninformative as an evaluation method.\n\nAs the reviewer correctly noted, merely comparing the intervals based on their width is difficult, as width does not directly map to coverage. Therefore, we interpret Fig. 7 as *suggestive* of the coverage (see Section 5.3) and perform the semi-synthetic experiment on the TCGA dataset to be able to properly evaluate our intervals on a medical dataset.\n\n**Action:**  We will address the challenges of real-world evaluation and clarify the meaning of Fig. 7 in the camera-ready version of our paper.\n\n&nbsp;\n\n**Section 4:**\n\nWe are happy to provide a more detailed explanation of our notation and the theoretical results, acknowledging that our paper may be notationally dense for readers unfamiliar with the related work. Specifically, we will discuss Section 4 based on (i) the reformulation as quantile regression in the high-level outline, (ii) the reformulation of Lemma 4.1 in terms of the dual optimization problem in Theorem 4.2 (and similarly the formulation of Eq. $P_S$ in terms of Theorem 4.5), and (iii) the challenges when employing our approach.\n\n**(i) Quantile regression** Eq. 34:\n\nEq. 3 & 4 represent the general form of a quantile regression as a minimization problem based on the so-called _pinnball-loss_ $l_{\\alpha}$, a standard result in the statistical regression literature (e.g. [1],[2]). Observe that $l_{\\alpha}$ can equivalently be represented as $l_{\\alpha}(\\theta, S) = (\\alpha - \\mathbb{1}_\\{[\\theta-S < 0]\\})(\\theta-S)$. This asymmetric linear loss penalizes under-prediction $(\\theta-S) > 0$ proportionally to $\\alpha$ and over-prediction $(\\theta-S) < 0$ proportionally to $1-\\alpha$, which results in the fitted function (see Eq. 3) targeting the $\\alpha$-th conditional quantile.\n\n**(ii) Dual optimization** Theorem 4.2 with primal optimization problem in Lemma 4.1: \n\nNote that directly **solving the problem given in Lemma 4.1 is not feasible**, as it would require solving Eq. 5 for all possible imputed values $S \\in \\mathbb{R}$ of $S_{n+1}$. It is unclear how to feasibly find the imputation of the “true” $S_{n+1}$ needed for the interval construction in Eq. 6. We thus exploit the efficient computation through dual optimization.\n\n\nWe can rewrite the primal problem in Lemma 4.1 as\n\n$$\n\\min_{\\theta > 0} \\quad \\sum_{i=m+1}^{n+1} (1-\\alpha)u_i + \\alpha v_i\\\\\n\\qquad\n\\text{ s.t. } S_i - \\theta \\frac{\\pi(a_i + \\Delta_A \\mid x_i)}{\\pi(a_i \\mid x_i)}) -u_i + v_i = 0, \\quad \\forall i=m+1\\ldots,n+1, S_{n+1} = S;\\\\\n\\quad\nu_i, v_i \\geq 0,  \\quad \\forall i=m+1\\ldots,n+1.\n$$\n\nNote that this is precisely the formulation of the respective primal problem stated in **Eq. P_S** as well as Theorem 4.5, where **$u_i$ and $v_i$ represent the primal optimization variables.**\n\n\nThe **Lagrangian** of the primal problem with Lagrangian multipliers $\\eta, \\gamma_1, \\gamma_2$ states\n\n$$\n\\mathcal{L} = \\sum_{i=m+1}^{n+1} (1-\\alpha) u_i + \\alpha v_i + \\sum_{i=m+1}^{n+1} \\eta_i \\left( S_i - \\theta \\frac{\\pi(a_i + \\Delta_A \\mid x_i)}{\\pi(a_i \\mid x_i)} - u_i+v_i \\right) - \\sum_{i=m+1}^{n+1} (\\gamma_{1_i}u_i + \\gamma_{2_i}v_i).\n$$\n\nleading to the **dual problem** \n\n$$\n\\max_{\\eta_i, i=m+1,\\ldots,n+1} \\min_{\\theta > 0} \\quad \\sum_{i=m+1}^{n} \\eta_i \\left(S_i - \\theta \\frac{\\pi(a_i + \\Delta_A \\mid x_i)}{\\pi(a_i \\mid x_i)} \\right) + \\eta_{n+1} \\left( S - \\theta \\frac{\\pi(a^{\\ast} \\mid x_{n+1})}{\\pi(a_{n+1} \\mid x_{n+1})} \\right)\n\\qquad\n\\text{ s.t.} -\\alpha \\leq \\eta_i \\leq 1-\\alpha, \\quad \\forall i=m+1,\\ldots,n+1, \n$$\n\nwith **dual optimization variables $\\eta_{m+1},...,\\eta_{n+1}$.** \n\nFor more background on **dual (convex) optimization theory**, we refer to, e.g., [3], [4]. Specific details on our derivations can be found in our **proofs of Theorems 4.2 and 4.5 in Appendix D**. For more details on the **coverage guarantees/reformulation from Eq. 6 to Eq. 9**, we refer to the respective part of our **proof of Thm. 4.2** in L 663ff in our manuscript.\n\n**(iii) Challenges:**\n\nThe main challenge of our approach is the need for sufficient prior knowledge to set a bound $M$ on the propensity estimation error (see “answer to limitations” below). However, this bound is far less restrictive than the assumption of a known propensity score, and is common in the literature (see Section 2). In practice, one aims for valid but also narrow intervals. If the calibration dataset is not representative of the full population, the resulting intervals are potentially wider than necessary for test samples in segments of the output space with few calibration data. For transparency, we discuss all limitations in Section 6 of our manuscript. \n\n**Action:** We will provide more background and explain the intuition for our derivations, as well as the challenges of our approach, in more detail in the camera-ready version.\n\n&nbsp;\n\n**Answer to questions: Numerical stability**\n\nWe are happy to discuss the numerical stability of our proposed optimization approach. In Scenario 1, the only source of potential instability can be a very low propensity score in low-overlap regions of the covariate space. This, however, is only a problem if $\\pi(a|x) <<\\pi(a+\\Delta|x)$. We can thus consider this unlikely in practice. For example, consider a patient who would be treated with a dosage of 10mg of some medication, which is prescribed in the range from 0 to 50mg. A practitioner is likely to be interested in the effect of an increase of the dosage to 15mg (i.e., $\\Delta=5$) in contrast to an increase to 50mg. Furthermore, it is reasonable to assume the propensity function to be locally smooth. Therefore, the instability of $\\pi(a|x) <<\\pi(a+\\Delta|x)$ is unlikely to occur.\n\nIn Scenario 2, we could additionally face an underflow of $\\exp(-\\frac{(a_i-a)^2}{2\\sigma^2})$ or a blow-up of the pre-factor $(\\sigma\\hat{\\pi}(x_i))^{-1}$. As a remedy, we can reparameterize the problem to work in the log domain and only exponentiate after subtracting a stable (maximum) constant across all datapoints. This additionally prevents the Jacobian/Hessian of the constraints from becoming nearly singular or wildly varying, causing Newton‐type steps to blow up or stall.\n\n**Action:** We will add a discussion on the stability of the optimization approaches to our manuscript.\n\n\n&nbsp;\n\n**Answer to limitations: Considerations for practical application**\n\n\nWe are happy to discuss the requirements of applying our method in practice. \nFirst, our method is designed for single **continuous treatments** with a univariate outcome, which is common in dosaging (e.g., determining the dosage of chemotherapy/ insulin/ hypertension drugs, etc). \n\n- In **randomized controlled trials** (when the propensity score is known), we can directly apply **Theorem 4.2** on top of the trained outcome prediction model $\\phi$ to construct our CP intervals at target coverage level $\\alpha$. \n- In observational studies, when the propensity score is **unknown**, we require the practitioner to have sufficient prior knowledge to set a **bound $M$** on the propensity estimation error. The practitioner should choose this bound with care and rather in a conservative way to prevent undercoverage. Then, the CP intervals can be derived based on **Theorem 4.5**. \n\nOur intervals do not suffer from undercoverage due to limited data. Importantly, CP guarantees are valid for **any sample size** and our method is kept fully **model agnostic**, enabling the practitioner to make reliable judgements based on limited data and an ML model of choice. \n\n**Action:** We will discuss practical considerations of applying our method in medicine. \n\n&nbsp;\n\n[1] Koenker, et al. Regression quantiles. Econometrica (1978): 33-50.\n\n[2] Koenker. Quantile regression. Vol. 38. Cambridge University Press, 2005.\n\n[3] Boyd, et al. Convex optimization. Cambridge University Press, 2004.\n\n[4] Luenberger, et al. Linear and nonlinear programming. Vol. 2. Reading, MA: Addison-Wesley, 1984."}]

Author rebuttal:

Meta-review: Evaluating causal effects with continuous treatments from observational data poses interesting challenges. First of all, there are the usual challenges of causal analysis (such as estimating propensity scores). Second, there is the fact that one rarely if ever sees two sample points that have received the same treatment, which requires some sort of smoothing over "treatment space". 

The present paper applies the methodology of conformal prediction to this task. The goal in the present setting is this: after training a model and calibrating CP, one wants to be able to produce, upon seeing a new covariate X_{n+1}, an interval containing the true treatment effect with high probability. The treatment to be applied to X_{n+1} might either be chosen by the user, or assumed to come from the same distribution. The assumption needed here is fairly strong, but quite classical: positivity, consistency, and unconfoundedness. Moreover, the authors assume that the propensity score is either known, or can be estimated up to a constant factors. 

The actual proposed method is kind of CP with covariate shift. However, it is quite subtle, as it requires an appeal to duality theory. As is usual for CP, the method gives no guarantees on the sizes of intervals, but it does give valid coverage at the desired level (at least asymptotically). In any case, experiments (including some reported in the discussion) suggest that it does have good coverage and decent-sized intervals in practice. 

The main strengths of the paper are the novel setting and the clearly important contribution of providing uncertainty estimates for treatment effects. On this point I confess that I should have probably pressed the reviewers further: the method, even though limited by its assumptions and intense computational cost, seems like an important contribution. The problem formulation and the proposed solution are quite interesting. (All reviewers acknowledged as much, but in a somewhat lukewarm fashion.)

On the weaknesses of the paper, I concur with the reviewers only partially. The paper does lack more extensive experiments and details on runtimes (though the authors intend to add more info on these points to the final version). On the other hand, it is clear that the "simple baselines" they suggested could never provide appropriate coverage. Some of the reviewers' comments were somewhat misguided, like the questions about l_\alpha (the "pingpong" loss that is standard in quantile regression) and about the assumptions (the ones used here are standard in the area). 

Overall, I found the discussion of the paper somewhat unproductive and a bit unfair to the paper. I tend to side with the authors insofar as they were able to answer all relevant questions. Whatever they did not answer fully seems better left for future work. 

My reading of the paper is that its merits outweigh its shortcomings by some margin. As pointed out above, the approach proposed is limited by its assumptions. These assumptions, however, are standard, and thus fine for a first paper on a topic. The comments on experiments are fine. Then again, I do not think that the first paper in a topic needs to be "complete" in this sense. Reading other papers in the causal area suggests that the present set of experiments is not atypically smaller than what one sees out there. 

Therefore, I suggest that the paper be accepted.
Decision: Accept (poster)