Title: Quantifying Aleatoric Uncertainty of the Treatment Effect: A Novel Orthogonal Learner
Venue: NeurIPS 2024
Accepted: yes
Keywords: causal inference

treatment effect estimation
Abstract: Estimating causal quantities from observational data is crucial for understanding the safety and effectiveness of medical treatments. However, to make reliable inferences, medical practitioners require not only estimating averaged causal quantities, such as the conditional average treatment effect, but also understanding the randomness of the treatment effect as a random variable. This randomness is referred to as aleatoric uncertainty and is necessary for understanding the probability of benefit from treatment or quantiles of the treatment effect. Yet, the aleatoric uncertainty of the treatment effect has received surprisingly little attention in the causal machine learning community. To fill this gap, we aim to quantify the aleatoric uncertainty of the treatment effect at the covariate-conditional level, namely, the conditional distribution of the treatment effect (CDTE). Unlike average causal quantities, the CDTE is not point identifiable without strong additional assumptions. As a remedy, we employ partial identification to obtain sharp bounds on the CDTE and thereby quantify the aleatoric uncertainty of the treatment effect. We then develop a novel, orthogonal learner for the bounds on the CDTE, which we call AU-learner. We further show that our AU-learner has several strengths in that it satisfies Neyman-orthogonality and, thus, quasi-oracle efficiency. Finally, we propose a fully-parametric deep learning instantiation of our AU-learner.

Quantifying Aleatoric Uncertainty of the Treatment Effect: A Novel Orthogonal Learner

Abstract

Estimating causal quantities from observational data is crucial for understanding the safety and effectiveness of medical treatments. However, to make reliable inferences, medical practitioners require not only estimating averaged causal quantities, such as the conditional average treatment effect, but also understanding the randomness of the treatment effect as a random variable. This randomness is referred to as aleatoric uncertainty and is necessary for understanding the probability of benefit from treatment or quantiles of the treatment effect. Yet, the aleatoric uncertainty of the treatment effect has received surprisingly little attention in the causal machine learning community. To fill this gap, we aim to quantify the aleatoric uncertainty of the treatment effect at the covariate-conditional level, namely, the conditional distribution of the treatment effect (CDTE). Unlike average causal quantities, the CDTE is not point identifiable without strong additional assumptions. As a remedy, we employ partial identification to obtain sharp bounds on the CDTE and thereby quantify the aleatoric uncertainty of the treatment effect. We then develop a novel, orthogonal learner for the bounds on the CDTE, which we call AU-learner. We further show that our AU-learner has several strengths in that it satisfies Neyman-orthogonality and, thus, quasi-oracle efficiency. Finally, we propose a fully-parametric deep learning instantiation of our AU-learner.

1 Introduction

Estimating causal quantities from observational data is crucial for decision-making in medicine [9, 12, 22, 30, 70]. For example, medical practitioners are interested in estimating the effect of chemotherapy vs. immunotherapy on patient survival from electronic health records to understand the best treatment strategies in cancer care. Here, common estimation targets are averaged causal quantities such as the average treatment effect (ATE) and the conditional average treatment effect (CATE), yet averaged causal quantities do not allow for understanding the variability of the treatment effect.

What is needed for the reliability of causal quantities in medicine? To obtain reliable causal quantities, one often needs to 'move beyond the mean' [44, 68] and consider the inherent randomness in the treatment effect as a random variable. This randomness is referred to as aleatoric uncertainty [17, 60, 110]. Quantifying the aleatoric uncertainty of the treatment effect is relevant in medical practice to understand the probability of benefit from treatment [26, 60] and the quantiles and variance of the treatment effect [5, 17, 26, 33, 59]. As an example, averaged quantities such as the CATE would simply suggest a positive effect for some patients, while the probability of benefit from treatment can inform patients about the odds of being negatively affected by the treatment. Hence, aleatoric uncertainty of the treatment effect promises additional, fine-grained insights beyond simple averages.

Methods for quantifying the aleatoric uncertainty of the treatment effect have gained surprisingly little attention in the causal machine learning community. So far, machine learning for treatment

conditional average treatment effect (CATE)

CDF of conditional distribution of treatment effect (CDTE)

Identification

Estimation

Treatment effect,

Patient  characteristic,

Treatment effect,

Patient  characteristic,

CDF

Point identification (backdoor adjustment)

Partial identification (Makarov bounds)

Mean squared error (MSE)

or

Target

risk

Covariate

adjusted

learner

Orthogonal

learner

Plug-in

estimator

(see. Eq. 5)

One-step bias correction

(see Eq. 13)

(see Eq. 14)

Challenges

X

X

1

Target

estimand

X

X

2

scaled one-step bias correction

DR-learner

AU-learner (CRPS)

AU-learner  (

)

Continuous ranked probability score (CRPS)

Squared Wasserstein-2 distance  (

)

?

X

X

3

or

effect estimation was primarily focused on estimating averaged causal quantities [20, 57, 64, 69, 96, 98, 114, 125, 128]. Some research aims to quantify the epistemic uncertainty in treatment effect estimation [51] or the total uncertainty (but without distinguishing the types of uncertainty) [1, 67, 76]. Other works focused on the aleatoric uncertainty of the potential outcomes [13, 31, 65, 66, 94] 1 or on contrasts between distributions of potential outcomes (also known as distributional treatment effects) [16, 29, 62, 92, 100]. 2 However, to the best of our knowledge, there is no comprehensive meta-learning theory for the estimation of the aleatoric uncertainty in the treatment effect .

In this paper, we aim to quantify the aleatoric uncertainty of the treatment effect at the covariateconditional level in the form of a conditional distribution of the treatment effect (CDTE). Knowing the CDTE would automatically allow one to compute the above-mentioned quantities of aleatoric uncertainty, namely, the probability of benefit from treatment and the quantiles and variance of the treatment effect, at both population and covariate-conditional levels.

1.1 Challenges

Yet, the identification and estimation of the CDTE in contrast to CATE come with three challenges as follows (see Fig. 1):

Challenge 1 is that the CDTE does not allow for point identifiable , neither in the potential outcomes framework nor in randomized control trials due to the fundamental problem of causal inference as counterfactual outcomes can not be observed [26, 88]. We thus employ partial identification [48] to obtain bounds on the CDTE and thereby quantify the aleatoric uncertainty of the treatment effect. Specifically, we focus on Makarov bounds [87, 126, 129] that give sharp bounds for both the cumulative distribution function (CDF) and the quantiles of the CDTE.

Challenge 2 is that there is no closed-form expression of the target estimand in terms of nuisance functions. Because of this, existing CATE learners cannot be directly adapted to our task of estimating Makarov bounds. For example, there are no orthogonal learners in the general setting, and existing

approaches only use naïve plug-in estimators/learners. Furthermore, even the derivation of the orthogonal loss is non-trivial as there is no efficient influence function at hand for the Makarov bounds.

Challenge 3 is that CATE is an unconstrained target estimand whereas Makarov bounds are monotonous and contained in the interval [0 , 1] . Notably, any constraints of the target estimand could be violated by orthogonal learners [71, 125]. Therefore, an orthogonal learner for Makarov bounds needs to be carefully adapted, especially to perform well in low-sample settings.

1.2 Our contributions

In this paper, we develop a novel, orthogonal learner for estimating Makarov bounds which we call AU-learner , which allows to quantify the a leatoric u ncertainty of the treatment effect. Our AU-learner addresses all of the above-mentioned challenges 1 -3 . Further, our AUlearner has several useful theoretical properties, such as satisfying Neyman-orthogonality and, thus, quasi-oracle efficiency [98]. Finally, we propose a flexible, fullyparametric deep learning instantiation of our AU-learner . For this, we make use of conditional normalizing flows and call our method AU-CNFs.

To summarize, our contributions are as follows: 3

Wederive a novel, orthogonal leaner called AU-learner to quantify the aleatoric uncertainty of the treatment effect. For this, we estimate Makarov bounds on the CDF/quantiles of the conditional distribution of the treatment effect (CDTE).

We prove several favorable theoretical properties of our AU-learner , such as Neyman-orthogonality and, thus, quasi-oracle efficiency.

Wepropose a flexible deep learning instantiation of our AU-learner based on conditional normalizing flows, which we call AU-CNFs, and demonstrate its effectiveness over several benchmarks.

2 Related Work

In the following, we briefly summarize the existing works on uncertainty quantification in the potential outcomes framework; on the identification of the CDTE; and on the estimation of Makarov bounds. For a more detailed overview of literature, we refer to Appendix A.

Uncertainty quantification in the potential outcomes framework. The (total) uncertainty of a predictive model in machine learning is generally split into (a) epistemic and (b) aleatoric uncertainty [41, 50]. 4 This split is important, as it informs a decision-maker about the source of uncertainty (see Fig. 2), especially in the context of the potential outcomes framework. (a) Epistemic uncertainty was studied for predictive models targeting at identifiable averaged causal quantities, such as conditional average potential outcomes (CAPOs) and CATE [51, 52]. (b) Aleatoric uncertainty, on the other hand, is only identifiable for potential outcomes [88]. Prominent methods focus on interventional (counterfactual) quantities such as: (i) CDF/quantiles estimation [6, 13, 31, 43, 86]; (ii) density estimation [65, 66, 91, 94, 97, 102, 124]; and (iii) distributional distances (also known as distributional treatment effects) estimation [16, 29, 62, 92, 100]. Yet, our work differs substantially from the above, as we aim at inferring the aleatoric uncertainty of the treatment effect, which is only partially identifiable.

Challenge

Treatment effect,

Patient  characteristic,

Partial identification

of aleatoric uncertainty

Total

uncertainty

Epistemic

uncertainty

Treatment effect,

Patient  characteristic,

?

?

Aleatoric uncertainty

?

1

Work

Covariate-

conditional

Limitations

Estimator/

learner

Orthogonal

Instantiation

Fan et al. [26]

(

✓

)

-

Plug-in

✗

Empirical CDF

Ruiz et al. [110]

✓

-

Plug-in

✗

Conditional mean with hold-out residuals

Lee [77], Cui et al. [17]

✓

-

Plug-in

✗

Kernel density estimation

Kallus [60]

✗

Binary outcome

A-IPTW/NO

✓

Random forest & causal forest

✗

Optimization assumptions

A-IPTW/NO

✓

Homoskedastic Gaussian linear model

✗

Propensity score is known

IPTW

✓

-

Our paper

(

AU-learner

)

✓

-

A-IPTW/NO

✓

Conditional normalizing flows (AU-CNFs)

(A-)IPTW: (augmented) inverse propensity of treatment weighted; NO: Neyman-orthogonal

Identification of the distribution of the treatment effect. Point identification of the distribution of the treatment effect (or, equivalently, a joint distribution of potential outcomes) is only possible under additional assumptions on the data-generating mechanism. A common example is, e. g., invertibility of latent outcome noise [2, 3, 6]. Other works have rather focused on partial identification [48]. For example, [88] proposed sharp bounds for the distribution of the treatment effect under a monotonicity assumption. Later, assumption-free sharp bounds were proposed for both the joint CDF of potential outcomes [26] and for the variance of treatment effect [5], both known as Fréchet-Hoeffding bounds [39, 49]. Finally, [26, 28, 33] proposed sharp bounds on the CDF/quantiles of the treatment effect without any additional assumptions, so-called Makarov bounds [87, 126]. Makarov bounds were further generalized [81, 129] and applied to other settings [17, 25, 27, 36, 59] but different from ours.

Estimation of Makarov bounds. Table 1 provides a comparison of key methods for estimating Makarov bounds, at both covariate-conditional and population levels. Existing methods build mainly upon plug-in (single-stage) estimators/learners. Examples are methods tailored for randomized controlled trials [26, 110] and for potential outcomes framework [17, 77, 110]. Crucially, these methods are not orthogonal and, thus, are sensitive to the misspecification of the nuisance functions. Nevertheless, we include the latter methods [17, 77] as baselines for our experiments as they use a highly flexible CDF estimator based on kernel density estimators. Some works also developed efficient estimators for Makarov bounds at the population level (analogous to the two-stage Neymanorthogonal learners at the covariate-conditional level) but only in highly restricted settings. In particular, [60] is restricted to binary outcomes, [112] assumed a known propensity score, and [54] made special optimization assumptions. 5 In addition, all three works [54, 60, 112] suggest fixing a value of δ/α , which the CDF/quantiles of the treatment effect are evaluated at; when our work suggests targeting at several values of δ/α at once. Therefore, the previous methods are not applicable to our general setting of estimating covariate-conditional level Makarov bounds.

Research gap. To the best of our knowledge, we are the first to propose an orthogonal learner for estimating Makarov bounds on the CDF/quantiles of conditional distribution of the treatment effect.

3 Identification of Distribution of Treatment Effect

Notation. Let capital letters X,A,Y, ∆ denote random variables and small letters x, a, y, δ their realizations from domains X , A , Y , ∆ . Let P ( Z ) denote a distribution of some random variable Z , and let P ( Z = z ) be the corresponding density or probability mass function. Furthermore, π ( x ) = P ( A = 1 | X = x ) is propensity score, µ a ( x ) = E ( Y | X = x, A = a ) are conditional expectations, and F a ( y | x ) = P ( Y ≤ y | x, a ) is a conditional outcome CDF. For other conditional quantities or distributions, we use short forms whenever possible; e. g., E ( Y | x ) = E ( Y | X = x ) . Further, P n { f ( Z ) } = 1 n ∑ n i =1 f ( z i ) is a sample average of a random f ( Z ) , where n is the sample size. We denote linear rectifier functions as [ x ] + = max( x, 0) and [ x ] -= min( x, 0) , and sup/inf convolutions of two functions [116] f 1 ( · | x ) , f 2 ( · | x ) as ( f 1 ∗ f 2 ) Y ( δ | x ) = sup y ∈Y { f 1 ( y | x ) -f 2 ( y -δ | x ) } and ( f 1 ∗ f 2 ) Y ( δ | x ) = inf y ∈Y { f 1 ( y | x ) -f 2 ( y -δ | x ) } .

Problem setup. We consider the standard setting of the Neyman-Rubin potential outcomes framework [109]. That is, we have an observational dataset D with a binary treatment A ∈ A = { 0 , 1 } , potentially high-dimensional covariates X ∈ X ⊆ R d x and a continuous outcome Y ∈ Y ⊆ R . For instance, a typical scenario is in cancer therapy, where the outcome is tumor growth, the treatment is whether chemotherapy is given, and the covariates include patient details like age and sex. We define a joint random variable Z = ( X,A,Y ) . D = { x i , a i , y i } n i =1 is sampled i.i.d. from the observational

distribution P ( Z ) = P ( X,Y,A ) , where n is the sample size. The potential outcomes framework then makes three (causal) assumptions, i. e., (1) consistency : if A = a , then Y [ a ] = Y ; (2) overlap : P (0 ≤ π ( X ) ≤ 1) = 1 ; and (3) exchangeability : A ⊥ ⊥ ( Y [0] , Y [1]) | X .

Treatment effect distribution. In this paper, we refer to the treatment effect ∆ = Y [1] -Y [0] as a random variable. 6 The CATE is given by τ ( x ) = E (∆ | x ) , which is identifiable as µ 1 ( x ) -µ 0 ( x ) under the causal assumptions (1)-(3). We are interested in identifying a conditional distribution of the treatment effect (CDTE) , specifically, its CDF or quantiles:

The CDF and the quantiles of the CDTE are point non -identifiable due to the fundamental problem of causal inference, i. e., that the counterfactual outcome, Y [1 -A ] , is never observed. This is illustrated in Fig. 3, where both conditional potential outcome distributions are identifiable as P ( Y [ a ] | x ) = P ( Y | x, a ) ; but a conditional joint distribution, P ( Y [0] , Y [1] | x ) , and the CDTE, P (∆ | x ) , are not.

Partial identification of the CDTE. Fan et al. [26] proposed pointwise sharp bounds on the CDF and the quantiles of the CDTE, so-called Makarov bounds [87, 126, 129]. Given that the outcome Y is continuous, the Makarov bounds for the CDF of the CDTE are given by linearly rectified sup/inf convolutions [116] of conditional CDFs of potential outcomes:

where δ ∈ ∆ , and F a ( y | x ) is the CDF of P ( Y [ a ] | x ) . Similarly, Makarov bounds can be formulated for the quantiles of the CDTE:

̸

̸

where α ∈ [0 , 1] , and F -1 a ( u | x ) are the quantiles of P ( Y [ a ] | x ) . Then, under the causal assumptions (1)-(3), conditional distributions of potential outcomes coincide with observed ones, i. e., P ( Y [ a ]) = P ( Y | x, a ) . Notably, Makarov bounds on the CDFs are CDFs themselves, but these CDFs do not correspond to the solution of the partial identification task (which implies pointwise sharpness). We refer to Appendix B for more illustrations about the inference of the Makarov bounds, the explanation of pointwise sharpness, and Makarov bounds for categorical/mixed-type outcomes.

4 An AU-learner for estimating Makarov bounds

In the following, we develop a theory of orthogonal learning for Makarov bounds, which then gives rise to our AU-learner . For this, we first review the plug-in learner and its shortcomings. Motivated by this, we then derive two-stage learners. Here, we first present a novel CA-learner in an intermediate step and finally our AU-learner . Note that both are novel but we frame our contributions around the AU-learner because of favorable theoretical properties. For notation, we use over- and underlines as in, e. g., F ( δ | x ) to refer to the upper and/or lower bound.

4.1 Single-stage learners for Makarov bounds

Plug-in learner. A naïve way to construct estimators for Makarov bounds [17, 26, 77, 110] is to estimate conditional outcome CDFs, ̂ F a ( y | x ) , and plug them into Eq. (3):

The plug-in learner for the bounds of the quantiles, ̂ F PI -1 ( α | x ) , can be obtained in a similar way but where one uses Eq. (4) and relies on the estimators of either the conditional outcome quantiles or inverse of the estimated conditional outcome CDFs, ̂ F -1 a ( u | x ) .

Γ&ΓKΓDΓOΓOΓHΓQΓJΓH

Γˇ

Shortcomings. The plug-in learner suffers from two important shortcomings [96]. (a) The plug-in learner does not account for the selection bias, meaning that F 1 is estimated better for the treated population and F 0 for the untreated. Hence, it could be necessary to re-weight the loss wrt. to the propensity score. A remedy is to employ an inverse propensity of treatment weighted (IPTW) learner for both F 0 and F 1 [6, 43] (see Appendix C). (b) The plug-in learner does not target the Makarov bounds directly but rather the conditional outcome distributions. Therefore, it is unclear how to incorporate an inductive bias that the Makarov bounds are less heterogeneous than either of the conditional outcome CDFs (i. e., the Makarov bounds can depend on a subset of covariates X ). The second shortcoming thus motivates our derivation of two-stage learners.

4.2 Two-stage learners for Makarov bounds

In order to address the above shortcomings of the plug-in learners, a two-stage learning theory was proposed [15, 35, 72, 96]. Yet, the two-staged learning theory is primarily built for simple target estimands (e. g., CATE) and therefore requires non-trivial adaptations by us to extend to Makarov bounds, which we do in the following.

Working model & target risk. Our two-stage learners seek to find the best approximation of the ground-truth Makarov bounds, functional target estimands, by a (parametric) working model G . Formally, the working model is given G = { g ( δ, x ) | g : ∆ ×X → [0 , 1]; g ( · , x ) is non-decreasing } . The best approximation g ∗ is then obtained by minimizing a (population) target risk via g ∗ = arg min g ∈G L ( g ) . 7 In our setting, L is chosen as some distributional distance between the target estimands and the working model. Specifically, we use continuous ranked probability score (CRPS) [40, 117] as a target risk for learning Makarov bounds on the CDF via

or squared Wasserstein-2 distance as a target risk for learning Makarov bounds on the quantiles via

The target risks in Eq. (6) and (7) cannot be directly minimized as we do not observe the ground-truth Makarov bounds. Yet, due to the identifiability results in Sec. 3, the Makarov bounds depend on the nuisance functions, F a ( y | x ) , which can be estimated from the observational data. We perform that in the following.

From now on, we denote the ground-truth nuisance functions as η and their estimates as ˆ η . Also, we make the dependence on the target risks of the nuisance functions explicit; that is, we write the target risk as L ( g, η ) and the target estimands as F ( δ | X ; η ) and F -1 ( α | X ; η ) .

Covariate-adjusted learner. A straightforward way to estimate and then minimize the target risk is to plug-in the estimates of conditional outcome CDFs, ̂ F a ( y | x ) , into Eq. (6) and (7), respectively. This yields the so-called covariateadjusted (CA) learner , which aims at minimizing the following losses (empirical risks):

where we call both F PI ( δ | x ; ˆ η ) = ̂ F PI ( δ | x ) and F PI -1 ( α | x ; ˆ η ) = ̂ F PI -1 ( α | x ) pseudo-CDFs and pseudo-quantiles , respectively, and where both can be obtained from Eq. (5).

The CA-learner addresses the shortcoming (b) of the plug-in learner from above in that loss minimization in the equations above targets directly at Makarov bounds. However, the shortcoming (a) of the selection bias still persists. Furthermore, a new shortcoming (c) now emerges: The losses can be highly sensitive to badly estimated nuisance functions so that ̂ L PI ( g, η ) and ̂ L PI ( g, ˆ η ) differ significantly. Next, we develop an orthogonal learner that addresses all of the shortcomings.

One-step bias correction. In order to address the before-mentioned shortcomings of the CA-learner, we employ the concept of (Neyman-)orthogonal losses [15, 35]. Informally, orthogonal losses are first-order insensitive to the misspecification of the nuisance functions, which introduces many favorable properties such as quasi-oracle efficiency [98] and double robustness [127]. The CA-learner losses in Eq. (8) and (9) can be made orthogonal by performing a one-step bias correction [10, 63]. The one-step bias correction requires the knowledge of an efficient influence function , which has not yet been derived for Makarov bounds ( → Challenge 2 ). Hence, the following theorem presents one of our main theoretical results.

Theorem 1 (Efficient influence function for Makarov bounds) . Let P denotes P ( Z ) = P ( X,A,Y ) , and let y ∗ Y ( · | x ) and u ∗ [ α, 1] ( · | x ) be argmax/argmin sets of the convolutions ( F 1 ∗ F 0 ) Y ( · | x ) and ( F -1 1 ∗ F -1 0 ) [ α, 1] ( · -0 | x ) , respectively. Then, under mild conditions on the conditional outcome distributions and for almost all values of δ ∈ ∆ and for all values of α ∈ (0 , 1) (see Appendix D), average Makarov bounds are pathwise differentiable. Further, the corresponding efficient influence functions, ϕ , are as follows:

where I ( X ; η ) = ✶ { ( F 1 ∗ F 0 ) Y ( δ | X ) > 0 } ; y ∗ is some value from the finite set y ∗ Y ( δ | X ) ; u ∗ is some value from the finite set u ∗ [ α, 1] ( α | X ) ; and C ( δ, Z ; η ) and C -1 ( δ, Z ; η ) can be then obtained by swapping the symbols {∗ , >, y ∗ Y , u ∗ [ α, 1] , -0 , +0 } to {∗ , <, y ∗ Y , u ∗ [0 ,α ] , -1 , +1 } .

In the above theorem, we use red color to show the nuisance functions of P that are influencing the target estimand, i. e., averaged Makarov bounds. Therein, we also provide a Corollary 1, where we derive efficient influence functions for the target risks from Eq. (6) and (7), namely ϕ ( L ( g ); P ) .

Note on infinite argmax/argmin sets. When argmax/argmin sets of the sup/inf-convolutions are infinite, average Makarov bounds are pathwise non-differentiable, and, thus, one-step bias correction is not possible as statistical inference becomes non-regular [47]. This result also holds for other causal quantities that contain sup/inf operators (e. g., for the policy value of the optimal treatment strategy [84, 106]). Although, there exist approaches to perform inference in the non-regular setting [84], we focus solely on the regular setting where pathwise differentiability holds (see Appendix D for a discussion on the generality of such a setting).

Orthogonal leaner ( AU-learner ). Given the derived efficient influence function for the target risks, we perform a γ -scaled one-step bias correction [63] of the CA-learner losses, namely, ̂ L PI ( g, ˆ η ) + γ P n { ϕ ( L ( g ); ˆ P ) } . The latter then yields our novel orthogonal AU-learner (see Corollary 2 in Appendix D). Our AU-learner effectively resolves all the above-mentioned shortcomings (see a comparison in Fig. 4). Formally, it aims at minimizing one of the following losses:

where C ( δ, Z ; ˆ η ) and C -1 ( α, Z ; ˆ η ) are given by Eq. (11) and (12), respectively; and γ ∈ (0 , 1] is a scaling hyperparameter. We present a meta-algorithm of our AU-learner (with the CRPS target risk) based on cross-fitting in Algorithm 1 ( AU-learner with the W 2 2 target risk follows analogously).

Scaling hyperparameter. The scaling hyperparameter γ is introduced to tackle Challenge 3 from above, namely, that the pseudo-CDF term, F AU ( δ, Z ; ˆ η, γ ) , is not guaranteed to be a valid CDF for γ > 0 (both monotonicity wrt. δ and [0 , 1] -constraint can be violated). 8 The same happens with the pseudo-quantiles of the AU-learner , F AU -1 ( α, Z ; ˆ η, γ ) , which could be non-monotonous wrt. α . The main intuition behind scaling is that it interpolates between the full AU-learner ( γ = 1 ), that has favorable theoretical properties; and the CA-learner ( γ = 0 ), for which the pseudo-CDFs and pseudoquantiles are valid CDFs and quantiles, respectively (we refer to Appendix E with visual examples). Hence, scaling mimics a learning rate of a Newton-Raphson method (usually considered as an analogy to the one-step bias correction [34]). We found fixed values for the scaling hyperparameter γ to work well in all of our experiments and to improve the low-sample performance of our AU-learner .

4.3 Theoretical properties of AU-learner

In the following, we formulate our second main theoretical result. For the results to hold, the nuisance functions η = ( π, F 0 , F 1 ) /η = ( π, F -1 0 , F -1 1 ) need to be estimated independently from the second stage model g/g -1 . This could be done by either assuming a not-too-flexible class of models (such as a Donsker class of estimators and fitting all the models on the same dataset D ) or by using a generic approach of cross-fitting [15, 35].

Theorem 2 (Neyman-orthogonality of

AU-learner (informal)) . Under the assumptions of the Theorem 1, the following holds for AU-learner from Algorithm 1 with the scaling hyperparameter γ = 1 :

Neyman-orthogonality. Losses in Eq. (13) and Eq. (14) are first-order insensitive wrt. to the misspecification of the nuisance functions.

Algorithm 1 AU-learner (CRPS) via cross-fitting

̸

1: Input. Training dataset D = { x i , a i , y i } n i =1 , scaling γ ∈ (0 , 1] , folds K ≥ 2 , δ -grid { δ j ∈ ∆ } n δ j =1 2: Output. Estimator of Makarov bounds ̂ g ( δ, x ) 3: for k ∈ { 1 , . . . , K } do ▷ First stage 4: Use { x i , a i , y i } k -1 =( i mod K ) to fit ˆ η = (ˆ π, ̂ F 0 , ̂ F 1 ) 5: for i : k -1 = ( i mod K ) do 6: Use ˆ η to infer the pseudo-CDF for the δ -grid: { F AU ( δ j , z i ; ˆ η ) } n δ j =1 (Eq. (13)) 7: end for 8: end for

9: Fit the working model based on δ -grid: ▷ Second stage ̂ g = arg min g ∈G ̂ L AU, CRPS ( g, ˆ η )

Quasi-oracle efficiency. The bias from the misspecification of the nuisance functions is of second order.

We refer to the Appendix D for the detailed formulation of the theorem and the proof. Notably, the quasi-oracle efficiency [98] means that our AU-learner with (sufficiently fast) estimated nuisance functions performs nearly identical to the AU-learner with the ground-truth nuisance functions.

5 Neural instantiation with AU-CNFs

Wenow introduce a flexible fully-parametric instantiation of our AU-learner , which we call AU-CNFs. Therein, we employ conditional normalizing flows (CNFs) [105, 121] as the main backbone for our AU-learner . CNFs are a flexible neural probabilistic method with tractable conditional densities, CDFs, and quantiles. Importantly, all three attributes of the CNFs (densities, CDFs, and quantiles) can be used for both training via back-propagation and inference [115], which makes them a perfect model for both stages of our AU-learner .

Architecture. The architecture of our AU-CNFs is inspired by interventional normalizing flows (INFs) [94] (a two-stage model for efficient estimation of potential outcomes densities). Our AUCNFs consist of several CNFs corresponding to the two stages of learning, namely a nuisance CNF , which fits the nuisance functions, (ˆ π, ̂ F 0 , ̂ F 1 ) or, equivalently, (ˆ π, ̂ F -1 0 , ̂ F -1 1 ) ; and two target CNFs , which implement second stage working models for upper and lower bounds, G and G , respectively.

Training & implementation. At the first stage of AU-CNFs learning, the nuisance CNF aims at maximizing the conditional log-likelihood and minimizing a binary cross-entropy via a joint loss. Then, we generate the pseudo-CDFs and pseudo-pseudo quantiles, as described in Algorithm 1. Therein, we set the δ / α -grid size to n δ = n α = 50 and discretize the Y -space/ [0 , 1] -interval to infer the argmax/argmin values, ˆ y ∗ / ˆ u ∗ . Then, we proceed with the second stage of AU-CNFs learning, where we set γ = 0 . 25 for the CRPS loss and γ = 0 . 01 for the W 2 2 loss. We found the fixed values of γ to work well in all of the synthetic and semi-synthetic experiments (except for the IHDP100 dataset, where the overlap assumption is violated). We use the same training data for two stages of learning, as (regularized) CNFs as neural networks belong to the Donsker class of estimators [123]. We refer to Appendix F for more details on our AU-CNFs.

6 Experiments

We now evaluate our AU-learner . For this, we use (semi-)synthetic benchmarks with the ground-truth conditional CDFs/quantiles of potential outcomes, i. e., F a ( y | x ) / F -1 a ( u | x ) . In this way, we can infer the ground-truth Makarov bounds and use them for evaluation.

Evaluation metric. We use evaluation metrics based on the target risks (as introduced in Sec 4.2). Specifically, we report root continuous ranked probability score (rCRPS) and Wasserstein-2 distance ( W 2 ) based on training data (in-sample) and test data (out-sample).

Baselines. We compare the proposed hierarchy of learners from Sec. 4 with CNFs as backbones. These are the plug-in learner (Plug-in CNF) , IPTW-learner (IPTW-CNF) , CA-learners (CA-CNFs (CRPS / W 2 2 )) , and AU-learners (AU-CNFs (CRPS / W 2 2 )) . The only relevant baseline found in the literature is a plug-in learner based on kernel density estimation [17, 77, 110]. 9 For this, we used distributional kernel mean embeddings [97] (Plug-in DKME) , a standard conditional kernel density estimation method. Details on the baselines are in Appendix G.

Synthetic data. We adapt the synthetic data generator ( d x = 2 ) from [61, 93] by creating three settings with different conditional outcome distributions: normal, multi-modal, and exponential (see data generation details in Appendix H). In the synthetic data, the ground-truth Makarov bounds are less heterogeneous than the potential outcomes , and, hence, two-stage learners are expected to perform the best. We sample n train ∈ { 100; 250; 500; 750; 1000 } training and n test = 1000 test datapoints. The out-sample results are in Fig. 5. Here, our AU-CNFs perform the best wrt. rCRPS in the majority of settings and different sizes of training data. We also report the results wrt. W 2 2 in Appendix I.

0

.

2

0

.

3

0

.

4

0

.

5

rCRPS

out

S: normal

|

B: upper

S: multi-modal

|

B: upper

S: exponential

|

B: upper

250

500

750

1000

n

train

0

.

2

0

.

3

0

.

4

0

.

5

rCRPS

out

S: normal

|

B: lower

250

500

750

1000

n

train

S: multi-modal

|

B: lower

250

500

750

1000

n

train

S: exponential

|

B: lower

Method

Plug-in learner

Plug-in DKME

Plug-in CNF

IPTW-learner

IPTW-CNF

CA-learner

CA-CNFs (CRPS)

CA-CNFs (

W

2

2

)

AU-learner

AU-CNFs (CRPS)

AU-CNFs (

W

2

2

)

HC-MNIST dataset. HC-MNIST is a high-dimensional semi-synthetic dataset ( d x = 785 ), built on top of the MNIST dataset [52] (see details in Appendix H). Here, the heterogeneity of Makarov bounds is also smaller than that of potential outcomes, reflecting inductive biases in the real world. We report the out-sample performance of different methods in Table 2 (the Plug-in DKME is omitted due to a too-long runtime). Therein, our AU-CNFs (CRPS) achieve the best performance and, thus, scale well with the dataset size and the dimensionality of covariates. Further, our AU-CNFs ( W 2 2 ) improve the performance of CA-CNFs ( W 2 2 ). In general, we observe that the loss based on the CDF distance (i. e., CRPS) has a lower variance and is easier to fit. In Appendix I, we additionally report the results for another popular semi-synthetic benchmark, IHDP100 [46, 114].

Case study. In Appendix J, we provide a real-world case study based on the observational dataset from [7]. Therein, we demonstrate how our AU-learner (AU-CNFs) can be used to estimate the effectiveness of lockdowns during the COVID-19 pandemic. We estimate the probability of benefit from intervention (a special case of Makarov bounds with δ = 0 ). As expected, we observe a drop in the incidence rate is highly probable after the implementation of a strict lockdown.

7 Discussion

Low-sample & asymptotic performance. In several experiments, especially in low-sample settings, the CA-learner or even the plug-in approach are performing nearly as well or even sometimes better than the AU-learner . This can be expected, as the best low-sample learner and the asymptotically best learner can, in general, be different [20], and there is no single 'one-fits-all' data-driven solution to choose the former one [19]. This can be explained by too small dataset sizes or the severe overlap violations (as is the case with the IHDP100 dataset; see Appendix I). Yet, only our quasi-oracle efficient AU-learner offers asymptotic properties in the sense that it is asymptotically closest to the oracle (see Fig. 4). We thus argue for a pragmatic choice in practice (i. e., in the absence of ground-truth counterfactuals or additional RCT data) where our AU-learner should be the preferred method for the covariate-conditional Makarov bounds even in low-sample data.

Future work. Our work sets a foundation for several extensions to estimate covariate-conditional Makarov bounds. For example, the estimation of the interval probabilities (see Appendix B) of the treatment effect can provide a connection with the existing works on total uncertainty with conformal prediction [1]. Additionally, one might want to study possible extensions of Makarov bounds tailored to high-dimensional outcomes.

Limitations & broader impact. Our work is subject to the standard assumptions of the potential outcomes framework. We further make assumptions on the outcome distribution, though these are very mild. Nevertheless, we expect our work to have a positive impact, as it will help to improve the reliability of decision-making in medicine and other safety-critical fields.

Conclusion. We are the first to offer a theory of orthogonal learning to quantify the aleatoric uncertainty of the treatment effect at the covariate-conditional level and present flexible neural instantiation.

Acknowledgments

This paper is supported by the DAAD program 'Konrad Zuse Schools of Excellence in Artificial Intelligence', sponsored by the Federal Ministry of Education and Research. Additionally, the authors would like to thank Lars van der Laan, Dennis Frauen, and Alicia Curth for their helpful remarks and comments on the content of this paper. SF also acknowledges funding from the Swiss National Science Foundation (SNSF) via Grant 186932.

References

Ahmed M. Alaa, Zaid Ahmad, and Mark van der Laan. 'Conformal meta-learners for predictive inference of individual treatment effects'. In: Advances in Neural Information Processing Systems . 2024.

Ahmed M. Alaa and Mihaela van der Schaar. 'Bayesian inference of individualized treatment effects using multi-task Gaussian processes'. In: Advances in Neural Information Processing Systems . 2017.

Ahmed M. Alaa and Mihaela van der Schaar. 'Bayesian nonparametric causal inference: Information rates and learning algorithms'. In: IEEE Journal of Selected Topics in Signal Processing 12.5 (2018), pp. 1031-1046.

Joshua D. Angrist, Guido W. Imbens, and Donald B. Rubin. 'Identification of causal effects using instrumental variables'. In: Journal of the American statistical Association 91.434 (1996), pp. 444-455.

Peter M. Aronow, Donald P. Green, and Donald K.K. Lee. 'Sharp bounds on the variance in randomized experiments'. In: The Annals of Statistics 42.3 (2014), pp. 850-871.

Sivaraman Balakrishnan, Edward Kennedy, and Larry Wasserman. 'Conservative inference for counterfactuals'. In: arXiv preprint arXiv:2310.12757 (2023).

Nicolas Banholzer et al. 'Estimating the effects of non-pharmaceutical interventions on the number of new infections with COVID-19 during the first epidemic wave'. In: PLoS one 16.6 (2021), e0252827.

Elias Bareinboim et al. 'On Pearl's hierarchy and the foundations of causal inference'. In: Probabilistic and Causal Inference: The Works of Judea Pearl . Association for Computing Machinery, 2022, pp. 507-556.

Ioana Bica et al. 'From real-world patient data to individualized treatment effects using machine learning: Current and future methods to address underlying challenges'. In: Clinical Pharmacology & Therapeutics 109 (2021), pp. 87-100.

Peter J. Bickel et al. Efficient and adaptive estimation for semiparametric models . Springer New York, 1993.

Matteo Bonvini et al. 'Sensitivity analysis for marginal structural models'. In: arXiv preprint arXiv:2210.04681 (2022).

Kevin G. Buell et al. 'Individualized treatment effects of oxygen targets in mechanically ventilated critically ill adults'. In: JAMA (2024).

Victor Chernozhukov, Iván Fernández-Val, and Blaise Melly. 'Inference on counterfactual distributions'. In: Econometrica 81.6 (2013), pp. 2205-2268.

Victor Chernozhukov, Sokbae Lee, and Adam M. Rosen. 'Intersection bounds: Estimation and inference'. In: Econometrica 81.2 (2013), pp. 667-737.

Victor Chernozhukov et al. 'Double/debiased machine learning for treatment and structural parameters'. In: The Econometrics Journal 21.1 (2018), pp. C1-C68.

Yoichi Chikahara, Makoto Yamada, and Hisashi Kashima. 'Feature selection for discovering distributional treatment effect modifiers'. In: Uncertainty in Artificial Intelligence . 2022.

Yifan Cui and Sukjin Han. 'Policy learning with distributional welfare'. In: arXiv preprint arXiv:2311.15878 (2024).

Alicia Curth, Ahmed M. Alaa, and Mihaela van der Schaar. 'Estimating structural target functions using machine learning and influence functions'. In: arXiv preprint arXiv:2008.06461 (2020).

Alicia Curth and Mihaela van der Schaar. 'In search of insights, not magic bullets: Towards demystification of the model selection dilemma in heterogeneous treatment effect estimation'. In: International Conference on Machine Learning . 2023.

Alicia Curth and Mihaela van der Schaar. 'Nonparametric estimation of heterogeneous treatment effects: From theory to learning algorithms'. In: International Conference on Artificial Intelligence and Statistics . 2021.

Alicia Curth et al. 'Really doing great at estimating CATE? A critical look at ML benchmarking practices in treatment effect estimation'. In: Advances in Neural Information Processing Systems . 2021.

Alicia Curth et al. 'Using machine learning to individualize treatment effect estimation: Challenges and opportunities'. In: Clinical Pharmacology & Therapeutics (2024).

Jacob Dorn and Kevin Guo. 'Sharp sensitivity analysis for inverse propensity weighting via quantile balancing'. In: Journal of the American Statistical Association 118.544 (2023), pp. 2645-2657.

Conor Durkan et al. 'Neural spline flows'. In: Advances in Neural Information Processing Systems . 2019.

Yanqin Fan, Emmanuel Guerre, and Dongming Zhu. 'Partial identification of functionals of the joint distribution of 'potential outcomes''. In: Journal of Econometrics 197.1 (2017), pp. 42-59.

Yanqin Fan and Sang Soo Park. 'Sharp bounds on the distribution of treatment effects and their statistical inference'. In: Econometric Theory 26.3 (2010), pp. 931-951.

Yanqin Fan, Robert Sherman, and Matthew Shum. 'Identifying treatment effects under data combination'. In: Econometrica 82.2 (2014), pp. 811-822.

Yanqin Fan and Sang Soo Park. 'Partial identification of the distribution of treatment effects and its confidence sets'. In: Nonparametric Econometric Methods . Emerald Group Publishing Limited, 2009, pp. 3-70.

Jake Fawkes et al. 'Doubly robust kernel statistics for testing distributional treatment effects'. In: Transactions on Machine Learning Research (2024).

Stefan Feuerriegel et al. 'Causal machine learning for predicting treatment outcomes'. In: Nature Medicine 30.4 (2024), pp. 958-968.

Sergio Firpo. 'Efficient semiparametric estimation of quantile treatment effects'. In: Econometrica 75.1 (2007), pp. 259-276.

Sergio Firpo, Nicole M. Fortin, and Thomas Lemieux. 'Unconditional quantile regressions'. In: Econometrica 77.3 (2009), pp. 953-973.

Sergio Firpo and Geert Ridder. 'Partial identification of the treatment effect distribution and its functionals'. In: Journal of Econometrics 213.1 (2019), pp. 210-234.

Aaron Fisher and Edward H. Kennedy. 'Visually communicating and teaching intuition for influence functions'. In: The American Statistician 75.2 (2021), pp. 162-172.

Dylan J. Foster and Vasilis Syrgkanis. 'Orthogonal statistical learning'. In: The Annals of Statistics 51.3 (2023), pp. 879-908.

Brigham R. Frandsen and Lars J. Lefgren. 'Partial identification of the distribution of treatment effects with an application to the knowledge is power program (KIPP)'. In: Quantitative Economics 12.1 (2021), pp. 143-171.

Dennis Frauen, Valentyn Melnychuk, and Stefan Feuerriegel. 'Sharp bounds for generalized causal sensitivity analysis'. In: Advances in Neural Information Processing Systems . 2023.

Dennis Frauen et al. 'A neural framework for generalized causal sensitivity analysis'. In: International Conference on Learning Representations . 2024.

Maurice Fréchet. 'Sur les tableaux de corrélation dont les marges sont données'. In: Ann. Univ. Lyon, 3ˆ e serie, Sciences, Sect. A 14 (1951), pp. 53-77.

Tilmann Gneiting and Adrian E. Raftery. 'Strictly proper scoring rules, prediction, and estimation'. In: Journal of the American Statistical Association 102.477 (2007), pp. 359-378.

Cornelia Gruber et al. 'Sources of uncertainty in machine learning - a statisticians' view'. In: arXiv preprint arXiv:2305.16703 (2023).

Wenshuo Guo et al. 'Partial identification with noisy covariates: A robust optimization approach'. In: Conference on Causal Learning and Reasoning . 2022.

Daeyoung Ham, Ted Westling, and Charles R. Doss. 'Doubly robust estimation and inference for a log-concave counterfactual density'. In: arXiv preprint arXiv:2403.19917 (2024).

James J. Heckman, Jeffrey Smith, and Nancy Clements. 'Making the most out of programme evaluations and social experiments: Accounting for heterogeneity in programme impacts'. In: The Review of Economic Studies 64.4 (1997), pp. 487-535.

Miguel A Hernán and James M. Robins. 'Instruments for causal inference: An epidemiologist's dream?' In: Epidemiology 17.4 (2006), pp. 360-372.

Jennifer L. Hill. 'Bayesian nonparametric modeling for causal inference'. In: Journal of Computational and Graphical Statistics 20.1 (2011), pp. 217-240.

Keisuke Hirano and Jack R. Porter. 'Impossibility results for nondifferentiable functionals'. In: Econometrica 80.4 (2012), pp. 1769-1790.

Kate Ho and Adam M. Rosen. 'Partial identification in applied research: Benefits and challenges'. In: Advances in Economics and Econometrics: Eleventh World Congress . Econometric Society Monographs. Cambridge University Press, 2017, pp. 307-359.

Wassilij Hoeffding. 'Masstabinvariante Korrelationstheorie'. In: Schriften des Mathematischen Instituts und Instituts für Angewandte Mathematik der Universität Berlin 5 (1940), pp. 181-233.

Eyke Hüllermeier and Willem Waegeman. 'Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods'. In: Machine learning 110.3 (2021), pp. 457-506.

Andrew Jesson et al. 'Identifying causal-effect inference failure with uncertainty-aware models'. In: Advances in Neural Information Processing Systems . 2020.

Andrew Jesson et al. 'Quantifying ignorance in individual-level causal-effect estimates under hidden confounding'. In: International Conference on Machine Learning . 2021.

Andrew Jesson et al. 'Scalable sensitivity and uncertainty analyses for causal-effect estimates of continuous-valued interventions'. In: Advances in Neural Information Processing Systems . 2022.

Wenlong Ji, Lihua Lei, and Asher Spector. 'Model-agnostic covariate-assisted inference on partially identified causal effects'. In: arXiv preprint arXiv:2310.08115 (2023).

Ying Jin, Zhimei Ren, and Emmanuel J. Candès. 'Sensitivity analysis of individual treatment effects: A robust conformal inference approach'. In: Proceedings of the National Academy of Sciences 120.6 (2023), e2214889120.

Ying Jin, Zhimei Ren, and Zhengyuan Zhou. 'Sensitivity analysis under the f -sensitivity models: A distributional robustness perspective'. In: arXiv preprint arXiv:2203.04373 (2022).

Fredrik D. Johansson et al. 'Generalization bounds and representation learning for estimation of potential outcomes and causal effects'. In: Journal of Machine Learning Research 23 (2022), pp. 7489-7538.

Jef Jonkers et al. 'Conformal Monte Carlo meta-learners for predictive inference of individual treatment effects'. In: arXiv preprint arXiv:2402.04906 (2024).

Tetsuya Kaji and Jianfei Cao. 'Assessing heterogeneity of treatment effects'. In: arXiv preprint arXiv:2306.15048 (2023).

Nathan Kallus. 'What's the harm? Sharp bounds on the fraction negatively affected by treatment'. In: Advances in Neural Information Processing Systems . 2022.

Nathan Kallus, Xiaojie Mao, and Angela Zhou. 'Interval estimation of individual-level causal effects under unobserved confounding'. In: International Conference on Artificial Intelligence and Statistics . 2019.

Nathan Kallus and Miruna Oprescu. 'Robust and agnostic learning of conditional distributional treatment effects'. In: International Conference on Artificial Intelligence and Statistics . 2023.

Edward H. Kennedy. 'Semiparametric doubly robust targeted double machine learning: A review'. In: arXiv preprint arXiv:2203.06469 (2022).

Edward H. Kennedy. 'Towards optimal doubly robust estimation of heterogeneous causal effects'. In: Electronic Journal of Statistics 17.2 (2023), pp. 3008-3049.

Edward H. Kennedy, Sivaraman Balakrishnan, and L.A. Wasserman. 'Semiparametric counterfactual density estimation'. In: Biometrika 110.4 (2023), pp. 875-896.

Kwangho Kim, Jisu Kim, and Edward H. Kennedy. 'Causal effects based on distributional distances'. In: arXiv preprint arXiv:1806.02935 (2018).

Danijel Kivaranovic et al. 'Conformal prediction intervals for the individual treatment effect'. In: arXiv preprint arXiv:2006.01474 (2020).

Thomas Kneib, Alexander Silbersdorff, and Benjamin Säfken. 'Rage against the mean - a review of distributional regression approaches'. In: Econometrics and Statistics 26 (2023), pp. 99-123.

Sören R. Künzel et al. 'Metalearners for estimating heterogeneous treatment effects using machine learning'. In: Proceedings of the National Academy of Sciences 116.10 (2019), pp. 4156-4165.

Milan Kuzmanovic et al. 'Causal machine learning for cost-effective allocation of development aid'. In: Conference on Knowledge Discovery and Data Mining . 2024.

Lars van der Laan, Marco Carone, and Alexander R. Luedtke. 'Combining T-learning and DR-learning: A framework for oracle-efficient estimation of causal contrasts'. In: arXiv preprint arXiv:2402.01972 (2024).

Lars van der Laan et al. 'Adaptive debiased machine learning using data-driven model selection techniques'. In: arXiv preprint arXiv:2307.12544 (2023).

Mark J. van der Laan and James M. Robins. Unified methods for censored longitudinal data and causality . Springer New York, 2003.

Mark J. van der Laan and Sherri Rose. Targeted Learning: Causal Inference for Observational and Experimental Data . Springer New York, 2011.

Yann LeCun. 'The MNIST database of handwritten digits'. In: http://yann.lecun.com/exdb/mnist/ (1998).

Hyun-Suk Lee et al. 'Robust recursive partitioning for heterogeneous treatment effects with uncertainty quantification'. In: Advances in Neural Information Processing Systems (2020).

Sungwon Lee. 'Partial identification and inference for conditional distributions of treatment effects'. In: Journal of Applied Econometrics 39.1 (2024), pp. 107-127.

Lihua Lei and Emmanuel J. Candès. 'Conformal inference of counterfactuals and individual treatment effects'. In: Journal of the Royal Statistical Society 83.5 (2021), pp. 911-938.

Alexander W Levis, Edward H. Kennedy, and Luke Keele. 'Nonparametric identification and efficient estimation of causal effects with instrumental variables'. In: arXiv preprint arXiv:2402.09332 (2024).

Kendrick Qijun Li et al. 'Doubly robust proximal causal inference under confounded outcomedependent sampling'. In: arXiv preprint arXiv:2208.01237 (2022).

Jiannan Lu, Peng Ding, and Tirthankar Dasgupta. 'Treatment effects on ordinal outcomes: Causal estimands and sharp bounds'. In: Journal of Educational and Behavioral Statistics 43.5 (2018), pp. 540-567.

Alexander R. Luedtke. Evaluating Optimal Individualized Treatment Rules . University of California, Berkeley, 2016.

Alexander R. Luedtke. 'Simplifying debiased inference via automatic differentiation and probabilistic programming'. In: arXiv preprint arXiv:2405.08675 (2024).

Alexander R. Luedtke and Mark J. van der Laan. 'Statistical inference for the mean outcome under a possibly non-unique optimal treatment strategy'. In: Annals of statistics 44.2 (2016), p. 713.

Alexander R. Luedtke and Mark J. van der Laan. 'Super-learning of an optimal dynamic treatment rule'. In: The international journal of biostatistics 12.1 (2016), pp. 305-332.

Yuchen Ma et al. 'DiffPO: A causal diffusion model for predicting potential outcomes of treatments'. In: Advances in Neural Information Processing Systems . 2024.

G.D. Makarov. 'Estimates for the distribution function of a sum of two random variables when the marginal distributions are fixed'. In: Theory of Probability & its Applications 26.4 (1982), pp. 803-806.

Charles F. Manski. 'Monotone treatment response'. In: Econometrica: Journal of the Econometric Society (1997), pp. 1311-1334.

Charles F. Manski. 'Nonparametric bounds on treatment effects'. In: The American Economic Review 80.2 (1990), pp. 319-323.

Myrl G. Marmarelis et al. 'Ensembled prediction intervals for causal outcomes under hidden confounding'. In: Causal Learning and Reasoning . 2024.

Diego Martinez-Taboada and Edward H. Kennedy. 'Counterfactual density estimation using kernel Stein discrepancies'. In: International Conference on Learning Representations . 2024.

Diego Martinez-Taboada, Aaditya Ramdas, and Edward Kennedy. 'An efficient doublyrobust test for the kernel treatment effect'. In: Advances in Neural Information Processing Systems . 2023.

Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. 'Bounds on representationinduced confounding bias for treatment effect estimation'. In: International Conference on Learning Representations . 2024.

Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. 'Normalizing flows for interventional density estimation'. In: International Conference on Machine Learning . 2023.

Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. 'Partial counterfactual identification of continuous outcomes with a curvature sensitivity model'. In: Advances in Neural Information Processing Systems . 2023.

Pawel Morzywolek, Johan Decruyenaere, and Stijn Vansteelandt. 'On a general class of orthogonal learners for the estimation of heterogeneous treatment effects'. In: arXiv preprint arXiv:2303.12687 (2023).

Krikamol Muandet et al. 'Counterfactual mean embeddings'. In: Journal of Machine Learning Research 22 (2021), pp. 1-71.

Xinkun Nie and Stefan Wager. 'Quasi-oracle estimation of heterogeneous treatment effects'. In: Biometrika 108 (2021), pp. 299-319.

Miruna Oprescu et al. 'B-Learner: Quasi-oracle bounds on heterogeneous causal effects under hidden confounding'. In: International Conference on Machine Learning . 2023.

Junhyung Park et al. 'Conditional distributional treatment effect with kernel conditional mean embeddings and U-statistic regression'. In: International Conference on Machine Learning . 2021.

Judea Pearl. Causality . Cambridge University Press, 2009.

Thong Pham et al. 'Scalable counterfactual distribution estimation in multivariate causal models'. In: Causal Learning and Reasoning . 2024.

Boris T. Polyak and Anatoli B. Juditsky. 'Acceleration of stochastic approximation by averaging'. In: SIAM Journal on Control and Optimization 30.4 (1992), pp. 838-855.

Ashesh Rambachan, Amanda Coston, and Edward Kennedy. 'Counterfactual risk assessments under unmeasured confounding'. In: arXiv preprint arXiv:2212.09844 (2022).

Danilo Rezende and Shakir Mohamed. 'Variational inference with normalizing flows'. In: International Conference on Machine Learning . 2015.

James M. Robins. 'Optimal structural nested models for optimal sequential decisions'. In: Second Seattle Symposium in Biostatistics: Analysis of correlated data . Springer New York. 2004, pp. 189-326.

James M. Robins and Andrea Rotnitzky. 'Semiparametric efficiency in multivariate regression models with missing data'. In: Journal of the American Statistical Association 90.429 (1995), pp. 122-129.

Jonas Rothfuss et al. 'Noise regularization for conditional density estimation'. In: arXiv preprint arXiv:1907.08982 (2019).

Donald B. Rubin. 'Estimating causal effects of treatments in randomized and nonrandomized studies'. In: Journal of Educational Psychology 66.5 (1974), p. 688.

Gabriel Ruiz and Oscar Hernan Madrid Padilla. 'Non-asymptotic confidence bands on the probability an individual benefits from treatment (PIBT)'. In: arXiv preprint arXiv:2205.09094 (2022).

Maresa Schröder et al. 'Conformal prediction for causal effects of continuous treatments'. In: arXiv preprint arXiv:2407.03094 (2024).

Vira Semenova. 'Adaptive estimation of intersection bounds: A classification approach'. In: arXiv preprint arXiv:2303.00982 (2023).

Vira Semenova. 'Debiased machine learning of set-identified linear models'. In: Journal of Econometrics 235.2 (2023), pp. 1725-1746.

Uri Shalit, Fredrik D. Johansson, and David Sontag. 'Estimating individual treatment effect: Generalization bounds and algorithms'. In: International Conference on Machine Learning . 2017.

Phillip Si, Allan Bishop, and Volodymyr Kuleshov. 'Autoregressive quantile flows for predictive uncertainty estimation'. In: International Conference on Learning Representations . 2021.

Thomas Strömberg. 'A study of the operation of infimal convolution'. PhD thesis. Luleå tekniska universitet, 1994.

Hongjun Su and Hong Zhang. 'Distances and kernels based on cumulative distribution functions'. In: Emerging Trends in Image Processing, Computer Vision and Pattern Recognition . Elsevier, 2015, pp. 551-559.

Erik Sverdrup and Yifan Cui. 'Proximal causal learning of conditional average treatment effects'. In: International Conference on Machine Learning . 2023.

Zhiqiang Tan. 'A distributional approach for causal inference using propensity scores'. In: Journal of the American Statistical Association 101.476 (2006), pp. 1619-1637.

Eric J. Tchetgen Tchetgen et al. 'An introduction to proximal causal learning'. In: Statistical Science 39.3 (2024), pp. 375-390.

Brian L. Trippe and Richard E. Turner. 'Conditional density estimation with Bayesian normalising flows'. In: arXiv preprint arXiv:1802.04908 (2018).

Anastasios A. Tsiatis. Semiparametric theory and missing data . Springer New York, 2006.

Aad W. van der Vaart. Asymptotic statistics . Vol. 3. Cambridge university press, 2000.

Toon Vanderschueren, Jeroen Berrevoets, and Wouter Verbeke. 'NOFLITE: Learning to predict individual treatment effect distributions'. In: Transactions on Machine Learning Research (2023).

Stijn Vansteelandt and Paweł Morzywołek. 'Orthogonal prediction of counterfactual outcomes'. In: arXiv preprint arXiv:2311.09423 (2023).

Robert C. Williamson and Tom Downs. 'Probabilistic arithmetic. I. Numerical methods for calculating convolutions and dependency bounds'. In: International Journal of Approximate Reasoning 4.2 (1990), pp. 89-158.

Andrew Ying. 'A geometric perspective on double robustness by semiparametric theory and information geometry'. In: arXiv preprint arXiv:2404.13960 (2024).

Yao Zhang, Alexis Bellot, and Mihaela van der Schaar. 'Learning overlapping representations for the estimation of individualized treatment effects'. In: International Conference on Artificial Intelligence and Statistics . 2020.

Zhehao Zhang and Thomas S. Richardson. 'Bounds on the distribution of a sum of two random variables: Revisiting a problem of Kolmogorov with application to individual treatment effects'. In: arXiv preprint arXiv:2405.08806 (2024).

A Extended Related Work

Partial identification and sensitivity models in the potential outcomes framework. Interventional quantities (such as CAPOs and distributions of potential outcomes) and some counterfactual quantities (e. g., CATE) are point identifiable in potential outcomes framework (see Fig. 6). However, relaxation of the unconfoundedness assumption renders those quantities partially identifiable or even nonidentifiable identifiable [89]. In this case, additional sensitivity models can be employed. Examples are the marginal sensitivity model [11, 23, 37, 38, 52, 53, 56, 119] and outcome sensitivity model [11, 104]. Other approaches suggest using instrumental variables [4, 45] or noisy proxy variables [42, 120]. Counterfactual quantities , on the other hand, are inherently non-identifiable (e. g., joint distribution of potential outcomes [26], expected counterfactual outcomes of (un)treated [95], and distribution of treatment effect [26]). There is no general approach for deriving sharp bounds for partially identifiable causal quantities, therefore the above-mentioned works are not relevant in our setting.

Counterfactual

Observational

Interventional

Ladder of causation

Potential

outcomes

framework,

RCTs

conditional distribution

of outcome

conditional average

potential outcome

conditional distribution of

treatment effect

conditional average

treatment effect

Example of causal diagrams

conditional average

outcome

conditional distribution

of potential outcome

Additivity of

expectation

Distributional treatment effects and the distribution of the treatment effect. There are two seemingly similar notions in the existing literature: (a) distributional treatment effects [16, 29, 62, 92, 100] and (b) the distribution of the treatment effect [6, 26, 28, 33, 88]. However, (a) and (b) have two important differences:

(i) Interpretation. Distributional treatment effects represent the differences between different distributional aspects of the potential outcomes (e.g., Wasserstein distances, KL-divergence, or the quantile differences) [62]. Hence, they can answer questions like 'How are 10% of the worst-possible outcomes with treatment different from the worst 10% of the outcomes without treatment?'. Here, the two groups (treated and untreated) of the worst 10% contain, in general, different individuals. This is problematic in many applications like clinical decision support and drug approval. Here, the aim is not to compare individuals from treated vs. untreated groups (where the groups may differ due to various, unobserved reasons). Instead, the aim is to accurately quantify the treatment response for each individual and allow for quantification of the personalized uncertainty of the treatment effect. The latter is captured in the distribution of the treatment effect, which allows us to answer the question about the CDF/quantiles of the treatment effect. For example, we would aim to answer a question like 'What are the worst 10% of values of the treatment effect?'. Here, we focus on the treatment effect for every single individual. The latter is more complex because we reason about the difference of two potential outcomes simultaneously. Hence, in natural situations when the potential outcomes are non-deterministic, both (a) the distributional treatment effect and (b) the distribution of the treatment effect will lend to very different interpretations, especially in medical practice. In particular, the distribution of the treatment effect (which we study in our paper) is important in medicine, where it allows quantifying the amount of harm/benefit after the treatment [60]. This may warn doctors about

situations where the averaged treatment effects are positive but where the probability of the negative treatment effect is still large.

(ii) Inference. The efficient inference of the distributional treatment effects only requires the estimation of the relevant distributional aspects of the conditional outcome distributions (e. g., quantiles) and the propensity score [62]. However, in our setting of the bounds on the CDF/quantiles of the treatment effect, we also need to perform sup/inf convolutions of the CDF/quantiles of the conditional outcomes distributions. Hence, while the definitions of (a) the distributional treatment effects and (b) the distribution of the treatment effect appear related, their estimation is very different.

Alternatives to Makarov bounds. Potential alternatives to Makarov bounds vary depending on the type of aleatoric uncertainty. For example, explicit or implicit sharp bounds were proposed for:

The variance of the treatment effect, Var( Y [1] -Y [0]) [5]. Here, the sharp bounds are explicitly given by Fréchet-Hoeffding bounds [39, 49] and are, in general, different from Makarov bounds.

The interval probabilities, P ( δ 1 ≤ Y [1] -Y [0] ≤ δ 2 ) [33]. The sharp bounds on the interval probabilities are only defined implicitly and are also, in general, different from Makarov bounds (see Appendix B).

We are not aware of other bounds for measuring aleatoric uncertainty (e. g., kurtosis, skewness, or entropy). Importantly, the above-mentioned bounds on different measures of uncertainty are orthogonal to our work, and we focus on the bounds on the CDF/quantiles of the treatment effect.

Efficient estimators and orthogonal learners. Efficient estimation of causal quantities (target estimands) was studied in the scope of (i) semi-parametric efficient estimation theory and, more general, (ii) orthogonal learning theory . Both theories (i) and (ii) rely on the concept of influence functions (pathwise derivatives) [63, 72, 122], which allow performing a one-step bias correction for (i) a plug-in estimator of the target estimand or (ii) the population risk containing target estimand, respectively. (i) Semi-parametric efficient estimation theory [10, 73, 74, 107, 122] provides asymptotically efficient estimators for finite-dimensional target estimands (e. g., average treatment effect (ATE) and average potential outcomes (APOs)). On the other hand, (ii) orthogonal learning theory (or debiased ML) [15, 35, 96, 125] was developed for infinitely-dimensional (functional) estimands, like, CATE and CAPO. Orthogonal learning theory (or orthogonal learners) estimates target estimands by minimizing (Neyman-)orthogonal losses that are first-order insensitive to the misspecification of the nuisance functions. Specific examples of orthogonal learners for identifiable quantities include CAPO learners [125] and CATE learners [18, 64, 71, 96, 98].

Estimation of partially identifiable quantities. Orthogonal learners were also proposed for bounds on partially identifiable causal quantities. Examples include different interventional quantities such, e. g., in marginal sensitivity model [99], in instrumental variables setting [79], in noisy proxy variables setting [80, 118]. Examples from counterfactual quantities are, e. g., the variance of treatment effect and joint CDF of potential outcomes (Fréchet-Hoeffding bounds) [6]. More generally, estimation and learning for partially identifiable quantities was studied in econometrics as estimation of intersection bounds [14, 112, 113]. In this paper, we construct a novel orthogonal learner targeting at Makarov bounds on the CDF/quantiles of the CDTE.

Total uncertainty quantification. Total uncertainty in potential outcomes framework can be generally quantified with two approaches: (i) Bayesian methods (e. g., Gaussian processes) and (ii) conformal prediction framework. (i) Bayesian methods [2, 3, 46] allow to infer posterior predictive distributions or credible intervals and for both potential outcomes and treatment effect, but under additional identifiability assumptions (e. g., an assumption of additive latent outcome noise, which renders treatment effect distribution identifiable). (ii) Conformal prediction aims at providing a valid predictive interval and was applied to quantify total uncertainty of predicting potential outcomes [76, 78, 90, 111] and treatment effect [1, 55, 58, 67, 78]. The latter works either make a similar additive latent outcome noise assumption or 'hide' non-identifiable treatment effect into the predictive interval (however, oracle predictive interval could never be reached in this case). We argue that total uncertainty needs to be split into epistemic and aleatoric explicitly to provide insights on the origin of uncertainty [50], especially in our setting where aleatoric uncertainty of the treatment effect itself is partially identifiable (see Fig. 2).

B Makarov Bounds

B.1 Bounds Construction

In the following, we provide an additional intuition on how Makarov bounds are inferred from conditional distributions of the potential outcomes, P ( Y [ a ] | x ) . For example, Makarov bounds for the CDF of the CDTE are a composition of the sup/inf convolutions, applied to the conditional CDFs of the potential outcomes, and the rectifier functions (see Fig. 7).

Makarov bounds can be inferred (i) analytically or (ii) numerically. (i) Analytic formulas were proposed for very simple distributions (e. g., a normal distribution [26]). At the same time, the (ii) numerical approach is more flexible. For example, we can discretize the Y -space or [0 , 1] -interval and perform maximization/minimization on that grid. Notably, in our experiments, we infer the ground-truth bounds with the approach (i), when the ground-truth conditional potential outcomes distributions are normal; and (ii) otherwise.

B.2 Pointwise and Uniformly Sharp Bounds

The sharpness of Makarov bounds proposed in [26] has one important characteristic. Specifically, although the Makarov bounds on the CDFs, F ( δ | X ) , are CDFs themselves, they are not valid solutions to the partial identification task. This can be easily checked, i. e., the expectation wrt. to the Makarov bounds does not coincide with the point-identifiable CATE, τ ( x ) = E (∆ | x ) :

Therefore, there is an important distinction between so-called pointwise sharpness and a uniform sharpness [33].

In the case of uniform sharpness, a sharp bound coincides with the solution to the partial identification task. This implies that if a bound is uniformly sharp, then the joint bound on the set of quantities,

evaluated in two (or more) points of ∆ or [0 , 1] , is also sharp. Many known bounds (e. g., FréchetHoeffding bounds [39, 49], and marginal sensitivity model bounds [23, 37, 38, 119]) are uniformly sharp. Yet, Makarov bounds are only pointwise sharp.

Recent works developed uniformly sharp bounds on the CDF of the CDTE [33]. However, their inference requires a special computational routine for every value of δ/α wrt. the CDF/quantiles of the CDTE. Their usage is further complicated by the fact that the CDFs of the uniformly sharp bounds correspond to mixed-type discrete/continuous random variables. We show an example of the uniformly sharp bounds on the CDF for δ = 0 in Fig. 8. The uniformly sharp bounds may be useful for more complex aleatoric uncertainty quantities (e. g., interval quantities like P ( δ 1 ≤ Y [1] -Y [0] ≤ δ 2 | x ) ) or simultaneous bounds on the variance and the CDF of the CDTE. Nevertheless, in many practical applications, pointwise sharp bounds (Makarov bounds) are enough and we focus on those in our paper.

-

4

-

2

0

2

4

6

8

10

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

CDF

(a)

F

(

δ

|

x

7

)

F

(

δ

|

x

7

)

τ

(

x

7

) =

E

(∆

|

x

7

)

F

0

(

δ

|

x

7

)

-

4

-

2

0

2

4

6

8

10

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

CDF

(b)

F

(

δ

|

x

7

)

F

(

δ

|

x

7

)

τ

(

x

7

) =

E

(∆

|

x

7

)

F

0

(

δ

|

x

7

)

δ

δ

B.3 Makarov bounds for mixed-type outcomes

In the following, we provide a connection of the Makarov bounds for the continuous outcome Y (the main target of our paper), with the bounds on the fraction of the negatively affected (FNA) from [60]. The FNA is defined for the binary outcome, Y ∈ { 0 , 1 } , as

As suggested by [60], sharp bounds on the FNA are given by

To connect the Makarov bounds for continuous outcome and the bounds on the FNA, we formulate the following remark.

Remark 1 (Makarov bounds for mixed-type outcome [129]) . Sharp bounds on the CDF of the treatment effect for the mixed-type outcome Y are F ( δ | x ) ≤ F ( δ | x ) ≤ F ( δ | x ) , which are

Notably, for (absolutely) continuous outcome P 0 = 0 , Eq. (18) matches Eq. (3) of Sec. 3.

Remark 1 immediately yields the bounds on the FNA, proposed by [60]:

C IPTW-learner

In the following, we develop an improved single-stage learner, namely, an inverse propensity of treatment weighted (IPTW) learner. First, we revisit the plug-in learner by defining its estimation objective. Then, we introduce the IPTW-learner, which addresses one of the shortcomings of the plug-in learner. At the end, we mention a surprising property of the IPTW-learner, namely, the orthogonality wrt. to target risk aiming at the potential outcome distributions.

Empirical risk of the plug-in learner. We assume that the plug-in learner (Sec. 4.1) uses the estimators of the conditional outcome CDFs, ̂ F a , that aim at minimizing the following empirical risk:

✶ where l ( · , · ) > 0 is a probabilistic loss (e. g., negative log-likelihood, check score, or CRPS with an empirical CDF). Here, the plug-in learner has two possible variants, namely, S- and T-learner, depending on whether the conditional outcome distribution is learned by a single model or two models [20, 69].

IPTW-learner. The IPTW-learner addresses the selection bias of the plug-in learner. For this, it additionally employs an estimated propensity score, ˆ π . The estimated propensity score is used to re-weight the original probabilistic loss, l ( · , · ) , in the following way:

where ˆ π a ( x ) = a ˆ π ( x ) + (1 -a ) (1 -ˆ π ( x )) . The IPTW-learner up-weights the loss of ̂ F 0 and ̂ F 1 in the treated and untreated populations, respectively.

Orthogonality of the IPTW-learner. Interestingly, the IPTW-learner from Eq. (29) can be seen as an orthogonal learner targeting at the following risk:

where g ( · , x ) ∈ G is a working model defined as in Sec. 4.2. This target risk aims to find the best approximation of the conditional potential outcome distribution, P ( Y [ a ] | x ) , with the working model, g ∈ G .

The orthogonality of the IPTW-learner wrt. the target risk aiming at the potential outcome distributions was formally proved in [125]. Therein, the authors notice that the target estimand (e. g., the CDF of one of the potential outcomes) coincides with one of the nuisance functions (i. e., F a ). Informally, the orthogonality follows from the fact that the working model, g , simultaneously fits the target estimand and the nuisance function.

The orthogonality of the IPTW-learner can also be seen by (1) performing a one-step bias-correction of Eq. (28) and (2) setting the same estimator for the working model and the nuisance function, i. e., g = ̂ F a . For example, if the probabilistic loss is the CRPS with the empirical CDF, l ( Y, g ( · , X )) = ∫ Y ( ✶ { Y ≤ y } -g ( y, X )) 2 d y , the (1) one-step bias-corrected loss is given by [6, 43]

where ˆ π a ( x ) = a ˆ π ( x ) + (1 -a ) (1 -ˆ π ( x )) . Then, after step (2), g = ̂ F a , we immediately yield the IPTW-learner from Eq. (29). Similarly, for the negative log-likelihood loss (NLL) l ( Y, g ( · , X )) = -log g ( Y, X ) , the (1) one-step bias-correction is given by [65, 94]

where ˆ π a ( x ) = a ˆ π ( x ) + (1 -a ) (1 -ˆ π ( x )) , and ̂ P is an estimator of the conditional density of the outcome. Again, after (2) setting g = ̂ P , it is easy to see that the minimization of one-step bias corrected loss in Eq. (32) is equivalent to the minimization of the IPTW-learner's objective in Eq. (29) (where ̂ P is used in place of ̂ F a ). This is possible due to two facts: (1) both entropy terms in Eq. (32), -∫ Y log g ( y, X ) g ( y, X ) d y , only require the minimization wrt. to the working model g under the logarithm; and (2) these entropies are minimal as the cross-entropy for any distribution is minimal when evaluated with itself.

D Proofs

In the following, we provide the main theoretical results of our paper. We use the following additional notation: δ {·} is a Dirac delta function, a ≲ b means there exists C ≥ 0 such that a ≤ C · b , and X n = o P ( r n ) means X n /r n p - → 0 . Also, in the following theorems, we use red color to show the nuisance functions of P that are influencing the target estimand.

D.1 Efficient influence functions

We start with deriving the efficient influence functions for the average Makarov bounds and, afterwards, for the target risks. For that, we make two mild assumptions: (1) one the conditional outcome distributions and (2) another on the set of δ where linear rectifiers are differentiable. These assumptions allow us to (1) handle sup-/inf-convolutions as max-/min-convolutions with a finite number of argmax/argmin values and to (2) get a derivative of the linear rectifiers.

Assumption 1 (Finite argument sets) . We assume that the outcome space Y is compact. Also, we assume that conditional outcome CDFs, F a ( y | x ) , are continuously differentiable and consist of a finite number of strictly concave / convex regions.

Assumption 1 implies that sup-/inf-convolutions are achieved by some finite set of values in Y . Furthermore, Assumption 1 is a special case of the margin assumption (Assumption 3.2) from [112]. Yet, we find our version to be more interpretable and many regular distributions satisfy it, e. g., an exponential family, finite mixtures of normal distributions, etc.

Assumption 2 (Differentiability of linear rectifiers) . Values of δ ∈ ∆ are considered to satisfy differentiability of linear rectifier for some x ∈ X if ( F 1 ∗ F 0 ) Y ( δ | x ) = 0 .

̸

It is easy to see that, if Assumption 1 holds, the new Assumption 2 will hold for almost all δ ∈ ∆ and x ∈ X , namely, P { ( F 1 ∗ F 0 ) Y ( δ | X ) = 0 } = 0 or, equivalently, P { | F 1 ∗ F 0 ) Y ( δ | X ) | ≤ ξ } ≤ ξ for some ξ > 0 . Also, no additional requirements are needed for the values of α ∈ [0 , 1] . Notably, the Assumption 2 is analogous to the margin assumptions, e. g., Assumption 3.2 of [84] or Eq. (3) of [112]. Therefore, the formulation of the following theorem is not too restrictive.

Theorem 1 (Efficient influence function for Makarov bounds) . Let P denotes P ( Z ) = P ( X,A,Y ) and let y ∗ Y ( · | x ) and u ∗ [ α, 1] ( · | x ) be argmax/argmin sets of the convolutions ( F 1 ∗ F 0 ) Y ( · | x ) and ( F -1 1 ∗ F -1 0 ) [ α, 1] ( · -0 | x ) , respectively. Then, under the mild assumption of the finite argument sets (Assumption 1), average Makarov bounds are pathwise differentiable for values of δ ∈ ∆ that satisfy the differentiability of linear rectifiers (Assumption 2) and for all values of α ∈ (0 , 1) . Further, the corresponding efficient influence functions, ϕ , are as follows:

where y ∗ is some value from y ∗ Y ( δ | X ) ; u ∗ is some value from u ∗ [ α, 1] ( α | X ) ; and C ( δ, Z ; η ) and C -1 ( δ, Z ; η ) can be then obtained by swapping the symbols {∗ , >, y ∗ Y , u ∗ [ α, 1] , -0 , +0 } to {∗ , <, y ∗ Y , u ∗ [0 ,α ] , -1 , +1 } .

Proof. We start the proof by employing the properties of the efficient influence functions, namely, product rule and chain rule; and some existing efficient influence functions (e. g., for conditional expectation [63]).

The efficient influence function of the lower averaged Makarov bound is as follows:

Furthermore, the inner term is

Then, the efficient influence function of the sup-convolution can be derived under the finite argument sets assumption and using the envelope theorem (we refer to Appendix C.2.6 and C.2.7 of [83] for further details):

where y ∗ is some value from y ∗ Y ( δ | x ) . The latter can be expanded by using product and chain rules and the efficient influence function for the conditional expectation. Specifically, we do the following:

The terms (2) and (4) yield well-known efficient influence function for CDFs, i. e.,

Let us now consider the remaining terms (1) and (3) :

where the last equality holds due to the properties of the argmax, y ∗ ∈ y ∗ Y ( δ | x ) . Namely, under the necessary condition for a local maximum, we have d d y ( F 1 ( y | x ) -F 0 ( y -δ | x )) = P ( Y = y | x, A = 1) -P ( Y = y -δ | x, A = 0) = 0 . In the context of the efficient influence functions, this

means that the Makarov bounds are first-order insensitive to the misspecification of argmax/argmin. Interestingly, a similar result was demonstrated for the efficient influence functions of the policy values of the optimal policies [82, 85].

Finally, we obtain an efficient influence function for the lower Makarov bound by expanding the part ( ∗ ) of Eq. (40):

where C ( δ, Z ; η ) is defined in Eq. (35).

The Makarov bounds on the quantiles are derived analogously. However, several last steps are different. The differences start from Eq. (44):

where u ∗ is some value from u [ α, 1] ( α | x ) .

Now, again, the Makarov bounds for quantiles are first-order insensitive wrt. argmax/argmin values misspecification. Thus, we can employ the efficient influence function for the quantiles [31, 32]:

The latter then yields the final part ( ∗ ) of Eq. (40) for the efficient influence function:

where C -1 ( α, Z ; η ) is defined in Eq. (36).

Corollary 1 (Efficient influence functions of the target risks) . Let the Assumption 1 of the Theorem 1 hold. Then, the efficient influence functions, ϕ , of the target risks in Eq. (6) and Eq. (7) are as follows:

where C ( δ, Z ; η ) and C -1 ( α, Z ; η ) are defined in Eq. (35) and (36) , respectively.

Proof. We start by using the properties of the efficient influence function, namely, chain and product rules. Then, the efficient influence function for the CRPS risk of the lower bound is as follows

We then note, that the inner term of the ( ∗ ) can be expanded as:

Finally, the term ( ∗ ) of Eq. (65) equals to

The derivations for the upper bound and for the W 2 2 target risk is fully analogous.

Corollary 2 (One-step bias-corrected estimator of the target risks) . Let the Assumption 1 of the Theorem 1 hold. Then, the γ -scaled one-step bias-corrected estimator of the target risks Eq. (6) and Eq. (7) is given by the following:

where C ( δ, Z ; ˆ η ) and C -1 ( α, Z ; ˆ η ) are given by Eq. (35) and (36) , respectively; and γ ∈ (0 , 1] is a scaling hyperparameter.

Proof. The γ -scaled one-step bias-corrected estimator of the CRPS target risk proceeds as follows:

The minimization of the latter wrt. g is then equivalent to the minimization of

The proof for the γ -scaled one-step bias-corrected estimator of the W 2 2 target risk is analogous.

D.2 Neyman-orthogonality and quasi-oracle efficiency

Now, we proceed with the second main theoretical result. Here, we use additional notation.

Definition 1 (Neyman-orthogonality [35, 96]) . A risk L is called Neyman-orthogonal if its pathwise cross-derivative equals to zero, namely,

where D f F ( f )[ h ] = d d t F ( f + th ) | t =0 and D k f F ( f )[ h 1 , . . . , h k ] = ∂ k ∂t 1 ...∂t k F ( f + t 1 h 1 + · · · + t k h k ) | t 1 = ··· = t k =0 are pathwise derivatives [35], g ∗ = arg min g ∈G L ( g, η ) , and η is the ground-truth nuisance function.

Informally, this definition means that the risk is first-order insensitive wrt. to the misspecification of the nuisance functions. Notably, the pathwise derivative in the direction of the Dirac delta distribution coincides with the efficient influence function [34, 63], i. e., D P F ( P )[ δ { Z - ·} -P ( Z = · )] = ϕ ( F ( P ); P ) , where P ( Z = · ) is the PDF of the P ( Z ) .

Theorem 2 (Neyman-orthogonality of AU-learner) . Under the assumptions of the Theorem 1, the following holds for AU-learner from Algorithm 1 with the scaling hyperparameter γ = 1 :

Neyman-orthogonality. Population versions of the empirical risks in Eq. (71) and Eq. (71) are first-order insensitive wrt. to the misspecification of the nuisance functions, i. e.,

where g ∗ = arg min g ∈G L AU , CRPS ( g, η ) and g -1 ∗ = arg min g -1 : g ∈G L AU ,W 2 2 ( g -1 , η ) .

Quasi-oracle efficiency. The bias from the misspecification of the nuisance functions is of second order. Specifically, the following two inequalities hold:

where ̂ g = arg min g ∈G L AU , CRPS ( g, ˆ η ) ; ̂ g -1 = arg min g -1 : g ∈G L AU ,W 2 2 ( g -1 , ˆ η ) ; ∥ f ( Z ) ∥ L 2 = √ E ( f ( Z )) 2 ; ∥ f ( δ, Z ) ∥ k, CRPS = ( E ( ∫ ∆ | f ( δ, Z ) | k d δ )) 1 /k ; ∥ f ( α, Z ) ∥ k,W 2 2 =

( E ( ∫ 1 0 | f ( α, Z ) | k d α )) (1 /k ) ; I ( X ; η ) = ✶ { ( F 1 ∗ F 0 ) Y ( δ | X ) > 0 } ; y ∗ is some value from y ∗ Y ( δ | X ) ; ˆ y ∗ is some value from ˆ y ∗ Y ( δ | X ) ; u ∗ is some value from u ∗ [ α, 1] ( α | X ) ; and ˆ u ∗ is some value from ˆ u ∗ [ α, 1] ( α | X ) . The inequalities corresponding to the upper Makarov bound, can be obtained by swapping the symbols {∗ , >, y ∗ Y , u ∗ [ α, 1] , -0 , +0 } to {∗ , <, y ∗ Y , u ∗ [0 ,α ] , -1 , +1 } . Furthermore, if the nuisance functions are estimated sufficiently fast, i. e., ∥ y ∗ -ˆ y ∗ ∥ 4 , CRPS = o P ( n -1 / 4 ) ; ∥ ∥ ∥ F 1 ◦ y ∗ -ˆ F 1 ◦ ˆ y ∗ ∥ ∥ ∥ 4 , CRPS = o P ( n -1 / 4 ) ; ∥ ∥ ∥ F 0 ◦ ( y ∗ -· ) -ˆ F 0 ◦ (ˆ y ∗ -· ) ∥ ∥ ∥ 4 , CRPS = o P ( n -1 / 4 ) ; ∥ ˆ π -π ∥ L 2 ∥ ∥ ∥ F 1 ◦ ˆ y ∗ -ˆ F 1 ◦ ˆ y ∗ ∥ ∥ ∥ 2 , CRPS = o P ( n -1 / 2 ) ; and ∥ ˆ π -π ∥ L 2 ∥ ∥ ∥ F 0 ◦ (ˆ y ∗ -· ) -ˆ F 0 ◦ (ˆ y ∗ -· ) ∥ ∥ ∥ 2 , CRPS = o P ( n -1 / 2 ) , then the AU- learner (CRPS) achieves the quasi-oracle property (analogous result holds for AU-learner ( W 2 2 )). This means that the estimation error of the second stage with the estimated nuisance functions behaves in the same way as if the ground-truth nuisance functions were used.

Proof. 1. Neyman-orthogonality. The Neyman-orthogonality follows by the construction of the AU-learner as a one-step bias-corrected estimator. Specifically, it is easy to verify that the pathwise cross-derivative from Eq. (78) is equal to zero.

Let us consider the CRPS risk. First, we find a pathwise derivative wrt. working model g :

Then, we find derivatives wrt. to different nuisance functions (e.g., the propensity score):

) ) ]

y

∗

-

δ

|

X

(

g

(

δ, X

)

-

g

∗

(

δ, X

)

)

d

δ

(90)

where I ( X ; η ) = ✶ { ( F 1 ∗ F 0 ) Y ( δ | X ) > 0 } or ✶ { ( F 1 ∗ F 0 ) Y ( δ | X ) < 0 } ; and y ∗ is some value from y ∗ Y ( δ | X ) .

]

The derivatives wrt. the conditional CDF F 1 is

where I ( X ; η ) = ✶ { ( F 1 ∗ F 0 ) Y ( δ | X ) > 0 } or ✶ { ( F 1 ∗ F 0 ) Y ( δ | X ) < 0 } ; ˜ y ∗ ( t ) is some value from ˜ y ∗ Y ( δ | X ) ; ˜ y ∗ Y ( · | X ) are the argmax/argmin sets of the convolutions ( F 1 + t ( ˆ F 1 -F 1 ) ∗ F 0 ) Y ( · | X ) ; and ( ∗ ) = 0 follows from the the same considerations as in Eq. (51). Analogously, the pathwise derivative wrt. F 0 can be shown to be equal to zero. We refer to the appendices of [96] for more details.

The Neyman-orthogonality of the W 2 2 population risk can be proved in a similar fashion. First, the pathwise derivative wrt. g -1 has a similar form to Eq. (85), namely

Furthermore, similarly to the CRPS target risk, the cross-derivative wrt. to the propensity score is

where u ∗ is some value from u ∗ [ α, 1] ( α | X ) . The cross-derivative for the upper bound follows similarly.

Finally, to show that a cross-derivative wrt. F -1 1 , we make use of the following property of the derivative of the quantiles (inverse function rule):

where P ( ˜ Y = · ; t ) is the density function for a distribution with quantiles F -1 ( α ) + t ( ˆ F -1 ( α ) -F -1 ( α )) . Then, the pathwise derivative is

Therefore, the cross-derivative is

where ˜ u ∗ ( t ) is some value from ˜ u ∗ [ α, 1] ( α | X ) ; and ˜ u ∗ [ α, 1] ( α | X ) is the argmin set of the convolution ( F -1 1 + t ( ˆ F -1 1 -F -1 1 ) ∗ F -1 0 ) [ α, 1] ( · -0 | X ) . The cross-derivatives for the upper bound wrt. F -1 1 and for both upper and lower bounds wrt. F -1 0 follow similarly.

2. Quasi-oracle efficiency. The result is a direct application of Theorem 1 in [35]: It is easy to see that the Assumptions 1-4 from [35] hold for the population versions of the empirical risks of our AU-learner (i.e., CRPS and W 2 2 ). In the following, we provide the derivation of the quasi-oracle efficiency property for the population version of CRPS loss (the derivation is similar to one in [96, 125]).

We first apply a functional Taylor expansion of L AU , CRPS ( ̂ g, ˆ η ) at the g ∗ :

Therefore, the following holds:

The latter term then also allows for the distributional Taylor expansion around η :

where the first term is known as a second-order remainder term, R 2 ( η, ˆ η ) , and the second term equals to -D g L AU , CRPS ( g ∗ , η )[ ̂ g -g ∗ ] . R 2 ( η, ˆ η ) can be further expressed as

where I ( X ; η ) = ✶ { ( F 1 ∗ F 0 ) Y ( δ | X ) > 0 } or ✶ { ( F 1 ∗ F 0 ) Y ( δ | X ) < 0 } ; y ∗ is some value from y ∗ Y ( δ | X ) ; ˆ y ∗ is some value from ˆ y ∗ Y ( δ | X ) ; and ˜ y ∗ is a value between y ∗ and ˆ y ∗ .

Considering there is such an ϵ > 0 , for which ϵ ≤ ˆ π ( x ) ≤ (1 -ϵ ) , the second-order remainder term can be upper bounded with the Cauchy-Schwarz inequality:

By combining the previous expressions, the following holds:

Furthermore, using the AM-GM inequality for the last three terms, for any constants δ 1 > 0 , δ 2 > 0 , δ 3 > 0 , δ 4 > 0 , δ 1 + δ 2 + δ 3 + δ 4 < 1 , we obtain:

We note that, given Assumptions 1 and 2, the term ∥ I ( · , ˆ η ) -I ( · , η ) ∥ 2 2 , CRPS can be upper bounded as follows

̸

where ξ > 0 and ( ∗ ) holds due to Markov's inequality.

On the other hand, since the term D g L AU , CRPS ( g ∗ , η )[ ̂ g -g ∗ ] is always positive (by virtue of the optimization), we finally yield the desired quasi-oracle efficiency:

The quasi-oracle efficiency of the W 2 2 loss follows similarly.

E Scaling in pseudo-CDFs and pseudo-quantiles

Fig. 9 provides a visual comparison of different pseudo-CDFs for our AU-learner , with and without scaling γ . Therein, we see that the scaling hyperparameter helps to enforce the constraints on the pseudo-CDFs (i. e., [0 , 1] -boundedness and monotonicity).

δ

-

1

.

5

-

1

.

0

-

0

.

5

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

Pseudo-CDF

B: upper; CA-learner

F

PI

(

δ

|

x

i

; ˆ

η

)

δ

B: upper; AU-learner (w/o scaling)

F

DR

(

δ

|

z

i

; ˆ

η, γ

= 1)

δ

B: upper; AU-learner (w/ scaling)

F

DR

(

δ

|

z

i

; ˆ

η, γ

= 0

.

25)

-

2

0

2

4

-

1

.

5

-

1

.

0

-

0

.

5

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

Pseudo-CDF

B: lower; CA-learner

F

PI

(

δ

|

x

i

; ˆ

η

)

-

2

0

2

4

B: lower; AU-learner (w/o scaling)

F

DR

(

δ

|

z

i

; ˆ

η, γ

= 1)

-

2

0

2

4

B: lower; AU-learner (w/ scaling)

F

DR

(

δ

|

z

i

; ˆ

η, γ

= 0

.

25)

δ

δ

δ

F Details on AU-CNFs

F.1 Architecture

Nuisance CNF

FC

FC

U/L target CNFs

CNF

CNF

2. Upper and lower target CNFs

1. Nuisance CNF

FC

Fully-connected subnetwork

CNF

Normalizing flow,

parametrized by

Losses

Gradient blocking

Noise regularization

Outputs with corresponding

distributions

Inputs / connections

Nuisance CNF

Legend

Our AU-CNFs allow us to implement the Algorithm 1 of our AU-learner (see Fig. 10) by combining several conditional normalizing flows (CNFs) [105, 121]. It consists of a (i) nuisance CNF and (ii) two target CNFs (upper and lower). (1) The nuisance CNF aims to fit the nuisance functions, (ˆ π, ̂ F 0 , ̂ F 1 ) or, equivalently, (ˆ π, ̂ F -1 0 , ̂ F -1 1 ) . (2) Upper and lower target CNFs constitute the second stage working models, namely, G and G , and minimize the loss of our AU-learner .

(1) Nuisance CNF. The nuisance CNF has three components, similarly to [94]. These are two fully-connected subnetworks (FC 1 and FC 2 ) and a CNF, parametrized by θ . The two subnetworks FC 1 and FC 2 form a hypernetwork, which outputs the conditional parameters, θ = θ ( X,A ) . This allows us to flexibly model the conditional outcome distribution.

The nuisance CNF has the following joint loss for the nuisance functions: L N = L NLL + α L π . Here, L NLL is a conditional negative log-likelihood loss, L π is a binary cross-entropy, and α > 0 is a hyperparameter. We additionally employed noise regularization to regularize the conditional negative log-likelihood loss [108].

(2) Upper and lower target CNFs. The upper and lower target CNFs use the pseudo-CDFs / pseudo-quantiles, generated by the nuisance CNF, and then implement a second stage loss of our AU-learner . Both target CNFs have the same structure. Specifically, they have a fully-connected subnetwork, FC 3 , and a CNF, parametrized by β . Analogously, FC 3 serves as a hypernetwork so that the parameters can be conditioned on X : β = β ( X ) .

To fit the target CNFs, we use a second stage loss of our AU-learner , namely, Eq. (13) or Eq. (14). For that, we discretize the Y -space or the [0 , 1] -interval of u into n d values and infer argmin/argmax values based on those grids. Then, to approximate the integrals, we do the same for the ∆ -space and the [0 , 1] -interval of α . The later creates a δ/α -grid with n δ /n α points. Those grids are later used for a rectangle quadrature integration. Furthermore, we also regularize the target CNFs by applying the noise regularization [108].

F.2 Implementation

Implementation. We implemented our AU-CNFs using PyTorch and Pyro. For the CNFs of both stages of learning, we used neural spline flows [24] with a standard normal distribution as a base distribution. Neural spline flows build an invertible transformation based on invertible rationalquadratic splines and, thus, allow the direct inference of the (conditional) log-probability, CDF, and quantiles. Neural spline flows are characterized by two main hyperparameters, namely, a number of knots n knots and a span of the transformation interval, [ -B,B ] . The number of knots, n knots , controls the expressiveness of the flow. The span B defines the support of the transformation. In

our experiments, we tune the number of knots n knots and set the span B via a heuristic depending on sample max/min values (as Y is assumed to be compact).

Training. To train our AU-CNFs, we make use of Algorithm 1. However, both first and second stage models are fit on the same training data D without cross-fitting as (regularized) CNFs as neural networks belong to the Donsker class of estimators [123]. Training of our AU-CNFs proceeds as follows: (1) we fit the nuisance CNF; (2) we freeze the nuisance CNF and generate pseudoCDFs/pseudo-quantiles; and (3) we train the upper and lower target CNFs. The hyperparameters are then as follows:

First stage. Weused stochastic gradient descent (SGD) with a minibatch size b N, n e, N = 200 epochs and a learning rate η N. Furthermore, we set the loss coefficient to α = 1 . Both the number of hidden units of FC 1 /FC 2 and the size of the output of the FC 1 are set to 10. The nuisance CNF has the number of knots n knots, N and the span B = max i (˜ y i ) -min i (˜ y i ) + 5 , where ˜ y i are standard normalized outcomes y i . The intensities of the noise regularization for the input and the output are set to σ 2 x and σ 2 y , respectively.

Intermediate stage. We set n d = 200 and n δ = n α = 50 . Furthermore, we set the scaling hyperparameters γ = 0 . 25 for the CRPS loss and γ = 0 . 01 for the W 2 2 loss. We clipped too low propensity scores (lower than 0.05).

Second stage. The upper and lower target CNFs are also fit via SGD with the minibatch size b T = 64 , n e, T = 200 epochs, and the learning rate η T = 0 . 005 . The intensities of the noise regularization for the input are the same as for the nuisance CNF, σ 2 x . The number of hidden units of FC 3 is also set to 10. The target CNFs have the number of knots twice larger than the nuisance flow, n knots, T = 2 n knots, N, and the span B = max i ( a i ˜ y i ) -min i ( a i ˜ y i ) + max i ((1 -a i ) ˜ y i ) -min i ((1 -a i ) ˜ y i ) + 5 , where ˜ y i are standard normalized outcomes y i . To further stabilize the training of the target CNFs, we employed an exponential moving average (EMA) of the target CNFs parameters [103] with a smoothing hyperparameter λ = 0 . 995 .

We demonstrate the detailed training procedure of our AU-CNFs (CRPS) in Algorithm 2 (AU-CNFs ( W 2 2 ) follow analogously).

Hyperparameter tuning. We performed extensive hyperparameter tuning only for the nuisance CNF. The following hyperparameters are subjects to tuning: the minibatch size n b, N, the learning rate η N, the number of knots n knots, N, and the intensities of the noise regularization, σ 2 x and σ 2 y . Further details of hyperparameter tuning are provided in Appendix G. The hyperparameters of the target CNFs for all the experiments are either kept fixed or are inherited from the nuisance CNF.

Algorithm 2 Training procedure of our AU-CNFs (CRPS)

1: Input. Training dataset D = { x i , a i , y i } n i =1 ; scaling γ ∈ (0 , 1] ; hyperparameter α ; number of epochs n e, N , n e, T; minibatch sizes b N , b T; learning rates η N , η T; intensities of the noise regularization σ 2 x , σ 2 y ; EMA smoothing λ ; δ -grid { δ j ∈ ∆ } n δ j =1 ; Y -grid { y j ∈ Y} n d j =1

2: Init. Parameters of the nuisance CNF: FC (0) 1 and FC (0) 2

3: for i = 0 to ⌈ n e, N · n/b N ⌉ do

4: Draw a minibatch B = { X,A,Y } of size b N from D

5: ( R, ˆ π a ( X ) ) ← FC ( i ) 1 ( X )

6: Noise regularization: ξ x ∼ N (0 , σ 2 x ) , ξ y ∼ N (0 , σ 2 y ) , ( ˜ R, ˜ Y ) ← ( R + ξ x , Y + ξ y )

7: θ ( X,A ) ← FC ( i ) 2 ( A, ˜ R )

9: ˆ L N ( ˆ P , ˆ π ) ← P b N { -log ˆ P ( Y = ˜ Y | X,A ) + α BCE(ˆ π A ( X ) , A ) }

8: ˆ P ( Y | X,A ) ← density of a CNF with parameters θ ( X,A )

10: ( FC ( i +1) 1 , FC ( i +1) 2 ) ← optimization step wrt. ˆ L N ( ˆ P , ˆ π ) with the learning rate η N

11: end for

12: Output. Estimator of the nuisance functions ˆ η = (ˆ π, ̂ F 0 , ̂ F 1 )

13: for i = 0 to n do

▷ Intermediate stage

14: Infer argmax/argmin of sup/inf-convolutions based on the Y -grid: { ˆ y ∗ Y -grid ( δ j | x i ) } n δ j =1

16: Use ˆ η to infer the pseudo-CDF for the δ -grid: { F AU ( δ j , z i ; ˆ η ) } n δ j =1 (Eq. (13))

15: Clip propensity scores for bias-correction terms { C ( δ j , z i ; ˆ η ) } n δ j =1 (Eq. (11))

17: end for

18: Init. Parameters of the upper and lower target CNFs: FC (0) 3 and FC (0) 3

19: for i = 0 to ⌈ n e, T · n/b T ⌉ do

20: Draw a minibatch B = { X,A,Y } of size b T from D

21: Noise regularization: ξ x ∼ N (0 , σ 2 x ) , ˜ X ← X + ξ x

22: β ( X ) ← FC ( i ) 3 ( ˜ X ) , β ( X ) ← FC ( i ) 3 ( ˜ X )

23: ̂ g ( · , X ) ← CDF of a CNF with parameters β ( X ) , ̂ g ( · , X ) ← CDF of a CNF with parameters β ( X )

26: FC ( i +1) 3 ← optimization step wrt. ̂ L AU, CRPS ( ̂ g, ˆ η ) with the learning rate η T

27: FC ( i +1) 3 ← optimization step wrt. ̂ L AU, CRPS ( ̂ g, ˆ η ) with the learning rate η T

28: EMA update for FC ( i +1) 3 and FC ( i +1) 3 with smoothing λ

29: end for

30: Output. Estimator of Makarov bounds with CNFs: EMA smoothed ̂ g and ̂ g

▷ First stage

▷ Second stage

G Hyperparameter tuning

We performed hyperparameters tuning of the nuisance function estimators for all the baselines based on five-fold cross-validation using the training dataset. For each baseline, we did a grid search wrt. different tuning criteria (see the details in Table 3). The optimal hyperparameters can be found as YAML files in our GitHub.

H Dataset details

H.1 Synthetic data

Our synthetic data generator is adapted from [61, 93]. Although the original synthetic benchmark contains hidden confounding, we include the confounder as the second observed covariate. We created three settings with different conditional outcome distributions: (1) normal, (2) multi-modal and (3) exponential. Specifically, synthetic covariates, X 1 , X 2 , a treatment, A , and an outcome, Y , are sampled from the following data generating mechanisms:

where X 1 , X 2 are mutually independent and P j ( Y | µ A ( X ) , A ) are defined by three settings j ∈ { 1 , 2 , 3 } :

where N ( µ, σ 2 ) is the normal distribution and where Exp( λ ) is the exponential distribution.

The synthetic benchmark allows us to infer or approximate the ground-truth Makarov bounds. For the (1) normal distribution, they are given by the analytical solution [26], namely, the CDFs of halfnormal distributions. However, in the settings (2) and (3), they need to be approximated numerically. Thus, for the (2) multi-modal distribution, we only infer the Makarov bounds on the CDF, as the quantiles are not directly available for the mixture distribution. For the (3) exponential distribution, however, we can infer both the Makarov bounds on the CDF and the quantiles.

H.2 HC-MNIST dataset

HC-MNIST dataset was introduced as a high-dimensional, semi-synthetic dataset [52] based on the MNIST image dataset [75]. The HC-MNIST dataset builds on n train = 60 , 000 train and n test = 10 , 000 test images. HC-MNIST takes original high-dimensional images and maps them onto a one-dimensional manifold, where potential outcomes depend in a complex way on the average intensity of light and the label of an image. The treatment also uses this one-dimensional summary, ϕ , together with an additional (hidden) synthetic confounder, U (we consider this hidden confounder as another observed covariate). HC-MNIST is then defined by the following data-generating mechanism:

where c is a label of the digit from the sampled image X ; µ N x is the average intensity of the sampled image; µ c and σ c are the mean and standard deviation of the average intensities of the images with the label c ; and Min c = -2 + 4 10 c, Max c = -2 + 4 10 ( c + 1) . The parameter Γ ∗ defines what factor influences the treatment assignment to a larger extent, i.e., the additional confounder or the one-dimensional summary. We set Γ ∗ = exp(1) . For further details, we refer to [52].

Similarly to the synthetic data with the normal distribution, the ground-truth Makarov bounds for the HC-MNIST dataset are given by the analytical solution, namely, the CDFs of half-normal distributions [26].

H.3 IHDP100 dataset

The Infant Health and Development Program (IHDP100) [46, 114] is a standard semi-synthetic benchmark for treatment effect estimation. It contains 100 train/test splits with n train = 672 , n test = 75 , and d x = 25 . Yet, this dataset contains severe overlap violations, which makes the methods using propensity re-weighting unstable [20, 21].

The IHDP100 dataset samples synthetic outcomes from the conditional normal distribution, N ( µ i , 1) , where µ i are CAPOs provided in the dataset. Therefore, the ground-truth Makarov bounds are given by the half-normal distributions [26].

I Additional results

In the following, we provide additional results for our experiments with synthetic data, the results of the semi-synthetic IHDP100 benchmark, and the runtime information for all the baselines.

I.1 Synthetic data

We provide additional results of our synthetic benchmark in Fig. 11. Therein, the out-of-sample performance is reported wrt. W 2 evaluation score for two settings, i.e., normal and exponential setting. 10 Our AU-CNFs achieve superior performance in the normal setting and perform well in the exponential setting. Notably, the IPTW-CNF also perform well in the exponential setting, mainly due to the orthogonality wrt. potential outcome distributions.

0

.

4

0

.

6

0

.

8

W

2 out

S: normal

|

B: upper

1

.

6

1

.

8

2

.

0

2

.

2

S: exponential

|

B: upper

200

400

600

800

1000

n

train

0

.

4

0

.

6

0

.

8

1

.

0

W

2 out

S: normal

|

B: lower

200

400

600

800

1000

n

train

2

.

8

2

.

9

3

.

0

3

.

1

3

.

2

S: exponential

|

B: lower

Method

Plug-in learner

Plug-in CNF

IPTW-learner

IPTW-CNF

CA-learner

CA-CNFs (CRPS)

CA-CNFs (

W

2

2

)

AU-learner

AU-CNFs (CRPS)

AU-CNFs (

W

2

2

)

I.2 IHDP100 dataset

We report the in- and out-sample results for the IHDP100 dataset in Table 4. As expected, our CA-CNFs achieve the best performance. This happens due to the severe overlap violations in the IHDP100 dataset [20, 21], and learners with propensity-score re-weighting are theoretically expected to perform worse.

I.3 Runtime comparison

Table 5 provides the runtime comparison of different methods used to estimate Makarov bounds. Therein, our AU-CNFs are well scalable.

Lower = better (best in bold, second best underlined)

≈

J Case study: Lockdown effectiveness

In the following, we provide a case study, where we apply our AU-learner to a real-world problem. Here, we want to study the effectiveness of lockdowns during the COVID-19 pandemic by using the observational data collected in the first half-year of 2020 [7]. Specifically, we aim to estimate the probability that the incidence falls after the implementation of the strict lockdown, i. e., a probability of individual benefit from treatment (intervention) (PITB).

J.1 Dataset

We used multi-county data provided by [7]. 11 The outcome Y ∈ [ -7 , 0] is defined as the relative case growth per week (in log), namely, the number of new cases divided by the number of cumulative cases. Then, the treatment A ∈ { 0 , 1 } is taken as an implementation of the strict lockdown one week before. We also choose three ( d x = 3 ) pre-treatment covariates X : the relative case growths from the previous week, the relative case growth from two weeks ago, and the implementation of the strict lockdown from two weeks ago. We assume that the data is i.i.d. and that the causal assumptions (1)-(3) are satisfied. We filtered out observations where the number of cumulative cases is fewer than 20. As a result, we ended up with n = n 0 + n 1 = 152 + 112 treated and untreated observations, respectively.

J.2 Results

We present the results for our case study in Fig. 12. Therein, we report two quantities: the estimated bounds on the probability of the individual treatment benefit (PITB), P ( Y [1] -Y [0] ≤ 0 | x ) , for 20 countries during week 15 of 2020. Additionally, we show bounds on a population analogue of PITB, namely, a probability of the population treatment benefit, P ( Y [1] -Y [0] ≤ 0) . We estimated the bounds on the PITB with both AU-CNFs (CRPS) and AU-CNFs ( W 2 2 ), and both methods produced very similar results (which implies a robustness of our AU-learner ). For the population analogue of the PITB, we first efficiently estimated the distributions of the potential outcomes with interventional normalizing flows (INFs) [94] and then used them to infer the Makarov bounds at the population level.

Average

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

PITB

Australia

Austria

Belgium

Canada

Denmark

Finland

France

Germany

Greece

Ireland

Italy

Luxembourg

Netherlands

Norway

Portugal

Spain

Sweden

Switzerland

United Kingdom

United States

Country

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

Method

INFs

AU-CNFs (CRPS)

AU-CNFs (

W

2

2

)

Bound

lower

upper

Week: 15

There are two important takeaways: (1) The bounds on the PITB are more shifted towards 1, suggesting the drop in the incidence is highly probable after the implementation of the strict lockdown in all the studied countries. (2) The bounds on PITB are much tighter than their population analogue (e. g., average upper-lower bound width is 0 . 66 for AU-CNFs (CRPS) and 0 . 88 for INFs). The latter implies that individualization enhances decision-making and makes the Makarov bounds on the aleatoric uncertainty tighter and, thus, more informative.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We include a paragraph discussing the limitations of our method in Sec. 7.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide the full set of Assumptions in Sec. 3 and 4, and Appendix D. The proofs for the theoretic results are provided in Appendix D.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide all the details needed to reproduce the experiments in Appendices F, G and H.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We used openly available (semi-)synthetic datasets in our experiments. Also, we provide a link to our code: https://github.com/Valentyn1997/AU-CNFs .

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide all the details needed to reproduce the experiments in Appendices G and H.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Yes, we report standard deviation in all of the results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide the details on the used resources and runtimes in the Appendix I.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We include a paragraph discussing the broader impacts of our method in Sec. 7. Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our work does not pose such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All the datasets, used in our work, are cited accordingly.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not provide any assets in our work.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our work does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our work does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper provides the orthogonal estimator for the distributional treatment effect (the conditional CDF of $Y[1]-Y[0]$).\n\nSTRENGTHS:\nThis paper is technically strong, demonstrating a high level of mathematic rigor and diligence. It presents an in-depth description of the proposed estimator. I think the proposed estimator is useful in practice.\n\nWEAKNESSES:\nDespite the strong technical details, the paper is poorly written in overall. \n\n__1. Weak motivation__\n\nA current shape of Introduction is weakly motivated. Firstly, the way that the paper motivates the problem is misleading:\n> Methods for quantifying the aleatoric uncertainty of the treatment effect have gained surprisingly little attention in the causal machine learning community.\n\nThis sentence is incorrect, since there are literatures on quantile regression and semiparametric density estimation, as reviewed in Section 2. \n\nMore importantly, the introduction doesn't provide the motivation of the problem against the following question: _why a community need the proposed estimator, given that the quantile estimator can capture the distributional treatment effect_.  \n\n\n__2. Difficult to understand due to insufficient information__\n\nAnother issue that the paper has (especially in Introduction) is its lack of back-ground information that readers may need to comprehend. Specifically, in Introduction, the paper doesn’t provide any definition or clue what CDTE is. I understand that the CDTE is defined in the caption of Figure 1 as $P(Y[1] - Y[0] \\leq \\delta \\mid x)$. However, $Y[a]$ is undefined, and this key quantity should be in the main body of the text. Also, even if Figure 1 aims to provide a whole summary of the paper, authors at Introduction have insufficient knowledge to comprehend it. In other words, Figure 1 is too detailed to be presented in Introduction section. Since Figure 1 can only be understood by those who entirely digested the paper from the beginning to the end, Introduction section is not the right position where the Figure 1 is located. I understand the goal of Figure 1, but it doesn't achieve the goal because of insufficient background information. The same issue happens to Figure 2 and Table 1. For example, in Table 1, technical terms like AIPTW, hold-out residual, optimization assumptions are undefined, so it's hard to appreciate the contribution of this paper. \n\n__3. Fuzzy description on contribution__ \n\nFirst, the terms like \"Aleatoric uncertainty\" and \"distributional treatment effect\" are used for denoting the same target quantity. Given that the \"distributional treatment effect\" is clearly describing the problem, I don't see why the authors want to use \"Aleatoric uncertainty\" as a title and employ these two words to denote the same estimand. \n\nSecond, the contribution is wrongly described. Consider this sentence: \n> AU-learner solves all of the above-mentioned challenges 1 – 3 .\n\nThe AU-learner doesn't address Challenge 1 and 2. Challenge 1 means that the distributional treatment effect is not identifiable. The Makarov's bound, _NOT_ AU-learner, is employed to address Challenge 1. Challenge 2 is actually the same as Challenge 1, since it means that there are no known nuisances-based functional for the distributional treatment effect. Again, the Makarov's bound is used to address Challenge 2. AU-learners are representing the approximated quantity of the target estimand in terms of nuisance functionals. \n\nFinally, third contribution \"flexible deep learning instantiation of our AU-learner\" is scarcely described only in Section 5. The description needs to be much improved. \n\n__4. Little focus on the real contribution__ \n\nThe real contribution of this paper, compared to the existing works in Table 1, is to provide the doubly robust conditional distributional treatment effect for the bounds of the conditional CDF of treatment effects. However, little focus and efforts have been made for this contribution. For example, if developing a doubly robust estimator is a contribution, then corresponding results such as detailed error analysis, a closed form of estimators, a detailed recipe of the proposed estimator for the specified working model, how to minimize the losses in Equations (8,9), a simple example, and assumptions should be described.\n\nQUESTIONS:\n1. Are rate-doubly-robustness and Neyman orthogonality violated when the scaling hyperpameters are not $1$? \n\n2. Is the Makarov bound sharp? \n\n3. What are the practical examples of the distributional treatment effect? \n\n4. In line 178, is this phenomenon officially termed the selection bias?\n\nLIMITATIONS:\nThe paper is limited to the setting where the ignitability holds.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a partial identification of quantiles of the individual treatment effect, which are not point-identifiable in general. The authors justifiably argue that characterizing the distribution of individual treatment effects gives a better idea of the aleatoric uncertainty in a causal-inference problem. The proposed method is doubly robust, works with heterogeneous treatment effects, and requires minimal additional assumptions about the data.\n\nSTRENGTHS:\nThe problem of aleatoric uncertainty in causal effects is clearly important, the contribution is significant, and the presentation is solid and concise. \n\n * Explanations of aleatoric versus empirical uncertainty are clear.\n * The numerous diagrams are informative.\n * Algorithm 1 is also easy to understand and succinct.\n\nWEAKNESSES:\n* The benefit and/or novelty of the CA-learner is questionable (also reflected in the results Table 2) and I wonder if its exposition is taking up valuable space. It is an insightful point on lines 182--185 that learning Makarov bounds could benefit from inductive biases of lower heterogeneity than the conditional CDFs. However, it is unclear if the CA-learner loss really incorporates that inductive bias and if so, how much.\n\n * The conclusion is a bit grand. Arguably, previous papers like those highlighted in Table 1 have proposed robust methods for quantifying a version of aleatoric uncertainty of causal effects.\n\nQUESTIONS:\nMy main question has to do with recent related work. In particular, Ji et al. [52] appear to solve a similar problem, and the quick dismissal of that approach in this paper because they \"made special optimization assumptions\" needs further discussion. It would be helpful to spell out what these assumptions are in a concrete sense. In doing so, the authors could also discuss whether these two approaches have any fundamental commonalities, or if the partial identifications are expected to be materially different. Finally, it would be nice to see [52] appear as a baseline in the empirical evaluations, although I understand this could be difficult to implement, especially since [52] is relatively recent.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose a method to quantify the aleatoric uncertainty of the treatment effect.\nFor this, authors estimated Makarov bounds on the CDF and quantiles of the CDTE, and then showed, how one can build a learner, which has properties of Neyman-orthogonality and double robustness.\nThe authors proved the abovementioned theoretical properties of the resulting learner and demonstrated the usefulness of the proposed approach in a series of experiments on synthetic and real-world data.\n\nSTRENGTHS:\nI think the paper has the following strengths: It\n- Addresses an important problem in the field of treatment effect estimation;\n- Proposed a theoretically grounded method, that has useful theoretical properties, to quantify the aleatoric uncertainty of the treatment effect;\n- Published code;\n\nWEAKNESSES:\nI don't have critical concerns about the paper, but I believe the following can improve the paper:\n- I find the paper logically well-structured, but at the same time challenging to read, as it is too concentrated with technical details. I would suggest authors reconsider the narrative, concentrating more on the conceptual ideas in the main part, and moving technical details to the Supplementary;\n- The field of treatment effect estimation is quite special and narrow (from my point of view) among the machine learning community, and it is worth adding some clarifications to the terms and notation used. For example, what is $Y$? It is just said that it is a continuous outcome, without any intuitions of what it could be.\n- I feel that the paper missing the discussion of the alternatives of the proposed approach. For example, why specifically Makarov bounds were chosen, but not other possible alternatives? \n- In Table 2, it is worth explicitly writing that CNF corresponds to the Plug-in learner (as it was called throughout the paper). I would suggest Plug-in-CNF (like written for IPTW-CNF).\n- In Line 268 there is a minor typo: Should be Eq.13 and Eq. 14.\n\nQUESTIONS:\n- What are the alternatives for Makarov's bounds?\n- Are such options like conformal predictions or plain confidence intervals somehow useful to estimate AU in the context of treatment effect? \n- Compared to the vanilla Plug-in estimator (say Plug-in-CNF), what is the additional computational overhead of AU-learner?\n- In lines 212-216, it is mentioned that CA-learned still has shortcomings (a) (and a new shortcoming (c)). In practice, how severe is the shortcoming (a) (selection bias) compared to AU-learner? Can you provide a toy example?\n\nLIMITATIONS:\nOne of the limitations, not mentioned by authors, is that the approach requires (conditional) Normalizing Flows, and hence might not work well in the high dimensional scenario.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce AU-learner, a method to estimate the conditional distribution of treatment effects (CDTE) and hence capture the  variability in the treatment effect. They use Makarov bounds for partial identification and use conditional normalizing flows for estimation. Further, they show that AU-learner satisfies Neyman-orthogonality and double robustness.\n\nSTRENGTHS:\n(S1) The paper is well written and the authors provide a good experimental validation of the proposed method.\n\n(S2) The authors are the first to propose a doubly robust estimator for the Makarov bounds on the distributional treatment effect.\n\nWEAKNESSES:\n(W1) I would have liked a more comprehensive discussion of how the proposed bounds compare against existing ones in terms of sharpness. For instance, it is not clear to me if the bounds you get are sharper than [58] in the binary outcomes setting. \n\n(W2) I think it would be useful to compare the proposed methods against the non-doubly robust estimators. In practical settings both outcome and propensity models are likely misspecified, hence the main advantage of doubly robust estimators are the faster theoretical rates. In this setting (with both nuisances misspecified), it is not clear if the doubly robust version of the bounds is always better in practice and some more experiments would be nice to explore this. \n\n(Minor) In the extended related work on partial identification and sensitivity models (line 658) you could also mention more recent approaches that incorporate RCT data together with proxy and instrumental variables, e.g. the works of [1] and [2].\n\n\n[58] Nathan Kallus. “What’s the harm? Sharp bounds on the fraction negatively affected by treatment”. In: Advances in Neural Information Processing Systems. 2022.\n\n[1] Falsification of Internal and External Validity in Observational Studies via Conditional Moment Restrictions. Hussain et al. AISTATS 2023.\n\n\n[2] Hidden yet quantifiable: A lower bound for confounding strength using randomized trials. De Bartolomeis et al. AISTATS 2024.\n\nQUESTIONS:\n(Q1) For a binary outcome, could you comment on how your bounds relate to Kallus [58]? Why don't you consider them as a baseline (say for estimating bounds on the quantiles of the treatment effect distribution)?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study the distribution over the individual treatment effect for binary treatments, continuous outcomes, and observed, potentially high-dimensional confounders. The authors build up on prior work on Makarov bounds, to develop a new method to lower/upper-bound the conditional CDF and the quantile function of the treatment effect. They develop a doubly robust learner and perform a theoretical and empirical evaluation. In a series of experiments, They compare several instances of their method with a baseline based on kernel density estimation. While the bound-based methods clearly outperform the baseline, they yield rather similar results.\n\nSTRENGTHS:\nThe authors clearly state the problem and derive a viable solution. They first argue that learning estimators of the CDFs of the two potential outcomes, and plugging them into the bound computation does not yield optimal bounds. Instead, they then propose to learn the outcome CDFs which directly target the Makarov bounds. Third, they augment this loss with a term accounting for selection bias, which yields a doubly robust learner. Finally, the authors derive an implementation based on neural normalizing flows.\n\nThe authors present a great set of supplementary material including additional discussion of related work, additional experiments, and implementation details. The material might be sufficient for a journal publication.\n\nWEAKNESSES:\nThe plugin estimator based on a conditional normalizing flow seems to perform rather well in the experiments. The CA learner w/o bias correction seems to perform as good as the AU learner (slightly worse in Table 2, slightly better in Table 4 in the appendix). The added complexity of the AU learner, in particular, the one-step bias correction which comes with an additional tuning parameter, renders the practical value questionable. It seems that a lot of work went into the AU learner; but it might still be worth focusing the presentation more on the simpler CA learner (e.g., discussing the role of g in the main doc).\n\nQUESTIONS:\nI do not fully understand the role of the working model g \\in G in the CA learner. The Makarov bound is a CDF and G may include all possible CDFs; so if G is rich enough, the CA learner may return the same CDF estimate as the plugin estimator. If we restrict G, why does this cause tighter bounds?\n\nLIMITATIONS:\nNeurIPS Paper Checklist is provided; no concerns.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}