Efficient Causal Decision Making with One-sided Feedback
Authors: Jianing Chu
Shu Yang
Wenbin Lu
PULAK GHOSH
Venue: ICLR 2025 Poster (2025)
Keywords: semiparametric efficiency
one-sided feedback
causal decision making
Primary area: causal reasoning

Abstract: We study a class of decision-making problems with one-sided feedback, where outcomes are only observable for specific actions. A typical example is bank loans, where the repayment status is known only if a loan is approved and remains undefined if rejected. In such scenarios, conventional approaches to causal decision evaluation and learning from observational data are not directly applicable. In this paper, we introduce a novel value function to evaluate decision rules that addresses the issue of undefined counterfactual outcomes. Without assuming no unmeasured confounders, we establish the identification of the value function using shadow variables. Furthermore, leveraging semiparametric theory, we derive the efficiency bound for the proposed value function and develop efficient methods for decision evaluation and learning. Numerical experiments and a real-world data application demonstrate the empirical performance of our proposed methods.

Published as a conference paper at ICLR 2025
EFFICIENT CAUSAL DECISION MAKING WITH ONE- SIDED FEEDBACK
Jianing Chu
Shu Yang & Wenbin Lu Department of Statistics North Carolina State University (syang24, w1 wl @ncsu. .ew.ssuuduuu
Amazon
j anning chu3@gma com
Pulak Ghosh
Indian Institute of Management pulak. pulak.ghosh@i mbb ac.. in
ABSTRACT
We study class of decision-making problems with one-sided feedback, where outcomes are only observable for specific actions. A typical example is bank loans, where the repayment status known only if loan is approved and re- mains undefined if rejected. In such scenarios, conventional approaches to causal decision evaluation and learning from observational data are not directly applica- ble. In this paper, we introduce novel value function to evaluate decision rules that addresses the issue of undefined counterfactual outcomes. Without assuming unmeasured confounders, we establish the identification of the value function using shadow variables. Furthermore, leveraging the semiparametric theory, we derive the efficiency bound for the proposed value function and develop efficient methods for decision evaluation and learning. Numerical experiments and a real- world data application demonstrate the empirical performance of proposed methods.
INTRODUCTION
Binary decision-making problems pervasive in the real world, encompassing domains such as bank loan approval (Pacchiano et al., 2021), job hiring (Raghavan et al., 2020), school admission (Baker & Hawn, 2022), and criminal recidivism prediction (Lakkaraju et al., 2017). Often, feedback in these scenarios is one-sided. Take bank loan approval as an example: decision-maker is pre- sented with covariates describing a loan applicant and decides whether to grant or deny the loan. If the loan is approved, feedback regarding the applicant's repayment is subsequently received. How- ever, if the loan is denied, no further information obtained. There are two main objectives in these maasinnnmmaingggg processes: (1) evaluating a decision rule that aims to approve loans for applicants likely to repay while denying loans to those unlikely to do so, based on the expected outcomes achieves; and (2) deriving an optimal decision rule that maximizes the expected outcome.
Decision-making with one-sided feedhack can be viewed as a special contextual bandit problem with two actions, "approve" and "reject", where the outcome is observable exclusively when an individual approved. Significant challenges arise due to the inherent heterogeneity between the approved and rejected groups--peecffiallly the conditional distribution of the outcome given the covariates may differ between these two groups. As a result, using an outcome model trained on approved samples to predict outcomes for the rejected group is generally unfeasible. To address model bias, one category of approaches uses exploration strategies to gather additional information from new samples, gradually reducing the bias 2021). However, most existing works are restricted to binary outcomes and specific outcome mod- els. lacking robustness to model misspecification and unable generalize to numerical outcomes. Moreover, in real-world applications, exploration can be costly, risky, or even unethical, such as in
time (e.g. Jiang al., 2021; Pacchiano al.,
"This work done prior joining Amazon.

---

Published as a conference paper at ICLR 2025
healthcare, finance, and education. This motivates to develop practical approaches to decision evaluation and learning for different types of outcomes from observational data (Dudik et al., 2014; Kallus & Uchara, 2020; Athey & Wager, 2021; Chu et al., 2023a:b).
As mentioned above, disparities between approved and rejected groups often lead to variations in outcome measures due to unobserved differences in action selection, which also serve predictors for the outcomes. This phenomenon violates a critical assumption in the causal inference literature for identifying and estimating the value function, known as the no unmeasured confounders (NUC) assumption (Imbens, 2004). This assumption, also referred to as strong ignorability (Rosenbaum & Rubin, 1983) or exogeneity (Imbens & Rubin, 2015), posits that actions are independent of potential outcomes given the covariates. Under this assumption, various approaches have been developed for estimating the value function, such as the inverse propensity weighting (IPW) method (Horvitz & Thompson, 1952) and the doubly robust (DR) method (Zhang et al., 2012; Dudík et al., 2014; Jiang & Li, 2016). The NUC assumption, however, can be often violated in many real-world scenarios. When the NUC assumption does not hold, the identifiability of the value function may be com- promised, and existing estimators under this assumption may no longer be consistent for the value function.
To deal with such violations, the utilization of instrumental variables (IVs) emerges as a well- established strategy in the literature (Angrist et al., 1996; Hernán & Robins, 2006; Aronow & Carnegie, 2013; Wang & Tchetgen Tchetgen, 2018). An IV is defined as a pretreatment variable that is independent of all unmeasured confounders, and does not have direct causal effect the outcome other than through the action. However, it is acknowledged that identifying suitable IVs poses a considerable challenge, given the potential existence of numerous unmeasured confounders and the difficulty in eliminating the possibility of an IV's dependence on all of them. In contrast to IVs, we consider an alternative approach using distinct type of variables known as shadow vari- ables (SVs) (Wang et al., 2014; Shao & Wang, 2016; Miao et al., 2016; Li et al., 2024). SVs are independent of the action after conditioning on fully observed covariates and the outcome itself. Meanwhile, SVs are related to the outcome, potentially through unmeasured confounders. For ex- employment, sensitive attributes such the age of candidates should be
ample, in fairness-oriente independent of the decision. However, these attributes may be related to the performance of candi- dates, thereby qualifying them as SVs. With the utilization of SVs, we show that the proposed value function is identifiable.
The contribution of this paper is multi-fold.
First, propose novel value function for miasiinn-maiinggg with one-sided feedback. Without assuming the NUC condition, we consider a model that involves both outcomes and covariates for the action assignment mechanism. We provide identification for the proposed value function under this model by leveraging SVs.
Second, derive the efficient influence function (EIF) and the semiparametric efficiency bound of the value function. Motivated by the EIF, we develop two different efficient estimators for the value function with binary and continuous outcomes, respectively. Our proposed estimation strategy does not require estimating the density when the outcome is continuous, thereby avoiding instability and distinguishing our methods from existing literature.
Third,
we establish theoretical properties for the proposed estimators. We show the estimators are consistent and achieve the semiparametric efficiency bound under mild conditions of nuisance func- tions approximation.
Fourth, we propose classification-bart based framework for learning the optimal decision rule, which allows us to leverage wide range of existing classification tools tailored to different classes of deci- sion rules. Through numerical experiments, we demonstrate that the proposed method significantly outperforms conventional decision learning methods.
RELATED WORK
Contextual Bandits, Off-policy Evaluation and Learning As formally described in Section 3, decision-makinn with one-sided feedback can be formulated as a special type of contextual bandits problem (Chu et al., 2011; Agrawal & Goyal, 2013; Zhou al., 2020). There are limited num-
2

---

Published as a conference paper at ICLR 2025
ber of works focusing one-sided feedback, with two notable related works in this setting. Jiang et al. (2021) considered binary outcomes and estimated outcome functions using generalized linear models, proposing adaptive online learning approach that integrates uncertainty into outcome estimation. Pacchiano et al. (2021) studied the same problem setting with binary outcomes, approx- imating the outcome function using deep neural networks and proposing an online algorithm train an optimistic decision-making model. However, their methods cannot be generalized to numerical outcomes and focus on the online learning setting. In contrast, the primary focus of our work is on decision evaluation and learning using observational data, commonly referred to as off- off-policy evalu- ation and learning in the context of contextual bandits. Off-policy methods have attracted significant interest, particularly in fields such as finance, medicine, and education, where experimentation and exploration can risky, costly, or even unethical (Dudík et al., 2014; Kallus & Uehara, 2020; Athey & Wageer, 2021; Chu et al., 2023a;b).
Labels Although we study the problem under the contextual ban- labels problems in semi- supervised learning (Misra et al., 2016; Kleinberg et al., 2018; Sohn et al., 2020; Coston et al., 2021). In these problems, only subset of instances receive labels, determined by the choices of mekesion-makers. This issue is further complicated by unmeasured confounders that influence both human decisions and the resulting outcomes. Lakkaraju al. (2017) proposed model evaluation method based on the assumption that the decisions in the historical dataset are made by different decision-makers with varying thresholds for their yes-no decisions. Sportisse et al. (2023) studied the problem in semi-supervised learning, adopting the assumption that the label-missing mechanism independent of covariates given the label itself, implying that all covariates are SVs. Based on this assumption, they constructed consistent estimators for the loss function by modeling the label- missing mechanism. Hu al. (2022) adopted the same assumption but proposed estimators without modeling the missing mechanism. The significant difference in our work is that we do not require all covariates to be SVs; instead, we allow the missing mechanism to depend on both the covari- ates and the outcome. More importantly, develop the most efficient estimator by utilizing the semiparametric theory.
dits setting, it is intrinsically related to the selecive/oon--raddom-iiiss
3 PRELIMINARIES
We consider binary action A E {0, 1), where action I denotes "approve" and action 0 denotes "reject". Let X E X C R denote vector of covariates, and Y E R denote the observed outcome of interest. We assume larger values of Y are preferred by convention. We study the problem under the counterfactual potential-outcome framework (Rubin, 2005). The potential outcomes Y (a). a = 0, 1, which are the outcomes that would be observed if subject received action = 0 or = 1. both are well-defined in conventional recision-making problems. Under the Stable Unit Treatment Value Assumption (SUTVA) (Rubin, 2005), we have Y = AY (1-A)) (0). However, under the one- sided feedback setting, only (1) is defined, and the outcome Y is only observed an individual is approved (A 1). In this case, the observed outcome is always Y (1).. The observed data are then (O; = (Y:A,, A, X,), 1,... n} and we assume they are independent and identically distributed.
A decision rule ग X' - [0, 1] is a map from covariates to a probability, so that decision maker, when presented with covariates X, will select action 1 with probability =(X), In conventional decision-making making, where potential outcomes are defined for both actions, implementing a decision rule π in a population would yield the population mean outcome, commonly referred as the value function, defined as follows:
V(*) =
+
(1)
Under the one-sided feedback setting, since Y()) is not defined, we can no longer use the definition of value function in (1). We define new value function
(i(T) (X))))((X)).
The interpretation of (I(T is straightforward. Consider practical example of bank loans and deterministic decision rule π (where π xX) can only take on values 0 or 1). Let Y denote the money earned by the bank if loan is approved. For an applicant with covariates X, if π (X) 1,
3

---

Published as a conference paper at ICLR 2025
indicating loan approval, then (11)=(X) = =(1) represents the potential financial outcome for the bank. On the other hand, if x()) = 0, indicating loan rejection, the bank neither carns nor loses any money. Therefore, the newly defined value function (at quantifies the expected monetary outcome for the bank when implementing decision rule æ for loan approvals. We define the optimal decision rule the one that maximizes the defined value function: "¹" arg max Vi (TT). Our first goal is to evaluate a given decision rule π by estimating (T( using the historical data (0; (Y,A,, Ad, X n... Our second goal learn the optimal decision rule n².
IDENTIFICATION, EIF, AND EFFICIENCY BOUND
In this section, we provide the identification of the value function V1 (1(1,,, and establish the corre- sponding EIF and efficiency bound under the semiparametric theory.
4.1 IDENTIFICATION
Without assuming the NUC condition that [I) | X, we consider general action assignment mechanism that depends not only on covariates but also on the potential outcome:
xxx, y) = P(A
x = Y(1) y}:
and we assume 0 < (x(x y) < 1. Let fxx denote the marginal density of X, and let fly x, 1)
denote the conditional density of Y (1) given X and 1. Let w(x) = P(A = X x). We can show that the value function (T(TT) has the following representation (details are given in Appendix A.1):
(x = ((Y())((X)) / [(x))(()))))))))))()))) {
f'v | x. dy (x))dxx.
(3)
4((x
Therefore, we can identify V (T) through identifying f(x), w(x), f(u x, 1), and ((xx v). The likelihood function for a single observation is
f(x)w(( (1 - (xx))> [ 1))
Thus, fx), wxx), and f(y x, 1) can be identified from the observed data distribution. However, as noted in the literature (e.g. Wang et al., 2014; Miao et al., 2016), cp(x y) is not identifiable without further assumptions.
We assume that covariates X can be partitioned into two subsets of variables U and Z, i.e. X = (U", z² U and Z are variables satisfying the following assumptions.
Assumption 4.1 ZLAA (1) and Z (1) U.
Assumption 4.2 For any function (IY U), E(h(Y(I),U) X.A 1} 0 implies
(LY, U) = 0 almost surely.
Assumption 4.1 indicates are SVs and sp(x, P x P U= Y (1) y} (u, y). For example, in fairness-orienter employment, sensitive attributes such as the age candidates should unrelated to the action assignment. If these attributes cor- relate with the performance of candidates, they can be considered SVs. SVs can be selected based on expert prior knowledge, or alternatielly, representations that serve the role of shadow variables can be generated directly from observed covariates without the need for prior knowledge (Li al., 2024). Assumption 4.2 is known as the conditional completeness assumption, which is widely used identification problems (Newey & Powell, 2003; Miao et al., 2015; Yang et al., 2019). This con- dition guarantees the uniqueness of (pu y). When both Y (1) and Z are categorical variables with / and m levels, respectively, Assumption 4.2 holds if ( < m. When Y (1) is continuous, Assumption 4.2 holds when fly x, 1) follows some common distributions, such as exponential families.
Theorem 4.3 Under Assumptions 4. and 4.2, fxx,, wxx), fly x, 1), and ylu, y) are identifiable, and thus (T( is identified by
(((+)=/ ((x))(x)) {
fly x xx, (( dy (x))xx.
(4)
4

---

Published as a conference paper at ICLR 2025
4.2 EIF AND EFFICIENCY BOUND
The identification (4) motivates a rich class of estimators for the value function. However, to guide the construction of more principled estimators, derive the EIF and the efficiency bound for the value function using the semiparemetric theory (Bickel et al., 1993; Tsiatis, 2006) in this section. Semiparametric models are sets of probability distributions that indexed by both finite- dimensional parametric and efficiency bound defined as the supremum of the Cramer- lower bounds for all parametric submodels. The EIF is the influence function of semiparametric regular and asymptotically linear estimator that achieves the semiparametric efficiency bound. We assume a general model for the ac- tion assignment mechanism, denoted as sp(u, ys 17), which is represented by parameter Consider the Hilbert space T of all measurable functions the observed data with mean zero and finite vari- ance, equipped with covariance inner product (h1, h2) first derive the nuisance tangent space and its orthogonal complement, where the nuisance tangent space is defined the mean squared closure of all parametric submodel nuisance tangent spaces (Bickel et al., 1993; Tsiatis, 2006). For the ease of exposition, we simplify (U, Y(1); as p(n) and 0ip(U, Y (1); as ((n)..
nonparametric components. The semiparametric
(1))(-))22) where h1, h2 T. We
Theorem 4.4 The Hilbert space T can be decomposed as T= A @ A
where
A1 (x)(X) [X) 0]]
A2 Ah2(X, (()) =(X)
B(b (X, (()) X) ((,, x. A 1) =0
1- (X)
((n) A [
A1
((1)
gXX is a function with the same dimension as 17, and the notation denotes the direct sum of two spaces that are orthogonal to each other.
Based on Theorem 4.4, the EIF for Vi (T) has the following form
+ w ((X) A
Doff = Mx( + Ab,
(()G(X
X) + D" S
w X)
EA1
EA
EAg
where [X)] = 0, (X, X, A = A 0. Sn.eff is the efficient score for 77, and D is vector with the same dimension 7). The efficient score Sn can be obtained by projecting the score function of 1) onto A1, as stated in the following theorem.
Theorem 4.5 Under Assumptions 4.1 and 4.2, the efficient score for is
p(n) ((n)
A
E
p()) P
X, A
Sy.eff
(x) ((1)2
X A A-A=1
By projecting the value function identification (4) onto A1,22, and A1, we can derive hT(X), (x),), and D. The EIF and semiparametric efficiency bound for the value function are given in the following theorem.
Theorem 4.6 Under Assumptions 4.1 and 4.2, the EIF for V1 V1(TT) is
A ((n)
A
E
1-((()) (())
Y
X,
(T) = +x))
Y
+
VT(TT) + D Sy,eff (5)
(n)
E
1-p(n)
X A
where D (Var(S
E
(X)
E 1-vin)
stuje
x|A111
x(n)
E
(X)E
2(n) Y X, p(m)
I A xxA==1]])
X,4=1
9(7)
-(1)
The semiparametric efficiency bound for (I(T)) is ((T) = E(02 (T)).
5

---

Published as a conference paper at ICLR 2025
5 EFFICIENT DECISION EVALUATION AND LEARNING
5.1 EFFICIENT VALUE ESTIMATION
Based on the EIF (5), since D is fixed vector and off is score function with mean zero, propose the following estimator for Ví
E
1-
(()) Y
x.
(I(T) = Pn
(x)
a
y
(
(6)
(p(T)
((())
E
1-p(n) p(m)
x
where [=]] = Σ h for any given function h(x), and quantities marked with hats are estimates of their unmarked counterparts. To obtain the value estimator, we first need to estimate n and two conditional expectations E metric estimator for can be obtained by solving the following equation:
1-s(g)y x and E I-ę(y)
+(n]² |x, 1 A general semipara-
((n)²
(u,
P
(u, y:m) g(x;; 0,
where 1(;; is a calibration function with the same dimension as n, Although this estimator achieves consistency and asymptotic normality under certain regularity conditions. its efficiency is not guaranteed. To ensure minimum estimation variability introduced by n. we need to derive the efficient estimator of 17, denoted as Ten- This estimator can be obtained by solving the estimation equation based on the efficient score Szfeff given in Theorem 4.5,
(nn) (p())
a
x, x,
Pr
0.
p P(n)
However, the closed forms of the two conditional expectations in (8) are unknown and need to be approximated. We consider the following two scenarios.
Scenario I: When the outcome Y is binary, say Y E {0,1}, we can specify model for P(Y = X, A 1) and we denote its estimator P(Y expectations in (8) can be estimated by Σ
X, 1). The conditional
p(m) (())²
X, A =1] ((U,
Bp(UU.1.)) P(Y =
8n
(p((U.0:)) 4 = 2(,0m))) Bq
(1 - feY 1 X, A = 1)), and T. (( pl
X, A=
-(U,1m))- = | 1) + (U.0:))
(UU,0:)) (1 f(YY - 1)}. Thus we can get the
(())
efficient estimator Not by solving (8). Next, the conditional expectations in (6) can be estimated by
1- (()
X, A } (U, P(Y = X, A = 1), and 1-p(g) P(Y
X, A-
(..
1U.. Ma
X, 1)
(U.00.))) (1 P(Y
X, A = 1)). By plugging
the estimators neh E ( p(n)2
y X, X,A= and E 1-p(n) p())
X, A } into (6), obtain the
value estimator and denote as (T).
still first model the conditional density f(Y A 1). However, the density estimation often requires large sample sizes and complex algorithms to achieve accurate estimates. This can be computationally intensive and prone high variance, particularly in high-dimensional spaces. Instead, we propose a two-step estimation strat- egy. In step 1, we find a root-n consistent estimator 1)()) For example, we can choose a simple calibration function g(x; 17) and solve the equation (7). In step 2, we construct pseudo-outcomes (00(1) (afl and (((7(1)1 ( and the estimators of the conditional expectations, Tº
Scenario II: When the outcome Y is continuous,
(()) X, A 1=
p(n)"
and E p(n)-1 X, A can then be obtained using regression with these pseudo-outcomes p(1)²
Thus we can get the efficient estimator Nel by solving (8). Similarly, to estimate the condi- tional expectations in (6), construct pseudo-outcomes 1-2( and 1-2(2e) The ((ha)" "0aa""

---

Published as a conference paper at ICLR 2025
estimators E
p(n) p(
Y X A 1 and
1).
((n) A =
1) can be obtained using regres-
p(1)2
sion with these pseudo-outeomess. By plugging the estimators Nef B
{ 1
X, A 5 3 and
X. = into (6), we obtain the value estimator and denote it as (T)
We now establish the theoretical results for the proposed value estimator. We first make the following assumptions for the nuisance functions and their approximations.
Assumption 5.1 For all E X, (i) { (x)), [K (x)l3
where (x = -E (((n)=
x, 1]:
f(1)
(ii) for any (x(xx)= E
e(n) (p())
x,
E
1-p(a) Y x, {x)(())),
1b2(())) oc. (iii) for
p())
any kx)x)= E
s(m)-1 P(D)
x,
I-sp(r) (P))
x, 1
E
pp()) (())))
x, (x() _ kx(x).
Assumption 5.1 (i) and (ii) require that the conditional expectations and their estimations are bounded. Assumption 5.1 (iii) requires that the conditional expectations are consistently estimated. In the case of a binary outcome, the estimation of P(Y = 1 X, A 1) is required to be consistent. For continuous outcomes, given the root-n consistency of 77 we only require that the regression with constructed pseudo-outcomes is consistent. This can be achieved by various machine and deep learning models (e.g. Kennedy, 2016; Farrell et al., 2021).
Theorem 5.2 Under Assumptions 4.1, 4.2, and 5.1 (i) (ü), Var(T) is a consistent estimator for (z).. Additionally, if Assumption 5.1 (iti) holds, (T achieves the semiparametric efficiency bound ((x).
5.2 FROM EFFICIENT DECISION EVALUATION LEARNING
In this section, we consider a deterministic decision rule class and propose method based on the efficient estimator (T) to learn the optimal decision rule, x = arg max Vi (T). A natural estimator for the optimal decision rule x² would be î = arg max=en V=*(*). However, this direct search poses a significant challenge as typically involves non-convex and non-smooth optimization problems and can be computationally expensive. We have the following proposition to transform it into a weighted classification problem.
Proposition 5.3 Maximizing the value estimator Ves( is equivalent to a weighted classification problem of minimizing the following loss function over II,
TH (xi, > 0) # x (x,,,.
where ((x;, Vi, (:)
ar 4i (Ratt)) Vi {I
1 y(x,,
vinlt
for sism.
pi (Rwer)
1-alt [x,,1]
w
With Proposition 5.3, we have transformed the optimal decision rule learning into a weighted clas- sification problem (9) where for subject i with features x; the true label is I[v(x, Ut, a:) 0} and the sample weight is (x,, Vi, az)]. The choice of classification approach dictates the restricted class II. Compared to direct search, a classification-bart optimizer facilitates handling more com- plex functional classes and allows for the use of off-the-shelf machine learning and deep learning software packages.
6 EXPERIMENTS
We have carried out extensive simulation studies and a real data application to evaluate the perfor- mance of the proposed methods.
7

---

Published as a conference paper at ICLR 2025
6.1 SYNTHETIC SCENARIOS
We compare the proposed method with three alternative methods. One consistent but not efficient estimator for is the solution to the estimation equation (7) with a simple choice g(x; 17). We denote this estimator as nuaive- The first estimator for the value function is the IPW estimator with naive: ViPw naive (TT = F'
(Thaaee UI (x)
The second estimator is also an IPW estimator but with
Netf Vipw-oft (TT = Pn (Patr) yT (x)
The third estimator is the DR estimator (Zhang et al.,
2012; Dudík et al., 2014): (T) = nr, (α(x)
E(y x) + R((y xx))=
a [xx
Decision Evaluation: We first generate covariates X = (X1, X2, Xz) ~ -1, E),
1
-0255
-0.25
where Σ
- 0.25
1
-0.25
We consider two types of potential outcome, continuous
0.25
-0.25
and binary.
Case I: The potential outcome Y()) is generated by Y(1) = 8X, 4X3 4X2- + 4X3 + 6,
where is generated from normal distribution with mean 0 and standard deviation 0.5. The action A is generated from ~ Bernoulli (x. (1)),, and (x,,t((((X,,)))) Y())) = 1/[1 + exp
X2 - 0.1Y (1) Thus, X3 is the shadow variable. We construct three different evaluation decision rules as mixtures of a deterministic decision rule (X) = I(2X, - X? -X2 X3 0) and the uniform random decision rule T. (X) by changing a mixture parameter a, i.e., i (X)
a (X) + (1 a) (X). The candidates of the mixture parameter a are {0.6, 0.3, 0.0}.
Case 2: The potential outcome (1) follows a Bernoulli distribution with probability of success 1/(1 + exp(X, + X₂ + Xa)}- The action A is generated from A ~ Bernoulli(s(X,, (())), and logit((p(X,, (L)))
1/[1 exp + X1 + X2 + 0.5Y (1)3). Thus, X3 is the shadow variable. We construct three different evaluation decision rules as mixtures of a deterministic decision rule (x( = 1(X,+ +X++Xx < 0) and the uniform random decision rule T. (X) by changing a mixture parameter a, i.e., क (X) = (X) +1-)))(1))))) (l=a)x(XX) The candidates of the mixture parameter a are 0.4, 0.03.
For both cases, the true value function for each evaluation decision rule is obtained by generating a large sample (x, Y,(1)) with size N = 10 and applying the empirical version of Vi(x) = ([)())=(X)). We consider correctly specified logistic regression model for (p(r). We obtain Thaive using n)) = (1, 11, 22, x2,x))² We obtain the efficient estimators îet and Vot(TT) using the approach introduced in Section 5. Specifically, in case 1, all the regressions with pseudo-outcomes are using random forest (RF) models. In case 2. we estimate P(YY 1 X,A 1) using a generalized additive model (GAM). For the DR estimator, we estimate w(x) using GAM in both cases. We estimate Eyy x) using RF in case and using GAM in case 2.
We consider samples with size = 1000, 2000. For each case, we conduct 500 replications. The root-mean-squar error (RMSE), the standard deviation (SD), and the bias results for cases and are reported in Table and Table 2.
Table 1: Decision evaluation results for case 1: (a) 0.0Ta + 1.0mu, (b) 0.3 + 0.7 (c) 0.6 + 0.4
(b)
RMSE
SD
Bias
RMSE
SD
Bias
RMSE
SD
Bias
1000
Var
0.3512
0.3480
0.0468
0.5509
0.5483
0.0530
0.7999
0.7977
0.0591
Viiw-
0.7893
0.7890
-0.0229
0,8279
0,8278
-0.0127
0,8740
0.8740
-0.0024
Vipw-ctt
0.6172
0.61 19
0.0807
0.8426
0.8387
0.0809
1.0852
1.0822
0.0810
VIDR
0.4421
0.1559
0.4138
0.4371
0.1842
0.3964
0.4364
0.2162
0,3790
n 2000
Vet
0.2003
0.1985
0.0274
0.2016
0.2005
0.0209
0.2169
0.2165
0.0143
Vipw-msiee
0.7057
0.7026
-0.0662
0.7363
0.7341
-0.0575
0.7733
0.7718
-0.0489
Vipw
0.2563
0.2539
0.0353
0.2771
0.2761
0.0228
0.3121
0.3119
0.0103
VIR
0.3647
0.1077
0.3485
0.3538
0.1245
0.3312
0.3455
0.1444
0,3139
We have the following observations. VoFF. ViPw-naise and VIPW are nearly unbiased with sam-
ple size
1000, 2000.
However, VDR has significantly larger bias when compared other
8

---

Published as a conference paper at ICLR 2025
Table 2: Decision evaluation results for case 2. (a) 0.0mg 1.0T (b) 0.4 + 0.6m (c) 0.7
0.
(a)
(b)
(c)
RMSE
Bias
RMSE
SD
Bias
RMSE
SD
Bías
1000
VaE
0.0172
0.0172
-0,0005
0.0207
0.0207
-0.0008
0.0239
0.0239
-0.0011
View-saine
0.0204
0.0204
-0.0001
0.0246
0.0246
-0,0003
0,0282
0.0282
-0.0005
Vitw-uit
0.0179
0.0179
-0.0006
0.0219
0.0219
-0.0009
0.0254
0.0253
-0.0012
Von
0.0196
0.0097
0.0170
0.0223
0.0124
0.0185
0.0248
0.0152
0.0196
n 2000
Var
0.0119
0.0119
-0.0005
0.0142
0.0142
-0.0009
0.0163
0.0163
-0.0013
Vapw-
0.0141
0.0141
-0.0003
0.0167
0.0167
-0.0006
0.0190
0.0190
-0.0009
Vil
0,0122
0.0122
-0.0004
0.0148
0.0147
-0.0007
0.0171
0.0170
-0.0009
VDR
0.0179
0.0069
0.0166
0.0198
0.0087
0,0178
0.0215
0.0106
0.0187
estimators. This is because the NUC assumption is violated in this setting. Among three consis- tent estimators Veft.ViPw ViPW and Vipw-eff. Vett has the smallest standard deviation and RMSE, which is expected. One interesting observation is that for case 1, when sample size = 1000, the standard deviations of ViPW-naive with decision rules (b) and (c) are smaller than those of ViPw-eft- One possible reason is that when the sample size is small, the performance of nonpara- metric regressions with pseudo-outcomes may have larger variation. As the sample size increases, the standard deviations and RMSEs of three consistent estimators Venr ViPw and VIPW become smaller.
Decision Learning: We consider the same covariates as those used in decision evaluation. The potential outcome is generated by Y(1) 8X, 6XX 4X2 + 2X + c, where c is generated from normal distribution with mean 0 and standard deviation 0.25, The action A is generated from ~ (ernoulliy((X, (1)) = 1/11eexp(0.5
Xi X2 - 00)15Y We construct four estimators following the same procedure in decision evaluation. We tree-based classification algorithm introduced Zhou et al. (2023) and focus on depth-2 decision trees for illustration. To evaluate and compare the performance of estimated optimal decision rules obtained by different methods, we compute the corresponding value functions and percentages of making correct decisions (PCD). Again, generate large sample (X,, (1)),, with size N 10 For fixed decision rule t, its value function is computed using the empirical version of (T) = E[Y (1)=(X)]. We then maximize the value function and obtain the oracle optimal depth-2 decision tree, denoted n² For each estimated optimal decision rule म. its associated value function is computed using the generated
large sample and the PCD is computed by N-1 ΣN [x(,) x² (X.)). We report the value and PCD results for the decision rules obtained by different methods in Figure 1. We observe that the decision rule obtained by our proposed method has best performance compared with other methods, in terms of values and PCDs. For our proposed method, as the sample size increases, the means of values become larger, PCDs get close to 1, and the standard deviations of values and PCDs become smaller.
4.-
43-
0.85
0.50
1000
2000
1000
2000
Sarple size
Sample size
Methods
eff
IPW-naive
IPW-eff
DR
Figure 1: The values and PCDs of estimated optimal decision rules.

---

Published as a conference paper at ICLR 2025
6.2 REAL DATA APPLICATION
In this section, we apply our method to a loan application dataset from a fintech company. A sim- ulated dataset based on the real data is available upon request. The fintech lender aims provide short-term credit to young salaried professionals by using their mobile and social footprints to deter- mine their peedit-worthiness To get a loan, customer needs to download the lending app, submit all the requisite details and documentation, and give permission to the lender to gather additional information from the smartphone, such as the number of apps and SMSs. We obtained data from the lending firm for all loans granted from February 2016 to November 2018. There are 42,777 customers in total. We select set of covariates X, which includes the applicants' age, salary, loan amount, CIBIL credit score, number of apps, number of SMSs, number of contacts, and number of social connections. The action A are whether or not the lender approves the loan applications. The outcome Y is defined 1 if the loan is repaid, and -1 if the applicant defaults the loan. We conduct hypothesis testing, and our analysis reveals no significant evidence suggesting that the number of social connections violates Assumption 4.1. Therefore, consider it SV.
We randomly sample the training data with a size 3000 and 5000. We compare the four estimators introduced in Section 6.1, which constructed using the same procedure for the binary outcome. Specifically, we estimate E(Y X) for the DR method and IP(Y X, A = 1) for the proposed method using GAM. For the DR method, we estimate wxX) using GAM as well. We consider logistic regression model for (nn) that uses all covariates (excluding the SV) and the potential out- come as predictors. We obtain Maaive using gX,,1) algorithm as in the synthetic scenarios to estimate the optimal decision rule. The proposed efficient estimator over the entire dataset used as the testing value. The training-testing procedure re- peated 100 times. We report the results of testing values in Figure 2. We observe that the average value proposed method is much larger than those other three methods, while the variability of proposed method is smaller. This implies the proposed method has better performance than other three methods.
(T. We use the same classification
0.25
0.20
Value
0.15
0.10
3000
5000
Sample size
Method
Proposed
Figure 2: The boxplots of testing values under estimated optimal decision rules by different methods.
7 CONCLUSION
In this paper, we propose a novel framework for causal decision making under the one- one-sided feed- back setting. Specifically, we define a new value function for this task and provide identification leveraging SVs, without assuming NUC. We develop efficient evaluation and learning methods mo- tivated by the semiparametric theory. Numerical experiments and a real-world data application demonstrate the empirical performance of our proposed methods. Although this work focuses on the contextual bandits setting, method has significant potential for extension many semi- supervised learning tasks (Hu et al., 2022; Sportisse et al., 2023) and generative models (Ma & Zhang, 2021; Ipsen al., 2021) with non-random missing data.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Shipra Agrawal and Navin Goyal. Thompson sampling for contextual bandits with linear payoff'.. In International conference on machine learning, pp. 127-135. PMLR, 2013.
Joshua Angrist, Guido W Imbens, and Donald Rubin. Identification of causal effects using instrumental variables. Journal of the American statistical Association, 91(434):444 455, 1996.
Peter M Aronow and Allison Carnegie. Beyond late: Estimation of the average treatment effect with an instrumental variable. Political Analysis, 21(4):492-506, 2013.
Susan Athey and Stefan Wager. Policy learning with observational data. Econometrica, 89(1): 133-161, 2021.
Ryan S Baker and Aaron Hawn. Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32(4): 1052--00511 2022.
Peter J Bickel, CAJ Klaassen, Y Ritov, and JA Wellner. Efficient and Adaptive Inference in Semi- parametric Models. Johns Hopkins University Press, Baltimore, 1993.
Jianing Chu, Wenbin Lu, and Shu Yang. Targeted optimal treatment regime learning using summary statistics, Biometrika, 110(4):913-93) 2023a.
Jianing Chu, Shu Yang, and Wenbin Lu. Multiply robust off-policy evaluation and learning under truncation by death. In International Conference on Machine Learning, pp. 6195-6222 6227. PMLR, 2023b.
Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff func- tions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 208-214. JMLR Workshop and Conference Proceedings, 2011.
Amanda Coston, Ashesh Rambachan, and Alexandra Chouldechova. Characterizing fairness over the set of good models under selective labels. In International Conference on Machine Learning, pp. 2144-2155. PMLR, 2021.
Miroslav Dudik, Dumitru Erhan, John Langford, and Lihong Li. Doubly robust policy evaluation and optimization. Statistical Science, (1(4),455-51,2000
Max H Farrell, Tengyuan Liang. and Sanjog Misra. Deep neural networks for estimation and infer- ence. Econometrica, (111::811213:) 2021.
Miguel A Hernán and James M Robins. Instruments for causal inference: an epidemiologist's
dream? Epidemiology, 17(4):360-33 -723600--32222 2006.
Daniel G Horvitz and Donovan Thompson. generalization of sampling without replacement from finite universe. Journal of the American statistical Association, 47(260):663-6688 685, 1952.
Xinting Hu, Yulei Niu, Chunyan Miao, Xian-Sheng Hua, and Hanwang Zhang. On non-random missing labels in semi-supervised learning. arXiv preprint arXiw:2006.14923 2022.
Guido W Imbens. Nonparametric estimation of average treatment effects under exogeneity: A
review. Review of Economics and Statistics, 86(1):4-29, 2004.
Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sci- ences. Cambridge University Press, 2015.
Niels Bruun Ipsen, elling with missing not at random data. In ICLR -nte-nnntaa Conference on Learning Representations, 2021.
Mattei, and Jes Frellsen. not-miwae: Deep generative mod-
Heinrich Jiang, Qijia Jiang, and Aldo Pacchiano. Learning the truth from only one side of the story. In International Conference on Artificial Intelligence and Statistics, pp. 2413-2421. PMLR, 2021.
Nan Jiang and Lihong Li. Doubly robust off tolico value evaluation for reinforcement learning. In International Conference Machine Learning, pp. 652-661. PMLR, 2016.

---

Published as a conference paper at ICLR 2025
Nathan Kallus and Masatoshi Uchara. Double reinforcement learning for efficient off- off- policy evalu- ation in markov decision processes. Journal of Machine Learning Research, 21(167), 2020.
Edward H Kennedy. Semiparametric theory and empirical processes in causal inference. Statistical Causal Inferences and Their Applications in Public Health Research, pp. 141-167, 2016.
Jon Kleinberg, Himabindu Lakkaraju. Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. Hu- man decisions and machine predictions. The quarterly journal of economics, 133(1):237-293, 2018.
Himabindu Lakkaraju, Jon Kleinberg, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. The selective labels problem: Evaluating algorithmic predictions in the presence of unobservables. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pp. 275 284, 2017.
Baohong Li, Haoxuan Li, Ruoxuan Xiong, Anpeng Wu, Fei Wu, and Kun Kuang. Learning shadow variable representation for treatment effect estimation under collider bias. In Proceedings of the s1 International Conference on Machine Learning, pp. 28146-28163, PMLR, 2024.
Chao Ma and Cheng Zhang. Identifiable generative models for missing not random data imputa- tion. Advances in Neural Information Processing Systems, 34:27645 276588 2021.
Wang Miao, Lan Liu, Eric Tchetgen Tchetgen, and Zhi Geng. Identification, doubly robust estima- tion, and semiparametric efficiency theory of nonignorable missing data with a shadow variable. 2015.
arXiv preprint
Wang Miao, Peng Ding, and Zhi Geng. Identifiability of normal and normal mixture models with nonignorable missing data. Journal of the American Statistical Association, 11165166)1633-168,,, 2016.
Ishan Misra, Lawrence Zitnick, Margaret Mitchell, and Ross Girshick. Sceing through the human reporting bias: Visual classifiers from noisy human-centric labels. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2930-29399 2016.
Whitney Newey and James Powell. Instrumental variable estimation of nonparametric models. Econometrica, 71(5):1565-1578 2003,
Aldo Pacchiano, Shaun Singh, Edward Chou, Alex Berg, and Jakob Foerster. Neural pseudo-label optimism for the bank loan problem. Advances in Neural Information Processing Systems, 34: 6580-6593, 2021.
Manish Raghavan, Solon Barocas, Jon Kleinberg, and Karen Levy. Mitigating bias in algorithmic hiring: Evaluating claims and practices. In Proceedings of the 2020 conference on fairness, accountability, and transparency. pp. 469-481, 2020.
Paul R Rosenbaum and Donald Rubin. The central role the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55 1983.
Donald B Rubin. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469):322-333 2005.
Jun Shao and Lei Wang. Semiparametric inverse propensity weighting for nonignorable missing data. Biometrika, 103(1):15--177, 2016.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems, 33:596-600 0089,66688,00 2020.
Aude Sportisse, Hugo Schmutz, Olivier Humbert, Charles Bouveyron, and Fierre-Alexandr Mattei. Are labels informative in semi-supervised learning? estimating and leveraging the missing-data mechanism. In International Conference on Machine Learning, pp. 32521-32539. PMLR, 2023.
Anastasios A Tsiatis. Semiparametric theory and missing data. Springer, 2006.
12

---

Published as a conference paper at ICLR 2025
Linbo Wang and Eric Tchetgen Tchetgen. Bounded, efficient and multiply robust estimation of average treatment effects using instrumental variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80:531-550, 2018.
Sheng Wang, Jun Shao, and Jae Kwang Kim. An instrumental variable approach for identification and estimation with nonignorable nonresponse. Statistica Sinica, pp. 1097-1116, 2014.
Shu Yang, Linbo Wang. and Peng Ding. Causal inference with confounders missing not at random. Biometrika, 106(4):875 888, 2019.
Baqun Zhang, Anastasios A Tsiatis, Eric B Laber, and Marie Davidian. robust method for esti- mating optimal treatment regimes. Biometrics, 68(4):1000-1018, 2012.
Dongruo Zhou, Lihong Li, and Quanquan Gu. Neural contextual bandits with ucb-based exploration. In International Conference on Machine Learning pp. 11492-11502, PMLR, 2020.
Zhengyuan Zhou, Susan Athey, and Stefan Wager. Offline autti-action policy learning: Generaliza- tion and optimization. Operations Research, 71(1):148-183, 2023.
13

---

Published as a conference paper at ICLR 2025
A TECHNICAL PROOFS
A.1 PROOF OF THEOREM 4.3
Proof.
(1 X x
x}
(L( X = -x=xxA=1]uxx] + (L) x 0}{1 w(x)3
+ (1 wxx)) { yf(y) x,0)dy
-w(x)
V x,1)dy
{ u(1 - w(x)}[(v) x, O)dyy
(x)
V x, x,1)dy
f(y
x, x,00][1 w(x)3
=w((x)
y x, 1,1)dy
y 1)
dy
f(y x, 1)
(x)
y y x, 1,1d
+ { u5(y x,
xxxx)
1
1
dy
(0(x,))
(x) { y x, 1)dy
+ w((x) { 1 (x/[[xxx1)) {
1
1
}}]
dy
y)
=w(x) /
[V x, 1, dy.
((xx v)
Therefore,
Vi(T)) = B(X())(()
=E (E((((())(())) X])
= / (x)))) x=x]dx
f f(x)((()) { f(y 1x,1) dy TT (xx)
v)
To identify VT( we need to identify fxx), w(x), f(y x, 1). and (p(x, w). The likelihood function for a single observation
- (1
1- f(y 1)
A key observation is that
w(x) = / f(y x) dy.
(x(x 1))
Under Assumption 4.1, (x(x y) = = y} = p(u, v), and the likelihood function becomes
= x x = x, Y() = = uu Y()
S(y x. dy f(x) { (
U
f( x, dy
T
1-0
fly 1)
(p(u,) v)
plu, v)
Assume we have two different sets of models f(x), fly ((u, y). such that
x, 1), plu, v), and f(x). fly x, 1),
-17 1-a
f(x) {ʃ
[( x)
du p(u.v)
f x) dy ((u,v) v)
1
f(y x, 1)
-17 1-a
{ ( dy
-a
My x, 1) dy
-s(x)
f(y xx,x),1_. (10)
(uu,
14

---

Published as a conference paper at ICLR 2025
Taking 0 in (10), we have
}
f(x) 1 {
1w x, 11y1x,1) ((u
-11
flu x, 1)
dy
= f(x) 1
G v) dy }
(11)
Taking a = and taking integration with respect to Y (1) on both sides of the above equation, we have
[(x) {
J(y M1 x2 [
1 i(x)
{
f(y xx dg
dy
(12)
ppu,
((, y)
By Equations (11) and (12), we have
f [iy x, dy /
( y)
f(y x; dy. ( y)
f(x) f(x) and
Taking = 1 in (10), we have
f(x) {
f(y x. [(y11x1)
fly x. f 1 {
fly x, ( y)
du)
flu x, x, 1).
co(, 4)
Thus, we have
f(y) x, 1) = =fly x, 1).
Finally, from
f(y xx,1) dg ʃ fly x, 1) spu, v)
dy.
(u y)
and Assumption 4.2, we have
p y) = ((,, v) Thus, f(x), w(x), f x, 1), and cx(x v) are all identified. The value function V1(T) then identified.
A.2 PROOF OF THEOREM 4.4
Proof. Let O = [AY, A, X) summarize the vector of observed variables with the likelihood factor-
ized as
S(O) = ((X)
-( f(Y A=
We consider a one-dimensional parametric submodel fo, (X) for f(X), and a one-dimensional
parametric submodel fe_(Y A 1) for f(Y X, A 1). respectively. The submodel
fe, (X) contains the true model f(X) at 0 = 0, i.c., fo, (X)
10,=0= fX)). Similarly, the
submodel for (Y X,A A 1) contains the true model f(Y A 1) 02 0, i.c.
fo, (Y X. A = 10 f(Y X, = 1). The submodel for the likelihood is
(e, fe, (X) we, [1 we,
(Y X,A = 1)4
i log fe. (O)
log fo, (X)
00,
30,
i log fo, (0)
i log fo, (Y
A 1) + Wo, (X) A
a log fo, (Y X, A x
E
002
002
We, (X)
00g
By the semiparametric theory (Bickel et al., 1993; Tsiatis, 2006), we have the nuisance tangent spaces
A (x) [X) = 0}},
A2 (X Y(1) + w((X) - A
E(h2(X, Y()) X) (X, x (X
1 (X)
15

---

Published as a conference paper at ICLR 2025
It is easy to verify that A1 1 Ag. Consider generic
element in A1, Ag1 (X, (1)) +
(1 A))2(X). Since A1 1 A1, for any measurable mean zero function (i(X),, we have
El{Agi (X, Y()) + (1 - ())
=B( (X, Y(1) + (1
(X) X
(X)(w(()) (X, (X, x, A 1} (1 - w (X)))
-0.
Therefore, w/ (x))E( 9) (x,
X,
w(X) 9 (X) is a constant and we denote
as c. Since Agı (X, (1)) + (1 (X) mean zero, we have
[[4g (X, Y())) +11- -
= [[ (X) [g) (X, Y(1) X + (1)
-B(()
Therefore, we have
W X (9 (L)) X (X
(13)
Since A2 1 ATs we have
E
(Agi (Xg,(X, + (1 (1-))
Aha(X, w(X) A
Efh (X. (X, ( x3
1)
w/ (X)
=E
[X)B (X)))X X, (x Y(I)) XH
h2,X, Y(1))
|x,A-1 }}
=F
(X (X, (X)))))XXXXXXXXX | - xx
w
X
(p(1)
E (x) { ( (X)) 9x(X)
h2(X, (()) (
=E
((1)
=0.
Therefore, 91 (x,X + Sa (x) =(n)
is a function of X and we denote it as k(X):
((X) = 91 (X. Y (1)) + 92(X)
(0(1)
Taking the conditional expectation on both sides, and by (13), we have
k(X) = EX, X,A = 1 + 92 (x)
w (X) (X).
Therefore, we have
92(X) 9XX, 9x
(1)))
Thus,
(p(1) A
(g, - (1- A)gz (X)
(1()) 91 (X),
and A1 - (())
91 (x) This completes the proof.
A.3 PROOF OF THEOREM 4.5
Proof. The score function for is
w
X
c ())
S, =
E
x
1
wl
X
((n)
The efficient score for 1j is the projection the score function S, onto the space A1. Notice that S, 1 A1. Therefore, we can write
A w (XX)
((n) (())
+ w (X) A w X)
E(b(X, (((()Y))) X) p(n) A
x Ab(X,
(p(1) c(X),
1 (X)
EAg
A1
(14)
16

---

Published as a conference paper at ICLR 2025
where Bfb(X, (1))
X, = 1 0. Let A = 1 in (14), we have
(p(n)
x} ((()) -
E
X b(x, Y()) - E( Y())
c(X).
(p(n)
(0(1)
By taking B(- X) on both sides, we have
E
X
E
p(())
X, A 1
PIN)
PD)
c (X)
1 E
((1))1 ((1))2
X, A
P(n)
Therefore,
(0(1)
A
E
((n)=
(p())
X, A =
Sg.off
(011)
(0()))
X, X A =
((()22
Let A = 0 in (14), we can further derive that
6(X, (p())
w(X) ox)).
A.4 PROOF OF THEOREM 4.6
Proof. We consider a one-dimensional parametric submodel [x(X) for X(X), and a one- dimensional parametric submodel fxxYY X, A 1) for J(Y X, 1), respectively. The submodel fo (X) contains the true model f(X) at a = i.c., foo (X) = f(X). Similarly, the submodel fY X, 1) contains the true model f(Y X,A 1) at So. i.c. faa (Y X, A = J(Y X. A = 1). Let 0 = (). The submodel for the likelihood can be represented as
Sa., (0) (0) (x (X) W3, (X) Sa(Y X,A= w3,, (x), (x))
which contains the true model at 0g (oo, Bo). For the ease of exposition, write Vi (T) V(T). We use 9 in the subscript to denote the quantity with respect to the submodel, e.g., Vo( is the value of V (=) in the submodel.
Let
a log So(O)
a log fa da
Sao
da
0=0a
a0000
a log (O)
b log fa
X, A 1)
w(X) A
alog fa X, A 1)
Sgo
x
03
03
1 w (X)
03
0=A0
8=30
9=3
i log Oo)
A - wi XX)
Ölog (p(n)
S.
E
Ōn
1 w (X)
On
B=8g
Blog AA=1))
@ log ((n)
Let 83,-
and sn
03
On
B=Do
By the semiparametric theory, the EIF for V V(T)) must have the form
(X)
A
doff (X + Ab)(X)
(ins(XY(()) X]+ + D Sn
w
EA1
EA
EA2
where Exh((X) = 0}, E(g X, A 1} = 0, and D is vector with the same dimen- sion as 1). The EIF doef for (TT) must satisfy
OVo(*)/0ol 000 E(sbeet Scn);
(Vo(T)/83)0=a, E(dbcff S ),0), (oo(T)/0) 10=00 E(dcat
17

---

Published as a conference paper at ICLR 2025
(I)
Y
|x, A 1 Sao
(Vo 10=00- = E T w (X)E
(p(m)
B(oottSoo) - (x (X)Sao).
We have
M(() =0===(XXjp x(X)
Y
|xAA=1] V(T).
(p(n)
(II)
(V))3) (x))) x))3,p),
Sp)oHtSpo) = [ (()) -- B(h5 Y x}
8800
0Vo
@Vo(T)/B8 1= -E(den
xxX)
=B
(())) X, (1)) 1 1-w((p) (X)
Ei (X, (1)) - (X) - - X
8000
xxX)
E(h5(X (X, Y(1))) X]
r(1 E(Y(1) XX
=E E h2(X, (
T(X)
(o(n)s80)
x
1 (X)
((n)
((T))
Since (1) = 03X, x()
(X)
E( must
(X)
P())
k(())
be function of X and denote ml (X):
XX)
Bih
Y(I)) X] =(X) - X (15)
m (X) h, -- (X)
(
(p(1)
Taking the conditional expectation on both sides, we have
Eth (X (L)) X}
m (X)
w (X)
Therefore, we have
E (X, Y(I)) x]
(X) (X, X]
Y(1) B(Y()) x)
X00++ wXX
(X))
(XX
(())
(p())
Taking B(-
X) on both sides,
E h X ((1))
X}
(X)
w Eth (()) (HE(1/<(()) X}
=B Y X 1-
- -(X) (E(Y())/((()
x (LY x3 X]].
We have
w - E (Y X - (L) X}E(L/{(1) X]
E(4)(X,Y()) |X]] X]
M (X)
w
E(1/p(n) X -1
(16)
By Equations (15) and (16),
1
1
E
Y(1) ven)
X
B(Y(1) X
((1)
x
Y(1) - -(Y X
h5(X, Y(L) =(X)
(X))
sp(1)
E(1/p(n) X) X1-1
sp(1))
18

---

Published as a conference paper at ICLR 2025
(III)
((1)
(-
X
@Vo(T)/Onlo-0, =
(X)
+(1)
{g(n)
E T(X)Y(1) (1)
1-p(m)
(0(n)
:p(1))
E
X
p(
E( (7)
By @Vg(T)/On"le-pp = ))
E
pin) (
X,
pp(1)
p(n))
Y X, A xx11=1)])
D = (Var(Sn,ott)
E
X
IF
n (X)E
E
(1)
X, A = 1]
(p())
(p())
p())
By (I),II), and (III), we complete the proof.
A.5 PROOF OF THEOREM 5.2
Proof.
E
1-p(n)
X. x111) 1)
X
A
p(1)"
E
π
(X)
Y
1
(0())
(pn))
E
1-p(1)
p())
A
-E
(X)
((1)
A
=E
X
((1)
A
Y()) x
E2
(X)
((1)
Y(1)
E (A)))
p(n)
(X)Y Vi().
Since solution Equation (7) is root-n estimator of 17, by the strong law of large numbers and uniform consistency, we have (T) = (=(()) o,.)
By Assumption 5.1 and the empirical process theory, have
4 (Melt)
(p(1)2
x,
1
(felt))
E v(n)
(p())
x,
PP,.
IP,.
((
E
((())=
x, 1
(P
E
(((n)=1
x, 1
(p()))
sp())
p(())
p(n))
=P
40 (eeH)
a
((())
xx
IP
4p Meet
a
v())
x,
+ op (17)
60 (fitt)
((n) -1
50 (ett))
(())
E
((())2
((n)²
19

---

Published as a conference paper at ICLR 2025
{ (2(7)
[((m)) x, ] and Eg E { |
p())2 x, 1 By Assumptions
For the ease of exposition, let E1 = E 5.1, we have
4p (
F.
(Telt) a 1Rq
}
E2 /
(Tel) )
R2
(P
10 (ott)
a
E1
IE
(Mett)
E2
12,
4 (ettt
IE
1
E1
40 (Telt)
E2
16
E2
E2
6 Meat 10 (ottt)
a
E1
18
E1
(E2
Eg)
P
E2
E2
50 (n 1/2) x (1)
0p 1/2).
(18)
By Equations (17) and (18), we have
pin) (
((()) F(m)
IP,
(
a
x,
Pn
(T
a
x.
+ Op(n 1/2)).
(Mttt)
(x)))
x,
1
40 (ett))
1F
((n)
x. 1
(())²
((
By taking Taylor expansion, we have
(f
(
s(n)
x, 1
P.
((MER))
p(n)-1
x,
(p())*
(ip(1))
E
(
((1)
x, 1
(Sm(e +P
(7-1) + tpppnn11/
(
E
p(n)
x, 1
p())²²
(S
Var(S (17 - 11) + Op (n 1/2).
(19)
By Assumption 5.1 and the empirical process theory, we have
E
1-s9
*())²²
x,
1
Vott =P,,
(xx)
y1
+
4p Foet
(eett)
1-p(())
x,
p(n)"
1-p(1) (
x,
{
{
E
1-p(n) Y x())
x
+ P
1
a
P
+ 0p(n-1122
p Moft)
E
1- (()) p(n)²
x, 1
sp
Nett
E
(-
x, 1
(())
(20)
20

---

Published as a conference paper at ICLR 2025
For the case of exposition, let E3 = B
I-p(n)y
x, }. By Assumptions 5.1, we have
((112
(T(ttt))
RF3
(M(e)t)) - 133
P
+ PP
TF2
(((ea)))
B2
E3
E3
P
(PMett)
E2
122
4 Mott)
E3
180
E3
E3
P
4 (Mettt)
12
12
E2
E2
40 (Texx) a
E3
E3
E3 (E.
- E2)
P
40 (Peett))
E2
E2
50 1/2) x (1())
=0p(n 1/2).
(21)
By Equations (20) and (21), we have
Van (T) Pn
T(x)
y
{
E
1 1-c(m)
x(n)
Y
x.
op(n 1/2). (112).
(T(tttt)
40 (Tott)
E
p()) p(n)²
x 1
By taking Taylor expansion, we have
Ven(n)
=Pa
T(x)
a p(())
4 +
{1
E
1-p(n)
p(1)
Y
x, 1
((n)
E
1-p(())
x, 1
p(
ap((1)
ap((1)
(-piin) ((n)2
Y
x 1] 1)
x 1
7
P
xx)
(7 - 1) +
8 (n)
E
I (-((()) (()))
(22)
By Equations (19) and (22), have
(eT(TT)
Vi(Tt)
E
1-p(n)
Y
x,
=P,
(xx
y1 +
(0(1)
(n))
I-(e(n) (())
|x
T
1-sp(1)
x; 1
+ P (xx)
aip(n)
(i)(n)
(p())*
() (n (S V((TT) + on
2 (m)
(p²(
E
1-sp(n)
x,
(())
1-s(())
=Pm
7 (x)
p(() y + {1 a
(p()) f
E }
*(1)²*
I-sp(y)
x,
+ DT D""PPSSn,ot VT(TT) + (p
((1)"*
=P,
(x)
((n) y + {1 1- pp())
E3
1-p(n)
(v))
Y xx, 1]
+ D S Vi((T) + top(n 1/2)
I-ve(n)
x, 1 (())"
=P, (T) op(n 11/2).
This completes the proof.
21

---

Published as a conference paper at ICLR 2025
A.6 PROOF OF PROPOSITION 5.3
arg max (TT sell
arg max Tell
T (x:) (xx))b((x)) Xi, Di, (4)
arg max
x (x, ) R50 x,, U4. a; (T) Vi, > 00 Ix, yc, s 0H]
1 (xi, yi, a: a.) >
arg max Tell
(xx Vi, a. (x;) (x: (x. (:) > ( 03]
(x,, Ur. [[ y1. a;) > 0}
arg max sell
(xx, ui (xx)) + (x,, > -)) (xx
arg max sell
(x, (x,, a.) > 0}
(xx ui (x) + 11 (xx B/ a;) 03 - 27 v1) 0
(xx (T, a:) > 0} - (1))) - TT > 01}²
aarg max sell
arg max xell
(x, Ui. [x)[[ - I (xx X yi. az) > 0}}
arg min well
(x,, Vo. (x)()(( 1f (x,, Mr, a.) > 07f²
arg min TEl
[xx, yr, ([T(x) # L[ yc, ax) > 0}}.
Therefore, the decision learning is equivalent to weighted classification problem, where for subject i with features x,, the true label is (x yi, a;) > 0} and the sample weight is [x,(x, yi, a;)].
22


Reviews:
[{"review_id": "EvHVEtUEJQ", "reviewer": "Reviewer_Wzn4", "summary": "This paper addresses decision-making with one-sided feedback, where outcomes are observable only for chosen actions, such as approved loans. The authors introduce a new value function for evaluating decisions without assuming the no unmeasured confounders (NUC) condition. Using shadow variables, they establish identifiability and derive the efficient influence function (EIF) and the semiparametric efficiency bound of the value function. Motivated by the EIF, they develop two efficient estimators for the value function, applicable to binary and continuous outcomes, respectively. They propose a weighted, classification-based framework to learn the optimal decision rule. Empirical results validate the method's effectiveness.", "questions": "I have some questions/comments for the authors:\n\n* The authors state that assumption 4.2 guarantees the uniqueness of $ \\phi(u,y) $. Can the authors explain how this assumption guarantees this? Also, is it a sufficient condition for uniqueness, or could there be necessary or more relaxed conditions?\n\n* Regarding assumption 4.2, it is worthwhile to discuss in the paper how restrictive this assumption is. Perhaps the authors could mention some cases where it does not hold.\n\n* How do high-dimensional categorical covariates affect the estimation for the conditional expectations as required by assumption 5.1?\n\n* The authors mentioned that the maximizer for the efficient estimator denoted by $ \\hat{\\pi} $ is a natural estimator for the true optimal decision rule $ \\pi^* $, which is a maximizer of the true value function. Although in experiments, the authors show percentages of making correct decisions (PCD), can they provide any discussion on how far $ \\hat{\\pi} $ can be from the true optimal rule $ \\pi^* $ in general?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The paper's main strength  is introducing a modified value function and establishing identifiability for this value function by leveraging so called shadow variables (SVs). The authors derive the efficient influence function (EIF) and the semiparametric efficiency bound of the value function. Additionally, in the case of continuous distributions, their proposed estimation strategy avoids estimating the density when the outcome is continuous, thus preventing instability. Furthermore, they demonstrate that their estimators are consistent and achieve the semiparametric efficiency bound under mild conditions for nuisance function approximation.", "weaknesses": "The assumption of knowing the shadow variables (SVs) may restrict the practical application of the proposed approach. Similarly, assumption 4.2 could also limit the applicability of the proposed methods. Although the authors cite some relevant papers for assumption 4.2, more discussion is needed to clarify how restrictive this assumption is.  The proposed methodology requires that conditional expectations \\( P(Y | A = 1, X) \\) be consistently estimated. This might be an issue particularly for high-dimensional discrete covariates, i.e., categorical variables that can take on a large number of distinct categories or levels.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zqDQoceUt7", "reviewer": "Reviewer_d6pz", "summary": "The paper defines a new value function for the problem of 'one-sided feedback' in causal inference for decision making.\n\nIt provides identification results, using the so called 'shadow variables'.\n\nThe authors also provide efficiency bounds using semi-parametric. \n\nFinally, they also provide empirical evidence for their new method.", "questions": "Can you help me understand why your new value function is of practical value and relevance?\n\n- Is the concept of 'opportunity cost' relevant? Why or why not?\n- Is there a practical background for this new value function? is it already being used in decision making elsewhere?\n- What was the inspiration for this value function? Is it simply the observation that we can't do anything otherwise or is there a more fundamental observation from e.g. banking where this function is being actively used?\n\nIf it's just a technical convenience, but no practical relevance, than the contribution of this paper is hard to justify without a stronger discussion of why a user should accept this value function.\n\nIf there is more to it than just a technical connivence, than I am very keen to learn about that!\n\nThanks!", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is strong in exposition of its core idea, methods, experiments and results.\n\nIt seems technically sound.", "weaknesses": "I struggle to accept the core assumption made in line 157:\n\n\"On the other hand, if π(X) = 0, indicating loan rejection, the bank neither earns nor loses any money. Therefore, the newly defined value function V1(π) quantifies the expected monetary outcome for the bank when implementing decision rule π for loan approvals\"\n\nI want to argue that the bank in fact does lose money when it rejects a loan. Some of the loans, might have in fact not defaulted and therefore produced a profit for the bank. This concept is known to me as \"opportunity cost\", sometimes defined as \"the loss of potential gain from other alternatives when one alternative is chosen\".\n\nIn this case, it'd be the loss of the potential gain from giving out the loan, when choosing to reject the loan applicant.\n\nThere seems to be no discussion of whether this new definition of the value function makes conceptually sense. It is assumed to be valid, and technical challenges are solved (probably successfully).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XoLPLC2rMr", "reviewer": "Reviewer_oarq", "summary": "The paper addresses decision-making scenarios where only specific actions yield observable outcomes. Since counterfactuals are undefined in this setting, a new method beyond conventional causal inference approaches is needed. The paper proposes a novel value function to evaluate decision rules in one-sided feedback contexts without assuming no unmeasured confounders. An efficiency bound, $\\Gamma(\\pi)$, is provided for the value function. The paper also introduces efficient estimators for decision evaluation and learning. Through simulations and applications on real-world data, including a bank loan dataset, the paper demonstrates that the proposed method surpasses traditional approaches in robustness and efficiency for evaluating and learning decision rules.", "questions": "Refer to the weaknesses section above, and find the additional questions below:\n1. Is it possible to extend the work to non-binary action settings and more general partial feedback causality settings?\n2. Including a wider range of settings in the numerical experiments would make the results more persuasive.\n3. The theoretical analysis considers binary and continuous outcome variables. Does the theoretical analysis also apply to categorical outcomes?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Please find the strengths below:\n1. The paper studies an important problem in practice. In applications like credit card applications and bank loans, the outcomes may not be observable when certain actions are taken.\n2. The algorithm for learning and evaluating the causal relationship is theoretically effective. The use of shadow variables is innovative and removes the requirement of the NUC assumption.\n3. The paper provides both numerical and practical experiments. The experiments show the efficiency of the proposed estimators for the one-sided feedback setting.\n4. The method in the paper appears flexible and adaptable to me, allowing it to be extended to a variety of settings with partial feedback.", "weaknesses": "Please find the weaknesses below:\n1. The paper only considers binary actions, i.e., $A=0,1$, which is relatively limited. Given that this is an initial work on partial feedback learning, this limitation is acceptable to me.\n2. I did not find any discussion on the optimality of the semiparametric efficiency bound $\\Gamma(\\pi)$.\n3. The numerical experiment is based on a specific setting, which is not general enough (both the action distribution and the outcome function are fixed). The experiment on the bank loan dataset does not show a very notable improvement of the proposed method over IPW-eff, compared to the results in the numerical experiment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ll8ZSS8gdD", "reviewer": "Reviewer_h5vG", "summary": "The authors study a class of decision-making problems with one-sided feedback, where outcomes are only observable for specific actions. They introduce a value function to evaluate decision rules that address the issue of undefined counterfactual outcomes. Without assuming no unmeasured confounders, they establish the identification of the value function using shadow variables. Furthermore, leveraging semiparametric theory, they derive the efficiency bound for the proposed value function and develop efficient methods for decision evaluation and learning. Numerical experiments and a real-world data application demonstrate the empirical performance of proposed methods.", "questions": "It seems that the framework essentially treats all $Y(0)$ as 0. Then is the task just to predict the sign of $Y(1)$?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is well-written. The paper establishes rigorous identification results and semiparametric theory.", "weaknesses": "Please refer to Questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)