RISE: Robust Individualized Decision Learning with Sensitive Variables
Authors: Xiaoqing Tan
Zhengling Qi
Christopher Warren Seymour
Lu Tang
Venue: NeurIPS 2022 Accept (2022)
Keywords: causal inference
individualized treatment rules
sensitive variables
robustness

Abstract: This paper introduces RISE, a robust individualized decision learning framework with sensitive variables, where sensitive variables are collectible data and important to the intervention decision, but their inclusion in decision making is prohibited due to reasons such as delayed availability or fairness concerns. A naive baseline is to ignore these sensitive variables in learning decision rules, leading to significant uncertainty and bias. To address this, we propose a decision learning framework to incorporate sensitive variables during offline training but not include them in the input of the learned decision rule during model deployment. Specifically, from a causal perspective, the proposed framework intends to improve the worst-case outcomes of individuals caused by sensitive variables that are unavailable at the time of decision. Unlike most existing literature that uses mean-optimal objectives, we propose a robust learning framework by finding a newly defined quantile- or infimum-optimal decision rule. The reliable performance of the proposed method is demonstrated through synthetic experiments and three real-world applications. 

RISE: Robust Individualized Decision Learning with Sensitive Variables
Xiaoqing Tan University of Pittsburgh xit31@pitt. edu
Zhengling Qi
George Washington University gizhengling@email.co gwu. edu
Christopher W. Seymour University of Pittsburgh seymourc@pitt... edu
Lu Tang"
University of Pittsburgh estang@pitt.... edu
Abstract
This paper introduces RISE, a robust individualized decision learning framework with sensitive variables, where sensitive variables collectible data and important to the intervention decision, but their inclusion decision making is prohibited due reasons such delayed availability fairness concerns. A naive baseline is to ignore these sensitive variables in learning decision rules, leading to significant uncertainty and bias. To address this, we propose decision learning framework to incorporate sensitive variables during offline training but not include them in the input the learned decision rule during model deployment. Specifically, from a causal perspective, the proposed framework intends to improve the worst-case outcomes of individuals caused by sensitive variables that are unavailable at the time of decision. Unlike most existing literature that uses mean-optimal objectives, we propose robust learning framework by finding newly defined quantile- or infinum-optimal decision rule. The reliable performance of the proposed method is demonstrated through synthetic experiments and three real-world applications.
Introduction
Recently, there has been widespread interest in developing methodology for individualized decision rules (IDRs) based on observational data. When deriving IDRs, some collectible data are important to the intervention decision, while their inclusion in decision making is prohibited due reasons such as delayed availability or fairness concerns. For example, sensitive characteristics of subjects regarding their income, sex, race and ethnicity may not be appropriate be used directly for decision making due to fairness concerns. In the medical field especially for patients in severe life-threen tife-threatening conditions such sepsis, timely bedside intervention decisions have to be made before lab measurements are ordered, assayed and returned to the attending physicians. However, due the delayed availability of lab results, most of the decisions are made with great uncertainty and bias due to partial information at hand. We define sensitive variables as variables whose inclusion into decision rules is prohibited. The formal definition of sensitive variables will be given in Section 3
In this work, we propose RISE (Robust Individualized decision learning with SEnsitive variables robust IDR framework to improve the outcome of individuals when there informative yet sensitive variables that are either not available or prohibited from using during IDR deployment. To achieve this, we propose estimate the optimal IDR by optimizing quantile- or infimum-based objective,
Corresponding author:
Python code is available at httccpp://ggthuuu.co///lllaaaaas
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

respectively, for continuous or discrete sensitive variables. This optimization problem is then shown to be equivalent a weighted classification problem where most existing machine learning classiffers can be readily applied. Our idea falls along the lines of work that considers algorithmic fairness [IS] while extending to the setting of causal inference [56] in the sense that decisions are driven by causality rather than general utility function. We show in our empirical analyses that this leads to fairer and safer real-life decisions with little sacritice of the overall performance.
Assuming that a larger outcome value is preferable, optimal IDRs are traditionally derived through maximizing the mean outcome of the sample population. In this paper, we are interested in a specific yet broadly applicable setting of learning that involves sensitive variables. We consider offline learning where sensitive variables are collected and can be used in training the IDRs, but they cannor be used as input in the resulting IDRs. This is a setting commonly considered in the faimess and privacy-related literature for classification (e.g., (33), but not from a causal standpoint. When there exist important variables that are simply left out from training, the estimated IDR will be biased. This bias can be removed if all important variables are used during training, which we will show in Section 3.1 mean-optimal approach. The optimal action maximizes the mean outcome where the mean is taken over the sensitive variables, conditioning on other variables. This method, however, has no control of the disparity in sensitive variables. Subjects with different sensitive values may report large outcome differences, hence unfairly or unsafely treated. Therefore, objective functions with robustness guarantees for sensitive variables are preferred, since they offer protection to subjects in the lower tail of the outcome distribution with regards to the sensitive values.
For illustration, we consider a toy example with binary actions, A E (-1,1). We remark that the decision can only be made based on the variable X whereas S is sensitive variable. The setup is
are shown in Table 1 and the oracle values under the mean-optimal rule and RISE are given in Table 2
The detailed setup can be found in Section 4.1 Jnder Example 1. We consider vulnerable subjects as those with low outcome values, as highlighted in red in Table [ (A full definition is given in Section 3.2). For X S 0.5, the mean-optimal rule would assign action = 1 as it tries to achieve the largest average reward across S = 0 and S = 1. Recall that S is not available at the time of decision. However, this action results in great harm for subjects with S = as they could get the worst expected outcome of 0. On the contrary, RISE improves the worst-case outcome by assigning A = 1. protecting the vulnerable subjects. Likewise, for X > 0.5, the mcan-optimal rule assigns A -1 1 while RISE assigns A 1 protecting those with S O that could have experienced an outcome of 5. Compared to the mean-optimal rule, the proposed rule achieves a larger reward among vulnerable subjects while maintaining comparable overall reward.
Table I: Toy example setup of [Y| A).
Table 2: Toy example results.
X <
0.5
X > 0.5
Average reward
S=0 0
S=1
S= 0
S ==
Overall
Vulnerable
A =
11
5
Mcan-optimal rule RISE
15.5
2.5
A
=
1
30
0
15
13
13
14
Main contributions. Methodology-wise, 1) we propose a novel framework, RISE, to handle sensitive variables in causality-driven decision making. Robustness is introduced to improve the worst-case outcome caused by sensitive variables, and as a result, it reduces the outcome variation across subjects. The latter is directly associated with fairness and safety in decision making. To the best of our knowledge, we are among the first to propose a robust-type fairnsss criterion under causal inference. 2) We introduce a classification-base based optimization framework that can easily leverage most existing classification tools catered to different functional classes, including state-of- the-art random forest, boosting, or neural network models. Application-wise 3) the consideration of sensitive variables in decision learning is important to applications in policy, education, healthcare, etc. Specifically, we illustrate the application of RISE using three real-world examples from fairness and safety perspectives where robust decision rules are needed, across which we have observed robust performance of the proposed approach. From a fairness perspective, we consider a job training program where age is considered as sensitive variable. From safety perspective, consider two applications to healthcare where lab measurements are considered as sensitive variables.
Foor the mean-optimal rule, overall average reward is calculated by (30 + 0 + 5 27) = 15.5. reward among vulnerable subjects is calculated by (0 + 5)/2 = 2.5; for RISE, overall average reward is calculated by (11 + 13 15- + 13)/4 13, reward among vulnerable subjects is calculated by (13 15)/2 14.
2

---

2 Related Work
Our work focuses on individualized decision rules, which aim at assigning treatment decision based on subject characteristics. Existing methods for deriving IDRs include bosel- methods such as Q-learning [75] 39 38] and A-learning [3 611, model-free policy search methods [77] 81 80), and contextual bandit methods [6]1] In Appendix A provide additional literature review on general IDRs under causal settings. Fairness, safety and robustness are topics of interest that extend well beyond causal inference. In the following, provide review of these areas, with focus given to work related to causal inference and IDRs.
Fairness and safety in IDRs. The consideration of fairness and safety in machine learning has seen an explosion of interest in the past few years [ 69 4 43 20 12 37 47 665 66), especially for solving classification and regression problems to help derive decisions that are not only accurate but also fair. In these work, sensitive variables are also referred to as sensitive, protected, or auxiliary attributes. We extend the definition of sensitive variables to include delayed information that is not available at deployment as it is also suitable for this framework.
Among earlier work, preprocessing [3
17 13 58
and inprocess training approaches 5
20
29
consider disentangling the input X from a known or unknown sensitive variable S so that the transformed X does not contain any information related to S. Due to the causal nature of IDRs, effect of IDRs cannot be estimated consistently when an informative S is left out and the resulting rule is suboptimal. This follows from the classic argument that any unmeasured confounding (i.e., S), if not accounted for, would lead to bias. Similar issues persist in contextual bandits [22, 44). 35 considers reducing the impact of auxiliary variables prediction under distributional shift. Although is motivated from a causal idea, its main focus is still on prediction. Inside the causal framework, 77 (2) extend fairness from prediction to policy learning using causal graphical models by incorporating fairness constraints. [10] considers counterfactual fairness that seeks to achieve conditional independence of the decisions via data preprocessing. Despite earlier efforts in bringing fairness into the causal framework, most of these approaches only ensure mean zero disparity in but do not have robustness guarantees in the sense that the variance of the disparity in S is not controlled. Besides, most examples consider a single categorical sensitive variable, but not multiple or continuous ones.
Robustness in IDRs. Recently the statistical literature has witnessed a growing interest in developing robust methods for estimating IDRs. They introduce robustness into the objective function by using quantile-optimal treatment regimes or mean-opting optimal treatment regimes under certain constraints to improve the gain of individuals at the lower tail of the reward spectrum [72
74
51
49
16] In
particular, 772 51 propose to estimate quantile- or tail-optimal treatment regimes. 74 studies the mean-optimal treatment regime under constraint control for the average potential risk. [49] proposes a decision-rule-base
optimized covariates dependent equivalent for tasks of individualized decision making. (16] considers mean and quantile objectives simultancously by maximizing the average value with the guarantee that its tail performance exceeds a prespecified threshold. Robustness, in their sense, pertains
the outcome distribution subject the sampling error. When sensitive variables are present, we consider instead the robustness of the outcome distribution subject to the uncertainty due to sensitive variables. providing more targeted way of ensuring robustness, which is directly related to fairnss and safety. Compared to algorithms based on explicit faimess constraints (for example 766 78 classification and [79 10] in causal inference) that seek to remove the disparity across different values of S, our method reduces the variance of disparity across S. In addition, constraint-basee approaches typically require specialized optimization procedures whereas our approach presents elegant and systematic way for optimization. To our knowledge, we are the first few to consider decision fairness via a robust objective under the causal framework.
3 Robust Decision Learning Framework with Sensitive Variables
3.1 Preliminaries
Notation. We let random variables be represented by upper-case letters, and their realizations be represented by lower-case letters. Suppose there are n independent subjects sampled from given population. For subject i, let A, E 1) denote a binary treatment assignment and Y, denote the corresponding outcome. Without loss of generality, we assume larger value of outcome is desirable.
3

---

Under the potential outcomes framework [55] 631, let (i(a) be the potential outcome had the subject been assigned to = a for or 1. Let X; be the feature vector and, for now, S. be single sensitive variable. We consider S E $ where $ (1..... ..... K] if S is discrete and & IR if S continuous. The extension multiple sensitive variables presented in Section 3.4
Definition of sensitive variables. We define sensitive variables S as variables that are important to the intervention decision, but their inclusion in decision making prohibited. Formally, consider variables X and S that are both available during model training and are both determinants of conditional average treatment effect [54] While S may be involved in training, the derived decision rule d(-) precludes the input of S due to sensitive concerns. Hence, the derived IDR is function of the form d X + A. Following the above definition, we consider an offline learning framework where sensitive variables are collected and can be used in obtaining the IDRs, but they cannot be used the resulting IDRs. causal diagram and decision diagram are provided in Figure] [ in Figure h. both X and S confound the effect of treatment A on outcome Y. The arrows represent the causal relationship between variables. Note that and S can be correlated. This causal diagram formalized in Assumption bebloo.. On the other hand in Figure [ in the decision diagram under our setting, S is shown in a dotted circle as S may not be readily available at the time of decision making. We connect S and A with dotted arrow to indicate that S is incorporated in the training of the decision rule, but it is not required at deployment.
As it shows
X
S
X
S
A
Y
A
Y
(a)
(b)
Figure 1: (a) A causal diagram. (b) A decision diagram.
Assumption 1. Assume the following conditions hold:
la Consistency: Y = -((1)) -1) + ((I)
1b Positivity: 0 Pr(A 1) 1.
Ic Unconfoundedness: (Y(-1), 1 A((, and (Y(-1), A|X.
Assumption a the standard consistency assumption in causal inference and Assumption [Tp states that every subject has nonzero probability of getting the treatment. Assumption [ states that given X and S, the potential outcomes are independent of the treatment assignments. Besides, unconfoundedness does not hold when only X is given, signifying the important role of S. Under causal settings, Assumption : implics that treatment effects cannot be non-patt non-parametrically identifiable without S 44 54]. Approaches such ones that disentangle X from S under supervised learning will introduce bias towards estimating IDRs.
settings mentioned in Section 2
Naive approaches that omit sensitive variables. When S' is not available for future deployment, = d(X))) over d using (X, Y) during the training
naive approach is to maximize [x{E(Y procedure. This approach introduces bias in the estimation of conditional treatment effect and leads to suboptimal IDR due to the unmeasured confounder S.
Mean-optimal approaches that use the sensitive variables. is thus important that one includes S into the training procedure. For example, if consider the value function framework (i.e., expected outcome) used by most existing works such as [36]] 52), we can show that
E(Y = Ex.s (d))x, (EX Fxx (F( S, = d(x))) # Ex A
where the third equality in I1 holds by Assumption | and the last inequality also indicates the naive approaches without using S will in general fail. Then valid approach is to maximize

---

Ex[ssx(EE(]]) S,A = d(x))} over d using (X,S,A,Y) The optimal IDR under this crite- rion is, for every X X.
d(x) E (gnn (F()), 1)) (
which finds the treatment that maximizes the conditional expected outcome given X by averaging out the effect of the sensitive variable S. Mean-optimal approaches, however, fail to control the disparities across realizations of the sensitive variables due to the integration over S, which may lead to unsatisfactory decisions to certain subgroups, illustrated in the toy example in Section
3.2 Robust Optimality with Sensitive Variables
Driven by the limitation of existing approaches, our goal is derive robust decision rule that maximizes the worst-case scenarios of subjects when some sensitive information is not available at the time of deploying the decision rule. Specifically, our robust decision learning framework draws decisions based on individuals' available characteristics summarized in the vector X without the sensitive variable while improving the worst-case outcome of subjects in terms of the sensitive variable in the population. Formally, given collection D of all treatment decision rules depending only on X, the proposed RISE approach estimates the following IDR, which is defined as
d' E arg max [Gsix (EXx)EE(XXXXA
d(x)))),
d(X)) where G-)x(-) could be chosen as some risk measure for evaluating - for each E S. Examples include variance, conditional value at risk, quantiles, etc. In this paper, we consider Gsix as the conditional quantiles (for a continuous S) or the infimum (for a discrete S) over S.
Specifically, for discrete S, Gsix x is considered as infimum operator of E(Y S.A A= d(X)) over We thus aim to find
d' E arg max Ex inn ( [F(V| -, =,A==(
where inf is the infimum taken with respect to E X, A d(X)) over E S. This implics that for given X, d' (X) assigns the treatment that yields the best worst-case scenario among all possible values of S for every E X, or equivalently,
d' (X) sgn(inf ( = A = 1) inf (F = = ==
For continuous S, we consider Gs|x (,, = d(x))) as ( a(x)))); which is the 7-th quantile of E S, A d(X)) and E (0, the quantile level of interest. Specifically, Q3 x [E(Y)X, S, A d(x)) inf F(t) 2 7} with F denoting the conditional distribution function of F A= d(X)) over 3 given X and d. Note the randomness behind E S, A d(X)) given and d fully determined by the sensitive variable S. Then optimal IDR under this criterion is defined
d' E arg Ex [Q (E(Y)X, S, = d(x)))
This implies that for a given X, d' (X) assigns treatment that yields the largest 7-th quantile of the outcome over the distribution related to S, or equivalently,
d' (X) sgn((Q)) (E()),
1))
We let 0.25 throughout the paper and suppress for simplicity. Results varying the value of T is provided Appendix D see Section 4.1 for details.
Identifying vulnerable subjects. Our RISE framework provides a natural way to define vulnerable groups. Specifically, for discrete S, {EY then arg inf ( X S, 0)} is vulnerable given X, otherwise arg E S, A - 1)} vulnerable. In other words, the vulnerable subjects are those in the worst-off group that needs protection. Similarly, for continuous S. if Qf E(Y S, A = 1)) (x(E(Y)X, S, 0)). then the set {5: F S, = 0) s (s(E(Y) X, S, A 0))} defines the vulnerable subjects given X, otherwise this group is defined {S E(Y X, S, = 1) (s(E(Y)x, S, 1))}.
S, = 0)}. 1)} > inf
5

---

3.3 Estimation and Algorithm
Here we provide a transformation of the proposed RISE from an optimization problem to a weighted classification problem. There several advantages to this conversion: 1) The optimization problem defined in (2) involves a nonsmooth and nonconvex objective function that could lead to computational challenges. 2) With multiple powerful statistical and machine learning toolbox to choose from, classification problem can be more readily solved in practice. Hyperparameter tuning and model selection could be conducted to further boost performance. 3) Compared to direct optimization of (2), a classification beadccaattio--beeeddddeeee optimizer allows the use of off-the-shelf software packages that can be tailored different functional classes or incorporate different properties such as model sparsity.
Proposition Maximizing the objective function in
2 equivalent to maximizing
Ex (x( = 1)( S, - G -1H}-
With Proposition andd a proper estimator of the outcome model E(Y|X, S, A) using training data D, (X,, St, A,, Y) replace the expectation of Y, by its estimate Y; and solve the following,
arg man Il=1[1(d(2)
(x;)((;;) (2(2))},
(3)
where 9i (2+) = G s, = 1)} and 92(2;) = ( s, a, = -1)]. We have the following proposition to address noncontinuity in (3) and transform it into a classification problem. Define FF as a class of all measurable functions over X.
Proposition 2. Maximizing the empirical objective in 33 is equivalent to a weighted classification problem of minimizing over f E F.
n1 E [sgn (g) (Ix)) - -2(x=)) ((x) < (9) - 92(x:)]
(4)
with features Vi, label sg (I.)) - -2((x:) and the sample weight lg ( - gp) Si i n.
With Proposition 2 we have transformed the optimization problem 22 into a weighted classification problem 44) where for subject i with features 2;, the true label is sgn(91 (z+) _2((x;)) and the
sample weight is [g (z;) (2(x;)]. The estimated optimal decision rule by (4) is then given by
d(z) = sgn (x) The proof of Proposition and Proposition 2 is presented in Appendix B
algorithmic overview of RISE. The inner expecta- tion E(Y|X, S, A) can be modeled Y(X,S. A) using a twin model separated by the treatment and control groups (i.e., T-learner as in (27) For continuous A)) is estimated via quantile regression of Y on X but without S. For discrete S, an esti- mate of G(X, A) = inf (E [X, , S, A)} is obtained by finding the minimum among {E(Y|X,S = 1, A,.)..... E(Y|],, S K, A).. The estimated decision rule then be obtained from the weighted classi- fication. In our implementation, neural net- works are used to fit models in the training data sets. The details on modeling and hy- perparameter tuning via cross- validations cross-validations
Algorithm [ provides
Algorithm 1: RISE (Robust individualized decision learning with sensitive variables)
Input: Training data D., (Yi, A., X,, S. Output: Estimated decision rule d for
do
S, G(X, A) = (
(i 81, a;) <- Model X, S, A a) using
D., with == and respectvelll..
if is continuous then
91 (x.) +- Model (E(|XX S, A a)} via quantile regressions of (1,,, 81, a.) on Zis for D, with a = 1;
g²(xx) +- Model QsIx.A S, a)} via quantile regressions of Yi 8;, as) on di. for D., with -
else if S is discrete then
9i (I:) &- Compute inf (Ý, (Ii, 8, a; = 1)]: (V (Iz. 8, 0; = -1)];
ga( < Compute inf.es (
end
given in Appendix C A Python pack- age rise based on neural networks is avail- able GitHuu httt:::///ithub.b com/ ellenxtann /rise). Note that the model choices are flexible.
d Build weighted classification model with features 21, label sgn (gi (xx) ga(xx)), }. and sample weight [g] (a:) g2 (x:) for S return d.
IIf (()) 0, assign random treatment.

---

3.4 Extension to Multiple Sensitive Variables
The extension from S being a single continuous variable to multiple continuous variables is straight- forward in Algorithm I For multiple discrete sensitive variables, similar estimation procedure can be conducted as outlined in Section 3.3 Suppose there are I. discrete sensitive variables, i.e., S = (St, S2 ................................ Sr.). The inner expectation S1 St, A) can be obtained with a twin model of Y on X and all S for each treatment level. The infimum over S is obtained by finding the minimum iterating space of possible parameter values for each sensitive variable. See Section (..tionn4.2]0 for an example of using multiple discrete sensitive variables. We will discuss in Section[5thh the challenges and future work related the scenario with a mixture of continuous and discrete sensitive variables and the identification of vulnerable subjects under these cases.
Numerical Studies
In this section, we perform extensive numerical experiments to investigate the merit of robustness of the proposed framework via simulations and three real-data applications. The results demonstrate that the proposed rules achieve a robust objective with sensitive variables unavailable at the time of decision while maintaining comparable mean outcomes.
Compared approaches. For comparison, we consider the naive and mean-optimal approaches described in Section 3.1] which correspond to different choices of G(.) functions. The naive decision rule that simply disregard information of S, denoted as Base, can be formulated in our optimization framework of (2) by letting G(X, A) = EY [X, A). The IDR can be estimated directly by fitting a model of on X in each treatment arm. The resulting IDR is not sensitive variables- aware and is biased due to confounding, as discussed. Another IDR that resembles traditional mean-optimal decision rules, denoted as Exp, can be formulated as G(X, S, A) = [ A.). This can be obtained by training classification model without S, i.e.. only using X, after obtaining an outcome model for the inner expectation E(Y [X, A). Note that this approach is not robust to extreme behaviors in S. The modeling approaches described in Jopendix[[appllee to here. We also include the double robus! [11 (8) versions of Base and Exp, respectively, by adapting Policytree (PT) [64] 3]. the latest sfate-of-the-aar policy learning method for maximizing the expected values. The double robust analogues of Base and Exp are termed PT-Base and PT-Fxp, respectively.
Evaluation metrics. 1) Objective: the quantile objective is estimated and reported for a continuous S and the infimum objective is for a discrete S. The objective, when T < 0.5, (here T = 0.25)
represents the value of the "low performers" among all possible value of S under given d. 2) Value: the value function, or expected reward used by the most existing methods, such as [36] 52], is defined as V(d)= = E{Y(d)), (d)). It represents the "average performers". For randomized trials, an unbiased estimator of Vdd) is given by ((d) ((,)))/(4A, K ET. where T is the sample size of the test data and x (AA is propensity score.
(Σi= (A, a(x,))/=(A,, x,)) / 1A,
For observational studies, the value is estimated with (d) 7 "I (2,,, 5;, a, d). We report the metrics among all subjects and among the potential vulnerable subgroup, respectively. For simulation, respectively. For real-data applications, we consider a 80-20 split of the dataset into a training data and testing data. Continuous covariates are standardized before the estimation. All results are based on 100 replications. Experiments are performed on a 6-core Intel Xeon CPU E5-2620 v3 2.40GHz equipped with 64GB RAM.
consider training data and testing data with sample sizes of 8,000 and 2,000,
4.1 Simulation Studies
Example 1. Here we provide details for the simulation of the motivating example introduced in
Section D The outcome is generated according to: Y, = L(X, > (5 (A1 = 1) + 225,
241 = 1).5.) + 1(X; < (11 + 191 = 1) + 25, - (2, = 1)5,} + 6, where the covariate X; Un, 1), treatment assignment A, ~ Bernoulli(0.5). and the noise 4 ~ NO, 1). For a discrete type S, S. ~ Bernoulli For a continuous type 5, S, is generated from a mixture of beta distributions, Beta(4, 1) and Bela(1, 4), with equal mixing proportions.
Example 2. We generate using the following model: Y, = {0.5 + 1(A, = 1) + exp(S.) 2,5 1), 2x X
+)x+2exxp(5)) -3.5S,
7

---

- 2
where X1, Unif(0,1), j= 1.....6... A satisfies X,, - X12 Xx3 Xx4 + X,5 xx)) and 6 ~ (O, 1). For continuous type S, S; is generated from mixture of beta distributions, Beta(4, 4, 1) and (1 (1,())))) 4), with equal mixing proportions; for a discrete type S, we consider a binary S; that satisfies (PS;: = 1 / PP(S) = ]] 2.5 + 0.8(X,1 X, + X13 + X14 + X + Xt))
Table summarizes the performance of the proposed IDRs compared the mean criterion for Example I and Example 2. The proposed RISE achieves the largest objectives and improves the value among vulnerable subjects, while maintaining comparative overall values. As for the objective, intuitively, the proposed rule is expected to achieve a larger objective than all other methods uniformly in X. We also point out that there no direct relationship between the objective among all subjects versus the objective among vulnerable subjects. For example. using the toy example with setup in Table and limiting to subjects with X S 0.5 only, S is vulnerable and assigned A by the proposed RISE. The objective among S = 1 is 13 but the objective among both S = 0 and = 12 (11 + /2, which is smaller than that among the vulnerable group. other words, by protecting the vulnerable subjects, the proposed rule may lead to increase in the outcome of the vulnerable group, and the gain may result in a higher outcome than the overall mean outcome. PT-Exp tends to show the best improvement in terms of the overall value, as the doubly robust-based estimators tend to reduce variance in value estimation. However, PT-Exp is shown to have minimal benefits for vulnerable subjects. RISE still shows the largest gain in the objective and value among vulnerable subjects among all compared methods.
Table 3: Simulation results for Example and Example 2. Standard error in parenthesis.
Example
Type $
IDR
Obj. (all)
Obj. (vulnerable)
Value (all)
Value (vulnerable)
Base
7.03 (0.03)
7.01 (0.04)
14.3 (0.15)
7.92 (0.06)
Exp
6.39 (0.03)
6.39 (0.04)
14.4 (0.05)
7.14 (0.06)
Disc.
PT- PT-Base
2.66 (0.02)
2.65 (0.02)
15.4 (0.05)
2.58 (0.02)
PT-Exp
2.62 (0.02)
2.62 (0.02)
15.5 (0.05)
2.55 (0.02)
RISE
12.0 (0.01)
12.0 (0.01)
13.0 (0.01)
14.0 (0.01)
Base
9.12 (0.03)
9.14 (0.04)
14.5 (0.08)
8.25 (0.11)
Exp
8.75 (0.03)
8.75 (0.04)
14.6 (0.08)
7,58 (0.06)
Cont.
PT-Base
6.71 (0.03)
6.72 (0.03)
15.3
4.52 (0.02)
PT--xpp
6.68 (0.02)
6.67 (0.02)
15.4 (0.05)
4.47 (0.02)
RISE
12.2 (0.02)
12.2 (0.03)
13.0 (0.01)
13.7 (0.01)
Base
7.79 (0.02)
8.66 (0.03)
19.4 (0.04)
11.4 (0.06)
Exp
9.12 (0.03)
10.1 (0.03)
19.5 (0.04)
14.4 (0.05)
Disc.
PT-Base
7.19 (0.03)
7.77 (0.03)
19.0 (0.05)
9.71 (0.05)
PT-Exp
8.30 (0.02)
9.03 (0.03)
19.1 (0.14)
12.2 (0.05)
RISH
13.5 (0.01)
14.0 (0.01)
17.4 (0.02)
22.1 (0.02)
2
Base
9.89 (0.02)
9.87 (0.03)
17.6 (0.02)
9.09 (0.04)
Exp
11.1 (0.02)
11.1 (0.02)
17.8 (0.02)
12.2 (0.04)
Cunl.
PT-Base
9.30 (0.02)
9,29 (0.03)
18.0 (0.03)
7.61 (0.04)
PT-Exp
9.41 (0.02)
9.41 (0.02)
18.1 (0.02)
7.92 (0.04)
RISE
14.1 (0.01)
14.2 (0.02)
17.0 (0.01)
20.3 (0.03)
In the appendix, we consider continuous S for different quantile criteria = 0.1 and 0.5 to test the robustness of RISE. Results show that when is small, there more strength the proposed method, as the algorithm aims to improve the worst-outcome scenarios. The proposed RISE has the largest gain in objective and value among vulnerable subjects when r 0. and has similar performance as the compared approaches when is 0.5. We also consider scenario where S is not involved in the data generation of Y, i.c., Assumption De is simplified as (-) 1 A|X. The estimated objective and value function are similar across all compared approaches, which indicates the robustness of RISE. Finally, we study the performances of our method when Assumption Ip is nearly violated or Assumption Te is violated. Similar patterns have been observed that the proposed RISE achieves the largest objectives and improves the value among vulnerable subjects, while maintaining comparable overall values. The details can be found in Appendix D

---

4.2 Real-data Applications
We present three real-data examples to showcase the robust performance of RISE. These applications
consider either fairness or safety in the context of policy (30 and healthcare 19 59 variables commonly exist.
where sensitive
Fairness in a job training program. To illustrate the implication of the proposed method from faimees perspective, we consider the National Supported Work (NSW) program (30] for improving personalized recommendations of a job training program on increasing incomes. This program intended to provide a 6 to 18-month training for individuals in face of economical and social stress such as former drug addicts and juvenile delinquents. The original experimental dataset consists of 185 individuals who received the job training program (A = 1) and 260 individuals who did not (A -1). The baseline covariates are age, years of schooling. race (1 African Americans or Hispanics, 0 == others), married (1 = yes, 0 no), high school diploma (1 = yes. 0 = no), carning in 1974, and earning in 1975. The outcome variable the earning in 1978. In the exploratory analysis using causal forest [7]] random forest-based method for causal inference, we observe that age may play an important role in the causal effect of the job training program on the long-term post-market earning. In this application example we use age as the sensitive variable S and other baseline covariates as X. The earnings in years 1974, 1975, and 1978 are transformed by taking the logarithm of the earning plus one.
To illustrate the implication of the proposed method from a safety perspective when there is delayed information, we consider the ACTG175 dataset among HIV positive patients [9]] The original study considers a total of 2.139 patients who were randomly assigned into four treatment groups. In this data application, we focus on finding the optimal IDRs between two treatments: zidovudine combined with didanosine (A 11) and zidovudine combined with zalcitabine (A = 1). The total number of patients receiving these two treatments is 1,046. The baseline covariates we consider are age, weight, CD4 T-cell amount at baseline, hemophilia (1 = yes, O = no), homosexual activity (1 = yes, 0 = no). Karnofsky score, history of intravenous drug use (1 = yes, 0 no), gender (1 = male, 0 == female), CD8 T-cell amount at baseline, race (1 = Cau-Caaa,,,,n, 0 = Caucasian), number of days of previously received antiretroviral therapy, use of zidovudine in the 30 days prior to treatment initiation (1 = yes. 0 = no), and symptomatic indicator (1 = symptomatic, 0 asymptomatic). The outcome variable is the CD4 T-cell amount at 96 + 5 weeks from the baseline. We consider CD8 T-cell amount at baseline as the sensitive variable. The response of CD8 T-cell among HIV positive patients has not been fully understood (7] Clinically, it is plausible that only CD4 is measured in clinical visits where treatments are based on, hence CD8 might not be measured and not used in decision making. As our exploratory analysis using causal forest shows, CD8 T-cell amount may play an important part in the treatment effect of the outcome.
Improvement of HIV treatment.
Safe resuscitation for patients with sepsis. For this application, we apply the proposed method to treating sepsis, a life-threatening disease. This application intends to provide an example to apply our method with multiple categorical sensitive variables in the scenario where there is missing yet important information at the time of decision making. We apply the proposed method to a sepsis study from the University of Pittsburgh Medical Center (UPMC). The original study cohort includes 30,687 patients with Sepsis-3 [59] within 6 hours of hospital arrival from 14 UPMC hospitals between 2013 and 2017. For our data analysis, sepsis onset, which includes patient demographics of age, gender (1 = male, 0 female), race (1 = Caucasian, 0= others), and weight, and vital signs of usage of mechanical ventilation (1 = yes, 0= no), respiratory rate, temperature, intravenous fluids (1 = yes, 0= no), Glasgow Coma Scale score, platelets, blood urea nitrogen, white blood cell counts, glucose, creatinine. We consider two sensitive variables, lactate and Sequential Organ Failure Assessment (SOFA) score 4 hours before sepsis onset. Lactate and SOFA score have been two important indicators of sepsis severity [2 26 60 62]
consider X to be baseline patient characteristics 4 hours before
Different from the baseline patient demographics or common vital signs that are typically obtained at the admission of patients, SOFA score combines performance of several organ systems in the body [59]. which requires additional calculation and cannot be obtained directly. Lactate labs measures the level of lactic acid in the blood ( and are less common in routine examination, which could be delayed in ordering. Hence, their measurements are obtained retrospectively after treatment decisions have been made and are not available at times of decision. We dichotomize lactate level at clinically meaningful value of 2 mmol/I. (60). and SOFA score at value of 6 for analysis (70] 18] The treatment option whether the patient took any vasopressors during the first 24 hours after sepsis onset. The outcome is hospital survival (Y 1) or death (Y = 0). The analysis cohort contains 6,539 patients

---

in total. We are interested in making decision about whether to treat patients with vasopressors in the first 24 hours after sepsis onset given the measurements of lactate and SOFA are not available at the time of decision making. Additional rationale and background on this example are provided in Appendix F.2
Table 4: Estimated objective and value of different IDRs for the three data applications. Standard
error in parenthesis. The outcome of each study is italicized.
Dataset
IDR
Obj. (all)
Obj. (vulnerable)
Value (all)
Value (vulnerable)
Base
5.26 (0.04)
5.28 (0.05)
6.32 (0.05)
6.33 (0.07)
Exp
(0.04)
5.24 (0.05)
6.37 (0.05)
6.37 (0.07)
NSW log(income+1)
PT-Base
4,97 (0.04)
5.08 (0.06)
6.40 (0.03)
6.38 (0.05)
PT-Exp
5,03 (0.04)
5.11 (0.0S)
6.43 (0.03)
6,40 (0.05)
RISE
5,43 (0.04)
5,44 (0.04)
6.42 (0.04)
6.42 (0.06)
Base
336.9 (1.65)
338.1 (2.23)
353.5 (1.86)
357.5 (2.24)
Exp
337.5 (1.65)
338.9 (1.80)
355.9 (1.95)
359.1 (2.21)
ACTG175
PT Base
299.7 (1.01)
299.5 (1.91)
356.9 (1.72)
350.7 (2.54)
CD4 T-cell amount
PT-Exp
300.1 (0.99)
299.9 (1.83)
357.1 (1.55)
352.7 (2.61)
RISE
351.5 (1.67)
351.2 (1.80)
351.8 (1.88)
363.1 (2.19)
Base
0.803 (0.001)
0.822 (0.001)
0.965 (0.001)
0,905 (0.002)
Exp
0.803 (0.001)
0.822 (0.002)
0.966 (0.001)
0,908 (0.002)
Sepsis survival rate
PT-Base
0.758 (0.001)
0.771 (0.002)
0.981 (0.001)
0.848 (0.003)
PT-Exp
0.758 (0.001)
0.772 (0.002)
0.984 (0.001)
0.875 (0.003)
RISE
0.836 (0.001)
0.833 (0.001)
0.972 (0.001)
0.923 (0.002)
Results. Table 4 presents the performance of various IDRs on the three applications. As expected, RISE has the largest objective as well as value among vulnerable subjects. The patterns are similar to that in Section 4.1 We apply the Shapley additive explanations (SHAP) approach (33) to help visual- ize and interpret the important covariates in the decision rules given by RISE and Exp, respectively. in Appendix EE.3] The SHAP approach provides unified values to describe the correlation between each feature and the predicted decision rule, respectively [33]. Overall, the direction of correlations similar for RISE and Exp. but their ranking of feature importance may be different.
Discussion
We have proposed RISE, robust decision learning framework with novel quantile- infimum- optimal treatment objective intended to improve the worst-case scenarios of individuals when de- cisions with uncertainty need to be made, but with sensitive yet important information missing. Our approach can be applied to a broad range of applications, including but not limited to policy, education, healthcare, etc. For mixture of continuous and discrete sensitive variables, the estimated rule can be obtained by first taking the infimum over the discrete ones as in Section ...4 then obtaining the quantile over the continuous ones. However, challenges remain in finding the vulnerable subjects described in Section 3.2 under these settings as it may be computationally difficult to find a vulnerable of S when it is multi-diiensional. Another future work includes the extension of the current binary treatment option to a multi-treatment option. It is also worth mentioning that our work can be naturally extended to the scenario where there exist unmeasured confounders. As long as the conditional average outcome given observed covariates can be identified (via instrumental variables such
73 or negative control variables such as (50), our method can be applied.
Acknowledgements
This research was supported in part by the Competitive Medical Research Fund of the UPMC Health System and the University of Pittsburgh Center for Research Computing through the resources provided. The authors thank Jason N. Kennedy for the sepsis data preparation, and Emily B. Brant, Chung-Chou H. Chang and Gong Tang for helpful discussions and feedback.
10

---

References
[1] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. (TensorFlow): A system for (Large-Scale) machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pages 265-283, 2016.
[2] Lars W. Andersen, Julie Mackenhauer, Jonathan C. Roberts, Katherine M. Berg. Michael N. Cocchi, and Michael W. Donnino. Etiology and therapeutic approach elevated lactate levels. Mayo Clinic Proceedings, 88(10):1127-1144 2013.
[3] Susan Athey and Stefan Wager. Policy learning with observational data. Econometrica, 89(1):333-161, 2021.
[4] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning. fairml- book.org, 2019. attp::/www. Faaimmaooooooooo...... org
[5] Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H Chi. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv: 000750 2017.
[6] Alberto Bietti, Alekh Agarwal, and John Langford. A contextual bandit bake-off. Journal of
Machine Learning Research, 22(133)):-49 2021.
[7] Sushma Boppana and Paul Goepfert. Understanding the CD8 T-cell response in natural HIV control. F1000Research, 2018.
[8] Bibhas Chakraborty and Erica E Moodie. Statistical Methods for Dynamic Treatment Regimes. Springer, 2013.
[9] Bibhas Chakraborty, Susan Murphy, and Victor Strecher. Inference for non-regular parameters in optimal dynamic treatment regimes. Statistical Methods in Medical Research, 19(3):31-3333, 2010.
[10] Haoyu Chen, Wenbin Lu, Rui Song, and Pulak Ghosh. On learning and testing of counterfactual fairness through data preprocessing. arXiv preprint arXiv:2202. 12440, 2022.
[11] Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21 21(1):C1-C6, 01 2018.
[12] Alexandra Chouldechova and Aaron Roth. A snapshot of the frontiers of fairness in machine learning. Communications of the ACM, 63(5):82-88) 89, 2020.
[13] Elliot Creager, David Madras, Jörn-Henrik Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, and Richard Zemel. Flexibly fair representation learning by disentanglement. In International Conference on Machine Learning, pages 1436-1445. PMLR, 2019.
[14] Jiewen Du, Xin Yan, Zhihong Liu, Lu Cui, Peng Ding, Xiaoqing Tan, Xiuming Li, Huihao Zhou, Qiong Gu, and Jun Xu. cBinderDB: a covalent binding agent database. Bioinformatics, 33(8):1258 1260, 2017.
[15] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pages 214-226, 2012.
[16] Ethan X. Fang, Zhaoran Wang, and Lan Wang. Faimess-orientt learning for optimal individu- alized treatment rules. Journal of the American Statistical Association, 0(0):1114, 2022.
[17] Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkata- subramanian. Certifying and removing disparate impact. In Proceedings of the 2 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 259-268.
2015.

---

[18] Flavio Lopes Ferreira, Daliana Peres Bota, Annette Bross, Christian Mélot, and Jean-Louis Vincent. Serial evaluation of the sofa score to predict outcome critically ill patients. The Journal of the American Medical Association, 286( (1 (366(1))177))) 1):1754-17558 2001.
[19] Scott M Hammer, David Katzenstein, Michael D Hughes, Holly Gundacker, Robert Schooley, Richard H Haubrich, W Keith Henry, Michael M Lederman, John P Phair, Manette Niu, et al. A trial comparing nucleoside monotherapy with combination therapy in hiv-infected adults with cd4 cell counts from 200 to 500 per cubic millimeter. New England Journal of Medicine, (35(15):1081-100)
1996.
[20] Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographies in repeated loss minimization. In International Conference on Machine Learning, pages 1929-1938. PMLR, 2018.
[21] Michael Howell, Michael Donnino, Peter Clardy, Daniel Talmor, and Nathan Shapiro. Occult hypoperfusion and mortality in patients with suspected infection. Intensive Care Medicine, 13(11):1822-189) 2007.
[22] Matthew Joseph, Michael Kearn,, Jamie H Morgenstern, and Aaron Roth. Fairness in learning: Classic and contextual bandits. Advances in Neural Information Processing Systems, 29, 2016.
[23] Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems, 33(1):1-33, 2012.
[24] Michael R Kosorok and Erica EM Moodie. Adaptive treatment strategies in practice: planning trials and analyzing data for personalized medicine. SIAM, 2015.
[25] Filippos Koutroumpakis, Anna Evans Phillips, Dhiraj Yadav, Jorge D) Machicado, Maaz Ahsan, Claudia Ramos Rivers, Xiaoqing Tan, Mare Schwartz, Siobhan Proksell, Elyse Johnston, et al. Serum igg4 subclass deficiency defines distinct, commonly encountered, severe inflammatory bowel disease subtype. Inflammatory Bowel Diseases, 855-88855-8 2021.
[26] Uma Krishna, Suresh P Joshi, and Mukesh Modh. An evaluation of serial blood lactate measurement as an early predictor of shock and its outcome patients of trauma or sepsis. Indian Journal of Critical Care Medicine, 13(2):66, 2009.
[27] Sören R Künzel, Jasjeet S Sekhon, Peter Bickel, and Bin Yu. Metalearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the National Academy of Sciences, (16-:44556-1655,, 2019.
[28] Eric B Laber, Daniel J Lizotte, Min Qian, William E Pelham, and Susan A Murphy. Dynamic treatment regimes: Technical challenges and applications. Electronic Journal of Statistics, 8(1):1225-1272, 2014.
[29] Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographics through adversarially reweighted learning. Advances in Neurul Information Processing Systems, 33:728-740, 2020.
[30] Robert J LaLonde. Evaluating the econometric evaluations of training programs with experi- mental data. The American Economic Review, pages 604-620, 1986.
[31] Lihong Li, Wei Chu, John Langford, and Xuanhui Wang. Unbiased offline evaluation of contextullbbandit-a
news article recommendation algorithms. In Proceedings of the fourth ACM international conference on Web search and data mining, pages 297-306, 2011.
[32] Gang Liu, Xiaoqing Tan, Chao Dang, Shuangquan Tan, Shihui Xing, Nianwei Huang, Kangqiang Peng, Chuanmiao Xie, Xiaoying Tang, and Jinsheng Zeng. Regional shape abnormal- ities thalamus and verbal memory impairment after subcortical infarction. Neurorehabilitatio and Neural Repair, 33(6):476-48) 2019.
[33] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 2017.
12

---

[34] Jorge D Machicado, Amir Gougol, Xiaoqing Tan, Xiaotian Gao, Pedram Paragomi. Ioannis Pothoulakis, Rupjyoti Talukdar, Rakesh Kochhar, Mahesh Goenka, Aiste Gulla, et al. Mortal- ity in acute pancreatitis with persistent organ failure is determined by the number, type, and sequence of organ systems affected. United European Gastroenterooogy Journal, 9(2):139-149) 2021.
[35] Maggie Makar, Ben Packer, Dan Moldovan, Davis Blalock, Yoni Halpern, and Alexander D' Amour. Causally motivated shortcut removal using auxiliary labels. In International Confer- ence on Artificial Intelligence and Statistics, pages 739-766. PMLR, 2022.
[36] Charles F Manski. Statistical treatment rules for heterogeneous populations. Econometrica, 72(4):1221-1246 2004.
[37] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. survey on bias and faimess in machine learning. ACM Computing Surveys (CSUR), 54(6):1-35, 2021.
[38] Erica EM Moodie, Thomas S Richardson, and David A Stephens. Demystifying optimal dynamic treatment regimes. Biometrics, 63(2):447-455, 2007.
[39] Susan A Murphy. Optimal dynamic treatment regimes. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 65(2):331-335 2003.
[40] Susan A Murphy, An experimental design for the development of adaptive treatment strategies. Statistics in Medicine, 24(10): 1455)):4555-44550 1481, 2005.
[41] Susan A Murphy, Mark van der Laan, James M Robins, and Conduct Problems Prevention Re- search Group. Marginal mean models for dynamic regimes. Journal of the American Statistical Association, 96(456): 1410-1411411 1423, 2001.
[42] Razieh Nabi, Daniel Malinsky, and Ilya Shpitser. Learning optimal fair policies. In International Conference on Machine Learning, pages 4674 4682. PMLR, 2019.
[43] Razieh Nabi and llya Shpitser. Fair inference outcomes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
[44] Jerzy Neyman. Sur les applications de la thar des probabilities aux experiences agaricales: Essay des principle. Excerpts reprinted (1990) in English. Statistical Science, 5:463-472, 1923.
[45] CA Onyemekwu, X Tan, J Gamboa, K Moghbeli, K Potter, N Prendergast, P Tiberio, F Billings, EW Ely, T Hulgan, al. Associations between plasma cell-free mitochondrial dna, delirium and coma during sepsis. In C15." SURVIVING SEPSIS" IN THE POST PANDEMIC ERA, pages A3700-A3700. American Thoracic Society, 2022.
[46] Vishakha Patil, Ganesh Ghalme, Vineet Nair, and Yadati Narahari. Achieving fairness in the stochastic multi-armed bandit problem. In Proceedings of the AAAI Conference Artificial Intelligence, pages 5379-5386, 2020.
[47] Dana Pessach and Erez Shmueli. A review on fairness in machine learning. ACM Computing Surveys (CSUR), 55(3):1-44, 2022.
[48] Krishnapriya Marangattu Prathapan, Claudia Ramos Rivers, Alyce Anderson, Filippos Koutroumpakis, Ioannis Koutroubakis, Dmitríy Babichenko, Xiaoqing Tan, Gong Tang. Mare Schwartz, Siobhan Proksell, et al. Peripheral blood cosinophilia and long-term severity in pediatric-onset inflammatory bowel disease. Inflammatory Bowel Diseases, 26012).180--1900. 2020.
[49] Zhengling Qi, Ying Cui, Yufeng Liu, and Jong-Shi Pang. Estimation of individualized decision rules based on an optimized covariate -eeeeddee equivalent of random outcomes. SIAM Journal on Optimization, 29(3):2337-2333
2019.
[50] Zhengling Qi, Rui Miao, and Xiaoke Zhang. Proximal learning for individualized treatment regimes under unmeasured confounding. arXiv preprint
2021.
13

---

[51] Zhengling Qi, Jong- Shi Pang, and Yufeng Liu. On robustness of individualized decision rules. Journal of the American Statistical Association, pages 1-15,
[52] Min Qian and Susan A Murphy, Performance guarantees for individualized treatment rules. Annals of Statistics, 39(2): 1 180, 2011.
[53] James M Robins, Miguel Angel Hernan, and Babette Brumback. Marginal structural models and causal inference in epidemiology. Epidemiology, 11(5):550-560, 2000.
[54] Donald B Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5):688, 1974.
[55] Donald B Rubin. Bayesian inference for causal effects: The role of randomization. The Annals of Statistics, pages 34-58, 1978.
[56] Donald B Rubin. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469):322-331, 2005.
[57] Yasser Sakr, Ulrich Jaschinski, Xavier Wittebole, Tamas Szakmany, Jeffrey Lipman, Silvio A Namendys-Sile Ignacio Martin-Loechess Mare Leone, Mary-Nicoleta Lupu, Jean-Louis Vincent, and ICON Investigators. Sepsis in intensive care unit patients: worldwide data from the intensive care over nations audit. Open Forum Infectious Diseases, 5(12)offy133 2018.
[58]
Prasanna Sattigeri, Samuel C Hoffman, Vijil Chenthamarakshann and Kush R Varshney. Fairness gan: Generating datasets with fairesss properties using generative adversarial network. IBM Journal of Research and Development, 63(4/5)33-1 2019.
[59] Christopher W Seymour, Vincent Liu, Theodore Iwashyna, Frank M Brunkhorst, Thomas D Rea, André Scherag, Gordon Rubenfeld, Jeremy M Kahn, Manu Shankar-Har Harki, Mervyn Singer, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (sepsis-3). The Journal of the American Medical Association, 31 (15(8):762-774, 2016.
[60] Manu Shankar-Hait Gary S Phillips, Mitchell L Levy, Christopher W Seymour, Vincent X Liu, Clifford S Deutschman, Derek C Angus, Gordon D Rubenfeld, Mervyn Singer, al. Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (sepsis-3). The Journal of the American Medical Association, 315(8):775-787, 2016.
[61] Chengchun Shi, Alin Fan, Rui Song, and Wenbin Lu. High-dimensional a-learning for optimal dynamic treatment regimes. Annals of statistics, 46(3):925, 2018
[62] Vikesh Singh, David C Whitcomb, Peter A Banks, Samer AlKaade, Michelle A Anderson, Stephen T Amann, Randall E Brand, Darwin L Conwell, Gregory A Cote, Timothy B Gardner, al. Acute pancreatitis precedes chronic pancreatitis in the majority of patients: Results from the naps2 consortium. Pancreatology, 12(8):1091-1099 2022.
[63] Jerzy Splawa-Neyman Dorota M Dabrowska, and TP Speed. On the application of probability theory to agricultural experiments. essay on principles. section 9, Statistical Science, pages 465 472, 1990.
[64] Erik Sverdrup, Ayush Kanodia, Zhengyuan Zhou, Susan Athey, and Stefan Wager. policytree: Policy learning via doubly robust empirical welfare maximization over trees Journal of Open Source Software, 5250) 2020.
[65] Xiaoqing Tan, Judah Abberbock, Priya Rastogi, and Gong Tang. Identifying principal stratum causal effects conditional on post-treatment intermediate response. In Conference on Causal Learning and Reasoning, pages 734-753. PMLR, 2022.
[66] Xiaoqing Tan, Chung-Chop H. Chang, Ling Zhou, and Lu Tang. A tree-based model averaging approach for personalized treatment effect estimation from heterogeneous data sources. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 21013-21036 PMLR, 17-23 Jul 2022.
14

---

[67] Xiaoqing Tan, Christopher A Ross, Michaeli Miller, and Xiaoying Tang. Changepoint analysis of putamen and thalamus subregions in premanifest huntington 's disease. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 531-535. IEER, 2018
[68] Xiaoqing Tan, Shu Yang. Wenyu Ye, Douglas E Faries, Ilya Lipkovich, and Zbigniew Kadziola. When doubly robust methods meet machine learning for estimating treatment effects from real-world data: A comparative study. arXiv preprint arXiv:2204. 10969, 2022.
[69] Kush R Varshney. Engineering safety in machine learning. In 2016 Information Theory and Applications Workshop (ITA), pages 1-5, IEEE, 2016.
[70] J-L Vincent, Rui Moreno, Jukka Takala, Sheila Willatts, Arnaldo De Mendonça, Hajo Bruining. CK Reinhart, PeterM Suter, and Lambertius G Thijs. The SOFA (sepsis-related organ failure assessment) score to describe organ dysfunctionnfaallure On behalf of the working group on sepsis-related problems of the european society of intensive care medicine. Intensive Care Medicine, 22(7):707-710, 1996.
[71] Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 1136623):228--1444 2018.
[72] Lan Wang, Yu Zhou, Rui Song, and Ben Sherwood. Quantile-optimal treatment regimes. Journal of the American Statistical Association, 1 12455))::243-1253 2018.
[73] Linbo Wang and Fric Tchetgen Tchetgen. Bounded, efficient and multiply robust estimation of average treatment effects using instrumental variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80(3):531-550, 2018.
[74] Yuanjia Wang. Haoda Fu, and Donglin Zeng. Learning optimal personalized treatment rules in consideration of benefit and risk: with an application to treating type diabetes patients with insulin therapies. Journal of the American Statistical Association, 113(521):1-13, 2011. 2018.
[75] Christopher JCH Watkins and Peter Dayan. Q-learning. Machine Learning, 8(3):279-292, 1992.
[76] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fairness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics, pages 962-970. PMLR, 2017.
[77] Baqun Zhang, Anastasios A Tsiatis, Eric B Laber, and Marie Davidian. A robust method for estimating optimal treatment regimes. Biometrics, 6814)1000- 2012.
[78] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335-340, 2018
[79] Junzhe Zhang and Elias Bareinboim. Fairness decision- decision-making- the causal explanation formula. In Proceedings of the AAAI Conference Artificial Intelligence, volume 32, 2018.
[80] Ying-Qi Zhao, Donglin Zeng. Eric B Laber, Rui Song, Ming Yuan, and Michael Rene Kosorok. Doubly robust learning for estimating individualized treatment with censored data. Biometrika, 102(1):151-168, 2015.
[81] Yingqi Zhao, Donglin Zeng, John Rush, and Michael R Kosorok. Estimating individual- ized treatment rules using outcome weighted learning. Journal of the American Statistical Association, 107(499): 1076499::100-11188, 100-1118,
2012.
15


Reviews:
[{"review_id": "oIG8vwHCl7Y", "reviewer": "Reviewer_ouLu", "summary": "The authors propose a robust decision learning framework with quantile or infimum optimal treatment objective. Their idea is to improve the worst-case scenarios of individuals when decisions, with uncertainty, are required, to be made with sensitive albeit important information missing. ", "questions": "See main review in Strengths and Weaknesses.", "limitations": "See main review in Strengths and Weaknesses.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Introduction\n\n- First paragraph: I do not think this is convincing. Certainly there are sensitive variables, but a few real-world examples would help solidify your examples. They certainly crop up in the media from time to time, but which organisation is actively pursuing this design philosophy in their models? Please give real examples.\n- L31: you say “The main question of interest is whether the learned IDR could yield similar outcomes across all realizations of the sensitive variables.” -  why is this the main question? I do not follow the logic as to why focus is on this.\n- Table 1: you’ll need to put some work into re-jigging this table. The expectations are the coloured (and black) entries but you’ve but the expectation column-header over the action-space column. This is confusing. Further, the whole example is somewhat confusing. Attack your example with the assumption that your reader is not an expert in your field and so start from first-principles, be verbose, and explain like you would to a novice.\n\n# Related work\n\n- You use a version of this comment a few times in this section: “See [your listed references] and references therein for a comprehensive review”. That is not helpful. Give us a summary of what those sources say rather than divert your reader to another suite of papers. You are meant to give us a micro-review herein as this is the related work section, that you have chosen to include (you don’t have to). Now you are just giving us a more reading to do. This is a conference paper, not a journal paper, so you are forced to compress and combine. Right now what you are doing is not helpful to our (the reader’s) understanding of the state-of-the-art in this area.\n\n# Robust Decision Learning Framework with Sensitive Variables \n\n- Equation 2: why is $d^*$ not equal to the right-hand side but lies in the set of optimal IDRs? Does equation (2) yield one optimal IDR or multiple? If it is the former then $\\in$ should be an equality instead? If the latter, please explain further how the argmax yields multiple optimal IDRs. Algorithm 1 seems to suggest that one IDR is output by RISE.\n- You appear to have dropped $d \\in \\mathbb{D}$ at the bottom of page four, in the aromas.\n- What is the function of $n^{-1}$ in equation 3? Or rather, why the inclusion of a weighting?\n- Algorithm 1 would be more elegant if you looped over n rather than saying “for every i”. \n\n# Numerical studies\n\n- Line 250: it would appear that you’re missing the formatting for the indicator function in your estimator for $V(d)$.\n- It would be helpful, at least for the synthetic experiments, to visualise the underlying causal structure and if possible, how it fits in the IDR - the causal diagram is a key-part of causal inference so it would be interesting to see how you graphically include it (if possible).\n- Line 309: what on earth is a “causal forest” your reference title, [45], says ‘random forest’ but you must be referring to something else, hence what is a causal forest? Is there a reference?\n- Line 328: again causal forest - reference?\n- Line 341: explain why “lactate and Sequential Organ Failure Assessment (SOFA) score 4 hours before sepsis onset” are sensitive variables?\n\n# Discussion\n\n- Line 372, typo: “find” —> “finding”", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "mLREF8Jlg4h", "reviewer": "Reviewer_6nvm", "summary": "This paper is concerned with identifying the best decision option for individuals in the presence of sensitive variables---variables that are relevant to the decisions but are not available at the time the decision is made---at inference time. Existing decision-making frameworks may simply ignore sensitive variables or seek to optimize the mean utility overall all the individuals. Such solutions might lead to bias decision-making or unfair outcome toward different subgroups in the population. To resolve these issues, the paper takes a robust approach that optimizes with respect to the best worst-case scenarios. It does so by solving a weighted classification problem. Experiments on both synthetic data and three real-world datasets are conducted to demonstrate the performance of the proposed method.", "questions": "* Can the authors please clarity what is the meaning of \"individualized\"? For example, in equation 2, there is an expectation over $X$, which seems to indicate the average utility rather than the individualized utility?\n* Related to this, in line 174-175, the author also mentioned that the decision rule is only related to $X$. Does that mean the decision rule is not related to $S$? If so, it is a bit counter-intuitive that it is still necessary to factor in $S$ in decision-making.", "limitations": "The paper discussed limitations. It would also be desirable to discuss whether it is possible to further relax some of the assumptions that the paper is based on (e.g. unconfoundedness).", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The paper is concerned with an interesting and important problem in decision making.\n* The proposed approach that optimizes with respect to the best worst-case scenarios makes sense, and the paper also offers a conversion of the problem into solving weighted classification problems.\n* The paper is clearly written overall with intuitive explanations and examples provided.\n\nWeakness:\n* While the paper offers an empirical evaluation of the proposed method over both synthetic and real-world datasets. It is not clear to me whether the competing approaches that the proposed method compared to are competitive, representative, and exhaustive.\n* The method also operates under the unconfoundedness assumption, which might not be easy to satisfy in practice. It will be desirable to carry out further synthetic experiments to understand the robustness of the proposed approach when various assumptions of the proposed approach is violated.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Q6ZA-CHz042", "reviewer": "Reviewer_Git7", "summary": "This paper proposes a method for robust decision-making when there is some important attribute that we have at training time, but not test time. Using a potential outcomes framework, they discuss an robust objective which aims to output good decisions for a range of values of the unavailable attribute. Studies with simulated and real data support that this method improves on \"vulnerable\" subjects.", "questions": "- seems fairly close in concept to Makar et al. \"Causally Motivated Shortcut Removal Using Auxiliary Labels\". Would be good to see some engagement with that work and comparison to it in terms of what each paper is aiming for (at least in the rebuttal)\n- language of \"sensitive\" and \"vulnerable\" don't align that closely IMO with the algorithm proposed - I'd either target that more specifically in experiments or change the language\n- how is the vulnerable group calculated in Table 4, and why is this a useful group to look at for evaluating our algorithms?\n- what if we don't observe many values of S for regions of X? Are we relying mostly on the extrapolation ability of our Y-hat model? Could this create problems?", "limitations": "no suggestions", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- This is a useful idea and not one that I think I've seen directly proposed before\n- pretty clear exposition throughout section 3 of the idea\n- I appreciate the datasets in section 4.2 - seems like the authors used pretty interesting, challenging data for this evaluation\n\nWeaknesses:\n- language around \"sensitive\" attribute seems a little general. I understand why it's used in some sense, but really it's just any attribute we won't have access to at deployment time - as shown in section 4, this could be a whole range of things which are not really aligned which what we consider \"sensitive\" information\n- similarly, I would say that the \"vulnerable\" language doesn't really align with what I would consider a vulnerable population. Rather, it's a post-hoc definition of who the standard mean-outcome model doesn't work for\n-for this reason, I find the results in Table 4 a little bit hard to interpret. The paper post-hoc defines who is in our vulnerable group as exactly those users that the baseline didn't work for, and now shows that the proposed method improves on them. This would be more convincing if it was a coherent group that we saw improvement on, as we could just as easily construct such a group with the reverse property (that the baseline performs better on). Unless I'm misunderstanding this!\n- unclear about how estimation works of the quantile objective when X and S are correlated - what if we don't observe many values of S for   regions of X? Are we relying mostly on the extrapolation ability of our Y-hat model?\n-it's a little odd that in Table 4 RISE is best on the whole data, makes me think the results may be fairly noisy\n- re: novelty, there is some related work I mention in the \"Questions\" section which may make the statement \"we are among the first to propose a robust-type fairness criterion under causal inference\" a little less true - although depends on exactly what those terms mean here\n\nMinor points:\n- line 5: it's not really convention in fair ML to ignore sensitive variables anymore\n- line 201: why would negativity make (3) hard to solve?\n- line 204: clarify you're minimizing over f, and its range\n- is the job training data from an RCT?\n-line 350: rational -> rationale\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GLLwezAQAJl", "reviewer": "Reviewer_XrZR", "summary": "This paper proposes RISE, a robust individualized learning framework that improves the worst-case outcomes of decision when sensitive variables are unknown in test time, thus reducing the variation across subjects. The robustness considers the uncertainty of outcome distribution due to sensitive variables and focuses on the protection to subjects in the lower tail of the outcome distribution. A corresponding learning objective is theoretically defined and is transformed to an equivalent weighted classification problem for optimization. Experiments on real-world examples show good performance of the method and its importance from fairness and safety perspectives.", "questions": "1. Should the framework of RISE be regarded as a generalized form of individualized decision rules rather than a robust one? The proposed objective considering quantile- or infimum-optimal is not a must for the derivation. In other specific scenarios, for example, the venture capital investment, the highest reward might be the concern, which can also be included in this framework.\n2. Is there any error between the simulation results shown in Table 2 and its theoretical results? The qualitative conclusion is clear, while the quantitative results are confusing. For example, does the numeral result 14.4 corresponds to a theoretical value of (30+0+5+27)/4=15.5?\n3. Does the framework in this article refer to some works in section 'Robustness in IDRs'? All these methods are summarized with only one sentence in line 109-111, which is not informative enough.\n4. The objective of RISE seems to be a protective weight increase of vulnerable groups, just like the false negative punishment in classical medical statistics to avoid serious consequences of undetected diseases. Therefore, this problem is obviously a weighted classification problem with an adaptive weight according to the objective function. Thus, the framework seems to be another paraphrase of a classic problem. Are there any other differences to be considered?\n5. Have you tried to optimize the objective function in (2) directly with a neural network? It does not seem to pose a challenge. The transformation to classification problem may only serves for traditional approaches as SVM or decision trees.\n6. Are there no other suitable baselines for your experiments? If so, the objective function of some other methods should be modified for your proposed task as comparison.", "limitations": "The discussion on limitations is sufficient.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths:\n1. This paper proposes a new perspective on measuring individualized decision with fairness or safety concerns, which can be extremely useful in many practical scenarios.\n2. The theoretical modeling and derivation are very clear and detailed.\n3. The paper is well organized and well written.\n\n### Weaknesses\n1. The related work on Robustness in individualized decision rules is insufficient, so it is hard to judge the innovation of this paper in terms of robustness learning, because similar treatment regimes as quantile-optimal have already widely used in various cases.\n2. This proposed robust learning objective is essentially a minor modification to change the expectation $E_{s|x}$ into another measure, $G_{s|x}$ , which is rather intuitive with no technical innovations.\n3. The derivation of the transformation from the raw problem to a weighted classification problem is not of great significance for practical training. As the optimization object is rather simple, it is reasonable to believe that neural networks can directly handle the original one.\n4. The comparison in the experiments is inadequate with only vanilla baselines provided. For a fair comparison, more modern methods should be included with appropriate adjustment on their objective function to fit the proposed learning objective in this paper.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept