SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Tao Shen
Yifan Cui*
A common concern when a policymaker draws causal inferences from and makes decisions based on observational data is that the measured covariates are insufficiently rich to account for all sources of confounding, i.e., the standard no confoundedness assumption fails to hold. The recently proposed proximal causal inference framework shows that proxy variables that abound in real-life scenarios can be leveraged to identify causal effects and therefore facilitate decision-making. Building upon this line of work, we propose a novel optimal individualized treatment regime based on so-called outcome and treatment confounding bridges. We then show that the value function of this new optimal treatment regime is superior to that of existing ones in the literature. Theoretical guarantees, including identification, superiority, excess value bound, and consistency of the estimated regime, are established. Furthermore, we demonstrate the proposed optimal regime via numerical experiments and a real data application.
Data-driven individualized decision-making has received tremendous attention nowadays due to its applications in healthcare, economics, marketing, etc. A large branch of work has focused on maximizing the expected utility of implementing the estimated optimal policy over a target population based on randomized controlled trials or observational studies, e.g., Athey and Wager (2021); Chakraborty and Moodie (2013); Jiang et al. (2019); Kitagawa and Tetenov (2018); Kosorok and Laber (2019); Murphy (2003); Qian and Murphy (2011); Robins (1986, 1994, 1997); Tsiatis et al. (2019); Wu et al. (2019); Zhao et al. (2012, 2019).
A critical assumption commonly made in these studies, known as unconfoundedness or exchangeability, precludes the existence of unmeasured confounding. Relying on an assumed ability of the decision-maker to accurately measure covariates relevant to a variety of confounding mechanisms present in a given observational study, causal effects, value functions, and other relevant quantities can be nonparametrically identified. However, such an assumption might not always be realistic in observational studies or randomized trials subject to non-compliance (Robins, 1994, 1997). Therefore, it is of great interest in recovering confounding mechanisms from measured covariates to infer causal effects and facilitate decision-making. A prevailing strand of work has been devoted to using instrumental variable (Angrist et al., 1996; Imbens and Angrist, 1994) as a proxy variable in dynamic treatment regimes and reinforcement learning settings (Cui, 2021; Cui and Tchetgen Tchetgen, 2021b,a; Han, 2023; Liao et al., 2021; Pu and Zhang, 2021; Qiu et al., 2021; Stensrud and Sarvet, 2022).
Recently, Tchetgen Tchetgen et al. proposed the so-called proximal causal inference framework, a formal potential outcome framework for proximal causal learning, which while explicitly acknowledging covariate measurements as imperfect proxies of confounding mechanisms, establishes causal
*Correspondence to Yifan Cui <cuiyf@zju.edu.cn>
identification in settings where exchangeability on the basis of measured covariates fails. Rather than as current practice dictates, assuming that adjusting for all measured covariates, unconfoundedness can be attained, proximal causal inference essentially requires that the investigator can correctly classify a subset of measured covariates  $L \in \mathcal{L}$  into three types: i) variables  $X \in \mathcal{X}$  that may be common causes of the treatment and outcome variables; ii) treatment-inducing confounding proxies  $Z \in \mathcal{Z}$ ; and iii) outcome-inducing confounding proxies  $W \in \mathcal{W}$ .
There is a fast-growing literature on proximal causal inference since it has been proposed (Cui et al., 2023; Dukes et al., 2023; Ghassami et al., 2023; Kompa et al., 2022; Li et al., 2023; Mastouri et al., 2021; Miao et al., 2018b; Shi et al., 2020b, 2021; Shpitser et al., 2023; Singh, 2020; Tchetgen Tchetgen et al., 2020; Ying et al., 2023, 2022 and many others). In particular, Miao et al. (2018a); Tchetgen Tchetgen et al. (2020) propose identification of causal effects through an outcome confounding bridge and Cui et al. (2023) propose identification through a treatment confounding bridge. A doubly robust estimation strategy (Chernozhukov et al., 2018; Robins et al., 1994; Rotnitzky et al., 1998; Scharfstein et al., 1999) is further proposed in Cui et al. (2023). In addition, Ghassami et al. (2022) and Kallus et al. (2021) propose a nonparametric estimation of causal effects through a min-max approach. Moreover, by adopting the proximal causal inference framework, Qi et al. (2023) consider optimal individualized treatment regimes (ITRs) estimation, Sverdrup and Cui (2023) consider learning heterogeneous treatment effects, and Bennett and Kallus (2023) consider off-policy evaluation in partially observed Markov decision processes.
In this paper, we aim to estimate optimal ITRs under the framework of proximal causal inference. We start with reviewing two in-class ITRs that map from  $\mathcal{X} \times \mathcal{W}$  to  $\mathcal{A}$  and  $\mathcal{X} \times \mathcal{Z}$  to  $\mathcal{A}$ , respectively, where  $\mathcal{A}$  denotes the binary treatment space. The identification of value function and the learning strategy for these two optimal in-class ITRs are proposed in Qi et al. (2023). In addition, Qi et al. (2023) also consider a maximum proximal learning optimal ITR that maps from  $\mathcal{X} \times \mathcal{W} \times \mathcal{Z}$  to  $\mathcal{A}$  with the ITRs being restricted to either  $\mathcal{X} \times \mathcal{W} \rightarrow \mathcal{A}$  or  $\mathcal{X} \times \mathcal{Z} \rightarrow \mathcal{A}$ . In contrast to their maximum proximal learning ITR, in this paper, we propose a brand new policy class whose ITRs map from measured covariates  $\mathcal{X} \times \mathcal{W} \times \mathcal{Z}$  to  $\mathcal{A}$ , which incorporates the predilection between these two in-class ITRs. Identification and superiority of the proposed optimal ITRs compared to existing ones are further established.
The main contributions of our work are four-fold. Firstly, by leveraging treatment and outcome confounding bridges under the recently proposed proximal causal inference framework, identification results regarding the proposed class  $\mathcal{D}_{\mathcal{Z}\mathcal{W}}^{\Pi}$  of ITRs that map  $\mathcal{X} \times \mathcal{W} \times \mathcal{Z}$  to  $\mathcal{A}$  are established. The proposed ITR class can be viewed as a generalization of existing ITR classes proposed in the literature. Secondly, an optimal subclass of  $\mathcal{D}_{\mathcal{Z}\mathcal{W}}^{\Pi}$  is further introduced. Learning optimal treatment regimes within this subclass leads to a superior value function. Thirdly, we propose a learning approach to estimating the proposed optimal ITR. Our learning pipeline begins with the estimation of confounding bridges adopting the deep neural network method proposed by Kompa et al. (2022). Then we use optimal treatment regimes proposed in Qi et al. (2023) as preliminary regimes to estimate our optimal ITR. Lastly, we establish an excess value bound for the value difference between the estimated treatment regime and existing ones in the literature, and the consistency of the estimated regime is also demonstrated.
2.1 Optimal individualized treatment regimes
We briefly introduce some conventional notation for learning optimal ITRs. Suppose  $A$  is a binary variable representing a treatment option that takes values in the treatment space  $\mathcal{A} = \{-1, 1\}$ . Let  $L \in \mathcal{L}$  be a vector of observed covariates, and  $Y$  be the outcome of interest. Let  $Y(1)$  and  $Y(-1)$  be the potential outcomes under an intervention that sets the treatment to values 1 and -1, respectively. Without loss of generality, we assume that larger values of  $Y$  are preferred.
Suppose the following standard causal assumptions hold: (1) Consistency:  $Y = Y(A)$ . That is, the observed outcome matches the potential outcome under the realized treatment. (2) Positivity:  $\mathbb{P}(A = a|L) > 0$  for  $a \in \mathcal{A}$  almost surely, i.e., both treatments are possible to be assigned.
We consider an ITR class  $\mathcal{D}$  containing ITRs that are measurable functions mapping from the covariate space  $\mathcal{L}$  onto the treatment space  $\mathcal{A}$ . For any  $d\in \mathcal{D}$ , the potential outcome under a hypothetical
intervention that assigns treatment according to  $d$  is defined as
$$
Y (d (L)) \stackrel {\triangle} {=} Y (1) \mathbb {I} \{d (L) = 1 \} + Y (- 1) \mathbb {I} \{d (L) = - 1 \},
$$
where  $\mathbb{I}\{\cdot\}$  denotes the indicator function. The value function of ITR  $d$  is defined as the expectation of the potential outcome, i.e.,
$$
V (d) \triangleq \mathbb {E} [ Y (d (L)) ].
$$
It can be easily seen that an optimal ITR can be expressed as
$$
d ^ {*} (L) = \operatorname {s i g n} \left\{\mathbb {E} (Y (1) - Y (- 1) | L) \right\}
$$
$$
d ^ {*} = \arg \max _ {d \in \mathcal {D}} \mathbb {E} [ Y (d (L)) ].
$$
There are many ways to identify optimal ITRs under different sets of assumptions. The most commonly seen assumption is the unconfoundedness:  $Y(a) \perp A|L$  for  $a = \pm 1$ , i.e., upon conditioning on  $L$ , there is no unmeasured confounder affecting both  $A$  and  $Y$ . Under this unconfoundedness assumption, the value function of a given regime  $d$  can be identified by (Qian and Murphy, 2011)
$$
V (d) = \mathbb {E} \left[ \frac {Y \mathbb {I} \{A = d (L) \}}{f (A | L)} \right],
$$
where  $f(A|L)$  denotes the propensity score (Rosenbaum and Rubin, 1983), and the optimal ITR is identified by
$$
d ^ {*} = \arg \max _ {d \in \mathcal {D}} V (d) = \arg \max _ {d \in \mathcal {D}} \mathbb {E} \left[ \frac {Y \mathbb {I} \{A = d (L) \}}{f (A | L)} \right].
$$
We refer to Qian and Murphy (2011); Zhang et al. (2012); Zhao et al. (2012) for more details of learning optimal ITRs in this unconfounded setting.
Because confounding by unmeasured factors cannot generally be ruled out with certainty in observational studies or randomized experiments subject to non-compliance, skepticism about the unconfoundedness assumption in observational studies is often warranted. To estimate optimal ITRs subject to potential unmeasured confounding, Cui and Tchetgen Tchetgen (2021b) propose instrumental variable approaches to learning optimal ITRs. Under certain instrumental variable assumptions, the optimal ITR can be identified by
$$
\underset {d \in \mathcal {D}} {\arg \max} \mathbb {E} \left[ \frac {M A Y \mathbb {I} \{A = d (L) \}}{\{\mathbb {P} (A = 1 | M = 1 , L) - \mathbb {P} (A = 1 | M = - 1 , L) \} f (M | L)} \right],
$$
where  $M$  denotes a valid binary instrumental variable. Other works including Cui (2021); Cui and Tchetgen Tchetgen (2021a); Han (2023); Pu and Zhang (2021) consider a sign or partial identification of causal effects to estimate suboptimal ITRs using instrumental variables.
2.2 Existing optimal ITRs for proximal causal inference
Another line of research in causal inference considers negative control variables as proxies to mitigate confounding bias (Kuroki and Pearl, 2014; Miao et al., 2018a; Shi et al., 2020a; Tchetgen Tchetgen, 2014). Recently, a formal potential outcome framework, namely proximal causal inference, has been developed by Tchetgen Tchetgen et al. (2020), which has attracted tremendous attention since proposed.
Following the proximal causal inference framework proposed in Tchetgen Tchetgen et al. (2020), suppose that the measured covariate  $L$  can be decomposed into three buckets  $L = (X, W, Z)$ , where  $X \in \mathcal{X}$  affects both  $A$  and  $Y$ ,  $W \in \mathcal{W}$  denotes an outcome-inducing confounding proxy that is a potential cause of the outcome which is related with the treatment only through  $(U, X)$ , and  $Z \in \mathcal{Z}$  is a treatment-inducing confounding proxy that is a potential cause of the treatment which is related with the outcome  $Y$  through  $(U, X, A)$ . We now summarize several basic assumptions of the proximal causal inference framework.
Assumption 1. We make the following assumptions:
(1) Consistency:  $Y = Y(A,Z)$ ,  $W = W(A,\overline{Z})$ .
(2) Positivity:  $\mathbb{P}(A = a\mid U,X) > 0$ $\forall a\in \mathcal{A}$
(3) Latent unconfoundedness:
$(Z,A)\perp (Y(a),W)\mid U,X,\forall a\in \mathcal{A}.$
The consistency and positivity assumptions are conventional in the causal inference literature. The latent unconfoundedness essentially states that  $Z$  cannot directly affect the outcome  $Y$ , and  $W$  is not directly affected by either  $A$  or  $Z$ . Figure 1 depicts a classical setting that satisfies Assumption 1. We refer to Shi et al. (2020b); Tchetgen Tchetgen et al. (2020) for other realistic settings for proximal causal inference.
Figure 1: A causal DAG under the proximal causal inference framework.
We first consider two in-class optimal ITRs that map from  $\mathcal{X} \times \mathcal{Z}$  to  $\mathcal{A}$  and  $\mathcal{X} \times \mathcal{W}$  to  $\mathcal{A}$ , respectively. To identify optimal ITRs that map from  $\mathcal{X} \times \mathcal{Z}$  to  $\mathcal{A}$ , we make the following assumptions.
Assumption 2. Completeness: For any  $a \in \mathcal{A}, x \in \mathcal{X}$  and square-integrable function  $g$ ,  $\mathbb{E}[g(U) \mid Z, A = a, X = x] = 0$  almost surely if and only if  $g(U) = 0$  almost surely.
Assumption 3. Existence of outcome confounding bridge: There exists an outcome confounding bridge function  $h(w, a, x)$  that solves the following equation
$$
\mathbb {E} [ Y | Z, A, X ] = \mathbb {E} [ h (W, A, X) | Z, A, X ],
$$
The completeness Assumption 2 is a technical condition central to the study of sufficiency in foundational theory of statistical inference. It essentially assumes that  $Z$  has sufficient variability with respect to the variability of  $U$ . We refer to Tchetgen Tchetgen et al. (2020) and Miao et al. (2022) for further discussions regarding the completeness condition. Assumption 3 defines a so-called inverse problem known as a Fredholm integral equation of the first kind through an outcome confounding bridge. The technical conditions for the existence of a solution to a Fredholm integral equation can be found in Kress et al. (1989).
Let  $\mathcal{D}_{\mathcal{Z}}$  be an ITR class that includes all measurable functions mapping from  $\mathcal{X} \times \mathcal{Z}$  to  $\mathcal{A}$ . As shown in Qi et al. (2023), under Assumptions 1, 2 and 3, for any  $d_z \in \mathcal{D}_{\mathcal{Z}}$ , the value function  $V(d_z)$  can be nonparametrically identified by
$$
V \left(d _ {z}\right) = \mathbb {E} [ h (W, d _ {z} (X, Z), X) ]. \tag {1}
$$
Furthermore, the in-class optimal treatment regime  $d_z^* \in \arg \max_{d_z \in \mathcal{D}_z} V(d_z)$  is given by
$$
d _ {z} ^ {*} (X, Z) = \operatorname {s i g n} \left\{\mathbb {E} \left[ h (W, 1, X) - h (W, - 1, X) | X, Z \right] \right\}.
$$
On the other hand, to identify optimal ITRs that map from  $\mathcal{X} \times \mathcal{W}$  to  $\mathcal{A}$ , we make the following assumptions.
Assumption 4. Completeness: For any  $a \in \mathcal{A}, x \in \mathcal{X}$  and square-integrable function  $g$ ,  $\mathbb{E}[g(U) \mid W, A = a, X = x] = 0$  almost surely if and only if  $g(U) = 0$  almost surely.
Assumption 5. Existence of treatment confounding bridge: There exists a treatment confounding bridge function  $q(z, a, x)$  that solves the following equation
$$
\frac {1}{\mathbb {P} (A = a | W , X)} = \mathbb {E} [ q (Z, a, X) | W, A = a, X ],
$$
Similar to Assumptions 2 and 3, Assumption 4 assumes that  $W$  has sufficient variability relative to the variability of  $U$ , and Assumption 5 defines another Fredholm integral equation of the first kind through a treatment confounding bridge  $q$ .
Let  $\mathcal{D}_{\mathcal{W}}$  be another ITR class that includes all measurable functions mapping from  $\mathcal{X} \times \mathcal{W}$  to  $\mathcal{A}$ . As shown in Qi et al. (2023), under Assumptions 1, 4 and 5, for any  $d_w \in \mathcal{D}_{\mathcal{W}}$ , the value function  $V(d_w)$  can be nonparametrically identified by
$$
V \left(d _ {w}\right) = \mathbb {E} \left[ Y q (Z, A, X) \mathbb {I} \left\{d _ {w} (X, W) = A \right\} \right]. \tag {2}
$$
The in-class optimal treatment regime  $d_w^* \in \arg \max_{d_w \in \mathcal{D}_{\mathcal{W}}} V(d_w)$  is given by
$$
d _ {w} ^ {*} (X, W) = \operatorname {s i g n} \left\{\mathbb {E} [ Y q (Z, 1, X) \mathbb {I} \{A = 1 \} - Y q (Z, - 1, X) \mathbb {I} \{A = - 1 \} | X, W ] \right\}.
$$
Moreover, Qi et al. (2023) consider the ITR class  $\mathcal{D}_{\mathcal{Z}} \cup \mathcal{D}_{\mathcal{W}}$  and propose a maximum proximal learning optimal regime based on this ITR class. For any  $d_{z \cup w} \in \mathcal{D}_{\mathcal{Z}} \cup \mathcal{D}_{\mathcal{W}}$ , under Assumptions 1-5, the value function  $V(d_{z \cup w})$  for any  $d_{z \cup w} \in \mathcal{D}_{\mathcal{Z}} \cup \mathcal{D}_{\mathcal{W}}$  can be identified by
$$
V \left(d _ {z \cup w}\right) = \mathbb {I} \left\{d _ {z \cup w} \in \mathcal {D} _ {\mathcal {Z}} \right\} \mathbb {E} [ h (W, d _ {z \cup w} (X, Z), X) ] + \mathbb {I} \left\{d _ {z \cup w} \in \mathcal {D} _ {\mathcal {W}} \right\} \mathbb {E} [ Y q (Z, A, X) \mathbb {I} \left\{d _ {z \cup w} (X, W) = A \right\} ]. \tag {3}
$$
The optimal ITR within this class is given by  $d_{z \cup w}^* \in \arg \max_{d_{z \cup w} \in \mathcal{D}_{\mathcal{Z}} \cup \mathcal{D}_{\mathcal{W}}} V(d_{z \cup w})$ , and they show that the corresponding optimal value function takes the maximum value between two optimal in-class ITRs, i.e.,
$$
V \left(d _ {z \cup w} ^ {*}\right) = \max  \left\{V \left(d _ {z} ^ {*}\right), V \left(d _ {w} ^ {*}\right) \right\}.
$$
2.3 Optimal decision-making based on two confounding bridges
As discussed in the previous section, given that neither  $\mathbb{E}[Y(a)|X,U]$  nor  $\mathbb{E}[Y(a)|X,W,Z]$  for any  $a\in \mathcal{A}$  may be identifiable under the proximal causal inference setting, one might nevertheless consider ITRs mapping from  $\mathcal{X}\times \mathcal{W}$  to  $\mathcal{A}$ , from  $\mathcal{X}\times \mathcal{Z}$  to  $\mathcal{A}$ , from  $\mathcal{X}\times \mathcal{W}\times \mathcal{Z}$  to  $\mathcal{A}$  as well as from  $\mathcal{X}$  to  $\mathcal{A}$ . Intuitively, policy-makers might want to use as much information as they can to facilitate their decision-making. Therefore, ITRs mapping from  $\mathcal{X}\times \mathcal{W}\times \mathcal{Z}$  to  $\mathcal{A}$  are of great interest if information regarding  $(X,W,Z)$  is available.
As a result, a natural question arises: is there an ITR mapping from  $\mathcal{X} \times \mathcal{W} \times \mathcal{Z}$  to  $\mathcal{A}$  which dominates existing ITRs proposed in the literature? In this section, we answer this question by proposing a novel optimal ITR and showing its superiority in terms of global welfare.
We first consider the following class of ITRs that map from  $\mathcal{X}\times \mathcal{W}\times \mathcal{Z}$  to  $\mathcal{A}$
$$
\mathcal {D} _ {\mathcal {Z W}} ^ {\Pi} \stackrel {\triangle} {=} \left\{d _ {z w} ^ {\pi}: d _ {z w} ^ {\pi} (X, W, Z) = \pi (X) d _ {z} (X, Z) + (1 - \pi (X)) d _ {w} (X, W), d _ {z} \in \mathcal {D} _ {\mathcal {Z}}, d _ {w} \in \mathcal {D} _ {\mathcal {W}}, \pi \in \Pi \right\},
$$
where  $\Pi$  is the policy class containing all measurable functions  $\pi : \mathcal{X} \to \{0,1\}$  that indicate the individualized predilection between  $d_z$  and  $d_w$ .
Remark 1. Note that  $\mathcal{D}_{\mathcal{Z}}, \mathcal{D}_{\mathcal{W}}$  and  $\mathcal{D}_{\mathcal{Z}} \cup \mathcal{D}_{\mathcal{W}}$  are subsets of  $\mathcal{D}_{\mathcal{Z}\mathcal{W}}^{\Pi}$  with a particular choice of  $\pi$ . For example,  $\mathcal{D}_{\mathcal{Z}}$  is  $\mathcal{D}_{\mathcal{Z}\mathcal{W}}^{\Pi}$  with restriction on  $\pi(X) = 1$ ;  $\mathcal{D}_{\mathcal{W}}$  is  $\mathcal{D}_{\mathcal{Z}\mathcal{W}}^{\Pi}$  with restriction on  $\pi(X) = 0$ ;  $\mathcal{D}_{\mathcal{Z}} \cup \mathcal{D}_{\mathcal{W}}$  is  $\mathcal{D}_{\mathcal{Z}\mathcal{W}}^{\Pi}$  with restriction on  $\pi(X) = 1$  or  $\pi(X) = 0$ .
In the following theorem, we demonstrate that by leveraging the treatment and outcome confounding bridge functions, we can nonparametrically identify the value function over the policy class  $\mathcal{D}_{\mathcal{Z}\mathcal{W}}^{\Pi}$ , i.e.,  $V(d_{zw}^{\pi})$  for  $d_{zw}^{\pi} \in \mathcal{D}_{\mathcal{Z}\mathcal{W}}^{\Pi}$ .
Theorem 1. Under Assumptions 1-5, for any  $d_{zw}^{\pi} \in \mathcal{D}_{\mathcal{ZW}}^{\Pi}$ , the value function  $V(d_{zw}^{\pi})$  can be nonparametrically identified by
$$
V \left(d _ {z w} ^ {\pi}\right) = \mathbb {E} \left[ \pi (X) h \left(W, d _ {z} (X, Z), X\right) + \left(1 - \pi (X)\right) Y q (Z, A, X) \mathbb {I} \left\{d _ {w} (X, W) = A \right\} \right]. \tag {4}
$$
One of the key ingredients of our constructed new policy class  $\mathcal{D}_{\mathcal{Z}\mathcal{W}}^{\Pi}$  is the choice of  $\pi (\cdot)$ . It suggests an individualized strategy for treatment decisions between the two given treatment regimes. Because we are interested in policy learning, a suitable choice of  $\pi (\cdot)$  that leads to a larger value function is more desirable. Therefore, we construct the following  $\bar{\pi} (X;d_z,d_w)$ ,
$$
\bar {\pi} (X; d _ {z}, d _ {w}) \triangleq \mathbb {I} \left\{\mathbb {E} [ h (W, d _ {z} (X, Z), X) | X ] \geq \mathbb {E} [ Y q (Z, A, X) \mathbb {I} \{d _ {w} (X, W) = A \} | X ] \right\}. \tag {5}
$$
In addition, given any  $d_z \in \mathcal{D}_{\mathcal{Z}}$  and  $d_w \in \mathcal{D}_{\mathcal{W}}$ , we define
$$
d _ {z w} ^ {\bar {\pi}} (X, W, Z) \triangleq \bar {\pi} (X; d _ {z}, d _ {w}) d _ {z} (X, Z) + (1 - \bar {\pi} (X; d _ {z}, d _ {w})) d _ {w} (X, W).
$$
We then obtain the following result which justifies the superiority of  $\bar{\pi}$ .
Theorem 2. Under Assumptions 1-5, for any  $d_z \in \mathcal{D}_{\mathcal{Z}}$  and  $d_w \in \mathcal{D}_{\mathcal{W}}$ ,
$$
V (d _ {z w} ^ {\bar {\pi}}) \geq \max  \{V (d _ {z}), V (d _ {w}) \}.
$$
Theorem 2 establishes that for the particular choice of  $\bar{\pi}$  given in (5), the value function of  $d_{zw}^{\bar{\pi}}$  is no smaller than that of  $d_z$  and  $d_w$  for any  $d_z\in \mathcal{D}_{\mathcal{Z}}$ , and  $d_w\in \mathcal{D}_{\mathcal{W}}$ . Consequently, Theorem 2 holds for  $d_z^*$  and  $d_w^*$ . Hence, we propose the following optimal ITR  $d_{zw}^{\bar{\pi} *}$ ,
$$
d _ {z w} ^ {\bar {\pi} ^ {*}} (X, W, Z) \stackrel {\triangle} {=} \bar {\pi} (X; d _ {z} ^ {*}, d _ {w} ^ {*}) d _ {z} ^ {*} (X, Z) + (1 - \bar {\pi} (X; d _ {z} ^ {*}, d _ {w} ^ {*})) d _ {w} ^ {*} (X, W),
$$
and we have the following corollary.
Corollary 1. Under Assumptions 1-5, we have that
$$
V \left(d _ {z w} ^ {\bar {\pi} *}\right) \geq \max  \left\{V \left(d _ {z} ^ {*}\right), V \left(d _ {w} ^ {*}\right), V \left(d _ {z \cup w} ^ {*}\right) \right\}.
$$
Corollary 1 essentially states that the value of  $d_{zw}^{\bar{\pi}*}$  dominates that of  $d_z^*$ ,  $d_w^*$ , as well as  $d_{z\cup w}^*$ . Moreover, the proposition below demonstrates the optimality of  $d_{zw}^{\bar{\pi}*}$  within the proposed class.
Proposition 1. Under Assumptions 1-5, we have that
$$
d_{z^{w}}^{\overline{\pi}^{*}}\in \arg \max_{d_{z^{w}}^{\pi}\in \mathcal{D}_{\mathcal{Z}\mathcal{W}}^{\Pi}}V(d_{z^{w}}^{\pi}).
$$
Therefore,  $d_{zw}^{\bar{\pi}*}$  is an optimal ITR of policymakers' interest.
3 Statistical Learning and Optimization
3.1 Estimation of the optimal ITR  $d_{zw}^{\bar{\pi}*}$