# Heterogeneous Treatment Effect with Trained Kernels of the Nadaraya-Watson Regression
## 3 A Formal Problem Statement  
Suppose that the control group of patients is represented as a set of \(c\) examples of the form \(\mathcal{C}=\{(\mathbf{x}_{1},y_{1}),\ldots,(\mathbf{x}_{c},y_{c})\}\), where \(\mathbf{x}_{i}=(x_{i1},\ldots,x_{id})\in\mathbb{R}^{d}\) is the \(d\)-dimensional feature vector for the \(i\)-th patient from the control group; \(y_{i}\in\mathbb{R}\) is the \(i\)-th observed outcome, for example, time to death of the \(i\)-th patient from the control group or the blood sugar level of this patient. The similar notations can be introduced for the treatment group containing \(t\) patients, namely, \(\mathcal{T}=\{(\mathbf{z}_{1},h_{1}),\ldots,(\mathbf{z}_{t},h_{t})\}\). Here \(\mathbf{z}_{i}=(z_{i1},\ldots,z_{id})\in\mathbb{R}^{d}\) and \(h_{i}\in\mathbb{R}\) are the feature vector and the outcome of the \(i\)-th patient from the treatment group, respectively. We will also use the notation of the treatment assignment indicator \(T_{i}\in\{0,1\}\) where \(T_{i}=0\) (\(T_{i}=1\)) corresponds to the control (treatment) group.  
Let \(Y\) and \(H\) denote the potential outcomes of a patient if \(T=0\) and \(T=1\) for the patient, respectively. Let \(\mathbf{X}\) be the random feature vector from \(\mathbb{R}^{d}\). The treatment effect for a new patient with the feature vector \(\mathbf{x}\), which shows how the treatment is useful and efficient, is estimated by the Individual Treatment Effect (ITE) defined as \(H-Y\). Since the ITE cannot be observed, then the treatment effect is estimated by means of CATE which is defined as the expected difference between two potential outcomes as follows [94]:  
\[\tau(\mathbf{x})=\mathbb{E}[H-Y\mid\mathbf{X}=\mathbf{x}]. \tag{3}\]  
The fundamental problem of computing CATE is that, for each patient in the training dataset, we can observe only one of outcomes \(y\) or \(h\). To overcome this problem, the important assumption of unconfoundedness [95] is used to allow the untreated units to be used to construct an unbiased counterfactual for the treatment group [76]. According to the assumption of unconfoundedness, the treatment assignment \(T\) is independent of the potential outcomes for \(Y\) or H conditional on \(\mathbf{x}=\mathbf{z}\), respectively, which can be written as  
\[T\perp\{Y,H\}\mid\mathbf{x}. \tag{4}\]Another assumption called the overlap assumption regards the joint distribution of treatments and covariates. According to this assumption, there is a positive probability of being both treated and untreated for each value of \(\mathbf{x}\). It is of the form:  
\[0<\Pr\{T=1\mid\mathbf{x}\}<1. \tag{5}\]  
If we are to accept these assumptions, then CATE can be represented as follows:  
\[\tau(\mathbf{x})=\mathbb{E}[H\mid\mathbf{x}=\mathbf{x}]-\mathbb{E}[Y\mid \mathbf{x}=\mathbf{z}]. \tag{6}\]  
The motivation behind unconfoundedness is that nearby observations in the feature space can be treated as having come from a randomized experiment [10].  
If we suppose that outcomes of patients from the control and treatment groups are expressed through the functions \(g_{0}\) and \(g_{1}\) of the feature vectors \(X\), then the corresponding regression functions can be written as  
\[y=g_{0}(\mathbf{x})+\varepsilon,\;\mathbf{x}\in\mathcal{C}, \tag{7}\]  
\[h=g_{1}(\mathbf{z})+\varepsilon,\;\mathbf{z}\in\mathcal{T}. \tag{8}\]  
Here, \(\varepsilon\) is a Gaussian noise variable such that \(\mathbb{E}[\varepsilon]=0\). Hence, there holds under condition \(\mathbf{x}=\mathbf{z}\)  
\[\tau(\mathbf{x})=g_{1}(\mathbf{x})-g_{0}(\mathbf{x}). \tag{9}\]  
An example illustrating sets of controls (small circles), treatments (small triangles), and the corresponding unknown function \(g_{0}\) and \(g_{1}\) are shown in Figure 1.