# Heterogeneous Treatment Effect with Trained Kernels of the Nadaraya-Watson Regression
## 4 The TNW-CATE Description
It is important to point out that the neural networks shown in Figures 4 and 5 are not trained on datasets \(\mathcal{C}\) and \(\mathcal{T}\). These datasets are used as testing examples. This is an important difference between the proposed approach and other classification or regression models. Let us return to the case \(\theta=0\) when only the control network is trained on the controls without the treatments. This case is interesting because it clearly demonstrates the transfer learning model when domains of source and target data are the same, but the tasks are different. Indeed, we train the kernels of the Nadaraya-Watson regression on the controls under the assumption that the domains of the controls and treatments are the same. Kernels learn the feature vector location. Actually, kernels are trained on controls by using outcomes \(y_{i}\). However, nothing prevents us from using the same kernels with different outcomes \(h_{i}\) if the structures of the feature vectors in the control and treatment groups are similar. We often use the same standard kernel with the same parameters, for example, the Gaussian one with the temperature parameter in machine learning tasks. The proposed method does the same, but with a more complex kernel. ## 5 Numerical Experiments

In this section, we provide simulation experiments evaluating the performance of metamodels for CATE estimation. In particular, we compare the T-learner, the S-learner, the X-learner, and the proposed method in several simulation studies. Numerical experiments are based on a random generation of the control and treatment data in accordance with different predefined controls and treatment outcome functions because the true CATEs are unknown due to the fundamental problem of the causal inference for real data [13]. ### General Parameters of Experiments

#### 5.1.1 CATE Estimators for Comparison

The following models are used for their comparison with TNW-CATE. 1. The T-learner [25] is a simple procedure based on estimating the control \(g_{0}(\mathbf{x})\) and treatment \(g_{1}(\mathbf{z})\) outcome functions by applying a regression algorithm, for example, the random forest [96]. The CATE in this case is defined as the difference \(g_{1}(\mathbf{z})-g_{0}(\mathbf{x})\). 2. The S-learner was proposed in [25] to overcome difficulties and disadvantages of the T-learner. The treatment assignment indicator \(T_{i}\) in the S-learner is included as an additional feature to the feature vector \(\mathbf{x}\). The corresponding training set in this case is modified as \(\mathcal{D}^{*}=\{(\mathbf{x}_{1}^{*},y_{1}),\ldots,(\mathbf{x}_{c+t}^{*},y_{ c+t})\}\), where \(\mathbf{x}_{i}^{*}=(\mathbf{x}_{i},T_{i})\in\mathbb{R}^{d+1}\) if \(T_{i}=0\), \(i=1,\ldots,c\), and \(\mathbf{x}_{c+i}^{*}=(\mathbf{z}_{i},T_{i})\in\mathbb{R}^{d+1}\) if \(T_{i}=1\), \(i=1,\ldots,t\). Then the outcome function \(g(\mathbf{x},T)\) is estimated by using the training set \(\mathcal{D}^{*}\). The CATE is determined in this case as \(\tau(\mathbf{x})=g(\mathbf{x},1)-g(\mathbf{x},0)\). 3. The X-learner [25] is based on computing the so-called imputed treatment effects and is represented in the following three steps. First, the outcome functions \(g_{0}(\mathbf{x})\) and \(g_{1}(\mathbf{z})\)are estimated using a regression algorithm, for example, the random forest. Second, the imputed treatment effects are computed as follows:

\[\mathbf{a}_{i}^{(r)} =\left(\mathbf{x}_{1}^{(r)},\ldots,\mathbf{x}_{n}^{(r)}, \mathbf{x}_{i,j}y_{1}^{(r)},\ldots,y_{n}^{(r)},y_{i}\right),\]6 \[\mathbf{a}_{i}^{(r)} =\left(\mathbf{x}_{1}^{(r)},\ldots,\mathbf{x}_{n}^{(r)}, \mathbf{x}_{i,j}y_{1}^{(r)},\ldots,y_{n}^{(r)},y_{i}\right),\]7

Third, two regression functions \(\tau_{1}(\mathbf{z})\) and \(\tau_{0}(\mathbf{x})\) are estimated for imputed treatment effects \(D_{1}(\mathbf{z}_{i})\) and \(D_{0}(\mathbf{x}_{i})\), respectively.