# Heterogeneous Treatment Effect with Trained Kernels of the Nadaraya-Watson Regression
## 4 The TNW-CATE Description
It is important to note that the models with the Gaussian kernel (T-NW, S-NW, and X-NW) provide the worst results. This example illustrates a case when the data structure is too complex to model by means of the standard Gaussian kernel. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**Functions**} \\ \hline
**Model** & **Spiral** & **Logarithmic** & **Power** & **Indicator** \\ \hline T-NW & \(3.806\pm 0.734\) & \(0.377\pm 0.163\) & \(1.722\pm 0.482\) & \(0.650\pm 0.208\) \\ S-NW & \(3.629\pm 0.605\) & \(0.341\pm 0.057\) & \(1.719\pm 0.287\) & \(0.549\pm 0.092\) \\ X-NW & \(3.279\pm 0.547\) & \(0.542\pm 0.090\) & \(0.632\pm 0.105\) & \(0.353\pm 0.059\) \\ T-RF & \(2.278\pm 0.380\) & \(0.051\pm 0.019\) & \(2.743\pm 0.457\) & \(0.337\pm 0.056\) \\ S-RF & \(2.575\pm 0.409\) & \(0.060\pm 0.018\) & \(0.839\pm 0.140\) & \(0.434\pm 0.072\) \\ X-RF & \(1.385\pm 0.231\) & \(0.202\pm 0.074\) & \(0.805\pm 0.134\) & \(\mathbf{0.061}\pm 0.010\) \\ TNW-CATE & \(\mathbf{0.232}\pm 0.069\) & \(\mathbf{0.026}\pm 0.008\) & \(\mathbf{0.353}\pm 0.089\) & \(0.257\pm 0.043\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The best MSE values and standard deviations of CATE for different models and by different generating functions. After analyzing the obtained numerical results, we can summarize the following:

1. The results significantly depend on the structure of the control and treatment data. The most complex structure among the considered ones is produced by the spiral function, and we observe that TNW-CATE outperforms the results in comparison with other models, especially when the number of controls is rather large. The large number of controls prevents the neural network from overfitting. The worst results provided by TNW-CATE and models based on the Gaussian kernels (T-NW, S-NW, and X-NW) for the indicator function are caused by the fact that the random forest can be regarded as one of the best models for the functions such as the indicator. The neural network cannot cope with this structure due to the properties of its training. 2. It can be seen from the experiments that the difference between TNW-CATE and other models increases as the number of treatments decreases. It does not mean that the MSE of TNW-CATE is increased. The MSE is decreased. However, this decrease is not as significant as in other models. This is caused by joint use of controls and treatments in training the neural network. It follows from the above that the co-training of the neural network on treatments corrects the network weights when the domain of the treatment group is shifted relative to the control domain. 3. The main conclusion from the experiments using the dataset IHDP is that the neural network is perfectly adapted to the data structure which is totally unknown. In other words, the neural network tries to implement is own distance function and a form of the kernel to fit the data structure. This is an important distinction of TNW-CATE from other approaches. ## 6 Conclusions

A new method (TNW-CATE) for solving the CATE problem has been studied. The main idea behind TNW-CATE is to use the Nadaraya-Watson regression with kernels that are implemented as neural networks of the considered specific form and are trained on many randomly selected subsets of the treatment and control data. With the proposed method, we aimed to avoid constructing the regression function \(g_{1}\) based only on the treatment group because it may be small. Moreover, we aimed to avoid using standard kernels, for example, Gaussian ones, in the Nadaraya-Watson regression. By training kernels on controls (or controls and treatments), we aimed to transfer knowledge of the feature vector structure in the control group to the treatment group. The proposed model and the learning algorithm can be used in many applications, for example, for drug development and related steps such as clinical trials and study design or selection of the optimal dose of medicine for a patient, etc.