# Heterogeneous Treatment Effect with Trained Kernels of the Nadaraya-Watson Regression
## 2 Related Work  
**Estimating CATE**. Computing CATE is a very important problem which can be regarded as a tool for implementing personalized medicine [47]. This fact motivated researchers to develop many efficient approaches solving the problem. An approach based on the Lasso model for estimating CATE was proposed by [PERSON] et al. [48]. Several interesting approaches using the SVM model were presented in [30; 49]. A \"honest\" model for computing CATE was proposed by [PERSON] and [PERSON] [18]. According to the model, the training set is split into two subsets such that the first one is used to construct the partition of the data into subpopulations that differ in the magnitude of their treatment effect, and the second subset is used to estimate treatment effects for each subpopulation. A unified framework for constructing fast tree-growing procedures solving the CATE problem was provided in [50; 51]. A modification of the survival causal tree method for estimating the CATE based on censored observational data was proposed in [52]. [PERSON] et al. [53] established the CATE detection problem as a false positive rate control problem, and they discussed in detail the importance of this approach for solving large-scale CATE detection problems. Algorithms for estimating CATE in the context of Bayesian nonparametric inference were studied in [12]. Bayesian additive regression trees, causal forest, and causal boosting were compared under condition of binary outcomes in [15]. An orthogonal random forest as an algorithm that combines orthogonalization with generalized random forests for solving the CATE estimation problem was proposed in [54]. Estimating CATE as the anomaly detection problem was studied in [55]. Many other approaches can also be found in [3; 47; 56; 57; 58; 59; 60; 61; 62; 63; 64]. A set of meta-algorithms or meta-learners, including the T-learner [25], the S-learner [25], the O-learner [65], and the X-learner [25] were investigated and compared in [25]. Neural networks can be regarded as one of the efficient tools for estimating CATE. As a result, many models based on using neural networks have been proposed [2; 13; 14; 31; 32; 66; 67; 68; 69; 70; 71; 72; 73]. Transformer-based architectures using attention operations [74] were also applied to solving the CATE estimating problem [33; 34; 35; 36]. Ideas of applying the transfer learning technique to CATE estimation were considered in [11; 13; 75]. Ideas of using the NadarayaWatson kernel regression in CATE estimation were studied in [76; 77] where it was shown that the Nadaraya-Watson regression can be used for CATE estimation. However, the small number of training examples in the treatment group does not allow us to efficiently apply this approach. Therefore, we aim to overcome this problem by introducing a neural network of a special architecture, which implements the trainable kernels in the Nadaraya-Watson regression. **The Nadaraya-Watson regression in machine learning**. There are several machine learning approaches based on applying the Nadaraya-Watson regression [78; 79; 80; 81; 82; 83; 84]. Properties of the boosting with kernel regression estimates as weak learners were studied in [85]. A metric learning model with the Nadaraya-Watson kernel regression was proposed in [86]. The high-dimensional nonparametric regression models were considered in [87]. Models taking into account the available correlated errors were proposed in [88]. An interesting work discussing a problem of embedding the Nadaraya-Watson regression into the neural network as a novel trainable CNN layer was presented in [89]. Applied machine learning problems solved by using the Nadaraya-Watson regression were considered in [90]. A method of approximation using the kernel functions made from only the sample points in the neighborhood of input values to simplify the Nadaraya-Watson estimator is proposed in [91].