# BENK: The Beran Estimator with Neural Kernels for Estimating the Heterogeneous Treatment Effect
## 6 Neural Network for Estimating CATE  
Let us consider how the Beran estimator with neural kernels can be implemented by means of a neural network of a special type. Our first aim is to implement kernels \(K(\mathbf{x},\mathbf{x}_{i})\) by means of a neural subnetwork, which is called the neural kernel and is a part of the whole network for implementing the Beran estimator. The second aim is for this network to learn on the control data. Having the trained kernel, we can apply it to compute the conditional survival function for controls, as well as for treatments, because the kernels in (14) do not directly depend on times to events \(f_{i}\) or \(h_{i}\). However, in order to train the kernel, we have to train the whole network because the loss function is defined through SF \(S_{0}(t\mid\mathbf{x})\), which represents the probability of survival of a control patient up to time \(t\), which is estimated by means of the Beran estimator. This implies that the whole network contains blocks of the neural kernels for computing kernels \(K(\mathbf{x},\mathbf{x}_{i})\), normalization for computing the kernel weights \(W(\mathbf{x},\mathbf{x}_{i})\) and the Beran estimator in accordance with (14). In order to realize a training procedure for the network, we randomly select a portion (\(n\) examples) from all control training examples and form a single specific example from \(n\) selected ones. This random selection is repeated \(N\) times to have \(N\) examples for training. Thus, for every \(\mathbf{x}_{i}\), \(i=1,\ldots,c\), from the control group, we add another vector \(\mathbf{x}_{k}\) from the same set of controls. By composing \(n\) pairs of vectors \((\mathbf{x}_{i},\mathbf{x}_{k})\), \(k=1,\ldots,n\), and including other elements of training examples (\(\delta_{i},f_{i}\)), we obtain one composite vector of data, representing one new training example for the entire neural network. Such new training examples can be constructed for each \(i=1,\ldots,c\). The formal construction of the training set is considered below. Having the trained neural kernel, it can be successfully used for computing SF \(S_{0}(t\mid\mathbf{z})\) of controls and SF \(S_{1}(t\mid\mathbf{z})\) of treatments for arbitrary vectors of features \(\mathbf{z}\), again applying the Beran estimator. Let us consider the training algorithm in detail. First, we return to the set of \(c\) controls \(\mathcal{C}=\{(\mathbf{x}_{i},\delta_{i},f_{i}),\;i=1,\ldots,c\}\). For every \(i\) from set \(\{1,\ldots,c\}\), we construct \(N\) subsets \(\mathcal{C}_{i}^{(r)}\), \(r=1,\ldots,N\), having \(n\) examples randomly selected from \(\mathcal{C}\backslash(\mathbf{x}_{i},\delta_{i},f_{i})\), which have indices from the index set \(\mathcal{I}^{(r)}\), i.e., the subsets \(\mathcal{C}_{i}^{(r)}\) are of the form  
\[\mathcal{C}_{i}^{(r)}=\{(\mathbf{x}_{k}^{(r)},\delta_{k}^{(r)},f_{k}^{(r)}), \;k\in\mathcal{I}^{(r)}\},\;r=1,\ldots,N. \tag{16}\]  
Here, \(N\) and \(n\) can be regarded as tuning hyperparameters. Upper index \(r\) indicates that the \(r\)-th example \((\mathbf{x}_{k}^{(r)},\delta_{k}^{(r)},f_{k}^{(r)})\) is randomly taken from \(\mathcal{C}\backslash(\mathbf{x}_{i},\delta_{i},f_{i})\), i.e., there is an example \((\mathbf{x}_{j},\delta_{j},f_{j})\) from \(\mathcal{C}\) such that \(\mathbf{x}_{k}^{(r)}=\mathbf{x}_{j}\), \(\delta_{k}^{(r)}=\delta_{j}\), \(f_{k}^{(r)}=f_{j}\). Each subset \(\mathcal{C}_{i}^{(r)}\), jointly with \((\mathbf{x}_{i},\delta_{i},f_{i})\), forms a training example \(\mathbf{a}_{i}^{(r)}\) for the control network as follows:  
\[\mathbf{a}_{i}^{(r)}=\left(\mathcal{C}_{i}^{(r)},\mathbf{x}_{i},\delta_{i},f_ {i}\right),\;i=1,\ldots,c,\;r=1,\ldots,N. \tag{17}\]  
The number of possible examples \(\mathbf{a}_{i}^{(r)}\) is \(c\cdot N\), and these examples are used for training the neural network, whose output is the estimate of SF \(\tilde{S}_{0}(t\mid\mathbf{x}_{i})\).