# BENK: The Beran Estimator with Neural Kernels for Estimating the Heterogeneous Treatment Effect
## 7 Numerical Experiments
### Study of the BENK Properties  
In all pictures illustrating results of numerical experiments, dotted curves correspond to the T-learner (triangle markers), the S-learner (triangle markers) or the X-learner (the circle marker) under the condition of using the Beran estimator with the Gaussian kernels. Dash-and-dot curves correspond to the Cox models. Dashed curves with the same markers correspond to the same models implemented using RSFs. The solid curve with cross markers corresponds to BENK. The PEHE metric is used to represent results of experiments. The smaller the values of the PEHE, the better the obtained results. To avoid clutter of curves on the figures, we pick the best model for each T-,S- or X-learner obtained in each experiment. First, we study different CATE estimators using different numbers \(c\) of controls, taking the values 100, 200, 300, 500, 1000. The number of treatments \(t\) is determined as 20% of the number of controls. Values of \(n\) are equal to \(\min\{t,100\}\). Figures 4-6 illustrate how values of the PEHE metric depend on the number \(c\) of controls for different estimators when different functions are used for generating examples. Figure 4 shows the difference between the PEHE metric of BENK and other models in the experiment, with the feature vectors located around the spiral. The T-SF, S-Beran and X-SF models are provided in Figure 4 because they show the best competitive metric values. In order to illustrate how the variance in results depends on the amount of input data, the error bars are also depicted in Figure 4. It can be seen from Figure 4 that the variance in results is reduced with the number of controls. This property of results indicates that the neural network is properly trained. We do not add the error bars to other graphs so as to not mask the relative positions of the corresponding curves. Figure 5 illustrates similar dependencies when the bell-shaped function is used for generating the feature vectors. The selected models in this case are T-Cox, S-SF and X-Cox. Figure 6 illustrates the relationship between different models obtained on the circular feature space. The competitive algorithms given in the picture are T-Beran, S-Beran and X-Beran. It can be seen from Figures 4-6 that the proposed model BENK provides better results in comparison with other models. The largest relative difference between BENK and other models can be observed when the feature vectors are generated in accordance with the spiral function. This function produces the most complex data structure, such that other studied models cannot cope with it. Another interesting question is how the CATE estimators depend on the proportion \(q\) of treatments and controls in the training set. Particularly, for the proposed BENK model, we try to study whether an increasing number of treatments (the set \(\mathcal{T}\)) provides better CATE results with an unchanged number of controls (the set \(\mathcal{C}\)). The corresponding numerical results are shown in Figures 7-9. One can see from Figures 7-9 that the enhancement in the PEHE is sufficient in comparison with other CATE estimators when \(q\) is changed from 10% to 20% in the experiments with the spiral and bell-shaped functions. Moreover, we again observe the outperformance of BENK in comparison with other estimators. Figure 4: The PEHE metric as a function of the number of the controls when the spiral function is used for generating examples. Figure 5: The PEHE metric as a function of the number of controls when the bell-shaped function is used for generating examples. Figure 6: The PEHE metric as a function of the number of controls when the circular function is used for generating examples. In the previous experiments, the amount of the censored data was taken \(p=33\%\) of all observations. However, it is interesting to study how this amount impacts the PEHE of the CATE estimators. Figures 10-12 illustrate the corresponding dependencies when different generating functions are used.