## 5 Experimental Analysis
### Numerical Experiments  
Tables 1 and 2 report the performance of the ML models in terms of \(|\epsilon_{\text{ATE}}|\) and the PEHE in both in-sample (i.e., counterfactual estimation of the training samples) and out-of-sample (i.e., factual and counterfactual estimation on the held-out test data), respectively, relative to the four different cases presented in the experimental setup. Regarding Case 1, the direct estimation of the ATE using Equation (7) leads to an estimated ATE of 0.58. In detail, in Case 1 there are no biases in the dataset and thus the estimated effect coincides with the ground truth ATE, as it would have been calculated through an RCT. On the other hand, the direct calculation of the effect of high-intensity exercise on BMD loss for the biased datasets is 0.87, which is an overestimation of the ATE compared to the real 0.58 value. The deviation in results compared to the ground truth is reasonable, as in this case the ATE measures the combined effect of a higher concentration \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline \multirow{3}{*}{**Model**} & \multicolumn{2}{c}{**Case 1**} & \multicolumn{2}{c}{**Case 2**} & \multicolumn{2}{c}{**Case 3**} & \multicolumn{2}{c}{**Case 4**} \\  & \(|\epsilon_{\text{ATE}}|=0\) & \(|\epsilon_{\text{ATE}}|=0\) & \(|\epsilon_{\text{ATE}}|=0.29\) & \(|\epsilon_{\text{ATE}}|=0.29\) & \(|\epsilon_{\text{ATE}}|=0.29\) & \(|\epsilon_{\text{ATE}}|=0.29\) \\  & \(|\epsilon_{\text{ATE}}|\) & **PEHE** & \(|\epsilon_{\text{ATE}}|\) & **PEHE** & \(|\epsilon_{\text{ATE}}|\) & **PEHE** & \(|\epsilon_{\text{ATE}}|\) & **PEHE** \\ \hline LR1 & 0.154 & 3.506 & 0.141 & 2.727 & 0.155 & 2.727 & 0.189 & 2.727 \\ LR2 & **0.007** & 3.305 & 0.150 & 2.342 & 0.158 & 2.386 & 0.194 & 2.653 \\ R-Forest & 0.066 & 3.062 & 0.175 & **2.186** & 0.177 & **2.183** & 0.118 & **2.265** \\ BART & 0.427 & 3.221 & 0.160 & 2.490 & 0.165 & 2.435 & 0.169 & 2.727 \\ TARnet & 0.055 & 3.295 & 0.166 & 2.262 & 0.150 & 2.281 & 0.095 & 2.641 \\ Dragonnet & 0.050 & **3.041** & 0.221 & 2.334 & 0.196 & 2.249 & 0.079 & 2.645 \\ NN-Dragonnet (C) & 0.159 & 3.263 & 0.106 & 2.260 & 0.140 & 2.240 & 0.093 & 2.734 \\ NN-Dragonnet (E) & 0.246 & 3.389 & 0.104 & 2.261 & 0.208 & 2.216 & 0.110 & 2.740 \\ NN-Dragonnet (M) & 0.257 & 3.390 & **0.094** & 2.261 & **0.126** & 2.242 & **0.031** & 2.656 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Experimental results on train set. Top results are specified in bold, while second best are underlined. The calculated ground truth ATE of “intense exercise” on “BMD loss” is 0.58.  
of calcium in the blood, along with high-intensity physical activity, leading to a higher inhibition of BMD loss. The interpretation of Tables 1 and 2 reveals that ML models achieve better estimation of the ATE (and CATE) in observational studies, as opposed to the direct estimation of the causal effect from the dataset. Therefore, the benefit of using these algorithms for causal effect estimation is apparent, as the estimation errors are considerably lower. Regarding to the estimation of the PEHE of each model, they are all close to the calculation of the PEHE for the average of the population, as it is calculated on the unbiased and biased dataset. Furthermore, it is worth noticing that in both non-imputed and imputed datasets, there is no significant difference in the estimation of the ATE in Cases 2 and 3, where the \"Calcium supplementation\", \"Nutrition rich in calcium\", \"Alcohol consumption\" and \"Vit. D in Blood\" are confounders, as there is still in the dataset the variable \"Calcium in blood\" acting as a proxy variable. Finally, the estimations of the ATE in Cases 2 and 4 are also very close, as the hidden variable \"Calcium in blood\" is represented by the variables \"Calcium supplement\", \"Alcohol consumption\", \"Calcitonin\", \"Vit. D in Blood\" and \"Nutrition rich in calcium\" in the dataset.