## 4 Results
### AD Definition as a Data Preprocessing Method (Substudy 1)
#### 4.1.4 Effects of X-outliers and Model-outliers on the Model Performance  
Previously, the effects of outliers, identified either in terms of the anomaly scores (X-outliers) or the standardized residuals (Model-outliers), on the performance of the ML models were studied individually. In this section, both aspects are considered combined, with _RF confidence_ for the X-outlier detection method. More precisely, starting from the preprocessed data, the three layouts described in Section 3.3 are compared, with thresholds of 2 kJ/mol and 150 kJ/mol, respectively, for the absolute standardized residual and the _RF confidence_ anomaly score.  
The results for enthalpy are shown in Figure 10. In particular, Figure 10a represents the whole preprocessed dataset, prior to any outlier elimination, while Figure 10b-d, correspond to the same data but with an additional step of outlier elimination, according to Layouts 1-3, respectively. It can be observed that each outlier elimination scenario enables to obtain a cloud of points that lies inside a more restricted window, with respect to the initial dataset, even if some points that were not outliers become outliers during the elimination process. In any case, the clouds of points obtained with the three layouts look similar. Their performance and amount of molecules are also close, as observed in Figure 11.  
Figure 9: Evolution of Lasso model performance in predicting the **enthalpy** under different thresholds for Model-outlier elimination in the preprocessed data.  
Figure 11: Effect of different scenarios of outlier elimination on Lasso model performance in predicting the **enthalpy**.  
Figure 10: Effect of different scenarios of outlier elimination on the data set of the **enthalpy**.  
All three layouts enable us to drastically improve the performance of the model (more than when only X-outliers or Model-outliers are removed with the same thresholds), achieving an overall two-fold decrease in the test \(MAE\). Among the three tested scenarios, a slightly better performance is obtained for Layout 3 (\(MAE\) train = 9.51 kJ/mol vs. 10.43 for Layout 1 and 10.51 for Layout 2; \(MAE\) test = 11.88 kJ/mol vs. 13.48 for Layout 1 and 13.58 for Layout 2), but this comes at the cost of a marginally increased elimination of molecules, with respect to the other two layouts (1531 remaining molecules vs. 1591 for Layout 1 and 1603 for Layout 2). Due to its higher performance, Layout 3 is therefore chosen as the outlier elimination method for the rest of this article. Note that Figure 11 highlights this compromise between the size of the AD and the reliability of the predictions, as already discussed.  
Layout 3 is also applied for the elimination of outliers for the entropy, but with different threshold values for X-outliers in view of a higher density of the cloud of points in a smaller region (see Figure 4). These thresholds are set to 50 J/mol/K and 2 J/mol/K for the \(RF\)_confidence_ anomaly score and the standardized residual, respectively. The resulting data are shown in Figure 12 and result again in a two-fold reduction in the model test \(MAE\), though it is slightly better for Layout 3 (\(MAE\) train = 7.49 J/mol/K vs. 8.61 for Layout 1 and 8.8 for Layout 2; \(MAE\) test = 8.19 J/mol/K vs. 9.82 for Layout 1 and 10 for Layout 2).