## 4 Results
### AD Definition as a Data Preprocessing Method (Substudy 1)
#### 4.1.3 Effects of Model-Outliers on the Model Performance  
The same procedure as in Section 4.1.2 applies here but, this time, outliers are removed on the basis of their standardized residual values (Model-outliers) instead of the anomaly scores (X-outliers). In this sense, different thresholds of standardized residuals are tested, namely 1, 2, 3, 5 and 10, in absolute values. This means that the molecules with absolute standardized residual values above 1, 2, 3, 5 and 10 are removed from the preprocessed data.  
Initially, the same effect (i.e., points that are not outliers before the removal of molecules become ones after the removal) as presented in Section 4.1.2 can be observed concerning the \"data-relative\" character of an outlier. Figure 7(b) shows the new standardized residuals versus the new _RF confidence_ anomaly scores after removal of the points with initial absolute standardized residual values above three (see Figure 7(a)), for the enthalpy. It can be observed again that some points have increased their absolute standardized residual values after the outlier removal (points in red in Figure 7(a),b). These points, corresponding mostly to molecules from the families of silicon and inorganic compounds, become less represented (at least, in a consistent manner in terms of the enthalpy values) in the new dataset, after the elimination of the Model-outliers. Besides, the elimination of the outliers with absolute standardized residuals above three seems to have reduced the range of standardized residuals, but not that of the _RF confidence_ anomaly scores.  
Figure 8: Comparison of the dataset (**a**) before and (**b**) after the removal of Model-outliers for **enthalpy** (elimination of the molecules with absolute standardized residuals above 3 kJ/mol in (**a**)). In red, the molecules that were not outliers in (**a**) become outliers in (**b**).  
Figure 7: Comparison of the dataset (**a**) before and (**b**) after the removal of X-outliers via the _RF confidence_ method for **enthalpy** (the molecules with anomaly scores above 200 kJ/mol in (**a**) are removed). In red, the molecules that were not outliers in (**a**) become outliers in (**b**).  
The effect of the elimination of outliers, based on their standardized residual, on the enthalpy model performance is presented in Figure 9. The x-axis represents the tested thresholds, a threshold of k corresponding to the elimination of all the molecules with absolute standardized residuals above k. Similarly as for the elimination of X-outliers based on anomaly scores, the model seems to improve its performance with the elimination of outliers with high absolute standardized residuals, while overfitting is also reduced. At the same time, the number of eliminated molecules is not as dramatic as in the case of X-outliers, allowing a wider margin of selection of the final threshold value. In fact, it is seen that only the strictest threshold of 1 kJ/mol, among the tested ones, results in a significant elimination of about 200 molecules, for a decrease in the _MAE_ that remains comparable to the rest of the tested thresholds.