Conversely, for the rest of the new species and/or for enthalpy, the *RF confidence* anomaly scores are higher and the standardized residuals fluctuate significantly. For example, both very low (e.g., Cat. A or B for enthalpy) and very high (e.g., Cat. C or D for enthalpy, Cat. A or B for entropy) values are observed for the standardized residuals. By highlighting the chemical families and the reference property values (Figures 20b,c,e,f–23b,c,e,f), similarly as in substudy 2, it is possible to observe that the new species that are not well predicted are those located in regions with no—or very few—training points, or those located in higher density regions but with very different property values. However, it remains difficult to understand why some species that have high anomaly scores and/or are isolated from the training data can be either very well predicted (e.g., Cat. A and B in the case of enthalpy) or not (e.g., Cat. A and B in the case of entropy). Further work needs to be conducted to better assess the reliability of the predictions for new species, as the *RF confidence* anomaly score seems not to be a sufficient criterion.

**5. Conclusions and Perspectives**

In this work, the applicability domain (AD) of machine learning (ML) models trained on high-dimensional data is investigated. The considered models are two ML models developed in a previous article for the prediction of the ideal gas enthalpy of formation and entropy of molecules based on their high-dimensional descriptors [15].

The AD is crucial, as it describes the space of chemical characteristics in which the model can make predictions with a given reliability. This work studies the AD definition of a ML model all along its development procedure: during data preprocessing, model construction and model deployment. Inside each of these steps, the AD definition fulfills a different function. Three AD definition methods, commonly used for outlier detection in high-dimensional problems, were compared: isolation forest (iForest), random forest prediction confidence (*RF confidence*) and k-nearest neighbors in the 2D projection of descriptor space obtained via t-distributed stochastic neighbor embedding (*tsNE2D/kNN*). These methods compute an anomaly score which can be used similarly to the distance metrics of classical AD definition methods. A molecule is inside the AD if its distance to the training domain (or anomaly score here) is below a given threshold. Due to the curse of dimensionality, classical AD definition methods are generally unsuitable for high-dimensional problems.

During data preprocessing, the three AD definition methods were used to identify outlier molecules and the effect of their removal was investigated. A more significant improvement in model performance could be observed when outliers identified with *RF confidence* were removed (e.g., for a removal of 30% of outliers, test *MAE* was divided by 2.5, 1.6 and 1.1 for *RF confidence*, *iForest* and *tsNE2D/kNN*, respectively). While these three methods identify X-outliers, the effect of other types of outliers, namely Model-outliers and y-outliers, was also investigated. In particular, the elimination of X-outliers followed by elimination of Model-outliers enabled us to divide *MAE* and *RMSE* by two and three, respectively, while reducing overfitting phenomenon. The elimination of y-outliers did not display significant improvement in the model performance.

During model construction, the AD serves to check how validation or test data are located in comparison with training data. All test data were close to training data according to the *RF confidence* method; however, some test data displayed high prediction errors and *tsNE 2D* representations were used to identify the possible causes (e.g., molecule with chemical information not well represented in training data).

Finally, the developed models were deployed to predict the thermodynamic properties of different categories of molecules. The results show that anomaly scores calculated using the *RF confidence* method are not sufficient to judge the reliability of a prediction for a new molecule. Indeed, some new molecules with low/high anomaly scores were poorly/well predicted. Further research needs to be conducted to identify more appropriate metrics, possibly in combination with the *RF confidence* method, to benefit from the information contained in each metric.
Conversely, for the rest of the new species and/or for enthalpy, the *RF confidence* anomaly scores are higher and the standardized residuals fluctuate significantly. For example, both very low (e.g., Cat. A or B for enthalpy) and very high (e.g., Cat. C or D for enthalpy, Cat. A or B for entropy) values are observed for the standardized residuals. By highlighting the chemical families and the reference property values (Figures 20b,c,e,f–23b,c,e,f), similarly as in substudy 2, it is possible to observe that the new species that are not well predicted are those located in regions with no—or very few—training points, or those located in higher density regions but with very different property values. However, it remains difficult to understand why some species that have high anomaly scores and/or are isolated from the training data can be either very well predicted (e.g., Cat. A and B in the case of enthalpy) or not (e.g., Cat. A and B in the case of entropy). Further work needs to be conducted to better assess the reliability of the predictions for new species, as the *RF confidence* anomaly score seems not to be a sufficient criterion.

## 5. Conclusions and Perspectives

In this work, the applicability domain (AD) of machine learning (ML) models trained on high-dimensional data is investigated. The considered models are two ML models developed in a previous article for the prediction of the ideal gas enthalpy of formation and entropy of molecules based on their high-dimensional descriptors [15].

The AD is crucial, as it describes the space of chemical characteristics in which the model can make predictions with a given reliability. This work studies the AD definition of a ML model all along its development procedure: during data preprocessing, model construction and model deployment. Inside each of these steps, the AD definition fulfills a different function. Three AD definition methods, commonly used for outlier detection in high-dimensional problems, were compared: isolation forest (iForest), random forest prediction confidence (*RF confidence*) and k-nearest neighbors in the 2D projection of descriptor space obtained via t-distributed stochastic neighbor embedding (*tSNE2D/kNN*). These methods compute an anomaly score which can be used similarly to the distance metrics of classical AD definition methods. A molecule is inside the AD if its distance to the training domain (or anomaly score here) is below a given threshold. Due to the curse of dimensionality, classical AD definition methods are generally unsuitable for high-dimensional problems.

During data preprocessing, the three AD definition methods were used to identify outlier molecules and the effect of their removal was investigated. A more significant improvement in model performance could be observed when outliers identified with *RF confidence* were removed (e.g., for a removal of 30% of outliers, test *MAE* was divided by 2.5, 1.6 and 1.1 for *RF confidence*, *iForest* and *tSNE2D/kNN*, respectively). While these three methods identify X-outliers, the effect of other types of outliers, namely Model-outliers and y-outliers, was also investigated. In particular, the elimination of X-outliers followed by elimination of Model-outliers enabled us to divide *MAE* and *RMSE* by two and three, respectively, while reducing overfitting phenomenon. The elimination of y-outliers did not display significant improvement in the model performance.

During model construction, the AD serves to check how validation or test data are located in comparison with training data. All test data were close to training data according to the *RF confidence* method; however, some test data displayed high prediction errors and *tSNE 2D* representations were used to identify the possible causes (e.g., molecule with chemical information not well represented in training data).

Finally, the developed models were deployed to predict the thermodynamic properties of different categories of molecules. The results show that anomaly scores calculated using the *RF confidence* method are not sufficient to judge the reliability of a prediction for a new molecule. Indeed, some new molecules with low/high anomaly scores were poorly/well predicted. Further research needs to be conducted to identify more appropriate metrics, possibly in combination with the *RF confidence* method, to benefit from the information contained in each metric.
More generally, the AD definition should be better integrated into any QSPR/QSAR procedure for later use of the developed models.

**Supplementary Materials:** The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/a16120573/s1, S1 (excel): Data, outliers and ML predictions.

**Author Contributions:** C.T.: literature review, conceptualization, methodology, data curation and modeling, writing (original draft preparation, review and editing). D.M.: supervision, methodology, writing (review and editing). S.L. and O.H.: data provision, molecular and thermodynamic analyses, generation of data for substudy 3, writing (review and editing). All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by MESRI (Ministère de l'Enseignement supérieur, de la Recherche et de l'Innovation), and by the Institute Carnot ICEEL (Grant: "Recyclage de Pneus par Intelligence Artificielle-RePnIA"), France.

**Data Availability Statement:** The authors do not have the permission to share the data from DIPPR; only some information about the descriptors and the predictions are available in the Supplementary Material (excel). The data of the new species used during model deployment cannot be published.

**Conflicts of Interest:** The authors declare no conflict of interest.

## Abbreviations

The following abbreviations are used in this manuscript:

<table><tr><td>AD</td><td>Applicability domain.</td></tr><tr><td>DIPPR</td><td>Design institute for physical properties.</td></tr><tr><td>DFT</td><td>Density functional theory.</td></tr><tr><td>H</td><td>Enthalpy for ideal gas at 298.15 K and 1 bar.</td></tr><tr><td>IQR</td><td>Interquartile range.</td></tr><tr><td>GA</td><td>Genetic algorithm.</td></tr><tr><td>GNN</td><td>Graph neural network.</td></tr><tr><td>GP</td><td>Gaussian processes.</td></tr><tr><td>iForest</td><td>Isolation forest.</td></tr><tr><td>kNN</td><td>k-nearest neighbors.</td></tr><tr><td>Lasso</td><td>Least absolute shrinkage and selection operator.</td></tr><tr><td>MAE</td><td>Mean absolute error.</td></tr><tr><td>ML</td><td>Machine learning.</td></tr><tr><td>OECD</td><td>Organisation for economic co-operation and development.</td></tr><tr><td>PCs</td><td>Principal components.</td></tr><tr><td>PCA</td><td>Principal component analysis.</td></tr><tr><td>QSAR</td><td>Quantitative structure-activity relationship.</td></tr><tr><td>QSPR</td><td>Quantitative structure-property relationship.</td></tr><tr><td>R<sup>2</sup></td><td>Coefficient of determination.</td></tr><tr><td>REACH</td><td>Registration, Evaluation, Authorization and Restriction of Chemicals.</td></tr><tr><td>RF</td><td>Random forest.</td></tr><tr><td>RF confidence</td><td>Random forest prediction confidence.</td></tr><tr><td>RMSE</td><td>Root mean square error.</td></tr><tr><td>S</td><td>Absolute entropy of ideal gas at 298.15 K and 1 bar.</td></tr><tr><td>tSNE</td><td>t-distributed stochastic neighbor embedding.</td></tr></table>
More generally, the AD definition should be better integrated into any QSPR/QSAR procedure for later use of the developed models.

**Supplementary Materials:** The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/a16120573/s1, S1 (excel): Data, outliers and ML predictions.

**Author Contributions:** C.T.: literature review, conceptualization, methodology, data curation and modeling, writing (original draft preparation, review and editing). D.M.: supervision, methodology, writing (review and editing). S.L. and O.H.: data provision, molecular and thermodynamic analyses, generation of data for substudy 3, writing (review and editing). All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by MESRI (Ministère de l'Enseignement supérieur, de la Recherche et de l'Innovation), and by the Institute Carnot ICEEL (Grant: "Recyclage de Pneus par Intelligence Artificielle-RePnIA"), France.

**Data Availability Statement:** The authors do not have the permission to share the data from DIPPR; only some information about the descriptors and the predictions are available in the Supplementary Material (excel). The data of the new species used during model deployment cannot be published.

**Conflicts of Interest:** The authors declare no conflict of interest.

**Abbreviations**

The following abbreviations are used in this manuscript:

<table><tr><td>AD</td><td>Applicability domain.</td></tr><tr><td>DIPPR</td><td>Design institute for physical properties.</td></tr><tr><td>DFT</td><td>Density functional theory.</td></tr><tr><td>H</td><td>Enthalpy for ideal gas at 298.15 K and 1 bar.</td></tr><tr><td>IQR</td><td>Interquartile range.</td></tr><tr><td>GA</td><td>Genetic algorithm.</td></tr><tr><td>GNN</td><td>Graph neural network.</td></tr><tr><td>GP</td><td>Gaussian processes.</td></tr><tr><td>iForest</td><td>Isolation forest.</td></tr><tr><td>kNN</td><td>k-nearest neighbors.</td></tr><tr><td>Lasso</td><td>Least absolute shrinkage and selection operator.</td></tr><tr><td>MAE</td><td>Mean absolute error.</td></tr><tr><td>ML</td><td>Machine learning.</td></tr><tr><td>OECD</td><td>Organisation for economic co-operation and development.</td></tr><tr><td>PCs</td><td>Principal components.</td></tr><tr><td>PCA</td><td>Principal component analysis.</td></tr><tr><td>QSAR</td><td>Quantitative structure-activity relationship.</td></tr><tr><td>QSPR</td><td>Quantitative structure-property relationship.</td></tr><tr><td>R<sup>2</sup></td><td>Coefficient of determination.</td></tr><tr><td>REACH</td><td>Registration, Evaluation, Authorization and Restriction of Chemicals.</td></tr><tr><td>RF</td><td>Random forest.</td></tr><tr><td>RF confidence</td><td>Random forest prediction confidence.</td></tr><tr><td>RMSE</td><td>Root mean square error.</td></tr><tr><td>S</td><td>Absolute entropy of ideal gas at 298.15 K and 1 bar.</td></tr><tr><td>tSNE</td><td>t-distributed stochastic neighbor embedding.</td></tr></table>
## Appendix A. Identification of Model-outliers and X-outliers in the Preprocessed Data for Enthalpy

<table><caption>**Table A1.** Top Model-outliers in the preprocessed data for enthalpy.</caption><thead><tr><th>Absolute Standardized Residual r<sub>i</sub></th><th>Chemid</th><th>Family</th></tr></thead><tbody><tr><td rowspan="4">r<sub>i</sub> ≥ 10</td><td>3608</td><td>Halogen Compounds</td></tr><tr><td>2628</td><td>Halogen Compounds</td></tr><tr><td>3862</td><td>Polyfunctional Compounds</td></tr><tr><td>7886</td><td>Polyfunctional Compounds</td></tr><tr><td rowspan="4">5 ≤ r<sub>i</sub> &lt; 10</td><td>7887</td><td>Polyfunctional Compounds</td></tr><tr><td>1840</td><td>Sulfur Compounds</td></tr><tr><td>2254</td><td>Organic Acids</td></tr><tr><td>1950</td><td>Inorganic Compounds</td></tr><tr><td rowspan="16">3 ≤ r<sub>i</sub> &lt; 5</td><td>1969</td><td>Silicon Compounds</td></tr><tr><td>2283</td><td>Organic Acids</td></tr><tr><td>3931</td><td>Silicon Compounds</td></tr><tr><td>3948</td><td>Silicon Compounds</td></tr><tr><td>3929</td><td>Silicon Compounds</td></tr><tr><td>4994</td><td>Silicon Compounds</td></tr><tr><td>9858</td><td>Organic Salts</td></tr><tr><td>2619</td><td>Halogen Compounds</td></tr><tr><td>6851</td><td>Other Compounds</td></tr><tr><td>2995</td><td>Silicon Compounds</td></tr><tr><td>3991</td><td>Silicon Compounds</td></tr><tr><td>3958</td><td>Silicon Compounds</td></tr><tr><td>2370</td><td>Esters/Ethers</td></tr><tr><td>2653</td><td>Halogen Compounds</td></tr><tr><td>2877</td><td>Polyfunctional Compounds</td></tr><tr><td>3974</td><td>Silicon Compounds</td></tr><tr><td>3898</td><td>Inorganic Compounds</td></tr><tr><td>6850</td><td>Organic Compounds</td></tr><tr><td>9866</td><td>Nitrogen Compounds</td></tr></tbody></table>

<table><caption>**Table A2.** Top 10 X-outliers in the preprocessed data for enthalpy.</caption><thead><tr><th rowspan="2">N°</th><th colspan="2">iForest</th><th colspan="2">RF Confidence</th><th colspan="2">tSNE2D/kNN</th></tr><tr><th>Chemid</th><th>Family</th><th>Chemid</th><th>Family</th><th>Chemid</th><th>Family</th></tr></thead><tbody><tr><td>1</td><td>3933</td><td>Silicon Compounds</td><td>2624</td><td>Halogen Compounds</td><td>3977</td><td>Silicon Compounds</td></tr><tr><td>2</td><td>3932</td><td>Silicon Compounds</td><td>3933</td><td>Silicon Compounds</td><td>1509</td><td>Halogen Compounds</td></tr><tr><td>3</td><td>2624</td><td>Halogen Compounds</td><td>1627</td><td>Halogen Compounds</td><td>1866</td><td>Halogen Compounds</td></tr><tr><td>4</td><td>3931</td><td>Silicon Compounds</td><td>1631</td><td>Halogen Compounds</td><td>9879</td><td>Nitrogen Compounds</td></tr><tr><td>5</td><td>2991</td><td>Silicon Compounds</td><td>1626</td><td>Halogen Compounds</td><td>9877</td><td>Nitrogen Compounds</td></tr><tr><td>6</td><td>1631</td><td>Halogen Compounds</td><td>3881</td><td>Polyfunctional Compounds</td><td>90</td><td>Alkanes</td></tr><tr><td>7</td><td>3877</td><td>Other Compounds</td><td>1930</td><td>Inorganic Compounds</td><td>5878</td><td>Organic Acids</td></tr><tr><td>8</td><td>2995</td><td>Silicon Compounds</td><td>1864</td><td>Halogen Compounds</td><td>1097</td><td>Ketones/Aldehydes</td></tr><tr><td>9</td><td>3348</td><td>Esters/Ethers</td><td>3932</td><td>Silicon Compounds</td><td>3056</td><td>Ketones/Aldehydes</td></tr><tr><td>10</td><td>1627</td><td>Halogen Compounds</td><td>834</td><td>Alkanes</td><td>7883</td><td>Nitrogen Compounds</td></tr></tbody></table>
## Appendix A. Identification of Model-outliers and X-outliers in the Preprocessed Data for Enthalpy

<table><caption>**Table A1.** Top Model-outliers in the preprocessed data for enthalpy.</caption><thead><tr><th>Absolute Standardized Residual r<sub>i</sub></th><th>Chemid</th><th>Family</th></tr></thead><tbody><tr><td rowspan="4">r<sub>i</sub> ≥ 10</td><td>3608</td><td>Halogen Compounds</td></tr><tr><td>2628</td><td>Halogen Compounds</td></tr><tr><td>3862</td><td>Polyfunctional Compounds</td></tr><tr><td>7886</td><td>Polyfunctional Compounds</td></tr><tr><td rowspan="4">5 ≤ r<sub>i</sub> &lt; 10</td><td>7887</td><td>Polyfunctional Compounds</td></tr><tr><td>1840</td><td>Sulfur Compounds</td></tr><tr><td>2254</td><td>Organic Acids</td></tr><tr><td>1950</td><td>Inorganic Compounds</td></tr><tr><td rowspan="16">3 ≤ r<sub>i</sub> &lt; 5</td><td>1969</td><td>Silicon Compounds</td></tr><tr><td>2283</td><td>Organic Acids</td></tr><tr><td>3931</td><td>Silicon Compounds</td></tr><tr><td>3948</td><td>Silicon Compounds</td></tr><tr><td>3929</td><td>Silicon Compounds</td></tr><tr><td>4994</td><td>Silicon Compounds</td></tr><tr><td>9858</td><td>Organic Salts</td></tr><tr><td>2619</td><td>Halogen Compounds</td></tr><tr><td>6851</td><td>Other Compounds</td></tr><tr><td>2995</td><td>Silicon Compounds</td></tr><tr><td>3991</td><td>Silicon Compounds</td></tr><tr><td>3958</td><td>Silicon Compounds</td></tr><tr><td>2370</td><td>Esters/Ethers</td></tr><tr><td>2653</td><td>Halogen Compounds</td></tr><tr><td>2877</td><td>Polyfunctional Compounds</td></tr><tr><td>3974</td><td>Silicon Compounds</td></tr><tr><td>3898</td><td>Inorganic Compounds</td></tr><tr><td>6850</td><td>Organic Compounds</td></tr><tr><td>9866</td><td>Nitrogen Compounds</td></tr></tbody></table>

<table><caption>**Table A2.** Top 10 X-outliers in the preprocessed data for enthalpy.</caption><thead><tr><th rowspan="2">N°</th><th colspan="2">iForest</th><th colspan="2">RF Confidence</th><th colspan="2">tSNE2D/kNN</th></tr><tr><th>Chemid</th><th>Family</th><th>Chemid</th><th>Family</th><th>Chemid</th><th>Family</th></tr></thead><tbody><tr><td>1</td><td>3933</td><td>Silicon Compounds</td><td>2624</td><td>Halogen Compounds</td><td>3977</td><td>Silicon Compounds</td></tr><tr><td>2</td><td>3932</td><td>Silicon Compounds</td><td>3933</td><td>Silicon Compounds</td><td>1509</td><td>Halogen Compounds</td></tr><tr><td>3</td><td>2624</td><td>Halogen Compounds</td><td>1627</td><td>Halogen Compounds</td><td>1866</td><td>Halogen Compounds</td></tr><tr><td>4</td><td>3931</td><td>Silicon Compounds</td><td>1631</td><td>Halogen Compounds</td><td>9879</td><td>Nitrogen Compounds</td></tr><tr><td>5</td><td>2991</td><td>Silicon Compounds</td><td>1626</td><td>Halogen Compounds</td><td>9877</td><td>Nitrogen Compounds</td></tr><tr><td>6</td><td>1631</td><td>Halogen Compounds</td><td>3881</td><td>Polyfunctional Compounds</td><td>90</td><td>Alkanes</td></tr><tr><td>7</td><td>3877</td><td>Other Compounds</td><td>1930</td><td>Inorganic Compounds</td><td>5878</td><td>Organic Acids</td></tr><tr><td>8</td><td>2995</td><td>Silicon Compounds</td><td>1864</td><td>Halogen Compounds</td><td>1097</td><td>Ketones/Aldehydes</td></tr><tr><td>9</td><td>3348</td><td>Esters/Ethers</td><td>3932</td><td>Silicon Compounds</td><td>3056</td><td>Ketones/Aldehydes</td></tr><tr><td>10</td><td>1627</td><td>Halogen Compounds</td><td>834</td><td>Alkanes</td><td>7883</td><td>Nitrogen Compounds</td></tr></tbody></table>
## Appendix B. Most Represented Families in the Eliminated Outliers (Layout 3) for Enthalpy and Entropy

<table><caption>Table A3. Most represented families in the eliminated outliers (Layout 3) for **enthalpy** and **entropy**.</caption><thead><tr><th colspan="2">Enthalpy</th><th colspan="2">Entropy</th></tr></thead><tbody><tr><td>Halogen Compounds</td><td>24%</td><td>Esters/Ethers</td><td>20%</td></tr><tr><td>Silicon Compounds</td><td>16%</td><td>Silicon Compounds</td><td>13%</td></tr><tr><td>Esters/Ethers</td><td>10%</td><td>Polyfunctional Compounds</td><td>9%</td></tr><tr><td>Nitrogen Compounds</td><td>9%</td><td>Nitrogen Compounds</td><td>9%</td></tr><tr><td>Inorganic Compounds</td><td>7%</td><td>Aromatics</td><td>7%</td></tr><tr><td>Total</td><td>66%</td><td>Total</td><td>58%</td></tr></tbody></table>

## Appendix C. AD Definition for Entropy During Model Construction

<figure><img src="image_35.png" /><figcaption>**Figure A1.** Parity plots for **entropy** after outlier elimination (Layout 3) and dimensionality reduction (GA).</figcaption></figure>
## Appendix B. Most Represented Families in the Eliminated Outliers (Layout 3) for Enthalpy and Entropy

<table><caption>Table A3. Most represented families in the eliminated outliers (Layout 3) for enthalpy and entropy.</caption><thead><tr><th colspan="2">Enthalpy</th><th colspan="2">Entropy</th></tr></thead><tbody><tr><td>Halogen Compounds</td><td>24%</td><td>Esters/Ethers</td><td>20%</td></tr><tr><td>Silicon Compounds</td><td>16%</td><td>Silicon Compounds</td><td>13%</td></tr><tr><td>Esters/Ethers</td><td>10%</td><td>Polyfunctional Compounds</td><td>9%</td></tr><tr><td>Nitrogen Compounds</td><td>9%</td><td>Nitrogen Compounds</td><td>9%</td></tr><tr><td>Inorganic Compounds</td><td>7%</td><td>Aromatics</td><td>7%</td></tr><tr><td>Total</td><td>66%</td><td>Total</td><td>58%</td></tr></tbody></table>

## Appendix C. AD Definition for Entropy During Model Construction

<figure><img src="image_38.png" /><figcaption>Figure A1. Parity plots for entropy after outlier elimination (Layout 3) and dimensionality reduction (GA).</figcaption></figure>
<figure><img src="image_36.png" /><figcaption>**Figure A2.** Visualization of the AD for entropy after outlier elimination (Layout 3) and dimensionality reduction (GA).</figcaption></figure>
<figure><img src="image_39.png" /><figcaption>**Figure A2.** Visualization of the AD for entropy after outlier elimination (Layout 3) and dimensionality reduction (GA).</figcaption></figure>
<figure><img src="image_37.png" /><figcaption>**Figure A3.** tSNE 2D representations for **entropy** after outlier elimination (Layout 3) and dimension-ality reduction (GA), **splits 2 and 4**.</figcaption></figure>
<figure><img src="image_40.png" /><figcaption>**Figure A3.** tSNE 2D representations for **entropy** after outlier elimination (Layout 3) and dimension-ality reduction (GA), **splits 2 and 4**.</figcaption></figure>
<figure><img src="image_38.png" /><figcaption>**Figure A4.** tSNE 2D representations for **entropy** after outlier elimination (Layout 3) and dimensionality reduction (GA), **split 5**.</figcaption></figure>

<figure><img src="image_39.png" /><figcaption>**Figure A5.** Chemical families considered in Figures 15b,e, 16b,e, A3b,e and A4b.</figcaption></figure>
<figure><img src="image_41.png" /><figcaption>**Figure A4.** tSNE 2D representations for **entropy** after outlier elimination (Layout 3) and dimensionality reduction (GA), **split 5**.</figcaption></figure>

<figure><img src="image_42.png" /><figcaption>**Figure A5.** Chemical families considered in Figures 15b,e, 16b,e, A3b,e and A4b.</figcaption></figure>
<figure><img src="image_40.png" /><figcaption>(a) Split 2</figcaption></figure>

<figure><img src="image_41.png" /><figcaption>(b) Split 4</figcaption></figure>

<figure><img src="image_42.png" /><figcaption>(c) Split 5</figcaption></figure>

**Figure A6.** Analysis of the causes of the high prediction errors for **entropy** after outlier elimination (Layout 3) and dimensionality reduction (GA).

## References

1. Netzeva, T.I.; Worth, A.P.; Aldenberg, T.; Benigni, R.; Cronin, M.T.; Gramatica, P.; Jaworska, J.S.; Kahn, S.; Klopman, G.; Marchant, C.A.; et al. Current status of methods for defining the applicability domain of (quantitative) structure-activity relationships. *ATLA Altern. Lab. Anim. 2005*, *33*, 155-173. [CrossRef]

2. McCartney, M.; Haeringer, M.; Polifke, W. Comparison of Machine Learning Algorithms in the Interpolation and Extrapolation of Flame Describing Functions. *J. Eng. Gas Turbines Power 2020*, *142*, 061009. [CrossRef]

3. Cao, X.; Yousefzadeh, R. Extrapolation and AI transparency: Why machine learning models should reveal when they make decisions beyond their training. *Big Data Soc.* **2023**, *10*, 20539517231169731. [CrossRef]

4. European Commission Environment Directorate General. *Guidance Document on the Validation of (Quantitative) Structure-Activity Relationships [(Q)Sar] Models*; OECD: Paris, France, 2014; pp. 1-154.

5. Dearden, J.C.; Cronin, M.T.; Kaiser, K.L. How not to develop a quantitative structure-activity or structure-property relationship (QSAR/QSPR). *SAR QSA Environ. Res.* **2009**, *20*, 241-266. [CrossRef] [PubMed]

6. Singh, M.M.; Smith, I.F.C. Extrapolation with machine learning based early-stage energy prediction models. In *Proceedings of the 2023 European Conference on Computing in Construction and the 40th International CIB W78 Conference, Crete, Greece, 10-12 July 2023*; Volume 4. [CrossRef]

7. Muckley, E.S.; Saal, J.E.; Meredig, B.; Roper, C.S.; Martin, J.H. Interpretable models for extrapolation in scientific machine learning *Digit. Discov.* **2023**, *2*, 1425-1435. [CrossRef]

8. Hoaglin, D.C.; Kempthorne, P.J. Influential Observations, High Leverage Points, and Outliers in Linear Regression: Comment. *Stat. Sci.* **1986**, *1*, 408-412. [CrossRef]

9. Aggarwal, C.C.; Yu, P.S. Outlier detection for high dimensional data. In *Proceedings of the ACM SIGMOD International Conference on Management of Data, Santa Barbara, CA, USA, 21-24 May 2001*; pp. 37-46. [CrossRef]

10. Akoglu, L.; Tong, H.; Koutra, D. Graph based anomaly detection and description: A survey. *Data Min. Knowl. Discov.* **2015**, *29*, 626-688. [CrossRef]

11. Souiden, I.; Omri, M.N.; Brahmi, Z. A survey of outlier detection in high dimensional data streams. *Comput. Sci. Rev.* **2022**, *44*, 100463. [CrossRef]

12. Smiti, A. A critical overview of outlier detection methods. *Comput. Sci. Rev.* **2020**, *38*, 100306. [CrossRef]
<figure><img src="image_43.png" /><figcaption>(a) Split 2</figcaption></figure>

<figure><img src="image_44.png" /><figcaption>(b) Split 4</figcaption></figure>

<figure><img src="image_45.png" /><figcaption>(c) Split 5</figcaption></figure>

**Figure A6.** Analysis of the causes of the high prediction errors for **entropy** after outlier elimination (Layout 3) and dimensionality reduction (GA).

## References

1. Netzeva, T.I.; Worth, A.P.; Aldenberg, T.; Benigni, R.; Cronin, M.T.; Gramatica, P.; Jaworska, J.S.; Kahn, S.; Klopman, G.; Marchant, C.A.; et al. Current status of methods for defining the applicability domain of (quantitative) structure-activity relationships. *ATLA Altern. Lab. Anim. 2005*, *33*, 155–173. [CrossRef]

2. McCartney, M.; Haeringer, M.; Polifke, W. Comparison of Machine Learning Algorithms in the Interpolation and Extrapolation of Flame Describing Functions. *J. Eng. Gas Turbines Power 2020*, *142*, 061009. [CrossRef]

3. Cao, X.; Yousefzadeh, R. Extrapolation and AI transparency: Why machine learning models should reveal when they make decisions beyond their training. *Big Data Soc.* **2023**, *10*, 20539517231169731. [CrossRef]

4. European Commission Environment Directorate General. *Guidance Document on the Validation of (Quantitative) Structure-Activity Relationships [(Q)Sar] Models*; OECD: Paris, France, 2014; pp. 1–154.

5. Dearden, J.C.; Cronin, M.T.; Kaiser, K.L. How not to develop a quantitative structure-activity or structure-property relationship (QSAR/QSPR). *SAR QSA Environ. Res.* **2009**, *20*, 241–266. [CrossRef] [PubMed]

6. Singh, M.M.; Smith, I.F.C. Extrapolation with machine learning based early-stage energy prediction models. In *Proceedings of the 2023 European Conference on Computing in Construction and the 40th International CIB W78 Conference, Crete, Greece, 10–12 July 2023*; Volume 4. [CrossRef]

7. Muckley, E.S.; Saal, J.E.; Meredig, B.; Roper, C.S.; Martin, J.H. Interpretable models for extrapolation in scientific machine learning *Digit. Discov.* **2023**, *2*, 1425–1435. [CrossRef]

8. Hoaglin, D.C.; Kempthorne, P.J. Influential Observations, High Leverage Points, and Outliers in Linear Regression: Comment. *Stat. Sci.* **1986**, *1*, 408–412. [CrossRef]

9. Aggarwal, C.C.; Yu, P.S. Outlier detection for high dimensional data. In *Proceedings of the ACM SIGMOD International Conference on Management of Data*, Santa Barbara, CA, USA, 21–24 May 2001; pp. 37–46. [CrossRef]

10. Akoglu, L.; Tong, H.; Koutra, D. Graph based anomaly detection and description: A survey. *Data Min. Knowl. Discov.* **2015**, *29*, 626–688. [CrossRef]

11. Souiden, I.; Omri, M.N.; Brahmi, Z. A survey of outlier detection in high dimensional data streams. *Comput. Sci. Rev.* **2022**, *44*, 100463. [CrossRef]

12. Smiti, A. A critical overview of outlier detection methods. *Comput. Sci. Rev.* **2020**, *38*, 100306. [CrossRef]
13. Cao, D.S.; Liang, Y.Z.; Xu, Q.S.; Li, H.D.; Chen, X. A New Strategy of Outlier Detection for QSAR/QSPR. *J. Comput. Chem.* **2010**, *31*, 592–602. [CrossRef]

14. De Maesschalck, R.; Estienne, F.; Verdu-Andres, J.; Candolfi, A.; Centner, V.; Despagne, F.; Jouan-Rimbaud, D.; Walczak, B.; Massart, D.L.; De Jong, S.; et al. The development of calibration models for spectroscopic data using principal component regression [Review]. *Internet J. Chem.* **1999**, *2*, 1–21.

15. Trinh, C.; Tbatou, Y.; Lasala, S.; Herbinet, O.; Meimaroglou, D. On the Development of Descriptor-Based Machine Learning Models for Thermodynamic Properties. Part 1—From Data Collection to Model Construction: Understanding of the Methods and their Effects. *Processes* **2023**, *11*, 3325. [CrossRef]

16. Sahigara, F.; Mansouri, K.; Ballabio, D.; Mauri, A.; Consonni, V.; Todeschini, R. Comparison of different approaches to define the applicability domain of QSAR models. *Molecules* **2012**, *17*, 4791–4810. [CrossRef] [PubMed]

17. Jaworska, J.; Nikolova-Jeliazkova, N.; Aldenberg, T. QSAR applicability domain estimation by projection of the training set in descriptor space: A review. *ATLA Altern. Lab. Anim.* **2005**, *33*, 445–459. [CrossRef] [PubMed]

18. Mathea, M.; Klingspohn, W.; Baumann, K. Chemoinformatic Classification Methods and their Applicability Domain. *Mol. Inform.* **2016**, *35*, 160–180. [CrossRef] [PubMed]

19. Roy, K.; Kar, S.; Ambure, P. On a simple approach for determining applicability domain of QSAR models. *Chemom. Intell. Lab. Syst.* **2015**, *145*, 22–29. [CrossRef]

20. Yalamanchi, K.K.; van Oudenhoven, V.C.; Tutino, F.; Monge-Palacios, M.; Alshehri, A.; Gao, X.; Sarathy, S.M. Machine Learning to Predict Standard Enthalpy of Formation of Hydrocarbons. *J. Phys. Chem. A* **2019**, *123*, 8305–8313. [CrossRef]

21. Yalamanchi, K.K.; Monge-Palacios, M.; van Oudenhoven, V.C.; Gao, X.; Sarathy, S.M. Data Science Approach to Estimate Enthalpy of Formation of Cyclic Hydrocarbons. *J. Phys. Chem. A* **2020**, *124*, 6270–6276. [CrossRef]

22. Aldosari, M.N.; Yalamanchi, K.K.; Gao, X.; Sarathy, S.M. Predicting entropy and heat capacity of hydrocarbons using machine learning. *Energy AI* **2021**, *4*, 100054. [CrossRef]

23. Aouichaoui, A.R.; Fan, F.; Abildskov, J.; Sin, G. Application of interpretable group-embedded graph neural networks for pure compound properties. *Comput. Chem. Eng.* **2023**, *176*, 108291. [CrossRef]

24. Balestriero, R.; Pesenti, J.; LeCun, Y. Learning in High Dimension Always Amounts to Extrapolation. *arXiv* **2021**, arXiv:2110.09485.

25. Ghorbani, H. Mahalanobis Distance and Its Application for detecting multivariate outliers. *Ser. Math. Inform.* **2019**, *34*, 583–595. [CrossRef]

26. De Maesschalck, R.; Jouan-Rimbaud, D.; Massart, D.L. The Mahalanobis distance. *Chemom. Intell. Lab. Syst.* **2000**, *50*, 1–18. [CrossRef]

27. Aouichaoui, A.R.; Fan, F.; Mansouri, S.S.; Abildskov, J.; Sin, G. Combining Group-Contribution Concept and Graph Neural Networks Toward Interpretable Molecular Property Models. *J. Chem. Inf. Model.* **2023**, *63*, 725–744. [CrossRef] [PubMed]

28. Mauri, A.; Bertola, M. Alvascience: A New Software Suite for the QSAR Workflow Applied to the Blood-Brain Barrier Permeability. *Int. J. Mol. Sci.* **2022**, *23*, 12882. [CrossRef] [PubMed]

29. Huoyu, R.; Zhiqiang, Z.; Zhanggao, L.; Zhenzhen, X. Quantitative structure–property relationship for the critical temperature of saturated monobasic ketones, aldehydes, and ethers with molecular descriptors. *Int. J. Quantum Chem.* **2022**, *122*, 1–10. [CrossRef]

30. Cao, L.; Zhu, P.; Zhao, Y.; Zhao, J. Using machine learning and quantum chemistry descriptors to predict the toxicity of ionic liquids. *J. Hazard. Mater.* **2018**, *352*, 17–26. [CrossRef]

31. Yousefinejad, S.; Mahboubifar, M.; Eskandari, R. Quantitative structure-activity relationship to predict the anti-malarial activity in a set of new imidazolopiperazines based on artificial neural networks. *Malar. J.* **2019**, *18*, 1–17. [CrossRef]

32. Asadollahi, T.; Dadfarnia, S.; Shabani, A.M.H.; Ghasemi, J.B.; Sarkhosh, M. QSAR models for cxcr2 receptor antagonists based on the genetic algorithm for data preprocessing prior to application of the pls linear regression method and design of the new compounds using in silico virtual screening. *Molecules* **2011**, *16*, 1928–1955. [CrossRef]

33. Kim, M.G. Sources of High Leverage in Linear Regression Model. *J. Appl. Math. Inform.* **2004**, *16*, 509–513.

34. Leys, C.; Klein, O.; Dominicy, Y.; Ley, C. Detecting multivariate outliers: Use a robust variant of the Mahalanobis distance. *J. Exp. Soc. Psychol.* **2018**, *74*, 150–156. [CrossRef]

35. Gramatica, P. Principles of QSAR models validation: Internal and external. *QSAR Comb. Sci.* **2007**, *26*, 694–701. [CrossRef]

36. Varamesh, A.; Hemmati-Sarapardeh, A.; Dabir, B.; Mohammadi, A.H. Development of robust generalized models for estimating the normal boiling points of pure chemical compounds. *J. Mol. Liq.* **2017**, *242*, 59–69. [CrossRef]

37. Sabando, M.V.; Ponzoni, I.; Soto, A.J. Neural-based approaches to overcome feature selection and applicability domain in drug-related property prediction. *Appl. Soft Comput. J.* **2019**, *85*, 105777. [CrossRef]

38. Huang, J.; Fan, X. Reliably assessing prediction reliability for high dimensional QSAR data. *Mol. Divers.* **2013**, *17*, 63–73. [CrossRef] [PubMed]

39. Rakhimbekova, A.; Madzhidov, T.; Nugmanov, R.I.; Baskin, I.; Varnek, A.; Rakhimbekova, A.; Madzhidov, T.; Nugmanov, R.I.; Gimadiev, T.; Baskin, I. Comprehensive Analysis of Applicability Domains of QSPR Models for Chemical Reactions. *Int. J. Mol. Sci.* **2021**, *21*, 5542. [CrossRef] [PubMed]

40. Kaneko, H.; Funatsu, K. Applicability domain based on ensemble learning in classification and regression analyses. *J. Chem. Inf. Model.* **2014**, *54*, 2469–2482. [CrossRef]

41. Rasmussen, C.E.; Williams, C.K.I. *Gaussian Processes for Machine Learning*; MIT Press: Cambridge, MA, USA, 2006.

42. Sushko, I. Applicability Domain of QSAR Models. Ph.D. Thesis, Technical University of Munich, Munich, Germany, 2011.
13. Cao, D.S.; Liang, Y.Z.; Xu, Q.S.; Li, H.D.; Chen, X. A New Strategy of Outlier Detection for QSAR/QSPR. *J. Comput. Chem.* **2010**, *31*, 592–602. [CrossRef]

14. De Maesschalck, R.; Estienne, F.; Verdu-Andres, J.; Candolfi, A.; Centner, V.; Despagne, F.; Jouan-Rimbaud, D.; Walczak, B.; Massart, D.L.; De Jong, S.; et al. The development of calibration models for spectroscopic data using principal component regression [Review]. *Internet J. Chem.* **1999**, *2*, 1–21.

15. Trinh, C.; Tbatou, Y.; Lasala, S.; Herbinet, O.; Meimaroglou, D. On the Development of Descriptor-Based Machine Learning Models for Thermodynamic Properties. Part 1—From Data Collection to Model Construction: Understanding of the Methods and their Effects. *Processes* **2023**, *11*, 3325. [CrossRef]

16. Sahigara, F.; Mansouri, K.; Ballabio, D.; Mauri, A.; Consonni, V.; Todeschini, R. Comparison of different approaches to define the applicability domain of QSAR models. *Molecules* **2012**, *17*, 4791–4810. [CrossRef] [PubMed]

17. Jaworska, J.; Nikolova-Jeliazkova, N.; Aldenberg, T. QSAR applicability domain estimation by projection of the training set in descriptor space: A review. *ATLA Altern. Lab. Anim.* **2005**, *33*, 445–459. [CrossRef] [PubMed]

18. Mathea, M.; Klingspohn, W.; Baumann, K. Chemoinformatic Classification Methods and their Applicability Domain. *Mol. Inform.* **2016**, *35*, 160–180. [CrossRef] [PubMed]

19. Roy, K.; Kar, S.; Ambure, P. On a simple approach for determining applicability domain of QSAR models. *Chemom. Intell. Lab. Syst.* **2015**, *145*, 22–29. [CrossRef]

20. Yalamanchi, K.K.; van Oudenhoven, V.C.; Tutino, F.; Monge-Palacios, M.; Alshehri, A.; Gao, X.; Sarathy, S.M. Machine Learning to Predict Standard Enthalpy of Formation of Hydrocarbons. *J. Phys. Chem. A* **2019**, *123*, 8305–8313. [CrossRef]

21. Yalamanchi, K.K.; Monge-Palacios, M.; van Oudenhoven, V.C.; Gao, X.; Sarathy, S.M. Data Science Approach to Estimate Enthalpy of Formation of Cyclic Hydrocarbons. *J. Phys. Chem. A* **2020**, *124*, 6270–6276. [CrossRef]

22. Aldosari, M.N.; Yalamanchi, K.K.; Gao, X.; Sarathy, S.M. Predicting entropy and heat capacity of hydrocarbons using machine learning. *Energy AI* **2021**, *4*, 100054. [CrossRef]

23. Aouichaoui, A.R.; Fan, F.; Abildskov, J.; Sin, G. Application of interpretable group-embedded graph neural networks for pure compound properties. *Comput. Chem. Eng.* **2023**, *176*, 108291. [CrossRef]

24. Balestriero, R.; Pesenti, J.; LeCun, Y. Learning in High Dimension Always Amounts to Extrapolation. *arXiv* **2021**, arXiv:2110.09485.

25. Ghorbani, H. Mahalanobis Distance and Its Application for detecting multivariate outliers. *Ser. Math. Inform.* **2019**, *34*, 583–595. [CrossRef]

26. De Maesschalck, R.; Jouan-Rimbaud, D.; Massart, D.L. The Mahalanobis distance. *Chemom. Intell. Lab. Syst.* **2000**, *50*, 1–18. [CrossRef]

27. Aouichaoui, A.R.; Fan, F.; Mansouri, S.S.; Abildskov, J.; Sin, G. Combining Group-Contribution Concept and Graph Neural Networks Toward Interpretable Molecular Property Models. *J. Chem. Inf. Model.* **2023**, *63*, 725–744. [CrossRef] [PubMed]

28. Mauri, A.; Bertola, M. Alvascience: A New Software Suite for the QSAR Workflow Applied to the Blood-Brain Barrier Permeability. *Int. J. Mol. Sci.* **2022**, *23*, 12882. [CrossRef] [PubMed]

29. Huoyu, R.; Zhiqiang, Z.; Zhanggao, L.; Zhenzhen, X. Quantitative structure–property relationship for the critical temperature of saturated monobasic ketones, aldehydes, and ethers with molecular descriptors. *Int. J. Quantum Chem.* **2022**, *122*, 1–10. [CrossRef]

30. Cao, L.; Zhu, P.; Zhao, Y.; Zhao, J. Using machine learning and quantum chemistry descriptors to predict the toxicity of ionic liquids. *J. Hazard. Mater.* **2018**, *352*, 17–26. [CrossRef]

31. Yousefinejad, S.; Mahboubifar, M.; Eskandari, R. Quantitative structure-activity relationship to predict the anti-malarial activity in a set of new imidazolopiperazines based on artificial neural networks. *Malar. J.* **2019**, *18*, 1–17. [CrossRef]

32. Asadollahi, T.; Dadfarnia, S.; Shabani, A.M.H.; Ghasemi, J.B.; Sarkhosh, M. QSAR models for cxcr2 receptor antagonists based on the genetic algorithm for data preprocessing prior to application of the pls linear regression method and design of the new compounds using in silico virtual screening. *Molecules* **2011**, *16*, 1928–1955. [CrossRef]

33. Kim, M.G. Sources of High Leverage in Linear Regression Model. *J. Appl. Math. Inform.* **2004**, *16*, 509–513.

34. Leys, C.; Klein, O.; Dominicy, Y.; Ley, C. Detecting multivariate outliers: Use a robust variant of the Mahalanobis distance. *J. Exp. Soc. Psychol.* **2018**, *74*, 150–156. [CrossRef]

35. Gramatica, P. Principles of QSAR models validation: Internal and external. *QSAR Comb. Sci.* **2007**, *26*, 694–701. [CrossRef]

36. Varamesh, A.; Hemmati-Sarapardeh, A.; Dabir, B.; Mohammadi, A.H. Development of robust generalized models for estimating the normal boiling points of pure chemical compounds. *J. Mol. Liq.* **2017**, *242*, 59–69. [CrossRef]

37. Sabando, M.V.; Ponzoni, I.; Soto, A.J. Neural-based approaches to overcome feature selection and applicability domain in drug-related property prediction. *Appl. Soft Comput. J.* **2019**, *85*, 105777. [CrossRef]

38. Huang, J.; Fan, X. Reliably assessing prediction reliability for high dimensional QSAR data. *Mol. Divers.* **2013**, *17*, 63–73. [CrossRef] [PubMed]

39. Rakhimbekova, A.; Madzhidov, T.; Nugmanov, R.I.; Baskin, I.; Varnek, A.; Rakhimbekova, A.; Madzhidov, T.; Nugmanov, R.I.; Gimadiev, T.; Baskin, I. Comprehensive Analysis of Applicability Domains of QSPR Models for Chemical Reactions. *Int. J. Mol. Sci.* **2021**, *21*, 5542. [CrossRef] [PubMed]

40. Kaneko, H.; Funatsu, K. Applicability domain based on ensemble learning in classification and regression analyses. *J. Chem. Inf. Model.* **2014**, *54*, 2469–2482. [CrossRef]

41. Rasmussen, C.E.; Williams, C.K.I. *Gaussian Processes for Machine Learning*; MIT Press: Cambridge, MA, USA, 2006.

42. Sushko, I. Applicability Domain of QSAR Models. Ph.D. Thesis, Technical University of Munich, Munich, Germany, 2011.
43. Kamalov, F.; Leung, H.H. Outlier Detection in High Dimensional Data. *J. Inf. Knowl. Manag.* **2020**, *19*, 1–15. [CrossRef]

44. Riahi-Madvar, M.; Nasersharif, B.; Azirani, A.A. Subspace outlier detection in high dimensional data using ensemble of PCA-based subspaces. In Proceedings of the 26th International Computer Conference, Computer Society of Iran, CSICC 2021, Tehran, Iran, 3–4 March 2021. [CrossRef]

45. Kriegel, H.P.; Kr, P.; Schubert, E.; Zimek, A. *Outlier Detection in Axis-Parallel Subspaces of High Dimensional Data*; Springer: Berlin Heidelberg, Germany, 2009; Volume 1, pp. 831–838.

46. Filzmoser, P.; Maronna, R.; Werner, M. Outlier identification in high dimensions. *Comput. Stat. Data Anal.* **2008**, *52*, 1694–1711. [CrossRef]

47. Angiulli, F.; Pizzuti, C. Fast outlier detection in high dimensional spaces. In Proceedings of the Principles of Data Mining and Knowledge Discovery, 6th European Conference PKDD, Helsinki, Finland, 19–23 August 2002; pp. 29–41.

48. Kriegel, H.P.; Schubert, M.; Zimek, A. Angle-based outlier detection in high-dimensional data. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, NV, USA, 24–27 August 2008; pp. 444–452. [CrossRef]

49. Liu, F.T.; Ting, K.M.; Zhou, Z.H. Isolation-based anomaly detection. *ACM Trans. Knowl. Discov. Data* **2012**, *6*, 1–39. [CrossRef]

50. Thudumu, S.; Branch, P.; Jin, J.; Singh, J.J. A comprehensive survey of anomaly detection techniques for high dimensional big data. *J. Big Data* **2020**, *7*, 42. [CrossRef]

51. Xu, X.; Liu, H.; Li, L.; Yao, M. A comparison of outlier detection techniques for high-dimensional data. *Int. J. Comput. Intell. Syst.* **2018**, *11*, 652–662. [CrossRef]

52. Zimek, A.; Schubert, E.; Kriegel, H.P. A survey on unsupervised outlier detection in high-dimensional numerical data. *Stat. Anal. Data Min.* **2012**, *5*, 363–387. [CrossRef]

53. Erfani, S.M.; Rajasegarar, S.; Karunasekera, S.; Leckie, C. High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. *Pattern Recognit.* **2016**, *58*, 121–134. [CrossRef]

54. Alvascience, AlvaDesc (Software for Molecular Descriptors Calculation), Version 2.0.8. 2021. Available online: https://www.alvascience.com (accessed on 1 January 2023).

55. Mauri, A. alvaDesc: A tool to calculate and analyze molecular descriptors and fingerprints. *Methods Pharmacol. Toxicol.* **2020**, *2*, 801–820. [CrossRef]

56. van der Maaten, L.; Hinton, G. Visualizing Data using t-SNE. *J. Mach. Learn. Res.* **2008**, *9*, 2579–2605.

57. Gaussian, Inc. *Gaussian 09*; Gaussian, Inc.: Wallingford, CT, USA, 2010.

58. Montgomery, J.A.; Frisch, M.J.; Ochterski, J.W.; Petersson, G.A. A complete basis set model chemistry. VI. Use of density functional geometries and frequencies. *J. Chem. Phys.* **1999**, *110*, 2822–2827. [CrossRef]

59. Becke, A.D. Thermochemistry. III. The role of exact exchange. *J. Chem. Phys.* **1993**, *98*, 5648–5652. [CrossRef]

60. Miyoshi, A. GPOP Software, Rev. 2022.01.20m1. Available online: http://akrmys.com/gpop/ (accessed on 1 January 2023).

61. Non-Positive Definite Covariance Matrices. Available online: https://www.value-at-risk.net/non-positive-definite-covariance-matrices (accessed on 1 June 2023).

62. Cruz-Monteagudo, M.; Medina-Franco, J.L.; Pérez-Castillo, Y.; Nicolotti, O.; Cordeiro, M.N.D.; Borges, F. Activity cliffs in drug discovery: Dr Jekyll or Mr Hyde? *Drug Discov. Today* **2014**, *19*, 1069–1080. [CrossRef]

63. Fechner, U.; Franke, L.; Renner, S.; Schneider, P.; Schneider, G. Comparison of correlation vector methods for ligand-based similarity searching. *J. Comput.-Aided Mol. Des.* **2003**, *17*, 687–698. [CrossRef]

64. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. *J. Mach. Learn. Res.* **2011**, *12*, 2825–2830.

65. Cao, D.; Liang, Y.; Xu, Q.; Yun, Y.; Li, H. Toward better QSAR/QSPR modeling: Simultaneous outlier detection and variable selection using distribution of model features. *J.-Comput.-Aided Mol. Des.* **2011**, *25*, 67–80. [CrossRef] [PubMed]

66. Insolia, L.; Kenney, A.; Chiaromonte, F.; Felici, G. Simultaneous feature selection and outlier detection with optimality guarantees. *Biometrics* **2022**, *78*, 1592–1603. [CrossRef] [PubMed]

67. Menjoge, R.S.; Welsch, R.E. A diagnostic method for simultaneous feature selection and outlier identification in linear regression. *Comput. Stat. Data Anal.* **2010**, *54*, 3181–3193. [CrossRef]

68. Kim, S.S.; Park, S.H.; Krzanowski, W.J. Simultaneous variable selection and outlier identification in linear regression using the mean-shift outlier model. *J. Appl. Stat.* **2008**, *35*, 283–291. [CrossRef]

69. Jimenez, F.; Lucena-Sanchez, E.; Sanchez, G.; Sciavicco, G. Multi-Objective Evolutionary Simultaneous Feature Selection and Outlier Detection for Regression. *IEEE Access* **2021**, *9*, 135675–135688. [CrossRef]

70. Park, J.S.; Park, C.G.; Lee, K.E. Simultaneous outlier detection and variable selection via difference-based regression model and stochastic search variable selection. *Commun. Stat. Appl. Methods* **2019**, *26*, 149–161. [CrossRef]

71. Wiegand, P.; Pell, R.; Comas, E. Simultaneous variable selection and outlier detection using a robust genetic algorithm. *Chemom. Intell. Lab. Syst.* **2009**, *98*, 108–114. [CrossRef]

72. Tolvi, J. Genetic algorithms for outlier detection and variable selection in linear regression models. *Soft Comput.* **2004**, *8*, 527–533. [CrossRef]

73. Wen, M.; Deng, B.C.; Cao, D.S.; Yun, Y.H.; Yang, R.H.; Lu, H.M.; Liang, Y.Z. The model adaptive space shrinkage (MASS) approach: A new method for simultaneous variable selection and outlier detection based on model population analysis. *Analyst* **2016**, *141*, 5586–5597. [CrossRef]
43. Kamalov, F.; Leung, H.H. Outlier Detection in High Dimensional Data. *J. Inf. Knowl. Manag.* **2020**, *19*, 1–15. [CrossRef]

44. Riahi-Madvar, M.; Nasersharif, B.; Azirani, A.A. Subspace outlier detection in high dimensional data using ensemble of PCA-based subspaces. In Proceedings of the 26th International Computer Conference, Computer Society of Iran, CSICC 2021, Tehran, Iran, 3–4 March 2021. [CrossRef]

45. Kriegel, H.P.; Kr, P.; Schubert, E.; Zimek, A. *Outlier Detection in Axis-Parallel Subspaces of High Dimensional Data*; Springer: Berlin Heidelberg, Germany, 2009; Volume 1, pp. 831–838.

46. Filzmoser, P.; Maronna, R.; Werner, M. Outlier identification in high dimensions. *Comput. Stat. Data Anal.* **2008**, *52*, 1694–1711. [CrossRef]

47. Angiulli, F.; Pizzuti, C. Fast outlier detection in high dimensional spaces. In Proceedings of the Principles of Data Mining and Knowledge Discovery, 6th European Conference PKDD, Helsinki, Finland, 19–23 August 2002; pp. 29–41.

48. Kriegel, H.P.; Schubert, M.; Zimek, A. Angle-based outlier detection in high-dimensional data. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, NV, USA, 24–27 August 2008; pp. 444–452. [CrossRef]

49. Liu, F.T.; Ting, K.M.; Zhou, Z.H. Isolation-based anomaly detection. *ACM Trans. Knowl. Discov. Data* **2012**, *6*, 1–39. [CrossRef]

50. Thudumu, S.; Branch, P.; Jin, J.; Singh, J.J. A comprehensive survey of anomaly detection techniques for high dimensional big data. *J. Big Data* **2020**, *7*, 42. [CrossRef]

51. Xu, X.; Liu, H.; Li, L.; Yao, M. A comparison of outlier detection techniques for high-dimensional data. *Int. J. Comput. Intell. Syst.* **2018**, *11*, 652–662. [CrossRef]

52. Zimek, A.; Schubert, E.; Kriegel, H.P. A survey on unsupervised outlier detection in high-dimensional numerical data. *Stat. Anal. Data Min.* **2012**, *5*, 363–387. [CrossRef]

53. Erfani, S.M.; Rajasegarar, S.; Karunasekera, S.; Leckie, C. High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. *Pattern Recognit.* **2016**, *58*, 121–134. [CrossRef]

54. Alvascience, AlvaDesc (Software for Molecular Descriptors Calculation), Version 2.0.8. 2021. Available online: https://www.alvascience.com (accessed on 1 January 2023).

55. Mauri, A. alvaDesc: A tool to calculate and analyze molecular descriptors and fingerprints. *Methods Pharmacol. Toxicol.* **2020**, *2*, 801–820. [CrossRef]

56. van der Maaten, L.; Hinton, G. Visualizing Data using t-SNE. *J. Mach. Learn. Res.* **2008**, *9*, 2579–2605.

57. Gaussian, Inc. *Gaussian 09*; Gaussian, Inc.: Wallingford, CT, USA, 2010.

58. Montgomery, J.A.; Frisch, M.J.; Ochterski, J.W.; Petersson, G.A. A complete basis set model chemistry. VI. Use of density functional geometries and frequencies. *J. Chem. Phys.* **1999**, *110*, 2822–2827. [CrossRef]

59. Becke, A.D. Thermochemistry. III. The role of exact exchange. *J. Chem. Phys.* **1993**, *98*, 5648–5652. [CrossRef]

60. Miyoshi, A. GPOP Software, Rev. 2022.01.20m1. Available online: http://akrmys.com/gpop/ (accessed on 1 January 2023).

61. Non-Positive Definite Covariance Matrices. Available online: https://www.value-at-risk.net/non-positive-definite-covariance-matrices (accessed on 1 June 2023).

62. Cruz-Monteagudo, M.; Medina-Franco, J.L.; Pérez-Castillo, Y.; Nicolotti, O.; Cordeiro, M.N.D.; Borges, F. Activity cliffs in drug discovery: Dr Jekyll or Mr Hyde? *Drug Discov. Today* **2014**, *19*, 1069–1080. [CrossRef]

63. Fechner, U.; Franke, L.; Renner, S.; Schneider, P.; Schneider, G. Comparison of correlation vector methods for ligand-based similarity searching. *J. Comput.-Aided Mol. Des.* **2003**, *17*, 687–698. [CrossRef]

64. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. *J. Mach. Learn. Res.* **2011**, *12*, 2825–2830.

65. Cao, D.; Liang, Y.; Xu, Q.; Yun, Y.; Li, H. Toward better QSAR/QSPR modeling: Simultaneous outlier detection and variable selection using distribution of model features. *J.-Comput.-Aided Mol. Des.* **2011**, *25*, 67–80. [CrossRef] [PubMed]

66. Insolia, L.; Kenney, A.; Chiaromonte, F.; Felici, G. Simultaneous feature selection and outlier detection with optimality guarantees. *Biometrics* **2022**, *78*, 1592–1603. [CrossRef] [PubMed]

67. Menjoge, R.S.; Welsch, R.E. A diagnostic method for simultaneous feature selection and outlier identification in linear regression. *Comput. Stat. Data Anal.* **2010**, *54*, 3181–3193. [CrossRef]

68. Kim, S.S.; Park, S.H.; Krzanowski, W.J. Simultaneous variable selection and outlier identification in linear regression using the mean-shift outlier model. *J. Appl. Stat.* **2008**, *35*, 283–291. [CrossRef]

69. Jimenez, F.; Lucena-Sanchez, E.; Sanchez, G.; Sciavicco, G. Multi-Objective Evolutionary Simultaneous Feature Selection and Outlier Detection for Regression. *IEEE Access* **2021**, *9*, 135675–135688. [CrossRef]

70. Park, J.S.; Park, C.G.; Lee, K.E. Simultaneous outlier detection and variable selection via difference-based regression model and stochastic search variable selection. *Commun. Stat. Appl. Methods* **2019**, *26*, 149–161. [CrossRef]

71. Wiegand, P.; Pell, R.; Comas, E. Simultaneous variable selection and outlier detection using a robust genetic algorithm. *Chemom. Intell. Lab. Syst.* **2009**, *98*, 108–114. [CrossRef]

72. Tolvi, J. Genetic algorithms for outlier detection and variable selection in linear regression models. *Soft Comput.* **2004**, *8*, 527–533. [CrossRef]

73. Wen, M.; Deng, B.C.; Cao, D.S.; Yun, Y.H.; Yang, R.H.; Lu, H.M.; Liang, Y.Z. The model adaptive space shrinkage (MASS) approach: A new method for simultaneous variable selection and outlier detection based on model population analysis. *Analyst* **2016**, *141*, 5586–5597. [CrossRef]
74. t-SNE: The Effect of Various Perplexity Values on the Shape. Available online: https://scikit-learn.org/stable/auto_examples/manifold/plot_t_sne_perplexity.html (accessed on 1 June 2023).

75. Wu, Z.; Pan, S.; Chen, F.; Long, G.; Zhang, C.; Yu, P.S. A Comprehensive Survey on Graph Neural Networks. *IEEE Trans. Neural Netw. Learn. Syst.* **2021**, *32*, 4–24. [CrossRef] [PubMed]

76. Xu, K.; Jegelka, S.; Hu, W.; Leskovec, J. How powerful are graph neural networks? In *Proceedings of the 7th International Conference on Learning Representations*, ICLR 2019, New Orleans, LA, USA, 6–9 May 2019; pp. 1–17.

77. Jiang, D.; Wu, Z.; Hsieh, C.Y.; Chen, G.; Liao, B.; Wang, Z.; Shen, C.; Cao, D.; Wu, J.; Hou, T. Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models. *J. Cheminform.* **2021**, *13*, 1–23. [CrossRef] [PubMed]

**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
74. t-SNE: The Effect of Various Perplexity Values on the Shape. Available online: https://scikit-learn.org/stable/auto_examples/manifold/plot_t_sne_perplexity.html (accessed on 1 June 2023).

75. Wu, Z.; Pan, S.; Chen, F.; Long, G.; Zhang, C.; Yu, P.S. A Comprehensive Survey on Graph Neural Networks. *IEEE Trans. Neural Netw. Learn. Syst.* **2021**, *32*, 4–24. [CrossRef] [PubMed]

76. Xu, K.; Jegelka, S.; Hu, W.; Leskovec, J. How powerful are graph neural networks? In *Proceedings of the 7th International Conference on Learning Representations*, ICLR 2019, New Orleans, LA, USA, 6–9 May 2019; pp. 1–17.

77. Jiang, D.; Wu, Z.; Hsieh, C.Y.; Chen, G.; Liao, B.; Wang, Z.; Shen, C.; Cao, D.; Wu, J.; Hou, T. Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models. *J. Cheminform.* **2021**, *13*, 1–23. [CrossRef] [PubMed]

**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.