## 3 Results
### Basic Model Scores for Unfiltered and Filtered Datasets
However, the high difference to the mean value indicates a high variation in the randomly initialized model runs and, therefore, a less robust model. All four datasets that include FFT features yield the worst scores for both kinds of datasets. All datasets except DS 3 profited from the reduction in data complexity. No significant difference can be seen in the use of composition features for both kinds of datasets. Two more values extend the view of the model performance. The first one is the mean MAE divided by the standard deviation of DM in the corresponding dataset, which allows comparing the model performance to the variation of the target value. The standard deviation and statistical evaluation of DM are depicted in Figure 11. The second one is the coefficient of determination R\({}^{2}\), which estimates the models' level of abstraction regarding the cause-effect correlations toward the target value. Figure 14 summarizes these metrics for all twelve models shown in Figure 13 and adds the highest achieved R\({}^{2}\)-value by any model for each dataset. DS 9 reaches the best R\({}^{2}\)-score for the models that performed best on average, which is also the closest to its highest-performing model. This low deviation between the best model on average and the best model overall indicates a robust parameter set. DS 11, which uses time-based features, achieves the overall highest R\({}^{2}\)-score. However, the difference between its mean best and overall best models is far higher than for DS 9. The highest achieved R\({}^{2}\)-score for any model is above 0.75, contrary to the diminishing scores shown in Figure 13. The increasing difference can be interpreted as less robust models. The mean MAE of the models depicted in Figure 13 divided by the target value's standard deviation ranges between 0.34 and 0.57. All model prediction errors variate far less than the target value in the datasets. Figure 14: Evaluation of R\({}^{2}\) and the mean MAE compared to the standard deviation (stdv) of the true values of DM. All values refer to the model predictions for the test data. Figure 15 illustrates the results for the validation data. Two sets of models are included. The yellow, full line datapoints represent the same models as in Figure 13. The green, separated line datapoints depict the model that achieved the lowest mean RMSE for each dataset for the validation data. These are different models than those in Figure 13. The best achieved RMSE of the ten random initialized runs is also included for both sets of models. An extreme variation in the results can be seen for DS 4, where the difference between the mean and best value is significant compared to the other datapoints. The same datasets achieve the lowest mean RMSE as for the test data, only with slight differences. However, all models perform far worse for the validation data than for the test data. They still show that the same models performing best for the training data also achieved the best values for the validation data. This fact indicates that these robust models can predict some of the outcomes of the validation dataset. However, it can be suspected that they are missing relevant information unique about sand-binder system D in the training process. On the other hand, other models resulted in better predictions for the validation data. These models were less accurate for the test data than the previously discussed models but have generalized more information that allows for better prediction values of the validation data. However, even these models are still significantly worse than the mean model scores for the train data. In summary, the trained models are partially suited for the validation data. Still, not all relevant influences on system D were included in the other systems used to train the models. Table 3 evaluates the results statistically, as described in the approach chapter. It lists the average, the standard deviation, and the \(p\)-value obtained by the two-sided [PERSON]'s \(t\)-test for each dataset.