# Statistical Validation Framework for Automotive Vehicle Simulations Using Uncertainty Learning
## 7 Validation and Discussion of the Framework
\begin{table}
\begin{tabular}{l c c} \hline \hline  & **Time** & **Location** \\ \hline Measure one configuration three times & 10 h – min – s & Test bench \\ Calculate model uncertainty of one config. & 14 min 13 s & Cluster \\ Calculate model uncertainty of all ten configs. & 2 h 22 min 18 s & Cluster \\ Train uncertainty learning model & 6 s & Cluster \\ Predict new parameter configuration & 14 min 32 s & Cluster \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Time needed to measure the parameter configurations on the roller dynamometer and to calculate them on a cluster with 40 Intel Xeon Processors at 2.4 GHz. Figure 10: Validation of the uncertainty prediction method showing the 95% confidence interval of the total output uncertainty of the two test application parameter configurations. The left side shows the predicted total uncertainty \(B_{\gamma_{\text{m}}}\) using the predictions of \(G_{\text{pred}}\) to predict the model uncertainty \(\hat{a}^{\text{a}}\). The right side shows the measured total uncertainty \(B_{\gamma_{\text{m}}}\) using additional measurements and the AVM to calculate the real model uncertainty \(u^{\text{v}}\). more sophisticated model with more uncertain parameters were to be implemented, the validation measurements can be reused to validate the new model. Nevertheless, the model uncertainty and the prediction model must be recalculated. It should be noted that although the new model might have more parameters and more uncertainties, the needed samples for each configuration will remain the same because relevant value is the output distribution of the SRQ (Section 5.3). If a prediction of new predictor parameters is needed, such as the consideration of the ambient temperature, additional validation parameter configurations must be measured and evaluated. The initial measurements can be reused. The next point is to discuss the amount of data points used for the prediction model. The minimum amount of data points \(n\) is defined in Equation (14), meaning that if only one parameter is to be predicted, at least three \(p<n\) validation configurations are needed. If no prediction is required, only one validation configuration is needed. Because the number of data points, the error of the response surface and the distance between training data and prediction are considered, a low quality fit with few training data and a large extrapolation range will lead to a high prediction uncertainty. If this uncertainty is unfeasibly high, more learning data must be added by measuring additional validation configurations. The framework becomes more valuable whenever more data about the system are available, because the predictions become more precise and narrower. As can be seen in the previous example in Figure 9, the prediction is narrow enough for the considered application. In this case, the ten learning points of the validation domain are enough. Finally, the method used for the learning model is discussed. If the prediction interval is not precise enough, there is the possibility to improve the learning model. As confirmed by the literature in Section 5.5, basic GP and PCE do not meet the requirements. Nevertheless, more sophisticated GPs can be used if they are combined with kernels for pattern discovery [58; 63]. Moreover, an improved PCE method can be used if it is expanded with a confidence interval, which can be constructed by a bootstrap method [59]. It should be noted that these algorithms need more validation data configurations, which significantly increases the validation effort considering that one data point needs 10 h of measuring time on the roller dynamometer. As the goal of the paper is to prove the feasibility and validity of the framework, the chosen extrapolation method is reasonable.