# Statistical Validation Framework for Automotive Vehicle Simulations Using Uncertainty Learning
## 7 Validation and Discussion of the Framework  
This section validates the uncertainty prediction method by using additional test data and to critically discuss the methods used in the framework. The uncertainty prediction method is considered to be valid if the predicted uncertainty is higher than or equal to the corresponding measured uncertainty. For practical use, a small interval is targeted. Since no interval has existed in the automotive domain until now, calculating any interval already improves validation in the automotive domain. Thus, this paper focuses more on a conservative prediction, while additionally making suggestions as to how to keep the predicted interval small. Two of the test parameter configurations shown in Table 3 are measured to validate the extrapolation capability, as well as the ten training parameter configurations. All three parameters \(\theta_{\text{p}}^{\text{a}}=[\hat{M}_{\text{veh}}\ \hat{c}_{\text{d}}\ \hat{p}\text{y}\text{r}]\) are changed randomly at the same time. The left side of Figure 10 shows the predicted total output uncertainty of the two application configurations in form of the p-box \(B_{\hat{p}\text{a}}\) and the 95% total prediction interval \(I_{\text{p}\text{a}}^{95}\). Using the measurements \(y_{\text{s},1-3}^{\text{a}}\) instead of the prediction model \(G_{\text{pred}}\) to calculate the real \(u^{\text{v}}\), the real total output uncertainty \(B_{\text{p}\text{a}}\) and the corresponding interval \(I_{\text{Ya}}^{95}\) can be calculated. It can be observed that the predicted values are larger and include the measured uncertainty. Consequently, this prediction method is defined as being valid. The overall result of the whole output of the framework is a confidence interval of the SRQ together with a high-information p-box. The result of a conventional validation method is a single value. Since no uncertainty learning is used in conventional validation, there is no information about the validity of the individual use-case in the application domain. Declaring this single value to be a valid result would be conjecture. Comparing both validation outputs, the VV+UQ framework provides a more precise reliability quantification across a large application domain and improves conventional validation. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**No.** & \(\hat{M}_{\text{veh}}\ \mathbf{in}\ \mathbf{kg}\) & \(\hat{c}_{\text{d}}\) & \(\hat{\rho}_{\text{yere}}\ \mathbf{in}\ \mathbf{bar}\ \mathbf{bar}\ \mathbf{\hat{u}}^{\text{v}}\ \mathbf{in}\ \mathbf{Wh}\) & \(\hat{\rho}_{\text{y}}^{\text{a}}\ \mathbf{in}\ \mathbf{Wh}\) & \(y_{\text{s},1-3}^{\text{a}}\ \mathbf{in}\ \mathbf{Wh}\) & \(u^{\text{v}}\ \mathbf{in}\ \mathbf{Wh}\) & \(I_{\text{p}\text{a}}^{95}\ \mathbf{in}\ \mathbf{Wh}\) \\ \hline
1 & \([925,933]\) & 0.32 & 3.0 & 91.40 & \(1171\pm 14\,\%\) & \((1134,1124,1121)\) & 38.74 & \(1171\pm 9\%\) \\
2 & \([1225,1233]\) & 0.47 & 2.0 & 76.29 & \(1584\pm 10\,\%\) & \((1533,1524,1518)\) & 53.93 & \(1584\pm 8\%\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Test data set with model uncertainties of WLTC depending on application parameters \(\theta^{\text{a}}\). Furthermore, additional aspects of the validation framework are discussed. The first is the framework's efficiency, where the measuring and computational costs of the validation are related to the costs saved through the reuse of the gathered information. As described in Table 4, it will take approximately 100 h on the test bench and 2.5 h on the described cluster to measure all validation parameter configurations on the test bench, validate them, train a prediction model and make the first prediction. To demonstrate the advantage of the framework, which enables the reuse of the data, three scenarios are considered. Because the data are reused, any future prediction takes about 15 min of calculation on the cluster instead of 10 h on the test bench. If a