# Statistical Validation Framework for Automotive Vehicle Simulations Using Uncertainty Learning
## 3 Statistical Validation Method
### Detailed Explanation  
The validation framework is explained in detail in Figure 4. The aim is to give an overview of the framework and to introduce symbols and the nomenclature. The framework considers a model \(G_{\rm m}\) shown in Equation (2). Based on the input of time-variant signals \(x\) and the time-invariant parameters \(\theta\), the simulation of the model mimics the behaviour of the system \(G_{\rm s}\) to approximate the system output \(\text{SRO}\ y_{\rm s}\). The simulation result \(y_{\rm m}\) is an estimate of the measured \(\text{SRO}\ y_{\rm s}\). As shown in Figure 2, all inputs and outputs of a model can be single-value, aleatory, epistemic or mixed. Before starting with the actions in Figure 4, the model must be verified and the model inputs and their uncertainties quantified. For detailed information on how to estimate and calibrate uncertain parameters, please refer to the four publications [1,4,30,43]. In the validation domain, the results of a real system \(y_{\rm s}^{\rm v}\) measured during a fixed validation parameter configuration \(\theta^{\rm v}\) are compared to the results of the model \(y_{\rm m}^{\rm v}\). A validation metric \(G_{\rm val}\) compares both results to get an estimate of the model error, which leads to the model uncertainty \(u^{\rm v}\). A non-deterministic model is needed to consider the uncertain model inputs. To cover the validation domain, several validation parameter configurations concluded in the validation configurations \(\boldsymbol{\Theta}^{\rm v}\) must be validated by estimating the corresponding model uncertainties concluded in \(\boldsymbol{u}^{\rm v}\). These tests are needed to enable validation uncertainty learning \(G_{\rm lea}\)  
Figure 3: Predicting the future reliability of an application domain by inter- and extrapolation from a validation domain based on [PERSON] et al. [30].  
The result reliabilities of new application parameter configurations \(\theta^{\text{a}}\) are estimated in the application domain. This process can be called the validation of an application parameter configuration. Because no system is available to provide real measurements \(y^{\text{a}}_{s}\), it is necessary to validate the result by approximating the model uncertainty \(\hat{a}^{\text{a}}\) through the validation uncertainty prediction \(G_{\text{pred}}\). This uses the data from uncertainty learning \(G_{\text{lea}}\) to inter- or extrapolate to the model uncertainty \(\hat{u}^{\text{va}}\) of the current application parameter configuration \(\theta^{\text{a}}\). The approximated model uncertainty \(\hat{a}^{\text{a}}\) and the simulation result of the application configuration \(y^{\text{a}}_{\text{m}}\) need to be integrated with the integration model \(G_{\text{int}}\) to calculate the total prediction uncertainty \(\hat{y}^{\text{a}}\). An application decision-making \(G^{\text{a}}_{\text{dec}}\) decides if the result of the total prediction uncertainty is accurate enough for the intended use case. The basic structure of the framework shown in Figure 4 stays the same. There are several possibilities to conduct the individual steps of the framework that are interchangeable, which leads to some modularity. Several variants, combinations and advantages of them are explained in the review of [PERSON] et al. [30]. One variant with all steps will be explained and validated in detail with real data in this paper.