# Statistical Validation Framework for Automotive Vehicle Simulations Using Uncertainty Learning
## 2 Model Validation
### Validation Processes  
There are widely accepted V+V processes to validate the M+S process. They focus on practical approaches and less on the truth in the philosophy of science [21]. As shown in Figure 1, [PERSON] [27] developed a V+V process and defined operational validity as the determination of whether the output behaviour of the model has a satisfactory range of precision for its intended purpose within the area of intended applicability. The V+V process of [PERSON] is similar to the approaches of [PERSON] and [PERSON] [28] and the ASME Guide [24]. This V+V process is one of the most broadly acknowledged in the validation of M+S [16] and will be referred to as the conventional process in this paper. Because conventional validation is strongly oriented towards its practicability, statistical validation methods are emerging that cover a better fulfilment of truth in the philosophy of science, while maintaining the necessary practicability in engineering [21]. Statistical validation assumes a natural true value \(y_{\text{nature}}\) of a real system's or entity's  
Figure 1: Simplified model development process according to [PERSON] [29]. output response quantity of interest (SRQ). This value is often called ground truth or true value [30]. When the output value \(y_{\text{s}}\) of the system \(G_{\text{s}}\)  
\[G_{\text{s}}:(x,\theta) \mapsto y_{\text{s}} \tag{1}\] \[G_{\text{m}}:(x,\theta) \mapsto y_{\text{m}} \tag{2}\]  
is measured, there will always be an error \(e_{y_{\text{s}},\text{obs}}\) between observation and true value. The simulation model \(G_{\text{m}}\) estimates the true value of the nature of the SRQ based on the input of time-variant signals \(x\) and the time-invariant parameters \(\theta\) through its output value \(y_{\text{m}}\) (2). The errors \(e\) indicate the discrepancy between the simulated and true value. They are inherent in every simulation model because a model is by definition a simplified abstraction of reality [26]. One source of errors results from the calculation of finite accuracy by computers compared to the exact solution of the mathematical model \(e_{\text{h}}\). A further source of error results from the model time-variant inputs \(e_{\text{x}}\) and the time-invariant parameters \(e_{\theta}\). Whereas they are usually assumed to be fully characterized in a conventional validation of deterministic simulations, these errors are mostly unknown in reality. Moreover, the model itself contains model form errors \(e_{\text{m}}\) due to the selection of an unsuitable equation. They can be quantified based on physical data from model validation experiments. Therefore, the experimental results of the physical system \(G_{\text{s}}\) and the simulation results of the model \(G_{\text{m}}\) are compared using validation metrics [31]. As defined in ASME [32], the involved errors can be summarized as:  
\[y_{\text{nature}} =G_{\text{s}}(x,\theta)-e_{y_{\text{s}},\text{obs}} \tag{3}\] \[y_{\text{nature}} =G_{\text{m}}(x,\theta,h)-(e_{\text{m}}+e_{\text{x}}+e_{\theta}+ e_{\text{h}}) \tag{4}\]  
The basic theory behind statistical methods states that because those errors cannot be exactly estimated, they need to be approximated through uncertainties, which are expressed in probability functions or confidence intervals [33]. The various sources of uncertainty are deeply interwoven and affect each other. Depending on the constellation, they may reinforce or compensate each other, which can result in a misleading trustworthiness of the simulation model ([21], p. 385). It is therefore helpful to quantify the different uncertainties separately. Additional sources such as observation errors \(e_{y,\text{obs}}\) or extrapolation errors caused by model predictions beyond the validity range make this even more difficult. As explained in Figure 2, the uncertainties can be point valued, if accurately known, epistemic, resulting in an interval, aleatory, resulting in a cumulative distribution function (CDF), or mixed, resulting in a probability box (p-box).