# Statistical Validation Framework for Automotive Vehicle Simulations Using Uncertainty Learning
## 1 Introduction  
The automotive development process is both highly complex and is becoming more diverse thanks to increasing possibilities in modelling, computing and analysing, especially in the field of new electric powertrain systems [1, 2, 3]. The early stages of the process offer a great deal of freedom when the development comes to realising the vehicle specifications [4]. In order to compare the different options, evaluating as many properties of the final product as possible [5] is important. As today's research and development work increasingly depends on predictions based on system simulations [6], their reliability quantification is essential for creating new knowledge, the transparent evaluation of options for action and to support safety-relevant decisions based on simulation results [7]. The modelling and simulation (M+S) process and analysis methods used in automotive development are undergoing a transformation and new requirements are emerging. An increasing system complexity and growing variant diversity lead to more complex and modular simulation models. This transformation needs virtual vehicle development, testing and approval in a large application domain [1, 8]. Better knowledge of systems, such as the ability to precisely quantify uncertain parameters, is needed to ensure a more accurate understanding of systems that affect the safety and efficiency of complex powertrains [9, 10] and non-linear dynamic systems [11, 12, 13]. As described in [PERSON] et al. [1], the new keyrequirements of validation include a more precise reliability quantification of simulation results in a large application domain, achieved at reasonable cost and time. What is meant by large application domain is a precise evaluation of the simulation's output reliability under changing system parameters, system scenarios or system topologies. Part of the information can be obtained from existing verification and validation (V+V) processes. Nonetheless, the information is not enough to fulfil the new requirements, because their validation provides scant information on the reliability of the simulation results [14; 15; 16]. [PERSON] et al. [1] identified four key insufficiencies that prevent conventional validation methods from meeting the new requirements:  
1. the negligence of uncertainty,
2. the binary, low-information validation result,
3. the low extrapolation capability of model reliability,
4. the small application domain size. This paper addresses those insufficiencies by developing a statistical verification, validation and uncertainty quantification (VV+UQ) framework. The framework focuses on dynamic systems with changing parameter configurations, thus enabling a flexible validation of complex total vehicle simulations including powertrain and consumption modelling. It uses non-deterministic models to include input uncertainties, applies uncertainty learning to predict inherent model uncertainties and thus enables the precise reliability quantification of arbitrary system parameter configurations to form a large application domain. The main contributions are:  
* Summarising the insufficiencies of validation in the automotive domain, which prevent a reliability assessment of simulation models and system safety. * A statistical VV+UQ framework with uncertainty learning for the precise validation of a large application domain. * First application of a statistical VV+UQ framework predicting model uncertainties of new parameter configurations. * Explanation, validation and discussion of the framework with real world data from a prototype electric vehicle on a roller dynamometer. * Solving the new key requirements of automotive validation and recommending four improvement strategies to allow efficient error targeting for revising automotive M+S processes and total system understanding. After defining validation and summarizing the problems in Section 2, the basic VV+UQ framework and the test system setup are explained in Sections 3 and 4.