# Applying Heuristics to Generate Test Cases for Automated Driving Safety Evaluation
## 2 Heuristics Methodology
### Evaluation of Severity and Exposure  
Safety relevance of a scenario was considered according to its potential consequences in the case of failure (severity) and the likelihood to occur in real traffic (exposure), as introduced in [33]. Criticality is, therefore, associated with both severity and exposure, implying that it can either result from consequences (in the case of failure), unfavorable handling at some point of the system's lifetime, or a scenario frequently occurring in real traffic. Consequently, safety relevant scenarios are severe, frequent, or a combination of both. The test cases towards severity aim to meet the former, while the test cases towards exposure aim to meet the latter. By explicitly targeting frequently occurring scenarios, the necessary breadth of testing is reached, and it is ensured that the system is capable of dealing with all occurring scenarios.  
Severity was evaluated by means of risk potential field (RPF), in accordance with [34; 35]. The RPF is a Bayesian network-based trajectory planner that calculates the overall risk potential of the vehicle's possible paths. For the required application of the database and the corresponding evaluation of test cases, the RPF evaluation was limited to the path associated with the analysed test case, the driving state of the vehicle under test (VuT) (as defined in [26]), and its interaction with the challenging vehicle (chall) (represented by the parameters characterizing the scenario, as in [36]).  
Exposure was addressed by targeting the full coverage of the patterns (parameter's individual distribution functions as well as the parameter dependencies from the regression models) found in the data set. Therefore, an evaluation was conducted to compare the previously assigned test cases and the newness of a parameter combination of a test case candidate. In order to avoid redundancy between different test cases and to optimize the coverage of the data, an objective measure of newness was incorporated.  
The newness evaluation of a test case candidate was conducted in several steps, by comparing each test case candidate with all previously assigned test cases (e.g., to define the fourth test case, a test case candidate was compared with the three test cases previously assigned to the test suite). For each Parameter (Param), the value difference between the test case candidate (cand) and the compared test case (comp) was divided by the parameter distribution range from the total data set, according to (1). Therefore, the equation provides values close to zero for similar candidates and compared parameters, as well as values close to one for very different parameters.  
\[\text{Newness}_{\text{param}}\ =\ \frac{\text{abs}(\text{Param}_{\text{cand}}- \text{Param}_{\text{comp}})}{(\text{Param}_{\text{max, data}}\ -\ \text{Param}_{\text{min, data}})} \tag{1}\]  
Based on the newness value for each individual parameter, the average newness value of the parameters that defined a test case was calculated. The minimal average newness of all parameters that defined a test case candidate was regarded as relevant, in comparison to all previously assigned test cases (e.g., the relevance of a candidate to become the fourth test case in the final test suite is judged based on the minimum newness value out from the comparison with each of the three previously assigned test cases).