# Applying Heuristics to Generate Test Cases for Automated Driving Safety Evaluation
## 2 Heuristics Methodology
### Data Split, Distribution Fitting, and Regression Analysis  
The scenario data set was split into two subsets, representative for critical and normal driving scenarios, respectively. Thereafter, for each subset, distributions were fitted to the data and analysed through regression models. These distributions and models served as input for the application of the heuristic methodology. The resulting regression models allowed us to consider parameter dependencies in the test case generation and to reflect the observed data patterns of the underlying naturalistic driving data. In that way, realism in the test case generation is accounted for in two manners: first, as naturalistic driving data build the basis for the test case generation procedure, it is ensured that no synthetic parameter values are applied. Moreover, by abstracting the original parameter values with distribution functions, no outlier parameter values are used in the test case generation.  
Second, current state-of-the-art approaches [24; 27] set parameter values in the test case generation independently, neglecting the fact that dependencies exist. Its importance has already been introduced in [26] and is now considered in this proposed methodology.  
With the severity evaluation of each scenario, by means of RPF, a relative split of the data set was conducted. For the current study, a 10% threshold was applied to split the relatively more severe data from the normal driving data. The threshold was determined iteratively during the methodology development, as a trade-off between the large number of scenarios in the transition from critical to normal driving scenarios, as well as the limited data sample size that can affect the statistical reliability of the results. Future work should address the threshold's value and kind (relative or absolute split), by analyzing multiple data sets and scenario types.  
From the distribution fitting, the resulting range, as well as the frequency of occurrences, were derived for each parameter. Univariate, generalized, extreme value distributions were applied to consider both skewness and sharpness at the parameter range boundaries. This resulted in parameter values for the expected value (\(\mu\)), the standard deviation (\(\sigma\)), and the skewness (\(k\)). The minimum and maximum of each individual parameter range, during the test case generation, was determined as the 0.1 and 99.9 percentiles of the corresponding distribution, as in [37].  
For the regression analysis, four types of independent parameter relations (constant, linear, pairwise-linear, and quadratic) were considered. The selection of these relations was conducted based on physical plausibility. For example, a second order dependency between distances and accelerations is physically plausible, therefore justifying the incorporation of quadratic relationships in the analysis. By this, the particularity of the AD systems is addressed, as there are physical dependencies between parameters, which is not generally applicable, in the context of software testing. Only the terms that improved the R\({}^{2}\)-value of the model by at least 0.1 were incorporated in the final regression models. The implementation was conducted by a stepwise regression, to assess all potential dependencies, while removing those that did not meet the pre-defined chosen criteria.