## 3 Crash Prediction Testing Methodology
### Test Scenario Catalog Development  
#### 3.5.1 Machine Learning Scenario Extraction  
One possibility for the extraction of the relevant test scenarios from the acquired accident database is the explorative machine learning approach. Since our goal is to investigate the occurrence of collision configurations in real-world accident data, no external labels should be defined for the data. Thus, an unsupervised machine learning approach without data labeling was chosen. Furthermore, the property of clustering to organize similar cases into one group makes the extraction of representative test scenarios possible. The clustering aim and input data selection are derived from the test scenario requirements previously defined in step C. The proposed scenario clustering methodology is shown schematically in Figure 6.  
Depending on the database and the selection of the input features, both accident scenarios and critical situations can be grouped by cluster analysis. However, the selected input features directly determine the clustering results. To test a Pre-Crash function, both the Pre-Crash and In-Crash information in the test scenario are required. The scale level of the different features should be considered. However, the characteristics of the ratio scale features are well suited for cluster analyses since they show rankings and interpretable distances and can be transformed in the preprocessing. Thus, variables of both participants,  
Figure 6: Novel unsupervised machine learning methodology for creating concrete and logical test scenario catalogs. such as collision velocity, impact angle, floating angle, impact point along the contour of the vehicles, vehicle mass and other In-Crash variables, can be used directly for collision configuration clustering. Additionally, the k-nearest neighbors (k-NN) algorithm is utilized to determine the representative cases within the clusters. Concrete representative cases from each cluster are selected to create a concrete test scenario catalog. For logical test scenarios, the parameter ranges of features used for scenario representation should be limited by a standard deviation of these features within the clusters. Through k-NN, the selection of more than one cluster representative, as a cluster center, is possible. In this way, test case catalogs with both concrete and logical scenarios can be created. Proof of concept clustering on the GIDAS dataset of vehicle to vehicle crashes, collected in the years 2000 to 2019, was performed to demonstrate the challenges in the evaluation of clustering results. The goal of this clustering was to identify representative collision configurations and to find the optimal number of clusters, to reduce the test effort of the system without losing the relevant information. The dataset was filtered by the area of action of the function under tests. A sample of 20,239 cases on the passenger level was used for clustering. A K-means++ clustering algorithm, which were introduced by [PERSON] and [PERSON] [31], was utilized. The difference to the conventional K-means algorithm is the initialization method of the cluster centers, which helps to increase the cluster quality and reduces the risk of falling into local data minima. The feature input set consisted of ten overall collision configuration features, such as the impact points, collision velocity, collision angle and technical crash severity dv for both the host and opponent vehicles. Additionally, the input features have been standardized. In unsupervised learning, no labels are used, which makes the evaluation of the clustering results more complicated. The external clustering validity criteria cannot be used here because no labeling of the generated clusters is available. Relative and internal criteria, such as distortion and silhouette scores, are applied to assess the cluster validity. Figure 7 shows the distortion and silhouette scores for the clustering results, from 1 to 100 clusters.