## 3 Results and Discussion
### Evaluation with the Reference Network in the Frequency Domain  
As described above, the data were recorded on four different dates. A k-fold cross-validation is applied for the following evaluation. This cross-validation represents a common technique for evaluating a model in machine learning. Here, the input data are partitioned and multiple models are trained with the partial data sets. The remaining part of the respective input data serves as a test data set. The overall success rate is calculated from the average of the individual success rates. This procedure prevents overfitting during training [21]. With respect to the present use case, this means that four networks are built up, each of which is based on the sound data of three different measurement dates, while the forth is used for testing. Table 3 gives an overview of the success rate achieved in the classification of the nozzle state with the reference network. Table 3 shows that the nozzle condition was correctly determined in 63% of the classifications by the network. The success rate obtained is the average of all four measurement dates. The success rate of each individual measurement date in turn represents an average value, which refers to ten separately trained ANNs. In addition to these mean values, the minimum and maximum of each measurement date are also shown in Table 3. Particularly noticeable is measurement date 3, where only 49.4% of all nozzle states can be correctly classified. The success rate of the other three measurement dates, on the other hand, is significantly higher. Therefore, for a more in-depth analysis, a confusion matrix of the test data from measurement date 3 is necessary. As the difference between minimum and maximum is small, the ANN with the lowest success rate of all ten trained networks is utilized as a worst case study. Figure 5 shows the confusion matrix of measurement date 3. A confusion matrix allows us to represent the results of a classification problem more comprehensively. The  
Figure 4: Spectral analysis for a shot with a functional and a clogged nozzle. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Date** & **Success Rate** & **(Min/Max)** \\ \hline Date 1 & 64.6\% & (Min: 53.4\%, Max: 85.6\%) \\ Date 2 & 72.2\% & (Min: 61.5\%, Max: 82.8\%) \\ Date 3 & 49.4\% & (Min: 48.3\%, Max: 51.7\%) \\ Date 4 & 65.7\% & (Min: 57.0\%, Max: 71.1\%) \\ \hline
**Average** & 63.0\% & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Correct classifications for each measurement date based on the training with data acquired on the remaining three dates. columns of the matrix represent the given target class, whereas the rows represent the output class determined from the ANN. Target class 1 represents a functional nozzle and Target class 2 represents a clogged nozzle. The correctly classified nozzle states can be taken from the main diagonal of the matrix. Of the total 120 nozzle shots, 58 were correctly identified, which is consistent with the success rate of the minimum from Table 3. The advantage of a confusion matrix is primarily the visualization of the misclassified nozzle states, which distinguishes between false positive and false negative errors [22]. The lower left red box of the matrix shows the number of nozzles that were functional during the experiments but were classified as clogged by the network. The upper right red field, on the other hand, shows the number of nozzles that were clogged during the experiments but were incorrectly detected as functional by the network. By dividing the falsely classified nozzles into false positive and false negative errors, it becomes clear that the network evaluates almost all nozzles as clogged, regardless of the actual nozzle condition. All 61 nozzles that were \"functional\" when the sound was recorded are misclassified. The reason for the large number of false evaluations could be the level of the amplitudes in the time domain.