## 5 Classification Performed by Machine Learning Algorithms
### Results from Machine Learning Algorithms  
The classification results obtained with the machine learning algorithms, both under BRIR matched and BRIR mismatched scenarios, are summarized in Figure 7. For comparison, the left pane of the figure shows the classification results reached by human listeners.  
Under the BRIR matched test scenario, the SVM, XGBoost, and CNN algorithms performed equally well, with the classification accuracy being equal to 83.9%, 82.7%, and 81.7%, respectively, with no statistically significant differences between the results according to a \(t\)-test (\(p>0.058\)). Logit was the worst performing algorithm, yielding an accuracy of 75.6%. The difference in the accuracy between the Logit algorithm and the above three algorithms was statistically significant at \(p<9.2\times 10^{-6}\) level. The examples of the confusion matrices obtained for all four algorithms are presented in Figure 8a-d. They show that the algorithms are relatively good at the classification of the first two scenes (_FB_ and _BF_) and slightly worse at the classification of the last scene (_FF_). The last two panes in Figure 8 illustrate the confusion matrices achieved for BRIR sets 2 and 7, respectively, demonstrating an exceptionally good BRIR-specific performance obtained using the SVM and CNN algorithms.  
Figure 7: Comparison of human against machine classification of spatial audio scenes. Red line denotes no-information rate. Error bars represent 95% confidence intervals.  
For the BRIR mismatched test scenario, the performance of all four algorithms markedly dropped down to the level of approximately 55%, with no statistically significant differences in performance between them (\(p>0.085\)), as indicated by overlapping confidence intervals in Figure 7. The figure demonstrates that the outcomes of the experiment strongly depend on the testing method.  
Figure 9 confirms the above observations, showing the confusion matrices obtained under the BRIR mismatched scenario. The first four panes of that figure illustrate the results obtained using CNN, XGBoost, SVM, and Logit algorithms, respectively. The confusion matrices are substantially worse than those obtained under the BRIR matched condition. The last two panes of Figure 9 show the BRIR-specific results. They illustrate the confusion matrices for BRIR set 9 achieved with the SVM and Logit algorithms, respectively. As before, visual inspection of these two confusion matrices confirms rather mediocre performance of the algorithms.  
For completeness, Figure 10 shows the overview of the BRIR-specific results. While the differences in performance between the algorithms are rather small under the BRIR matched test (Figure 10a), they are much more pronounced under the BRIR mismatched scenario (Figure 10b). For example, CNN was markedly outperformed by the traditional algorithms for BRIR set 1 under the mismatched test condition (\(p<1.3\times 10^{-5}\)). However, its performance was superior for BRIR set 7 (\(p<1.6\times 10^{-7}\)), indicating that the strengths and weaknesses of the classification methods could be BRIR-specific.  
Figure 8: Examples of confusion matrices obtained using machine learning algorithms under BRIR matched conditions: (**a**) convolutional neural network CNN, BRIR sets 1-11; (**b**) extreme gradient boosting framework (XGBoost), BRIR sets 1-11; (**c**) support vector machine (SVM), BRIR sets 1-11; (**d**) logistic regression (Logit), BRIR sets 1-11; (**e**) SVM BRIR set 2; (**f**) CNN, BRIR set 7. Vertical axis represents the classification results obtained from the classification algorithms.