## 4 Performance Evaluation Results
### Confusion Matrices for the Combined Scenario  
Figure 9 shows the confusion matrices for the combined scenarios for the data split {80:20}% and \(M=8\). The confusion matrices provide insights into the (mis)classification statistics of the different ML algorithms. In Figure 9, each confusion matrix highlights the precision and recall percentages for each beam in the codebook, with precision values being displayed at the bottom of each table for each algorithm while the recall values are shown on the right sides of the confusion  
Figure 8: Beam Prediction accuracy comparison for different dataset splits for \(M=8\). matrices. The classes are listed in the same order in both the rows and the columns, ensuring consistency between the true and predicted classifications. As a result, the correctly classified elements are located on the main diagonal of the confusion matrix, extending from the top left to the bottom right. These values correspond to the number of times that both the true and predicted classifications agree. By examining the values along the main diagonal, one can assess the accuracy of the classification model, as it reflects the number of correctly classified instances for each class. The off-diagonal elements provide insights into the misclassifications, showing where the model's predictions diverge from the true classifications. To further show the misclassification statistics, Table 3 shows the percentages of the misclassified beams across the 8-beam codebook. The diagonal values are shown with * because they represent the correctly classified beams, while empty cells in Table 3 indicate no prediction for such beams. Based on the misclassification spread in the confusion matrices and on Table 3, beams have higher tendencies or probabilities to be misclassified as adjacent beams than as beams further away. Using beam 2 in the KNN 8-beam codebook as an example, out of the 25.5% misclassified beams, adjacent beam 1 and beam 3 received 12.77% and 12.41% misclassification respectively, while beam 4 that is further away from  
Figure 9: Confusion Matrices for the Combined Scenario with the \(M=8\) beams (Blue color represents correct classifications while orange represents misclassifications; the darker the color tone, the higher the value). To further show and compare the performances of the four considered ML algorithms, Tables 4-6 respectively show the results for codebook sizes \(M=8\), 16, and 32 for the combined dataset, based on the evaluation metrics given by Equations (1)-(5) (i.e., accuracy, precision, recall, specificity, and F1-score, respectively). For all these five metrics, the closer the values to one, the better the algorithm/model's performance. High accuracy indicates high correct predictions, while high precision shows high true positive prediction among all positive predictions by the classifier. In a similar vein, high recall shows that the model is effective at identifying positive instances, while high specificity shows that the model is effective at identifying negative instances. A higher F1-score translates to a better balance between the precision and recall values. The results in Tables 4-6 show that the performances of the algorithms decreased as the codebook size increased. This indicates that an increase in codebook size leads to a higher rate of misclassification. Further, the results in Tables 4-6 show a beam prediction accuracy of \(\sim\)73% (for DT, \(M=8\)) for the best case and as low as 18% (for NB, \(M=32\)) for the worst case. Similarly, the F1-score statistics also reveal the best case of 0.71 and worst case value of 0.16 for the considered scenarios and codebooks. Overall, considering the performance of the four algorithms for \(M=8\) beams in Table 4, the algorithm with the best performance is DT, followed by KNN, then SVM and NB. The algorithms' performance order for \(M=16\) (Table 5) and \(M=32\) (Table 6) follow the same pattern as those of \(M=8\), thus retaining their positions though with different performance values.