# Rainy Environment Identification Based on Channel State Information for Autonomous Vehicles
## 5 Simulation Analysis
### Model Evaluation  
The confusion matrix, as an important tool for evaluating classification performance, not only intuitively displays the model's identification capabilities across various categories but also provides detailed insights into the model's misclassification in specific categories. The main diagonal of each matrix shows the true positives for each category, while the off-diagonal elements represent the model's misjudgments, thus offering a dimension to evaluate the model's detailed performance. Figure 5 presents the confusion matrix for the proposed CNN model under a dual-channel configuration. The matrix reveals that the CNN model can reliably identify various vehicular environments in rainy conditions based on CSI values, achieving an overall test accuracy of 95.7%. Specifically, the model's identification accuracies for H-NLOS, H-LOS, R-LOS, U-LOS, and U-NLOS environments are 95.2%, 96.1%, 97.3%, 95.3%, and 94.8%, respectively. Additionally, the proposed CNN model was compared with an ANN model that includes four dense, fully connected layers to ensure compatibility with the number of environments to be identified. The comparative analysis also considers a Random Forest (RF) classifier with 100 decision trees, a K-Nearest Neighbors classifier (K-NN, with K set to five neighbors), Gaussian Naive Bayes (GNB), and a Support Vector Machine (SVM) with a linear kernel, as shown in Table 3. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Approach** & **Accuracy (\%)** & **Prediction Time (\(\mu\)s)** \\ \hline Our Model & 95.7 & 43.82 \\ ANN & 81.9 & 27.45 \\ RF & 65.6 & 36.27 \\ K-NN & 66.8 & 9723 \\ GBN & 26.5 & 5.79 \\ SVM & 35.2 & 16,724 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Classification accuracy and average prediction time comparison. Figure 5: The confusion matrix of the CNN model. To further demonstrate the performance of various methods in accurate classification, the confusion matrices of each method are detailed in Figures 5-10. These matrices provide a clear visual representation for evaluating and comparing the effectiveness of different models in environmental identification tasks. They not only intuitively display the predictive accuracy of classification models across various categories but also reveal the strengths and weaknesses of the models under specific conditions. The analysis of the confusion matrices indicates that different machine learning models exhibit varied performances in the task of vehicular environment identification under rainy conditions. The ANN model performed admirably with an accuracy rate of 81.9%, while the K-NN and RF models achieved an accuracy of 80% in H-NLOS and U-NLOS environments, demonstrating their effectiveness under specific conditions. However, these two models  
Figure 6: The confusion matrix of the ANN model. Figure 7: The confusion matrix of the KNN model. performed poorly in H-LOS, R-LOS, and U-LOS environments, with accuracy rates below 65%, which may suggest insufficient feature learning in these settings. On the other hand, the GNB and SVM classifiers generally performed poorly across all test environments, which may imply their limited adaptability to complex environment identification tasks. The proposed convolutional neural network (CNN) model was compared with a series of advanced deep learning models, including ResNet50 [30], Xception [31], InceptionV3 [32], DenseNet201 [33], and MobileNetV2 [34]. To ensure fairness in comparison, all models were trained on the same training dataset, and their classification performance was evaluated on the test set. In response to the specific data input requirements of this study, the input layer shape of all models was adjusted accordingly, and the output layer structure was updated to include five categories, accurately matching the types of vehicular environments to be identified. Figure 8: The confusion matrix of the RF model. Figure 9: The confusion matrix of the GNB model.