## 4 Simulation and Experimental Results and Discussion
### ML Models Prediction Accuracy
#### 4.3.2 Prediction Accuracy of ML Models  
Both the CNN and MLP models were compiled with categorical cross-entropy loss using the Adam optimizer and the optimal hyperparameters discussed in Section 3.4.2. Table 3 summarizes the prediction accuracy and standard deviation achieved by our models for both low- and high-resolution datasets. The CNN model achieved more than 93% and 94% prediction accuracy for U10 and V10, respectively, for the high resolution. In the case of low resolution, the CNN model achieved more than 98% and 97% prediction accuracy for U10 and V10, respectively. The MLP model achieved over 97% prediction accuracy for both the U10 and V10 vector components in the low-resolution dataset. However, for the high-resolution dataset, the MLP model achieved 72% and 64% accuracy rates for the U10 and V10 wind components, respectively. A possible explanation for the lower accuracy of the MLP model in the case of high-resolution data could be that the high-resolution data encompasses a smaller geographic area compared with the low-resolution domain and that features that are present in the larger area help in identifying local phenomena. As a final step, cross-validation tests [87; 88] were performed on the low-resolution forecasts for both CNN and MLP and evaluated with a high accuracy rate of 97% and above for both ML models (see Table 4). As part of our experimental analysis, we performed cross-validation (CV, Table 4) for both the CNN and MLP models. The CV was conducted by randomly selecting the training and validation datasets using the same 80/20 split between the training and validation datasets. This process was repeated five times, and the mean was extracted after removing the best and worst outliers [87; 88]. To assess the ML models' performance comprehensively, we conducted further analysis by constructing a confusion matrix (CM) and evaluating the ML model accuracy, as illustrated in Figure 8 for the low-resolution model. In an ideal CM, each class aligns with its counterpart, reflecting a similar prevalence across the diagonal cells. The CM is vital in evaluating the performance of our ML models. First, it provides a detailed breakdown of a model's predictions, revealing true positives, true negatives, false positives, and false negatives for each class. In addition, it allows for a comprehensive performance assessment, error analysis, and detection of class imbalances. In essence, the CM offers nuanced insights beyond basic accuracy metrics, guiding decisions on model refinement and deployment. As shown in Figure 8a,b, the CM illustrates favorable outcomes for both the CNN and MLP models, revealing accurate predictions, small percentages of misclassifications, and a well-balanced representation across various classes. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Min Accuracy** & **Max Accuracy** & **St. dev.** &
\begin{tabular}{c} **Post Trimming** \\ **Average** \\ \end{tabular} \\ \hline CNN & 97.3\% & 98.3\% & 0.42 & 97.9 \\ MLP & 97.2\% & 98.0\% & 0.33 & 97.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Cross-validation results summary. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**(a) High Resolution**} \\ \hline
**Model** & **U10 avg. Accuracy** & **V10 avg. Accuracy** & **U10 st. dev.** & **V10 st. dev.** \\ \hline CNN & 93.7\% & 94.8\% & 1.5 & 0.2 \\ MLP & 72.0\% & 54.8\% & 1.6 & 2.6 \\ \multirow{3}{*}{**Model**} & \multirow{3}{*}{**U10 avg. accuracy**} & **(b) Low resolution** & & \\  & & **V10 avg. accuracy** & **U10 st. dev.** & **V10 st. dev.** \\ CNN & 98.2\% & 97.7\% & 0.14 & 0.46 \\ MLP & 97.7\% & 97.5\% & 0.14 & 0.60 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Final ML model evaluation results. An additional analysis was conducted to investigate the convergence of the CNN and MLP models during the training process, as illustrated in Figure 8c. Notably, 50 epochs are sufficient to attain stable and converged training accuracy.