# Improving Air Pollution Prediction System through Multimodal Deep Learning Model Optimization
## 6 Experimental Evaluation
The RMSE of Multimodal3, LSTM_AE + VGG19, was 14.59, which decreased by 2.15% compared with Multimodal2. Table 3 shows the top five out of 10 experiments for proposed models in the order of good performance. However, unlike Table 2, in the case of the multimodal deep learning model, numerical data used the same data as LSTM_AE, but all image data used cropped image data. Again, the performance is compared based on the minimum and average values among the experimental results. Figure 10: Average Test RMSE for each model using the original image. Figure 9: Minimum Test RMSE for proposed models using the original image. Figure 11 shows the minimum Test RMSE (Rank 1) of each model shown in Table 3. The RMSE of LSTM_AE, which processes numerical data, showed a value of 15.96, and the RMSE of Multimodal1, which is LSTM_AE + CNN, was 14.03, decreased by 12.09% compared with LSTM_AE. The RMSE of Multimodal2, LSTM_AE + VGG16, was 13.88, which decreased by 1.07% compared with Multimodal1. The RMSE of Multimodal3, LSTM_AE + VGG19, was 13.7, which decreased by 1.3% compared with Multimodal2. Figure 12 shows the average Test RMSE (Average of Rank) of each model shown in Table 3. The RMSE of LSTM_AE, which processes numerical data, showed a value of 16.91, and the RMSE of Multimodal1, which is LSTM_AE + CNN, was 15.43, decreased by 8.75% compared with LSTM_AE. The RMSE of Multimodal2, LSTM_AE + VGG16, was 15.34, which decreased by 0.58% compared with Multimodal1. The RMSE of Multimodal3, LSTM_AE + VGG19, was 15.01, which decreased by 2.15% compared with Multimodal2. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model Rank** & **Multimodal1** & **Multimodal2** & **Multimodal3** \\ \hline Rank 1 & 14.03 & 13.88 & 13.7 \\ Rank 2 & 14.71 & 15.31 & 14.78 \\ Rank 3 & 15.28 & 15.51 & 15.12 \\ Rank 4 & 16.41 & 15.73 & 15.65 \\ Rank 5 & 16.71 & 16.27 & 16.14 \\ \hline Avg of Rank & 15.43 & 15.34 & 15.01 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance comparison of proposed models using cropped image (Test RMSE). Figure 11: Minimum Test RMSE for each model using the cropped image. Figure 13 compares minimum Test RMSE for all experimental results in Tables 2 and 3, and C means using cropped images. Overall, the performance was better when cropped images were applied, and Multimodal3(C) was the best among them. The RMSE of Multimodal1(C) was 14.03, which decreased by 3.7% compared with Multimodal1. The RMSE of Multimodal2(C) was 13.88, which decreased by 2.11% compared with Multimodal2. The RMSE of Multimodal3(C) was 13.7, which decreased by 1.86% compared with Multimodal2. Figure 14 compares the average Test RMSE for all experimental results in Tables 2 and 3, and C means using cropped images. Unlike the minimum Test RMSE, the performance was not good when cropped images were applied. The RMSE of Multimodal(C) is 15.43, which is very similar to Multimodal1, and the RMSE of Multimodal2(C) is 15.34, increased  
Figure 12: Average Test RMSE for each model using the cropped image. Figure 13: Minimum Test RMSE of all experimental results. by 2.88% compared with Multimodal2. The RMSE of Multimodal3(C) was 15.01, which increased by 2.88% compared with Multimodal3. Combining the experimental results, the multimodal deep learning model (Multimodal1, Multimodal2, Multimodal3) using both numerical and image data performed better than a single deep learning model (LSTM_AE) using only numerical data. If the image data is the original image, the minimum Test RMSE of the multimodal deep learning model decreased by approximately 8.71-12.53% compared with the single deep learning model, and the average Test RMSE of the multimodal deep learning model was reduced by approximately 8.69-13.72% compared with the single deep learning model.