# Hybrid LSTM Model to Predict the Level of Air Pollution in Montenegro
## 4 Results and Discussion
### CAQI Prediction Model Implementation
\begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter** & **Value** \\ \hline Number of features & 4 \\ Number of outputs & 1 \\ Number of time steps (selected by GA) & 24 \\ Hidden layer size (selected by GA) & 15 \\ Batch size & 8 \\ Number of epochs & 300 \\ Dropout rate & 0.2 \\ Recurrent dropout & 0.2 \\ Learning rate & 0.0001 \\ Optimizer & Adam \\ Activation & Relu \\ Number of bags & 5 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: LSTM hyperparameter setting. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Crossover probability & 0.9 \\ Number of generations & 4 \\ Mutation probability & 0.3 \\ Population size & 4 \\ Fitness function & MSE \\ \hline \hline \end{tabular}
\end{table} 
Table 3: GA parameter setting. The performance of the best LSTM model with applied GA for hyperparameter tuning was evaluated using the MSE, MAE, MAPE, and \(R^{2}\) evaluation metrics, as shown in the third column of Table 5. The learning curve is displayed in Figure 8a, showing the decreasing trend of the loss function with each epoch. This rapid decline in loss is attributed to the high quality of the model and the well-tuned model parameters. The convergence graph for the five models in the bagging ensemble is shown in Figure 8b. The training and testing MSE and MAE history for the ensemble model with bagging are presented in Figure 9.  
The performance of the ensemble model with bagging was evaluated using four evaluation metrics, as presented in the fourth column of Table 5. The actual and predicted values on a subset of the test dataset for the ensemble model with bagging are displayed in Figure 10.  
In our analysis, the performance difference between the LSTM model and the ensemble model was assessed, revealing a reduced MSE difference value of \(-9.3688\), indicating improved algorithm performance in terms of lower error and better prediction. As a result, the quality of the model was increased. The performance difference between the best LSTM  
Figure 8: Convergence graphs representing **(a)** the best LSTM model without bagging, **(b)** the five models in the bagging ensemble. Figure 9: Training and testing performance metric history for the ensemble model with bagging: (a) MSE, **(b)** MAE. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Performance Metric** & **SVM Model** & **Best LSTM Model** & **Ensemble Model** \\ \hline MSE & 428.2885 & 58.8046 & 56.5810 \\ MAE & 13.8739 & 4.6657 & 4.5560 \\ MAPE & 0.2685 & 0.0923 & 0.0912 \\ \(R^{2}\) & 0.3820 & 0.9135 & 0.9168 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: SVM, best LSTM, and ensemble model evaluation results in hourly CAQI prediction. model without bagging and the ensemble model was also assessed, resulting in an MSE difference of \(-2.2236\) (see Table 6). Regarding the SVM model, preliminary tests to adjust its hyperparameters were performed. The best-performing SVM model, which was used for the comparison with the ensemble model with bagging, had the following hyperparameters: the radial basis function was used as the kernel function, the parameter that represented the penalty factor was set to 1, and the regularization parameter was set to 2. This value of the penalty factor enabled the resulting SVM model to be less sensitive to outliers and focus more on the general patterns of the data. Additionally, a relatively large value of the regularization parameter ensured that the model had a wider margin of tolerance for errors. This enabled the comparison between the best-performing SVM model and the ensemble model with bagging. The performance evaluation of the best-performing SVM model yielded a high value of MSE (428.2885) and a low \(R^{2}\) measure (0.3820), as shown in the second column of Table 5. These low performance results of the SVM model can also be seen in Figure 11, where the actual and predicted values on a subset of the test dataset for the SVM model are presented.