# Hybrid LSTM Model to Predict the Level of Air Pollution in Montenegro
## 3 Implementation Methodology  
This paper focuses on the hybridization of multiple algorithms. To develop our hybrid model, the Python libraries Keras and Scikit-Learn and their modules _keras.optimizers_ and _sklearn.model_selection_ were used, as well as the necessary functions and algorithms contained in these modules. A comprehensive description of the proposed system architecture is represented in Figure 1. The various phases employed to obtain hourly air pollution predictions in Nikisic, Montenegro are presented. The first step involves collecting air quality data from an air monitoring station. The collected data undergo preprocessing and feature engineering procedures to ensure better training and testing results. After this, the data are partitioned, scaled, and fed into the proposed hybrid LSTM model for further analysis and prediction. In order to enhance the performance of the LSTM model, a GA is employed for parameter selection. This hybridization with the metaheuristic algorithm helps to find the best combination of hyperparameters for LSTM, resulting in the improved predictive capabilities of the model. The bagging technique is applied to the best-performing LSTM model. This approach involves training multiple instances of the LSTM model on distinct subsets of the data and combining their predictions, leading to improved overall accuracy and robustness. Based on its predictive performance, the implemented hybrid LSTM model is thoroughly tested and compared with the SVM approach. These two final models are evaluated and analyzed to provide insights into their suitability for air quality prediction. The LSTM algorithm is a type of RNN architecture specifically designed to handle sequential data and effectively capture long-term dependencies [21]. LSTM networks use specialized memory cells and gates to effectively manage and control the flow of information at various time steps. This enables them to effectively model and retain important patterns and dependencies in the input data. The LSTM algorithm addresses the vanishing gradient problem that occurs in RNN algorithms, where gradients become too small to update weights effectively over long sequences. By using memory cells and gates, LSTM allows gradients to flow through time more easily. Figure 2 illustrates the architecture of the LSTM unit, while the corresponding Equations (1)-(6) for the LSTM algorithm are provided below:  
\[in_{t}=\sigma(V_{in}x_{t}+\omega_{in}h_{t-1}+\beta_{in}), \tag{1}\]  
\[frg_{t}=\sigma(V_{frg}x_{t}+\omega_{frg}h_{t-1}+\beta_{frg}), \tag{2}\]  
\[out_{t}=\sigma(V_{out}x_{t}+\omega_{out}h_{t-1}+\beta_{out}), \tag{3}\]  
\[\mathcal{C}_{t}=tanh(V_{c}x_{t}+\omega_{c}h_{t-1}+\beta_{c}), \tag{4}\]  
\[\mathcal{C}_{t}=frg_{t}\otimes\mathcal{C}_{t-1}+in_{t-1}\otimes\mathcal{C}_{t}, \tag{5}\]  
\[h_{t}=out_{t}\otimes tanh(\mathcal{C}_{t-1}), \tag{6}\]  
where the input gate (\(in_{t}\)), forget gate (\(frg_{t}\)), and output gate (\(out_{t}\)) are controlled by the weights (\(V_{in}\), \(V_{frg}\), \(V_{out}\)) connecting them to the input. The weights (\(\omega_{in}\), \(\omega_{frg}\), \(\omega_{out}\)) connect the input, forget, and output gates to the hidden layer. The bias vectors (\(\beta_{in}\), \(\beta_{frg}\), \(\beta_{out}\)) are associated with the input, forget, and output gates, respectively. The state of the cell at the  
Figure 1: Proposed system architecture for hourly CAQI prediction in Montenegro. previous time point is denoted by \(\mathcal{C}_{t}\) and the current state of the cell by \(\mathcal{C}_{t}\). The outputs of the cell at the previous and current time points are denoted by \(h_{t-1}\) and \(h_{t}\), respectively [22]. The GA is an evolutionary-based metaheuristic algorithm that employs the principles of natural adaptation and selective breeding [8]. It operates on a population of individuals, where each individual represents a potential solution to a specific problem. These individuals are characterized by a genetic code, which is a sequence of characters (genes) from some alphabet.