# Hybrid LSTM Model to Predict the Level of Air Pollution in Montenegro
## 3 Implementation Methodology
By decoding each individual and assessing its fitness value, the algorithm determines the quality of solutions within the population. Through iterative processes of selection, crossover, and mutation, the GA seeks to optimize the solutions until a specific stopping criterion is met. This criterion can be a fixed number of generations or a condition where the algorithm terminates if there is no improvement in the best individual over a defined number of generations. The SVM algorithm belongs to the class of well-known supervised ML algorithms. It shows good performance when it is required to fit functions to the training data and at the same time to minimize produced errors. To handle the demanding relationships between the input features and the target variable, and to ensure the detection of linear decision boundaries, SVM incorporates kernel functions, allowing the mapping of the input data into a multidimensional feature space. The optimization process in SVM involves minimizing the loss function, which consists of a margin violation error and a regularization term [23]. The regularization term maintains a trade-off between the complexity of the model and the training error, thus mitigating the risk of overfitting. The SVM function is defined by Equation (7):  
\[f(x_{i})=v\varphi(x_{i})+\beta, \tag{7}\]  
where \(\varphi(x_{i})\) describes the non-linear mapping function, \(x_{i}\) represents the input vector, and \(z_{i}\) represents the corresponding target value. The values \(v\) and \(\beta\) are the weight factor and bias, respectively [24]. The estimation of the parameters \(w\) and \(b\) is achieved through the minimization of the regularized risk function \(\rho(v)\), as denoted by Equation (8):  
\[\rho(v)=\frac{1}{2}\|v\|^{2}+P\sum_{i=1}^{n}\mathcal{L}_{e}(z_{i},f(x_{i})). \tag{8}\]  
The regularization term \(\frac{1}{2}\|v\|^{2}\) balances the trade-off between the empirical risk and model flatness. The penalty coefficient \(P\) determines the extent of this trade-off. The \(e\)-insensitive loss function \(\mathcal{L}_{e}(z_{i},f(x_{i}))\) is defined by Equation (9) and is used to handle errors within a tolerance level \(e\):  
\[in_{t}=\sigma(V_{in}x_{t}+\omega_{in}h_{t-1}+\beta_{in}), \tag{1}\]0  
\[in_{t}=\sigma(V_{in}x_{t}+\omega_{in}h_{t-1}+\beta_{in}), \tag{1}\]1 The structure of the memory unit in the LSTM layer. If the predicted value falls within the threshold, its contribution to the loss function will be ignored. However, if the predicted value exceeds the threshold, the loss function will take on a value greater than \(e\). To measure the distance between the actual values and the corresponding boundary values of the \(e\)-tube, two positive slack variables, \(\delta\) and \(\delta^{*}\), are introduced. This transformation results in the constrained form of Equation (8),  
\[in_{t}=\sigma(V_{in}x_{t}+\omega_{in}h_{t-1}+\beta_{in}), \tag{1}\]2  
subject to  
\[ \begin{cases}z_{i}-v\varphi(x_{i})-\beta\leq e+\delta,&\delta\geq 0\\ v\varphi(x_{i})+\beta-z_{i}\leq e+\delta^{*},&\delta^{*}\geq 0.\end{cases}  \tag{10}\]  
The Lagrangian function, defined by Equation (11), is used to solve the previously defined optimization problem (10),  
\[maxF(\lambda_{i},\lambda_{i}^{*})= -\frac{1}{2}\sum_{i=1}^{n}\sum_{j=1}^{n}(\lambda_{i}-\lambda_{i}^ {*})(\lambda_{j}-\lambda_{j}^{*})\kappa(x_{i},x_{j})\] \[+\sum_{i=1}^{n}z_{i}(\lambda_{i}-\lambda_{i}^{*})-e\sum_{i=1}^{n} z_{i}(\lambda_{i}+\lambda_{i}^{*})\]  
subject to  
\[\sum_{i=1}^{n}\lambda_{i}-\lambda_{i}^{*}=0,\quad\lambda_{i},\lambda_{i}^{*} \in[0,P]. \tag{11}\]  
Equation (12) describes the method of calculating the regression function \(f(x)\),  
\[f(x)=\sum_{i=1}^{n}(\lambda_{i}-\lambda_{i}^{*})\kappa(x_{i},x_{j})+\beta, \tag{12}\]  
where the Lagrangian multipliers satisfy the constraints \(\lambda_{i}\geq 0\) and \(\lambda_{i}^{*}\geq 0\). The kernel function \(\kappa(x_{i},x_{j})\) allows for the non-linear mapping of the original data into a higher-dimensional feature space.