## 4 PM2.5 Prediction Modeling Based on Stacking
### Model Structure  
We individually train Convolutional Neural Networks (CNN), long short-term memory (LSTM), Random Forest (RF), XGBoost, and a Stacking ensemble learning model for predicting PM2.5 concentrations in Macau.  
1. Convolutional Neural Networks (CNNs) are deep learning models specifically designed for processing data with a distinct grid-like structure, such as images. They use convolutional layers to automatically and efficiently capture the spatial and temporal local correlations present in the input data. CNNs are widely applied in various domains including image recognition, video analysis, and natural language processing [24]. Our CNN architecture, illustrated in Figure 6, integrates a single convolutional layer comprising 20 filters with a kernel size of 2, and utilizes the ReLU activation function for nonlinear transformations. The input time step is set at 24. During the training phase, the Adam optimizer is selected, with the training conducted over 60 epochs, and a batch size of 32 samples per batch.  
2. LSTM (long short-term memory) networks are a variant of recurrent neural networks (RNNs) that are extensively utilized for processing and modeling time series data. The design of LSTM is aimed at addressing the issues of vanishing and exploding gradients that traditional RNNs encounter when dealing with long sequences [25]. As depicted in Figure 7, our model employs a two-layer LSTM to capture the long-term dependencies present in sequence data, and utilizes the ReLU activation function for nonlinear transformations. The input time step is set at 24. During the training phase, the Adam optimizer is chosen, with the training conducted over 60 epochs, and a batch size of 48 samples per batch.  
Figure 6: CNN structure.  
3. Random Forest is an ensemble learning method that enhances prediction accuracy and stability by constructing multiple decision trees and aggregating their predictions. This approach effectively reduces model overfitting and enhances generalization capabilities [26]. As illustrated in Figure 8, our model is configured with 60 decision trees for ensemble learning, with the input time step set at 24.  
Figure 8: RF structure.  
Figure 7: LSTM structure.  
4. Extreme Gradient Boosting (XGBoost) is an ensemble learning technique used for supervised learning that enhances the overall model's predictive ability by integrating multiple weak learners (decision trees). Each decision tree is trained on the residuals of the previous tree, progressively reducing the prediction error and thereby increasing the model's accuracy [27]. As depicted in Figure 9, our model's input time step is set at 24. In each iteration, the model randomly selects 30% of the features to build the trees, with a learning rate set at 0.1, a maximum tree depth of 5, and a regularization weight of 10, training a total of 60 decision trees.  
5. LightGBM is an efficient machine learning algorithm based on the gradient boosting framework, provided by Microsoft Research. As a type of Gradient Boosting Decision Tree (GBDT) algorithm, it is particularly effective in handling large-scale data due to its lower memory consumption and faster training speed [28], as illustrated in Figure 10.  
Figure 10: LightGBM structure.  
Figure 9: XGBoost structure.