# An Improved Hybrid Transfer Learning-Based Deep Learning Model for PM2.5 Concentration Prediction
## 2 Proposed Model
### DSTP-DANN Based on Transfer Learning  
Figure 2 shows the proposed DSTP-DANN structure based on transfer learning (defined as TL-DSTP-DANN). The TL-DSTP-DANN structure consists of three main components: feature extractor, regression predictor, and domain classifier. The feature extractor is based on the improved DSTP model (see Section 2.2), and the regression predictor and domain classifier are both fully connected layers. The training optimization loss of the TL-DSTP-DANN model includes regression loss and domain classification loss. The regression loss for PM2.5 prediction is defined as the mean squared error:  
\[L_{y}^{i}(\theta_{f},\theta_{y}) = \frac{1}{n}\sum_{i=1}^{n}\left(y_{i}-y_{i}^{\prime}\right)^{2}, \tag{16}\]  
where \(n\) is the batch size of training data; \(y_{i}\) and \(y_{i}^{\prime}\) represent the actual and predicted values of PM2.5, respectively. The loss for domain label classification is defined as the dichotomous cross-entropy:  
\[L_{d}^{i}(\theta_{f},\theta_{d})=\frac{1}{n}\sum_{i=1}^{n}l_{i}\log\frac{1}{l _{i}^{\prime}}+\frac{1}{n}\sum_{i=1}^{n}\left(1-l_{i}\right)\log\frac{1}{1-l_ {i}^{\prime}}, \tag{17}\]  
where \(l_{i}\) and \(l_{i}^{\prime}\) represent the actual domain label and the prediction domain label, respectively. In this study, we set the source domain label to 0 and the target domain label to 1. In the training process, to obtain domain invariant features, the distribution of two features is as similar as possible. The parameter \(\theta_{f}\) of feature mapping is found to maximize the loss of the domain classifier, and at the same time, the parameter \(\theta_{d}\) of the domain classifier is found to minimize the loss of the domain classifier. The minimum and maximum change between losses cannot be directly realized by gradient update in the back-propagation process of neural networks. The difference between these two losses is achieved by inserting a gradient reversal layer (GRL) between the feature extractor and the domain classifier. In this paper, DANN is used to search for domain-invariant features between source domain and target domain through the domain adaptation of DSTP feature extractor and domain classifier. The main reason for using the DANN is that it can combine domain adaptation and feature learning in a training process, so that the parameters learned can be directly applied to the target domain without reducing its prediction accuracy due to the domain deviation [27]. The idea of this paper is very similar to the generative adversarial networks (GANs). The generating model G: Equivalent to a feature extractor, the goal is to make the domain classifier not correctly identify the domain labels (the two feature distributions should be as similar as possible). The discriminant model \(D\): Determine whether a label is the label of the target domain, and the target is to distinguish whether the extracted features come from the source domain or the target domain. GANs is implemented by the competition between \(G\) and \(D\). During the training process, the two models \(G\) and \(D\) can be enhanced simultaneously by competing with each other. Because of the existence of discriminant model \(D\), \(G\) can learn the similar features of the two distributions well without a lot of prior knowledge and prior distribution, and finally make the data generated by the model achieve the effect of faking the truth (that is, \(D\) cannot distinguish whether the features extracted by \(G\) come from the source domain or the target domain, so \(G\) and \(D\) reach a certain Nash equilibrium [33]). In the proposed TL-DSTP-DANN model, GRL acts as a constant transform during forward-propagation, gaining gradients at the latter level and changing its sign during  
Figure 2: The proposed TL-DSTP-DANN structure. (1) Feature extractor: The feature extractor \(G_{f}\) is used to extract the temporal and spatial features of the input time-series data.