# Air Quality Prediction Model Based on Spatiotemporal Data Analysis and Metalearning

Authors: Kejia Zhang; Xu Zhang; Hongtao Song; Haiwei Pan; Bangju Wang
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

With the continuous improvement of people’s quality of life, air quality issues have become one of the topics of daily concern. How to achieve accurate predictions of air quality in a variety of complex situations is the key to the rapid response of local governments. This paper studies two problems: (1) how to predict the air quality of any monitoring station based on the existing weather and environmental data while considering the spatiotemporal correlation among monitoring stations and (2) how to maintain the accuracy and stability of the forecast even when the available data is severely insufficient. A prediction model combining Long Short-Term Memory networks (LSTM) and Graph Attention (GAT) mechanism is proposed to solve the first problems. A metalearning algorithm for the prediction model is proposed to solve the second problem. LSTM is used to characterize the temporal correlation of historical data and GAT is used to characterize the spatial correlation among all the monitoring stations in the target city. In the case of insufficient training data, the proposed metalearning algorithm can be used to transfer knowledge from other cities with abundant training data. Through testing on public data sets, the proposed model has obvious advantages in accuracy compared with baseline models. Combining with the metalearning algorithm, it gives a much better performance in the case of insufficient training data.


---

## Body

## 1. Introduction

Because of the increasingly serious air pollution all over the world, air quality has become one of the most socially concerned issues. In many countries, air quality has become a key indicator to measure the happiness index of residents. In order to achieve real-time monitoring of air quality, almost all countries have arranged a large number of air quality monitoring stations in major cities. Besides, more and more mobile portable monitoring devices are participating in air quality monitoring [1, 2]. Although many monitoring methods have been applied, it is still extremely challenging to make accurate predictions of air quality. Especially in the case of insufficient monitoring data or poor data quality, it is more difficult to maintain the accuracy and stability of the prediction model.Air quality is affected by a variety of complex factors [3–6], including meteorological factors, industrial factors, fuel factors, traffic factors, and other human activity factors. The development of related monitoring equipment leads the collection of air quality data more and more comprehensive [7]. With the application of a series of spatiotemporal prediction models [8], air quality prediction has made considerable progress. Time correlation refers to the impact of historical monitoring data on future data, and spatial correlation refers to the mutual influence among adjacent monitoring stations. Most of the existing research works [3, 4] focus on establishing prediction models based on time correlation, while there are obvious shortcomings in the study of spatial correlation. The reason for this phenomenon is because the diffusion of air pollutants is affected by various factors such as geographical location, wind direction, wind speed, air pressure, and air humidity. The impact of each factor on the relevance of different regions is difficult to accurately model. In this paper, we propose a spatiotemporal model for air quality prediction. The proposed model combines Long and Short-Term Memory networks (LSTM) and Graph ATtention (GAT) mechanism [9], where LSTM is used to capture the correlation in the time domain and GAT is used to model the spatial correlation among different regions.In recent years, many deep learning models [3, 4, 10–13] have achieved good results in air quality prediction. However, the accuracy of these predicting models highly depends on the sufficiency of training data. In reality, a lack of sufficient training data is the most common situation. As we all know, air quality monitoring in developing countries mainly depends on monitoring stations arranged by the government. There are few or no such stations in small cities and towns. Insufficient training data makes it difficult for the existing prediction models to achieve accurate results in these small cities and towns. Even in large cities, government-monitoring stations are very sparse. Although there are some unofficial monitoring devices that can provide data as a supplement, the data collected by these simple monitoring devices are often of poor quality with large amounts of various dirty data and missing values. Therefore, making accurate predictions based on insufficient training data is a realistic and challenging problem. Transfer learning (metalearning) [14] is currently the most effective method to solve this problem. Some transfer learning models [15–17] have been proposed to predict air quality with insufficient data. However, these methods require a strong similarity between the source domain and the target domain. Different cities and towns (especially large cities and small towns) have huge differences in pollution levels, climate, pollutant diffusion conditions, and density of monitoring sites. This makes it difficult for the existing transfer learning technology to successfully transfer the knowledge acquired in large cities to the air quality prediction of small and medium cities. To meet these challenges, based on the proposed prediction model, we give a metalearning algorithm for knowledge transfer among cities with huge differences.The main contributions of this paper are as follows:(i)
Proposing a spatiotemporal model by combining LSTM and GAT for accurate air quality prediction(ii)
Designing a metalearning algorithm for the proposed model, which can transfer knowledge among different cities and make an accurate prediction in case of insufficient training data(iii)
Verifying the advantages of the proposed model and meta-learning algorithm in the aspect of prediction accuracy through a large number of experimentsThe rest of this paper is organized as Section2 introduces some related research works in the area of air quality prediction, transfer learning, and metalearning; Section 3 gives the definition of the problems; the proposed prediction model and metalearning algorithm are introduced in Section 4; after showing the experimental results to prove the effectiveness of the proposed model and metalearning algorithm in Section 5, Section 6 summarizes the whole paper.


## 2. Related Works

This section will briefly present the related research works in the area of air quality prediction, transfer learning, and metalearning.


### 2.1. Air Quality Prediction

The machine learning models for air quality prediction can be divided into two categories: basic learning models and deep learning models. Basic learning models include linear regression, supporting vector regression, random forest, and LightGBM. Land Use Regression (LUR) [5, 6] makes air quality predictions through a linear regression model that takes into account multiple factors like regional population level, traffic condition, and land use condition. LUR does not consider the complicated spatiotemporal correlation of air pollution data, so the accuracy of prediction is poor. Later, the basic time series model autoregressive integrated moving average model (ARIMA) [18] appeared, which was used for time series forecasting with strong periodicity. However, it does not perform well for complex weather conditions. Random forest [19], LightGBM [20], deep learning methods have become widely used methods in air pollution prediction. Later, in order to further improve the accuracy of prediction, Zheng et al. [21] proposed U-Air, which uses a spatial classifier based on an artificial neural network (ANN) and a temporal classifier based on the linear-chain conditional random field (CRF) to capture temporal and spatial characteristics. Convolutional neural networks (CNN) are used to process data from Euclidean structures. For example, they are very effective in the field of image recognition, and it is impractical to use CNN directly to capture the spatial relationships between monitoring stations for sparse graph structures consisting of monitoring stations. The ConvLSTM model proposed in [22] combines CNN and LSTM to characterize the spatiotemporal relationship between monitoring stations, and it is still applicable to the spatial relationship in Euclidean space. The emergence of Graph Convolutional Networks (GCN) [23, 24] has made up for the deficiencies of CNN and is widely used in traffic data. GCN has realized the full use of the traffic network. GAT [9] are proposed on the basis of GCN, using an attention mechanism, and are good at capturing dynamic relationships between nodes. The ST-GAT model proposed by Zhang et al. [25] can dynamically capture the dynamic dependencies in the traffic network, making the traffic speed prediction results more advanced than existing models.


### 2.2. Transfer Learning and Meta-Learning

To improve the practicality of air quality prediction models, the obstacles caused by insufficient data must be resolved. Transfer learning can be divided into three categories according to the difference in source domains and target domains and tasks, namely, inductive transfer learning, transitive transfer learning, and unsupervised transfer learning [14]. In recent years, transfer learning combined with deep neural networks (DNN) has been widely used. The VGG model proposed in the image field [26], with the help of this model, can achieve fast and accurate model training under a small number of sets. Unlike image data, air quality data are more complex in spatial and temporal distribution. Hu et al. [27] proposed a DNN-based sharing model that fused multisource wind speed data together to solve the problem of insufficient wind farm data. However, this model does not provide a solution to the knowledge transfer of spatially related data.Metalearning [28–30] can quickly initialize the model by learning knowledge in multiple different learning tasks in order to widely adapt to a variety of situations. Literature [29] firstly proposes the concept of metalearning, also known as learning. The goal is to train a metalearning model on multiple learning tasks, so as to use a small number of training samples to solve new learning tasks. A model-independent metalearning algorithm MAML is proposed in [28]. MAML deals with the situation of insufficient training data by transferring data and models among multiple learning tasks. Each update step consists of multitask pretraining, model migration, target task training, and model parameter synchronization. Unlike previous metalearning methods, MAML uses gradients to update model and does not introduce additional parameters. Literature [27] proposes a MAML-based spatiotemporal prediction model, which is used for urban traffic prediction and water quality prediction by transferring knowledge among multiple cities.


### 2.3. Summary of Related Works

Through the introduction of the above related works, it can be seen that the existing air quality prediction methods rarely consider the spatial relationship between multiple monitoring stations. A few spatiotemporal prediction models lack the ability to dynamically model spatial correlation based on weather and other related factors. The only methods that can dynamically model spatial correlation do not consider how to deal with insufficient training data. Some existing methods in the area of transfer learning and metalearning can solve the insufficient-training-data situation to a certain extent by transferring the knowledge from other source domains, but these methods lack the ability to adapt to the air quality spatiotemporal prediction models and cannot be directly applied to the scenarios targeted in this article. For this reason, this paper proposes a spatiotemporal model for air quality prediction and a metalearning algorithm for this model. The prediction model can dynamically and accurately model the temporal and spatial correlation in air quality prediction. The metalearning algorithm is used to establish a more accurate prediction model in the case of insufficient training data. As far as we know, it is the first time that metalearning has been used for air quality prediction.


## 2.1. Air Quality Prediction

The machine learning models for air quality prediction can be divided into two categories: basic learning models and deep learning models. Basic learning models include linear regression, supporting vector regression, random forest, and LightGBM. Land Use Regression (LUR) [5, 6] makes air quality predictions through a linear regression model that takes into account multiple factors like regional population level, traffic condition, and land use condition. LUR does not consider the complicated spatiotemporal correlation of air pollution data, so the accuracy of prediction is poor. Later, the basic time series model autoregressive integrated moving average model (ARIMA) [18] appeared, which was used for time series forecasting with strong periodicity. However, it does not perform well for complex weather conditions. Random forest [19], LightGBM [20], deep learning methods have become widely used methods in air pollution prediction. Later, in order to further improve the accuracy of prediction, Zheng et al. [21] proposed U-Air, which uses a spatial classifier based on an artificial neural network (ANN) and a temporal classifier based on the linear-chain conditional random field (CRF) to capture temporal and spatial characteristics. Convolutional neural networks (CNN) are used to process data from Euclidean structures. For example, they are very effective in the field of image recognition, and it is impractical to use CNN directly to capture the spatial relationships between monitoring stations for sparse graph structures consisting of monitoring stations. The ConvLSTM model proposed in [22] combines CNN and LSTM to characterize the spatiotemporal relationship between monitoring stations, and it is still applicable to the spatial relationship in Euclidean space. The emergence of Graph Convolutional Networks (GCN) [23, 24] has made up for the deficiencies of CNN and is widely used in traffic data. GCN has realized the full use of the traffic network. GAT [9] are proposed on the basis of GCN, using an attention mechanism, and are good at capturing dynamic relationships between nodes. The ST-GAT model proposed by Zhang et al. [25] can dynamically capture the dynamic dependencies in the traffic network, making the traffic speed prediction results more advanced than existing models.


## 2.2. Transfer Learning and Meta-Learning

To improve the practicality of air quality prediction models, the obstacles caused by insufficient data must be resolved. Transfer learning can be divided into three categories according to the difference in source domains and target domains and tasks, namely, inductive transfer learning, transitive transfer learning, and unsupervised transfer learning [14]. In recent years, transfer learning combined with deep neural networks (DNN) has been widely used. The VGG model proposed in the image field [26], with the help of this model, can achieve fast and accurate model training under a small number of sets. Unlike image data, air quality data are more complex in spatial and temporal distribution. Hu et al. [27] proposed a DNN-based sharing model that fused multisource wind speed data together to solve the problem of insufficient wind farm data. However, this model does not provide a solution to the knowledge transfer of spatially related data.Metalearning [28–30] can quickly initialize the model by learning knowledge in multiple different learning tasks in order to widely adapt to a variety of situations. Literature [29] firstly proposes the concept of metalearning, also known as learning. The goal is to train a metalearning model on multiple learning tasks, so as to use a small number of training samples to solve new learning tasks. A model-independent metalearning algorithm MAML is proposed in [28]. MAML deals with the situation of insufficient training data by transferring data and models among multiple learning tasks. Each update step consists of multitask pretraining, model migration, target task training, and model parameter synchronization. Unlike previous metalearning methods, MAML uses gradients to update model and does not introduce additional parameters. Literature [27] proposes a MAML-based spatiotemporal prediction model, which is used for urban traffic prediction and water quality prediction by transferring knowledge among multiple cities.


## 2.3. Summary of Related Works

Through the introduction of the above related works, it can be seen that the existing air quality prediction methods rarely consider the spatial relationship between multiple monitoring stations. A few spatiotemporal prediction models lack the ability to dynamically model spatial correlation based on weather and other related factors. The only methods that can dynamically model spatial correlation do not consider how to deal with insufficient training data. Some existing methods in the area of transfer learning and metalearning can solve the insufficient-training-data situation to a certain extent by transferring the knowledge from other source domains, but these methods lack the ability to adapt to the air quality spatiotemporal prediction models and cannot be directly applied to the scenarios targeted in this article. For this reason, this paper proposes a spatiotemporal model for air quality prediction and a metalearning algorithm for this model. The prediction model can dynamically and accurately model the temporal and spatial correlation in air quality prediction. The metalearning algorithm is used to establish a more accurate prediction model in the case of insufficient training data. As far as we know, it is the first time that metalearning has been used for air quality prediction.


## 3. Problem Formulation

This paper will solve two problems: prediction problem and transfer learning problem. The prediction problem is how to build a prediction model for the target pollutant in the city with sufficient training data. The transfer learning problem is how to build a prediction model in the target city with insufficient training data, given the source cities with sufficient data. The symbols used in this paper are given in Table1.Table 1
Symbols and explanations.
SymbolMeaningS=s1,s2,⋯,snSet of monitoring stations in the target citykLength of the historical datatCurrent timelLength of the forecast windowxsiConcentration of target pollutant monitored by stations at time i (past)xs=xst−k;xst−k+1;⋯,;xstHistorcical data vector of stationsX=xs1,xs2,⋯,xsnHistorcical data vectors of all stationswit−k≤i≤tHistorical weather of the target city at timeiWh=wt−k,wt−k+1,⋯,wtHistorical weather datasetwit+1≤i≤t+lWeather forecast of the target city at timeiWf=wt+1,wt+2,⋯,wt+lWeather forecast datasetT=Dsc1,Dsc2,⋯,DscmSet of training dataset from source citiesDtcTraining dataset from the target cityysit+1≤i≤t+lConcentration of target pollutant monitored by stations at time i (future)ys=yst+1;yst+2;⋯,yst+lConcentration of target pollutant at stations in the futurefθPrediction model with parameterθLBfθLoss of modelfθ on training batch By^s=y^st+1;y^st+2;⋯,y^st+lPredicted value ofys.rInfluence radius of monitoring stationsGrDirected graph built on all the monitoring stationsNrsNrs=s∪s′∣s′,s∈GrhsiOutput vector of the LSTM unit on stations at time icsiCell state vector of the LSTM unit on stations at time izsiOutput vector of GAT on stations at time iαs′,sGAT’s similarity score between nodes′ and sα^s′,sWeight of the edges′,s in Gr calculated by GAT


### 3.1. Prediction Problem

Suppose that there is a set of urban monitoring stationsS=s1,s2,⋯,sn in the target city. We use a fixed time interval while counting historical data and making predictions. The prediction problem is building a model to predict the concentration of a certain pollutant sampled by a specified monitoring station in the future. The target air pollutant can be one of PM2.5, PM10, SO2, NO2, O3, CO, AQI (can be regarded as a comprehensive pollutant). Suppose that the current time is t. The input of the prediction model contains (1) a specified monitoring station, (2) the historical monitoring data of the target pollutant sampled from time t−k to time t, (3) the historical weather information from time t−k to time t, and (4) the weather forecast information from time t+1 to time t+l. The output of the prediction model is the predicted value of the target pollutant sampled by the specified station from time t+1 to time t+l. In practical applications, we usually set k+1=2l.For a monitoring stations∈S, define the historical data vector of s as xs=xst−k;xst−k+1;⋯,;xst, where xsi (t−k≤i≤t) is the concentration of the target pollutant sampled by station s at time i. The historical data vectors of all monitoring stations can be represented by a matrix X=xs1,xs2,⋯,xsn.The weather information used by the prediction model includes temperature, humidity, pressure, wind direction, and wind speed. The historical weather dataset of the target city is expressed asWh=wt−k,wt−k+1,⋯,wt, in which wit−k≤i≤t represents the vector of the above weather indexes sampled at time i. The weather forecast dataset of the target city is expressed as Wf=wt+1,wt+2,⋯,wt+l, in which wit+1≤i≤t+l represents the forecast value of the above weather indexes at time i. Usually, Wh and Wf are given by the meteorological department.Given the target monitoring stations∈S, our goal is to predict the concentration of the target pollutant sampled by s from time t+1 to time t+l, which can be expressed as a vector ys=yst+1;yst+2;⋯,yst+l. Suppose that fθ with parameter θ is the model we build, so we have
(1)y^s=fθX,Wh,Wf,s,where y^s is the predicion of ys.LetDtc be the training dataset of the target city. Dtc contains the historical monitoring data of all monitoring stations, the historical weather data, and the historical weather forecast data collected from the target city over a period of time. The prediction problem can be formally defined as how to build an accurate prediction model fθ based on Dtc.


### 3.2. Transfer Learning Problem

In addition to constructing the prediction model, another important issue to be solved in this paper is how to make accurate predictions when there is little training data. In this case, we will transfer knowledge from the source cities with sufficient training data to the target city with insufficient data. Suppose that we havem source cities with sufficient training data. Let Dsc1, Dsc2, ...,Dscm be the training datasets collected from the source cities, respectively. Let Dtc be the insufficient training dataset collected from the target city. The transfer learning problem is defined as how to build an accurate prediction model fθ for the target city based on Dtc∪Dsc1∪Dsc2∪⋯∪Dscm.


## 3.1. Prediction Problem

Suppose that there is a set of urban monitoring stationsS=s1,s2,⋯,sn in the target city. We use a fixed time interval while counting historical data and making predictions. The prediction problem is building a model to predict the concentration of a certain pollutant sampled by a specified monitoring station in the future. The target air pollutant can be one of PM2.5, PM10, SO2, NO2, O3, CO, AQI (can be regarded as a comprehensive pollutant). Suppose that the current time is t. The input of the prediction model contains (1) a specified monitoring station, (2) the historical monitoring data of the target pollutant sampled from time t−k to time t, (3) the historical weather information from time t−k to time t, and (4) the weather forecast information from time t+1 to time t+l. The output of the prediction model is the predicted value of the target pollutant sampled by the specified station from time t+1 to time t+l. In practical applications, we usually set k+1=2l.For a monitoring stations∈S, define the historical data vector of s as xs=xst−k;xst−k+1;⋯,;xst, where xsi (t−k≤i≤t) is the concentration of the target pollutant sampled by station s at time i. The historical data vectors of all monitoring stations can be represented by a matrix X=xs1,xs2,⋯,xsn.The weather information used by the prediction model includes temperature, humidity, pressure, wind direction, and wind speed. The historical weather dataset of the target city is expressed asWh=wt−k,wt−k+1,⋯,wt, in which wit−k≤i≤t represents the vector of the above weather indexes sampled at time i. The weather forecast dataset of the target city is expressed as Wf=wt+1,wt+2,⋯,wt+l, in which wit+1≤i≤t+l represents the forecast value of the above weather indexes at time i. Usually, Wh and Wf are given by the meteorological department.Given the target monitoring stations∈S, our goal is to predict the concentration of the target pollutant sampled by s from time t+1 to time t+l, which can be expressed as a vector ys=yst+1;yst+2;⋯,yst+l. Suppose that fθ with parameter θ is the model we build, so we have
(1)y^s=fθX,Wh,Wf,s,where y^s is the predicion of ys.LetDtc be the training dataset of the target city. Dtc contains the historical monitoring data of all monitoring stations, the historical weather data, and the historical weather forecast data collected from the target city over a period of time. The prediction problem can be formally defined as how to build an accurate prediction model fθ based on Dtc.


## 3.2. Transfer Learning Problem

In addition to constructing the prediction model, another important issue to be solved in this paper is how to make accurate predictions when there is little training data. In this case, we will transfer knowledge from the source cities with sufficient training data to the target city with insufficient data. Suppose that we havem source cities with sufficient training data. Let Dsc1, Dsc2, ...,Dscm be the training datasets collected from the source cities, respectively. Let Dtc be the insufficient training dataset collected from the target city. The transfer learning problem is defined as how to build an accurate prediction model fθ for the target city based on Dtc∪Dsc1∪Dsc2∪⋯∪Dscm.


## 4. Methodology

### 4.1. Monitoring Station Graph

In order to measure the mutual influence among different monitoring stations, we initially model all monitoring stations as a directed graphGr. Monitoring stations are represented by the nodes (vertices) in Gr. Given two nodes si and sj, there are directed edges si,sj and sj,si if the Euclidean distance between si and sj is less than or equal to r. r is the influence radius of monitoring stations, i.e., the maximum range affected by the pollutant in the diffusion process. As shown in Figure 1, by setting r=20km, we get the graph among 34 monitoring stations located in Beijing, China. The weights of the edges in Grwill be calculated by GAT mechanism and change over time. Hereinafter, we use the term “node” to refer to monitoring station and define set Nrs as
(2)Nrs=s∪s′∣s′,s∈Gr.Figure 1
The Graph of 34 monitoring stations in Beijing (r=20km).


### 4.2. Air Quality Prediction Model

To solve the prediction problem, we propose a spatiotemporal prediction model (referred as GAT-LSTM) as shown in Figure2. The model is built by a recurrent neural network incorporating graph attention mechanism, which means that it has encoder-decoder structure. The encoder is used to embed historical data, and the decoder is used to generate the predicted value in the future. It uses LSTM to model time correlation of a node’s own data and uses GAT to capture spatial correlation among nodes.Figure 2
The air quality prediction model GAT-LSTM.Suppose that the current time ist. Take node s as an example. In the encoding phase, we use k+1 LSTM units to receive s’s historical data from time t−k to time t, and form them into a recurrent neural network. For time it−k≤i≤t, the input of the corresponding LSTM unit is xsi;wi, where xsi is the concentration of target pollutant monitored by s at time i, and wi is the historical weather data at time i. Let hsi and csi be the output vector (blue lines in Figure 2) and the cell state vector (gray lines in Figure 2) of the LSTM unit at time i, respectively. Unlike the traditional approach passing hsi and csi directly to the next LSTM unit, we pass hsi to GAT to find spatial correlation among different nodes. Let zsi be the output vector of GAT for node s at time i. In the end, zsi and csi are passed to the next LSTM unit. The structure of LSTM unit is as shown in Figure 3.Figure 3
The LSTM unit in GAT-LSTM.In the decoding phase,l LSTM units are used to generate y^st+1;y^st+2;⋯,y^st+l, i.e., the predicted concentration of the target pollutant sampled by node s. For time jt+1≤j≤t+l, the input of the corresponding LSTM unit is ysj−1;wj, where ysj−1 is the prediction of the previous moment and wj is the weather forecast data at timej. As with the coding phase, we pass the LSTM’s output vector hsj to GAT to generate vector zsj. In addition to being passed to the next LSTM unit, zsj is also passed to a Feedforward Neural Network (FNN) to generate the output ysj.Base on the monitoring station graphGr, we use a GAT to model the spatial relationship among different nodes. In GAT, each node uses the attention mechanism [31] to collect information from neighbor nodes (weighting and summing the feature vectors of neighbor nodes) and uses the collected information to update its own feature vector. Unlike GCN, the weight of an edge in GAT is calculated based on the similarity of the feature vectors of the two corresponding nodes and changes dynamically with the change of the node’s data. GAT is very sensitive to the changes of the spatial correlation among nodes caused by weather factors such as wind speed and wind direction.The GAT mechanism can be demonstrated by Figure4. At any time, the input of GAT is the output vectors of all nodes’ LSTM units, i.e., hs1,hs2,⋯,hsn. The output of GAT is zs1,zs2,⋯,zsn. Each zs is passed to the next LSTM unit on the corresponding node as a hidden state vector. To get zs, for each s′∈Nrs, GAT firstly calculates the similarity score between node s′ and s by
(3)αs′,s=vTtanhU1hs′+U2hs,where vector v and matrix U1 and U2 are the parameters that need to be learned. Then, α^s′,s is calculated by normalizing all the αs′,s through the softmax layer:
(4)α^s′,s=softmaxαs′,s=expαs′,s∑u∈Nrsexpαu,s.α^s′,s can be seen as the weight of the edge s′,s in Gr. Finally, zs is calculated by a weighted summation of all its neighbors’ h, i.e.,
(5)zs=∑s′∈Nrsα^s′,shs′.Figure 4
The GAT mechanism for node 1 (suppose that node 2 ~ 6 are 1’s neighbors inGr).


### 4.3. Metalearning Algorithm

To solve the transfer learning problem, we propose a metalearning algorithm named MetaGAT-LSTM (given by Algorithm1) for training the GAT-LSTM model in the target city with insufficient training data. The algorithm will build an accurate prediction model by transferring knowledge from source cities with sufficient data. It uses a modified version of Model-Agnostic Meta-Learning (MAML) [28] as the parameter learning method.LetT=Dsc1,Dsc2,⋯,Dscm be the set of datasets from source cities. Define the distribution over T as PT, in which the probability of choosing dataset D∈T is
(6)PrD=DΣD′∈TD′.Letfθ with parameter θ be the prediction model at the beginning of each training iteration. At first, with respect to PT, we sample k datasets D1,D2,⋯,Dk from T with replacement (Line 3 in Algorithm 1). Then, get the next training batches B1,B2,⋯,Bk from D1,D2,⋯,Dk, respectively (Line 4 in Algorithm 1) and get the next training batch B0 from Dtc (Line 5 in Algorithm 1). The model’s parameter is updated in two steps. In the first step (Line 6 ~ 10 in Algorithm 1), for each Bi (1≤i≤k), we get the first-step adapted parameter θi by
(7)θi=θ−β∇LBifθ.Algorithm 1: MetaGAT-LSTM.
Input:T=Dsc1,Dsc2,⋯,Dscm: The set of the training datasets from source cities;Dtc: Datasets from target city;PT: Distribution over T;β,γ: Learning ratesOutput: fθ: The GAT-LSTM model for the target city1. Randomly initializeθ2.While not done do:3.  Samplek datasets D1,D2,⋯,Dk from T with replacement w.r.t. PT4.  Get next training batchesB1,B2,⋯,Bk from D1,D2,⋯,Dk, respectively5.  Get next training batchB0 from Dtc6.ForBi in B1,B2,⋯,Bkdo:7.   Calculate∇LBifθ with respect to Bi8.   Calculate first-step adapted parameterθi⟵θ−β∇LBifθ9.   Calculate∇LB0fθi with respect to B010.End for11.  Calculate second-step adapted parameterθ⟵θ−γ∑i=1k∇LB0fθi12.End whilewhereLBifθ is the loss of the original model fθ on training batch Bi, ∇LBifθ is gradient of LBifθ and β is the learning rate in the first step. In the second step (Line 11 in Algorithm 1), we get the second-step adapted parameter by
(8)θ=θ−γ∑i=1k∇LB0fθi,where LB0fθi is the loss of first-step adapted model fθi on training batch B0 and γ is the learning rate in the second step.


## 4.1. Monitoring Station Graph

In order to measure the mutual influence among different monitoring stations, we initially model all monitoring stations as a directed graphGr. Monitoring stations are represented by the nodes (vertices) in Gr. Given two nodes si and sj, there are directed edges si,sj and sj,si if the Euclidean distance between si and sj is less than or equal to r. r is the influence radius of monitoring stations, i.e., the maximum range affected by the pollutant in the diffusion process. As shown in Figure 1, by setting r=20km, we get the graph among 34 monitoring stations located in Beijing, China. The weights of the edges in Grwill be calculated by GAT mechanism and change over time. Hereinafter, we use the term “node” to refer to monitoring station and define set Nrs as
(2)Nrs=s∪s′∣s′,s∈Gr.Figure 1
The Graph of 34 monitoring stations in Beijing (r=20km).


## 4.2. Air Quality Prediction Model

To solve the prediction problem, we propose a spatiotemporal prediction model (referred as GAT-LSTM) as shown in Figure2. The model is built by a recurrent neural network incorporating graph attention mechanism, which means that it has encoder-decoder structure. The encoder is used to embed historical data, and the decoder is used to generate the predicted value in the future. It uses LSTM to model time correlation of a node’s own data and uses GAT to capture spatial correlation among nodes.Figure 2
The air quality prediction model GAT-LSTM.Suppose that the current time ist. Take node s as an example. In the encoding phase, we use k+1 LSTM units to receive s’s historical data from time t−k to time t, and form them into a recurrent neural network. For time it−k≤i≤t, the input of the corresponding LSTM unit is xsi;wi, where xsi is the concentration of target pollutant monitored by s at time i, and wi is the historical weather data at time i. Let hsi and csi be the output vector (blue lines in Figure 2) and the cell state vector (gray lines in Figure 2) of the LSTM unit at time i, respectively. Unlike the traditional approach passing hsi and csi directly to the next LSTM unit, we pass hsi to GAT to find spatial correlation among different nodes. Let zsi be the output vector of GAT for node s at time i. In the end, zsi and csi are passed to the next LSTM unit. The structure of LSTM unit is as shown in Figure 3.Figure 3
The LSTM unit in GAT-LSTM.In the decoding phase,l LSTM units are used to generate y^st+1;y^st+2;⋯,y^st+l, i.e., the predicted concentration of the target pollutant sampled by node s. For time jt+1≤j≤t+l, the input of the corresponding LSTM unit is ysj−1;wj, where ysj−1 is the prediction of the previous moment and wj is the weather forecast data at timej. As with the coding phase, we pass the LSTM’s output vector hsj to GAT to generate vector zsj. In addition to being passed to the next LSTM unit, zsj is also passed to a Feedforward Neural Network (FNN) to generate the output ysj.Base on the monitoring station graphGr, we use a GAT to model the spatial relationship among different nodes. In GAT, each node uses the attention mechanism [31] to collect information from neighbor nodes (weighting and summing the feature vectors of neighbor nodes) and uses the collected information to update its own feature vector. Unlike GCN, the weight of an edge in GAT is calculated based on the similarity of the feature vectors of the two corresponding nodes and changes dynamically with the change of the node’s data. GAT is very sensitive to the changes of the spatial correlation among nodes caused by weather factors such as wind speed and wind direction.The GAT mechanism can be demonstrated by Figure4. At any time, the input of GAT is the output vectors of all nodes’ LSTM units, i.e., hs1,hs2,⋯,hsn. The output of GAT is zs1,zs2,⋯,zsn. Each zs is passed to the next LSTM unit on the corresponding node as a hidden state vector. To get zs, for each s′∈Nrs, GAT firstly calculates the similarity score between node s′ and s by
(3)αs′,s=vTtanhU1hs′+U2hs,where vector v and matrix U1 and U2 are the parameters that need to be learned. Then, α^s′,s is calculated by normalizing all the αs′,s through the softmax layer:
(4)α^s′,s=softmaxαs′,s=expαs′,s∑u∈Nrsexpαu,s.α^s′,s can be seen as the weight of the edge s′,s in Gr. Finally, zs is calculated by a weighted summation of all its neighbors’ h, i.e.,
(5)zs=∑s′∈Nrsα^s′,shs′.Figure 4
The GAT mechanism for node 1 (suppose that node 2 ~ 6 are 1’s neighbors inGr).


## 4.3. Metalearning Algorithm

To solve the transfer learning problem, we propose a metalearning algorithm named MetaGAT-LSTM (given by Algorithm1) for training the GAT-LSTM model in the target city with insufficient training data. The algorithm will build an accurate prediction model by transferring knowledge from source cities with sufficient data. It uses a modified version of Model-Agnostic Meta-Learning (MAML) [28] as the parameter learning method.LetT=Dsc1,Dsc2,⋯,Dscm be the set of datasets from source cities. Define the distribution over T as PT, in which the probability of choosing dataset D∈T is
(6)PrD=DΣD′∈TD′.Letfθ with parameter θ be the prediction model at the beginning of each training iteration. At first, with respect to PT, we sample k datasets D1,D2,⋯,Dk from T with replacement (Line 3 in Algorithm 1). Then, get the next training batches B1,B2,⋯,Bk from D1,D2,⋯,Dk, respectively (Line 4 in Algorithm 1) and get the next training batch B0 from Dtc (Line 5 in Algorithm 1). The model’s parameter is updated in two steps. In the first step (Line 6 ~ 10 in Algorithm 1), for each Bi (1≤i≤k), we get the first-step adapted parameter θi by
(7)θi=θ−β∇LBifθ.Algorithm 1: MetaGAT-LSTM.
Input:T=Dsc1,Dsc2,⋯,Dscm: The set of the training datasets from source cities;Dtc: Datasets from target city;PT: Distribution over T;β,γ: Learning ratesOutput: fθ: The GAT-LSTM model for the target city1. Randomly initializeθ2.While not done do:3.  Samplek datasets D1,D2,⋯,Dk from T with replacement w.r.t. PT4.  Get next training batchesB1,B2,⋯,Bk from D1,D2,⋯,Dk, respectively5.  Get next training batchB0 from Dtc6.ForBi in B1,B2,⋯,Bkdo:7.   Calculate∇LBifθ with respect to Bi8.   Calculate first-step adapted parameterθi⟵θ−β∇LBifθ9.   Calculate∇LB0fθi with respect to B010.End for11.  Calculate second-step adapted parameterθ⟵θ−γ∑i=1k∇LB0fθi12.End whilewhereLBifθ is the loss of the original model fθ on training batch Bi, ∇LBifθ is gradient of LBifθ and β is the learning rate in the first step. In the second step (Line 11 in Algorithm 1), we get the second-step adapted parameter by
(8)θ=θ−γ∑i=1k∇LB0fθi,where LB0fθi is the loss of first-step adapted model fθi on training batch B0 and γ is the learning rate in the second step.


## 5. Experimental Results and Analysis

### 5.1. Dataset

We use real datasets collected from four cities in China (Beijing, Tianjin, Shenzhen, Guangzhou) to verify the effectiveness and efficiency of the proposed model and meta-learning algorithm. These cities are very different in geographic coordinates, city size, population density, etc., resulting in very different air quality distributions. For example, the air pollution situation in Beijing and Tianjin is much more serious than that in Shenzhen and Guangzhou. Each dataset contains the air quality data (from all monitoring stations), weather data, and weather forecast data collected from a city within one year. The period of data sampling is one hour. Taking the dataset of Beijing as an example, the air quality data contains the concentration of six major pollutants (PM2.5, PM10, SO2, NO2, O3, CO) and AQI sampled by 36 monitoring stations within one year. The weather data contains basic weather, temperature, humidity, air pressure, wind speed, and wind direction collected within one year. Weather forecast data contains the forecast value of the above weather indexes published by Beijing Meteorological Bureau. There are missing and dirty values in these datasets. In order to exploit the data as much as possible, we fill in missing values with the mean in a period of time and delete the tuples with too many consecutive missing data; Table2 shows the details of these datasets.Table 2
Data details.
AttributeNumber of different itemsBeijingTianjinShenzhenGuangzhouTime span2014/5/1-2015/4/302014/5/1-2015/4/302014/5/1-2015/4/302014/5/1-2015/4/30AQIs27802318960488139281436Station ID36271142Major pollutantsPM1027802318960488139281436NO2COO3SO2Historical weatherDistrict ID172075Basic weather1168671066143030555632TemperaturePressureHumidityWind speedWind directionWeather forecastDistrict ID172065Basic weather39070236162410638051870Wind strengthWind direction


### 5.2. Experiment Setting

There are two groups of experiments. In the first group, we compare GAT-LSTM with the most effective method to date and benchmark models. For each dataset, we divide it into train set and test set, then train these models on the same train set and evaluate their effectiveness on the same test set. The models used for comparison include the following:(i)
ARIMA: Auto Regressive Integrated Moving Average. ARIMA is the most common statistical model used for time series forecasting(ii)
LSTM [32]. LSTM can learn the time dependence in time series. Compared with RNN, they can deal with longer time series and obtain better results(iii)
ST-DNN [33]. The spatiotemporal models combing of Convolution Neural Networks (CNN) and LSTM for air quality prediction. ST-DNN is the most effective method to dateIn the second group, we set one city as target city and the other cities as source cities. Then, delete most of the data from the target city and only keep a small part for training. The proposed metalearning algorithm is used to build a prediction model by transfer learning knowledge from source cities with sufficient data. We compare the proposed metalearning algorithm MetaGAT-LSTM with the following transfer learning methods:(i)
Fine-Tuning. First, use the data of a single city to pretrain the GAT-LSTM model and, then, fine-tune the model on the target city, which is called the single-source domain fine-tuning (Single-FT). Secondly, use the data from multiple source cities to pretrain the GAT-LSTM model then fine-tune it on the target city, which is called the multisource domain fine-tuning (Multi-FT).(ii)
MAML [28]. Use data from all cities to jointly train the model for the target. MAML is implemented based on the metalearning methodThe target pollutant is AQI (can be seen as a single pollutant). We use Root Means Square Error (RMSE), Mean Absolute Error (MAE), and ACCuracy (ACC) to evaluate models, which are defined as(9)RMSE=1T∑X,y∈TfX−y22,(10)MAE=1T∑X,y∈TfX−y1,(11)ACC=1−1T∑X,y∈TfX−y1y1.Here,T is the test set. y is sample’s label (true monitoring data in the future) and fX is the predicted value of y. ∙1 and ∙2 are L1 and L2 norm, respectively.In GAT-LSTM, the dimension of the GAT’s output vector, the LSTM’s output vector, and cell state vector are all set to 128. While training, we use dropout [34] and batch normalization [35] to strengthen the training effect. The batch size is set to 64. The number of epochs is set to 3.


### 5.3. Experiment Results

At first, we need to find an appropriate influence radiusr for building the directed graph Gr in GAT-LSTM, so we compare the performance of GAT-LSTM with different r. Table 3 and Figure 5 give the comparison results on the dataset from Beijing. They show that when r<20km, the accuracy of GAT-LSTM increases as r increases. The reason for this phenomenon is that when r is within a reasonable range, a larger r allows the model to consider more spatial correlation, thereby providing a more accurate prediction. The accuracy reaches its peak at r=20km. When r>20km, the accuracy decreases slightly as r increases. This phenomenon is because too large r will cause the model to incorrectly estimate the correlation among some remote monitoring sites based on the data similarity. Thus, we set r=20 km in the following experiments.Table 3
The performance of GAT-LSTM with different influence radiusr on Beijing dataset (l=6).
r (km)RMSEMAEACC539.124.80.7671033.321.90.7782028.520.10.7992530.320.50.7974029.720.20.7966028.820.50.79810029.320.20.796Figure 5
The relationship between influence radiusr and prediction accuracy (l=6).In the first group of experiments, we use the dataset from Beijing to evaluate all the prediction models. We divide the dataset into training set (70%), validation set (20%), and test set (10%). Each model takes data from the past 48 hours as input (k+1=48), then outputs prediction values for the next l hours. Table 4 shows the experiment results with different l. The best results are marked in bold. It can be seen that the traditional linear model ARIMA does not perform well under the influence of multiple complex factors. LSTM’s performance is acceptable for short-term prediction and drops quickly with the increase of l. Spatial correlation plays an important role in air quality prediction. By using CNN to extract spatial correlation among monitoring stations, the ST-DNN performs much better than ARIMA and LSTM. However, the spatial correlation built by ST-DNN cannot change dynamically with the change of weather, which reduces its predictive effects. By using GAT to dynamically model spatial correlation, GAT-LSTM gives the best performance in all cases. The performance of all models declines with the increase of l, but the decline rate of GAT-LSTM is lower than the other three, which shows that it is suitable for long-term prediction.Table 4
Comparison of mean prediction results by different methods among 36 monitoring stations in Beijing (RMSE, MAE, ACC).
Methods+6 h+12 h+24 h+48 hRMSEMAEACCRMSEMAEACCRMSEMAEACCRMSEMAEACCARIMA53.340.70.62270.759.20.43294.168.20.321104.378.20.201LSTM48.234.50.72357.347.30.58271.255.10.47385.766.30.367ST-DNN35.623.80.76152.237.10.6763.249.30.54670.457.70.474GAT-LSTM28.520.10.79947.934.80.69855.745.20.62165.848.90.501In the second group of experiments, we execute two experiments by taking Beijing and Shenzhen as the target cities, respectively. We delete most of the data from the target city and only keep a small part for training. With different sizes of the training dataset (in target city), the results of comparing MetaGAT-LSTM with other transfer learning methods are given by Tables5 and 6. It can be seen all the methods performs better with larger training dataset. Table 5 shows that Single-FT from the Tianjin is better than that from the other two cities. Table 6 shows that Single-FT from Guangzhou is better than that from the other two cities. The climate and geographical location cause similarity of air conditions in Tianjin and Beijing, as well as the similarity of air conditions in Guangzhou and Shenzhen. This can be proven by Figure 6, in which the AQI distribution of the four cities from 2014/5/1 to 2015/4/30 is given. The more similar the two cities’ datasets are, the better Single-FT performs. Multi-FT enriches the training samples by using the data from all source cities. It is better than Single-FT in some cases. However, because of simply mixing all datasets, it may cause negative migration and give an even worse performance compared with Single-FT in some cases. Both MAML and MetaGAT-LSTM are better than the fine-tuning methods. MetaGAT-LSTM outperforms MAML in all cases by more rationally integrating data from all cities for joint training.Table 5
RMSE of transfer learning methods over different sizes of training dataset in the target city. (taking Beijing as the target city and other cities as the source cities).
Methods24 h72 h240 h720 h2400 hSingle-FTTianjin62.762.153.550.545.6Shenzhen71.370.767.661.055.2Guangzhou70.969.066.259.854.5Multi-FT63.161.255.352.543.1MAML59.457.849.445.136.2MetaGAT-LSTM55.853.745.340.231.4Table 6
RMSE of transfer learning methods over different sizes of training dataset in the target city. (taking Shenzhen as the target city and other cities as the source cities).
Methods24 h72 h240 h720 h2400 hSingle-FTTianjin77.376.272.066.358.1Beijing79.277.273.168.959.2Guangzhou65.660.351.949.343.7Multi-FT66.365.158.956.745.4MAML61.560.551.648.240.3MetaGAT-LSTM58.257.552.347.735.6Figure 6
AQI distribution map of four cities.


## 5.1. Dataset

We use real datasets collected from four cities in China (Beijing, Tianjin, Shenzhen, Guangzhou) to verify the effectiveness and efficiency of the proposed model and meta-learning algorithm. These cities are very different in geographic coordinates, city size, population density, etc., resulting in very different air quality distributions. For example, the air pollution situation in Beijing and Tianjin is much more serious than that in Shenzhen and Guangzhou. Each dataset contains the air quality data (from all monitoring stations), weather data, and weather forecast data collected from a city within one year. The period of data sampling is one hour. Taking the dataset of Beijing as an example, the air quality data contains the concentration of six major pollutants (PM2.5, PM10, SO2, NO2, O3, CO) and AQI sampled by 36 monitoring stations within one year. The weather data contains basic weather, temperature, humidity, air pressure, wind speed, and wind direction collected within one year. Weather forecast data contains the forecast value of the above weather indexes published by Beijing Meteorological Bureau. There are missing and dirty values in these datasets. In order to exploit the data as much as possible, we fill in missing values with the mean in a period of time and delete the tuples with too many consecutive missing data; Table2 shows the details of these datasets.Table 2
Data details.
AttributeNumber of different itemsBeijingTianjinShenzhenGuangzhouTime span2014/5/1-2015/4/302014/5/1-2015/4/302014/5/1-2015/4/302014/5/1-2015/4/30AQIs27802318960488139281436Station ID36271142Major pollutantsPM1027802318960488139281436NO2COO3SO2Historical weatherDistrict ID172075Basic weather1168671066143030555632TemperaturePressureHumidityWind speedWind directionWeather forecastDistrict ID172065Basic weather39070236162410638051870Wind strengthWind direction


## 5.2. Experiment Setting

There are two groups of experiments. In the first group, we compare GAT-LSTM with the most effective method to date and benchmark models. For each dataset, we divide it into train set and test set, then train these models on the same train set and evaluate their effectiveness on the same test set. The models used for comparison include the following:(i)
ARIMA: Auto Regressive Integrated Moving Average. ARIMA is the most common statistical model used for time series forecasting(ii)
LSTM [32]. LSTM can learn the time dependence in time series. Compared with RNN, they can deal with longer time series and obtain better results(iii)
ST-DNN [33]. The spatiotemporal models combing of Convolution Neural Networks (CNN) and LSTM for air quality prediction. ST-DNN is the most effective method to dateIn the second group, we set one city as target city and the other cities as source cities. Then, delete most of the data from the target city and only keep a small part for training. The proposed metalearning algorithm is used to build a prediction model by transfer learning knowledge from source cities with sufficient data. We compare the proposed metalearning algorithm MetaGAT-LSTM with the following transfer learning methods:(i)
Fine-Tuning. First, use the data of a single city to pretrain the GAT-LSTM model and, then, fine-tune the model on the target city, which is called the single-source domain fine-tuning (Single-FT). Secondly, use the data from multiple source cities to pretrain the GAT-LSTM model then fine-tune it on the target city, which is called the multisource domain fine-tuning (Multi-FT).(ii)
MAML [28]. Use data from all cities to jointly train the model for the target. MAML is implemented based on the metalearning methodThe target pollutant is AQI (can be seen as a single pollutant). We use Root Means Square Error (RMSE), Mean Absolute Error (MAE), and ACCuracy (ACC) to evaluate models, which are defined as(9)RMSE=1T∑X,y∈TfX−y22,(10)MAE=1T∑X,y∈TfX−y1,(11)ACC=1−1T∑X,y∈TfX−y1y1.Here,T is the test set. y is sample’s label (true monitoring data in the future) and fX is the predicted value of y. ∙1 and ∙2 are L1 and L2 norm, respectively.In GAT-LSTM, the dimension of the GAT’s output vector, the LSTM’s output vector, and cell state vector are all set to 128. While training, we use dropout [34] and batch normalization [35] to strengthen the training effect. The batch size is set to 64. The number of epochs is set to 3.


## 5.3. Experiment Results

At first, we need to find an appropriate influence radiusr for building the directed graph Gr in GAT-LSTM, so we compare the performance of GAT-LSTM with different r. Table 3 and Figure 5 give the comparison results on the dataset from Beijing. They show that when r<20km, the accuracy of GAT-LSTM increases as r increases. The reason for this phenomenon is that when r is within a reasonable range, a larger r allows the model to consider more spatial correlation, thereby providing a more accurate prediction. The accuracy reaches its peak at r=20km. When r>20km, the accuracy decreases slightly as r increases. This phenomenon is because too large r will cause the model to incorrectly estimate the correlation among some remote monitoring sites based on the data similarity. Thus, we set r=20 km in the following experiments.Table 3
The performance of GAT-LSTM with different influence radiusr on Beijing dataset (l=6).
r (km)RMSEMAEACC539.124.80.7671033.321.90.7782028.520.10.7992530.320.50.7974029.720.20.7966028.820.50.79810029.320.20.796Figure 5
The relationship between influence radiusr and prediction accuracy (l=6).In the first group of experiments, we use the dataset from Beijing to evaluate all the prediction models. We divide the dataset into training set (70%), validation set (20%), and test set (10%). Each model takes data from the past 48 hours as input (k+1=48), then outputs prediction values for the next l hours. Table 4 shows the experiment results with different l. The best results are marked in bold. It can be seen that the traditional linear model ARIMA does not perform well under the influence of multiple complex factors. LSTM’s performance is acceptable for short-term prediction and drops quickly with the increase of l. Spatial correlation plays an important role in air quality prediction. By using CNN to extract spatial correlation among monitoring stations, the ST-DNN performs much better than ARIMA and LSTM. However, the spatial correlation built by ST-DNN cannot change dynamically with the change of weather, which reduces its predictive effects. By using GAT to dynamically model spatial correlation, GAT-LSTM gives the best performance in all cases. The performance of all models declines with the increase of l, but the decline rate of GAT-LSTM is lower than the other three, which shows that it is suitable for long-term prediction.Table 4
Comparison of mean prediction results by different methods among 36 monitoring stations in Beijing (RMSE, MAE, ACC).
Methods+6 h+12 h+24 h+48 hRMSEMAEACCRMSEMAEACCRMSEMAEACCRMSEMAEACCARIMA53.340.70.62270.759.20.43294.168.20.321104.378.20.201LSTM48.234.50.72357.347.30.58271.255.10.47385.766.30.367ST-DNN35.623.80.76152.237.10.6763.249.30.54670.457.70.474GAT-LSTM28.520.10.79947.934.80.69855.745.20.62165.848.90.501In the second group of experiments, we execute two experiments by taking Beijing and Shenzhen as the target cities, respectively. We delete most of the data from the target city and only keep a small part for training. With different sizes of the training dataset (in target city), the results of comparing MetaGAT-LSTM with other transfer learning methods are given by Tables5 and 6. It can be seen all the methods performs better with larger training dataset. Table 5 shows that Single-FT from the Tianjin is better than that from the other two cities. Table 6 shows that Single-FT from Guangzhou is better than that from the other two cities. The climate and geographical location cause similarity of air conditions in Tianjin and Beijing, as well as the similarity of air conditions in Guangzhou and Shenzhen. This can be proven by Figure 6, in which the AQI distribution of the four cities from 2014/5/1 to 2015/4/30 is given. The more similar the two cities’ datasets are, the better Single-FT performs. Multi-FT enriches the training samples by using the data from all source cities. It is better than Single-FT in some cases. However, because of simply mixing all datasets, it may cause negative migration and give an even worse performance compared with Single-FT in some cases. Both MAML and MetaGAT-LSTM are better than the fine-tuning methods. MetaGAT-LSTM outperforms MAML in all cases by more rationally integrating data from all cities for joint training.Table 5
RMSE of transfer learning methods over different sizes of training dataset in the target city. (taking Beijing as the target city and other cities as the source cities).
Methods24 h72 h240 h720 h2400 hSingle-FTTianjin62.762.153.550.545.6Shenzhen71.370.767.661.055.2Guangzhou70.969.066.259.854.5Multi-FT63.161.255.352.543.1MAML59.457.849.445.136.2MetaGAT-LSTM55.853.745.340.231.4Table 6
RMSE of transfer learning methods over different sizes of training dataset in the target city. (taking Shenzhen as the target city and other cities as the source cities).
Methods24 h72 h240 h720 h2400 hSingle-FTTianjin77.376.272.066.358.1Beijing79.277.273.168.959.2Guangzhou65.660.351.949.343.7Multi-FT66.365.158.956.745.4MAML61.560.551.648.240.3MetaGAT-LSTM58.257.552.347.735.6Figure 6
AQI distribution map of four cities.


## 6. Conclusions

In this paper, we propose a spatiotemporal model GAT-LSTM by combining LSTM and GAT for air quality prediction, then design a metalearning algorithm for GAT-LSTM for transfer learning. By more accurately modeling the temporal and spatial correlation of pollutants at all monitoring stations, GAT-LSTM gives a better performance compared with the up-to-date air quality prediction models. In the case of insufficient training data from the target city, the proposed metalearning algorithm for GAT-LSTM can effectively transfer knowledge from source cities with sufficient data and jointly training an accurate prediction model. A number of comparative experiments show the effectiveness of the proposed prediction model and metalearning algorithm. In the future, we may consider more factors related to air quality to improve prediction’s accuracy. On the other hand, it will be reasonable to apply the proposed model and metalearning algorithm to other fields.



---

*Source: 9627776-2021-08-28.xml*