AirPhyNet: Harnessing Physics-Guided Neural Networks for Air Quality Prediction
Authors: Kethmi Hirushini Hettige
Jiahao Ji
Shili Xiang
Cheng Long
Gao Cong
Jingyuan Wang
Venue: ICLR 2024 poster (2024)
Keywords: air quality prediction
physics-informed
spatiotemporal-learning
interpretability
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
TL;DR: AirPhyNet is a physics-guided deep learning framework for air quality prediction. It shows superior performance in lead times upto 72-hours especially in sparse data scenarios while generating forecasts with a real physical meaning.

Abstract: Air quality prediction and modelling plays a pivotal role in public health and environment management, for individuals and authorities to make informed decisions. Although traditional data-driven models have shown promise in this domain, their long-term prediction accuracy can be limited, especially in scenarios with sparse or incomplete data and they often rely on black-box deep learning structures that lack solid physical foundation leading to reduced transparency and interpretability in predictions. To address these limitations, this paper presents a novel approach named Physics guided Neural Network for Air Quality Prediction (AirPhyNet). Specifically, we leverage two well-established physics principles of air particle movement (diffusion and advection) by representing them as differential equation networks. Then, we utilize a graph structure to integrate physics knowledge into a neural network architecture and exploit latent representations to capture spatio-temporal relationships within the air quality data. Experiments on two real-world benchmark datasets demonstrate that AirPhyNet outperforms state-of-the-art models for different testing scenarios including different lead time (24h, 48h, 72h), sparse data and sudden change prediction, achieving reduction in prediction errors up to 10\%. Moreover, a case study further validates that our model captures underlying physical processes of particle movement and generates accurate predictions with real physical meaning. The code is available at: https://github.com/kethmih/AirPhyNet

Published as a conference paper at ICLR 2024
AIRPHYNET: HARNESSING PHYSICS-GUIDED NEU- RAL NETWORKS FOR AIR QUALITY PREDICTION
Kethmi Hirushini Hettige
Jiahao Ji
School Computer Science and Engineering
School of Computer Science and Engineering Beihang University, China jiahao ji@buaa. edu. en
Nanyang Technological University, Singapore kethmihi 00100. attu.edu.5.
Shili Xiang
Cheng Long & Gao Cong
Institute for Infocomm Research A STAR, Singapore
School of Computer Science and Engineering Nanyang Technological University, Singapore {c long, gaocong] (aaooconneentuu @ntu. edu. sg
sxiangei2: a-ssar edu. sg
Jingyuan Wang
School of Computer Science and Engineering Beihang University, China
jywang@buaa www.://aaaaauucucaoooooo edu.
ABSTRACT
Air quality prediction and modelling plays a pivotal role in public health and en- vironment management, for individuals and authorities to make informed deci- sions. Although traditional data-driven models have shown promise in this do- main, their long-term prediction accuracy can be limited, especially in scenarios with sparse or incomplete data and they often rely on black-box deep learning structures that lack solid physical foundation leading to reduced transparency and interpretability in predictions. To address these limitations, this paper presents a novel approach named Physies guided Neural Network for Air Quality Prediction (AirPhyNet). Specifically, we leverage two well-established physics principles air particle movement (diffusion and advection) by representing them as differ- ential equation networks. Then, we utilize a graph structure to integrate physics knowledge into a neural network architecture and exploit latent representations to capture spatio-tempora relationships within the air quality data. Experiments on two real-world benchmark datasets demonstrate that AirPhyNet outperforms state-of-the-are models for different testing scenarios including different lead time (24h, 48h, 72h), sparse data and sudden change prediction, achieving reduction in prediction errors up 10%. Moreover, case study further validates that our model captures underlying physical processes of particle movement and gener- ates accurate predictions with real physical meaning. The code is available at: htttps://qitie com wwt.w/tttw//c////cc/oototttttttt hwwehhmmmihAixxhhett....
INTRODUCTION
Air pollution refers to the disruption of natural traits of the atmosphere through contamination of the environment by chemical, physical, or biological substances. World Health Organization (WHO) reveals that nearly the entire global population (99%) inhales polluted air surpassing the WHO guideline limits (Kan et al.] 2023). The resultant challenges have urged the need for advanced solutions in mitigating air quality issues. Thus, air quality prediction, involving the forecasting of future pollutant concentrations based on historical observations and other associated factors, has become a focal point of research.
In the literature, the air quality prediction problem is addressed under two primary paradigms: physics-based based and data-driven. Physics-hased models inspired by atmospheric science and lever-

---

Published as a conference paper at ICLR 2024
age domain-specific knowledge to formulate governing equations that represent atmospheric processes. For example, Chemical Transport Models (CTMs) simulate atmospheric dynamics such as diffusion, advection, and deposition using ordinary partial differential equations eti al. [023 Jacob, 1999). However, these models involve solving complex differential equations and are often employed on larger spatial scales due their higher computational overhead (Bauer et al.) 2015 2020). This restricts the real-time and fine-grained predictions. Moreover, the majority of these models require parameter calibration which may limit their ability intricate real-world conditions such as complex human behaviors.
On the other hand, data-driven approaches utilize historical pollution data to learn complex re- lationships without the need for knowledge of the underlying physical processes. Among these methods, deep learning-
approaches have gained substantial attention over the past few years due to their enhanced expressive ability. For example, Spatio Temporal Graph Neural Networks (STGNNs) (Ji et al. 2023) which integrate GNNs and sequential models such as Recurrent Neural Networks (RNNs) have been employed to learn both spatial and temporal dependencies, thereby enhancing the prediction accuracy (Chen et al. 202T). Furthermore, recently attention- -based mod- els (Wang al., 2022 Liang et al., 2023) have emerged as a robust alternative in the domain of air quality prediction specifically enhancing forecasts in the long term. However, despite the promising outcomes, these models also have their limitations. The majority of these models require extensive training data to achieve accurate long-term predictions. Furthermore, the absence of physics con- straints in these models can limit their generalizability, potentially compromising their effectiveness in sparse data scenarios. Moreover, the inherent "black-box" nature of deep learning models raises challenges on interpretability which affects the decision-makin in urban planning.
To alleviate the respective limitations of each paradigm, we present a novel hybrid learning ap- proach: Physics guided Neural Network for Air Quality Prediction (AirPhyNet). This model at- tempts to leverage the strengths of both physics-based and data-driven methods contributing to a comprehensive framework, which can generate accurate air quality forecasts with a physical mean- ing. Specifically, the major motivation behind AirPhyNet is to use underlying physics related to air particle movement as domain knowledge to design an efficient and interpretable learning system. Accordingly, analogous to the differential equation-based traffic forecasting networks (Ji et al., 2022]. in this work we exploit diffusion and advection processes within an end-to-end deep learning framework.
The overall AirPhyNet architecture consists of three components: 2) an RNN-based encoder that ex- tracts temporal information from the historical air quality data, ii) GNN-based differential equation network that captures the air pollutant dispersion and flow dynamics, and iid) a decoder that gen- erates the final pollutant concentrations based on the learned dynamics of physical processes. We evaluate our approach against several state-of-the-ae baselines for forecasting air quality (PM2.5 concentrations in particular) up days ahead. Evaluations on two real-world benchmark air qual- ity datasets show that AirPhyNet consistently outperforms baselines by a large margin, especially in sparse data scenarios. In summary, our contributions are as follows:
We investigate the fundamental physics processes governing air pollutant transport and incorporate them into a comprehensive physics-guided deep learning framework. To the best of our knowledge, this is the first physics-guided hybrid deep learning approach of its kind introduced for outdoor air quality prediction.
We propose novel deysics-guided deep learning model which integrates the physics-based dispersion processes and neural networks into one framework with tailored GNN-based differential equation network.
We conduct extensive experiments on two real-world air quality datasets and show that the proposed model outperforms the state-of-the-arr models by significant margin. Further- more, case study validates our model's 's ability to unveil the physical mechanism of air pollutant transport, thereby verifying its interpretability.
2 RELATED WORK
Air Quality Prediction Air quality prediction one of the fundamental objectives within the scope of smart city and urban planning initiatives. Significant research efforts have been dedicated to
2

---

Published as a conference paper at ICLR 2024
this domain over the years. Existing approaches can be primarily divided into two classes. (1) Physics based Models and (2) Data driven models. The former category generates predictions by emulating the dispersion of different air pollutants, relying the principles of physical laws (Vardoulakis et al. 2003 Liu et al. 2005). These models portray the dynamics of air pollution, including processes like diffusion, advection, and chemical transformation a of differential equations, which are then solved through numerical simulations (Arystanbekova) [2044] Daly & Zannetti 2007). However, they often fail capture the real-world air pollution dynamics and de- mands for higher computational power. Recently, data-driven approaches have gained prominence, where historical observations and other contextual data (POI distributions, weather data) are used to capture spatiotemporal correlations (Han et al., 2021 2023). The shallow machine learning methods (Suárez Sánchez al., 2013, Su et (023) have limited capacity to model complex relationships while several deep learning models (Luo et al., 2019; Wang et al., 2022, Liang et al., 2023) have showed enhanced performance capturing complex spatiotemporal dependencies. However, these models require ample data for precise air quality prediction and they fail to encompass physical knowledge leading to limited generalization capability and interpretabilityy.
Physies Guided Deep Learning Recently, several studies have been introduced to incorporate phys- ical knowledge for data driven approaches (Raissi et al., 2017]] Peng et al.] (022). While some of these approaches incorporate physical constraints into the loss function, other approaches present hybrid frameworks. For example, Jia et al. (2019) introduces a physics based regularized loss func- tion in predicting lake temperature Wang al. (2020a) proposes composite framework for turbu- lent flow prediction exploiting critical physical characteristics of the process while Ji et al. (2020) and Wang et al. (2023) present an interpretable model for grid based traffic flow prediction based on potential energy field. Similarly, Ji et al. (2022) presents physics guided neural network ap- proach for road network based traffic flow prediction. Moreover, Mohammadshiraazi et al.) (2023)
introduces framework which combines physics based state-space models with machine learning techniques for indoor air quality prediction. To the best of our knowledge, we introduce the pioneer- ing work applying physics-guide deep learning in the realm of outdoor air quality prediction by proposing a hybrid learning framework, that integrates the physics- based dispersion processes into deep learning framework with custom designed deep neural networks.
METHODOLOGY
3.1 AIR QUALITY PREDICTION PROBLEM
The goal of air quality prediction is forecast future PM2.5 concentration based historical con- centrations and other related variables gathered from N monitoring stations within a sensor net- work. We represent the sensor network as a directed graph g (V, E, W) where V is a set of nodes V N, E' a of edges and W E N is weighted adjacency matrix indicating the proximity of nodes in terms of distance or flow field (see details in Section 2.3). A node v; E V represents an air quality monitoring station while Ei,; E corresponds a directed edge from node V; to node V, which a potential pathway for air pollutant transport. Let X, E R and Pt E R Nx2 denote the node features at time step t. The air quality prediction problem aims to learn function h(.) that predicts concentration over the next steps minimizing prediction error:
{ = (X(=-T+1:) P(-T-1:0) G)
(1)
where X(-T41=#) E T represents the historical PM2.5 concentrations, P(--+4::) e xxxN22 represents the wind related attributes (i.e. wind speed and direction), 9 represents the corresponding graph network and X((+1+++)) E RTxNx1 represents the predicted future PM2.5 concentrations.
3.2 PHYSICAL DYNAMICS OF AIR POLLUTANT TRANSPORT
This work aims to design a physics guided deep learning model that incorporates physics based principles of air particle movement into deep neural network. Thus, first provide an overview of the physical techniques which are utilized on this idea and formulate them in the context of air pollutant transport as the foundation for this approach. All these techniques derived based
3

---

Published as a conference paper at ICLR 2024
GNN Based DE Network
Sampling
Encoder
210
Decoder
21.
Inyrst Kou7426
plano Xp-rea)))
ODEStbwe(ng Fo:th 10 bl
Ordynii
F Fa t,
Ba
Gated Fusion
Hag
Hady
DIRGON
ADVGON
Diffusion
Advection
Wind
Figure The overall architecture of AirPhyNet framework consisting of a RNN Encoder, GNN based DE Network and Decoder. Heatmap indicate the PM2.5 concentrations while the dashed arrows represent the air pollutant transport between nodes due to diffusion and advection.
on the continuity equation which is one of the fundamental principles in physics and engineering. describes the principle of conservation of mass within closed system in mathematical form (Lienhard & Lienhard 2008). In relation to our problem statement, here we use the continuity equation describe the movement particles through space as:
ax Øt
div F
(2)
where X is PM2.5 concentration, F is the flux of particles which describes the transport of PM2.5 particles and div is the divergence operator. In simple terms, div describes how the concentration changes at particular point in space due to the transport of particles into or out of that point.
Diffusion is a key physical phenomenon that characterizes the random motion particles from high concentration place to low concentration place within medium. Diffusion process can be formulated using the continuity equation and Fick's Law (Grady al.] 2010). Eq. explains the relation between the change of concentration of substance and transport of that substance in space. On the other hand, Fick's Law (Paul al. 2014) describes the flux of particles due diffusion and states that the magnitude of flux directly proportional to the concentration gradient, i.e. = -V X where k represents the diffusion coefficient and V X represents the concentration gradient. In simple terms this equation indicates that the flux moves from high concentration regions to low concentration regions, with a magnitude proportional to the concentration gradient. By substituting the expression for the flux F into 2 follows:
we obtain the diffusion equation for PM2.5 Concentration as
0X Øt = div
(3)
Advection another fundamental physical process that describes the transport of particles due to flow field like bulk movement of air, typically driven by wind patterns or other large-scale air motions. In contrast to the diffusion process which explains the transport of particles due to the difference in concentration gradients, advection explains the transport of particles due to the effect of an external flow field. Thus, an advection process the flux is formulated in terms of a vector field. i.e. F = E (Grady et al. 2010). By substituting this formulation of flux F into Eq 2 we obtain the the advection equation for PM2.5 Concentration as follows:
0x
0t = -iu (TX)
(4)
4

---

Published as a conference paper at ICLR 2024
3.3 DIFFUSION-ADVECIII DIFFERENTIAL EQUATION
Air pollutant transport is a classical scenario which is governed by the aforementioned processes of diffusion and advection. Accordingly, this section, we define these two processes on sensor network and thereby propose equation for change of PM2.5 concentration over space and time incorporating the air pollutant transport dynamics within a sensor network.
Diffusion on sensor network refers to the dispersion of air pollutants within the network based on the concentration gradients between neighbouring sensor nodes. According to [Bronstein et al. (217)), the graph laplacian operator A can be expressed in terms of the divergence of the gradient as A V. Hence we re-write the diffusion equation (Eq(3) in terms of the graph laplacian as follows:
0x at
diiff - AX = - LLX
(5)
where L is the scaled laplacian computed using the distance based weighted adjacency matrix Wa such that w = 1/dey where dej is the haversine distance between nodes and j. In this study we consider the diffusion coefficient as & 0.1 (Cussler 2009).
Advection on a sensor network refers to the movement of air pollutants within a network, due to the effect of flow field. To define advection on the sensor network, we follow the formulation proposed by Chapman & Chapman (2015). Accordingly, the discrete analogue of advection equation (Eq. 4
can be stated as:
0x
Σ Vili+ta
X
Σ vkli-tk
- -MM
(6)
Ot
which states that the rate of change of PM2.5 concentration of node i is the difference between the flow of particles in and out the node due to the effect of the flow field. Here we consider Mg as modified laplacian determined using the flow field information edge weights of the graph G.
Generally, the flow field represents a vector field that characterizes the velocity of air particles at different locations and times within the system. In this work, we derive the flow field primarily based on wind speed and direction. Accordingly, we initially take the wind features P(:T++1:)) and transform to a high dimensional space using Multi Layer Perceptron (MLP). Then, we obtain the edge weight by the difference of the transformed wind representation of each source and destination node pair such that W p = =p' - p¹ where p² and p' are transformed wind representations of source node (i) and destination node (j) respectively (see Appendix A.2 for more details). Accordingly, the final advection differential equation can be stated in terms of scaled laplacian (M) computed using the flow-field based weighted adjacency matrix (Wp) as:
0x 1e
adv - [MX]
Diffusion-Advenn Differential Equation Function In a real world scenario. the chemical con- centration undergoes changes due to both of the above explained physical processes. To consider both diffusion and advection simultaneously, sion process results in the following function which we name as the Diffusion-AAvvection Differential Equation (DE):
gated fusion mechanism. This adaptive fu-
ax 0x,
dx 0
ax
a
diff + (1 (1-a) a) o
ade
01
01
0x
= a o Haffy + (1- 3 Hadv
(8)
01
with
a= o oHHai W + Hade W. + +ba)
where Hasfy and Hade are the outputs of the diffusion and advection processes respectively, Wa.1 Wa and ba are learnable parameters, represents the element-wine product, a(.) denotes the sigmoid activation and a is the gate. Eq describes the dynamic evolution of PM2.5 concentration within the sensor network.
5

---

Published as a conference paper at ICLR 2024
3.4 PHYSICS GUIDED DIFFERENTIAL EQUATION NETWORK
Based on the physical processes elaborated above and the derived Diffusion-AAvectioon DE, we propose the Physics guided Neural Network for Air Quality Prediction (AirPhyNet). The model architecture of AirPhyNet (see Figure D is inspired by the seq2seq Neural ODE (Chen et al. 2018) which extends conventional neural networks into continuous structure with ordinary differential equations, Nir consists of three main components: an RNN-based encoder to encode PM2.5 concentrations into an initial state, a GNN-based differential equation network governed by Eq. B that captures the physical dynamics of air pollutant transport and decoder that generates the final PM2.5 concentrations based on the learnt dynamics of physical processes.
We initially employ Gated Recurrent Unit (GRU) (Chung et al. 2014) as the RNN Encoder to extract the temporal information from the historical PM2.5 concentration sequence. Then, fully connected network g() is employed to determine the mean and standard deviation of #ig from the final hidden states of GRU:
(12g 1 Oz1g J (GRU (X (X(+-++14))
Following (Ji et al. 2022 we determine the 2t, based reparameteriization trick Kingma & Welling 2013). I... 2to Eiflz + Jgg where E, is sampled from a standard normal distribution. Inspired by the neural ODEs, we then model the Diffusion-Advection DE in Eq. 8 as follows:
Oz,
Öt Fg (1,1,
(10)
where & represents trainable parameters including the diffusion coefficient (k) and Fg is a neural network informed by the physics based DE in Eq. 8 Accordingly, the function Fg is formulated in terms of residual graph convolution network (GCN) layers in the form of:
Fg (Ø, (x) -a ④ k @ Tanh (IX)- - (1 a)) @ Tanh (MX)
(11)
where graph L and .M are the graph laplacian operators which use adjacency matrices with distance- based edge weights (W2) and flow field-based edge weights (W,) respectively. These operators calculate how the concentration of a particular node changes due to the diffusion and advection of pollutants from its neighbouring nodes. Tanh() is the activation function (see Appendix |A.3 for details). The results of the GCN layers corresponding to each process are aggregated using the gated fusion mechanism employed in Eq Given the initial state Zto and F&g, we apply the neural ODE solver (Chen et al. 2018) to compute the future states ((tr:t));
P(ta:t,) ODESolve (Zto Fg. Φ, [to...,,+))
(12)
Finally, the decoder module uses the output from the GNN-based DE Network representing the latent space, to generate predictions.I employs reshaping and aggregation steps to produce the desired output format with the correct dimensionality at each time step. AirPhyNet is trained through back- propagation by minimizing the Mean Absolute Error (MAE) (see A.4) between predicted values and ground truth values.
4 EXPERIMENTS
4.1 DATA DESCRIPTION
We assess the performance of model on real world air quality datasets collected from two urban centres in China, Beijing and Shenzhen. Beijing datase has 35 major monitoring stations and data spans from 2017/01/01 to 2018/05/30 while the Shenzen datase has 11 monitoring stations and data spans from 2014/05/01 to 2015/04/30. Both datasets consist of hourly air quality and weather related observations which include concentrations of major pollutants (PM2.5, PM10, O3, NO2, SO2, and CO), temperature, pressure, humidity, wind speed and wind direction. Following majority of the previous work in the domain, this study focuses on PM2.5 concentration as the target variable. Additionally, we consider wind speed and wind direction as auxiliary variables. The missing values are imputed with the preceding 24-hour mean value of the respective station. We split each dataset chronologically in the ratio of 7:1:2 to generate distinct training, validation, and test sets respectively. Similar to the prior studies (Wang et al. 2020b Liang et al. (023) we consider 3 hours as a time step and use past 72-hour (24 steps) data to predict the future 72 hours.
http:///dwwweeeeeaaaaaaaattddddc 195009

---

Published as a conference paper at ICLR 2024
4.2 EXPERIMENTAL SETTINGS
Baselines We compare our proposed AirPhyNet model with ten baselines under three categories. 1) Classical Methods: Historical Average (HA) and Vector Auto-Regression (VAR) 2)
Deep Learning Methods:Diffusion Convolution Recurrent Neural Network (DCRNN) (Li et al. Spatio-temporal Graph Convolutional Network (STGCN) (Yu et al. (0177) GMAN (Zheng al. 2020) and GTS (Shang et al. (021) along with two powerful air quality prediction models
2017
PM2.5-GNN (Wang et al. 2020b and AirFormer (Liang et al. 2023). 3) Differential equation
network based methods: Latent-ODE (Rubanova et al. (2099] and ODE-LSTM (Lechner & Hasani
2020).
See Appendix A.3 for more details on baselines.
Implementation Details Our model is implemented in PyTorch 2.0.1 using NVIDIA GeForce RTX 3070 GPU. Adam optimizer is used for training the model. We set the batch size to 32 and use an initial learning rate of 5e-4 which decays over specific steps with a decay rate of 0.1. We use a GRU in the RNN encoder to encode the sequence and obtain the initial state. A grid search is conducted over (16, 32, 64, 128), and 64 is chosen as number of hidden units of the GRU. In the ODESolver, use dopri5 the numeral integration method with relative tolerance (rtol) and absolute tolerance (atol) set to le-5. Experiments on all deep learning models are conducted five times and the average results are reported. We use Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) to evaluate the chosen baselines.
4.3 PERFORMANCE COMPARISON
Table the performance comparison of AirPhyNet with baselines over three time horizons on two datasets. Accordingly, AirPhyNet outperforms all competing baselines in both metrics, across all time horizons for both Beijing and Shenzen data. Compared to the second-best best method, on average it achieves reduction in MAE and RMSE by 3.7% and 6.1% respectively which shows the effectiveness of employing physics based knowledge in a deep learning framework in capturing complex spatiotemporal relationships.
Table I: Overall prediction performance comparison on metries MAE and RMSE. The bold and underlined font show the best and the second best result respectively.
Beijing Data
Shenzen Data
Model
24h
48h
MAE
RMSE
MAE
RRMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
HA
83.91
8.35
12.52
9.72
14.34
10.54
15.39
VAR
60.10
102.92
60.44
103.02
60.64
103.07
21.50
26.09
21.84
26.50
22.17
26.92
DCRNN
35.99
52.55
49.66
67.50
57.01
74.67
7.76
11.54
9.99
14.11
10.98
15.
STGCN
7.23
10.61
9.35
12.99
9.97
13.59
GMAN
50.62
66.05
50.73
66.07
50.69
65.87
9.76
12.70
10.08
13:22
10.07
12.99
GTS
34.99
51.45
54.18
71.87
73.50
89.59
6.58
9.55
8.70
12.15
10.54
13.94
PM25GINN
50.94
65.87
48.81
65.64
51.51
66.55
9.90
12.31
10.02
12.63
10.30
12.85
AirFormer
29.62
46.49
38.43
56.52
43,39
58.68
7.24
10.83
9.66
13.47
10.21
13.98
LatenrODE
44.83
53.96
45.95
55.44
47.14
57.39
9.85
12.30
10.24
12.89
10.73
13.33
LDE
46.19
57.56
49.18
62.39
51.45
63.66
10.55
13.24
11.36
14.18
12.03
14.81
Net
29.11
42.16
36.69
48.66
42.23
53.07
6.38
9.48
8.18
11.38
9.51
12.51
% Improvement
1.73
9.33
4.53
11.49
2.66
6.19
3.01
0.76
5.97
6.38
4.58
2.64
From Table I we can also observe that: 1) Deep-learning based approaches outperform the classical methods including HA and VAR demonstrating their capability of learning complex relationships. 2) Spatio-temporal Deep Learning approaches initially designed for traffic forecasting such as GTS and STGCN demonstrate strong adaptability for the air quality forecasting problem and yield competing results as the state of the art air quality prediction methods, such as the AirFormer.3) Neural DE net- work based approaches have demonstrated less effective performance compared to spatio-temporal deep learning methods due to their inability to capture spatial relationships which are vital for ac- curate air quality prediction. All in all, these results verify the ability of AirPhyNet in achieving precise air quality predictions while leveraging the physics based domain knowledge of air particle movement.
7

---

Published as a conference paper at ICLR 2024
We also evaluated the performance of models in predicting sudden changes in air quality. Compared to the baselines, AirPhyNet shows reduction prediction errors upto 8% in sudden change prediction exhibiting its' effectiveness in understanding the system in terms of physical processes and thereby predicting abrupt shifts in air quality. Complementary to the primary evaluations, we additionally conducted experiments assess how well our model generalizes to unseen data, which is identified as common advantage of physics-based models, Based on the results, AirPhyNet has outperformed all baselines across all time horizons by significant margin. Specifically, the MAE and RMSE have been reduced by around 8% on average compared to the second best method. This emphasizes the superiority of our model's ability to generalize effectively and achieve precise long term air quality predictions particularly in scenarios with limited data (i.e. regions with limited number of monitoring stations). See A.6 A.6 for detailed results.
4.4 ABLATION STUDY
Effect of Physical knowledge: To further illustrate the effect of incorporating physical knowledge compared AirPhyNet with two other variants. AdvOnly model which only considers the ad- vection process b) DiffOnly model which only considers the diffusion process. Fig. 2a shows the results comparison for Beijing dataset.
TCNEnc
AirPhyNet
oifoclly
Artthybeet
21.5
(a) Effect of Physical Knowledge in AirPhyNet
(b) Effect of Encoder
Figure 2: Results of Ablation Study
According to the results, we observe that AirPhyNet outperforms both AdvOnly and DiffOnly mod- els. demonstrating the advantage of leveraging physical knowledge predict air quality. Moreover, DiffOnly model performs better than AdvOnly model which shows that the advection process alone cannot adequately capture the the air pollutant transport dynamics while the diffusion process has shown better capability in doing so. However, DiffOnly model exhibits slightly lower performance compared to AirPhyNet due to the absence of the advection component. Thus, it highlights the im- portance of identifying the appropriate combination physical processes to comprehensively address the problem.
Effect of Encoder:
To study the efficacy of the RNN-based encoder in capturing temporal depen- dencies, we compare our model with another variant TCNEnc which replaces the RNN encoder with a Temporal Convolutional Network (TCN) based encoder. According to the results in Fig 2b] we observe that AirPhyNet outperforms TCNEnc model by relatively by a large margin, demonstrating the advantage of utilizing GRU in capturing temporal dependencies.
4.5
CASE STUDY
In this section, we conduct case study to demonstrate the strong performance of our approach qualitatively in terms of the two physical processes diffusion and advection. To illustrate the in- terpretability with regard to advection, we visualize the predicted PM2.5 concentrations together with the wind direction attributes for both datasets. Fig. show the spatial distribution of monitoring stations of the Beijing dataset and Shenzen dataset at two timesteps which are four hours apart. In Beijing dataset, wind flows towards the southwest direction in the respective times. It can be ob- served that the PM2.5 particles have moved towards the direction of the wind with time, specifically in the circled regions. PM2.5 concentration has reduced in the circled region over the four-hour pe- riod and the particles have moved and joined the PM2.5 cloud heading towards the southwest wind direction. Similarly in Shenzen dataset, the wind flows in the East direction. PM2.5 concentration
8

---

Published as a conference paper at ICLR 2024
has increased over the four-time steps in the circled region and joined the PM2.5 cloud moving towards the east wind direction.
Bejlnng Dataset: Southwrst Wind
FREE
Shendhen Wind
Shenchen Wind
Kong
Kong
Figure 4: Visualization of predicted PM2.5 concentrations and diffusion from Station 2017-11-13 14:00. line segments rep- resent diffusion from Station to its neigh- bouring stations and the magnitude of diffu- sion is reflected by the line colour.
Figure 3: Visualization of predicted PM2.5 concentrations and wind direction. Heatmap represents the PM2.5 concentrations and the arrows indicate the wind direction
Next, visualize the predicted PM2.5 concentrations and magnitude of diffusion Fig. shows the spatial distribution of subset of stations of the Beijing dataset where the PM2.5 concentra- tions predicted by the AirPhyNet neighbouring stations is represented by line segments. The visualization shows that the diffusion of particles from Station_3 3 relatively higher to stations with low concentrations (Station_1. Station_7) while it's low to stations with higher concentration (Station 2). Therefore, the predicted concentra- tions governed by physical process capture real mechanism of physical diffusion process which states that the particles move from places with high concentration to low concentration. Thus, the predicted concentrations can be interpreted in terms diffusion. In summary, the findings this section affirm that AirPhyNet successfully captures the underlying physical principles of particle movement and yields precise predictions with a physical meaning.
represented by heat map and diffusion from Station 3 to its
CONCLUSION AND FUTURE WORK
In this paper, introduced approach for air quality prediction based the application of diffu- sion and advection based differential equations which serve as the foundational principles governing the air pollutant transport in the atmosphere. We propose novel Physies guided Neural Network known as AirPhyNet that leverages these fundamental physical equations and seamlessly incorpo- rates them into deep learning architecture. Experiments conducted on two real world datasets show that AirPhyNet reduced prediction eтrors by a significant margin (1%))))) compared to the existing methods. case study further affirms model's capability generate precise predictions that can be interpreted within physical context. However, the applicability of our approach to pollutants remains uncertain as such complex chemical behaviours are not captured by the model. In the future, we plan to expand the applicability of our model across wide spectrum of air pollutants by further augmenting it with chemical kinetics. could enhance the predictive capabilities for pollutants governed by complex chemical processes. Moreover, aim to explore its capabilities for cross-city air quality prediction through transfer learning, PM2.5 source tracking and plume modelling.
highly

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENT
This work is partially supported by the National Research Foundation, Prime Minister's Office, Singapore under the Aviation Transformation Programme. Kethmi Hirushini Hettige's 's work is sup- ported by the Singapore International Graduate Award. Jingyuan Wang's work is supported by the National Natural Science Foundation of China No. 72222022. We would also like express our sin- cere gratitude to the anonymous reviewers for their invaluable feedback, which greatly contributed to the improvement of this manuscript.
REFERENCES
N Kh Arystanbekova. Application of gaussian plume models for air pollution simulation at instan- taneous emissions. Mathematics and Computers in Simulation, 67(4-5):451-4555 2004.
Peter Bauer, Alan Thorpe, and Gilbert Brunet. The quiet revolution of numerical weather prediction. Nature, 525(7567):47-55, 2015.
Peter Bauer, Tiago Quintino, Nils Wedi, Antonio Bonanni, Marcin Chrust, Willem Deconinck, Michail Diamantakis, Peter Dübben, Stephen English, Johannes Flemming, et al. The ECMWF scalability programme: Progress and plans. European Centre for Medium Range Weather Fore- casts, 2020.
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geomet- ric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18-42, 2017.
Airlie Chapman and Airlie Chapman. Advection on graphs. SemiiAutonomous Networks: Effective Control of Networked Systems through Protocols, Design, and Modeling, pp. 3-16, 2015.
Ling Chen, Jiahui Xu, Binqing Wu, Yuntao Qian, Zhenhong Du, Yansheng Li, and Yongjun Zhang. Group-aware graph neural network for nationwide city air quality forecasting. arXiv preprint arXiv:2108. 12238, 2021.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
Edward Lansing Cussler. Diffusion: mass transfer in fluid systems. Cambridge university press, 2009.
Aaron Daly and Paolo Zannetti. Air pollution modeling-an overview. Ambient air pollution, pp. 15-28, 2007.
Leo J Grady, Jonathan R Polimeni. Leo J Grady, and Jonathan R Polimeni. Discrete calculus: History and future. Discrete Calculus: Applied Analysis on Graphs for Computational Science, pp. 1-9, 2010.
Jindong Han, Hao Liu, Hengshu Zhu, Hui Xiong, and Dejing Dou. Joint air quality and weather prediction based on multi-adversarial spatiotemporal networks. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 35, pp. 4081-4089, 2021.
Jindong Han, Hao Liu, Hengshu Zhu, and Hui Xiong. Kill two birds with one stone: multi-view auvrs-aaversariall learning approach for joint air quality and weather prediction. IEEE Transac- tions on Knowledge and Data Engineering, 2023.
Daniel Jacob. Introduction to atmospheric chemistry. Princeton university press, 1999.
Jiahao Ji, Jingyuan Wang. Zhe Jiang, Jingtian Ma. and Hu Zhang. Interpretable spatiotemporal deep learning model for traffic flow prediction based on potential energy fields. 2020 JEEE International Conference Data Mining (ICDM), pp. 1076-1081. 2020.
10

---

Published as a conference paper at ICLR 2024
Jiahao Ji, Jingyuan Wang, Zhe Jiang, Jiawei Jiang, and Hu Zhang. Stden: Towards physics-guided guided neural networks for traffic flow prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 4048 40584 2022.
Jiahao Ji, Jingyuan Wang, Chao Huang, Junjie Wu, Boren Xu, Zhenhe Wu, Zhang Junbo, and Yu Zheng. Spatio-temporal self-supervised learning for traffic flow prediction. Proceedings of the AAAI Conference on Artificial Intelligence, 37(4):4356-433 4364, 2023.
Xiaowei Jia, Jared Willard, Anuj Karpatne, Jordan Read, Jacob Zwart, Michael Steinbach, and Vipin Kumar. Physics guided mns for modeling dynamical systems: A case study in simulating lake temperature profiles. In Proceedings of the 2019 SIAM international conference data mining, pp. 558-566. SIAM, 2019.
Haidong Kan, Tiantian Li, and Zhenyu Zhang. Air pollution and cardiovascular heath in changing climate, 2023.
Diederik P Kingma and Max Welling. arXiv:1312.6114, 2013.
Auto-encoding variational bayes.
arXiv preprint
Mathias Lechner and Ramin Hasani. Learning long-term dependencies in irregulaty-saappled time series. arXiv preprint arXiv:2006.0444 2020.
Lianfa Li, Roxana Khalili, Frederick Lurmann, Nathan Pavlovic, Jun Wu, Yan Xu. Yisi Liu, Karl O' Sharrrky,, Beate Ritz, Luke Oman, et al. Physics-informed deep learning to reduce the bias in joint prediction of nitrogen oxides. arXiv preprint arXiv:2308.0744: 07444, 2023.
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffussio convolutional recurrent neural net- work: Data-driven traffic forecasting, arXiv preprint arXiv:1707.01926, 2017.
Yuxuan Liang, Yutong Xia, Songyu Ke, Yiwei Wang, Qingsong Wen, Junbo Zhang, Yu Zheng, and Roger Zimmermann. Airformer: Predicting nationwide air quality in china with transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 14329-14333 2023.
JI Lienhard and J Lienhard. Doe fundamentals handbook: thermodynamics, heat transfer, and fluid flow. Washington, DC: US Department of Energy, 2008
Chun-Ho Liu, Dennis YC Leung, and Mary C Barth. On the prediction of air and pollutant ex- change rates street canyons of different aspect ratios using large-eddy simulation. Atmospheric Environment, 1999)1556715 2005.
Zhipeng Luo, Jianqiang Huang, Ke Hu, Xue Li, and Peng Zhang. Accuair: Winning solution to air quality prediction for kdd cup 2018. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, pp. 1842-1850, 2019.
Ahmad Mohammadshirazi, Aida Nadafian, Amin Karimi Monsefi, Mohammad H Rafiei, and Rajiv Ramnath. Novel physics-based machine-learning models for indoor air quality approximations. arXiv preprint wxXiv:2388.01433 2023.
Aloke Paul, Tomi Laurila, Vesa Vuorinen, Sergiy V Divinski, Aloke Paul, Tomi Laurila, Vesa Vuori- nen, and Sergiy V Divinski. Fick's laws of diffusion. Thermodynanics, diffusion and the kirk- endall effect solids, pp. 115-139, 2014.
Ciyuan Peng, Feng Xia, Vidya Saikrishna, and Huan Liu. Physics-informed informed graph learning. In 2022 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 732-739. IEEE, 2022.
Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics informed deep leamn ing (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint 17111 10566, 2017.
Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud. Latent ordinary differential equations for irreullrly-samplled time series. Advances in neural information processing systems, 32, 2019.

---

Published as a conference paper at ICLR 2024
Chao Shang, Jie Chen, and Jinbo Bi. Discrete graph structure learning for forecasting multiple time series. arXiv preprint arXiv:2101.0666/ 2021.
Mengshuai Su, Hui Liu, Chengqing Yu, and Zhu Duan. A novel aqi forecasting method based on fusing temporal correlation forecasting with spatial correlation forecasting. Atmospheric Pollution Research, (4(4):10171) 2023.
Ana Suárez Sánchez, Paulino José García Nieto, Francisco Javier tonio Vilán Vilán. Nonlinear air quality modeling using support vector machines in gijón urban area (northern spain) at local scale. International Journal of Nonlinear Sciences and Numerical Simulation, 14(5):291- 1-305, 2013.
and José An-
Sotiris Vardoulakis, Bernard EA Fisher, Koulis Pericleous, and Norbert Gonzalez-Flesca. Modelling air quality in street canyons: a review. Atmospheric environment, 37(2):1555 182, 2003.
Jingyuan Wang, Jiahao Ji, Zhe Jiang, and Leilei Sun. Traffic flow prediction based on spatiotemporal potential energy fields. IEEE Transactions on Knowledge and Data Engineering. 35(9):9073- 9087, 2023.
Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Yu. Towards physics- informed deep learning for turbulent flow prediction. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 1457-1466, 2020a.
Shun Wang, Lin Qiao, Wei Fang, Guodong Jing, Victor S Sheng, and Yong Zhang. Air pollution pre- diction via graph attention network and gated recurrent unit. Computers, Materials & Continua, 73(1), 2022.
Shuo Wang, Yanran Li, Jiang Zhang, Qingye Meng, Lingwei Meng, and Fei Gao. Pm2. 5-gnn: A domain knowledge enhanced graph neural network for pm2. 5 forecasting. In Proceedings of the 28th international conference on advances in geographic information systems, pp. 163-166, 2020b.
Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint 170.00170 2017.
Chuampan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. Gman: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 1234-1241, 2020.
Yu Zheng, Xiuwen Yi, Ming Li, Ruiyuan Li, Zhangqing Shan, Eric Chang, and Tianrui Li. Fore- casting fine-grained air quality based big data. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 2267-2276, 2015.
A
APPENDIX
A.1
NOTATION
A.2
COMPUTATION OF LAPLACIAN MATRICES FOR DIFFUSION AND ADVECTION
On the sensor network g = (V. E, W) with N monitoring stations we define two weighted adja- cency matrices. The weighted adjacency matrix based distance (W,) determines the edge weights as follows:
W = 1/dkj
where dy the haversine distance between nodes and j.On the other hand, the weighted adjacency matrix based on flow field (W,) determines the edge weights as follows:
pd Flow:
D¹ = FlowNell(P)
Wis =p² pp
12

---

Published as a conference paper at ICLR 2024
Table 2: Summary of Notations used in the paper
Symbol
Description
Graph
N
Number of Nodes in the Sensor Network
V, N
Nodes of the Graph and i-th node
E
Edges the Graph
Wa.
w
Distance based weighted adjacency matrix and its entries
W
W
Flow-field based weighted adjacency matrix and its entries
X,
PM2.5 Concentration at timestep (X; IR NxI)
P.
Wind features timestep (Pr R Nx2)
F
Flux of Particles
K
Diffusion Coefficient
U
Vector Field
L
Scaled laplacian computed based Wa
M
Scaled laplacian computed based on W.
where P and pd represent the wind related attributes of source node (i) and destination node (j) and FlowNet is MLP used obtain the flow field representation from corresponding wind data.
Eq is formulated to define the physical process of diffusion within a sensor network. In other words, it describes the change in concentration due to the dispersion of air pollutants within the network. Thus the distance based scaled graph laplacian L is computed as follows:
L = 1- Wa D
L 2I Xmaz
where L is the laplacian matrix, Wa is the distance based weighted adjacency matrix, D is the diagonal degree matrix and Amaz is the largest eigen value of L.
Similarly, Eq is formulated to define the physical process of advection within sensor network which describes the change in concentration due to the movement of air pollutants within the net- work due the effect of flow field. Accordingly, the flow field based scaled graph laplacian M is computed as follows:
M = -- M
where M is the laplacian matrix, W, the distance based weighted adjacency matrix, D is the diagonal degree matrix and Amaz is the largest eigen value of M.
A.3 GCNS USED IN FORMULATION OF F&
As stated in Eq, Fg is neural network guided by the Diffusiion-Avecttoon DE. Fgg is formulated in terms of residual graph convolution network (GCN) layers as follows.
GCN for Diffusion Process
K-1
H diff
Σ k=
02
(1)
diff
Hasfy =
H
1=0
where H (1 is the updated state at layer I, a is the activation function, 0g are the Chebyshev coefficients, and & represents the kth power of the scaled laplacian calculated using distance based
13

---

Published as a conference paper at ICLR 2024
weighted adjacency matrix Wa Hasyy represents the final output of the GCN designed to capture the diffusion process.
GCN for Advection Process
K- M
1 adv
= a
Σ
Ok
H ((-1) adv
k=
Hadv Σ H
where H()) is the updated state at layer a is the activation function, 02 are the Chebyshev coef- adv ficients, and M represents the kth power of the scaled laplacian calculated using flow field based weighted adjacency matrix Wp. Hado represents the final output of the GCN designed to capture the advection process.
A.4 EVALUATION METRICS
Let (#i...... Fn) represents the ground truth, and a (#...... )n) represents the predicted PM2.5 concentration values. The evaluation metrics we used in this paper are defined as follows:
Mean Absolute Error (MAE)
MAB(a, .)
34
1=1
Root Mean Square Error (RMSE)
RMSE(2, 8)
(13)
71
A.5 DETAILED SETTINGS OF AIRPHYNET AND BASELINES
GRU as the RNN encoder to encode the sequence and obtain the initial state. We conduct grid search over [16, 32, 64, 128). and choose 64 as the number of hidden units of the GRU. A fully connected layer with 50 hidden units is then used transform the hidden states of GRU and obtain the mean and the standard deviation of the initial state. In the ODESolver, use dopri5 the numeral integration method with relative tolerance (rtol) and absolute tolerance (atol) set to le-5. Adam optimizer is used for training the model. We set the batch size to 32 and use an initial learning rate of 5e-4 which decays over specific steps with decay rate 0.1. We also employ an early stop strategy with patience of 20, allowing for a maximum of 100 epochs.
AirPhyNet: In our model
HA: Historical Average is tradtional statistical method which predicts the PM2.5 concentration by computing the average value of historical readings in corresponding time intervals. Here, we use day as the period and consider the average of the historical concentration of the same time interval from previous 4 days prediction.
VAR: VAR model is implemented using the statsmodels python packages and the number of lags
set to 3.
DCRNN Diffusion Convolution Recurrent Neural Network introduced in the domain of traffic fore- casting uses bidirectional random walks on the graph structure to capture the spatial dependencies while an encoder-decoder architecture used to capture the temporal dependencies. We use two recurrent layers each with 64 hidden units, in both encoder and decoder. Initial learning rate is set to 0.01 and it is reduced with a rate of decay rate of 0.1 starting at the 20th epoch. The maximum steps of random walks (K) is set to 3.
STGCN -patioltemporal Graph Convolution Network integrates graph convolutions with gated con- volutions to capture spatio-temporal relationships opposed employing regular convolutional
14

---

Published as a conference paper at ICLR 2024
and recurrent units. It consists of two ST-Conv blocks and the channels the three layers of each block are 64, 16, 64 respectively. The kernel size of graph convolution (K) and temporal convolu- tion (K₂) is 3. The initial learning rate is set at 1e-3 and it decays with rate of 0.7 after every 5 epochs.
GMAN Graph Multi-Attention network is introduced for the task of traffic prediction which con- sists of an encoder-decoder architecture with multiple spatio-temporal attention blocks. An adam optimizer with initial learning rate of le-3 is used to train the model. The number of STAttention blocks (L) is set to 3 and use 8 attention heads ((K)) each having a dimensionality of 8 ((d))
GTS Graph for time series is designed simultaneously learns graph structure among multiple time series to enhance time series forecasting. Adam is used as the optimizer and initial learning rate is set 1e-3. Dropout rate is considered 0.2, the value of in kNN is set to 30 and the weight regularization is 1.
PM2.5-GNN PM2.5 -N.N is one of the powerful domain knowledge enhanced networks introduced to predict PM2.5 concentration. Although the original model employs multiple features to integrate domain knowledge, we only consider wind speed and wind direction features in this study. RMSprrp is used as the optimizer with a learning rate of 5e-4.
transformer based air quality prediction model designed to forecast air quality across thousands of locations simultaneously. Initial learning rate is set to 5e-4 and it decays with a rate of 0.5 after every 3 epochs. Adam optimizer is used to train the model. For the dart- board spatial attention module, we partition the space by two circles with radius 50km and 200km respectively. We use 4 Airformer blocks in the model and set the hidden dimension to 32.
AirFormer Airformer
Latent-ODE
This model extends the capabilities of RNNs by introducing continuous- -itteouuss---t-eeeeeee hidden dynamics governed by ordinary differential equations (ODEs). It uses ODE-RNN as the encoder. The initial learning rate is set to 5e-4 and the latent dimension of the GRU is 100.dopri5 is used as the numeral integration method with relative tolerance (rtol) and absolute tolerance (atol) set to le-5.
ODE-LSTM ODE-LSTM is introduced to address the limitations of recurrent neural networks (RNNs) with continuous-time hidden states in modeling irregularly-sampled time series by intro- ducing a novel algorithm that separates memory from continuous-time dynamics. Here we set the latent dimension to 64, Explicit Euler is used as the ODE solver iswith relative tolerance (rtol) and absolute tolerance (atol) set to le-5. The learning rate 5e-4.
All the baseline models were trained both with and without wind-related attributes (i.e., wind speed and wind direction) as auxiliary variables. However, with the exception of PM2.5-GNN, the rest of the baseline models exhibited degraded performance when wind-related attributes were incorporated in the respective models. Therefore, the reported baseline performances do not include wind-related attributes, except for PM2.5-GNN.
A.6 ADDITIONAL EXPERIMENTAL RESULTS
Sudden Change Prediction: In alignment with the studies by Zheng al. (2015) and Liang et al. (203)). we also evaluated the performance of deep learning based methods in predicting sudden changes of air quality. Accordingly, the sudden changes are defined as instances where PM2.5 concentration exceeds 50 ug/m² and 20 ug/m² and exhibits change of more than +20 pg/m² in the following three hours for Beijing Data and Shenzen Data respectively.As illustrated in Table AirPhyNet shows the best performance in predicting sudden changes in both datasets. AirFormer (Liang et al., 2023) has also shown competing performance compared to the other baselines due to its' utilization of stochastic latent spaces. Compared to the second best method (i.c. AirFormer), AirPhyNet shows an average reduction of 6.2% and 4.6% in MAE and RMSE respectively.
In order to assess the generalizability of our approach, we evaluated the models using a smaller training dataset with valinet split as 3:1:6 and obtain the results shown in Table 4 Based on the results, AirPhyNet has outperformed all baselines across all time horizons by a significant margin. Specifically, the MAE and RMSE have been reduced by 8.3% and 8.1 % on average respectively compared to the second best method.
Sparse Data Prediction:
15

---

Published as a conference paper at ICLR 2024
Table Sudden Change prediction performance comparison on metrics MAE and RMSE. The bold and underlined font show the best and the second best result respectively.
Beijing Data
Shenzhen Data
Model
MAE
RMSE
MAE
RMSE
DCRNN
18.616
53.0629
3.3468
8.0377
STGON
21.821
56.8061
3,2087
8.1773
GMAN
48.2294
80.4201
4.6336
9.9652
GTS
22.5072
63.1401
3.7127
8.1299
PM25GNN
28.196
69.6186
4.9186
10.2459
AirFormer
14.1553
48.4329
3.1317
7.9183
LatentODE
43.5
76.8365
3.549
8.639
ODE-LSTM
38.248
76.2235
3.6311
8.6356
AirPhyNet
13.4987
47.7057
2.8904
7.3148
% Improvement
4.64
1.50
7.71
7.62
Table 4: Sparse Data prediction performance comparison on metries MAE and RMSE. The bold and underlined font show the best and the second best result respectively.
Beijing Data 48h
Shenzen Data
Model
24h
48h
72h
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
DCRNN
44.24
56.03
68.13
64.90
78.24
7.99
11.48
10.11
13.99
11.06
15.01
STGCN
33.73
44.90
39.93
50.51
46.62
57.53
7.25
10.50
8.96
12.13
9.75
12.92
GMAN
46.61
55 59
47.60
56.62
48.65
57.63
9.69
12.35
9.88
12.68
9.91
12.64
GTS
38.09
48.97
60.28
71.45
80.42
92.06
6.60
9.61
8.70
11.87
10.09
13.37
PM25GNN
47.34
61 55
46.33
60.81
47.64
61 61.65
10.78
13.18
10.84
13.22
10.90
13.23
AirForer
31.47
43.20
41.64
53.22
44.50
56.39
7.14
10.22
8.93
12.17
9.25
12.51
LatentODE
44.02
51.40
45.94
53.28
47.68
54,51
9.81
12.15
10.01
12.38
10.25
12.54
ODE-LSTM
49.95
61.16
51.93
63.19
53 44
19
10.36
13.26
11.01
13.97
11.49
14.39
AirPhyNet
27.99
38.48
35.78
45.82
40.45
50.37
6.47
9.50
7.75
10.76
8.63
11.58
Improvement
11.08
10.93
10.39
9.29
9.09
10.68
1.88
1.09
10.92
9.34
6.73
7.43
A.7 DISCUSSION
According to the results presented this study, AirPhyNet reveals consistent performance in air quality prediction over competing baselines across various metrics and time horizons. Notably, the model excels in capturing complex spatiotemporal relationships, showcasing significant reduction in MAE and RMSE compared to alternative methods. The incorporation of physics physics-based based knowl- edge into deep learning framework proves advantageous, particularly in predicting sudden changes in air quality. Additionally, the model exhibits robust generalization to unseen data, highlighting its potential for application in regions with limited monitoring station Furtherrtteeoorc. the compari- son with variants focusing solely on advection diffusion processes underscores the model's ability to comprehensively address the air pollutant transport dynamics while the case study further vali- dates this fact. All in all, the AirPhyNet model has shown significant capability in yielding precise air quality forecasts with a physical meaning.
These findings hold significant implications for air quality prediction. AirPhyNet's performance with lower prediction errors, suggests its potential real-world applicability in scenarios where precise forecasts are crucial. The ability generalize well to unseen data implies its' adaptability across diverse environments, making it useful tool for regions with limited monitoring infrastructure. Moreover, the integration of physics- knowledge not only enhances prediction accuracy but also provides insights which could be beneficial in decision making related to urban planning.
While the results are promising, it vital to acknowledge certain limitations in this study. Our focus solely PM2.5 concentration, while significant, may not account for the complete spectrum of pol- lutants that contribute to overall air pollution. There exist pollutants with diverse chemical properties and reactivity patterns which might exhibit behaviors that not captured by our model. Therefore,
16

---

Published as a conference paper at ICLR 2024
the applicability of our method highly chemical-reactinng pollutants remains uncertain. Addition- ally, our deep learning architecture emphasizes the incorporation physical processes, limiting its suitability for pollutants heavily influenced by chemical reactions. This could potentially hinder the model's 's accuracy in scenarios dominated by chemical reactions rather than physical transport.
To address these limitations, future studies could broaden the scope predict multiple air pollutants and determine the overall air quality. Integration chemical reaction information into the modeling framework, possibly by incorporating chemical kinetics with deep learning, could enhance the pre- dictive capabilities for pollutants governed by complex chemical processes. These models can be further enhanced to capture the interplay between different domains through insightful collaboration with experts in chemistry and environmental science. Another potential future direction is to lever- age this architecture for cross-city air quality prediction. This involves training our model on data from one city and adapting it to predict air quality in a different city through appropriate fine-tuning and transfer learning techniques.
17


Reviews:
[{"review_id": "0BILgPRjmj", "reviewer": "Reviewer_pK4K", "summary": "The study introduces AirPhyNet, a physics-guided neural network designed for enhanced air quality prediction. This method incorporates fundamental physics principles into the network architecture, improving predictive performance and interpretability. For this, it draws from existing literature in physics guided ML and neural ODEs. Tests on real-world data showcase its potential to improve over existing methods.", "questions": "- What is the reason for selecting so few cities?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- Putting together multiple complex concepts and methods is indeed a difficult task and requires a thorough understanding of physical dynamics and deep learning.\n\n- The paper addresses a significant and timely problem.\n\n- The narrative is clear and accessible.\n\n- The case study illustrates some of the physics that the model captures.", "weaknesses": "- My main concern with this work is the lack of contributions to hybrid AI or AI in general. Authors did not identify any technical gaps in our current hybrid AI methods. Instead, authors take what other researchers have developed for physics-guided ML in a variety of domains (e.g., physics) and use them for air quality prediction. Therefore, the method appears to be a combination of multiple well known methods with some developments in how to incorporate the specific physics priors for air quality priors. The air quality priors are just new equations and do not pose a significant technical challenge. Therefore, I do not think this is not a significant contribution for ICLR's research track. Perhaps the paper's contribution is better suited to a domain journal or the applied track of an AI conference.\n\n- Liang et al. 2023 (cited by authors in experimental setup) performed experiments in 342 cities in China and data appears publicly available. However, authors of this paper performed experiments only in 2 cities.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YxSyXTXOOq", "reviewer": "Reviewer_6p5Q", "summary": "In the presented study, the authors address the limitations of traditional data-driven models for air quality prediction, which often lack long-term accuracy and transparency due to their black-box deep learning nature. They introduce a novel approach called AirPhyNet, which integrates well-known physics principles of air particle movement (diffusion and advection) into a neural network using differential equation networks and a graph structure. This method not only enhances the model's interpretability by tying it to real-world physics but also shows superior performance on real-world datasets, outperforming state-of-the-art models in various test scenarios and reducing prediction errors by up to 10%. The model's ability to accurately capture the underlying physical processes of particle movement is further validated through a case study.", "questions": "1. Minor spacing issues in sentences. For instance, in abstract. \n2. I did not see the full form of DE Network. If you are using any abbreviation, please make sure to introduce the full form of it along with the short form. For instance, “Differential Equation (DE) is something. DE does that……”. \n3. I would suggest you shift the section of Related Work before Methodology, after introduction. \n4. Some of the discussions from the findings are mentioned in Section 3.4 (in the first paragraph of Page 8). However, I would suggest you add it in the discussion section (as mentioned in Weakness #4) along with the possible reasons behind the claims.\n5. Please add the limitation subsection.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. Good writeup style, in terms of grammar and readability. In Particular, the methodology has a comparatively good readability.", "weaknesses": "1. Research question is not specified. It is better to specify it. \n2. Before diving into the main work, the related work should be discussed. \n3. Section 4: Air Quality Prediction: Most of the papers from this subsection that have been cited, are outdated. I would request to add critical discussions of all such works and how the present approach overcomes them.\nFor instance: \na. https://ieeexplore.ieee.org/document/10152272\nb. https://arxiv.org/abs/2308.03200\nc. https://www.nature.com/articles/s41598-022-12355-6\nd. https://www.sciencedirect.com/science/article/pii/S1309104223000715?casa_token=1NXW1K1A37EAAAAA:NOxq1SvOhxDOOuqWmSssZAMZYUeApCukMcQGYNRWgAkeNKWBamlEBoWke0IfgmZNpPBtT3vElOc\ne. https://ieeexplore.ieee.org/abstract/document/9877800\n4. There is no discussion section. I would highly suggest adding it. \n5. There is no limitation mentioned in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4XqxtYlVpr", "reviewer": "Reviewer_8KYF", "summary": "This paper introduces AirPhyNet, a physics-informed graph neural network for air quality forecasting. Specifically, a diffusion-advection differential equation is first established to represent the physical process of air particle movement. Then, a physics-guided model is proposed to capture air pollution dynamics and generate physically consistent forecasting results by seamlessly integrating the predefined differential equation into a graph neural network. Experimental results on two real-world benchmark datasets demonstrate the superiority of AirPhyNet over several state-of-the-art baseline models in various forecasting scenarios. Moreover, a case study is also included to show the potential interpretability of the proposed model.", "questions": "See weakness", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. As far as I know, this is the first few attempts to combine physical modeling and deep learning for air quality forecasting.\n2. It is reasonable to take the advection and diffusion of air pollutants into consideration when building forecasting models.\n3. The paper is well-organized and easy to follow. There are also enough experiments to demonstrate the effectiveness and interpretability of the proposed method.", "weaknesses": "1. The difference between diffusion and advection is unclear. From my understanding, they all describe the transport of air pollutants over space and time. Although the authors briefly explained the difference in Section 2.2, the details are not clearly stated. More discussions would be better.\n2. In Equation 9, the authors claim they adopt a reparametrization trick to derive the hidden representation z_{t_0}. The rationale behind this choice is not fully explained. Why not directly use the final hidden state of GRU as z_{t_0}?\n3. In Section 2.4, the authors mentioned they leverage a decoder to generate the prediction results, but it lacks sufficient explanations. For example, the instantiation of the decoder should be illustrated in the paper. Moreover, the loss function should be formally defined.\n4. Some existing forecasting methods, such as DCRNN and AirFormer used in this paper, can also model the diffusion and advection process to some extent. So, what’s the major advantage of injecting physical principles into machine learning models? More discussions are appreciated.\n5. There are some typos. For example, in Section 3.1, “Shanghai” should be “Shenzhen”.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dHjLTwt2Ay", "reviewer": "Reviewer_tKfZ", "summary": "This paper considers the task of air quality prediction (PM2.5) simultaneously over a network of weather monitoring stations, using wind direction and speed as covariates. This is achieved with the help of an enoder-decoder architecture which can represent past values of the target variables and covariates, including the spatial correlations between the different stations. The encoded structure is used to solve ODEs based on advection and diffusion through a suitably designed Graph Neural Network, which can represent the spatial relations between the different stations. Instead of using computationally expensive Chemical Transport Models (CTM), this physics-based NN model can predict the state at the end of the prediction period (24/48/72 hours) and this is used to generate the final predictions of PM2.5 using decoder. It is shown that the proposed model can perform better than existing NN-based models and traditional statistical models for air quality prediction, including in situations where there is abrupt air quality changes. Some intuition is also provided through a case study, where the pollution hot-spots are found to shift in the direction of winds.", "questions": "1) What was the architecture of the \"decoder\"? I did not find this anywhere.\n2) What is the significance of the encoded state \"z\" ? Is it possible to visualize it? Is it possible to run the Neural ODE on the original data itself, without the encoding?\n3) What is the impact of the lookback window size T? \n4) The model seems to have been trained separately for the two cities in question. Is it possible to train it on one city and use it on the other? Which parameters will need fine-tuning in that case?\n5) Is the data periodic in nature? How strong is the spatial correlation between the different stations?\n6) Two covariates are considered- wind direction and wind speed. Do the competitor methods also use these? How will the proposed model perform if these covariates are not considered? Or if more covariates are considered?\n7) How are the two \"traditional\" methods - HA and VAR implemented? Specifically, are they done specific to each station separately? If so, it may not be a fair comparison.\n8) How does the proposed approach compare with CTM?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper proposes an architecture that achieves two things: i) solves differential equations for advection and diffusion over a network using a GNN with Neural ODE solver, and ii) uses this to make spatio-temporal predictions of air quality using wind direction+speed as covariates. \n\nThe strengths of the paper lies in the facts that:\ni) the proposed framework is quite novel \nii) the reported results are very strong", "weaknesses": "The weak points of the work are mostly in the experimental part:\n\ni) The station sizes are relatively small (35 and 11). It is not clear how the method will scale to bigger networks\nii) The comparisons provided are mostly against other neural network based methods. The work is motivated by stating that CTMs are very expensive. Yet, no comparison with CTM in term of either computational cost or accuracy is provided. Two \"traditional methods\" are mentioned (HA and VAR), but no details of those are provided. \niii) There is no analysis of the nature of the data that is being dealt with.\n\nApart from these, the analysis leaves a few loose ends, mentioned in the next section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)