SPIGAN: A G ENERATIVE A DVERSARIAL N ETWORKS UPERVISED BY S PARSE I DENTIFICATION TO L EARNG OVERNING E QUATIONS FROM S CARCE DATA

1

Shen Yue* 1 and Chen Yu†1
Department of Human and Engineered Environmental Studies
Graduate School of Frontier Sciences
The University of Tokyo
5-1-5 Kashiwanoha, Kashiwa-shi, Chiba, 277-8561, JapanA BSTRACTThis work introduces SPIGAN to discover governing partial differential equations from scarce data
for nonlinear spatiotemporal systems, which leverages the strengths of GAN-based neural networks
for physics embedding, automatic differentiation, and data generation, along with the advantages ofsparse identification in identifying key derivative terms in an end-to end manner. The efficacy androbustness of SPIGAN are demonstrated with Burgers’ equation under different parameter settings.
The ablation tests further highlight the advantages of SPIGAN over original sparse identification or
GAN approaches.
Keywords Physics-informed machine learning · Generative adversarial networks · Equation learning

1

Introduction

Constructing a coherent description of natural laws using differential equations poses a significant challenge acrossvarious scientific fields. The prevailing approach to modeling complex dynamical systems relies heavily on ordinary
and/or partial differential equations (ODEs, PDEs) that govern the system’s behaviors. Traditionally, in order to analyze
the underlying dynamics of the concerned phenomenon, scientists would derive these equations from the first principle
or just intuitively write down mathematical equations despite that a reasonable amount of data has already been obtained.
However, these classical approaches have shown limited efficacy and slow progress. Fortunately, the availability of richobservational datasets offers an alternative avenue for distilling underlying equations from data. Recent advancements inmachine learning (ML) have significantly accelerated the field of equation learning (EQL), which aims to automaticallyand directly extract interpretable analytic equations from big data.
There has been vast research on data-driven discovery of dynamical systems which involves a variety of approaches.
Recently, an impressive breakthrough made by Brunton et al.[1, 2] called sparse identification of nonlinear dynamics(SINDy) has been successfully applied to uncovering the dynamic dependencies in spatio-temporal data generated
by numerical simulations of dynamic models[3, 4, 5, 6]. However, when it comes to real-case data-driven discovery
of ODEs/PDEs, the SINDy framework faces a critical bottleneck which arises from its strong dependence on thequality and quantity of measurement data. Due to the high costs associated with data collection, generation, andlabeling, as well as data quality issues such as missing data, it is often the case that the collected data fail to meetthese requirements. In such instances, we encounter data with sparsely and irregularly evaluated temporal or spatial
intervals. The direct impact of scarce data on SINDy is most noticeable when numerical differentiation is estimatedusing traditional algorithms like finite differences. Since the estimated differentiation is subsequently used as inputs for∗†shenyue@s.h.k.u-tokyo.ac.jp
chen@edu.k.u-tokyo.ac.jp

International Conference on Scientific Computing and Machine Learning 2024 (SCML2024)

SPIGANsparse regression, the inaccuracy of this stage significantly influences the overall performance of SINDy. Recently,
some scholars [7] have tried to resort to physics-informed neural networks to reduce noise and obtain robust derivatives,
which calls for further exploration with other network structures.
To solve this problem, we introduce a novel approach called SPIGAN (SINDY with Physics-informed GAN) to uncover
the governing PDEs of nonlinear spatiotemporal systems using scarce data. Our approach harnesses the combined
strengths of GANs for robust representation learning, automatic differentiation for precise derivative calculation, anddata generation to address the inherent limitations of existing methods that struggle with data scarcity, while maintaining
the ability of inferring ODEs/PDEs in a explicit form from SINDy. SPIGAN allows us to overcome the fundamental
challenges associated with scarce data and offers a more effective solution for discovering the underlying dynamics ofcomplex systems.

2

Methods

2.1

Sparse Identification Pipeline

Consider a modeled system of the ODE formdX(t) = F (X(t))(1)
dtwhere the vector X(t) = [x1 (t), x2 (t), · · · , xn (t)] denotes the states of system at time t, and the function F (X(t))
describes the dynamics of the system. A key assumption here is that the function F = F (X, t) consists of only a few
terms, making the sparse regression methodologies applicable in consideration of both efficiency and accuracy.
The SINDy pipeline begins by collecting all the time series data and building them into a matrix form X ∈ Rm×n ,the superscript of which represents m time points and n dimensions. Numerical methods (e.g., the finite differencedX(t). The third step
method) are then applied to calculate the discrete derivatives X t ∈ Rm×n as an estimation of dt
m×(N +Nq )of the EQL pipeline requires the construction of a nonlinear library Θ(F , Q) ∈ R
as the candidates of
explanatory variables for the original data and their time derivatives. Each column of the library matrix corresponds toa specific candidate term in the right hand side of the governing equation. The column vector Q ∈ Rm×Nq contains
additional terms such as the known constants or other external interacting variables. If we assume that Θ(F , Q)
is an over-complete library, for example, the ODE evolution can be expressed with a sparse vector of coefficientsξ ∈ R(N +Nq )×n as follows
X t = Θ(F , Q)ξ
(2)
Each nonzero entry of the sparse vector ξij corresponds to the existence of term represented by the ith column of Θ indthe right hand side of the differential equation dt
xj (t). The least absolute shrinkage and selection operator (LASSO)
regression or other sparse regression methods are commonly applied to Eq. (2) for determining the active nonlinearities.
2.2

SINDy with Physics-informed GAN

Physics supervision is integrated into the adversarial learning framework. Specifically, the physics residuals are used tocompute a physics consistency score η for each prediction, indicating the likelihood of the prediction being physicallyconsistent. The physics consistency scores are incorporated into the discriminator as additional inputs, such that thediscriminator not only distinguishes between real and fake samples like the original GAN architecture but also using
the physics supervision provided by the consistency scores. The physics consistency scores of a prediction ŷ regarding
to the k−th physical constraint is defined using the following equation:
ηk = exp (−λ||R(k) (x, ŷ)||2 )(3)

where R(k) (x, ŷ) is the physics residual calculated by the k−th ODE/PDE given x and the predicted value ŷ. λ is a
hyperparameter that controls the weight of the physics consistency term.
To increase the training volume and balance the data distribution, the physics-informed GAN allows the use of unlabeleddata instances for training both the generator and discriminator models. In this context, labeled data consists of pairs(xui , yui ) for i = 1, 2, · · · , Nu , where the input-output values are known in advance. Unlabeled data, on the other
hand, consists of input points xfj sampled from the domain of the dynamic system being investigated, with unknown
corresponding output values. The training objective of physics-informed GANs is modified accordingly as follows:
LossG =

Nf
Nu
1 X
1 X
D(xui , ŷui , ηui ) +D(xfj , ŷfj , ηfj )Nu i=1
Nf j=1

2(4)

SPIGAN

LossD = −

Nu
Nu
1 X
1 X
log(D(xui , ŷui , ηui )) −log(1 − D(xui , yui , 1))
Nu i=1
Nu i=1

Nf
Nf
1 X
1 X
−
log(D(xfj , ŷfj , ηfj )) −log(1 − D(xfj , ŷfj , 1))
Nf j=1
Nf j=1(5)where ŷui = G (xui , zui ), ŷfj = G (xfj , zfj ), zui and zfj are the sampled noise.
Our proposed framework SPIGAN (SINDY with physics-informed GAN) is based on physics-informed GAN but
without the reliance on prior knowledge. In the SPIGAN architecture, the physics residuals are not calculated based on
the pre-defined but instead obtained through the SINDy algorithm, resulting the discriminator in SPIGAN to learn thefollowing mapping.
D : D(x, y, ηSINDy ) → Ω ∈ [0, 1](6)

3

Results

In our experiments, we deliberately create scarce data by sampling the velocity variable sparsely in the spatial andtemporal domains for the Burgers’ equation to highlight the challenges faced by the vanilla SINDy approach. Burgers’
equation is a fundamental partial differential equation and convection–diffusion equation. In its well-known form, we
can write it as follows:
∂u
∂2u
∂u
= −u
+ ν 2 , 0 < x < L, 0 < t < T
∂t
∂x
∂x
(7)
u(x, 0) = ϕ(x), 0 < x < L,u(0, t) = ζ1 (t), u(L, t) = ζ2 (t), 0 < t < T
where u, x, t and ν are the velocity, spatial coordinate, time and kinematic viscosity, respectively.
ζ1 (t), ζ2 (t) and ϕ(x) are prescribed boundary conditions depending upon the specific conditions for the problem to be solved.
We consider the Burgers’ equation under two conditions.
In Case 1, we set the Burgers’ equation with a small viscosity parameter. The initial condition is set as a sine
wave, which leads to the formation of two transmitted
waves that meet in the middle of the region (x = 0.5L).
In Case 2, the initial condition is an activation function.
This case demonstrates the behavior of the Burgers’ equation in the presence of diffusion, where the velocity field
diffuses and smoothens over time. In both of the twocases the boundary condition is set to 0. The dynamics
of the two cases are shown in Figure 1, where the color
plot represents the velocity u(x, t) at different spatial positions x and time steps t. To simulate the problem ofscarce data, we randomly place several sensors at typicaltime intervals and select p = 10% of data points from thefull data volume, resulting in limited observations of the Figure 1: Visualization of ground truth dynamics of Burgers’
system’s behavior.
equation.
3.1

Parameter Settings

The network G : G (x, t, z) → u is a network with 4
hidden layers with 50 neurons each. The network D : D(x, t, u, η) → Ω ∈ [0, 1] is a network with 2 hidden layerswith 50 neurons each. To infer the PDE solutions, for every discriminator update, the generator is updated 5 times forall of our SPIGAN. The batch size is set to 256. λ used in Eq. 3 is set to 0.05. We train the baseline models for 20000
epochs, which is common practice in the existing literature. For each training, we sample Nf = 10000 unlabeled points
uniformly across the input space using latin hypercube sampling (LHS)[8]. This provides a diverse set of unlabeledpoints to help stabilize the training process. We use the Adam optimizer with a learning rate of 0.0001. The library used
∂2u
∂u
∂2u
in SINDy process is also built based on second order derivatives {u, ∂u∂x , ∂x2 , u ∂x , u ∂x2 }. Additionally, before starting
the training process, we standardize the spatio-temporal coordinates of the input data based on the sampled unlabeledpoints. This standardization helps to balance the range of input data and improve the training performance.
3

SPIGAN

3.2

Discover Burger’s Equation under Default Parameter Settings

In the test phase of the SPIGAN framework, we evaluate
the performance of the generator by comparing the generated output values with the ground truth values at thecoordinates of unlabeled points. In Figure 2(a), we provide visualizations of the generator’s output after training
the SPIGAN model on the Burgers’ equation data. The
output is averaged over 50 runs to reduce the influence ofnoise sampling, and the variance is also shown in Figure2(b) to indicate the uncertainty of the predictions. We
can see that the generator captures the system dynamics
of both Case 1 and Case 2 to a large extent. Figure 2(c)illustrates the absolute error within the domain. The final relative L2 -error of the predictions achieved by the
SPIGAN model is 0.212 for Case 1 and 0.096 for Case
2. These values are close to those reported in previous
GAN-based neural network studies (0.215 for Case 1, noresult for Case 2). Though in our setting the volume oflabeled data used is much larger than the previous study,
Figure 2: Visualization of (a) generator output, (b) variance our approach requires less reliance on prior knowledgeand (c) absolute error of SPIGAN for Burgers’ Equation or predefined physics constraints. Similarly, SPIGAN
can be used in cases in cases where sensors are placed
under default parameter setting in both two cases.
randomly at several locations. This scenario mimics situations where data scarcity occurs in the spatial domain.
The SPIGAN framework preserves the ability to fit and predict within the domain, demonstrating its effectiveness inhandling scarce data scenarios.
It should be noticed that, however, the selection of labeleddata will affect the performance to some extent. As for the
finally inferred equation, we analyze the coefficients of
the inferred equation of SPIGAN and compare them withthe ground truth equation and the SINDy results in both
Case 1 and Case 2. To demonstrate the effect of random
sampling of labeled data, the experiment is repeated threetimes with different selections of labeled data. It can beobserved that when the SINDy method is applied to thescarce data, the inferred equations deviate greatly fromthe ground truth. As the data becomes extremely limited,the coefficients of the inferred equations fluctuate greatly
depending on the sampled data, indicating the instabilityof finite differentiation when facing unequal intervals. Figure 3: Coefficients of inferred equation of SPIGAN for
Typically, as Burgers’ equation contains a second-order Burgers’ equation.
derivative in the spatial domain, when data is collected
at random locations, the results will be even worse. As
Figure 3 shows, at the same time, although there may be some deviation in the coefficients for certain terms, SPIGAN
demonstrates better stability in its results. The coefficients of the inferred equation from the three trials generally alignwith each other, which is a huge advantage over the vanilla SINDy.

4

Conclusion

Due to the high costs associated with data collection and limitation of facilities, it is often the case that the collected
data will be extremely scarce. Vanilla SINDy will not perform well in these situations. Therefore, we have introduced
SPIGAN, a novel GAN-based learning method for discovering physical laws from scarce data. SPIGAN provides an
end-to-end solution for Equation Learning (EQL) that integrates data-driven and physics-informed approaches. The
experiments on Burger’s equation with different boundary conditions have demonstrated the effectiveness of SPIGAN.
The generator in SPIGAN is able to recover the system dynamics using only 10% of the data required by vanilla SINDy.
The inferred form of the governing equation(s) shows good consistency, albeit with some deviations from the groundtruth.
4

SPIGANAcknowledgments
This study was supported by the World-leading Innovative Graduate Study Program in Proactive Environmental Studies(WINGS-PES), The University of Tokyo.

References[1] Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Discovering governing equations from data by sparse
identification of nonlinear dynamical systems. Proceedings of the national academy of sciences, 113(15):3932–3937,
2016.
[2] Markus Quade, Markus Abel, J Nathan Kutz, and Steven L Brunton. Sparse identification of nonlinear dynamics
for rapid model recovery. Chaos: An Interdisciplinary Journal of Nonlinear Science, 28(6):063116, 2018.
[3] Jean-Christophe Loiseau and Steven L Brunton. Constrained sparse galerkin regression. Journal of Fluid Mechanics,
838:42–67, 2018.
[4] Jean-Christophe Loiseau, Bernd R Noack, and Steven L Brunton. Sparse reduced-order modelling: sensor-based
dynamics to full-state estimation. Journal of Fluid Mechanics, 844:459–490, 2018.
[5] Moritz Hoffmann, Christoph Fröhner, and Frank Noé. Reactive sindy: Discovering governing reactions fromconcentration data. The Journal of chemical physics, 150(2):025101, 2019.
[6] Bhavana Bhadriraju, Abhinav Narasingam, and Joseph Sang-Il Kwon. Machine learning-based adaptive model
identification of systems: Application to a chemical process. Chemical Engineering Research and Design,
152:372–383, 2019.
[7] Zhao Chen, Yang Liu, and Hao Sun. Physics-informed learning of governing equations from scarce data. Naturecommunications, 12(1):6136, 2021.
[8] Michael D McKay, Richard J Beckman, and William J Conover. A comparison of three methods for selecting
values of input variables in the analysis of output from a computer code. Technometrics, 42(1):55–61, 2000.

5