Enforcing physics-based algebraic constraints for inference of PDE models on unstructured grids

1 INTRODUCTION . Multiple works have shown the capability of neural networks to solve complex physical problems and learn the behavior of physical systems from data . Examples include learning and solving ordinary differential equations ( ODEs ) [ 6 ] , partial differential equations ( PDEs ) [ 28 ; 20 ] and rigid body dynamics [ 31 ; 5 ] . Purely data-driven models are typically not forced to satisfy physical constraints of the system that generated the data . This might lead to unrealistic predictions that violate some known properties of the underlying physical system . Incorporation of relevant constraints allows to make a better use of the available data and makes predictions more physically plausible . The field dealing with physics-constrained learning is diverse and offers many approaches to adding constraints to models . We refer the reader to many reviews for details [ 30 ; 3 ; 36 ; 19 ] . The approach we consider in this work is based on forcing a model to satisfy algebraic constraints represented by a set of equalities and inequalities . This is the most commonly used approach which allows to represent a wide range of constraints and has been shown to work well in many cases [ 18 ; 17 ; 25 ] . However , while many constraints can be represented algebraically , it is not always clear how to evaluate and enforce them . Currently available approaches to enforcing algebraic constraints are limited to uniform grids and have a very narrow range of constraints they can enforce ( e.g . only pointwise , or specific differential constraints ) , see Section 5 for details of related work . Such approaches can be readily applied to models based on convolutional neural networks ( CNNs ) but can not be extended to recently developed models based on graph neural networks ( GNNs ) [ 33 ; 27 ; 15 ] and other models working on unstructured grids . We propose a much more general method which allows to enforce pointwise , differential and integral constraints on unstructured spatial grids and demonstrate its applicability in learning of PDE-driven dynamical systems and distributions of physical fields . The method is based on using a models ’ s output at the nodes of a grid to construct an interpolant and applying constraints directly to that interpolant ( Section 3 ) . Code and data will be made publicly available . 2 BACKGROUND . PDE-driven dynamical systems . Many physical systems can be described in terms of PDEs . Such systems are defined on a bounded domain on which they evolve over time . We consider continuous dynamical systems with state u ( t , x ) ∈ Rp that evolves over time t ∈ R≥0 and spatial locations x ∈ Ω ⊂ RD . For physical systems , D is typically limited to { 1 , 2 , 3 } although our method will work with any value of D. We assume the system is governed by an unknown PDE ∂u ( t , x ) ∂t = F ( x , u ( t , x ) , ∇xu ( t , x ) , ∇2xu ( t , x ) , ... ) ( 1 ) which describes the temporal evolution of the system in terms of the locations x , state u and its first and higher-order partial derivatives w.r.t . x . The goal of a data-driven PDE model is to learn the dynamics F from data . Data for learning F is collected by measuring the state of the system at observation locations ( x1 , . . . , xN ) over increasing time points ( t0 , . . . , tM ) . This results in a dataset ( y ( t0 ) , . . . , y ( tM ) ) , where y ( ti ) = ( u ( ti , x1 ) , . . . , u ( ti , xN ) ) is a collection of observations . The dataset is used to train the model to predict ( y ( t1 ) , . . . , y ( tM ) ) starting from the initial state y ( t0 ) . Training is typically done by minimizing an average loss between the model ’ s predictions u ( t ) and the data y ( t ) . PDE models differ in restrictions they impose on time points ( temporal grid ) and observation locations ( spatial grid ) . Some models require both grids to be uniform [ 23 ] , other models relax these requirements and allow arbitrary spatial [ 27 ] and spatio-temporal grids [ 15 ] . We build our algebraic constraints method using the model from [ 15 ] as the most general one . The model is based on application of the method of lines [ 32 ] to Equation 1 which results into a system of ODEs u̇ ( t ) : = du ( t , x1 ) dt ... du ( t , xN ) dt ≈ Fθ ( x1 , xN ( 1 ) , u1 , uN ( 1 ) ) ... Fθ ( xN , xN ( N ) , uN , uN ( N ) ) ( 2 ) which approximates the solution of Equation 1 at the observation locations xi using their neighboring points N ( i ) , where xN ( i ) and uN ( i ) are the neighbors ’ positions and states respectively , and ui is u ( t , xi ) . The approximate solution converges to the true solution as N increases . The true dynamics F is approximated by a parametric model Fθ whose parameters θ are learned by minimizing the difference between the model ’ s predictions u ( t ) = u ( 0 ) + ∫ t 0 u̇ ( τ ) dτ ( 3 ) and the data y ( t ) . The integral in Equation 3 is solved using a numerical ODE solver . In [ 15 ] , the function Fθ was represented by a graph neural network ( GNN ) which takes states and locations at an observation point i and its neighboring points N ( i ) . The observation points are connected into a grid using Delaunay triangulation which allows to naturally defineN ( i ) as a set of points connected to the point i . However , Fθ can be represented by other models and a different neighbor selection criterion can be used . The model parameters θ are learned by minimizing the MSE between y ( t ) and u ( t ) Ldata = 1 M M∑ i=1 ‖u ( ti ) − y ( ti ) ‖22 . ( 4 ) The gradient of Ldata w.r.t . θ is evaluated using the adjoint method as shown in [ 7 ] . Generative Adversarial Networks One of the tasks that we consider is learning distributions of physical fields . For that purpose we utilize generative adversarial networks ( GANs ) . A GAN is a generative model consisting of a generator and a discriminator [ 12 ] . The generator , G , learns to transform a random variable Z ∼ pZ over a latent space Z to the data space Y in such a way that the discriminator , D , can not tell the difference between samples generated by G and samples from the data distribution pdata . Both , G and D are learned by solving the following minimax problem min G max D V ( G , D ) = EY∼pdata [ logD ( Y ) ] + EZ∼pZ [ log ( 1−D ( G ( Z ) ) ) ] . ( 5 ) Solution of this problem exists and is unique with the optimal generator perfectly mimicking the data distribution [ 12 ] . 3 METHODS . In this section we presents an approach to evaluating pointwise , differential and integral constraints on unstructured grids . Then , we demonstrate how this approach can be used to enforce arbitrary soft and linear hard constraints . 3.1 EVALUATING CONSTRAINTS ON UNSTRUCTURED GRIDS . We assume the data y ( t ) is available at observation points ( x1 , . . . , xN ) and time points ( t1 , . . . , tM ) and that a model makes predictions u ( t ) at these points . We assume the predictions to be evaluations of an unknown underlying function . Since the underlying function is unknown , we can not impose constraints on it directly . Instead , we approximate it by an interpolant uf ( t , x ) and impose constraints on uf ( t , x ) ( Figure 1 ) . The approximation is constructed from u ( t ) by placing a basis function at each xi and representing uf ( t , x ) as uf ( t , x ) = N∑ j=1 αj ( t ) φj ( x ) , ( 6 ) where φj is a scalar basis function at xj and αj ∈ Rp . The coefficients αj ( t ) are obtained from u ( t ) ( see Section 3.4 ) . Next , we show how to evaluate constraints on uf ( t , x ) using basic building blocks . To avoid cluttered notation , we consider equality constraints and assume u ( t , x ) , x ∈ R. Generalization to inequality constraints , vector fields and higher spatial dimensions is straightforward . Pointwise constraints . Consider points z = ( z1 , . . . , zK ) in Ω on which a pointwise constraint h ( uf ( t , zi ) ) = 0 should be evaluated . Assume the function h : R→ R is representable in terms of a finite number of functions γm ( uf ( t , zi ) ) : R → R indexed by m. For example , should the constraint be h ( uf ) = 3uf + u2f = 0 , then we would define γ1 ( uf ) = uf , γ2 ( uf ) = u2f and h ( uf ) = 3 · γ1 ( uf ) + γ2 ( uf ) = 0 . Then , h can be evaluated by evaluating each γm as γm ( uf ( t , zi ) ) = γm N∑ j=1 αj ( t ) φj ( zi ) = γm ( Φi , ·α ( t ) ) , ( 7 ) where α ( t ) = ( α1 ( t ) , . . . , αN ( t ) ) T , Φ is K-by-N matrix with elements Φi , j = φj ( zi ) , and Φi , · is the i ’ th row of Φ . Differential constraints . Consider the same setup as before but now h ( uf ( t , zi ) ) = 0 consists of differential operators and is representable in terms of a finite number of functions ∂ qγm ( uf ( t , zi ) ) ∂zqi : R→ R indexed bym , where the derivative order q could be different for eachm . For example , should the constraint be h ( uf ) = 3uf + uf · ∂u2f ∂x = 0 , then we would define γ1 ( uf ) = uf , γ2 ( uf ) = u 2 f and h ( uf ) = 3 · γ1 ( uf ) + γ1 ( uf ) · ∂γ2 ( uf ) ∂z = 0 . Then , h can be evaluated by evaluating each ∂qγm ( uf ( t , zi ) ) ∂zqi using the generalization of the chain rule ( Appendix A ) which contains only two types of terms . The first type of terms dγmduf , . . . , dqγm duqf can be evaluated using automatic differentiation while the second type of terms ∂uf∂zi , . . . , ∂quf ∂zqi can be evaluated as ∂quf ∂zqi = N∑ j=1 αj ( t ) ∂qφj ( zi ) ∂zqi = Φ ( q ) i , · α ( t ) , ( 8 ) where Φ ( q ) i , j = ∂qφj ( zi ) ∂zqi . Mixed partial derivatives can be handled in a similar way ( Appendix A ) . Integral constraints . Consider the same setup as before but with h ( uf ( t , x ) ) =∫ Ω τ ( uf ( t , x ) ) dx = 0 , where the function τ : R → R is representable in terms of functions γm ( uf ( t , zi ) ) : R→ R similarly to the pointwise constraints . Then , ∫ Ω τ ( uf ( t , x ) ) dx can be evalu- ated using a numerical integration technique , e.g . midpoint rule , Gaussian quadrature or Monte-Carlo integration , as ∫ Ω τ ( uf ( t , x ) ) dx ≈ K∑ i=1 τ ( uf ( t , zi ) ) µi , ( 9 ) where K is the number of integration points , µi are integration coefficients which depend on the grid and integration method , and τ ( uf ( t , zi ) ) is evaluated as in Equation 7 .

Summary: The paper proposes a two-folded method to enforce constraints of different natures (differential, integrals….) on a statistical model learned from physical data. The constraints are not enforced directly on the model but rather on “interpolant” functions that aims at completing the original model in between the observed grid points.