Combining Physics and Machine Learning for Network Flow Estimation

1 INTRODUCTION . In many applications , ranging from road traffic to supply chains to power networks , the dynamics of flows on edges of a graph is governed by physical laws/models ( Bressan et al. , 2014 ; Garavello & Piccoli , 2006 ) . For instance , the LWR model describes equilibrium equations for road traffic Lighthill & Whitham ( 1955 ) ; Richards ( 1956 ) . However , it is often difficult to fully observe flows in these applications and , as a result , they rely on off-the-shelf machine learning models to make predictions about missing flows ( Li et al. , 2017 ; Yu et al. , 2018 ) . A key limitation of these machine learning models is that they disregard the physics governing the flows . So , the question arises : can we combine physics and machine learning to make better flow predictions ? This paper investigates the problem of predicting missing edge flows based on partial observations and the underlying domain-specific physics defined by flow conservation and edge features ( Jia et al. , 2019 ) . Edge flows depend on the graph topology due to a flow conservation law—i.e . the total inflow at every vertex is approximately its total out-flow . Moreover , the flow at an edge also depends on its features , which might regularize the space of possible flow distributions in the graph . Here , we propose a model that learns how to predict missing flows from data using bilevel optimization ( Franceschi et al. , 2017 ) and neural networks . More specifically , features are given as inputs to a neural network that produces edge flow regularizers . Weights of the network are then optimized via reverse-mode differentiation based on a flow estimation loss from multiple train-validation pairs . Our work falls under a broader effort towards incorporating physics knowledge to machine learning , which is relevant for natural sciences and engineering applications where data availability is limited ( Rackauckas et al. , 2020 ) . Conservation laws ( of energy , mass , momentum , charge , etc . ) are essential to our understanding of the physical world . The classical Noether ’ s theorem shows that such laws arise from symmetries in nature ( Hanc et al. , 2004 ) . However , flow estimation , which is an inverse problem ( Tarantola , 2005 ; Arridge et al. , 2019 ) , is ill-posed under conservation alone . Regularization enables us to apply domain-knowledge in the solution of inverse problems . We motivate our problem and evaluate its solutions using two application scenarios . The first is road traffic networks ( Coclite et al. , 2005 ) , where vertices represent locations , edges are road segments , flows are counts of vehicles that traverse a segment and features include numbers of lanes and speed limits . The second scenario is electric power networks ( Dörfler et al. , 2018 ) , where vertices represent power buses , edges are power lines , flows are amounts of power transmitted and edge features include resistances and lengths of lines . Irrigation channels , gas pipelines , blood circulation , supply chains , air traffic , and telecommunication networks are other examples of flow graphs . Our contributions can be summarized as follows : ( 1 ) We introduce a missing flow estimation problem with applications in a broad class of flow graphs ; ( 2 ) we propose a model for flow estimation that is able to learn the physics of flows by combining reverse-mode differentiation and neural networks ; ( 3 ) we show that our model outperforms the best baseline by up to 18 % ; and ( 4 ) we provide evidence that our model learns interpretable physical properties , such as the role played by resistance in a power transmission network and by the number of lanes in a road traffic network . 2 FLOW ESTIMATION PROBLEM . We introduce the flow estimation problem , which consists of inferring missing flows in a network based on a flow conservation law and edge features . We provide a list of symbols in the Appendix . Flow Graph . Let G ( V , E , X ) be a flow graph with vertices V ( n= |V| ) , edges E ( m= |E| ) , and edge feature matrix X ∈ Rm×d , where X [ e ] are the features of edge e. A flow vector f ∈ Rm contains the ( possibly noisy ) flow fe for each edge e ∈ E . In case G is directed , f ∈ Rm+ , otherwise , a flow is negative if it goes against the arbitrary orientation of its edge . We assume that flows are induced by the graph , and thus , the total flow—in plus out—at each vertex is approximately conserved : ∑ ( vi , u ) ∈E f ( vi , u ) ≈ ∑ ( u , vo ) ∈E f ( u , vo ) , ∀u ∈ V In the case of a road network , flow conservation implies that vehicles mostly remain on the road . Flow Estimation Problem . Given a graph G ( V , E , X ) with partial flow observations f̂ ∈ Rm′ for a subset E ′ ⊆ E of edges ( f̂e is the flow for e ∈ E ′ , m′= |E ′| < m ) , predict flows for edges in E \ E ′ . In our road network example , partial vehicle counts f̂ might be measured by sensors placed at a few segments , and the goal is to estimate counts at the remaining segments . One would expect flows not to be fully conserved in most applications due to the existence of inputs and outputs , such as parking lots and a power generators/consumers . In case these input and output values are known exactly , they can be easily incorporated to our problem as flow observations . Moreover , if they are known approximately , we can apply them as priors ( as will be detailed in the next section ) . For the remaining of this paper , we assume that inputs and outputs are unknown and employ flow conservation as an approximation of the system . Thus , different from classical flow optimization problems , such as min-cost flow ( Ahuja et al. , 1988 ) , we assume that flows are conserved approximately . Notice that our problem is similar to the one studied in Jia et al . ( 2019 ) . However , while their definition also assumes flow conservation , it does not take into account edge features . We claim that these features play important role in capturing the physics of flows . Our main contribution is a new model that is able to learn how to regularize flows based on edge features using neural networks . 3 OUR APPROACH : PHYSICS+LEARNING . In this section , we introduce our approach for the flow estimation problem , which is summarized in Figure 1 . We formulate flow estimation as an optimization problem ( Section 3.1 ) , where the interplay between the flow network topology and edge features is defined by the physics of flow graphs . Flow estimation is shown to be equivalent to a regularized least-squares problem ( Section 3.2 ) . Moreover , we describe how the effect of edge features and the graph topology can be learned from data using bilevel optimization and neural networks in Section 3.3 . Finally , we propose a reverse-mode differentiation algorithm for flow estimation in Section 3.4 . 3.1 FLOW ESTIMATION VIA OPTIMIZATION . The extent to which flow conservation holds for flows in a graph is known as divergence and can be measured using the oriented incidence matrix B ∈ Rn×m of G. The matrix is defined as follows , Bij = 1 if ∃u such that ej = ( vi , u ) ∈ E , Bij = −1 if ∃u such that ej = ( u , vi ) ∈ E , and Bij = 0 , otherwise . Given B and f , the divergence at a vertex u can be computed as : ( Bf ) u = ∑ ( vi , u ) ∈E f ( vi , u ) − ∑ ( u , vo ) ∈E f ( u , vo ) ( 1 ) And thus , we can compute the total ( squared ) divergence in the graph as ||Bf ||22 = fᵀBᵀBf =∑ u∈V ( ( Bf ) u ) 2 . One could try to solve the flow estimation problem by minimizing ||Bf ||22 while keeping the observed flows fixed , however , this problem is ill-posed—there might be multiple solutions to the optimization . The standard approach in such a scenario is to resort to regularization . In particular , we apply a generic regularization function Φ with parameters Θ as follows : f∗ = arg min f∈Ω ||Bf ||22 + Φ ( f , X ; f ( 0 ) ; Θ ) st. fe = f̂e , ∀e ∈ E ′ ( 2 ) where Ω is the domain of f , f ( 0 ) ∈ Rm is a prior for flows , fe ( f̂e ) are entries of f ( f̂ ) for edge e and the constraint guarantees that observed flows are not changed . Priors f ( 0 ) , not be confused with observed flows f̂ , should be set according to the application ( e.g. , as zero , based on a black-box model or historical data ) . Regarding the domain Ω , we consider Ω = Rm and Ω = Rm+ . The second case is relevant for directed graphs—when flows must follow edge orientations ( e.g. , traffic ) . In Jia et al . ( 2019 ) , the authors set Φ ( f , X , f ( 0 ) ; Θ ) as λ2||f ||22 for a regularization parameter λ , which implies a uniform zero prior with anL2 penalty over edges . We claim that the regularization function plays an important role in capturing the physics of flow graphs . As an example , for a power network , Φ should account for the resistance of the lines . Thus , we propose learning the regularization from data . Our approach is based on a least-squares formulation , which will be described next . 3.2 REGULARIZED LEAST-SQUARES FORMULATION . Flow estimation problem can be viewed as an inverse problem ( Tarantola , 2005 ) . Let x ∈ Rm−m′ be the vector of missing flows and H ∈ Rm×m−m′ be a matrix such that Hij = 1 if fi maps to xj ( i.e. , they are associated to the same edge ) , and Hi , j = 0 , otherwise . Moreover , let f̃ ∈ Rm be such that f̃e = f̂e if e ∈ E ′ and f̃i = 0 , otherwise . Using this notation , we define flow estimation as BHx = −Bf̃ + , where BH is a forward operator , projecting x to a vector of vertex divergences , and −Bf̃ + is the observed data , capturing ( negative ) vertex divergences for observed flows . The error can be interpreted as noise in observations or some level of model misspecification . We can also define a regularized least-squares problem with the goal of recovering missing flows x : x∗ = arg min x∈Ω′ ||BHx +Bf̃ ||22 + ||x− x ( 0 ) ||2Q ( X ; Θ ) ( 3 ) where Ω′ is a projection of the domain of f to the space of x , ||x||2M = xᵀMx is the matrixscaled norm of x and x ( 0 ) ∈ Rm−m′ are priors for missing flows . The regularization function Φ ( f , X ; f ( 0 ) , Θ ) has the form ||x−x ( 0 ) ||2Q ( X ; Θ ) , where the matrixQ ( X ; Θ ) is a function of parameters Θ and edge features X . We focus on the case where Q ( X ; Θ ) is non-negative and diagonal . Equation 3 has a Bayesian interpretation , with x being a maximum likelihood estimate under a Gaussian assumption—i.e. , x ∼ N ( x ( 0 ) , Q ( X ; Θ ) −1 ) and Bf̃ ∼ N ( 0 , I ) ( Tarantola , 2005 ) . Thus , Q ( X ; Θ ) captures the variance in flow observations f̂ in prior estimates f ( 0 ) compared to the one . This allows the regularization function to adapt to different edges based on their features . For instance , in our road network example , Q ( X ; Θ ) might place a lower weight on flow conservation for flows at a road segment with a small number of lanes , which are possible traffic bottlenecks . Given the least-squares formulation described in this section , how do we model the regularization function Q and learn its parameters Θ ? We would like Q to be expressive enough to be able to capture complex physical properties of flows , while Θ to be computed accurately and efficiently . We will address these challenges in the remaining of this paper .

Summary: The authors propose a parametric regularizer for estimating unobserved flows in networks, incorporating edge features and other side information. The parameters of the regularizer are learned by means of minimizing the empirical cross-validated MSE. Regularization is necessary because the basic problem, while convex, typically is under-constrained; resulting in a infinite space of solutions which match the observed data.