Combining Physics and Machine Learning for Network Flow Estimation

1 INTRODUCTION . In many applications , ranging from road traffic to supply chains to power networks , the dynamics of flows on edges of a graph is governed by physical laws/models ( Bressan et al. , 2014 ; Garavello & Piccoli , 2006 ) . For instance , the LWR model describes equilibrium equations for road traffic Lighthill & Whitham ( 1955 ) ; Richards ( 1956 ) . However , it is often difficult to fully observe flows in these applications and , as a result , they rely on off-the-shelf machine learning models to make predictions about missing flows ( Li et al. , 2017 ; Yu et al. , 2018 ) . A key limitation of these machine learning models is that they disregard the physics governing the flows . So , the question arises : can we combine physics and machine learning to make better flow predictions ? This paper investigates the problem of predicting missing edge flows based on partial observations and the underlying domain-specific physics defined by flow conservation and edge features ( Jia et al. , 2019 ) . Edge flows depend on the graph topology due to a flow conservation law—i.e . the total inflow at every vertex is approximately its total out-flow . Moreover , the flow at an edge also depends on its features , which might regularize the space of possible flow distributions in the graph . Here , we propose a model that learns how to predict missing flows from data using bilevel optimization ( Franceschi et al. , 2017 ) and neural networks . More specifically , features are given as inputs to a neural network that produces edge flow regularizers . Weights of the network are then optimized via reverse-mode differentiation based on a flow estimation loss from multiple train-validation pairs . Our work falls under a broader effort towards incorporating physics knowledge to machine learning , which is relevant for natural sciences and engineering applications where data availability is limited ( Rackauckas et al. , 2020 ) . Conservation laws ( of energy , mass , momentum , charge , etc . ) are essential to our understanding of the physical world . The classical Noether ’ s theorem shows that such laws arise from symmetries in nature ( Hanc et al. , 2004 ) . However , flow estimation , which is an inverse problem ( Tarantola , 2005 ; Arridge et al. , 2019 ) , is ill-posed under conservation alone . Regularization enables us to apply domain-knowledge in the solution of inverse problems . We motivate our problem and evaluate its solutions using two application scenarios . The first is road traffic networks ( Coclite et al. , 2005 ) , where vertices represent locations , edges are road segments , flows are counts of vehicles that traverse a segment and features include numbers of lanes and speed limits . The second scenario is electric power networks ( Dörfler et al. , 2018 ) , where vertices represent power buses , edges are power lines , flows are amounts of power transmitted and edge features include resistances and lengths of lines . Irrigation channels , gas pipelines , blood circulation , supply chains , air traffic , and telecommunication networks are other examples of flow graphs . Our contributions can be summarized as follows : ( 1 ) We introduce a missing flow estimation problem with applications in a broad class of flow graphs ; ( 2 ) we propose a model for flow estimation that is able to learn the physics of flows by combining reverse-mode differentiation and neural networks ; ( 3 ) we show that our model outperforms the best baseline by up to 18 % ; and ( 4 ) we provide evidence that our model learns interpretable physical properties , such as the role played by resistance in a power transmission network and by the number of lanes in a road traffic network . 2 FLOW ESTIMATION PROBLEM . We introduce the flow estimation problem , which consists of inferring missing flows in a network based on a flow conservation law and edge features . We provide a list of symbols in the Appendix . Flow Graph . Let G ( V , E , X ) be a flow graph with vertices V ( n= |V| ) , edges E ( m= |E| ) , and edge feature matrix X ∈ Rm×d , where X [ e ] are the features of edge e. A flow vector f ∈ Rm contains the ( possibly noisy ) flow fe for each edge e ∈ E . In case G is directed , f ∈ Rm+ , otherwise , a flow is negative if it goes against the arbitrary orientation of its edge . We assume that flows are induced by the graph , and thus , the total flow—in plus out—at each vertex is approximately conserved : ∑ ( vi , u ) ∈E f ( vi , u ) ≈ ∑ ( u , vo ) ∈E f ( u , vo ) , ∀u ∈ V In the case of a road network , flow conservation implies that vehicles mostly remain on the road . Flow Estimation Problem . Given a graph G ( V , E , X ) with partial flow observations f̂ ∈ Rm′ for a subset E ′ ⊆ E of edges ( f̂e is the flow for e ∈ E ′ , m′= |E ′| < m ) , predict flows for edges in E \ E ′ . In our road network example , partial vehicle counts f̂ might be measured by sensors placed at a few segments , and the goal is to estimate counts at the remaining segments . One would expect flows not to be fully conserved in most applications due to the existence of inputs and outputs , such as parking lots and a power generators/consumers . In case these input and output values are known exactly , they can be easily incorporated to our problem as flow observations . Moreover , if they are known approximately , we can apply them as priors ( as will be detailed in the next section ) . For the remaining of this paper , we assume that inputs and outputs are unknown and employ flow conservation as an approximation of the system . Thus , different from classical flow optimization problems , such as min-cost flow ( Ahuja et al. , 1988 ) , we assume that flows are conserved approximately . Notice that our problem is similar to the one studied in Jia et al . ( 2019 ) . However , while their definition also assumes flow conservation , it does not take into account edge features . We claim that these features play important role in capturing the physics of flows . Our main contribution is a new model that is able to learn how to regularize flows based on edge features using neural networks . 3 OUR APPROACH : PHYSICS+LEARNING . In this section , we introduce our approach for the flow estimation problem , which is summarized in Figure 1 . We formulate flow estimation as an optimization problem ( Section 3.1 ) , where the interplay between the flow network topology and edge features is defined by the physics of flow graphs . Flow estimation is shown to be equivalent to a regularized least-squares problem ( Section 3.2 ) . Moreover , we describe how the effect of edge features and the graph topology can be learned from data using bilevel optimization and neural networks in Section 3.3 . Finally , we propose a reverse-mode differentiation algorithm for flow estimation in Section 3.4 . 3.1 FLOW ESTIMATION VIA OPTIMIZATION . The extent to which flow conservation holds for flows in a graph is known as divergence and can be measured using the oriented incidence matrix B ∈ Rn×m of G. The matrix is defined as follows , Bij = 1 if ∃u such that ej = ( vi , u ) ∈ E , Bij = −1 if ∃u such that ej = ( u , vi ) ∈ E , and Bij = 0 , otherwise . Given B and f , the divergence at a vertex u can be computed as : ( Bf ) u = ∑ ( vi , u ) ∈E f ( vi , u ) − ∑ ( u , vo ) ∈E f ( u , vo ) ( 1 ) And thus , we can compute the total ( squared ) divergence in the graph as ||Bf ||22 = fᵀBᵀBf =∑ u∈V ( ( Bf ) u ) 2 . One could try to solve the flow estimation problem by minimizing ||Bf ||22 while keeping the observed flows fixed , however , this problem is ill-posed—there might be multiple solutions to the optimization . The standard approach in such a scenario is to resort to regularization . In particular , we apply a generic regularization function Φ with parameters Θ as follows : f∗ = arg min f∈Ω ||Bf ||22 + Φ ( f , X ; f ( 0 ) ; Θ ) st. fe = f̂e , ∀e ∈ E ′ ( 2 ) where Ω is the domain of f , f ( 0 ) ∈ Rm is a prior for flows , fe ( f̂e ) are entries of f ( f̂ ) for edge e and the constraint guarantees that observed flows are not changed . Priors f ( 0 ) , not be confused with observed flows f̂ , should be set according to the application ( e.g. , as zero , based on a black-box model or historical data ) . Regarding the domain Ω , we consider Ω = Rm and Ω = Rm+ . The second case is relevant for directed graphs—when flows must follow edge orientations ( e.g. , traffic ) . In Jia et al . ( 2019 ) , the authors set Φ ( f , X , f ( 0 ) ; Θ ) as λ2||f ||22 for a regularization parameter λ , which implies a uniform zero prior with anL2 penalty over edges . We claim that the regularization function plays an important role in capturing the physics of flow graphs . As an example , for a power network , Φ should account for the resistance of the lines . Thus , we propose learning the regularization from data . Our approach is based on a least-squares formulation , which will be described next . 3.2 REGULARIZED LEAST-SQUARES FORMULATION . Flow estimation problem can be viewed as an inverse problem ( Tarantola , 2005 ) . Let x ∈ Rm−m′ be the vector of missing flows and H ∈ Rm×m−m′ be a matrix such that Hij = 1 if fi maps to xj ( i.e. , they are associated to the same edge ) , and Hi , j = 0 , otherwise . Moreover , let f̃ ∈ Rm be such that f̃e = f̂e if e ∈ E ′ and f̃i = 0 , otherwise . Using this notation , we define flow estimation as BHx = −Bf̃ + , where BH is a forward operator , projecting x to a vector of vertex divergences , and −Bf̃ + is the observed data , capturing ( negative ) vertex divergences for observed flows . The error can be interpreted as noise in observations or some level of model misspecification . We can also define a regularized least-squares problem with the goal of recovering missing flows x : x∗ = arg min x∈Ω′ ||BHx +Bf̃ ||22 + ||x− x ( 0 ) ||2Q ( X ; Θ ) ( 3 ) where Ω′ is a projection of the domain of f to the space of x , ||x||2M = xᵀMx is the matrixscaled norm of x and x ( 0 ) ∈ Rm−m′ are priors for missing flows . The regularization function Φ ( f , X ; f ( 0 ) , Θ ) has the form ||x−x ( 0 ) ||2Q ( X ; Θ ) , where the matrixQ ( X ; Θ ) is a function of parameters Θ and edge features X . We focus on the case where Q ( X ; Θ ) is non-negative and diagonal . Equation 3 has a Bayesian interpretation , with x being a maximum likelihood estimate under a Gaussian assumption—i.e. , x ∼ N ( x ( 0 ) , Q ( X ; Θ ) −1 ) and Bf̃ ∼ N ( 0 , I ) ( Tarantola , 2005 ) . Thus , Q ( X ; Θ ) captures the variance in flow observations f̂ in prior estimates f ( 0 ) compared to the one . This allows the regularization function to adapt to different edges based on their features . For instance , in our road network example , Q ( X ; Θ ) might place a lower weight on flow conservation for flows at a road segment with a small number of lanes , which are possible traffic bottlenecks . Given the least-squares formulation described in this section , how do we model the regularization function Q and learn its parameters Θ ? We would like Q to be expressive enough to be able to capture complex physical properties of flows , while Θ to be computed accurately and efficiently . We will address these challenges in the remaining of this paper .

Summary: In this paper, the authors introduce a method for missing flow estimation. These method has potential to address some important applications in transportation, power systems and water management. One major difference compared with the previous work is that edge features are incorporated into the optimization process so that the model has a better chance at learning edge-specific patterns. The experimental results have shown some success of the proposed method in traffic and power datasets.