Physics-Informed Implicit Representations of Equilibrium Network Flows
Anonymous Author(s)
AfﬁliationAddress
Abstract
Flow networks are ubiquitous in natural and engineered systems, and in order tounderstand and manage these networks, one must quantify the ﬂow of commoditiesacross their edges.
This paper considers the estimation problem of predictingunlabeled edge ﬂows from nodal supply and demand.
We propose an implicitneural network layer that incorporates two fundamental physical laws: conservationof mass, and the existence of a constitutive relationship between edge ﬂows andnodal states (e.g., Ohm’s law).
Computing the edge ﬂows from these two laws isa nonlinear inverse problem, which our layer solves efﬁciently with a specializedcontraction mapping.
Using implicit differentiation to compute the solution’sgradients, our model is able to learn the constitutive relationship within a semisupervised framework.
We demonstrate that our approach can accurately predictedge ﬂows in AC power networks and water distribution systems.
Introduction
Network ﬂows are a fundamental aspect of modern society, from trafﬁc and communication networksto power and water distribution systems.
Many critical infrastructures are well-modeled as graphs,with edges that transport vital commodities [1].
Beyond infrastructure, network ﬂows are also centralto models in epidemiology, ecology, medicine, and chemical networks.
Their dynamics have beenwell-studied in compartmental systems theory [2].
Given the prevalence of ﬂow networks in naturaland engineered systems, predicting ﬂows in these networks is an important learning task that mayfacilitate monitoring, control, optimization, and protection of these networks.
While domain-speciﬁc tools to predict network ﬂows have been around for a while, the machinelearning community has only recently taken an interest in general-purpose models for networkﬂows.
[3] predicts edge ﬂows from partial measurements by making a smoothing assumption, i.e., byminimizing nodal ﬂow divergence.
[4] improves on this approach by adding a trainable regularizer thatcan incorporate side information.
Both of these approaches are centered on a notion of approximateconservation, i.e., that the net inﬂow to each node should be near zero.
Since conservation of mass isa universal constraint on network ﬂows, imposing this conservation law is an important step towardembedding physics into the model.
But the conservation law alone is not enough to uniquely determine ﬂow, which is why both [3] and[4] rely on heuristic regularizers to select the “best” conservation-respecting ﬂow.
In fact, physicalnetworks are often governed by a pair of physical laws: the conservation law, and a constitutiverelationship , which speciﬁes the magnitude and direction of each edge ﬂow based on “effort” variablesat each incident node (e.g., pressure or voltage).
For example, in DC circuits, currents are conservedaccording to Kirchoff’s current law, and Ohm’s law is the constitutive relationship that relates currentﬂows to nodal potentials.
The conservation law and the constitutive relationship together deﬁne theunique edge ﬂows (and nodal efforts).
1.1 Contributions
This paper proposes a model for network ﬂows that embeds both the conservation law and existenceof a constitutive relationship.
Our model, which we call an Implicit Flow Network (IFN), predictseach edge ﬂow using a trainable nonlinear function of latent nodal variables.
These latent variablesare constrained to a manifold wherein the conservation law is satisﬁed.
In addition to introducing
IFN, we offer the following contributions: (i) a contraction algorithm that is able to both evaluatethe IFN layer and backpropagate gradients through it, (ii) an explicit upper bound on the numberof iterations required by this algorithm, (iii) a rigorous theoretical comparison between IFN andthe state-of-the-art ﬂow estimation methods in [3, 4], and (iv) numerical experiments from several AC power networks and water distribution systems that indicate IFN can signiﬁcantly outperformthese baselines on the ﬂow estimation task.
Additionally, because IFN requires a nonlinearity with aconstrained slope, we provide (v) a novel “derivative-constrained perceptron”, which is essentially atrainable activation function with upper and lower bounds on its slope.
1.2 Related Work
Network Flow Estimation Flows on graphs are a classical topic in computer science [5], and ﬂowforecasting has long been studied in speciﬁc domains like trafﬁc [6], but interest in the ﬂow estimationtask from a machine learning perspective appears to be relatively recent.
Deep learning algorithmshave been used to predict trafﬁc ﬂows [7, 8] and power ﬂows [9], but [3] and [4] appear to be the ﬁrstpapers to propose methods for domain-agnostic ﬂow prediction, based on the notion of divergenceminimization.
Implicit Neural Networks IFN belongs to a growing class of models called implicit neural networks, which do not explicitly state the output of the model; rather, they describe a desired relationshipbetween the model’s inputs and outputs.
In the prevailing implicit framework, the output is deﬁned asa ﬁxed point of a trainable perceptron.
This approach was introduced in [10] as a “deep equilibriumnetwork”.
Subsequent work has developed new frameworks for ensuring the existence of the ﬁxedpoint and computing it [11, 12, 13, 14, 15].
Other types of implicit neural networks include neural
ODEs [16] and layers that solve convex optimization problems [17].
Graph Neural Networks Graph neural networks (GNN) are a diverse family of models for networkrelated learning tasks that incorporate graph structure directly into the model.
GNNs can typically beclassiﬁed into three types, in increasing order of generality [18, §5.3]: convolutional models [19, 20],attentional models [21], and message-passing models [22, 23].
Recently, [24] proposed an implicitgraph convolutional network.
Analogously, IFN can be interpreted as an implicit message-passing
GNN, with ﬂows serving as messages and latent nodal variables acting as an embedding.
1.3 Preliminaries and Notation
Given a directed graph G =
( V , E ) , the signed incidence matrix B ∈ {− 1 , 0 , 1 } |V|×|E| is the matrixwith entries
B i,e =   1 , i is the head of e − 1 , i is the tail of e 0 , else , ∀ i ∈ V and e ∈ E
For an undirected graph, the signed incidence matrix is obtained by assigning an aribtrary orientationto each edge. For each i ∈ V , let N in ( i ) , N out ( i ) ⊂
V be the in-neighbors and out-neighbors of i . Given a vector x ∈ R n , we use the notation [ x ] to denote the diagonal matrix diag( x ) ∈ R n × n .
Where such notation would be unclear (e.g., may be confused with brackets to indicate order ofoperations), we fall back on the diag( · ) notation.
We write x ⊥ to refer to the vector space that isorthogonal to x , i.e., the space { x (cid:48) ∈ R n : x (cid:62) x (cid:48)
= 0 } Flow
Nodal Variable h
( y ) =
[Table]
Implicit Flow Networks
IFN is inspired by the physics of network systems.
In many physical networks, nodes “communicate”through the exchange of a commodity, like power, water, or force, which can be represented as edgeﬂows.
Flows obey a conservation law: for all i ∈ V ,
[Equation] [Table] [Equation] [Table]
[Equation]
[Table] [Equation] [Table] [Equation]
(1) where u ∈ R |V| are nodal inﬂows from outside the network, and f ∈ R |E| are the edge ﬂows.
Furthermore, the ﬂows are related to nodal variables through a constitutive relationship (CR); there issome strictly increasing function h such that, for all ( i, j ) ∈ E ,
[Equation]
(2) where a ∈ R |E| are edge weights and x ∈ R |V| are nodal “efforts” or “potentials.”
For example, in
DC power networks, the CR is Ohm’s law f ( i,j ) = r − 1 ( i,j ) ( x i − x j ) , where r are resistances and x arevoltages. In lossless AC networks, the CR is the active power ﬂow equation f
( i,j )
= a ( i,j ) sin( x i − x j ) ,where the edge weights are a function of line parameters and x are voltage angles [31, §6.4].
In waterdistribution systems, the CR is the Hazen-Williams formula [32, Sec. 8.15].
Table 1 lists several ﬂownetworks, the physical interpretation of the effort variables x , and the ﬂow function h .
We propose IFN as a layer that predicts edge ﬂows based on these two physical laws—conservationand the existence of a CR:
Deﬁnition 2.1 (Implicit Flow Network) .
An implicit ﬂow network (IFN) is a module with thefollowing components:
ﬁxed parameters 0 < d min ≤ d max ,2.
trainable parameters θ ∈ R r for some r , and3.
a family of differentiable functions h θ : R → R such that d min ≤ h (cid:48) θ ( y ) ≤
d max for ally ∈ R and θ ∈ R r , which we call ﬂow functions .
The module requires each of the following inputs:
a weighted, connected, undirected graph G =
( V , E , a ) with edge weights a ∈ R |E| > 0 , and2.
a supply / demand vector u ∈ R |V| such that (cid:80) i ∈V u i = 0 .
The module outputs the unique vector f ∈ R |E| for which there exists x ∈ R |V| such that
[Equation]
where B ∈ {− 1 , 0 , 1 } |V|×|E| is the signed incidence matrix of G , and h θ is applied element-wise.
We use the notation FN h,θ ( G, u ) to represent the solution f given inputs G and u , ﬂow functions h ,and parameters θ .
We will prove that IFNs are well-posed in Theorem 2.2.
Note that (3) and (4) are just vectorizedstatements of the conservation law (1) and the CR (2), so these two physical laws directly deﬁne the
[Figure] output. The IFN’s only trainable component is its ﬂow function, parameterized by θ .
In practice, wewill only make calls to the inverse of the ﬂow function when evaluating and backpropagating through IFN layers, so it is convenient to learn the inverse ﬂow function directly.
We emphasize that IFNs are layers that can be situated in more complex architectures, with othermodels upstream estimating the supply / demand vector, edge weights, or even the topology.
Forexample, in power systems, demand forecasting is a very well-studied problem [25, 26], and one cansolve the economic dispatch problem to forecast power generation at each node [27], collectivelyleading to an estimate of the supply / demand vector.
2.1 Evaluating the Implicit Flow Network
Our approach to evaluating the implicit ﬂow network is adapted from [28] and is illustrated in Figure1.
Any undirected graph G induces a direct decomposition of the edge ﬂow space R |E| : given theincidence matrix B ∈ {− 1 , 0 , 1 } |V|×|E| , the cycle space ker( B ) and cutset space Img( B (cid:62) ) areorthogonal, and R |E| =
ker( B ) ⊕ Img( B (cid:62) ) .
We refer the reader to [29, §9.4] for a primer on cycleand cutset spaces.
Accordingly, we decompose the vector f = FN
h,θ ( G, u ) as f = f cyc + f cut ,where f cyc ∈ ker( B ) and f cut
∈ Img( B (cid:62) ) .
The cutset component is readily determined from (3),since Bf = Bf cut = u implies that f cut
= B † u .
Then we must analyze (4) to solve for f cyc .
Deﬁnea cycle projection matrix P ∈ R m × m as the oblique projection onto ker( B ) parallel to Img([ a ] B (cid:62) ) :
[Equation]
Based on this projection, we deﬁne a map T : ker( B ) → ker( B ) for all f cyc ∈ ker( B ) by
[Equation]
We can show that f cyc is the unique ﬁxed point of T , and that T is a contraction mapping, leading toa simple algorithm to compute this ﬁxed point.
Theorem 2.2 (Properties of T ) .
Consider an implicit ﬂow network with parameters d min , d max , andθ , with ﬂow functions h θ .
Suppose that the inputs G =
( V , E , a ) and u ∈ 1 ⊥|V| are given, and let
B ∈ {− 1 , 0 , 1 } |V|×|E| be the signed incidence matrix of G .
The following are true:
T is a contraction mapping with respect to || · || 2 , [ a ] − 1 , with Lipschitz constant Lip( T ) ≤ 1 − d min d max ,
the sequence of iterates f ( k +1) cyc
= T ( f ( k ) cyc ) starting from any initial condition f (0)cyc ∈ ker( B )
converges to a unique ﬁxed point f cyc ,
the output of the implicit ﬂow network is unique and given by
[Equation]
Consequently, IFN is well-posed.
Theorem 2.2 provides a simple algorithm for computing the IFN output f : pick any f (0)cyc ∈ ker( B ) ,repeatedly apply the map T until approximate convergence, then add B † u .
Some care is requiredwhen implementing this map.
Since P is a dense matrix with |E| 2 entries, it is undesirable to explicitlyconstruct the cycle space projection matrix for large networks.
Instead, in order to project a vectorv ∈ R |E| , we can use the fact that
[Equation]
so the projection is evaluated as Pv = v − [ a ] B (cid:62) w .
Using this method of projection to implement
T , the ﬁxed point iteration to compute FN h,θ ( G, u ) is stated in Algorithm 1.
Algorithm 1 Evaluating the implicit ﬂow network.
[Equation]
[Equation] [Equation] [Equation] 2 (cid:9) 8: ∆ f cyc ← v − [ a ] B (cid:62) w 9: f cyc ← f cyc − ∆ f cyc 10: end while 11: f ← f cyc + f cut 12: return f
Theorem 2.3 (Implicit Flow Networks, Forward Pass) .
Consider an implicit ﬂow network withparameters d min , d max , and θ , with ﬂow functions h θ .
Suppose that the inputs G =
( V , E , a ) andu ∈ 1 ⊥|V| are given, and let B ∈ {− 1 , 0 , 1 } |V|×|E| be the signed incidence matrix of G .
The followingare true of Algorithm 1, with a tolerance of (cid:15) > 0 :1.
for each iteration k = 1 , 2 , . . .
of the loop, let f ( k ) cyc represent the new value of f cyc deﬁnedon line 9; and let f (0)cyc = 0 m .
Then
[Equation]
the algorithm converges with at most k ∗ iterations of the while loop, where
[Equation]
and ρ = || [ a ] 12 h − 1 θ ([ a ] − 1 B † u ) || 2 ; and3.
the algorithm returns f ∈ R |E| , where
[Equation] [Equation]
If evaluating h − 1 θ is sufﬁciently simple, then the most expensive step in the iteration is solving theordinary least squares problem on line 7.
Using a general-purpose solver, the complexity of this operation is roughly O ( |V| 3 ) .
But B [ a ] B (cid:62) is a sparse Laplacian matrix, so we can use a specialized
Laplacian solver that reduces the complexity to O ( |E| log k |E| ) for some constant k [30].
The bound on the number of iterations k ∗ can be computed before any forward pass, since evaluatingh − 1 θ does not require solving the IFN equations.
But we can further simplify the bound by approximating h − 1 θ (0) = 0 , which is often justiﬁed because physical ﬂow functions generally have a root atthe origin.
Using the fact that ( h − 1 θ ) (cid:48) ( y ) ≤ d − 1min y , we can then eliminate the dependence on h − 1 θ :
[Equation] 2.2 Computing the Gradients
In order to train the ﬂow function and any upstream models, it is necessary to backpropagate gradientsthrough the IFN layer.
We can perform this backward pass using implicit differentiation, and it turnsout that the gradients of FN h,θ ( G, u ) with respect to the parameters θ , a , and u can also be computedusing Algorithm 1, i.e., by writing the gradient as the output of an auxiliary implicit ﬂow network.
Theorem 2.4 (Gradients) .
Consider an implicit ﬂow network with parameters d min , d max , and θ ,with ﬂow functions h θ .
Suppose that the inputs G =
( V , E , a ) and u ∈ 1 ⊥ |V| are given, and let
B ∈ {− 1 , 0 , 1 } |V|×|E| be the signed incidence matrix of G .
Let f = FN
h,θ ( G, u ) , and let w be ascalar entry of θ , a , or u .
We can compute the derivatives dfdw as follows.
Deﬁne a vector of ﬂow functions g : R |E| → R |E| by
[Equation]
[Equation]
where D ∈ R |E|×|E| is the diagonal matrix with entries
[Equation]
and v = [ a ] h − 1 θ ([ a ] − 1 f cyc + [ a ] − 1 B † u ) .
Then
[Equation] [Equation]
(We use the notation · in place of θ , since g has no trainable parameters.)
Furthermore, the derivativeconstraint parameters d min , d max from the original implicit ﬂow network are valid for the new implicitﬂow network.
In other words, to compute the gradient with respect to a parameter, we perform a single evaluationof the implicit ﬂow network.
In order to compute the derivatives with respect to some parameter orinput w , we ﬁrst evaluate the partial derivatives ∂v∂w and the total derivatives dudw .
Then we constructthe ﬂow functions g according to (10), and solve an implicit ﬂow network to ﬁnd dfdw according to(12).
It is easy to evaluate dudw , but for convenience, we provide the values of ∂v∂w below:
[Equation]
Comparison with Optimization Models
Both of the state-of-the-art methods for ﬂow estimation, from [3] and [4], use an optimization problem to predict ﬂows.
After a suitable transformation to incorporate external ﬂow injections u , we canstate this optimization problem as
[Equation]
where λ > 0 , and q > 0 m is some vector of edge weights.
In [3], q = 1 m , while [4] allows q to bethe output of a neural network.
IFN is not explicitly an optimization problem, but it can be cast asone that is similar (13):
Theorem 3.1 (Optimization Form of IFN) .
Consider an IFN with ﬂow function h θ .
Suppose that theinputs G =
( V , E , a ) and u ∈ 1 ⊥|V| are given, and let B ∈ {− 1 , 0 , 1 } |V|×|E| be the signed incidencematrix of G .
Then the IFN output can be stated as the solution of a convex optimization problem:
[Equation]
Theorem 3.1 can be interpreted as a nonlinear generalization of the Thomson principle from electricalcircuits theory [34].
Interestingly, the theorem sets up a direct comparison between IFN and themodels in [3] and [4].
If the ﬂow function h θ is the identity map, then (14) can be simpliﬁed as
[Equation]
Ignoring the constraints from labeled ﬂows, we can interpret (13) as using a penalty method toapproximate the output of an IFN with a linear ﬂow function.
Thus, we have three distinct differencesbetween IFN and the optimization-based approaches.
First, IFN allows for a nonlinear ﬂow function,while [3] and [4] implicitly assume a linear CR.
Second, IFN imposes ﬂow conservation as a hardconstraint rather than an approximate constraint (which is a limitation if u is uncertain).
Finally, IFNdoes not incorporate ﬂow measurements directly; rather, the model exploits these measurementsduring training to learn the proper ﬂow function (and train any upstream models for the IFN inputs),making it less sensitive to noise in the labeled ﬂows.
Models for Flow Functions
In order to implement an IFN, it is necessary to parameterize its inverse ﬂow function h − 1 θ .
Since theﬂow function is essentially a trainable activation function, i.e., a scalar nonlinearity, simple modelsare likely to be sufﬁcient.
The main difﬁculty with selecting a ﬂow function is that its slope must bebounded by d min ≤ h (cid:48) θ ( y ) ≤
d max for all y ∈ R .
This section proposes a simple scalar nonlinearitythat is guaranteed to respect arbitrary upper and lower bounds on its slope.
Deﬁnition 4.1 (Derivative-Constrained Perceptron) .
Let k ∈ Z > 0 be a hidden layer size, let a, b, c ∈
R k be freely trainable parameters (encoded within the parameter vector θ ), and let σ be a nonexpansive activation.
Let p, q ≥ 1 such that p − 1 + q − 1 = 1 , and let ¯ d min ≤ ¯ d max ∈ R .
Then thederivative-constrained perceptron N ( x, θ ) is the 3-layer neural network deﬁned by
[Equation]
Intuitively, (L1) re-scales c so that the perceptron in (L2) is guaranteed to be non-expansive in x . Then(L3) re-centers and re-scales the derivatives of the perceptron from the range [ − 1 , 1] to [ ¯ d min , ¯ d max ] .
Theorem 4.2 (Derivative-Constrained Perceptron) .
Let N( x, θ ) be a derivative-constrained perceptron with ¯ d min ≤ ¯ d max ∈ R .
Then for all parameter values θ ,
[Equation]
[Equation]
Note that the values ¯ d min , ¯ d max in Deﬁnition 4.1 and Theorem 4.2 are distinct from the IFN parametersd min , d max .
Since we parameterize the inverse ﬂow function h − 1 θ in IFN, one shoudl set ¯ d min = d − 1maxand ¯ d max = d − 1min to implement h − 1 θ with a derivative-constrained perceptron.  Numerical Experiments We studied the transductive task of predicting unlabeled ﬂows, given that some labeled ﬂows in thesame network are known.
If the edges E are partitioned into a labeled set E l and an unlabeled set E u , the task is to predict the missing ﬂows { f e : e ∈ E u } given the labeled ﬂows { f e : e ∈ E l } . For eachnetwork, we randomly selected a fraction of the edges to be labeled edges, and we trained IFN andbaselines on the labeled edges. Then we evaluated the RMSE of the ﬂows predicted for the unlabelededges E u to compute the testing error. See Appendix B in the supplementary material for full details.
AC Power We selected 6 standard power network test cases.
The ﬁrst 4 test cases (IEEE-57, IEEE118, IEEE-145, and IEEE-300) are synthetic transmission system test cases, while the remainingcases ACTIVSg200 and ACTIVSg500 are similar to the Illinois and South Carolina power grids,respectively [35].
Each test case contains the topology and electrical parameters of the power network,as well as baseline demands and power injections at each node.
While branch resistances are typicallysmall, we set them to zero to ensure lossless transmission lines.
We used the MATPOWER toolbox[36] to solve the power ﬂow equations, then recorded the active power ﬂows on each branch ( f ),computed the net active power injections at each node ( u ), and selected relevant electrical parametersas edge attributes (series reactance, tap ratio, and voltage magnitude at the two incident nodes).
Water Distribution We selected 3 sample water distribution networks from the ASCE Task Committee on Research Databases for Water Distribution Systems database [37], representing municipalwater distribution systems in Fairﬁeld, CA, Bellingham, WA, and Harrisburg, PA. Each networkcontains the topology of the distribution system, as well as the characteristics of pipes and othernetwork elements and nodal demands. We used the WNTR package [38] to compute the ﬂow ratesthrough each pipe ( f ), net inﬂow rate at each node ( u ), and edge weights associated with each pipe.
5.2 Models and Experiment Details
IFN Architecture In order to use the IFN layer to predict power ﬂows, we created a two-layermodel.
The ﬁrst layer estimates positive edge weights a ∈ R |E| according to a e = exp ( L ( z e )) for alle ∈ E , where L is a linear module, and z e is the log-transformed vector of edge attributes. The secondlayer is an IFN.
To predict water ﬂows, we used an IFN layer alone, supplying the edge weightsfrom the dataset as input (rather than learning them from other edge attributes).
For both water andpower, the IFN layer uses a derivative-constrained perceptron as the inverse ﬂow function ( k = 128 ,p = q = 12 ) with a ReLU activation function. For power, we set d min = 0 . 4 and d max = 2 ; and forwater, d min = 0 .
2 and d max = 20 .
Baselines We compared the IFN model against four baselines.
The minimum divergence method( Div ) from [3] minimizes the nodal divergence ||
Bf || 22 and a regularization term
λ ||
f ||
The bileveloptimization methods from [4] replace the uniform regularizer with a weighted regularizer
||
f || 22 , [ q ] ,where q is a vector of weights.
In Bil-MLP and Bil-GCN , q is the output of either a 2-layer MLPor GCN model with edge attributes as inputs (we use 64 nodes in each hidden layer with ReLUactivations).
In Bil-True , we specify q as the reciprocal of the coefﬁcient in the linearized CR for AC power networks, so that Bil-True approximates (15) with a as the ground-truth edge weight. Forwater experiments, Bil-True uses the same edge weights as the IFN model.
All of the baselines assume that nodal divergence Bf should be approximately zero, but nodes inpower networks inject and withdraw power according to the supply / demand vector u , resulting innonzero divergence.
Thus, when we evaluate the baselines, we transform the power network into adivergence-free network by introducing a “source node”, adding an edge from the source node to allnodes in V , and treating the entries of u as the ﬂows along each corresponding virtual edge.
5.3 Results
Figure 2 reports the results for the AC power networks, and Figure 3 reports the results for waterdistribution systems.
In both types of networks, the IFN model signiﬁcantly outperforms the baselineson all of the networks when a small fraction of edges are labeled (less than 80% in power and less than60% in water).
While the other baselines tend to improve as more labeled edges are made availablefor training, IFN achieves near-optimal performance with as few as 10% of the edges labeled.
[Figure]
[Figure]
Conclusion
In this paper, we have introduced an implicit model for network ﬂows that incorporates physicsthrough a conservation law and through the existence of a constitutive relationship between ﬂowsand nodal variables.
We have demonstrated that a simple architecture using this model can learn toaccurately predict active power ﬂows in AC networks and water distribution systems.
Future workmay investigate more elaborate architectures using IFN as a layer, wherein the supply / demandvector, edge weights, or even the graph itself could be predicted from upstream models, and the ﬂowsthemselves used for downstream tasks.
Another interesting extension may be to extend our method tonetworks with higher-order interactions, i.e., hypergraphs [39] and simplicial complexes [40, 41].
IFN has some limitations that should also be addressed in future work.
IFN assumes that the graph isundirected, which does not adequately model networks with unidirectional ﬂows (e.g., trafﬁc) or lossyﬂows (e.g., resistive power grids).
IFN also assumes a CR that depends on the difference betweennodal variables.
This form appears frequently in physical systems, but in other network ﬂow models(like Daganzo trafﬁc models [42]), the CR has a more general dependence on the nodal variables.
These limitations may be addressed with extensions of IFN’s contraction algorithm.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
We do not
forsee any negative societal impacts from ﬂow estimation.
On the contrary, we hope
that these techniques will be useful for managing and protecting critical infrastructure.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[Yes] Code and
detailed instructions are included in the supplementary material.
(b) Did you specify all the training details (e g., data splits, hyperparameters,
how they
were chosen)?
[Yes] Training details are discussed in the supplementary material.
(c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[No] While error bars could be included in Figure 2, we felt itadded considerable clutter to an already dense plot.
(d) Did you include the total amount of compute and the type of resources used (e.g.,type of GPUs, internal cluster, or cloud provider)?
[Yes] As mentioned in Section 5.2,
computation was performed in a Google Colab notebook.
GPU type varies in Colab,so we did not report the speciﬁcs.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes](b) Did you mention the license of the assets?
[No](c) Did you include any new assets either in the supplemental material or as a URL?
[Yes](d) Did you discuss whether
and how consent was obtained from people whose data you’reusing/curating?
[N/A](e)
Did you discuss whether the data you are using/curating contains personally identiﬁableinformation or offensive content?
[N/A]5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A](b)
Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A](c)
Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]
A Proofs
A.1 Proof of Theorem 2.2
To prove statement 1, choose any f cyc ∈ ker( B ) , let y =
[ a ] − 1 f cyc +
[ a ] − 1 B † u for brevity, andobserve that
[Equation]
− 1 [ a ]
[Equation]
[Equation]
[ a ]
[Equation]
where || [ a ] − 12 P [ a ] 12 || 2 = 1 because [ a ] − 12 P [ a ] 12 is a symmetric and idempotent matrix, i.e., anorthogonal projection.
Then
[Equation]
Hence
[Equation]
Then statement 2 follows from statement 1 and the Banach ﬁxed point theorem.
To prove statement 3,observe that f cyc =
Pf cyc , so f cyc = T ( f cyc )
if and only if
[Equation]
(17) where f = f cyc + B † u .
But ker( P [ a ]) =
Img( B (cid:62) ) , so (17) is equivalent to the existence of x ∈ R nsuch that
[Equation]
and (18) is equivalent to (4).
A.2 Proof of Theorem 2.3
To prove statement 1, let k ≥ 0 and consider iteration k
+ 1 of the loop.
The iteration ﬁrst deﬁnesv =
d min [ a ] h − 1 θ (cid:16) [ a ] − 1 f ( k ) cyc + [ a ] − 1 f cut (cid:17) on line 6.
Then on line 7,
[Equation]
and line 8 deﬁnes
[Equation]
Finally, on line 9,
[Equation]
A simple inductive argument shows that f ( k ) cyc ∈ ker( B ) .
The base case f (0)cyc = 0 m is trivial, for allk (cid:48) ≥ 0 , line 9 ensures that f ( k (cid:48) +1) cyc ∈ ker( B ) so long as f ( k (cid:48) )
cyc ∈ ker( B ) .
Hence f ( k ) cyc =
Pf ( k ) cyc , andwe conclude that
[Equation]
[Table]
To prove statement 2, recall from Theorem 2.2 that Lip( T ) ≤ 1 − d − 1max d min , which (together withstatement 1) implies that, for all k ≥ 0 ,
[Equation]
The algorithm terminates after iteration k if and only if || f ( k ) cyc − f ( k − 1)
cyc
|| 2 , [ a ] − 12 ≤ (cid:15) , so the algorithmwill have terminated after k ∗ iterations if
[Equation]
which is equivalent to
[Equation]
Finally, to prove statement 3, note that the algorithm terminates after iteration k as soon as
[Equation]
If f cyc is the true ﬁxed point of T , then using a general property of contraction mappings,
[Equation]
Therefore, the vector f returned by the algorithm satisﬁes
[Equation]
A.3 Proof of Theorem 2.4
Let v =
[ a ] h − 1 θ ([ a ] − 1 f cyc + [ a ] − 1 B † u ) .
From Theorem 2.2, we can write f = f cyc + B † u , wheref cyc is the unique ﬁxed point of T .
Therefore dfdw =
df cyc dw + B † dudw , so the remainder of the proof isto show that df cyc dw
=
FN g, · ( G, 0 n ) .
Since f cyc = T ( f cyc ) , and Pf cyc = f cyc , we have
[Equation]
so an equivalent characterization of f cyc is the unique solution to the equations
[Equation]
Since
[Equation]
then differentiating and factoring out [ a ] , we obtain
[Equation]
Since ker( P [ a ])
=
Img( B (cid:62) ) , there exists x ∈ R n such that
[Equation]
which we can re-write as
[Equation]
Hence df cyc dw is the solution to
[Equation]
which is identical to (3)–(4) with df cyc dw in place of f , 0 n in place of u , and g in place of h θ .
Furthermore, g respects the same d min , d max derivative constraints as h θ , since for each e ∈ E ,
[Equation]
It follows that df cyc dw is the output of the implicit ﬂow network with ﬂow functions g and parametersd min , d max , evaluated on the original graph G and nodal demands 0 n .
A.4 Proof of Theorem 3.1
Since h − 1 θ is increasing, the optimization problem in (14) has a convex cost function with linearconstraints, so the KKT conditions are necessary and sufﬁcient.
Letting x ∈ R
m be a vector of
Lagrange multipliers, the Lagrangian is
[Equation]
leading to the stationarity condition
[Equation]
which is equivalent to (4).
Additionally, the primal constraint Bf = u is equivalent to (3), so theminimizer of the optimization problem is identical to the output of the IFN.
A.5 Proof of Theorem 4.2
Due to (L3), it is clear that the derivative bounds (16) hold if
and only if
[Equation]
[Equation]
For all x, x (cid:48) ∈ R , by Hölder’s inequality,
[Equation]
Since σ is non-expansive, its Lipschitz constant with respect to the q -norm is
[Equation]
≤and thus
[Equation]
[Table]
Furthermore, by (L1),
[Equation]
so that
[Equation]
for all x, x (cid:48) .
Hence (21) is satisﬁed.
B Experiment Details
B.1 AC Power Datasets
We created datasets from 6 AC power network test cases.
Each dataset that we created represents asnapshot of an AC power network in its steady state, consisting of four components: the networktopology (as an oriented, undirected graph), four attributes on each edge (voltage magnitude at thetwo incident nodes, series reactance, and tap ratio), the net power injection at each node, and theactive power ﬂow through each branch.
Original Data We generated our datasets using MATPOWER, an open-source toolbox for powersystem simulation in MATLAB [36].
The toolbox includes many standard test cases, which containa network topology and tables of electrical and economic parameters for each bus (node), branch(edge), and generator.
We selected 6 test cases, listed in Table 2.
The raw data ﬁles for these test casesare available from the MATPOWER source 1 , and details on the test case ﬁle format are contained in
Appendix B of the user manual 2 .
Data Generation After loading each test case into MATPOWER, we performed the following twomodiﬁcations of the network parameters:
We set branch resistances (column 3 in the branch data table) to zero, so that transmission
lines in the system are lossless.
This step was necessary because IFN is limited to undirected
graphs, while lossy lines are more appropriately modeled with a pair of directed edges,since the power injected at one endpoint does not equal the power withdrawn from the other
endpoint.
Fortunately, branch resistances are typically small before this modiﬁcation.
We replaced any negative series reactances (column 4 in the branch data table) with a
positive value, chosen as the median of the positive series reactances in the same network.
We performed this modiﬁcation because negative series reactances results in decreasingconstitutive relationships on the corresponding edges, whereas IFN assumes that the consti-
tutive relationship is increasing.
This modiﬁcation only affected two networks: IEEE-145, inwhich 24 (4.2%) of the branches were assigned a series reactance of 0.2306; and IEEE-300,in which 1 (0.1%) of the branches was assigned a series reactance of 0.059.
We then computed the resulting power ﬂows using the runpf function and recorded the results
V
E
Fairﬁeld
Bellingham
Harrisburg
Pre-Processing Finally, we converted the results from the MATPOWER simulation into a PyTorch
Geometric data object, with the following attributes:
• edge_index , the edge index tensor, containing the topology from the test case.
• x , a tensor of net active power injections at each node, which has the property that 1 (cid:62) n x = 0 .
(This tensor is identical to the supply / demand vector u in the paper.)
• edge_attr , a tensor of four relevant attributes for each edge: the voltage magnitudes at the
two incident nodes, the series reactance, and the tap ratio.
• f_true , the tensor of active power ﬂows on each edge simulated by MATPOWER.
The net active power injections at each node are computed according to
[Equation]
where PG i is active power generated at i , PD i is active power demanded, GS i is shunt conductance,and VM i is the voltage magnitude.
B.2 Water Distribution Dataset
We created 3 datasets representing snapshots of municipal water distribution networks in their steadystate, consisting of four components: the network topology (as an oriented, undirected graph), weightsfor each edge, the net inﬂow rates at each node, and the ﬂow rate through each pipe.
Original Data Each of the datasets is based on a network from the ASCE Task Committee on
Research Databases for Water Distribution Systems database [37].
Networks in this database containa distribution network topology and tables of hydraulic parameters and operating characteristics foreach node, pipe (edge), pump, reservoir, and storage tank in the network.
We selected 3 networks,listed in Table 3 and plotted in Figure 4.
The raw data ﬁles are available online 3 .
Data Generation and Preprocessing We loaded each network INP ﬁle into WNTR and ran the
WNTR simulator with a hydraulic accuracy of 10 − 8 .
We then converted the results into a PyTorch
Geometric data object, with the following attributes:
• edge_index , the edge index tensor, containing the topology from the test case.
• x , a tensor of net inﬂows at each node, which has the property that 1 (cid:62) n x = 0 .
• edge_attr , a tensor of three relevant attributes for each edge: the pipe length, pipe diameter,and pipe roughness coefﬁcient.
• f_true , the tensor of ﬂow rates through each pipe simulated by WNTR.
Edge weights are computed according to the formula
[Equation]
(22) where C e is the roughness coefﬁcient (unitless), D e is the diameter (meters), and L e is the pipe length(meters) 4 .
B.3 Details on IFN
Our IFN implementation uses Algorithm 1 to compute the layer’s forward pass.
We set the maximumnumber of iterations in this algorithm to 100, with a tolerance of (cid:15) =
10 − 2 for power and (cid:15) = 10 − 4for water.
With the release of PyTorch 1.11.0, the torch.linalg.lstsq method 5 now supportsautomatic differentiation, allowing PyTorch to automatically backpropagate through the Algorithm 1iterations, instead of using Theorem 2.4.
We found that Algorithm 1 terminated with a small enoughnumber of iterations that automatic differentiation was faster, so we opted to use this rather thanthe method from Theorem 2.4.
We trained the IFN models to minimize the RMSE loss function byminimizing the RMSE loss function
[Equation]
B.4 Details on Baselines
We implemented all of the baselines by adapting Silva’s code 6 from [4], refactoring some utilityfunctions to decrease runtime.
Following [4], we perform the following two data normalization steps:
negative ﬂows are converted into
positive
ﬂows
by ﬂipping the orientation of the corresponding edges and replacing the entries of f_true with their absolute value, and
ﬂows are proportionally normalized to the range [0 ,
within each network
After training with the normalized ﬂows and computing the missing ﬂow predictions, the predictionsare denormalized before computing the testing RMSE.
Div
The minimizing divergence baseline from [3] has a single hyperparameter, λ , from the regularization term λ
2 ||
f || 22 in the loss function.
We set λ = 0 .
1 for all networks and fractions of labelededges by hand-tuning the parameter to the proper order of magnitude.
Bilevel Baselines
All three of the bilevel baselines (Bil-MLP, Bil-GCN, and Bil-True) have severalhyperparameters related to the bilevel optimization algorithm.
For most of these parameters, we usethe same settings as [4]: the number of iterations for the inner optimization problem is 300 duringtraining and 3000 during evaluation, and the number of k-fold cross validation folds is 10; however,we increased the number of iterations of the outer optimization problem from 10 to 100, with an earlystopping interval of 10, to ensure that the outer optimization problem was given sufﬁcient time toconverge.
As with [4], we used a 2-layer MLP and GCN in Bil-MLP and Bil-GCN, respectively, butwe increased the size of the hidden layer to 64.
Bil-True Like IFN, the baselines Bil-MLP and Bil-GCN train a model to predict edge weightsfrom side information (if we interpret Q as a diagonal matrix of edge weights).
We devised Bil-Trueas a third baseline to use the “ground-truth edge weights” instead of training a model.
For waterexperiments, these ground-truth edge weights are given by (22).
For the power experiments, wecompute these edges weights from the AC active power ﬂow equation: in a lossless AC power grid,active power ﬂows f ij on each edge { i, j } ∈ E are given by
[Equation]
(23) where v i , v j are the voltage magnitudes on incident nodes, x ij is the series reactance, τ ij is the tapratio, and θ i , θ j are the incident voltage angles.
Since (23) is the constitutive relationship for ACpower networks, examining its linear approximation in light of (15) suggests using x ij τ ij /v i v j as theregularizer weight on f ij .
B.5 Details on Training
We trained all models in a Google Colab notebook, using the Adam optimizer.
We used an initiallearning rate of 0 . 01 , which we found to have good performance for all of the models.
Training wasterminated when the training loss had not decreased for 10 epochs.