Physics-Informed Implicit Representations of Equilibrium Network Flows
Abstract
Flow networks are ubiquitous in natural and engineered systems, and in order to understand and manage these networks, one must quantify the ﬂow of commodities across their edges.
This paper considers the estimation problem of predicting unlabeled edge ﬂows from nodal supply and demand.
We propose an implicit neural network layer that incorporates two fundamental physical laws: conservation of mass, and the existence of a constitutive relationship between edge ﬂows and nodal states (e.g., Ohm’s law).
Computing the edge ﬂows from these two laws is a nonlinear inverse problem, which our layer solves efﬁciently with a specialized contraction mapping.
Using implicit differentiation to compute the solution’s gradients, our model is able to learn the constitutive relationship within a semisupervised framework.
We demonstrate that our approach can accurately predict edge ﬂows in AC power networks and water distribution systems.
Introduction
Network ﬂows are a fundamental aspect of modern society, from trafﬁc and communication networks to power and water distribution systems.
Many critical infrastructures are well-modeled as graphs, with edges that transport vital commodities [1].
Beyond infrastructure, network ﬂows are also central to models in epidemiology, ecology, medicine and chemical networks.
Their dynamics have been well-studied in compartmental systems theory [2].
Given the prevalence of ﬂow networks in natural and engineered systems, predicting ﬂows in these networks is an important learning task that may facilitate monitoring, control, optimization, and protection of these networks.
While domain-speciﬁc tools to predict network ﬂows have been around for a while, the machine learning community has only recently taken an interest in general-purpose models for network ﬂows.
[3] predicts edge ﬂows from partial measurements by making a smoothing assumption i e., by minimizing nodal ﬂow divergence.
[4] improves on this approach by adding a trainable regularizer that can incorporate side information.
Both of these approaches are centered on a notion of approximate conservation, i e., that the net inﬂow to each node should be near zero.
Since conservation of mass is a universal constraint on network ﬂows, imposing this conservation law is an important step toward embedding physics into the model.
But the conservation law alone is not enough to uniquely determine ﬂow, which is why both [3] and [4] rely on heuristic regularizers to select the “best” conservation-respecting ﬂow.
In fact, physical networks are often governed by a pair of physical laws: the conservation law, and a constitutive relationship , which speciﬁes the magnitude and direction of each edge ﬂow based on “effort” variables at each incident node (e.g., pressure or voltage).
For example, in DC circuits, currents are conserved according to Kirchoff’s current law, and Ohm’s law is the constitutive relationship that relates current ﬂows to nodal potentials.
The conservation law and the constitutive relationship together deﬁne the unique edge ﬂows (and nodal efforts).
1.1 Contributions
This paper proposes a model for network ﬂows that embeds both the conservation law and existence of a constitutive relationship.
Our model, which we call an Implicit Flow Network (IFN), predicts each edge ﬂow using a trainable nonlinear function of latent nodal variables.
These latent variables are constrained to a manifold wherein the conservation law is satisﬁed.
In addition to introducing
IFN, we offer the following contributions: (i) a contraction algorithm that is able to both evaluate the IFN layer and backpropagate gradients through it, (ii) an explicit upper bound on the number of iterations required by this algorithm, (iii) a rigorous theoretical comparison between IFN and the state-of-the-art ﬂow estimation methods in [3, 4], and (iv) numerical experiments from several
AC power networks and water distribution systems that indicate IFN can signiﬁcantly outperform these baselines on the ﬂow estimation task.
Additionally, because IFN requires a nonlinearity with a constrained slope, we provide (v) a novel “derivative-constrained perceptron”, which is essentially a trainable activation function with upper and lower bounds on its slope.
1.2 Related Work
Network Flow Estimation Flows on graphs are a classical topic in computer science [5], and ﬂow forecasting has long been studied in speciﬁc domains like trafﬁc [6], but interest in the ﬂow estimation task from a machine learning perspective appears to be relatively recent.
Deep learning algorithms have been used to predict trafﬁc ﬂows [7, 8] and power ﬂows [9], but [3] and [4] appear to be the ﬁrst papers to propose methods for domain-agnostic ﬂow prediction, based on the notion of divergence minimization.
Implicit Neural Networks IFN belongs to a growing class of models called implicit neural networks, which do not explicitly state the output of the model; rather, they describe a desired relationship between the model’s inputs and outputs.
In the prevailing implicit framework, the output is deﬁned as a ﬁxed point of a trainable perceptron.
This approach was introduced in [10] as a “deep equilibrium network”.
Subsequent work has developed new frameworks for ensuring the existence of the ﬁxed point and computing it [11, 12, 13, 14, 15].
Other types of implicit neural networks include neural
ODEs [16] and layers that solve convex optimization problems [17] and Nash equilibria [18].
Graph Neural Networks Graph neural networks (GNN) are a diverse family of models for networkrelated learning tasks that incorporate graph structure directly into the model.
GNNs can typically be classiﬁed into three types, in increasing order of generality [19, §5.3]: convolutional models [20, 21], attentional models [22], and message-passing models [23, 24].
Recently, [25] proposed an implicit graph convolutional network.
Analogously, IFN can be interpreted as an implicit message-passing
GNN, with ﬂows serving as messages and latent nodal variables acting as an embedding.
1.3 Preliminaries and Notation
Given a directed graph G =
( V , E ) , the signed incidence matrix B ∈ {− 1 , 0 , 1 } |V|×|E| is the matrix with entries
B i,e =   1 , i is the head of e − 1 , i is the tail of e 0 , else , ∀ i ∈ V and e ∈ E
For an undirected graph, the signed incidence matrix is obtained by assigning an aribtrary orientation to each edge.
For each i ∈ V , let N in ( i ) , N out ( i ) ⊂
V be the in-neighbors and out-neighbors of i .
Given a vector x ∈ R n , we use the notation [ x ] to denote the diagonal matrix diag( x ) ∈ R n × n .
Where such notation would be unclear (e.g., may be confused with brackets to indicate order of operations), we fall back on the diag( · ) notation.
We write x ⊥ to refer to the vector space that is orthogonal to x , i.e., the space { x (cid:48) ∈ R n : x (cid:62) x (cid:48)
= 0 } .
Given a positive deﬁnite diagonal matrix
D ∈ R n × n , we write || x || 2 ,D to represent the weighted
[Table]
Implicit Flow Networks
IFN is inspired by the physics of network systems.
In many physical networks, nodes “communicate” through the exchange of a commodity, like power, water, or force, which can be represented as edge ﬂows.
Flows obey a conservation law: for all i ∈ V ,
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Equation]
) where u ∈ R |V| are nodal inﬂows from outside the network, and f ∈ R |E| are the edge ﬂows.
Furthermore, the ﬂows are related to nodal variables through a constitutive relationship (CR); there is some strictly increasing function h such that, for all ( i, j ) ∈ E ,
[Equation]
(2) where a ∈ R |E| are edge weights and x ∈ R |V| are nodal “efforts” or “potentials.”
For example, in
DC power networks, the CR is Ohm’s law f
( i,j ) =
r − 1 ( i,j ) ( x i − x j ) , where r are resistances and x are voltages.
In lossless AC networks, the CR is the active power ﬂow equation f
( i,j )
= a ( i,j ) sin( x i − x j ) , where the edge weights are a function of line parameters and x are voltage angles [26, §6.4].
In water distribution systems, the CR is the Hazen-Williams formula [27, Sec. 8.15].
Table 1 lists several ﬂow networks, the physical interpretation of the effort variables x , and the ﬂow function h .
We propose IFN as a layer that predicts edge ﬂows based on these two physical laws—conservation and the existence of a CR:
Deﬁnition 2.1 (Implicit Flow Network) .
An implicit ﬂow network (IFN) is a module with the following components:
ﬁxed parameters 0 < d min ≤ d max , 2.
trainable parameters θ ∈ R r for some r , and 3.
a family of differentiable functions h θ : R → R such that d min ≤ h (cid:48) θ ( y ) ≤
d max for all y ∈ R and θ ∈ R r , which we call ﬂow functions .
The module requires each of the following inputs:
a weighted, connected, undirected graph G =
( V , E , a ) with edge weights a ∈ R |E| > 0 , and 2.
a supply / demand vector u ∈ R |V| such that (cid:80) i ∈V u i = 0 .
The module outputs the unique vector f ∈ R |E| for which there exists x ∈ R |V| such that
[Equation]
where B ∈ {− 1 , 0 , 1 } |V|×|E| is the signed incidence matrix of G , and h θ is applied element-wise.
We use the notation FN h,θ ( G, u ) to represent the solution f given inputs G and u , ﬂow functions h , and parameters θ . [Figure] [Figure]
We will prove that IFNs are well-posed in Theorem 2.2.
Note that (3) and (4) are just vectorized statements of the conservation law (1) and the CR (2), so these two physical laws directly deﬁne the output.
The IFN’s only trainable component is its ﬂow function, parameterized by θ .
In practice, we will only make calls to the inverse of the ﬂow function when evaluating and backpropagating through
IFN layers, so it is convenient to learn the inverse ﬂow function directly.
We emphasize that IFNs are layers that can be situated in more complex architectures, with other models upstream estimating the supply / demand vector, edge weights, or even the topology.
For example, in power systems, demand forecasting is a very well-studied problem [28, 29], and one can solve the economic dispatch problem to forecast power generation at each node [30], collectively leading to an estimate of the supply / demand vector.
2.1 Evaluating the Implicit Flow Network
Our approach to evaluating the implicit ﬂow network is adapted from [31] and is illustrated in Figure 1.
Any undirected graph G induces a direct decomposition of the edge ﬂow space R |E| : given the incidence matrix B ∈ {− 1 , 0 , 1 } |V|×|E| , the cycle space ker( B ) and cutset space Img( B (cid:62) ) are orthogonal, and R |E| =
ker( B ) ⊕ Img( B (cid:62) ) .
We refer the reader to [32, §9.4] for a primer on cycle and cutset spaces.
Accordingly, we decompose the vector f = FN
h,θ ( G, u ) as f = f cyc + f cut , where f cyc ∈ ker( B ) and f cut
∈ Img( B (cid:62) ) .
The cutset component is readily determined from (3), since Bf = Bf cut = u implies that f cut
= B † u .
Then we must analyze (4) to solve for f cyc .
Deﬁne a cycle projection matrix P ∈ R m × m as the oblique projection onto ker( B ) parallel to Img([ a ] B (cid:62) ) :
[Equation]
Based on this projection, we deﬁne a map T : ker( B ) → ker( B ) for all f cyc ∈ ker( B ) by
[Equation]
We can show that f cyc is the unique ﬁxed point of T , and that T is a contraction mapping, leading to a simple algorithm to compute this ﬁxed point.
Theorem 2.2 (Properties of T ) .
Consider an implicit ﬂow network with parameters d min , d max , and θ , with ﬂow functions h θ .
Suppose that the inputs G =
( V , E , a ) and u ∈ 1 ⊥ |V| are given, and let
B ∈ {− 1 , 0 , 1 } |V|×|E| be the signed incidence matrix of G .
The following are true:
T is a contraction mapping with respect to || · || 2 , [ a ] − 1 , with Lipschitz constant
[Equation]
the sequence of iterates f ( k +1) cyc
= T ( f ( k ) cyc ) starting from any initial condition f (0)cyc ∈ ker( B ) converges to a unique ﬁxed point f cyc , 3.
the output of the implicit ﬂow network is unique and given by
[Equation]
Consequently, IFN is well-posed.
Theorem 2.2 provides a simple algorithm for computing the IFN output f : pick any f (0)cyc ∈ ker( B ) , repeatedly apply the map T until approximate convergence, then add B † u .
Some care is required when implementing this map.
Since P is a dense matrix with |E| 2 entries, it is undesirable to explicitly construct the cycle space projection matrix for large networks.
Instead, in order to project a vector v ∈ R |E| , we can use the fact that
[Equation]
so the projection is evaluated as Pv = v − [ a ] B (cid:62) w .
Using this method of projection to implement
T , the ﬁxed point iteration to compute FN h,θ ( G, u ) is stated in Algorithm 1.
Algorithm 1: B ← signed incidence matrix of G 2: f cut ← argmin f cut ∈ R m {|| Bf cut − u || 2 } 3: f cyc ← 0 m 4: ∆ f cyc ← ∞ 1 m 5: while || ∆ f cyc || 2 , [ a ] − 1 > (cid:15) do 6: v ← d [Equation]
a
[Equation]
h
[Equation] [ a [Equation]
f
[Equation]
[ a ]
[Equation]
f
[Equation]
(cid:1) 7: w ← argmin
[Equation]
8: ∆ f cyc ← v − [ a ] B (cid:62) w 9: f cyc ← f cyc − ∆ f cyc 10: end while 11: f ← f cyc + f cut 12: return f
Theorem 2.3 (Implicit Flow Networks, Forward Pass) .
Consider an implicit ﬂow network with parameters d min , d max , and θ , with ﬂow functions h θ .
Suppose that the inputs G =
( V , E , a ) and u ∈ 1 ⊥|V| are given, and let B ∈ {− 1 , 0 , 1 } |V|×|E| be the signed incidence matrix of G .
The following are true of Algorithm 1, with a tolerance of (cid:15) > 0 :
for each iteration k = 1 , 2 , . . .
of the loop, let f ( k ) cyc represent the new value of f cyc deﬁned on line 9; and let f (0)cyc = 0 m .
Then
[Equation]
the algorithm converges with at most k ∗ iterations of the while loop, where
[Equation]
and ρ = || [ a ] 12 h − 1 θ ([ a ] − 1 B † u ) || 2 ; and 3.
the algorithm returns f ∈ R |E| , where
[Equation]
If evaluating h − 1 θ is sufﬁciently simple, then the most expensive step in the iteration is solving the ordinary least squares problem on line 7.
Using a general-purpose solver, the complexity of this operation is roughly O ( |V| 3 ) .
But B [ a ] B (cid:62) is a sparse Laplacian matrix, so we can use a specialized
Laplacian solver that reduces the complexity to O ( |E| log k |E| ) for some constant k [33].
The bound on the number of iterations k ∗ can be computed before any forward pass, since evaluating h − 1 θ does not require solving the IFN equations.
But we can further simplify the bound by approximating h − 1 θ (0) = 0 , which is often justiﬁed because physical ﬂow functions generally have a root at the origin.
Using the fact that ( h − 1 θ ) (cid:48) ( y ) ≤ d − 1min y , we can then eliminate the dependence on h − 1 θ :
[Equation]
2.2 Computing the Gradients
In order to train the ﬂow function and any upstream models, it is necessary to backpropagate gradients through the IFN layer.
We can perform this backward pass using implicit differentiation, and it turns out that the gradients of FN h,θ ( G, u ) with respect to the parameters θ , a , and u can also be computed using Algorithm 1, i.e., by writing the gradient as the output of an auxiliary implicit ﬂow network.
Theorem 2.4 (Gradients) .
Consider an implicit ﬂow network with parameters d min , d max , and θ , with ﬂow functions h θ .
Suppose that the inputs G =
( V , E , a ) and u ∈ 1 ⊥ |V| are given, and let
B ∈ {− 1 , 0 , 1 } |V|×|E| be the signed incidence matrix of G .
Let f = FN
h,θ ( G, u ) , and let w be a scalar entry of θ , a , or u .
We can compute the derivatives dfdw as follows.
Deﬁne a vector of ﬂow functions g : R |E| → R |E| by
[Equation]
[Equation]
where D ∈ R |E|×|E| is the diagonal matrix with entries
[Equation]
[Equation]
and v = [ a ] h − 1 θ ([ a ] − 1 f cyc + [ a ] − 1 B † u ) .
Then
[Equation]
12) (We use the notation · in place of θ , since g has no trainable parameters.)
Furthermore, the derivative constraint parameters d min , d max from the original implicit ﬂow network are valid for the new implicit ﬂow network.
In other words, to compute the gradient with respect to a parameter, we perform a single evaluation of the implicit ﬂow network.
In order to compute the derivatives with respect to some parameter or input w , we ﬁrst evaluate the partial derivatives ∂v∂w and the total derivatives dudw .
Then we construct the ﬂow functions g according to (10), and solve an implicit ﬂow network to ﬁnd dfdw according to (12).
It is easy to evaluate dudw , but for convenience, we provide the values of ∂v∂w below:
[Equation]
Comparison with Optimization Models
Both of the state-of-the-art methods for ﬂow estimation, from [3] and [4], use an optimization problem to predict ﬂows.
After a suitable transformation to incorporate external ﬂow injections u , we can state this optimization problem as
[Equation]
where λ > 0 , and q > 0 m is some vector of edge weights.
In [3], q = 1 m , while [4] allows q to be the output of a neural network.
IFN is not explicitly an optimization problem, but it can be cast as one that is similar to (13):
Theorem 3.1 (Optimization Form of IFN) .
Consider an IFN with ﬂow function h θ .
Suppose that the inputs G =
( V , E , a ) and u ∈ 1 ⊥|V| are given, and let B ∈ {− 1 , 0 , 1 } |V|×|E| be the signed incidence matrix of G .
Then the IFN output can be stated as the solution of a convex optimization problem:
[Equation]
Theorem 3.1 can be interpreted as a nonlinear generalization of the Thomson principle from electrical circuits theory [34].
Interestingly, the theorem sets up a direct comparison between IFN and the models in [3] and [4].
If the ﬂow function h θ is the identity map, then (14) can be simpliﬁed as
[Equation]
Ignoring the constraints from labeled ﬂows, we can interpret (13) as using a penalty method to approximate the output of an IFN with a linear ﬂow function.
Thus, we have three distinct differences between IFN and the optimization-based approaches.
First, IFN allows for a nonlinear ﬂow function, while [3] and [4] implicitly assume a linear CR.
Second, IFN imposes ﬂow conservation as a hard constraint rather than an approximate constraint (which is a limitation if u is uncertain).
Finally, IFN does not incorporate ﬂow measurements directly; rather, the model exploits these measurements during training to learn the proper ﬂow function (and train any upstream models for the IFN inputs), making it less sensitive to noise in the labeled ﬂows.
Models for Flow Functions
In order to implement an IFN, it is necessary to parameterize its inverse ﬂow function h − 1 θ .
Since the ﬂow function is essentially a trainable activation function, i.e., a scalar nonlinearity, simple models are likely to be sufﬁcient.
The main difﬁculty with selecting a ﬂow function is that its slope must be bounded by d min ≤ h (cid:48) θ ( y ) ≤
d max for all y ∈ R .
This section proposes a simple scalar nonlinearity that is guaranteed to respect arbitrary upper and lower bounds on its slope.
Deﬁnition 4.1 (Derivative-Constrained Perceptron) .
Let k ∈ Z > 0 be a hidden layer size, let a, b, c ∈
R k be freely trainable parameters (encoded within the parameter vector θ ), and let σ be a nonexpansive activation.
Let p, q ≥ 1 such that p − 1 + q − 1 = 1 , and let ¯ d min ≤ ¯ d max ∈ R .
Then the derivative-constrained perceptron N ( x, θ ) is the 3-layer neural network deﬁned by
[Equation]
Intuitively, (L1) re-scales c so that the perceptron in (L2) is guaranteed to be non-expansive in x . Then (L3) re-centers and re-scales the derivatives of the perceptron from the range [ − 1 , 1] to [ ¯ d min , ¯ d max ] .
Theorem 4.2 (Derivative-Constrained Perceptron) .
Let N( x, θ ) be a derivative-constrained perceptron with ¯ d min ≤ ¯ d max ∈ R .
Then for all parameter values θ , ¯
[Equation]
[Equation]
Note that the values ¯ d min , ¯ d max in Deﬁnition 4.1 and Theorem 4.2 are distinct from the IFN parameters d min , d max .
Since we parameterize the inverse ﬂow function h − 1 θ in IFN, one shoudl set ¯ d min = d − 1max and ¯ d max = d − 1min to implement h − 1 θ with a derivative-constrained perceptron.
Numerical Experiments
We studied the transductive task of predicting unlabeled ﬂows, given that some labeled ﬂows in the same network are known.
If the edges E are partitioned into a labeled set E l and an unlabeled set E u , the task is to predict the missing ﬂows { f e : e ∈ E u } given the labeled ﬂows { f e : e ∈ E l } .
For each network, we randomly selected a fraction of the edges to be labeled edges, and we trained IFN and baselines on the labeled edges.
Then we evaluated the RMSE of the ﬂows predicted for the unlabeled edges E u to compute the testing error.
See Appendix B in the supplementary material for full details.
Code is available at https://github.com/KevinDalySmith/implicit-flow-networks .
5.1 Datasets
AC Power We selected 6 standard power network test cases.
The ﬁrst 4 test cases (IEEE-57, IEEE118, IEEE-145, and IEEE-300) are synthetic transmission system test cases, while the remaining cases ACTIVSg200 and ACTIVSg500 are similar to the Illinois and South Carolina power grids, respectively [35].
Each test case contains the topology and electrical parameters of the power network, as well as baseline demands and power injections at each node.
While branch resistances are typically small, we set them to zero to ensure lossless transmission lines.
We used the MATPOWER toolbox [36] to solve the power ﬂow equations, then recorded the active power ﬂows on each branch ( f ), computed the net active power injections at each node ( u ), and selected relevant electrical parameters as edge attributes (series reactance, tap ratio, and voltage magnitude at the two incident nodes).
Water Distribution We selected 3 sample water distribution networks from the ASCE Task Committee on Research Databases for Water Distribution Systems database [37], representing municipal water distribution systems in Fairﬁeld, CA, Bellingham, WA, and Harrisburg, PA.
Each network contains the topology of the distribution system, as well as the characteristics of pipes and other network elements and nodal demands.
We used the WNTR package [38] to compute the ﬂow rates through each pipe ( f ), net inﬂow rate at each node ( u ), and edge weights associated with each pipe.
5.2 Models and Experiment Details
IFN Architecture In order to use the IFN layer to predict power ﬂows, we created a two-layer model.
The ﬁrst layer estimates positive edge weights a ∈ R |E| according to a e = exp ( L ( z e )) for all e ∈ E , where L is a linear module, and z e is the log-transformed vector of edge attributes.
The second layer is an IFN.
To predict water ﬂows, we used an IFN layer alone, supplying the edge weights from the dataset as input (rather than learning them from other edge attributes).
For both water and power, the IFN layer uses a derivative-constrained perceptron as the inverse ﬂow function ( k = 128 , p = q = 12 ) with a ReLU activation function.
For power, we set d min = 0 .
4 and d max = 2 ; and for water, d min = 0 .
2 and d max = 20 .
Baselines We compared the IFN model against four baselines.
The minimum divergence method ( Div ) from [3] minimizes the nodal divergence ||
Bf || 22 and a regularization term
λ ||
f ||
The bilevel optimization methods from [4] replace the uniform regularizer with a weighted regularizer
||
f || 22 , [ q ] , where q is a vector of weights.
In Bil-MLP and Bil-GCN , q is the output of either a 2-layer MLP or GCN model with edge attributes as inputs (we use 64 nodes in each hidden layer with ReLU activations).
In Bil-True , we specify q as the reciprocal of the coefﬁcient in the linearized CR for
AC power networks, so that Bil-True approximates (15) with a as the ground-truth edge weight.
For water experiments, Bil-True uses the same edge weights as the IFN model.
All of the baselines assume that nodal divergence Bf should be approximately zero, but nodes in power networks inject and withdraw power according to the supply / demand vector u , resulting in nonzero divergence.
Thus, when we evaluate the baselines, we transform the power network into a divergence-free network by introducing a “source node”, adding an edge from the source node to all nodes in V , and treating the entries of u as the ﬂows along each corresponding virtual edge.
5.3 Results
Figure 2 reports the results for the AC power networks, and Figure 3 reports the results for water distribution systems.
In both types of networks, the IFN model signiﬁcantly outperforms the baselines on all of the networks when a small fraction of edges are labeled (less than 80% in power and less than 60% in water).
While the other baselines tend to improve as more labeled edges are made available for training, IFN achieves near-optimal performance with as few as 10% of the edges labeled.
[Figure] [Figure]
Conclusion
In this paper [Figure] we have introduced an implicit model for network ﬂows that incorporates physics through a conservation law and through the existence of a constitutive relationship between ﬂows and nodal variables.
We have demonstrated that a simple architecture using this model can learn to accurately predict active power ﬂows in AC networks and water distribution systems.
Future work may investigate more elaborate architectures using IFN as a layer, wherein the supply / demand vector, edge weights [Figure] or even the graph itself could be predicted from upstream models [Figure]
and the ﬂows themselves used for downstream tasks.
Another interesting extension may be to extend our method to networks with higher-order interactions, i [Figure] e., hypergraphs [39] and simplicial complexes [40, 41].
IFN has some limitations that should also be addressed in future work.
IFN assumes that the graph is undirected, which does not adequately model networks with unidirectional ﬂows (e.g., trafﬁc) or lossy ﬂows (e.g., resistive power grids).
IFN also assumes a CR that depends on the difference between nodal variables.
This form appears frequently in physical systems, but in other network ﬂow models (like Daganzo trafﬁc models [42]), the CR has a more general dependence on the nodal variables.
These limitations may be addressed with extensions of IFN’s contraction algorithm.
A Proofs A.1 Proof of Theorem 2.
To prove statement 1, choose any f cyc ∈ ker( B ) , let y =
[ a ] − 1 f cyc +
[ a ] − 1 B † u for brevity, and observe that
[Equation]
[Equation]
[Equation]
where || [ a ] − 12 P [ a ] 12 || 2 = 1 because [ a ] − 12 P [ a ] 12 is a symmetric and idempotent matrix, i.e., an orthogonal projection.
Then (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)
[Equation]
Hence
[Equation]
Then statement 2 follows from statement 1 and the Banach ﬁxed point theorem.
To prove statement 3, observe that f cyc =
Pf cyc , so f cyc = T ( f cyc )
if and only if
[Equation]
(17) where f = f cyc + B † u .
But ker( P [ a ]) =
Img( B (cid:62) ) , so (17) is equivalent to the existence of x ∈ R n such that
[Equation]
and (18) is equivalent to (4).
A.2 Proof of Theorem 2.
To prove statement 1, let k ≥ 0 and consider iteration k
+ 1 of the loop.
The iteration ﬁrst deﬁnes v = d min [ a ] h − [Equation]
θ (cid:16) [ a ] − 1 f ( k ) cyc + [ a ] [Equation] 1 f cut (cid:17) on line 6.
Then on line 7,
[Equation]
and line 8 deﬁnes
[Equation]
Finally, on line 9,
[Equation]
A simple inductive argument shows that f ( k ) cyc ∈ ker( B ) .
The base case f (0)cyc = 0 m is trivial, for all k (cid:48) ≥ 0 , line 9 ensures that f ( k (cid:48) +1) cyc ∈ ker( B ) so long as f ( k (cid:48) ) cyc ∈ ker
( B ) .
Hence f ( k ) cyc =
Pf ( k ) cyc , and we conclude that
[Equation]
= T ( f ( k ) cyc )
To prove statement 2, recall from Theorem 2.2 that Lip( T ) ≤ 1 − d − 1max d min , which (together with statement 1) implies that, for all k ≥ 0 ,
[Equation]
The algorithm terminates after iteration k if and only if || f ( k ) cyc − f ( k − 1)
cyc
|| 2 , [ a ] − 12 ≤ (cid:15) , so the algorithm will have terminated after k ∗ iterations if
[Equation]
which is equivalent to
[Equation]
Finally
[Equation]
to prove statement 3, note that the algorithm terminates after iteration k as soon as
[Equation]
If f cyc is the true ﬁxed point of T , then using a general property of contraction mappings,
[Equation]
Therefore, the vector f returned by the algorithm satisﬁes
[Equation]
A.3 Proof of Theorem 2.
Let v =
[ a ] h − 1 θ ([ a ] − 1 f cyc + [ a ] − 1 B † u ) .
From Theorem 2.2, we can write f = f cyc + B † u , where f cyc is the unique ﬁxed point of T .
Therefore dfdw =
df cyc dw + B † dudw , so the remainder of the proof is to show that df cyc dw =
FN g, · ( G, 0 n ) .
Since f cyc = T ( f cyc ) , and Pf cyc = f cyc , we have
[Equation]
so an equivalent characterization of f cyc is the unique solution to the equations Bf cyc
[Equation]
0 n
Pv = 0 m
Since
[Equation]
then differentiating and factoring out [ a ] , we obtain
[Equation]
Since ker( P [ a ])
=
Img( B (cid:62) ) , there exists x ∈ R n such that
[Equation]
which we can re-write as
[Equation]
Hence df cyc dw is the solution to
[Equation]
[Equation]
which is identical to (3)–(4) with df cyc dw in place of f , 0 n in place of u , and g in place of h θ .
Furthermore, g respects the same d min , d max derivative constraints as h θ , since for each e ∈ E ,
[Equation]
It follows that df cyc dw is the output of the implicit ﬂow network with ﬂow functions g and parameters d min , d max , evaluated on the original graph G and nodal demands 0 n .
A.4 Proof of Theorem
Since h − 1 θ is increasing, the optimization problem in (14) has a convex cost function with linear constraints, so the KKT conditions are necessary and sufﬁcient.
Letting x ∈ R
m be a vector of
Lagrange multipliers, the Lagrangian is
[Equation]
leading to the stationarity condition
[Equation]
which is equivalent to (4).
Additionally, the primal constraint Bf = u is equivalent to (3), so the minimizer of the optimization problem is identical to the output of the IFN.
A.5 Proof of Theorem 4.
Due to (L3), it is clear that the derivative bounds (16) hold if
and only if
[Equation]
For all x, x (cid:48) ∈ R , by Hölder’s inequality,
[Equation]
Since σ is non-expansive, its Lipschitz constant with respect to the q -norm is
[Equation]
and thus
[Equation]
[Table]
Furthermore, by (L1),
[Equation]
so that
[Equation]
[Table]
[Equation]
for all x, x (cid:48) .
Hence (21) is satisﬁed.
B Experiment Details B.1 AC Power Datasets
We created datasets from 6 AC power network test cases.
Each dataset that we created represents a snapshot of an AC power network in its steady state, consisting of four components: the network topology (as an oriented, undirected graph), four attributes on each edge (voltage magnitude at the two incident nodes, series reactance, and tap ratio), the net power injection at each node, and the active power ﬂow through each branch.
Original Data We generated our datasets using MATPOWER, an open-source toolbox for power system simulation in MATLAB [36].
The toolbox includes many standard test cases, which contain a network topology and tables of electrical and economic parameters for each bus (node), branch (edge), and generator.
We selected 6 test cases, listed in Table 2.
The raw data ﬁles for these test cases are available from the MATPOWER source 3 , and details on the test case ﬁle format are contained in
Appendix B of the user manual 4 .
Data Generation After loading each test case into MATPOWER, we performed the following two modiﬁcations of the network parameters:
We set branch resistances (column 3 in the branch data table) to zero, so that transmission lines in the system are lossless.
This step was necessary because IFN is limited to undirected graphs, while lossy lines are more appropriately modeled with a pair of directed edges, since the power injected at one endpoint does not equal the power withdrawn from the other endpoint.
Fortunately, branch resistances are typically small before this modiﬁcation.
We replaced any negative series reactances (column 4 in the branch data table) with a positive value, chosen as the median of the positive series reactances in the same network.
We performed this modiﬁcation because negative series reactances results in decreasing constitutive relationships on the corresponding edges, whereas IFN assumes that the constitutive relationship is increasing.
This modiﬁcation only affected two networks: IEEE-145, in which 24 (4.2%) of the branches were assigned a series reactance of 0.2306; and IEEE-300, in which 1 (0.1%) of the branches was assigned a series reactance of 0.059. We then computed the resulting
power ﬂows using the runpf function and recorded the results
Test Case |V| |E|
Fairﬁeld
[Table]
Pre-Processing Finally, we converted the results from the MATPOWER simulation into a PyTorch
Geometric data object, with the following attributes:
• edge_index , the edge index tensor, containing the topology from the test case.
• x , a tensor of net active power injections at each node, which has the property that 1 (cid:62) n x = 0 .
(This tensor is identical to the supply / demand vector u in the paper.)
edge_attr , a tensor of four relevant attributes for each edge: the voltage magnitudes at the two incident nodes, the series reactance, and the tap ratio.
• f_true , the tensor of active power ﬂows on each edge simulated by MATPOWER.
The net active power injections at each node are computed according to
[Equation]
where PG i is active power generated at i , PD i is active power demanded, GS i is shunt conductance, and VM i is the voltage magnitude.
B.2 Water Distribution Dataset
We created 3 datasets representing snapshots of municipal water distribution networks in their steady state, consisting of four components: the network topology (as an oriented, undirected graph), weights for each edge, the net inﬂow rates at each node, and the ﬂow rate through each pipe.
Original Data Each of the datasets is based on a network from the ASCE Task Committee on
Research Databases for Water Distribution Systems database [37].
Networks in this database contain a distribution network topology and tables of hydraulic parameters and operating characteristics for each node, pipe (edge), pump, reservoir, and storage tank in the network.
We selected 3 networks, listed in Table 3 and plotted in Figure 4.
The raw data ﬁles are available online 5 .
Data Generation and Preprocessing We loaded each network INP ﬁle into WNTR and ran the
WNTR simulator with a hydraulic accuracy of 10 − 8 .
We then converted the results into a PyTorch
Geometric data object, with the following attributes:
• edge_index , the edge index tensor, containing the topology from the test case
• x , a tensor of net inﬂows at each node, which has the property that 1 (cid:62) n x = 0 .
edge_attr , a tensor of three relevant attributes for each edge: the pipe length, pipe diameter, and pipe roughness coefﬁcient.
• f_true , the tensor of ﬂow rates through each pipe simulated by WNTR.
Edge weights are computed according to the formula
[Equation]
22) where C e is the roughness coefﬁcient (unitless), D e is the diameter (meters), and L e is the pipe length (meters) 6 .
B.3 Details on IFN
Our IFN implementation uses Algorithm 1 to compute the layer’s forward pass.
We set the maximum number of iterations in this algorithm to 100, with a tolerance of (cid:15) =
10 − 2 for power and (cid:15) = 10 −for water.
With the release of PyTorch 1.11.0, the torch.linalg.lstsq method 7 now supports automatic differentiation, allowing PyTorch to automatically backpropagate through the Algorithmiterations, instead of using Theorem 2.4.
We found that Algorithm 1 terminated with a small enough number of iterations that automatic differentiation was faster, so we opted to use this rather than the method from Theorem 2.4.
We trained the IFN models to minimize the RMSE loss function by minimizing the RMSE loss function
[Equation]
B.4 Details on Baselines
We implemented all of the baselines by adapting Silva’s code 8 from [4], refactoring some utility functions to decrease runtime.
Following [4], we perform the following two data normalization steps:
negative ﬂows are converted into positive ﬂows by ﬂipping the orientation of the corresponding edges and replacing the entries of f_true with their absolute value, and 2.
ﬂows are proportionally normalized to the range [0 , 1] within each network.
After training with the normalized ﬂows and computing the missing ﬂow predictions, the predictions are denormalized before computing the testing RMSE.
Div
The minimizing divergence baseline from [3] has a single hyperparameter, λ , from the regularization term λ
2 ||
f || 22 in the loss function.
We set λ = 0 .
1 for all networks and fractions of labeled edges by hand-tuning the parameter to the proper order of magnitude.
Bilevel Baselines
All three of the bilevel baselines (Bil-MLP, Bil-GCN, and Bil-True) have several hyperparameters related to the bilevel optimization algorithm.
For most of these parameters, we use the same settings as [4]: the number of iterations for the inner optimization problem is 300 during training and 3000 during evaluation, and the number of k-fold cross validation folds is 10; however, we increased the number of iterations of the outer optimization problem from 10 to 100, with an early stopping interval of 10, to ensure that the outer optimization problem was given sufﬁcient time to converge.
As with [4], we used a 2-layer MLP and GCN in Bil-MLP and Bil-GCN, respectively, but we increased the size of the hidden layer to 64.
Bil-True Like IFN, the baselines Bil-MLP and Bil-GCN train a model to predict edge weights from side information (if we interpret Q as a diagonal matrix of edge weights).
We devised Bil-True as a third baseline to use the “ground-truth edge weights” instead of training a model.
For water experiments, these ground-truth edge weights are given by (22).
For the power experiments, we compute these edges weights from the AC active power ﬂow equation: in a lossless AC power grid, active power ﬂows f ij on each edge { i, j } ∈ E are given by
[Equation]
(23) where v i , v j are the voltage magnitudes on incident nodes, x ij is the series reactance, τ ij is the tap ratio, and θ i , θ j are the incident voltage angles.
Since (23) is the constitutive relationship for AC power networks, examining its linear approximation in light of (15) suggests using x ij τ ij /v i v j as the regularizer weight on f ij .
B.5 Details on Training
We trained all models in a Google Colab notebook, using the Adam optimizer.
We used an initial learning rate of 0 . 01 , which we found to have good performance for all of the models.
Training was terminated when the training loss had not decreased for 10 epochs.