Physics-Informed Implicit Representations of Equilibrium Network Flows
Authors: Kevin Daly Smith
Francesco Seccamonte
Ananthram Swami
Francesco Bullo
Venue: NeurIPS 2022 Accept (2022)
Keywords: network flow estimation
implicit neural networks
contraction theory

Abstract: Flow networks are ubiquitous in natural and engineered systems, and in order to understand and manage these networks, one must quantify the flow of commodities across their edges. This paper considers the estimation problem of predicting unlabeled edge flows from nodal supply and demand. We propose an implicit neural network layer that incorporates two fundamental physical laws: conservation of mass, and the existence of a constitutive relationship between edge flows and nodal states (e.g., Ohm's law). Computing the edge flows from these two laws is a nonlinear inverse problem, which our layer solves efficiently with a specialized contraction mapping. Using implicit differentiation to compute the solution's gradients, our model is able to learn the constitutive relationship within a semi-supervised framework. We demonstrate that our approach can accurately predict edge flows in several experiments on AC power networks and water distribution systems.

Physics-Informed Implicit Representations of Equilibrium Network Flows
Kevin D. Smmth' Francesco Seccamonte'
Ananthram Swamit
Francesco Bullo"
Abstract
Flow networks are ubiquitous in natural and engineered systems, and in order to understand and manage these networks, one must quantify the flow of commodities across their edges. This paper considers the estimation problem of predicting unlabeled edge flows from nodal supply and demand. We propose an implicit neural network layer that incorporates two fundamental physical laws: conservation of mass, and the existence of constitutive relationship between edge flows and nodal states (e.g.. Ohm's law). Computing the edge flows from these two laws is nonlinear inverse problem, which our layer solves efficiently with a specialized contraction mapping. Using implicit differentiation to compute the solution's 's gradients, our model able to learn the constitutive relationship within semi- supervised framework. We demonstrate that our approach can accurately predict edge flows in AC power networks and water distribution systems.
Introduction
Network flows are a fundamental aspect of modern society, from traffic and communication networks power and water distribution systems. Many critical infrastructures are well-modeled as graphs, with edges that transport vital commodities [1]. Beyond infrastructure, network flows are also central models in epidemiology, ecology, medicine, and chemical networks. Their dynamics have been well-studied in compartmental systems theory [2]. Given the prevalence of flow networks in natural and engineered systems, predicting flows these networks is an important learning task that may facilitate monitoring, control, optimization, and protection of these networks.
While domain-specific tools predict network flows have been around for while, the machine learning community has only recently taken an interest in general-purpose models for network flows. [3] predicts edge flows from partial measurements by making smoothing assumption, i.c., by minimizing nodal flow divergence. [4] improves on this approach by adding trainable regularizer that can incorporate side information. Both of these approaches are centered on notion of approximate conservation, i.e., that the net inflow to each node should be near zero. Since conservation of mass is universal constraint on network flows, imposing this conservation law an important step toward embedding physics into the model.
But the conservation law alone is not enough uniquely determine flow, which why both [3] and [4] rely on heuristic regularizers to select the "best" conservatioonrespeetie networks are often governed by a pair of physical laws: the conservation law, and constitutive relationship, which specifies the magnitude and direction of each edge flow based on "effort" variables each incident node (c.g.. pressure or voltage). For example, in DC circuits, currents are conserved according to Kirchoff's current law, and Ohm's 's law is the constitutive relationship that relates current flows nodal potentials. The conservation law and the constitutive relationship together define the unique edge flows (and nodal efforts).
flow. In fact, physical
"Center for Control, Dynamical Systems, and Computation, University of Califoria, Santa Barbara, CA 93106 USA, (kevinsmith, fseccamonte, bullo]@ucsb. U.S. Army Research Laboratory, Adelphi, MD 20783 USA, ananthram. swami. civdarmy.ni mil
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

1.1 Contributions
This paper proposes model for network flows that embeds both the conservation law and existence of a constitutive relationship. Our model, which we call an Implicit Flow Network (IFN), predicts each edge flow using trainable nonlinear function of latent nodal variables. These latent variables are constrained to a manifold wherein the conservation law is satisfied. In addition to introducing IFN, we offer the following contributions: (i) contraction algorithm that is able to both evaluate the IFN layer and backpropagate gradients through it, (ii) an explicit upper bound on the number of iterations required by this algorithm, (iii) rigorous theoretical comparison between IFN and the state-of-ther oftt----tt flow estimation methods in [3, 4], and (iv) numerical experiments from several AC power networks and water distribution systems that indicate IFN can significantly outperform these baselines on the flow estimation task. Additionally, because IFN requires a nonlinearity with a constrained slope, we provide (v) novel dlerivative-consrranne perceptron", which is essentially trainable activation function with upper and lower bounds on its slope.
1.2 Related Work
Network Flow Estimation Flows on graphs classical topic in computer science [5], and flow forecasting has long been studied in specific domains like traffic [6], but interest the flow estimation task from machine learning perspective appears be relatively recent. Deep learning algorithms have been used to predict traffic flows [7, 8] and power flows [9], but [3] and [4] appear to be the first papers to propose methods for domain-agnostic flow prediction, based on the notion of divergence minimization.
Implicit Neural Networks IFN belongs to growing class of models called implicit neural net- works, which do not explicitly state the output of the model; rather, they describe desired relationship between the model's inputs and outputs. In the prevailing implicit framework, the output is defined as fixed point of trainable perceptron. This approach was introduced in [10] as "deep equilibrium network". Subsequent work has developed new frameworks for ensuring the existence of the fixed point and computing it [I1, 12, 13, 14, 15]. Other types of implicit neural networks include neural ODEs [16] and layers that solve convex optimization problems [17] and Nash equilibria [18].
Graph Neural Networks Graph neural networks (GNN) are diverse family of models for network- related learning tasks that incorporate graph structure directly into the model. GNNs can typically be classified into three types, in increasing order of generality [19, $5.3]: convolutional models [20, 211, attentional models [22], and message-passing models [23, 24]. Recently, [25] proposed an implicit graph convolutional network. Analogously, IFN can be interpreted an implicit message-passine GNN, with flows serving as messages and latent nodal variables acting as an embedding.
1.3 Preliminaries and Notation
Given directed graph G (V,E), the signed incidence matrix B (-1,,0 is the matrix
with entries
1, i is the head of
Bie
-, is the tail of e Vi E V and E E
else
For an undirected graph, the signed incidence matrix is obtained by assigning an aribtrary orientation to each edge. For each i E V. let Nin (i), (t) V be the in-neighbors and out-neighbors of i.
Given a vector I E RR, we use the notation [x]] to denote the diagonal matrix diag(z) E B=** Where such notation would be unclear (e.g., may be confused with brackets to indicate order of operations), we fall back on the diag(-) notation. We write x to refer to the vector space that is orthogonal to x, i.e., the space (x' E R x TTx x = 0}. Given a positive definite diagonal matrix D E Rax*, we write || to represent the weighted 2-norm ||D2 Given any matrix M, M; is the ith column vector of M, and M is the transpose of the jth row vector.

---

Flow
Nodal Variable
h(y)
DC Current
Voltage
y
DC Power
Voltage
y²
AC Power (lossless)
Voltage Angle
sin(y)
Water Flow Rate
Hydraulic Head
sign (v) y 1/..54
Mechanical Force Networks
Position
Table 1: Examples of physical flow networks and their constitutive relationships.
Implicit Flow Networks
IFN is inspired by the physics of network systems. In many physical networks, nodes "communicate" through the exchange of a commodity, like power, water, force, which can be represented as edge flows. Flows obey conservation law: for all E V.
inflow
outflow
0
+
Σ
(ENim((
('ENou(()
where E R M Furthermore, the flows are related nodal variables through constitutive relationship (CR); there is some strictly increasing function h such that, for all (i,j) E E,
are nodal inflows from outside the network, and f E R are the edge flows.
f(a) h(z =2)).
(2)
where a E I 8l are edge weights and E IR MI are nodal "efforts" or "potentials." For example, in DC power networks, the CR is Ohm's ' law ((i))) و (2; xz). where are resistances and x are voltages. In lossless AC networks, the CR is the active power flow equation f(i,)) a(i.j) sin(r;- where the edge weights are a function of line parameters and r are voltage angles [26, $6.4). In water distribution systems, the CR is the -illlimms formula [27, Sec. 8. 8.15]. Table I lists several flow networks, the physical interpretation of the effort variables x, and the flow function
We propose IFN layer that predicts edge flows based on these two physical conseronstvvtton and the existence of CR:
Definition 2.1 (Implicit Flow Network), An implicit flow network (IFN) is module with the following components:
fixed parameters 0 < dmii S d
2. trainable parameters / IR' for some r, and
3. a family of differentiable functions he R + R such that dmin I kg(y) S dmax for all
y R and 0 IR", which call flow functions.
The module requires each of the following inputs:
1. a weighted, connected, undirected graph G EV,E,) a) with edge weights a RS0 and
lEl
2. supply/ demand vector u E R such that Σiev U:
The module outputs the unique vector f E R for which there exists R B such that
B
(3)
[a]ha(B (Tx))
where E (-1,0, 111XX1E is the signed incidence matrix of G, and ha is applied element-wise. We use the notation PN,,oGG, u) to represent the solution given inputs G and u, flow functions and parameters 0.
3

---

Inputs (From data or upstream models)
Implicit Flow Nenwork Layer
Cutset Component eun B'u
u (supply/ demand)
f (flows)
B (incidence matrix)
Cycle Component
(edge weights)
Saye T(Jey))
Inverse Flow Fne.
Figure 1: Diagram of the IFN. Inputs are the supply / demand vector u, incidence matrix B, and edges weights which are either known or output from upstream models. The IFN layer separately computes the cutset component and cycle component of the flows, with trainable model for the inverse of the flow function in the CR. These components are summed and output as the flow prediction, for downstream use.
We will prove that IFNs are well-posed in Theorem 2.2. Note that (3) and (4) are just vectorized statements of the conservation law (1) and the CR (2), so these two physical laws directly define the output. The IFN's only trainable component is its flow function, parameterized by 0. In practice, we will only make calls to the inverse of the flow function when evaluating and backpropagating through IFN layers, so it is convenient to learn the inverse flow function directly.
We emphasize that IFNs are layers that can be situated in more complex architectures, with other models upstream estimating the supply demand vector, edge weights, or even the topology. For example, in power systems, demand forecasting is very well-studied problem [28, 29], and one can solve the economic dispatch problem to forecast power generation at each node [30], collectively leading to estimate of the supply demand vector.
2.1 Evaluating the Implicit Flow Network
Our approach to evaluating the implicit flow network adapted from [31] and is illustrated in Figure Any undirected graph G induces direct decomposition of the edge flow space RI given the incidence matrix B E {-1, 1 x(e1, the cycle space ker(B) and cutset space Img( (B) are orthogonal, and | = ker B) @ Im)(BT). We refer the reader to [32, 89.4] for a primer on cycle and cutset spaces. Accordingly, we decompose the vector = FN,,, (, as f = Saye feut. where faye ker(B) and feut E lm The cutset component is readily determined from (3), since Bʃ B implies that feut Biu. Then we must analyze (4) to solve for Saye- Define cycle projection matrix P E mm as the oblique projection onto ker (B) parallel to Img([a]
p= Im [a] B (B(a)BT)* B
(5)
Based on this projection, we define a map T ker(B) - ker(B) for all Seye E ker(B) by T(feye)= = P (feyc - dmin [a] h2 ([a] Soye + Ba]- We can show that feye is the unique fixed point of T, and that T is contraction mapping, leading to a simple algorithm to compute this fixed point.
(6)
Theorem 2.2 (Properties of T). Consider an implicit flow network with parameters dmin, dmax- and EV,E,a) and 1My are given, and let be the signed incidence matrix of G. The following are true:
with flow functions ha. Suppose that the imputs
B E 1-1,0, 13 [I
1. T is contraction mapping with respect to || [12,,0) 1, with Lipschitz constant
dmin
Lip(T) < dmax

---

2. the sequence of iterates p(k+1) converges to a unique fixed point feye.
T(feee) starling from any initial condition Soye ker (B)
3. the output of the implicit flow network is unique and given by
FN u) = Seyc Blu
Consequently, IFN is well-posed.
Theorem 2.2 provides simple algorithm for computing the IFN output f: pick any foye ker ker(B). repeatedly apply the map T until approximate convergence, then add Biu. Some care is required when implementing this map. Since is a dense matrix with |&)² entries, it undesirable to explicitly construct the cycle space projection matrix for large networks. Instead, in order to project vector IRe fl,
we can use the fact that
(B(a]B D Be argmin [1] Bell2) wER"
T w. Using this method of projection implement so the projection is evaluated as Pv [a]
T, the fixed point iteration to compute FN e u) is stated in Algorithm I.
Algorithm Evaluating the implicit flow network.
1: B t signed incidence matrix of G
2: feut f- argmin, f.u ER Baut - =u[2]
3: Seye f- Om
4: Aayc f- 201m
5: while ||A > do
6: v t+ + ([a]~ feye + [a] feut)
7:
w - argmin [|a B w B
8:
ASere + v 1o-aaaa B w
9: Seye Seye Alyy
10: end while
11: f + Saye + Seut
12: return f
Theorem 2.3 (Implicit Flow Networks, Forward Pass). Consider an implicit flow network with parameters dmin dmaxx and O, with flow functions hg- Suppose that the imputs G (V,E,o) and itv are given, and let B {-1,0, 1,, [iiIEE be the signed incidence matrix of G. The following are true of Algorithm 1, with a tolerance of
1. for each iteration is = 1,2,... of the loop, let ((0) represent the new value of feye defined line 9; and let (00 Om. Then
s(k++1) T(f()), Wk: 2 0:
2. the algorithm converges with at most k* iterations of the while loop, where
log (dmLapp1
h- = 1+
(8)
log 1
dani danax
and [|[a] - Btu)lla and
3. the algorithm returns f E |l where
(
dmni
f FN (G, w)lla,1)1 S
dani
evaluating h is sufficiently simple, then the most expensive step in the iteration is solving the ordinary least squares problem line 7. Using general-purpose solver, the complexity of this
5

---

In other words, to compute the gradient with respect to a parameter, we perform a single evaluation of the implicit flow network. In order to compute the derivatives with respect to some parameter or input to, we first evaluate the partial derivatives de The and the total derivatives dhe operation is roughly O(D1S). But B"a] is a sparse Laplacian matrix, so we can use specialized Laplacian solver that reduces the complexity to O(EE) log" (E|) for some constant & [33].
The bound the number of iterations k* can be computed before any forward pass, since evaluating h does not require solving the IFN equations. But we can further simplify the bound by approxi- mating h (0) O, which is often justified because physical flow functions generally have root at the origin. Using the fact that (hg ((y) S dmiad. we can then eliminate the dependence on h
[
dmax (loge - log (IIa) BBuull))
dmin
h- < 1 + log
2.2 Computing the Gradients
In order train the flow function and any upstream models, is necessary to backpropagate gradients through the IFN layer. We can perform this backward pass using implicit differentiation, and turns out that the gradients of oo G, u) with respect to the parameters 0, and also be computed using Algorithm 1, i.e., by writing the gradient as the output of an auxiliary implicit flow network. Theorem 2.4 (Gradients). Consider implicit flow network with parameters dmin dmax and O, with flow functions hg- Suppose that the inputs = (V, a,a) and u E 1 1iy are given, and let B E (-1,0, 1]1 be the signed incidence matrix of Let f = FN (G, u), and let w be of scalar entry of 0, a, or u. We can compute the derivatives du as follows.
Define vector of flow functions g .RlEE -+ [1 by
g(n) = D-1 [a]
Ov dw
Vn E E
(10)
where D E TR| [R|[[ is the diagonal matrix with entries
Dec dhg (ye) dye
E
(11)
and v [a]haa ([a))"((a]--// Joye + [a]- Blu)). Then
du dw (We use the notation in place of 0, since has trainable parameters.) Furthermore, the derivative constraint parameters dmiu, dmax from the original implicit flow network are valid for the new implicit flow network.
df = FNg.
(12)
dw
du Then we construct
the flow functions g according (10), and solve implicit flow network to find df according to du
(12). is easy to evaluate dw- but for convenience, we provide the values of
8w below:
dv
dhg ([a) [(a]-
Ov diag (h ([a]==)) - -a)-
Ou
[a]
(DB)),
00,
d0,
Da.
Du.
Comparison with Optimization Models
Both of the -tate-of-the-ar methods for flow estimation, from [3] and [4], use an optimization problem to predict flows. After suitable transformation to incorporate externl flow injections u, we can state this optimization problem
f argmin SeRiEI
[1 + XB ull s.t. ff. V labeled edges E EE}
(13)
where À > O. and y O. is some vector of edge weights. In [3]. 1m while [4] allows to be the output of neural network. IFN is not explicitly an optimization problem, but can be cast as one that is similar to (13):

---

Theorem 3.1 (Optimization Form of IFN). Consider an IFN with flow function hg. Suppose that the inputs G = (V,&, a) and Liv are given, and let B E 1. El be the signed incidence matrix of G. Then the IFN output can be stated as the solution of a convex optimization problem:
ΣJ to hg (a,'z) dz s.t. Bf
FN (N, u) argmin fER
(14)
BE
Theorem 3.1 can be interpreted as nonlinear generalization of the Thomson principle from electrical circuits theory [34]. Interestingly, the theorem sets up a direct comparison between IFN and the models in [3] and [4]. If the flow function he is the identity map, then (14) can be simplified
FN u) arggin
Σ [11/112 s.t. Bf
(15)
SER"
Ignoring the constraints from labeled flows, we can interpret (13) as using a penalty method to approximate the output of an IFN with linear flow function. Thus, we have three distinct differences between IFN and the optimization based approaches. First, IFN allows for a nonlinear flow function, while [3] and [4] implicitly assume linear CR. Second, IFN imposes flow conservation as a hard constraint rather than an approximate constraint (which a limitation if u is uncertain). Finally, IFN does not incorporate flow measurements directly; rather, the model exploits these measurements during training to learn the proper flow function (and train any upstream models for the IFN inputs), making it less sensitive noise in the labeled flows.
4 Models for Flow Functions
In order to implement an IFN, it is necessary to parameterize its inverse flow function hg __ Since the flow function is essentially trainable activation function, i.e., scalar nonlinearity, simple models are likely to be sufficient. The main difficulty with selecting a flow function that its slope must be bounded by dmin h2 (v) dmax for all R. This section proposes simple scalar nonlinearity that is guaranteed to respect arbitrary upper and lower bounds on its slope.
Definition 4.1 ceeivaaive-Coostraainedd Perceptron). Let k Z be hidden layer size, let a, b, R be freely trainable parameters (encoded within the parameter vector 0), and let a be a non- expansive activation. Let q 2 such that p=1 + q- = 1, and let dmin S dmax E IR. Then the derivative perceptron N(2, O) the 3-layer neural network defined by
a(0) = (1 (1- (1cc1//|a|1e -1+ |||||||aal4
(L1)
No (Jr, 0) - (0))
(L22
O)x,0) [mmxx dmin No(a, 0)
(dmnxx + dmin
x
(L.3)
2
2
Intuitively, (LI) re-scales e so that the perceptron in (L2) is guaranteed be non-expansive in Then (L3) re-centers and re-scales the derivatives of the perceptron from the range [-1, to [dmin dmax].
Theorem 4.2 (Derivative-CCl Connstrriee Perceptron). Let (xx, 0) be a derivaative-consrranne percep- tron with dmin I dmax E IR. Then for all parameter values 0,
dunin S dx N(x.0) S danxx
Vx E
(16)
Note that the values duiin Juax in Definition 4.1 and Theorem 4.2 are distinct from the IFN parameters dming dmax- Since we parameterize the inverse flow function h, in IFN, one shoudl set dmii and d d=1 to implement hī with derivative--oostrainne perceptron.
d dmax
5 Numerical Experiments
We studied the transductive task of predicting unlabeled flows, given that some labeled flows in the same network are known. If the edges & are partitioned into labeled set E, and an unlabeled set E.,.
7

---

the task is to predict the missing flows {fe e E given the labeled flows {fe e Et}. For each network, we randomly selected fraction of the edges to be labeled edges, and we trained IFN and baselines on the labeled edges. Then we evaluated the RMSE of the flows predicted for the unlabeled edges E to compute the testing error. See Appendix the supplementary material for full details. Code is available at https:/ httbbb...tt....tg// com/KevinDaly th/ htplicippliiiiiim cit- flow- networks.
5.1 Datasets
AC Power We selected standard power network test cases. The first 4 test cases (IEEE-57, IEEE- 1 118, IEEE-145, and IEEE-300) are synthetic transmission system test cases, while the remaining cases ACTIVSg200 and ACTIVSg500 are similar the Illinois and South Carolina power grids, respectively [35]. Each test case contains the topology and electrical parameters of the power network, as well as baseline demands and power injections each node. While branch resistances are typically small, we set them to zero to ensure lossless transmission lines. We used the MATPOWER toolbox [36] to solve the power flow equations, then recorded the active power flows on each branch (f), computed the net active power injections at each node (u), and selected relevant electrical parameters as edge attributes (series reactance, tap ratio, and voltage magnitude at the two incident nodes).
Water Distribution We selected sample water distribution networks from the ASCE Task Com- mittee on Research Databases for Water Distribution Systems database [37], representing municipal water distribution systems in Fairfield, CA, Bellingham, WA, and Harrisburg, PA. Each network contains the topology of the distribution system, as well as the characteristics of pipes and other network elements and nodal demands. We used the WNTR package [38] to compute the flow rates through each pipe (f), net inflow rate each node (u), and edge weights associated with each pipe.
5.2 Models and Experiment Details
IFN Architecture In order to use the IFN layer to predict power flows, we created a two-layer model. The first layer estimates positive edge weights a E IR 81 according to de = exp (Lx(z)) for all E &, where L is a linear module, and se is the log- -rg-transformed vector of edge attributes. The second layer is an IFN. To predict water flows, we used an IFN layer alone, supplying the edge weights from the dataset input (rather than learning them from other edge attributes). For both water and power, the IFN layer uses a derivative-constraine
perceptron as the inverse flow function (k = 128, p = 9 = 2 with ReLU activation function. For power, we dmiu 0.4 and dmax 2: and for water, dmai = 0.2 and dmax 20.
Baselines We compared the IFN model against four baselines. The minimum divergence method (Div) from [3] minimizes the nodal divergence [B and a regularization term \|ls The bilevel optimization methods from [4] replace the uniform regularizer with a weighted regularizer ||/112. where q is vector of weights. In Bil-MLP and Bil-GCN, q is the output of either 2-layer MLP or GCN model with edge attributes as inputs (we use 64 nodes in each hidden layer with ReLU activations). In Bil-True, we specify q as the reciprocal of the coefficient in the linearized CR for AC power networks, so that Bil-True approximates (15) with a as the ground-truth edge weight. For water experiments, Bil-True uses the same edge weights the IFN model.
All of the baselines assume that nodal divergence Bf should be approximately zero, but nodes in power networks inject and withdraw power according to the supply / demand vector resulting in nonzero divergence. Thus, when we evaluate the baselines, we transform the power network into a divergence free network by introducing "source node", adding an edge from the source node to all nodes in V, and treating the entries of u as the flows along each corresponding virtual edge.
5.3 Results
Figure 2 reports the results for the AC power networks, and Figure 3 reports the results for water distribution systems. both types of networks, the IFN model significantly outperforms the baselines on all of the networks when small fraction of edges are labeled (less than 80% in power and less than 60% in water). While the other baselines tend to improve as more labeled edges are made available for training, IFN achieves near-optimal performance with few 10% of the edges labeled.
perffrmannce
8

---

IEEE-57
IEEE-118
IEEB-145
10a
10-2
IEBB-300
ACTIVag200
ACTIVSg500
10a
10a
10
0.3
0.5
0.5
0.5
0.7
Fraction of Labeled Edges
Div
BU-MLP
Bi-GON
BR-True
JFN
Figure 2: Results for missing flow prediction in AC power networks. Reported values are the RMSE (in units of MW) on the testing set, averaged across 10 trials. Note the vertical axis is in a log scale.
Fairfield
Bellingham
Harrisburg
10-3
10-9
0.3
0.5
0.3
0.5
0.7
0.1
0.7
0.9
Fraction of Labeled Edges
MLP BILGON Brue
Figure 3: Results for missing flow prediction in water distribution systems. Reported values are the RMSE (in units of m² /s) on the testing set, averaged across 10 trials.
Conclusion
In this paper, we have introduced an implicit model for network flows that incorporates physics through conservation law and through the existence of constitutive relationship between flows and nodal variables. We have demonstrated that a simple architecture using this model can learn to accurately predict active power flows in AC networks and water distribution systems. Future work may investigate more elaborate architectures using IFN as a layer, wherein the supply demand vector, edge weights, even the graph itself could be predicted from upstream models, and the flows themselves used for downstream tasks. Another interesting extension may be to extend our method to networks with higher-order interactions, i.c., hypergraphs [39] and simplicial complexes [40, 41].
IFN has some limitations that should also be addressed in future work. IFN assumes that the graph is undirected, which does not adequately model networks with unidirectional flows (e.g., traffic) or lossy flows (e.g., resistive power grids). IFN also assumes CR that depends on the difference between nodal variables. This form appears frequently in physical systems, but in other network flow models (like Daganzo traffic models [42]), the CR has a more general dependence on the nodal variables. These limitations may be addressed with extensions of IFN's contraction algorithm.

---

Acknowledgments and Disclosure of Funding
This work was supported by AFOSR under grant FA9550-22-1-0050 US Army ERDC under grant W912HZ-22-2-0010, and DTRA under grant HDTRAI-19-1-0017.
References
[1] S. M. Rinaldi, J. P. Peerenboom, and T. Kelly. Identifying, understanding, and analyzing critical infrastructure interdependencies. IEEE Control Systems Magazine, (1(6):1-25, 2001.
[2] J. A. Jacquez and C. P. Simon. Qualitative theory of compartmental systems. SIAM Review, 35(1):43-79,
1993.
[3] J. Jia, M. T. Schaub, S. Segarra, and A. R. Benson. Graph- based semi-supervised & active learning for edge flows. In ACM International Conference on Knowledge Discovery and Dato Mining (SIGKDD), page 761-771 July 2019.
[4] A. Silva, F. Kocayusufoglu, S. Jafarpour, F. Bullo, A. Swami, and A. K. Singh. Combining physics and machine learning for network flow estimation. In International Conference on Learning Representations, Online, May 2021.
[5] I. Ford and D. Fulkerson. Maximal flow through network. In Canadian Journal of Mathematics, volume 8, pages 399-404, 1956.
[6] M. Lippi, M. Bertini, and P. Frasconi. Short-term traffic flow forecasting: An experimental comparison of time-series analysis and supervised learning. IEEE Transactions on Intelligent Transportation Systems, 14(2):871-88) 2013.
[7] Y. Li, R. Yu, C. Shahabi, and Y. Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In International Conference on Learning Representations, 2018.
[8] H. Yao, X. Tang, H. Wei, G. Zheng, and Li. Revisiting spatial-temporal similarity: deep learning framework for traffic prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 5668-6673 5675, 2019.
[9] L. Böttcher, H. Wolf, B. Jung, P. Lutat, M. Trageser, O. Pohl, A. Ulbig, and M. Grohe. Solving AC power flow with graph neural networks under realistic constraints. arXiv preprint arXiv:2204.07000 2022.
[10] S. Bai, J. Z. Kolter, and V. Koltun. Deep equilibrium models. In Advances in Neural Information Processing Systems, 2019.
[11] E. Winston and J. Z. Kolter. Monotone operator equilibrium networks. Advances Neural Information Processing Systems, 2020.
[12] M. Revay, R. Wang. and R. Manchester. Lipschitz bounded equilibrium networks. 2020.
[13] L. El Ghaoui, F. Gu, B. Travacca, A. Askari, and A. Tsai. Implicit deep learning. SIAM Journal on Mathematics of Data Science, 3(3):930-90) 958, 2021.
[14] S. Jafarpour, A. Davydov, A. V. Proskurnikov, and F. Bullo. Robust implicit networks via -uccliidd contractions. In Advances in Neural Information Processing Systems, December 2021.
[15] S. W. Fung, H. Hcaton, Q Li, D. McKenzie, S. Osher, and W. Yin. Fixed point networks: Implicit depth models with Jacobian- free backprop, 2021. ArXiv e-print.
[16] R. T. Q. Chen, Rubanova, Bettencourt, and Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems, 2018.
[17] A. Agrawal, B. Amos, S. Barratt, S. Boyd, S. Diamond, and Z. Kolter. Differentiable convex optimization layers. In Advances in Neural Information Processing Systems, 2019.
[18] H. Heaton, D. McKenzie, Q. Li, S. W. Fung, S. Osher, and W. Yin. Learn to predict equilibria via fixed point networks. arXiv preprint
(0l::2200.00006 2021.
[19] M. M. Bronstein, Bruna, T. Cohen, and P. Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104. /3478, 2021.
[20] T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arxiv: arXiv:16090088 02907, 2016.
[21] W. Hamilton, Z. Ying. and J. Leskovec, Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
[22] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio. Graph attention networks. arXiv preprint 17X0::1710.10000 2017.
[23] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl. Neural message passing for quantum chemistry. In International Conference Machine Learning, pages 1263-1272. PMLR, 2017.
10

---

[24] P. Battaglia, R. Pascanu. M. Lai. D. Jimenez Rezende, and K. Kavukcuoglu. Interaction networks for learning about objects, relations and physics. In Advances in Neural Information Processing Systems. 2016.
[25] F. Gu, H. Chang, W. Zhu, S. Sojoudi, and L. El Ghaoui. Implicit graph neural networks. In Advances in Neural Information Processing Systems, 2020.
[26] P. Kundur. Power System Stability and Control. McGraw-Hill, 1994.
[27] E. John Finnemore and Joseph B. Franzini. Fluid mechanics with engineering applications. McGraw-Hil tenth edition, 2002.
[28] A. K. Singh, Ibraheem, S. Khatoon, M. Muazzam, and D. K. Chaturvedi. Load forecasting techniques and methodologies: A review. In 2012 2nd International Conference Power, Control and Embedded Systems. TEFE, 2012.
[29] G. M. U. Din and A. K. Marnerides. Short term power load forecasting using deep neural networks. In 2017 International conference computing, networking and communications (ICNC), pages 594-598. IEEE, 2017.
[30] K. D. Smith and K. Studarus. Limited-knowledge economic dispatch prediction using bayesian averaging of single-node models. In 2018 IEEE International Conference on Probabilistic Methods Applied to Power Systems (PMAPS). IEEE, 2018.
[31] S. Jafarpour, E. Y. Huang, K. D. Smith, and F. Bullo. Flow and elastic networks on the r-torus: Geometry, analysis and computation. SIAM Review, 64(1):59-1044 2022.
[32] F. Bullo. Lectures on Network Systems. Kindle Direct Publishing. 1.6 edition, January 2022.
[33] N. K. Vishnoi. Lx = b. Laplacian solvers and their algorithmic applications. Theoretical Computer Science, 8(1-2):1-141, 2013.
[34] P. G. Doyle and L. Snell. Random Walks and Electric Networks. Mathematical Association of America, 1984.
[35] A. B. Birchfield, T. Xu, M. Gegner, K. S. Shetye, and Overbye. Grid structural characteristies as validation criteria for synthetic networks. IEEE Transactions on power systems, 32(4):3258 3265, 2016.
[36] R. D. Zimmerman, C. E. Murillo-Sáncher and R. Thomas. MATPOWER: Steady-state operations, planning, and analysis tools for power systems research and education. IEEF Transactions Power Systems, 26(1) 26(1):12-19, 2011.
[37] E. Hernadez, S. Hoagland, and L. Ormsbec. Water distribution database for research applications. In World Environmental and Water Resources Congress, pages 465-474, 2016.
[38] K. A. Klise, R. Murray, and T. Haxton. An overview of the water network tool for resilience (WNTR). In Proceedings of the 1st Internationaa WDSA/CCWI Joint Conference, volume 1, 2018.
[39] Jo. Back, Lee, Kim, M. Kang, and S. Hwang. Edge representation learning with hypergraphs. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neurul Information Processing Systems, volume 34, pages 7534-7546. Curran Associates, Inc., 2021.
[40] T. Mitchell Roddenberry, Nicholas Glaze, and Santiago Segarra. Principled simplicial neural networks for trajectory prediction. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 9020 9029. PMLR, 18 24 Jul 2021.
[41] Maosheng Yang, Elvin Isufi, Michael Schaub, and Geert Leus. Simplicial convolutional filters. arXiv prepint arXiv:2201. //720, 2022.
[42] G. Como. On resilient control of dynamical flow networks, Annual Reviews in Control, 43:80-90, 2017,


Reviews:
[{"review_id": "vat1UIJQ9Aq", "reviewer": "Reviewer_JgzR", "summary": "IFN provides an implicit neural network layer, which can be used to predict unlabeled edge flows (f) from nodal supply and demand (u), which can be formulated as a nonlinear inverse problem. It also takes advantage of the physics of the flow network, namely conservation of mass at nodal points and constitutive relationship (Eqns 1 and 2) to train h_\\theta. Analysis is given for the convergence and existence of the IFN. To demonstrate, examples in the context of AC power networks are given.\n", "questions": "Minor suggestion: maybe theorem 2.2 can be expressed as a corollary right after theorem 2.3. Of course. authors can still mention the well-posedness of IFN before section 2.1, whose proof will be provided later.  \n\nCan we have an estimation of k* a-priori? Since it is a function of h_{\\theta}, it seems like it can be only found after the training. But it might be nice to have an approximation of the number of iterations.\n\nAny comments for the time-dependent flow networks? Please note that for various physics-informed networks, a major limitations is capturing the time-dependencies, so it's a non-trivial task (but feasible). \n\nThe authors clearly show how nonlinearities in equation 2 are captured, but seems like equation 1 needs to be linear. This may be a limiting factor. Any comments for a nonlinear version of equation 1 (for example conservation of momentum). \n\nAlthough IFN does not include measurements; can you consider it an advantage? Let's say constitutive law is not completely understood. So at the time of training, h_\\theta and f are evaluated based on the current model but such model may bifurcate or change in action. Since IFN does not take advantage of measurements directly, it may be prone to error in such circumstance. Any comments on robustness of IFN?\n\n", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe method to enforce physical laws of eqns 1 and 2 into the training is elegant. It comes with guarantees on the existence of the solution (contraction of T, which also makes the inverse problem a well-posed one), algorithm with clear mathematical foundation (with a finite number of iteration for convergence k*), and convergence analysis on FN (forward pass). \nThe authors further compute gradients required for training in manner specific to IFN.\nThe distinction with optimization-based approaches is clear.\n\nWeaknesses:\nThe implicit flow network represented by a neural network is not new; but addition of physical laws (conservation and constitutive) is novel. Authors also develop a nice method to achieve this. But although they try to generalize this for a verity of physical systems in section 3.1, I don't think it is clear how can one extend the framework for more complicated networks such as traffic flow or collective system of rigid bodies. Furthermore, seems like the method can only be applied to steady-state systems which is a very serious limitations. I think such limitations should reflect in the title of the paper as the current one may sound too generic.  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "_da2Fcnv9_k", "reviewer": "Reviewer_XX9M", "summary": "The submission proposes a solution to solve a conservation law and the existence of a constitutive relationship via implicit function. They shows that this model can estimate flows in AC networks. ", "questions": "Is this work still working when the number of nodes of the graph increases or decreases?", "limitations": "This work is limited to the undirected graph.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- It improved the previous general models predicting network flows not only by considering conservation law but also by considering a constitutive relationship.\n- This work clearly and accurately incorporates the constitutive relationship in the implicit model.\n- This work shows detailed descriptions for the forward and backward pass, where a contraction algorithm is used for evaluation and backward pass.\n\nWeaknesses:\n\n- This work is limited to an undirected graph.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "D9mH-dzla5R", "reviewer": "Reviewer_k7us", "summary": "The paper introduces an implicit neural network to compute (nonlinear) network flows, extending previous works that predict flows based on a divergence minimization principle.", "questions": "Why do the authors only consider primarily power flow problems in their evaluation? \n\nIt appears that one of the strengths of the proposed framework would be that it can be adjusted to different constitutive laws, covering other scenarios as well. It would be nice to see an evaluation of this. Several further datasets (traffic, etc.) are available from Ref [3] here:\nhttps://github.com/000Justin000/ssl_edge/tree/master/data\n\nI think a somewhat broader discussion of related literature would be of interest to the reader.\n\nIn the context of solving power flows with (graph) neural networks, see. e.g.:\nBöttcher, Luis, et al. \"Solving AC Power Flow with Graph Neural Networks under Realistic Constraints.\" arXiv preprint arXiv:2204.07000 (2022).\nThere are also several neural networks for edge flow predictions:\n\nRoddenberry, T. Mitchell, Nicholas Glaze, and Santiago Segarra. \"Principled simplicial neural networks for trajectory prediction.\" International Conference on Machine Learning. PMLR, 2021.\nEbli, Stefania, Michaël Defferrard, and Gard Spreemann. \"Simplicial Neural Networks.\" NeurIPS 2020 Workshop on Topological Data Analysis and Beyond. 2020.\n\nFinally, related works can also be found in the graph signal processing literature, e.g.,\nYang, Maosheng, et al. \"Simplicial Convolutional Filters.\" arXiv preprint arXiv:2201.11720 (2022).\n\n", "limitations": "n/a -- limitations are not discussed", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- well written paper\n- clear theoretical characterization of results\n\nWeaknesses\n- somewhat limited numerical evaluation (only power flow datasets)\n- related literature could be improved", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9DGmIHQvZuc", "reviewer": "Reviewer_dwJZ", "summary": "This paper incorporates two physical laws, namely conservation of mass and the existence of a constitutive relationship between edge flows and nodal states, into an implicit neural network layer to solve the estimation problem of unlabeled edge flows.\n\nRating raised from 4 to 5 after revision.", "questions": "N/A", "limitations": "Yes. The authors address the limitations of the implicit neural network layer that needs to be improved in the future.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Theory: \n1. Applying these two physical laws to flow prediction seems natural and reasonable. I’m not sure if others have made similar attempts. \n2. The mathematics in the paper is cool, although it might be a little hard to follow.\n\nPresentation: \n1. How the physical laws are incorporated is not highlighted in the paper and is hard to find. Section 3.1 is too far away from Equations 1 and 2.  After the revision, this point has been demonstrated more clearly.\n2. The physical laws incorporated are not elaborated.\n3. Figures sometimes are too far away from the corresponding text. Solved after revision.\n\nExperiment: Although the paper lists a few potential applications of flow prediction, the experiment focuses only on AC power networks. Thus, only one example of the second physical law, which is Ohm’s law, is examined. More experiments are included after the revision.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept