Flow-Attentional Graph Neural Networks
Authors: Pascal Plettenberg
Dominik Köhler
Bernhard Sick
Josephine Thomas
Venue: Accepted by TMLR ()

Abstract: Graph Neural Networks (GNNs) have become essential for learning from graph-structured data. However, existing GNNs do not consider the conservation law inherent in graphs associated with a flow of physical resources, such as electrical current in power grids or traffic in transportation networks, which can lead to reduced model performance. To address this, we propose flow attention, which adapts existing graph attention mechanisms to satisfy Kirchhoff’s first law. Furthermore, we discuss how this modification influences the expressivity and identify sets of non-isomorphic graphs that can be discriminated by flow attention but not by standard attention. Through extensive experiments on two flow graph datasets—electronic circuits and power grids—we demonstrate that flow attention enhances the performance of attention-based GNNs on both graph-level classification and regression tasks.

Published in Transactions on Machine Learning Research (09/2025)
Flow-Attentional Graph Neural Networks
Pascal Plettenberg
plettenberyyhuui-kkssccl
Intelligent Embedded Systems, University of Kossel
Dominik Köhler
dkochler@unii-assee..e
Intelligent Embeedee Systems, University of Kossel
Bernhard Sick
bsick warcchhrikkkssll.coo
Intelligent Embedded Systems, University of Kassel
Josephine M. Thomas
thomasj@uniigraafwuull. de
GAIN Group, Institute of Data Science, University of Greifswald
Reviewed on OpenReview: https: // openreview. net/ forum? id= 7UxTP
/ttps:///penrevveuu
Abstract
Graph Neural Networks (GNNs) have become essential for learning from graph-structured data. However, existing GNNs do not consider the conservation law inherent in graphs associated with flow of physical resources, such as electrical current in power grids or traffic in transportation networks, which can lead to reduced model performance. To ad- dress this, we propose flow allention, which adapts existing graph attention mechanisms to satisfy Kirchhoff's first law. Furthermore, we discuss how this modification influences the expressivity and identify sets of non-isomorphic graphs that can be discriminated by flow attention but not by standard attention. Through extensive experiments on two flow graph datasets electronic circuits and power grids we demonstrate that flow attention enhances the performance of attention-based GNNs both graph-level classification and regression tasks.
1
Introduction
have emerged as a powerful framework that extends the scope of deep learning from Euclidean to graph-structured data, which is preva- recommender systems (Wul
Graph neural networks (GNNs) (Sarrelll et al. 2008 Kipf & Welling 2017)
lent across many real-world domains, such social networks (Fan et al. 2019
et al.
2022 materials science (Reiser et al., 2022), or epidemiology (Liu et al.
2024 Especially attention- GNNs s have become increasingly popular due to their ability to select relevant features adaptively
based
(Sun et al.
2023). As graph data becomes increasingly common, advancing GNN architectures is crucial for
improving performance in tasks such as node classification (Hamilton et al. 2017 graph regression Giimer
et al. 2017 or link prediction (Zhang Chen 2018
In many important applications of GNNs, graphs are naturally associated with a flow of physical resources, such electrical current in electronic circuits (Sánchez et al.) 2023) power grids (Lioo et al. 2021 traffic in transportation networks (Jiang & Luo 2022 water in river networks (Sun et al. 2021).
materials and goods in supply chains (ooasshh & Brintrup 2022 All nodes in these resource flow graphs, except for source and largel nodes, are subject to Kirchhofl's s first law, which states that the sum of all incoming and outgoing flows must be zero, reflecting the conservation of resources. By contrast, informational graphs-sute as computation graphs, social networks, or citation networks are not associated with any physical flow but rather represent relationships or information transfer. Information can be arbitrarily duplicated in these graphs, unlike in flow graphs, where such duplication would violate the conservation law.

---

Published in Transactions on Machine Learning Research (09/2025)
a
810
Informational Graphs
x
X
sin
sin
b
R
Resource Flow Graphs
R
In
Out
In
R
Out
R
R
R
Figure 1: Two non-isomorphic graphs that are equivalent as informational graphs, but different as resource flow graphs a The two different directed graph structures represent the same computation (example adapted from Zhang et al. (2019)) ). b The same graph structures as above represent different electronic circuits.
As result, two non-isomorphic graphs may be equivalent informational graphs but non-equivalent flow graphs. For example, in computational graph, the output of sinc operation can be froely duplicated. Thus, the two non-isomorphic graph structures in Fig. Ia represent the same computation. However, the same graph structures may also represent electronic circuits governed by Kirchhoff's ss first law (see Fig. [b] In this case, the two circuits are different because combining or splitting resistors changes their electrical properties.
Electronic circuits and other resource flow graphs can often be represented directed acyclic graphs (DAGs). However, GNNs specifically tailored DAGs typically encode the rooted trees of the output nodes rather than the exact graph structure. Hence, if two DAGs exhibit the same computation tree (e.g., the graphs in Fig. D. they cannot. be distinguished, which limits the performance in many graph learning tasks. While these graph structures might be interchangeable for pure "Information" tasks, sufficiently expressive GNN should map them to distinct representations when performing lasks where physical flows are relevant.
Main Contributions. Inspired by the conservation law in resource flow graphs, we propose flow attention on graphs, which normalizes attention scores across outgoing neighbors instoad of incoming ones. This simple but effective modification can be applied to any attention-based GNN and allows the model to better capture the physical flow of a graph. We discuss the expressivity of the resulting models and demonstrate that flow attention enables the discrimination of any DAG from its computation tree. Based on this observation, we propose FlowDAGNN, a flow-attentional GNN for DAGs. Finally, we conduct extensive experiments on multiple datasets, including cascading failure analysis on power grids and property prediction on elec- tronic circuits, covering undirected graphs and DAGs, Our results indicate that flow attention improves the performance of attention-baan based GNNs on graph-level classification and regression tasks.
2 Related Work
In recent years, many new GNN models have been specifically designed for different graph types TTomas et al. 2023). Despite their fundamental differences, informational graphs and flow graphs are mainly treated by the same basic message- passiin layers, such as GraphSAGE (Hamilton et al. 2017). GAT (Veličkovi ett al. 2018), or GIN (Xu al.] 2019). Tn these models, messages exchanged between neighboring nodes do not depend on the number of recipients. Instead, the information is arbitrarily duplicated and passed to
all neighbors. GCN (Kipf & Welling 2017) applics symmetric neighborhood normalization but exhibits limited expressivity and performance. Attention-basee GNNs, such GAT, GATv2 (Brody et al. 2022 UnIMP (Shi et al. 2021) adaptively weight neighboring node features, leading to improved represontation
The code available at https://gsthibb.cm//aapg/// ccwm/ppssplltt//oooott
2

---

Published in Transactions on Machine Learning Research (09/2025)
learning. However, the attention weights are obtained through normalization across incoming neighbors, allowing for arbitrary message duplication. Our approach normalizes across outgoing neighbors instead, which better captures the physical flow of a graph while preserving the benefits of graph attention.
Many flow graphs, inchuding the example graphs in Fig. ! can be naturally expressed as DAGs, e.g., op- erational amplifiers (Dong et al. 2023) or material flow networks (Perera et al. 2018). Before GNNs were introduced, recursive neural networks were applied DAGs (Sperduti & Starita| 1997 Frasconi et al. 1998), and contextual recursive cascade correlation was proposed to overcome limitations in expressivity (Hammer le al. 2005). However, these early works lacked the advantages of modern GNNs, which have been extended to DAGs in recent years (Zhang et al.] 2019); Thost & Chen 2021). In directed acyclic GNNs, nodes are typically updated sequentially following the partial order of the DAG, and the final target node representa- tions are used as the graph embedding. Although these sequential models outperform undirected GNNs on DAG datasets, they still aggregate node neighborhood information similarly. This means they only encode the computation tree of the output nodes and not the exact structure of the DAG, resulting in limitod expressivily.
A possible approach to overcome the problem of indistigguishable flow graphs is to use node indices or random features as input node features {Loukas 2020 Sato et al. 2021), which enables the model to uniquely identify each node. However, the resulting GNN model is no longer permutation invariant, which reduces its generalization capability. Similar problems arise for Transformer based models (Vaswani et al. 2017) such
as PACE (Dong et al. 2022), which incorporate the relational inductive bias (Battaglia et al. 2018 via positional encodings. A different strategy would be introduce Kirchhof'ss first law through an additional physics-informed loss term (Donon et al., 2020), which considerably increases the training complexity and is only useful if the target variable is the resource flow itself. Our approach enhances the expressivity of aatention-based GNNs on flow graphs while preserving permutation invariance and computational efficiency.
3 Preliminaries
Graph. A directed graph can be defined as a tuple G = EV, containing a set of nodes V C N and a set of directed edges & _ VxV. Thereby, we define e = (u, v) as the directed edge from node u to node v. An edge is called undirected (u, E whenever (v, E E. Furthermore, we call the set Nim = {u E V (u, E E] the incoming neighborhood of v and the set (N) {u e V (v, u) E E} the outgoing neighborhood of v.
Node Multisct. For cach node in a graph, the feature vectors of set of incoming nodes can be represented as multisel (Xu et al. 2019). A multisel is pair (5,m), where S is set of distinct elements (the node features) and m S + N is the multiplicity of each element. We call two multisets X1 = (S,m)), X2 = (5, m2) equally distributed if m2 = ---m m- with & E N21.
Directed Acyelic Graph. A directed graph without cycles is called a directed acyelic graph (DAG). In the context of DAGs, we call the incoming neighborhood the predecessors of a node, and the outgoing neighborhood the successors of a node. The set of all ancestors of node u contains all nodes u E V such that is reachable from Similarly, the descendants are the nodes the set of nodes without predecessors is called the set of start or initial nodes, denoted by I c V, and the set of nodes without successors is called the set of end or final nodes, denoted by FCV.
V that are reachable from v. Finally,
Computation Tree. Let D = Ev) be a rooted DAG with a unique final node called the root r. Its computation tree r(D) obtained by leaving each node e with at most one successor, and for any node e with n 2 2 successors, replacing v by n copics vi,..., Un, each connected to exactly one of v's successors and inheriting all of v's incoming edges. This procedure yields a rooted tree with the same root r. See App. D for visualization.
Flow Graph. Let G (V,E) be a graph and S, C V be the sources and targets of G. A flow on g is mapping sb 6:E 3 + R. that satisfies Kirchhofl's first law:
{(, = Σ
((,,e YoEV\(S,T)
(1)
(v)
NENou. (v)
3

---

Published in Transactions on Machine Learning Research (09/2025)
If a graph is associated with flow & as defined above, refer to it as flow graph. In DAGs, the start nodes are sources, and the end nodes are targets: I I 8 and F ST.
Graph Neural Networks. Graph Neural Networks (GNNs) transfer the concept of traditional neural networks to graph data. Thereby, the node representations {h, E R' I E VJ with the feature dimension P are updated iteratively by aggregating information from neighboring nodes via message-passing. The updated node representations hi i.e,, the output of the network layer, are given by
h/ =
his jENi.(1)
f (h,)
(2)
with learnable message function f. aggregator @ e.g., choice of o, @ and f defines the design of specific GNN model.
mean, and an update function ø. The
Directed Acyelic Graph Neural Networks. The main idea of GNNs for DAGs is to process and update the nodes sequentially according to the partial order defined by the DAG. Thereby, the update of a node representation h; is computed based on the current-layer node representations of node i's predecessors. The cessage-passiigg scheme of directed acyelic GNNs can therefore be expressed as
h1 =
his + f (k₂)) jENin(i)
(3)
The most widely used directed acyclic GNNs are D-VAE Zhang et al. 2019) and DAGNN (Thos & Chen 2021b, the latter of which uses standard attention for aggregation. Both models utilize gated recurrent units (GRU) as the update function ø and are briefly explained in App. B As an alternative to sequential models, DAGs can be encoded using Transformer-bast based architectures, such as PACE (Dong et al. 2022) or DAGformer [uuo et al. 2023
4 Graph Attention and its Limitations
In this section, we demonstrate why standard graph attention is insufficient for flow graphs. First, we show that standard attention cannot distinguish node neighborhoods with equal distribution of node features, which generally limits its expressivity. Next, we prove that attention- based directed acyclic GNNs cannot discriminate between DAG and its computation tree. As result, they cannot distinguish non-isomorphic DAGs that exhibit the same computation tree, e.g., the example graphs from Fig. 1
4.1 Attentional GNNs
An attentional GNN uses scoring function Ee x R + E. to compute attention coefficients
eij = e hj),
(4)
indicating the importance of the features of node j node i. The computed attention coefficients ei are normalized across all incoming neighboring nodes using softmax:
exp(ey)
0i softmax, EkeNm exp(exx)
(5)
Finally, the aggregation corresponds to a weighted average of the incoming messages:
h = Σ
asy (hau.s) )
(6)
(t)

---

Published in Transactions on Machine Learning Research (09/2025)
Standard Graph Attention
b
Flow Graph Attention
0.09
B,
f22
012 =z 0.35
h12
Pig
0111
11,07
0.91
01.07
h1
update
h
h1
updoue
h
0.43
Bis =: 0.52
O1 0.15
Aia = 0.44
343 0.36
h-
h4
ha
hA
Pao
0,12
As. 0.41
Bas
Flow-Attentional Directed Acyelic GNN (FlowDAGNN)
1) Reverse Pass with Standard Attention
2) Forward Pass with Flow Attention
0.47
0.41
ONE
OCE =
AJ LT
hy
821
1.0
h81
iv
Fee
h
OEd
h4
1.0
D12 = 0.59
0.53
0
computation of sequential node updates
Figure 2: a Standard graph attention mechanism as it is applied in attentional GNNs. The attention weights associated with edges of the same color sum to 1. b The proposed flow attention mechanism applied in FlowGNNs. The flow atlention weights associated with edges of the same color sum to e Two snapshots during the reverse and forward pass of FloNDAGNN, Nodes marked in green have already been updated.
BBody et al., 2022 and Trans- formerConv (TC) Shi et al. 2021), which mainly differ in the choice of the scoring function e (see App. The graph attention mechanism is visualized in Fig. 2a
Popular attentional GNNs include GAT (Velčcković et al., 2018),
GATv2
aggregation limits the expressivity of attention-based GNNs. Similar the mean aggregator, standard attention does not capture the exact node neighborhood but the distribution of nodes in the neighborhood.
The weighted
Lemma 4.1. Assume Xi = m)) and X/2 = (S, k-m) are multisets with the same distribution, with E N21- Then h ( (X)), for any choice of & and f.
Proofs of all Lemmas and Corollaries can be found in the Appendix A
4.2 Attention on DAGs
The node update of an attentional directed acyclic GNN can be expressed as
W = hatt,ss
Σ anj (Kases))
(7)
JEN
A directed acyclic GNN sequentially updates each node's feature vector until it arrives at more final
nodes. If there are multiple final nodes, their representations can be gathered into single virtual node.
PPleasc note that TransformerCony is not Graph Transformer with global attention modulc (such GraphCPS Ram-
pášek ) message- message-passingg GNN with local attention mechanism adopted from the vanilla Transformer (Vswan originally introduced under the name Graph Transformer part of the UniMP model (Shi 2021
5

---

Published in Transactions on Machine Learning Research (09/2025)
Since all other nodes are ancestors of this final node, its representation can be used to characterize the whole DAG. The computation history of the final node representation can be visualized as a rooted subtree, which means that if two non-isomorphic DAGs represent the same computation, the rooted subtree structures of their final node representations are equivalent. Therefore, since standard attention weights only depend on incoming neighborhoods, directed acyclic GNNs with standard atlention mechanisms cannol. discriminate a DAG from its computation tree (D) (see Fig. 3 in App. D)
Corollary 4.2. For every DAG DD E A and every directed acyelic GNN Ant with slandord allention mechanism, it holds:
(au(D) Aau((r(D)).
As a consequence, an attentional directed acyclic GNN cannot. distinguish between the two graphs in Fig. 1 since they cxhibit the same computation tree. Note that Corollary 4.2 is also valid for directed acyclic GNNs with other aggregators that are independent of outgoing node neighborhoods, e.g., sum or mean.
5 The Flow Attention Mechanism
In this section, we introduce the flow attention mechanism and discuss its influence expressivity, Next, we define flow attention on DAGs, which enables the discrimination of a DAG from its computation tree. Finally, we propose FlowDAGNN, a directed acyclic GNN model for flow graphs.
5.1 Flow-Attentional GNNs
Standard graph attention mechanisms normalize attention scores across all incoming edges. Therefore, a message does not depend on the number of nodes it is forwarded to and can be duplicated arbitrarily, contradicting the resource conservation concept inherent in flow graphs. Therefore, we propose an alternative graph attention mechanism that normalizes the attention scores across outgoing edges instead (see Fig. 2b We denote the resulting flow attention weights as Biy to distinguish them from the standard attention weights Osgy
exp(es))
Bij = (oftmax((ex) = EmENaeLii exp(ex))
Although the attention scores are normalized across outgoing edges, we still aggregate incoming messages order update the hidden state of node
Σ Bigl (haswws) (EV
k = Φ
However, since the messages are multiplied with the flow attention weights Big, they now also depend on the neighborhood of the message's sender, i.e, node j. In this way, we ensure that a message transmitted by any node cannot be duplicated arbitrarily but instead is distributed among all outgoing neighbors. We define flow-attentional graph neural network (FlowGNN) as a modified version of an attentional GNN, which utilizes the flow attention mechanism from Eq. for aggregating node neighborhood information instead of standard attention. Furthermore, denote the corresponding FlowGNN versions of staudard attentional GNNs as FlowGAT, FlowGATv2, FlowTC, etc.
The flow attention weights determine how a nodc's message is distributed among its outgoing neighbors, i.e., Bij can be seen as the relative flow from node j to node i. An absolute flow wa can be calculated by iteratively multiplying subsequent flow attention weights in graph.
Lemma 5.1. Let G = (V,E) be a graph with fired source nodes S C V and target nodes T C V. We define b for every directed edge (j,i) E as
Bi Σ (o(k.)), if e VJV.ST
kENin())
(o.))))
otherwise.

---

Published in Transactions on Machine Learning Research (09/2025)
Thereby, (.. is the absolute outgoing flow from source or target node. Then th is flow G that
salisfies Kirchhoff's 's first law.
Since the absolute flow v satisfies Kirchhoff's first law, FlowGNN is capable of implicitly taking into account the underlying resource flow of graph via the flow attention weights Piy. Another advantage of the flow attention mechanism is that the aggregation over incoming neighbors is not a weighted mean but a weighted sum. Therefore, in contrast to standard attention, flow attention can discriminate between multisets with the same distribution.
Lemma 5.2. Assume X1 = (S,m) and X2 = (S, k m) are mullisets with the some distribution of elements for some k E N22. Furthermore, assume that all nodes s E S have the same outgoing neighborhood No)((s) X1 and X2. Then $ such that h (X)) *
(X))
Therefore, flow-attentional GNNs are particularly well-suited for tasks where not only statistical information but also the precise graph structure plays a crucial role. This is especially true when features occur repeatedly, e.g.: in the case of multiple identical resistors in an electronic circuit.
5.2 Flow-Attention on DAGs
We define the node update for flow-attentional directed acyclic GNN similar to Eq.
1/ = & haow,ss
Bill (Bhows) JENu.(()
Σ
(10)
If all nodes in a graph have only one outgoing edge, c-g., the graph is a rooted tree, then 3. = =1 14,, j. In this case, the flow altention mechanism degenerates to sum aggregation, which can be maximally expressive for the right choice of á and (Xu al. 2019).
Corollary 5.3. Let I A be the subset of all DAGs, where each node has at most one outgoing edge (i.e., the set of all rooted trees). Then, for any two rooted trees Tı, To with Tı T2, there crists flow-attentional directed acyelic GNN Apow such that:
A Aoow (Ti) # ARtow
Furthermore, contrary standard attention, flow-attentional directed acyelic GNNs can distinguish between graphs and their message- geassage-passing passing computation trees.
Theorem 5.4. For every DAG D E with D É T (i.e., a true DAG) and ils computation tree (D), there ewists floow-attentional directed acyclic GNN ARoW such that:
Afow (D) + (
Proof. We take an arbitrary but fixed intermediate node of the DAG and denote its representation under the flow-allentional directed acyclic GNN by
h; now and h;
(D) iiw..i
for the graph D and its computation tree (D), respectively. Since D not a tree, there is at least one node j with more than one successor. Hence, there is a predecessor j for which Big < We then need show that h A h/:
(hs Σ f1)()) + oh. Σ f(1))) JEN...
JENim.
By choosing f in the following way:
38 ff:((/)) 2 f(h)) Vj.k E Nm..
(11)
7

---

Published in Transactions on Machine Learning Research (09/2025)
it follows due to =j: 9, < 1:
Eq
11
Σ
Baf(h()) Σ
f(h))
Σ jENin,
f(h)).
JENi
JENi..
Hence, with choosing @ to be injective, Anow distinguishes D from 7(D).
From Theorem 5.4 we conclude that a flow-attentional directed acyclic GNN can discriminate the example DAGs from Fig. for the right choice & and f. In practice, we can model the composition f()) o(1-1)
the [-th GNN layer by universal approximator, e.g.: GRU (Schäfer & Zimmermanni 2006 hoon Song
et al.
2023
5.3 FlowDAGNN
We propose a flow-attentional version of the attention-based DAGNN (Thost & Chen 2021 A naive approach would be to simply replace the attention weights asy with flow attention weights Bky. Due to the sequential nature of directed acyelic GNNs, the computation of the flow attention weight Bg in DAG only depends on the node and all its ancestors. However, in contrast to standard attention weights, flow atlention weights are forward-directed, which means that they should also be conditioned all descendants of the node Analogously, the electrical current splitting up from one node into multiple branches depends on the whole branch and not only on the first node in cach branch. In App. an electronic circuit illustrating this situation.
we give a simple example of E
Hence, we construct a FlowDAGNN layer from two sublayers (see Fig. 2ch In the first sublayer (we call it the reverse pass), we invert all edges of the DAG D and apply a standard DAGNN layer to the reverse DAG D. This is equivalent to performing the aggregation over all successor nodes in the original DAG D instead of over all predecessors:
m.
Σ
Onj (he, hT) hTy
(12)
jENou (3)
Oij (hi, h) softmax ((wFT)T)) (w (w)))) Th,"),
(13)
jENour(i)
h h GRU(h; m,"")
(14)
In the second sublayer, we perform forward pass on the original DAG G. However, this time we are applying the flow attention mechanism described in Section 5.1 compute flow attention weights B,y:
Fw m
Σ
Big (hT", fo h.w
(15)
jENou (1)
Bi (hg h
softmax ((w #ENou(()
(w They +
(16)
h Fw
GRU(H," fw
(17)
Since the hidden states ht of the reverse pass contain information about all descendants of the node i, and the hidden states hew contain information about all ancestors of the node j, the computation of the flow attention weights Biy essentially takes into account information about all nodes of the graph that are connected to the node i.
After Flow layers, we compute the graph-level representation from both the reverse pass repre- sentations of the start nodes as well the forward pass representations of the end nodes and concatenate across layers:
(ii n.v.d
Max-Pool (i fmd. JEF
hg MaM--oxo
Max-Pooo IET
(18)
1=0
1-0
8

---

Published in Transactions on Machine Learning Research (09/2025)
5.4 Applicability Beyond Flow Graphs
The flow attention mechanism can be integrated into any GNN with local attention mechanism. Thus, flow-attentional GNNs can technically be applied to the same graph datasets as their base models, which means their applicability is not necessarily limited to flow graphs. However, it might be less effective on other types of graph datasets.
In flow attention, attention scores are normalized across outgoing neighbors, guiding a model to learn how to distribute a node's message among them. This is especially useful on flow graphs governed by Kirchhoff's first law, such as power grids, electronic circuits, or traffic networks. In contrast, other types of graph datasets, such citation graphs social networks, do not have an inherent structure that requires an information distribution across outgoing nodes. For example, there may be nodes that broadcast information to many adjacent nodes, e.g., inftuencers in social networks (Chen et al.) 2009).
Furthermore, as pointed out in the Introduction, some non-isomorphic graph structures might be equivalent in the context of informational graphs (e.g., computational graphs) and therefore should be mapped to the same representation, consistency not guaranteod by a flow-attentional GNN due to Theorem 5.4 We therefore expect FlowGNNs be less effective on these types of graph datasets.
On the other hand, the flow-attention mechanism has the advantage over standard attention that it distinguish between node multisets with the same distribution (Lemmaa 5.2). This could lead to performance improvements on graphs where certain node features occur repeatedly, which is often the case in flow graph applications (e.g., multiple identical components in electronic circuits), but can also be the case outside of the flow graph context (c.g., multiple identical atoms in molecules).
In this study, we focus on evalualing the effectivenens the flow attention mechanism on flow graph datasets, including power grids and electronic circuits. However, additional experimental results on standard graph datasets can be found in App.
6 Experiments
We perform two different experiments. First, we perform graph-level multiclass classification on undirected flow graphs, comparing the effectiveness of our flow attention mechanism against standard graph attention. In the second experiment, we perform graph regression on DAGs compare our proposed FlowDAGNN model with relevant directed acyelic GNN baselines.
6.1 Graph Classification on Undirected Flow Graphs
Dataset. We use the publicly available power grid data from the PowerGraph benchmark dataset (Varbella ct al. 2024), which encompasses the IEEE24, IEEE39, 1EEE118, and UK transmission systems. The graphs contained in these datasets are undirected and cyclic and represent lest power systems mirroring real-world power grids. The test systems differ scale and topology, covering various relevant parameters. Further details can be found in App.
Task. We perform cascading failure analysis graph-level multiclass classification task. Thereby, we utilize the attributed graphs provided by the PowerGraph dataset, each representing unique pre-outage operating conditions along with a set of outages corresponding to the removal of single or multiple branches. An outage may result in demand not served (DNS) by the grid, and a cascading failure may occur, meaning that one or more additional branches trip after the initial outage. The model supposed to predict whether the grid is stable (DNS = MW) or unstable (DNS > MW) after the outage, and additionally, whether cascading failure occurs, resulting in four distinct categories representing the possible combinations of stable/une unsttblle and cascading failure yes/no.
Models and Basclines. We take three widely used attention- GNNs (GAT, GATv2, and Transformer- Conv (TC)) and compare them against the corresponding flow-attentional variants FlowGAT, FlowGATv2, and FlowTC, Additionally, we compare against three popular non non-attentional GNN baselines (GCN, Graph- SAGE, and GIN). For each model, we perform hyperparameter optimization by varying the number of message-passe paagggggasnnggggg layers (1, 2, 3) and the hidden dimension (8, 16, 32). Between subsequent message-pant passing layers, we apply the ReLU activation function followed by a dropout of 10%. To obtain graph embeddings
9

---

Published in Transactions on Machine Learning Research (09/2025)
Table 1: Test set balanced accuracy (%, t) and Fl-score (macro-averaged, %, t) for the cascading failure analysis (multiclass classification) on four different power grid test systems from the PowerGraph dataset. Reported results represent the average over five training runs with different random seeds, along with the standard deviation. The best result for each test system is marked in bold.
IEEE24
IEEE39
IEEE11S
UK
MODEL
Bal.
(1)
Macro-F1 (t)
Bal. Ace. (1)
MacroFF1 (1)
Bal. Aec. (t)
Macro-F1 (1)
Hal. Acc. (t)
Macro-F1 (1)
GCN
89.4
78.2 +:
77.7 + 6.7
88.4 1.2
86.3 1.6
GraphSAGE
95.6 0.2
95.4 0.4
88.7
89.1 +
98.7 =
98.4 + 0.1
95.2 0.3
95.2 0.3
GIN
98.0 0.8
97.3 0.9
95.2 1.7
94.6 1.4
96.5 + 2.5
96.3 + 2.5
97.1 0.2
96.4 0.2
GAT
90.8 = 4.1
90.6 + 3.6
90.9 = 2.7
90.0 + 2.7
92.1 = 1.5
91.2 = 1.5
91.9 + 1.0
94.3 = 1.3
FlowGAT
93.3 = 1.1
93.0 1.0
94.1 = 1.3
93.2 4: 1.3
96.2 + 2.7
96.1 + 2.6
94.9 + 0.4
94.3 # 0.4
GATv2
91.9 4.0
90.0 + 4.7
87.8
86.3 + 2.2
92.6 + 1.6
91.7 + 1.3
95.3 + 0.7
94.8 1.0
2
96.1 = 0.9
95.6 + 0.8
95.9 4 0.7
95.0 0.9
99.1 + 0.1
99.0 + 0.1
96.9 + 0.3
96.4 = 0.4
TC
97.3 0.5
96.7 + 0.6
90.7 = 2.3
90.8 +: 2.0
98.7 + 0.1
98.4 + 0.1
96.5 + 0.3
96.2 = 0.2
FloWTC
98.7 0.3
98.1 0.3
96.0 0.6
91.4 + 0.6
98.8 = 0.2
98.6 - 0.2
97.1 = 0.4
96.8 + 0.5
from the node embeddings, we apply a global maximum pooling operator. For the final prediction, we use single linear layer or a two-layer perceptron with a LeakyReLU activation, depending on which type of prediction layer was used for the corresponding model in the original PowerGraph benchmark.
The messago-passing GNNs compared in this experiment are important components of many recent Graph Transformer models Shehzad et al. (2024), which combine local sarsaggepassingg with global attention. In
App. G.2 we report additional experimental results with recent state-of- toefthhh-arttt Graph Transformer models: SAT Chen et al., 2022), GraphGPS (Rampášck et al. 2022). and Exphormer (Shirzad et al. 2023). We will investigate the integration of flow-attentional GNNs into Graph Transformer models in future work.
Experimental Setting. We stick closely the original benchmark setting in Varbella al. (2024) by split- ting the datasets into trainvvaidaaion /test with ratios 85/5/ 5/1% 10% and using the Adam optimizer (Kingma scheduler that reduces the learning rate by factor
2014) with an initial learning rate of 10-3 as well of five if the validation accuracy plateaus for ten epochs. The negative log-likelihood is used as the loss function, and balanced accuracy [Brodersen et al. 2010) is used as the primary evaluation metric due to the strong class imbalance (sce App. F). We train all models with a batch size of 16 for a maximum number of 500 epochs and apply early stopping with a patience of 20 epochs. Each training run is repeated five times with different random seeds.
Results. The balanced accuracies and macro-F1 scores on the Lest are reported for each model on each of the four test systems in Tab. ! We only report the results for the best model architecture from the hyperparameter optimization. Thereby, we noticed that the accuracy mostly improves with more message- passing layers, which has already been observed for power grid data in Ringsquandl et al. (002)). The FlowGNNs perform better than their corresponding standard GNN version in most cases. FlowGAT shows a higher balanced accuracy compared to GAT for the test systems IEEE24, IEEE39, and IEEE118, and comparable performance on the UK test system. In the case of GATv2, the FlowGNN version even outperforms its standard counterpart systems except for IEEE118, where it shows comparable performance. On all test systems, the best- performing model among the tested ones is flow-attentional GNN. Overall, these results indicate that the flow attention mechanism, which is the only applied change to the corresponding baselines, can enhance the performance of attention-based GNNs on undirected flow graph data.
all Lest systems. FlowTC performs better than TC on all test
6.2 Graph Regression on Directed Acyclic Flow Graphs
Dataset. We utilize the Ckt-Bench101 dataset from the publicly available Open Circuit Benchmark (OCB) (Dong et al., 2023), which was developed evaluate methods for electronic design automation. The datasct contains 10,000 operational amplifiers (Op-Amps) represented DAGs and provides circuit specifications (e.g., gain and bandwidth) obtained from simulations. Details can be found in App. F
10

---

Published in Transactions on Machine Learning Research (09/2025)
Table 2: Test set. RMSE and Pearson's R (%) for the prediction of three different Op-Amp properties
from the Ckt-Bench101 dataset. Reportod results represent the average over ten training runs with different random seeds, along with the standard deviation. The best result for each properly is marked in bold.
GAIN
BANDWIDTH
FoM
MODEL
RMSE (4)
Pearson's is R (1)
RMSE (1)
Pearson's 's R (1)
RMSE (J)
Pearson's 's R (t)
PACE
0,253 0.009
97.1 1
0.443 0.014
90.9 1 0.5
0.443 0.009
90.8 0.5
DAGformer (SAT)
0.234 0.012
97.2 = 0.3
0.459 0.010
89.2 + 0.4
0.450 0.015
89.6 = 0.7
D-VAE
0.229 + 0.004
97.3 + 0.1
0.430 + 0.008
90.6 + 0.3
0.421 = 0.011
91.0 = 0.4
DAGNN
0.215 + 0.002
97.6 1 0.0
0.396 1: 0.008
92.1 1 0.3
0.396 + 0,005
92.0 = 0.2
FlowDAGNN
0.209 + 0.007
97.8 + 0.1
0.371 + 0.008
93.1 + 0.3
0.366 + 0.008
93.3 + 0.3
Task. We perform graph-level regression to predict the properties of the Op-Amps. For this purpose, we train three separate instances of each model for the prediction of gain, bandwidth, and figure of merit (FoM), respectively. The FoM a measure of the circuit's overall performance and depends on gain, bandwidth, and phase margin.
Models and Baselines. We compare our proposed model, FlowDAGNN, against widely used baseline models from the literature, including GNN- and Transformer-based models tailored to DAGs: D-VAE, DAGNN, DAGformer (building upon the Structure-Awars Transformer (SAT, Chen et al. (2022))) )), and PACE. Thereby, we use the default parameters from Dong et al. (2023) where applicable, and from the original publications elsewhere, as well as the model-specific readout layers. For FlowDAGNN, we use two layers as described in Sec. 5.3 (each comprising one reverse and one forward pass) and adopt all other model parameters from DAGNN. The final prediction done using two-layer perceptron with a. ReL.U activation in between. Right before these final layers, we apply dropout of 50% for regularization.
Experimental setting. We split the dataset into train/ hali//validaiio/teestt with ratios 80/10/10% and select the same test set as in Dong et al. (2023). Furthermore, we use the AdamW optimizer (Loshchilov 2017) with an initial learning rate of 10 and train each model using the mean squared error (MSE) as the loss function with a batch size of 64 for a maximum of 500 epochs, but apply early stopping with a patience of 20 epochs. Each training run is repeated ten times with different random seeds.
Results. The RMSEs on the test set for all models and all OpAmp target properties are presented in Tab. 2 Among all tested models, FlowDAGNN shows the best performance on all target properties. Especially, it performs better than DAGNN, the standard attentional model which it is originally based. Thereby, FlowDAGNN performs slightly better in prodicting the gain property and significantly better in the other two properties.
7 Conclusion
In this paper, proposed the flow attention mechanism, which adapts existing standard graph attention mechanisms to be more suitable for learning tasks on flow graph datasets, where the flow of physical resources (e.g,, electricity) plays an important role. Inspired by Kirchhoff's s first law, the mechanism normalizes atten- tion scores across outgoing edges instead of incoming ones, which avoids unrestricted message duplication and better captures the underlying physical flow of the graph. We discussed the influence of this architec- tural change on the model expressivity and showed that flow-attentional GNNs, in contrast to GNNs using standard attention, can distinguish node neighborhoods with the same distribution. Since the proposed modification of the standard graph attention is simple and minimal, it can be easily implemented in practice and does not significantly increase the computational effort (see App. G.4 for more details on computational efficiency).
Since many flow graphs can be naturally expressed as directed acyelic graphs (DAGs), we also extended the flow attention mochanism DAGs and proposed specific model, namely FlowDAGNN. We proved that this model can distinguish non-isomorphic directed acyelic graphs that were so far indistinguishable for existing GNNs tailored to DAGs. We validated our theoretical findings with extensive experiments on power grids
11

---

Published in Transactions on Machine Learning Research (09/2025)
and electronic circuit datasets, including undirected graphs and DAGs, respectively. Our results indicate that the flow attention mechanism considerably improves the performance of their standard counterparts on graph-level regression and classification tasks.
In the future, we want to analyze how the proposed models scale to larger circuits and power grids. Another interesting direction will be to evaluate the performance on node- and edge-level tasks, as well as on other flow graph data, such as traffic networks or supply chains. Finally, we also want to investigate the usage of flow- attentional GNNs local message-passing components in Graph Transformer models (see also App. G.2
Acknowledgments
This work was funded by the German Federal Ministry of Research, Technology and Space (16ME0877). We also acknowledge the helpful feedback from Mohamed Hassouna, Clara Holzhüter, Lukas Rauch, and Jan Schneegans.
References
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama, Optuna: A next- generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD interna- tional conference on knowledge discovery & data mining, pp. 1623-2633 2019.
Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Gonvalz, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. ar preprint or Arviv:1806.001 01261, 2018.
Ali Behrouz and Farnoosh Hashemi. Graph mamba: Towards learning on graphs with state space models. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pp. 119-130, 2024.
Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M Buhmann. The balanced accuracy and its posterior distribution. In 2010 20th international conference pattern recogrition, pp. 3121-3124. IEEE, 2010.
Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks? International Conference on Learning Representations, 2022.
Dexiong Chen, Leslie O'Bray, and Karsten Borgwardt. Structure-aware transformer for graph representation learning. In International conference machine learning, pp. 3469 3489. PMLR, 2022.
Wei Chen, Yajun Wang, and Siyu Yang. Efficient influence maximization in social networks. pp. 199 208, 2009.
Kyunghyun Cho, Bart van Merrienboer Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rmn encoder-decoder for statisti- cal machine translation. arXiv preprint arXiv:1406.1000 1078, 2014.
Zehao Dong, Muhan Zhang, Fuhai Li, and Yixin Chen. Pace: A parallelizable computation encoder for directed acyclic graphs. In International Conference on Machine Learning, pp. 5360-5377. PMLR, 2022.
Zehao Dong, Weidong Cao, Muhan Zhang, Dacheng Tao, Yixin Chen, and Xuan Zhang. Cktgnn: Circuit graph neural network for electronic design automation. International Conference on Learning Represen- tations, 2023.
Balthazar Donon, Rémy Clément, Benjamin Donnot, Antoine Marot, Isabelle Guyon, and Marc Schoenauer. Neural networks for power flow: Graph noural solver. Electric Power Systems Research, 189:106547, 2020.
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. Graph neural networks for social recommendation. In The world wide web conference, pp. 417-426, 2019.
12

---

Published in Transactions on Machine Learning Research (09/2025)
Paolo Frasconi, Marco Gori, and Alessandro Sperduti. A general framework for adaptive processing of data structures. IEEE transactions on Neural Networks, 0(5):768-788)) 1998.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chomistry. In International conference on machine learning, pp. 1263-1272. PMLR, 2017.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Barbara Hammer, Alessio Micheli, and Alessandro Sperduti. Universal approximation capability of cascade correlation for structures. Neural Computation, 17(5):1109-119,, 2005.
Chang hoon Song, Geonho Hwang, Jun ho Lee, and Myungjoo Kang. Minimal width for universal properly of deep m.n Journal of Machine Learning Research, 24(121):1-41, 2023.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datascts for machine learning on graphs. Advances in neural information processing systems, 33:22118 22133, 2020.
Weiwei Jiang and Jiayun Luo. Graph neural network for traffic forecasting: survey, Expert systems with applications, 207:117921, 2022.
Dicderik P Kingma. Adam: A method for stochastic optimization. arXiv preprint 1rXiv:1412.6900 2014.
Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. Inter- national Conference on Learning Representations, 2017.
Edward Elson Kosasih and Alexandra Brintrup. A machine learning approach for predicting hidden links in supply chain with graph neural networks. International Journal of Production Research, 60(17):5380-55 53993, 2022.
Wenlong Liao, Birgitte Bak-Jensen, Jayakrishnan Radhakrishna Pillai, Yuclong Wang, and Yusen Wang. A review of graph neural networks and their applications in power systems. Journal of Modern Power Systems and Clean Energy, 10(2):345-360, 2021.
Zewen Liu, Guancheng Wan, В Aditya Prakash, Max SY Lau, and Wei Jin. A review of graph neural networks in epidemic modeling. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6577 1577-6587, 2024.
Tlya Loshchilov. Decoupled weight decay regularization. arXiu preprint arXiv:1711.05100 2017.
Andreas Loukas. What graph neural networks cannot learn: depth vs width. International Conference on Learning Representations, 2020.
Yuankai Luo, Veronika Thost, and Lei Shi. Transformers over directed acyclic graphs. Advances in Neural Information. Processing Systems, 36:47764-47782, 2023.
Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the Construe- tion of Internet Portals with Machine Learning. Information Retrieval, 3(2):127-163, 2000.
Galilco Namata, Ben London, Lise Getoor, Bert Huang, and U Edu. Query-driven active surveying for collective classification. In 10th international workshop on mining and learning with gruphs, volume 8, pp. 1, 2012.
Supun S Percra, Michael GH Bell, Mahendrarajah Piraveenan, Dharshana Kasthurirathna, and Mamata Parhi. Topological structure of manufacturing industry supply chain networks. Complerity, 2018(1): 3924361, 2018.
13

---

Published in Transactions on Machine Learning Research (09/2025)
Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. Recipe for a general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems, 35:14501-14555 2022.
Patrick Reiser, Marlen Neubert, André Eberhard, Luca Torresi, Chen Zhou, Chen Shao, Houssam Metni, Clint van Hoesel, Henrik Schopmans, Timo Sommer, et al. Graph neural networks for materials science and chemistry. Communications Materials, 3(1):93, 2022.
Martin Ringsquandl, Houssem Sellami, Marcel Hildebrandt, Dagmar Beyer, Sylwia Henselmeyer, Sebastian Weber, and Mitchell Joblin. Power to the relational inductive bias: Graph neural networks in electrical power grids. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 1538 1538-1547, 2021.
Daniela Sánchez, Lorenzo Servadei, Gamze Naz Kiprit, Robert Wille, and Wolfgang Eeker. comprehensive survey on electronic design automation and graph neural networks: Theory and applications. ACM Transactions on Design Aulomalion of Electronic Systems, 28(2):1 27, 2023.
Ryoma Sato, Makoto Yamada, and Hisashi Kashima. Random features strengthen graph neural networks. In Proceedings of the 2021 SIAM international conference on data mining (SDM), pp. SIAM, 2021.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE transactions on neural networks, 20(1):61-80, 2008.
Anton Maximilian Schäfer and Hans Georg Zimmermann. Recurrent neural networks universal approx- imators. In Artificial Newral W'tworks-ICAAN 2006: 16th International Conference, Athens, Greece, September 10-14, 2006. Proceedings, Part 16, pp. 632-640. Springer, 2006.
Prithviraj Sen, Galilco Namata, Mustafa Bilgic, Lise Getoor, Brian Gallighor, and Tina Eliassi-Rad. Collec- live classification network data. Al magazine, 29(3):93 93, 2008.
Ahsan Shehzad, Feng Xia, Shagufta Abid, Ciyuan Peng, Shuo Yu, Dongyu Zhang, and Karin Verspoor. Graph transformers: A survey. arXie preprint 407.022407.09777, 2024.
Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked label pre- diction: Unified message passing model for cumi-supervieed classification. In Proceedings of the Thirtieth International Joinl Conference on Arlificial Intellgence, 2021.
Hamed Shirzad, Ameya Velingker, Balaji Venkatachalam, Danica Sutherland, and Ali Kemal Sinop. Ex- phormer: Sparse transformers for graphs. In International Conference on Machine Learning, pp. 31613- 31632. PMLR, 2023.
Alessandro Sperduti and Antonina Starita. Supervised neural networks for the classification of structures. IEEE transactions on neural networks, 833):714-735, 1997.
A Alexander Y Sun, Peishi Jiang, Maruti K Mudunuru, and Xingyuan Chen. Explore spatio-temporal learning of large sample hydrology using graph neural networks. Water Resources Research, 57(12): 22021WR030944, 2021.
Chengeheng Sun, Chenhao Li, Xiang Lin, Tianji Zheng, Fanrong Meng, Xiaobin Rui, and Zhixiao Wang. Attention-based graph neural networks: survey. Artificial Intelligence Review, 56(Suppl 2):2263-2311 2023.
Josephine Thomas, Alice Moalleny-Ourch, Silvia Beddar-Wicsing, and Clara Holzhüter. Graph neural networks designed for different graph types: survey. Transactions on Machine Learning Research, 2023.
Veronika Thost and Jie Chen. Directed acyclic graph neural networks. In International Conference on Learning Representations, 2021.
14

---

Published in Transactions on Machine Learning Research (09/2025)
Anna Varbella, Kenza Amara, Blazhe Gjorgiev, Mennatallah El-Assady, and Giovanni Sansavini. Power- graph: A power grid benchmark dataset. for graph neural networks. Advances in Neural Information Processing Systems, 37:110784 1100804, 2024.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L. ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017.
Petar Veličkovié, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pictro Lio, and Yoshua Bengio. Graph atlention networks. International Conference Learning Represenlations, 2018.
Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. Graph neural networks in recommender systems: a survey, ACM Computing Surveys, 55(5):1-335 37, 2022.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? International Conference on Learning Representations, 2019.
Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. Advances in neural information processing systems, 31, 2018
Muhan Zhang, Shali Jiang, Zhicheng Cui, Roman Garnett, and Yixin Chen. D-vae: A variational autoencoder for directed acyclic graphs. Advances in neural information processing systems, 32, 2019.
15

---

Published in Transactions on Machine Learning Research (09/2025)
A Proofs
Proof of Lemma 4.1
We are given the node update of attentional GNN from Equation 6 adapted to the
multiset X = (5,m) input:
Σ Eans)(s) (hantess)
Wau Φ
with a1s being the softmax over the edge importance scores:
exp(e1s)
O1s = Eveg exp(e1>))
For the two given multisets X1, X2, W gets the same result for any choice of e and
1 (Xi) Φ m eexpeenn))
EveS m(s') exp(ei>)
= Eess expls)) (hatts))
Evesh ki m(s)) exx((es)))))
= /h'u (22).
Proof of Corollary 4.2 This follows from the definition of computation tree: Each node in (D) gets the same representation as the corresponding node in D, the aggregation is carried out over the same multiset of node features and does not take into account the outgoing neighborhoods of the nodes contained in the multiset. Hence, the representation of the (virtual) output node is the same in both cases, leading to equal DAG representations.
Proof of Lemma 5.1 We recall the definition of the from Lemma 5.1
Big Σ
()(,))), if (
i)
kE
(19)
vo((,
otherwise.
Since the flow attention weights correspond to the attention scores normalized across outgoing edges, it holds that
Bij 1.
tEN
Multiplying on both sides with the sum of v across all incoming edges of node j gives:
Σ
Big
Σ
tha (k
Σ
()(
iE.ow.
kENinG)
kEN
Using the definition of th from Eq.
19 we arrive at Kirchhofl's first law:
Σ
Σ (p(kk)) VjEV (8
16

---

Published in Transactions on Machine Learning Research (09/2025)
Proof of Lemma 5.2 We are given the node update of a flow-attentional GNN from Equation 9 adapted to the multiset X' (5.m) as input:
nnw & [ B1) (hatow.s)
.ES
Since the elements of S have the same features and outgoing neighborhoods 21 and X2, the flow attention weights are the same both cases. Therefore, Φ injective, h/ gets different. results:
Waon((X)) = Φ Σ
(iam(s)((ttooe)))
$22,570
Σ
kBrsm(s)S(haaww.l
L
= Now (22).
Proof of Corollary
We are given the node update of flow-attentional GNN from Equation 9 adapted the multiscts X St m E {1,2] as input:
5.3
hatt (I E)ssm((s)) (haatss))
NEES
with B1s being equal to 1, all nodes have most one outgoing edge in rooted tree. Then, we fulfill the preroquisites of Lemma 5 from Xu et al. (2019), which states that for the right choice of and Ø, any two different multisets can be distinguished. Thus, Anow can distinguish the trees Tj and T2.
B Directed Acyclic GNN Baselines
In the encoder of the D- D-VAE model (Zhang al. 2019), the aggregation corresponds to gated sum using mapping network and gating network and the update function & is gated recurrent unit (GRU)
Cho et al. 2014
m Σ
g(h)) @ m())),
(20)
(e
h/ GRU(h, m!).
(21)
Another popular model is the DAGNN Thost & Chen 2021), which also uses a GRU for the update function but the message function is an attention mechanism with model parameters we and u22
m Σ jEN
aij h/r h/y.
(22)
0iy softmax (w h4 + +w h())
(23)
JENi.(f)
Since the embeddings of the (possibly multiple) end nodes contain information the whole DAG, they are typically used for computing the graph-level representations. After L layers, the graph-level embedding be obtained by concatenating the end node representations from all layers followed by max-pooling across all end nodes:
hgg Max-Pool YEF
h/
(24)
17

---

Published in Transactions on Machine Learning Research (09/2025)
As DAGs are treated as sequences by the above models, they also be processed in reversed order by inverting the edges. Therefore, directed acyclic GNNs are also capable of bidirectional processing. Using the tilde notation to denote node representations in the reverse DAG, the readout function for bidirectional processing can then be expressed as:
hg = FC Maxx-Pool
|| Rt) ||
Max-Pool( JEF ft m3))
(25)
KEI
1=
Note that the representations of the forward and reverse processing are computed independently, which is different from the reverse and forward pass in FlowDAGNN, In all our experiments, we use bidirectional processing for D-VAE and DAGNN.
C
Scoring Functions of Attentional GNN Baselines
In GAT (Veličković et al. 2018), the scoring function is defined ECAT (hi, h2) Leaky (a". (Wh; Why]).
(26)
Thereby, the linear layers a and W are applied consocutively, making it possible to collapse them into a single linear layer.
strictly more expressive attention mechanism is proposed, in which the second linear layer a is applied after the nonlincarity:
In GATv2 (Brody et al. 2022
eGATV2 (h; h2) a LeaaLe ReL. (W. [h; h,])
(27)
Thus, GATv2 is effectively using multi-layer perceptron (MLP) to compute the attention scores, allowing for dynamic attention compared to the static attention performed by GAT.
Finally, Trams (TC) (Shi et al. 2021 is transferring the attention mechanism of the Transformer
model [Vaswani et al. 2017) to graph learning:
q. WGh; + by
(28)
kj Wih; + bb,
(29)
erc (h,, h,) = 9. kj
(30)
Vd
where qs E Rdd is the query vector, ky Rd is the key vector and Wy. We. by. by are trainable parameters.
All of the above scoring functions can be extended to multi-head attention and can incorporate edge features as well. Furthermore, it is possible to include self-loops. These characteristics are naturally inherited by the corresponding FlowGNNs.
DAGs and Computation Trees
Fig. 3 again shows the two different DAG structures from the examples in Fig. [ together with their corre- sponding computation trees defined in Section 3 Although the two DAGs are different, they have the same computation tree. Since standard attention weights they are the same for both DAGs. However, the flow allention weights are obtained by normalizing across outgoing neighbors. Since the grey and blue nodes (colors represent distinct node features) exhibit different outgoing neighborhoods in the two DAGs, the flow attention weights are different. Thus, a flow-attentional directed acyclic GNN can distinguish the two DAGs while a standard attentional one cannot.
computed by normalizing over incoming neighbors,
E
Example Circuit Motivating the Reverse Pass in FlowDAGNN
Fig. shows an example for an electronic circuit modeled as DAG, which motivates the necessity for the pass in FlowDAGNN. If FloNDAGNN would only compute node representations via the forward
18

---

Published in Transactions on Machine Learning Research (09/2025)
Computation Tree
DAG
with Attention Weights
with Flow Attention Weights
0.6
0.4
1.0
1.0
1.0
L1.
1.0
1.0
1.0
LO
0.5
0.5
0.6
0.4
1.0
1.0
1.0
1.0
1.0
1.0
Figure Two non-isomorphic DAGs together with their corresponding computation trees, which are equivalent. Distinct node features are visualized by different colors. The middle and right columns show example standard and flow attention weights. While the standard attention weights always the same for both DAGs, the flow attention weights are different.
pass, the flow attention weight Bg would only depend on all ancestors node This means that the edge from In to R in the upper branch would receive the same flow attention weight as the edge from In R in the lower branch. However, since Ri R2, the electrical current in the upper branch would be much smaller than the current in the lower branch. Therefore, FlowDAGNN would not be capable of modeling the electrical current flow via the flow attention weighls. Only if the reverse pass is applied before the forward pass, the flow attention weights can also be conditioned on the descendants of node i.
R
R
In
Out O
R < 2
R
Figure 4: A simple example circuit described as a DAG, which explains why the reverse pass necessary in FlowDAGNN. Without the reverse pass, the flow atlention weights from the input node to the Rú nodes would be identical, whereas the electrical current flow is different due to R & R2.
F Details on PowerGraph and Ckt-Bench101
PowerGraph. The PowerGraph dataset (Varbella et al., 2024) contains four different transmission systems (IEEE24, IEEE39, 1EEE118, UK) with unique graph structures. For the cascading failure analysis, each test system was simulated for different operating conditions torether with specific initial outage, resulting in
19

---

Published in Transactions on Machine Learning Research (09/2025)
Table 3: Number of nodes and edges for each test system as well the number of corresponding graph samples contained in the PowerGraph dataset (sce Varbella et al. (2024).
Test system
No. Nodes
No. Edges
No. Graphs
IEEE24
24
38
21500
IBEE39
39
46
28000
IEEE118
118
186
122500
UK
29
99
64000
Table Distribution of the classification labels for each test system in the PowerGraph dataset (see Varbella et al. (2024)). ). DNS stands for 'demand not served" and e.f. stands for "cascading failure', corresponding to at least tripping branch after the initial outage.
Category DNS > MW DNS MW
Category
Category C
Category D
DNS 0 MW
DNS 0 MW
Test system
f.
no c. f.
c. f.
f.
IEEE24
15.8%
4.3%
0.1%
79.7%
IEEE39
0.55%
8.4%
0.45%
90.6%
IEEE118
>0.1%
5.0%
0.9%
93.9%
UK
3.5%
0%
3.8%
92.7%
large number of graph samples. The number of nodes and edges in each test system well as the number of graph samples are reported in Tab. 3
Tab. 4 shows how the classification labels are distributed the PowerGraph dataset for each test system. Due to the strong class imbalance, the balanced accuracy BA is used as the evaluation metric (Brodersen) et al. 2010), which is defined as the mean of sensitivity and specificity:
1 (
TN 2 TP+FN + TN+
TP
BA BA
(31)
Here, IP/EP/TN/EN represent true/false
predictions.
CktBench-101. The CktBennch1101 dataset from the Open Circuit Benchmark Dong et al. 2023 contains 10,000 artificially generated operational amplifiers represented as DAGs. Fig. 5 shows the distribution the number of nodes and the number of edges among all graphs in the dataset. The average number of nodes is 9.6 with standard deviation of 2.1. The average number of edges is 14.5 with standard deviation of 5.3. We are using the most recent update of the CktBench-101 dataset, which does not contain any failed simulations anymore.
G Additional Experiments
G.1 Correlation between (Flow) Attention Weights
Fig. 0 shows the Pearson correlation between the (flow) attention weights of different models trained on the four test systems contained in the PowerGraph dataset. Models with the same type of attention (e.g., GAT and GATv2, FlowGAT and Flow'IC) exhibit small medium positive correlations (0.17 0.56) between their attention weights. On the other hand, while models with different types of attention (c.g., GAT and FlowGAT, or GATv2 and FlowTC) do not show any considerable correlations on the IEEE24 and UK test systems, small negative correlations can be observed between attention and flow attention weights on the IEEE39 and IEEE118 test systems. These small negative correlations can be explained by a higher number of links between nodes with very different node degrees, resulting small standard attention weights and large flow attention weights vice versa, depending the edge direction. Apart from that, the difference
20

---

Published in Transactions on Machine Learning Research (09/2025)
1750
1500
800
1250
600
Counts
Counts
1000
750
400
500
200
250
15
10
15
20
25
30
35
Number of Nodes
Number of Edges
Figure 5: Distribution of the number of nodes (left) and number of edges (right) within the Ckt-Bench101 dataset (Dong et al. 2023
IEEE24
IEEE39
GAT
0.46
-0.1
-0.09
0.18
0.35
-0.06
-0.07
GATv2
0.26
0.33
-0.07
0.04
0.02
1.0
0.44
-0.16
-0.15
-0.25
0.75
TC
0.46
0.33
1.0
-0.13
-0.0
-0.03
0.35
0.44
1.0
-0.27
-0.27
-0.33
0.50
FlowGAT
.0.1
-0.07
.0.13
1.0
0.12
0.31
-0.06
-0.16
.0.27
0.59
0.39
FlowGAT
Pearson correlation
0.25
FlowTC
-0.09
0.02
-0.03
0.31
0.28
1.0
-0.07
-0.25
-0.33
0.39
0.29
1.0
IEEE1S
UK
0.000
GAT
1.0
0.56
0.35
-0.15
-0.19
-0.01
1.0
0.43
0.17
-0.05
-0.02
-0.11
-025
GATv2
0.56
0.39
-0.16
-0.24
-0.06
1.0
0.23
-0.09
-0.03
-0.02
TC
0.18
0.23
0.12
FlowGAT
-0.15
-0.16
-0.22
1.0
0.51
0.34
-0.05
-0.09
-0.03
1.0
0.41
0.32
FlowGATv2
-0.19
-0.24
-0.16
0.51
1.0
0.42
-0.02
-0.03
0.03
0.41
1.0
0.36
-0.75
FlowTC
0.12
0.36
-1.00
GAT
GATv2
TC
GAT
FAT
FlowTC
GAT
GATv2
TC
FlowGAT
Flo
FlowTC
Figure 6: The Pearson correlation between the (flow) attention weights of different models on the four test systems from the PowerGraph dataset, which were also used in the experiments in Sec.
21

---

Published in Transactions on Machine Learning Research (09/2025)
Table 5: Test set balanced accuracy (%, f) and Fl-score (macro-averaged, %,,) f) of state-of-the-art Graph Transformer models in standard configuration for the cascading failure analysis (multiclass classifi- cation) four different power grid test systems from the PowerGraph dataset. Reported results represent the average over five training runs with different random seeds along with the standard deviation. The best result for cach test system is marked in bold.
IEEE24
IEEE39
IEEE118
UK
MODEL
Bal. Acc. (t)
Macro (t)
Bal. Acc. (t)
Macro F1 (1)
Bal. Acc. (f)
Macro F1 (f)
Bal. Arc. (t)
Macro F1 (t)
SAT
99.6 + 0.3
99.0 0.6
99.4 + 0.2
98.8 + 0.5
99.8 + 0.1
99.6 = 0.1
99.4 0.1
98.9 + 0.3
GPS
99.2 =
98.5 0.4
98.6 0.4
97.4 + 0.7
99.8 + 0.1
99.5
99.2 0.1
98.6 0.3
Exphormer
99.3 = 98.8 0.5
98.1 0.8
97.2 + 1.5
99.0 = 0.6
98.9
98.7 0.4
98.3 0.3
Table 6: Test set results for flow-attentional GNNs and their standard-atttenioonal base models on popular standard graph datasets beyond the flow graph context. The performances are averaged over 10 runs with different random sceds. The best results for cach dataset are marked in bold.
Cora
CiteSeer
PubMed
ogbg-molhiv
MODEL
Acc. (%, f)
Aec. (%, t)
Acc. (%, t)
ROC-AUC (%.
GAT
81.2 + 1.1
69.1 +
76.6 +
74.9 3.7
FloWGAT
80.3 + 0.8
67.7 + 1.3
77.4 + 0.9
75.9 0.6
GATv2
79.6 + 1.4
69.6 +: 0.7
77.4 +: 0.4
74.5 + 1.9
F
80.4 1: 1.0
67.7 +: 1.9
76.7 +: 1.1
75.8 +: 1.5
TC
78.2 + 1.3
67.3 + 1.2
75.9 + 0.7
77.5 + 0.7
FlowTTC
77.3 + 0.6
67.3 + 1.1
74.6 + 0.5
77.7 + 0.7
in normalization causes the flow-attentional GNNs to learn attention patterns different to those of their corresponding base models, resulting in performance improvements as demonstrated by our experiments.
G.2 Transformers on PowerGraph
Graph Transformers have recently emerged as a promising alternative to messago-passing GNNs (hehhzdd et al. 2024 These models include global attention modules addition local message-passinng layers. This enables a better modeling of long-range interactions and reduces over-smoothing. While showing better empirical performance across many graph datasets and learning tasks, the quadratic complexity of global attention mechanisms limits their scalability to larger graphs (Behrouz & Hashemi 2024 c.g., real-world power grids with thousands of nodes, particularly for real-time decision making.
In Tab. 5 we report additional experimental results on the PowerGraph dataset (cascading failure analysis) with state-of-the-art. Transformer models for graph learning: SAT (Chen et al., 2022), GraphGPS [Ram- pášek et al. 2022), and Exphormer (Shirzad et al. 2023). Thereby, we train all three models with 3 layers and a hidden dimension of 32, and adopt the remaining hyperparameters from the original publications. On all test systems, these models perform considerably better compared all message-passing GNNs listed in Tab. [ However, the superior performance comes the cost of increased computational complexity. Depending the task at hand, it is therefore still very valuable achieve high performance with pure local pessago-passing GNNs, such as the How-attentional GNNs.
Additionally, note that many Graph Transformers rely on message-passnng GNNs to learn local node repre- sentations. Therefore, they could also be used with flow-attentional GNN, e.g., FlowGAT, FlowGATv2, or FlowTC. In future work, we want investigate the usage of flow-attentional GNNs compared to other local message-pass paassggg GNNs within Transformer frameworks like GraphGPS, since we believe that their performance flow graph datasets could be further improved in this way.
22

---

Published in Transactions on Machine Learning Research (09/2025)
G.3 Experiments Standard Graph Datasets
In Sec. we argued that flow-attention might be less effective on graph datasets outside of the flow graph context, because these graphs do not require an information distribution across outgoing neighbors. In addition to that, FlowGNNs might map non-isomorphic but equivalent informational graphs to different representations due to Theorem 5.4 On the other hand, flow-altention could be beneficial on graphs with repetitive node features (e.g., molecular graphs) due to Lemma 5.2
To complement this discussion, we performed additional experiments with flow-attentional GNNs on standard graph datasets, inchuding node classification on the widely used citation networks Cora (McCallum et al, 2000,, CiteSoer (Sen et al., 2008), and PubMed (Namata et al.) 2012). as well as graph classification on the molecular property prediction dataset ogbg-molhiv (Hu et al. 2020 Thereby we compared FlowGAT, FlowGATv2, and FCo to the corresponding slandard-atllentionss baselines.
For all datasets, we utilized public data splits and optimized hyperparameters the validation set using the TPE algorithm from Optuna (Akiba et al., 2019). For the ogbg-molhiv, we used sum pooling followed by a multi-layer perceptron for graph-level readout. We report the test set results for the best hyperparameter configurations of each model in Tab. G.3 On the citation networks, the performance of flow-attentional GNNs is slightly lower than that of their standard-atteentioaal baseline in most cases. In contrast, they perform slightly better compared their standard-attentional counterparts ogbg-molhiv. These observations consistent with discussion the applicability of FlowGNNs beyond flow graphs in Sec. 5.4
G.4
Efficiency Comparison
4.5
Training
Inference
17.5
Training
Inference
15.0
Aveage Epoch Time 69
3.5
Avenge Epoch Time
12.5
3.0
2.5
10.0
2.0
7.5
5.0
1.0
0.5
0.0
GAT
FlawGAT
GAYv2
FlowGATv2
TC FlowTC
PACE
FlowDAGNN
DAGNN
D-YAE
Figure 7: The training and inference times for processing the training set the IEEE24 dataset from the PowerGraph benchmark, averaged over 10 runs.
Figure The training and inference times for processing the training of the CktBench-101 dataset from the Open Circuit Benchmark, aver- aged over 10 runs.
Undirected Graphs. We compare the average training and inference times of the considered models for processing the training set of the IEEE24 dataset from the PowerGraph benchmark using a batch size of 64 (see Fig. 77. We do not observe significant differences in computational efficiency between any flow- attentional GNN and its standard-attentional counterpart.
DAGs. We compare the average training and inference times of the considered directed acyelic GNN models for processing the training set of the CktBench- 101 dataset using a batch size of 64 (see Fig, S) Thereby, we compare our implementation of FlowDAGNN against the original implementations from the authors of the baseline models. While PACE the most efficient model, FlowDAGNN only shows slightly higher training time. DAGNN and D-VAE (using bidirectional processing, App. B appear to be slightly less
23

---

Published in Transactions on Machine Learning Research (09/2025)
efficient than FlowDAGNN. Note that these differences could be caused not only by architectural but also by implementation differences.
All efficiency experiments were carried out on NVIDA V100 GPUs.
24


Reviews:
[{"review_id": "4Qmxkyq0od", "reviewer": "Reviewer_S68T", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The problem tackled in this paper is essential in graph neural networks and corresponding applications.\n2. The theoretical analysis provided in the paper shows the effectiveness of the paper to some extent.\n\nWeaknesses:\n1. The proposed approach is compared with some standard attention-based GNNs, including GAT, GATv2, and Graph Transformer. The authors are suggested to compare the proposed approach to other standard message-passing GNNs, e.g., those released after 2023.\n2. GNNs built upon partial/ordinal differential equations (P/ODE) can also capture the flow properties in the graph. How does the proposed flow attention perform when compared with these P/ODE GNNs?\n3. Some case studies should have been presented in the paper to clearly show the properties of the proposed flow attention. It would be great if the authors could show examples from real-world test datasets that can demonstrate how the proposed flow attention is different from standard attention mechanisms and other message-passing paradigms when learning representations. The reasons why the proposed flow attention is better than existing approaches can also be demonstrated with examples.\n4. Theoretical analysis shows that the proposed flow attention is more expressive than standard graph attention mechanisms in DAG cases, which limits the contributions of the paper.\n5. The overall design of flow attentions is simple despite being effective (row normalization in standard attentions versus column normalizations in flow attentions). The authors are suggested to find other ways to introduce this new attention mechanism and manifest its significance.\n6. The generalizability/learning capabilities of the proposed flow attention mechanism are not tested with other types of graph data.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents flow attention, a new attention mechanism for graph neural networks. Flow attention is designed to capture the conservation laws in graphs so that the learned representations can be implanted with physical resource flows such as electric circuits or traffic networks. Theoretical analysis is provided to show flow attention can identify cases that standard attention cannot. Experimental results show the proposed flow attention performs well on electronic circuit and power grid datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The authors are suggested to address the points listed under \"Weaknesses.\"", "broader_impact_concerns": "NIL.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KY6UlyyBjy", "reviewer": "Reviewer_VkJn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The proposed mechanism is theoretically well grounded. It is shown that the proposed attention mechanism can distinguish neighborhoods with the same distribution (which cannot be distinguished by existing mechanisms). It is also shown that the FlowDAGNN model can distinguish non-isomorphic DAGs which cannot be distinguished by existing GNNs for DAGs.\n\n- The empirical results demonstrate that the proposed mechanism is effective. The proposed approach improves the performance of the base GNN models on the PowerGraph datasets. Furthermore, FlowDAGNN outperforms all the baselines on the Ckt-Bench101 dataset.\n\n- The paper is well-written and easy to read. The introduction section builds the necessary motivation and context for the work.\n\nWeaknesses:\n\n- Some qualitative results are missing from the paper. The authors could extract and visualize the learned attention coefficients on some toy dataset to validate the proposed method. It is also not clear whether the learned attention weights are correlated with those of the base models such as GAT or GATv2.\n\n- It is not clear whether GNN models equipped with the proposed attention mechanism can be applied to standard node and graph classification or regression datasets (e.g., citation networks, molecules, etc), and whether those models would be effective or not. Some empirical results are missing from the paper.\n\n- The current baseline methods (in Table 1) appear to be outdated. Including more recent and competitive baselines would strengthen the paper and is also important to validate the significance of the results.\n\n- Even though it is clearly mentioned that the mechanism is inspired by Kirchhoff's first law, some further explanation is needed for why attention scores are normalized across outgoing edges. Couldn't these normalized coefficients be learned by some model that doesn't normalize the coefficient at all?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents an attention mechanism for graph learning tasks, which is inspired by Kirchhoff's first law. The main difference between this attention mechanism and previous mechanisms is that it normalizes attention scores across outgoing neighbors instead of incoming ones. The authors show that there exists scenarios where the proposed mechanism is more expressive than existing mechanisms (e.g., neighborhoods with the same distribution). The mechanism is also extended to directed acyclic graphs (DAGs) and the FlowDAGNN model is proposed. The proposed mechanism is evaluated on power grids and electronic circuit datasets that contain undirected graphs and DAGs, where it improves the performance of the base GNNs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The requested changes listed below are related to the aforementioned weaknesses, and they are essential to meet the standards required for acceptance:\n\n- Compare the learned attention scores against those learned by existing models such as GAT or GATv2. Train the model on some toy dataset and visualize the learned attention coefficients to validate that nodes can actually attend to their most important neighbors.\n\n- Discuss whether GNN models, such as GAT, equipped with the proposed attention mechanism can be applied to standard graph learning datasets, and conduct experiments on a few datasets (e.g., node classification or graph classification).\n\n- Compare the proposed method against more recent baselines that reflect the current state-of-the-art.\n\n- Provide some further explanation about the proposed attention mechanism and why the authors chose to normalize scores across outgoing edges.", "broader_impact_concerns": "There are no concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6mSK6Ut4KD", "reviewer": "Reviewer_4Amk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. The motivation of this research is intriguing, as the flow mechanism is important to understand the nature of many real graphs.\n2. The paper is generally written well. It is easy to understand as it is self-contained and equipped with good visualizations such as Figure 1.\n3. The proposed method shows good empirical performance on various datasets.\n\nWeaknesses\n1. Most GNNs are designed for undirected graphs. If we naturally extend them to directed graphs, I think Eq. (2) should take the information of outgoing neighbors as well as incoming neighbors, possibly with a simple way to integrate the two types of messages.\n2. Regarding W1, you may consider GNNs designed specifically directed graphs to strengthen your claim:\n    1. https://arxiv.org/abs/2004.13970\n    2. https://proceedings.neurips.cc/paper/2020/hash/cffb6e2288a630c2a787a64ccc67097c-Abstract.html\n    3. https://papers.neurips.cc/paper_files/paper/2021/file/e32084632d369461572832e6582aac36-Paper.pdf\n3. Regarding W1, if we change the definition of a GNN to taking the information of outgoing neighbors as well, is Lemma 4.1 satisfied?\n4. The experiments in Section 6.1 seem irrelevant to the main claim of the paper, as the datasets are undirected. There are few mentions on undirected graphs in the method section.\n5. Graph transformers are being widely studied, and there are many recent models that are designed to address the limitations of early versions, e.g., dense attention, computational complexity, etc. I believe these models should be included in this work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Existing GNNs do not consider the conservation law inherent in graphs associated with a flow of physical resources, such as electrical current in power grids or traffic in transportation networks. To address this, this work proposes flow attention, which adapts existing graph attention mechanisms to satisfy Kirchhoff’s first law. The authors also discuss how this modification influences the expressivity and identify sets of non-isomorphic graphs that can be discriminated by flow attention but not by standard attention.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the weaknesses above and revise the paper accordingly.", "broader_impact_concerns": "I have no concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)