Port-Hamiltonian Architectural Bias for Long-Range Propagation in Deep Graph Networks
Authors: Simon Heilig
Alessio Gravina
Alessandro Trenta
Claudio Gallicchio
Davide Bacciu
Venue: ICLR 2025 Poster (2025)
Keywords: graph representation learning
long-range propagation
ordinary differential equations
Primary area: learning on graphs and other geometries & topologies

Abstract: The dynamics of information diffusion within graphs is a critical open issue that heavily influences graph representation learning, especially when considering long-range propagation. This calls for principled approaches that control and regulate the degree of propagation and dissipation of information throughout the neural flow. Motivated by this, we introduce port-Hamiltonian Deep Graph Networks, a novel framework that models neural information flow in graphs by building on the laws of conservation of Hamiltonian dynamical systems. We reconcile under a single theoretical and practical framework both non-dissipative long-range propagation and non-conservative behaviors, introducing tools from mechanical systems to gauge the equilibrium between the two components. Our approach can be applied to general message-passing architectures, and it provides theoretical guarantees on information conservation in time. Empirical results prove the effectiveness of our port-Hamiltonian scheme in pushing simple graph convolutional architectures to state-of-the-art performance in long-range benchmarks.

Published as a conference paper at ICLR 2025
PORT-HAMILTONIAN ARCHITECTURAL BIAS FOR LONG-RANGE PROPAGATION IN DEEP GRAPH NET- WORKS
Simon Heilig' Alessio Gravina Alessandro Trenta", Claudio Gallicchio", Davide Bacciu²
Ruhr University Bochum, Department of Computer Science, Germany University of Pisa, Department of Computer Science, Italy
ABSTRACT
The dynamics of information diffusion within graphs is a critical open issue that heavily influences graph representation learning, especially when considering long- range propagation. This calls for principled approaches that control and regulate the degree of propagation and dissipation of information throughout the neural flow. Motivated by this, we introduce nort-Hamiltonie Deep Graph Networks, a novel framework that models neural information flow in graphs by building on the laws of conservation of Hamiltonian dynamical systems. We reconcile under single theoretical and practical framework both -issiisiie long-ran rangraage propagation and non-conservative behaviors, introducing tools from mechanical systems to gauge the equilibrium between the two components. Our approach can be applied to general message-passingg architectures, and it provides theoretical guarantees on information conservation in time. Empirical results prove the effectiveness our nort-Hamiltonian scheme in pushing simple graph convolutional architectures state-of- sice-of-the-aae art performance in long-range benchmarks.
INTRODUCTION
The conjoining of dynamical systems and deep learning has become a topic of great interest recent years. In particular, neural differential equations (neural DEs) demonstrate that neural networks and differential equations are two sides of the same coin (Haber Ruthotto 2017 Chen et al. 2018 Chang et al. 2019). This connection has been pushed to the domain of graph learning (Bacciu et al. 2020: Wu et al. (020). forging the field of diffcrentiallequations inspired Deep Graph Networks
(DE-DGNs) (Poli et al. 2019 Chamberlain et al. 2021a Gravina al.) 2023
Han et al. 2024
Gravina et al. 2025a).
In this paper, are interested in designing the information flow within graph solution of port-Hamiltonian system (Van der Schaft, 2017). which is general formalism for physical systems that allows for both conservative and non-conservative dynamics, with the aim of allowing flexible long-range propagation in DGNs. Indeed, long-range propagation is an ongoing challenge that limits the power of the Message-Pussing Neural Network (MPNN) family (Gilmer et al. 2017). as their capacity to transmit information between nodes exponentially decreases as the distance increases (Alon Yahav 202T Di Giovanní et al. (023). This prevents DGNs from effectively solving real-world tasks, e.g., predicting anti-bacterial properties of peptide molecules (Dwivedi jet al. 2022). While recent literature proposes various approaches to mitigate this issue, such as graph rewiring (Gasteiger et al. 2019 Topping et al. [022] Gutteridge et al.) 2023) and graph transformers (Shi et 2021 Dwivedi & Bresson 2021 Wu et al. 2023), here we aim to address this problem providing theoretically grounded framework through the prism of port- nort-Hmiltonian inspired DE-DGNs. Therefore, we propose port- Hamiltaallon Deep Graph Network (PH-DGN) a new message-passing scheme that, by design, introduces the flexibility balance non-dissipative long- range propagation and non-conservative behaviors as required by the specific task at hand. Therefore, when using purely conservative dynamics, our method allows the preservation and propagation of
"Equal Contribution. Correspondence: immon.eiiligee ruuh.de, alessio. wwwisioogravinadeeem ieriiirravii@ddiiunniii. @di.unipi.it

---

Published as a conference paper at ICLR 2025
long-range information by obeying the conservation laws. In contrast, when method is used to its full extent, internal damping and additional forces can deviate from this purely conservative behavior, potentially increasing effectiveness in the downstream task. To the best of our knowledge, we are the first to propose a sont-Hamilooiin-insphhee systems, we provide theoretical guarantees that information is conserved over time. Lastly, the general formulation of our approach can seamlessly incorporate any neighborhood aggregation function (i.e., DGN), thereby endowing these methods with the distinctive properties of our PH-DGN.
DE-DGNs. Leveraging the connection with Hamiltonian
Our main contributions can be summarized as follows: (i) We introduce PH-DGN, a novel general DE-DGN inspired by port-Hamiltoniar dynamics, which enables the balance and integration of non- dissipative long-range propagation and non-conservative behavior while seamlessly incorporating the most suitable aggregation function; (ii) We theoretically prove that, when pure conservative dynamic is employed, both the continuous and discretized versions of our framework allow for rong-range propagation in the message passing flow, since node states retain their past; (iii) We introduce tools inspired by mechanical systems that deviate from such conservative behavior, thus facilitating clear interpretation from the physics perspective; and (iv) We conduct extensive experiments to demonstrate the benefits of our method and the ability stack thousands of layers. Our PH-DGN outperforms existing state-of oftte-o---h-rrr the-art methods on both synthetic and real-world tasks.
PORT-HAMILTONIAN DEEP GRAPH NETWORK
We consider the problem of learning node embeddings for a graph G = EV, where V is a set of entities (the nodes) interacting through relations (i.e., edges) in & C V x V. Each node E V is
associated to state x (4) E Rd, that is the representation of the node at time L. The term X() E R the matrix of all node states in graph G.
We introduce new DE-DGN framework that designs the information flow within a graph as the solution of Hamiltanmnn system (Van der Schafi 2017). Hamiltonian mechanics is a formalism for physical systems based on the Hamiltonian function (P,, q, t), which represents the generalized energy of the system with position q and momentum p. classic example of a Hamiltonian system is that of a simple mass-spring pendulum, with mass attached to a spring with constant k having position q = x and momentum p = mm. The Hamiltonian of the system is the total energy
fmaā and P the spring potential component P krz. Hamilton's 's equations then defined as:
H = K + P where K is the kinetic component K
OH 0g
OH
= kæx q Op
(I)
from which we recover the well-known mass mass-npping pendulum equation mä -kr. The Hamilto- nian formalism allows for an easy description of the dynamics of system based on its energy and provides theoretical results that will allow to guarantee relevant properties our system, such as energy preservation. It is widely used both in classical mechanics (Arnold et al., 2013) as well as in quantum mechanics (Griffiths & Schroeter 2018) due its generality. In the port-Hamiltonian formulation, the system allows for energy exchange between subsystems and interaction with external environments. Therefore, Hort-tamltonnan systems let us introduce non con-conservative phenomena in the system, such as internal dampening D(q)p and external forcing F(g. t), acting on the momentum equation as p = 8H D(g)p + F(q. t).
Here, we show how the port- Harttaamilonian formulation provides the backing to preserve and propagate long-range information between nodes the absence of non-conservative behaviors, thus in adherence to the laws of conservation. The casting of the system in the more general, full Hami-toaitnn setting, then, introduces the possibility of trading non-dissipation with non-conservative behaviors when needed by the task at hand. Our approach is general, as it can be applied to any message- passing DGN, and frames in a theoretically sound way the integration of non-dissipative propagation and non-conservative behaviors. In the following, we refer to our framework as port- nomm-laamitonian Deep Graph Nenwork (PH- (PH-DGN). Figure shows our high-level architecture hinting at how the initial state of the system is propagated up to the terminal time T. While the state evolves preserving energy, internal dampening and additional forces (in the following denoted as driving forces) can intervene to alter its conservative trajectory.
2

---

Published as a conference paper at ICLR 2025
Initial Contition
Energy preservation
Final Embedding
Internal Dampening
(e)
((+1)
P
Pu
u
D
T
Nu
X,
and
g
Xu
and
(E)
(e+1)
qu
qu
External
Forcing
N
Fly,
Continuous time
T
t+1
Layers
L
Figure 1: A high-level overview of the proposed HarttHamiltonian Deep Graph Network.
marizes how the initial node state x, (0) is propagated by means of energy preservation up until the terminal time T (i.e., layer L), x, (T). While the global system's 's state y evolves preserving energy, external forces (i.e., dampening Dyy) and external control Fly, t)) can intervene to alter its conservative trajectory. The gray trajectories between the initial and final states represent the continuous evolution of the system. The discrete message passing step from layer & to & 1, which is shown in middle of the figure, is given by the coupling of coordinates q and momenta p terms of neighborhood aggregation &g and influence to adjacent neighbors Φz. Self-influence on both q and p from the previous step are omitted for simplicity.
In the following, we present our method in bottom-up fashion. Thus, we start by deriving our PH-DGN from purely conservative system, proving its conservative behavior theoretically, and then extend it by integrating non-conservative behaviors.
Conservative message passing. To inject purely conservative behavior inspired by port- Hamiltoninan dynamics into a DE-DGN, we start by considering the graph Hamiltonian system described by the following ODE
dy(() JVH((y())) dt
for time / E [0, T] and subject to an initial condition y(0) = y The term y() E Bnd is the vectorized view of X(t) that represents the global state of the graph at time I, with even dimension d, following the notation of Hamiltonian systems (Hairer et al. 2006). Hg Ind - R a neural- parameterized Hamiltonian function capturing the energy of the system. The skew-symmetric matrix
J J= 0 -ndd/2
-[(nn//2 of the gradient WHg and couples the position and momentum of the system.
0 with In4/2 being the identity matrix of dimension nd/2, reflects a rotation
Since we are dealing with a port-Hamiltnnian system, the global state y() is composed by two components which are the momenta, pt) (p1 (t),..., (t)). and the position, q(t) (t)....... (t)), of the system, thus (t) = (p(t), q(t). Therefore, from the node (local) per- expressee spective, each node state is expressed as x, (t) (P. (t), (t)).
Under this local node-wise perspective, Eq. 2 can be equivalently written
(t)
(p.(1) qm
( q. (p()))) q(1)) V p. Hg (p(1), q(1) Yu E V,
With the aim of designing purely conservative port- hort-Hmilttniin system (i.e., driving forces are null, reducing it to Hamiltonian system) based message passing, instantiate the Hamiltonian
3

---

Published as a conference paper at ICLR 2025
function Hg
Holy(t)) = Σ ã (Wx,(()) + (g(((x(t))ENN. + b) ld.
WEV
where o--) is the anti-derivative of monotonically non-decreasing activation function a. My is the neighborhood of node u, and &g is neighborhood aggregation permutation-invariann function. Terms W E RRdxd and E Rd are the weight matrix and the bias vector, respectively, containing the trainable parameters of the system; la denotes vector of ones of length d.
By computing the gradient Vx. xx. Hy((tt) we obtain an explicit version of Eq. (3. which can be rewritten from the node-wise perspective of the information flow the sum of the self-node evolution influence and its neighbor's evolution influence (referred to as 2zz), More formally, for each node
(t) dt
Ju WT WTo o(Wx.(t) + +
(PG((X)(())EEN,) o (Wxx(() + ()((x(())) +b) +b)]
(5)
rEAGU(u) (1,
Here, Tu has the same structure as J, but the identity blocks have dimension d/2 as it is applied to the single node u. Notice that the system in Eq. (5) implements Hamiltonian system, so it adheres solely to conservation laws.
Now, given an initial condition x,, for node u, and the other nodes in the graph, the ODE defined in Eq. (5) is a continuous information processing system over graph governed by conservation laws that computes the final node representation x., (T'). This is visually summarized in Figure) when dampening and external forcing are excluded.
Moreover, we observe that general formulation of the neighborhood aggregation function (t), allows implementing any function that aggregates nodes (and edges) information. Therefore, &g (x. (e)),)) allows enhancing a standard DGN with our Hamiltonian conservation. As demonstration of this, in Section 3) experiment with neighhorhood aggregation functions, which are the classical GCN aggregation (Kipf & Welling 2017) and
Φb (b)(xx(())EEx))
Σ Vx,
vEN.
Further details about the discretization of the purely conservative PH-DGN are in Appendix A.3
Purely conservative PH-DGN allows long-range propagation. We show that our PH-DGN in Eq. (5) adheres to the laws of conservation, allowing long-range propagation in the wessage-passing flow.
As discussed in (Haber & Ruthotto 2017 Gravina et al. 2023), non-dissipative propagation is directly linked the sensitivity the solution of the ODE to its initial condition, thus to the stability of the system. Such sensitivity is controlled by the Jacobian's 's eigenvalues of Eq. (5). Under the assumption that the Jacobian varies sufficiently slow over time and its eigenvalues are purely imaginary. then the initial condition is effectively propagated into the final node representation, making the system both stable and non-dissipative, thus allowing for long-range propagation.
Theorem 2.1. The Jacobian matrix of the system defined by the ODE Eq. (5) possesses eigenvalues purely on the imaginary axis, i.e.,
0 (2. (.Nxx,)(((())) =0,
Re
Vi,
where X; represents the i-th eigenvalue of the Jacobian.
We report the proof in Appendix B.I Then, we take a further step and strengthen such result by proving that the nonlinear vector field defined by conservative PH-DGN is divergence- thus
4

---

Published as a conference paper at ICLR 2025
preserving information within the graph during the propagation process and helping maintain informative node representations. In other words, the PH-DGN's dynamics possess a non-dissipative behavior independently of both the assumption regarding the slow variation of the Jacobian and the position of the Jacobian eigenvalues on the complex plane.
Theorem 2.2. The autonomous Hamiltonian Hg of the system in Eq. (5) with learnable weights shared across time stays constant at the energy level specified by the initial value Hg(y(0)), i.e. dHg/dt O, and possesses divergence-free nonlinear vector field
V V.J.Vx H.(xtt, 0, E [0.7].
See proof in Appendix [.2 This allows us to interpret the system dynamics purely rotational, without energy loss, and demonstrates that our PH-DGN is governed by conservation laws when driving forces are null.
We now provide a sensitivity analysis, following Chang et al. (2019): Gravina et al. (2023) Galimberti
et al. 2023), to prove that our conservative PH-DGN effectively allows for long- rangrrange information propagation. Specifically, we measure the sensitivity of node state after an arbitrary time T of the information propagation with respect to its previous state, [[xx,(T), (T') /0x, (T' f) In other words, we compute the backward sensitivity matrix (BSM). We now provide theoretical bound of our PH-DGN, with its proof in Appendix [33
Theorem 2.3. Considering the continuous system defined by Eq. ( the backward sensitivity matrix (BSM) is bounded from below:
0x,, (T)
2 1, V [0,7].
0x,, (T
We note that since 0x matrix norms, c.g., p-norm and Frobenius norm. The result of [he that the gradients in the backward pass do not vanish, enabling the effective propagation of previous node states through successive transformations to the final nodes' representations. Therefore, whenever driving forces null, PH-DGN has conservative message passing, where the final representation of each node retains its complete past. We observe that Theorem holds even during discretization when the Symplectic Euler method is employed (see Appendix A.3)
- 1) Rdddd our statement is valid for all sub-multiplicative
To give the full picture of the time dynamics of the gradients, we present a similar analysis and provide an upper bound of the BSM in Theorem A. Recently, TTopiigg al. (2022); DD Giovanni al. (2023) proposed to evaluate the long-range propagation ability of a model by measuring the sensitivity of the node embedding after é layers with respect to the input of another node, i.c., ||0x!! bounding such a measure on a MPNN:
(2) /0x! (0) [|.
Oxp
S (cowd) ((c,I )a
(8)
Ox L
(0)
where is the Lipschitz constant of non linearity a,, w is the maximal entry-value over all weight matrices, the embedding dimension, and Gr and ea being the weighted contributions of the residual and aggregation term, respectively. Following a similar amalysis, in Theorem 2.4 we provide a bound for our PH-DGN when Symplectic Euler method used as discretization method.
Theorem 2.4. Considering our PH-DGN discretized via Symplectic Euler method (Eqs. and 13 with neighborhood aggregation function of the form &g = EvEN. Vx.. then
Ox!!
(
I (dwNeo) ((wI + w(N + +1)A))
(9)
Ox
L1
where N max, |. and e. Ca 1 for simplicity.
See proof in Appendix B.6 The result of Theorem 2.4 indicates that our upper bound on the
right-hand side of the inequality is at least ' times bigger than the one computed for an MPNN. Therefore, together with previous theoretical results, it holds the capability PH-DGN to perform long-range propagation effectively.
5

---

Published as a conference paper at ICLR 2025
Introducing dissipative components. Without driving forces, a purely conservative inductive bias forces the node states to follow trajectories that maintain constant energy, potentially limiting the effectiveness of the DGN downstream tasks by restricting the system's 's ability to model all complex nonlinear dynamics. To this end, we complete the formalization of our port-Hamiltoniall framework by introducing tools from mechanical systems, such friction and external control, learn how much the dynamic should deviate from this purely conservative behavior. Therefore, we extend the dynamics in Eq. (5) by including two new terms D(q)= E Rd/2xd/2 and F(q,t) E Rd/22 i.e.
dx..(t) Tn (P(g(t)) 1 Ho(y(()) (Flg(t).t) VVEV.
V
(10)
di
Depending on the definition of D(q(()) we can implement different forces. Specifically, if D(q(O) is positive semi-definite then it implements internal dampening. while negative semi-definite implementation leads to internal acceleration. A mixture of dampening and acceleration is obtained otherwise. In the case of dampening. the energy decreased along the flow of the system (Van der Schaft 2017). To further enhance the modeling capabilities, we integrate the learnable state- and time- dependent external force F(q(t), t), which further drives node representation trajectories. Figure visually summarizes how such tools can be plugged in our framework during node update.
Although (t)) and (() can be implemented as static (fixed) functions, in our experiments in Section 3 we employ neural networks to learn such terms. We provide additional details on the specific architectures in Appendix AA.2 We provide further details about the discrctization of PH-DGN in Appendix A.3 Additional theoremes supporting the long-range propagation capability of our PH-DGN with driving forces are provided in Appendix|B.. B.7
EXPERIMENTS
We empirically verify both theoretical claims and practical benefits of our framework on popular graph benchmarks for long-range propagation. First (Section 3.1). we conduct a controlled synthetic test showing non-vanishing gradients even when thousands of layers are used. Afterward (Section (.2). we run graph transfer task inspired by Di Giovanni et al. (2023) to assess the efficacy in preserving long- rong-range information between nodes. Then, we assess our framework in popular benchmark tasks requiring the exchange of messages at large distances over the graph, including graph property predic- tion (Section (.3) and the long-range graph benchmark (Dwivedi et al.) (202)) (Section (3.4). An addi- tional ablation on the impact of the different dissipative components on the Minesweeper (Luo et al.) 2024) and graph transfer tasks is reported in Appendix D.4 We compare our performance to state-of- the-art methods, such as MPNN-based models, DE-DGNs (including Hamiltonian-inspiree DGNs), higher-order DGNs, and graph transformers. Notice that DE-DGNs represent a direct competitor to method. Additional details on literature methods are in Appendix C.I] We investigate two neigh- borhood aggregation functions for our PH-DGN, which are the classical GCN aggregation and that in Eq. (6) Our model is implemented in PyTorch (Paszke et al. (017) and TyTorch-Geometee (Fey & Lenssen 2019). We release openly the code implementing our methodology and reproducing our empirical analysis at lhttps:/ /ggithubbb.... www. s imonhe / httpg/////aaaa..m 1t onnan- Our experimental results were obtained using NVIDIA A100 GPUs and Intel Xeon Gold 5120 CPUs.
3.1 NUMERICAL SIMULATIONS
Setup. We empirically verify that our theoretical considerations on the purely conservative PH-DGN (i.e.. the driving forces are null) hold true by an experiment requiring to propagate information within a Carbon-60 molecule graph without training on any specific task, i.e., we perform no gradient update step. While doing so, we measure the energy level captured in Hfg(y(EE) in the forward pass , I. in the backward pass. We consider the 2-d position of the atom in the molecule the input node features, fixed terminal propagation time T = 10 with various integration step sizes e E {0.1, 0.01, 0.001] and T = 300 with 0.3. Note that the corresponding number of layers is computed as L T/c, i.e., we use tens to thousands of layers. For the ease of the simulation, we use nomh-nonlinearity fixed learnable weights that are randomly initialized, and the aggregation function in Eq. (6)
and the sensitivity, ||xx! / from each intermediate layer L. 1,...,
Results. In Figure 2a we show the energy difference Hg (f())) - (0) for different step sizes. For a fixed time T, smaller step size is related to a higher number of stacked layers.

---

Published as a conference paper at ICLR 2025
We note that the energy difference oscillates around zero, and the smaller the step size the more accurately the energy is preserved. This supports our intuition of the conservative PH-DGN being a discretization of divergence free continuous Hamiltonian dynamic, that allows for non-dissipative forward propagation, as stated in Theorrm 2.1 and Theorem 2.2 Even for larger step sizes, energy is neither gained nor lost.
Regarding the backward pass, Figures 2b 2c assert that the lower bound [[xx(L)/xx())) 2 stated in Theorem 2.3 and its discrete version in [heorem leads non-vanishing gradients. In particular, Figure 2c shows a logarithmic-linea increase of sensitivity with respect to the distance to the final layer, hinting at the exponential upper bound derived in Theorem A.I This growing behavior can be controlled by regularizing the weight matrices, or by use of normalized aggregation functions, as in GCN (Kipf & Welling 2017).
C-1o01 ... c=1002 16
10
Helyit)) -Hel(())
100|1 1/ox (L11 11
0.8
8
50
0.6
40
30
0.4
1a (8))
20
0.0
0.0
2.5
5.0
7.5
10.0
25
50
75
100
0
500
1000
Time
Layer
Layer
(a)
(b)
(c)
Figure 2: (a) Time evolution the energy difference to the initial state y(0) = yo obtained from one forward pass of conservative PH-DGN with fixed random weights on the Carbon-60 graph with three different numbers layers given by T/e. The sensitivity ||xx!!" /ax(." || of 15 different node states their final embedding obtained by backpropagation
the Carbon-60 graph after (b) T 10 and e == 0.1 (i.e., 100 layers) and (c) T 300 and e = 0.3 (i.e., 1000 layers). The log scale's horizontal line 0 indicates the theoretical lower bound.
3.2 GRAPH TRANSFER
Setup. We build on the graph transfer tasks by Di Giovanni et al. (2023) and consider the problem of propagating label from source node target node located at increasing distances & in the graph, following the setting proposed by Gravina et al. (2025a). We use the same graph distributions in the original work, i.e., line, ring, and crossed-ring graphs. To increase the difficulty of the task, we randomly initialize intermediate nodes with a feature uniformly sampled in [0, 0.5). Source and destination nodes are initialized with labels "1" and "0", respectively. We considered problems distances & E {3, 5, 10, 50}, thus requiring incrementally higher efficacy in propagating long-range information. Given the conservative nature of the task, focus assessing our PH-DGN without driving forces. More details on the task and the hyperparameters can be found in the Appendix C.2 and
Results. Figure 3 reports the test mean-squared error (and std. dev.) of PH-DGN compared to literature models. appears that classical MPNNs not effectively propagate information across long ranges as their performance decreases when lo increases. Differently. PH-DGN achieves low errors even at higher distances, i.e., 2 10. The only competitor to our PH-DGN is A-DGN, which is another non-dissipative method. Overall, PH-DGN outperforms all the classical MPNNs baseline while having on average better performance than A-DGN, thus empirically supporting our claim of long-ra range capabilities while introducing new architectural bias. Morcover, results highlight how our framework can push simple graph convolutional architectures to state-of of-the-art performance when imbuing them with dynamics capable of long-range message exchange.
3.3 GRAPH PROPERTY PREDICTION
Setup. We consider three graph property prediction tasks introduced in Corso et al. (020)) under the experimental setting of Gravina et al. (023). We investigate the performance of port-Hamiltonis
7

---

Published as a conference paper at ICLR 2025
GCN
GAT
GraphSAGE
GIN
A-DGN
PH-DGN
Mean Squared Error
Mean Squared Error
Mean Squared Error
10
10
10
10
10
10-1
10-6
10-5
35 10
35 10
35 10
Source-Target distance (#hops)
Source-Target distance (#hops)
Source-Target distance (#hops)
(a) Line
(b) Ring
Crossed-Ring
Figure Information transfer performance on (a) Line, (b) Ring, and (c) Crossed- graphs.
Overall, baseline approaches are not able to transfer the information accurately as distance increase, while non-dissipative methods like A-DGN and our PH-DGN achieve low errors.
framework in predicting graph diameters, single source shortest paths (SSSP), and node eccentricity on synthetic graphs. Note that effectively solve these tasks, it is fundamental propagate not only the information of direct neighborhoods, but also the information coming from nodes far away in the graph. Therefore, good performance in these tasks highlights long-range propagation capabilities. In this experiment, we investigate the performance of our complete framework (i.e., including driving forces), and present the pure conservative PH-DGN (referred as PH-DGNc) as an ablation study. More details on the task and the hyperparameters can be found in Appendix C.3 and C.5
Results. We present the results on the graph property prediction reporting (Mg1)(MSE) as evaluation metric. We observe that our PH- DGN show strong improvement with respect to baseline methods, achieving new state-of-the-art per- formance on all the tasks. Our ab- lation reveals that the purely con- servative model has, on average, a (MSE) that is 0.33 lower than the best baseline. Such gap is pushed to 0.81 when the full port- Hamiltonian bias (i.e., PH-DGN) is employed, marking significant de- crease in the test loss. The largest gap is achieved by PH-DGN the eccentricity task, where it improves the (MSE) performance of the best baseline by 1.36. Moreover, note that our PH-DGN more ef- fective than its purely conservative version and existing Hamiltonian- inspired DE-DGN, i.e., HamGNN and HANG, highlighting the signif- icance of Hart-tamilnn dynam- ics with respect to purely conser-
tasks
Table
Table Mean test (ogIe(MSE)) and std average over 4 training seeds on the Graph Property Prediction. Our methods and DE- DGN baselines are implemented with weight sharing. The first, second, and third best scores colored.
Model
Eccentricity
Diameter
SSSP
MPNNs GCN
0.84684 0028 0.742440.0166
0,9499-0.000r
GAT
0.7909+0.00 0222
0.822
00.0752
0.695 0.6951 10.199
GraphSAGE
0.7863 10.2007
0.8645 10.0000
0.2863 10.1843
GIN
0.950410.000
0.6131 110. 0990
-0.5408 40.4193
GCNII
0.764040.0355
0.5287 +0. 0570
-1. -1.1329
DE-DGNs
DGC
0.826140.0032
0.602840.00.3
-0, -0.1483=0.0231
GraphCON
0.6833
0.0964
-1.3836 10333600.0000 0.002
GRAND
0.6602+ 1393
0.6715
-0,0942 0.3897
A-DGN
0.4296 1003
-0.5188
10.118
-3.2417 0.00551
HamGNN
0,7851 0.6762
10.621..131
0,9449 10.0008
HANG
0.830210.0051
1.103610.1022
0.167110.0160
Ours
PH-DGN -0.7248 10.1068 -0.5473. 10.1074 -3.0467 10.1615
PH-DGN -0.9348 -0.5385 4.2993
vative inductive bias. This effectiveness is also reflected in the computational cost, as shown in Appendix D.3 where our PH-DGN results to be more efficient both in terms of speed and memory usage compared to HamGNN and HANG.
Although purely conservative PH-DGN shows improved performance with respect to all base- lines, it appears that relaxing such bias via PH-DGN is more beneficial overall, leading to even greater improvements in long-ram anggrange information propagation. Our intuition is that such tasks do not require purely conservative behavior since nodes need to count distances while exchanging more messages with other nodes, similar to standard algorithmic solutions such Dijkstra (1959). Therefore, the
8

---

Published as a conference paper at ICLR 2025
energy may not be constant during the resolution of the task, hence benefiting from the non-purely conservative behavior of PH-DGN.
As for the graph transfer task, our results demonstrate that our PH-DGNs effeetively learn and exploit long-range information while pushing simple graph neural architectures to state-of-the-an performance when modeling dynamics capable of long-range propagation.
3.4 LONG-RANGE GRAPH BENCHMARK
Setup. We assess the performance of our method the real-world long-range graph benchmark (LRGB) from Dwivedi et al. (2022), focusing on the Peptide-fune and Peptide-struct tasks. As in Section 3.3 we decouple our method into PH-DGN and PH-DGNc provide an ablation study on the strictly conservative behavior. For our evaluation, we follow the experimental setting in Dwivedi al.] (2022). Acknowledging the results from Tönshoff et al. (2023), we also report results with a 3-layer MLP readout. While some baselines leverage positional or structural encodings, our approach
does not depend on these mechanisms. More details on the task and the hyperparameters can be and C.5
found in Appendix C.4
Results. We report results on the LRGB tasks in Table 2 (extended results are reported Appendix D.2 due to space limits). Our results show that both PH- DGNç and PH-DGN outperform classi- cal MPNNs, graph transformers, most of the multi-hop DGNs, and recent DE- DGNs (which represent direct competi- tor to our method). Overall, our port- Hamiltonian framework (with and with- out driving forces) shows great benefit in propagating long-range information without requiring additional strategies such as global position encoding (as ev- idenced by comparisons with MPNN- based models using positional encod- ing), global attention mechanisms (as seen in comparisons with Transformer- based models), or rewiring techniques (as shown in comparisons with Multi- hop DGNs) that increase the overall com- plexity of the method. Consequently, our results reaffirm the effectiveness of our framework in enabling efficient long- range propagation, even simple DGNs characterized by purely local message ex- changes. Lastly, we believe that our PH- DGN with driving forces achieves bet- ter performance than PH-DGN, on the real-world LRGB because the learned driving forces act as an adaptive filter mechanism that filters noisy information, facilitating the learning of relevant infor- mation.
Table 2: Results for Peptides-func and Peptides-struct averaged over training seeds. The first, second, and third best scores are colored. Extended version of this table is provided in Appendix [.2] "+PE/SE" indicates the use of positional or structural encoding. We have detailed the type of encoding wherever the original source explicitly specifies it.
Peptides-func AP
Peptides-struct MAE
Model
Modified MPNNs, Tönshoff et al. (2023)
GCN+PE/SE GCNII+PE/SE GINE+PE/SE
0.6860 00.0050
0.2460 00.060=0.0007
0.6444 00011
0.2507=0.000
0.6621 000067
0.2473 000017
GatedGCN+PE/SE
0.6765 000047
0.2077
Multi-hop DGNs, Gutteridge et al. (2023)
DIGL+MPNN+LappE MixHop-GGCN+LLppPT 0.6843 10.0049
0.6830 L0.0026
0.2616_10.0018
0.261410.0023
DRew-GGN+LaapP 0.7150 +0.0044
0.253620.0015
Transformers, Gutteridge et al. (2033
Transformer+Lappe SAN+LapPE GraphGPS+LapPE
0.6326=
0.2529 +0.001
0.6384=0.0121
0.2683
0.653520.0043 20.0041
0.250020.0000 0005
DE-DGNS
GRAND
0.5789 20.0022
0.3418 0000015
GraphCON
0.6022 10.006
0,2778
0.0018
A-DGN
0.5975 10.0044
0.2874
1 0.0021
SWAN
0.6751 0.0033
0.2485
0.0009
Ours
PH-DGNG
0.6961 16961=000070 0.0070
0 +0.0020
PH-DGN
0.7012
0.2465 o 0.0020
RELATED WORKS
DGN based on differential equations. Recent advancements in the field of representation learn ing have introduced new architectures that establish connection between neural networks and dynamical systems. Inspired by pioneering works on recurrent neural networks (Chen et al.] 2018 Haber & Ruthotto 2017 Chang al. 2019: Galimberti al. 2023), such connection has been

---

Published as a conference paper at ICLR 2025
pushed to the domains of DGNs (Han al. 2024). Indeed, works like GDE (Poli et al. GRAND (Chamberlain et al. 202 PDE- GCN (Eliasof et al. (021). DGC (Wag et al. 2021 p.
2
GRAND++ (Thorpe et al. (022) propose to interpret DGNs as discretization of ODEs and PDEs. The conjoint of dynamical systems and DGNS have found favorable consensus, as these new methods exploit the intrinsic properties of differential equations to extend the characteristic of message passing within DGNs. GRAND, GRAND++, and DGC bias the node representation trajectories to follow the heat diffusion process, thus performing a gradual smoothing of the initial node states, On the contrary, GraphCON (Rusch et al. 2022) used oscillatory properties to enable linear dynamics that preserve the Dirichlet energy encoded in the node features; PDE-GCNM (Eliasof et al. 2021) uses an interpolation between anisotropic diffusion and conservative oscillatory properties; more recently, A-DGN (Grvv ina et al. 2023) introduces an anti-symmetric mechanism that leads to node-wise non-dissipative dynamics. Related work in the line of Hamiltonian systems for DGNs, such as HamGNN (Kang et al., 2023). exclusively leverage Hamiltonian dynamics to encode node input features, which are then fed into classical DGNs to enhance their conservative properties. Similarly, HANG (Zhao et al. 2023 leverages Hamiltonian dynamics to improve robustness to adversarial attacks to the graph structure. Differently from HamGNN and HANG, our PH-DGN is (to the best of our knowledge) the first DGN that leverages Hat-Hiiiltoninn dynamics, thus balancing non-dissipative and non-conservative behaviors while providing theoretical guarantees of long-range propagation. A deeper discussion on the differences with HamGNN and HANG is provided in Appendix D.3
While the aforementioned works focus on the spatial aggregation term DE-DGNS the temporal domain has also been studied in works such as Eliasof et al. (2024); Gravina et al. (024aa); Jin et al. (2022) Gravina et al. (0026b).
Long-range propagation on DGNs. Effectively transferring information across distant nodes is still an open challenge in the graph representation learning community (Shi et al., (023). Various strategies have been explored in recent years to address this challenge, including regularizing the model's weight space (Gravina et al., 2023 2025b a). filter messages the information flow (Errica et al. 2024). and graph rewiring. In this latter setting, methods like SDRF (Topping et al.) (022).
GRAND (Chamberlain et al. 2021a), BLEND (Chamberlain et al., 2021b), and DRew (Gutteridge ct al. 2023) (dynamically) alter the original edge set to densify the graph during preprocessing to facilitate node communication. Differently, Transformer-baas
methods (Shi et al. 2021 Dwivedi & Bresson 2021 Ying al.] [202]] Wu et al.) (023) enable message passing between all node pairs.
FLODE (askey et al.] (224)) incorporates non-local dynamics by using a fractional power of the graph shift operator. Although these techniques are effective in addressing the problem of long-range communication, they can also increase the complexity of information propagation due to denser
graph shift operators.
CONCLUSIONS
In this paper, we have presented port-Haniltonaan Deep Graph Network (PH-DGN), a general framework that gauges the equilibrium between non-dissipative long-range propagation and non- conservative behavior while seamlessly incorporating the most suitable neighborhood aggregation function. We theoretically prove that, when pure conservative dynamic is employed, both the con- tinuous and discretized versions of our framework allow for long-range propagation in the message passing flow since node states retain their past. To demonstrate the benefits of including port- Hamiltonian dynamics DE-DGR DE-DGNs, we conducted several experiments on synthetic and real-world benchmarks requiring long- raage interaction. Our results show that our method outperforms state-of- ahe-art models and that the inclusion of data-driven forces that deviate from a purely conservative behavior is often key to maximize efficacy of the approach on tasks requiring long-range propagation. Indeed, in practice, effective information propagation requires a balance between long-term memo- rization and propagation and the ability selectively discard and forget information when necessary. Looking ahead to future developments, our port-Hamillonia dynamic can be extended to handle time- varying streams of graphs (Gravina & Bacciu 2024) and can be evaluated with alternative discretization methods, e.g., adaptive multistep schemes (Rufai et al. 2023).
10

---

Published as a conference paper at ICLR 2025
ETHICS AND REPRODUCIBILITY STATEMENTS
Ethics Statement. In this work, we do not release any datasets or models that could pose a significant risk of misuse. We believe our research does not have any direct indirect negative societal implications or harmful consequences, as we do not utilize sensitive, privacy-related data, nor do we develop methods that could be applied for harmful purposes. As far as we are aware, this study does not raise any ethical concerns or potential negative impacts. Furthermore, our research does not involve human subjects, nor does it employ crowdsourcing methods. We confirm there are no potential conflicts of interest or sponsorship affecting the objectivity or outcomes of this study.
Reproducibility Statement. In Section we outline the setups employed in our experiments, while in Appendix C we provide comprehensive supplementary information, including references to the baselines, detailed dataset deseriptions, the experimental settings for each task, and the peppaaraaeterr grids used in our study. All experiments presented in Sections 3.3 and 3.4 are conducted on publicly available benchmarks. To further support reproducibility, we openly release all the data and code to reproduce our empirical evaluation upon acceptance.
ACKNOWLEDGMENTS
This research was partially supported by EU-EIC EMERGE (Grant No. 101070918), as well as partially funded in the course of TRR 391 Spatio-temporal Statistics for the Transition of Energy and Transport (520388526) by the Deutsche Forschungsgemeinschaff (DFG, German Research Foundation). SH acknowledges funding from ELSA European Lighthouse on Secure and Safe Al (Grant No. 101070617) by the European Union
REFERENCES
Sami Abu-EI-Haijj Hl-Haija Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. Mixhop: Higher-order graph convolutional ar- chitectures via sparsified neighborhood mixing. In International Conference on Machine Learning, pp. 21-29. PMLR, 2019.
Uri Alon and Eran Yahav. On the Bottleneck of Graph Neural Networks and Practical Implications. In International Conference Learning Representations, 2021. URL htttpsss//opeereeiieee net
V.I. Arnold, Vogtmann, and Weinstein. Mathematical Methods of Classical Mechanics. Graduate Texts in Mathematics. Springer New York, 2013. ISBN 9781475716931.
Davide Bacciu, Federico Errica, Alessio Micheli, and Marco Podda. A gentle introduction to deep learning for graphs. Neural Networks, 129:203-221, 2020. ISSN 0893-6080.
Xavier Bresson and Thomas Laurent.
Residual gated graph convnets. arXiv preprint
arXiv:1711.07553, 2017.
Ben Chamberlain, James Rowbottom, Maria I Gorinova, Michael Bronstein, Stefan Webb, and Emanuele Rossi. GRAND: Graph neural diffusion. In International Conference on Machine Learning, pp. 1407-1418. PMLR, 2021a.
Benjamin Chamberlain, James Rowbottom, Davide Eynard, Francesco Di Giovanni, Xiaowen Dong, and Michael Bronstein. Beltrami Flow and Neural Diffusion on Graphs. In M. Ranzato, A. Beygelz- imer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 1594-1609. Curran Associates, Inc., 202
Bo Chang, Minmin Chen, Eldad Haber, and Ed H. Chi. AntisymmetricNNN: A dynamical system view recurrent neural networks. In International Conference Learning Representations, 2019. URL
net/ htt.://wweeet..o.mm..eeeeeeepppo
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and Deep Graph Convolutional Networks. In Hal Daumé III and Aarti Singh (eds.). Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 1725-1735. PMLR, 13-18 Jul 2020.

---

Published as a conference paper at ICLR 2025
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural Ordinary Differential Equations. In S. Bengio, H. Wallach, H. Larochelle. K. Grauman, N. Cesa- cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, and Petar Veličković. Principal Neighbourhood Aggregation for Graph Nets. In Advances in Neural Information Processing Systems, volume 33, pp. 13260-13271. Curran Associates, Inc., 2020.
Shaan A. Desai, Marios Mattheakis, David Sondak, Pavlos Protopapas, and Stephen J. Roberts. Port-hamiltonian neural networks for learning explicit time-dependent dynamical systems. Phys. Rev. E, 104:034312, 9 2021.
Francesco Di Giovanni, Lorenzo Giusti, Federico Barbero, Giulia Luise, Pietro Liò, and Michael Bronstein. On over-squashing message passing neural networks: the impact of width, depth, and topology. In Proceedings of the 40th International Conference on Machine Learning, ICML'23, JMLR.org. 2023.
Edsger. W. Dijkstra. note on two problems in connexion with graphs. Numerische Mathematik, 1: 269-271, 1959.
Vijay Prakash Dwivedi and Xavier Bresson. A Generalization of Transformer Networks to Graphs. AAAI Workshop on Deep Learning on Graphs: Methods and Applications, 2021.
Vijay Prakash Dwivedi, Ladislav Rampášek, Michael Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, and Dominique Beaini. Long Range Graph Benchmark. In Advances in Neural Information Processing Systems, volume 35. pp. 22326 22326-22340.
22340. Curran Associates, Inc., 2022.
Moshe Eliasof, Eldad Haber, and Eran Treister. PDE-GCN: Novel Architectures for Graph Neu- ral Networks Motivated by Partial Differential Equations. In Advances in Neural Information Processing Systems, volume 34, pp. 3836-386 3849. Curran Associates, Inc., 2021.
Moshe Eliasof, Eldad Haber, Eran Treister, and Carola-Bibiane Schönlieb. On the temporal domain of differential equation inspired graph neural networks. In Sanjoy Dasgupta, Stephan Mandt, and Yingzhen Li (eds.), Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, pp. 1792-1800. PMLR, 02-04 May 2024.
Federico Errica, Henrik Christiansen, Viktor Zaverkin, Takashi Maruyama, Mathias Niepert, and Francesco Alesiani. Adaptive message passing: A general framework to mitigate oversmoothing, oversquashing, and underreaching, 2024. URL / /wtpaaaaivv....///aa/eaaeeaaaala www.waiwww..w..aaaaaoo.o..ooooot abs /2lss.2311... 16560
Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop Representation Learning Graphs and Manifolds, 2019.
Clara Lucía Galimberti, Luca Furieri, Liang Xu, and Giancarlo Ferrari-Trecate. Hamiltonian deep neural networks guaranteeing nonvanishing gradients by design. IEEE Transactions on Automatic Control, (8(55)3155-3166 2023.
Johannes Gasteiger, Stefan Weiß enberger, and Stephan Günnemann. Diffusion Improves Graph Learning. Advances in Neural Information Processing Systems, volume 32, Curran Associates, Inc., 2019.
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for Quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of ICML'17, pp.
JMLR.org, 2017.
Alessio Gravina and Davide Bacciu. Deep Learning for Dynamic Graphs: Models and Benchmarks. IEEE Transactions on Neural Networks and Learning Systems, 3178): 2024. doi: 10.1 1001///NNN..
Alessio Gravina, Davide Bacciu, and Claudio Gallicchio. Anti- Symmetrciiiic DGN: stable architecture for Deep Graph Networks. In The Eleventh International Conference on Learning Representations, 2023. URL /ttttp::/oppnreette ew. net. hww..eet..ooormm.a---37///77////
12

---

Published as a conference paper at ICLR 2025
Alessio Gravina, Giulio Lovisotto, Claudio Gallicchio, Davide Bacciu, and Claas Grohnfeldt. Long Range Propagation on Continuous- Time Dynamic Graphs. In Proceedings of the 41st International Conference Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 16206-16225. PMLR, 21-27 Jul 2024a.
Alessio Gravina, Daniele Zambon, Davide Bacciu, and Cesare Alippi. Temporal Graph ODEs for Srrguuarry-Samplle Time Series. In Kate Larson (ed.), Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, pp. 4025 4025-4034. International Joint Conferences on Artificial Intelligence Organization, 8 2024b, doi: 10.2490 10.24965/jcca.22244445 24tpg//....943/jaa.... 2024//4/4222244444 2024/445
URL /tt.pppp///oo.ooo///2222////1/..
Main Track.
Alessio Gravina, Moshe Eliasof, Claudio Gallicchio, Davide Bacciu, and Carola-Bibiana Schönlieb. On Oversquashing in Graph Neural Networks Through The Lens of Dynamical Systems. In The 39th Annual AAAI Conference on Artificial Intelligence, 2025a.
Alessio Gravina, Claudio Gallicchio, and Davide Bacciu. Non- Non-Dissipative Propagation by Random- ized Anti- Symmetric Deep Graph Networks. In Machine Learning and Principles and Practice of Knowledge Discovery in Databases, pp. 25-36, Cham, 2025b. Springer Nature Switzerland. ISBN 978-3-031--4644-77
David J. Griffiths and Darrell F. Schroeter. Introduction to quantum mechamics. Cambridge University Press, Cambridge: New York, NY, third edition edition, 2018. ISBN 178-1-107-18963-8
Benjamin Gutteridge, Xiaowen Dong, Michael M Bronstein, and Francesco Di Giovanni. Drew: Dynamically rewired message passing with delay. In International Conference on Machine Learning, pp. 12252-12267. PMLR, 2023.
E. Haber and L. Ruthotto. Stable architectures for deep neural networks. Inverse Problems, 34(1), 2017.
Emst Hairer, Christian Lubich, and Gerhard Wanner. Geometric numerical integration. Springer Series Computational Mathematics. Springer, Berlin, Germany, 2 edition, 2006.
William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, pp. 1025-1035. Curran Associates Inc., 2017. ISBN
Andi Han, Dai Shi, Lequan Lin, and Junbin Gao. From Continuous Dynamics to Graph Neural Networks: Neural Diffusion and Beyond. Transactions on Machine Learning Research, 2024. Survey
ISSN 2835-8856. URL httt::::/openeeeeee..eeeee/eeeee Certification.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for Pre-training Graph Neural Networks. In International Conference on Learning Representations, 2020. URL http:t/see/oppeeeeie.eeeeeee//ee
Ming Jin, Yu Zheng, Yuan-Fang Li, Siheng Chen, Bin Yang, and Shirui Pan. Multivariate time series forecasting with dynamic graph neural odes. IEEE Transactions on Knowledge and Data Engineering, 2022.
Qiyu Kang, Kai Zhao, Yang Song, Sijie Wang, and Wee Peng Tay. Node Embedding from Neural Hamiltonian Orbits in Graph Neural Networks, In International Conference on Machine Learning, 2023.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URLhttps: http//axivvv.. www.ii.wr//........o.oooooooo www.:/r.ggbbabbaea.o..
6980
Thomas N. Kipf and Max Welling. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations, 2017. URL https: 7/openrevier wtt..entoooooooooodooooaaaa
13

---

Published as a conference paper at ICLR 2025
Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létoumeau, and Prudencio Tossou. Rethinking graph transformers with spectral attention. Advances in Neural Information Processing Systems, 111111118922,, 2021.
Yuankai Luo, Lei Shi, and Xiao-Ming Wu. Classic GNNs are strong baselines: Reassessing GNNs for node classification. In The Thirty- Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https httpsss//pppreeewwww....oo forummi/ xk1 4E
Sohir Maskey, Raffaele Paolino, Aras Bacho, and Gitta Kutyniok. A fractional graph laplacian approach oversmoothing. Advances in Neural Information Processing Systems, 36, 2024.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang. Zachary De Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In Advances in Neural Information Processing Systems, 2017.
Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look at the evaluation of GNNs under heterophily: Are we really making progress? In The Eleventh International Conference on Learning Representations, 2023. URL https
net 7 wttp:/ieeweee/ooooooeoeeeeeooo /
Michael Poli, Stefano Massaroli, Junyoung Park, Atsushi Yamashita, Hajime Asama, and Jinkyoo Park. Graph neural ordinary differential equations. arXiv preprint arXiv: 1.005732, 2019.
Ladislav Rampášek, Mikhail Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Do- minique Beaini. Recipe for a General, Powerful, Scalable Graph Transformer. Advances in Neural Information Processing Systems, 35, 2022.
Mufutau Ajani Rufai, Thanh Tran, and Zacharias A. Anastassi. A variable step-size imple- mentation of the hybrid nyström method for integrating hamiltonian and stiff diffcrential sys- tems. Computational and Applied Mathematics, 42(4):156, Apr 2023. ISSN 1807-0302. doi: 00007//40331-002-0222200
T Konstantin Rusch, Ben Chamberlain, James Rowbottom, Siddhartha Mishra, and Michael Bronstein. Graph-coupled oscillator networks. In International Conference on Machine Learning, pp. 18888- 18909. PMLR, 2022.
Dai Shi, Andi Han, Lequan Lin, Yi Guo, and Junbin Gao. Exposition on over-squashing problem on gnns: Current methods, benchmarks and challenges. arXiv preprint arXiv:2311.07072 2023.
Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjing Wang, and Yu Sun. Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification. In Zhi-Hua Zhou (ed.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI- 21. pp. 1548-1554. International Joint Conferences on Artificial Intelligence Organization, 8 2021. doi: 21.44963/jcai.2222/2144, URL http::///ee...og//0/2222///////0 2222122111222111
Main Track.
Matthew Thorpe, Tan Minh Nguyen, Heidi Xia, Thomas Strohmer, Andrea Bertozzi, Stanley Osher, and Bao Wang. Grand++: Graph neural diffusion with source term. In International Conference on Learning Representation (ICLR), 2022.
Jan Tönshoff, Martin Ritzert, Eran Rosenbluth, and Martin Grohe. Where did the gap go? reassessing the long-range graph benchmark. In The Second Learning on Graphs Conference, 2023.
Jake Topping. Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, and Michael M. Bronstein. Understanding over-squashing and bottlenecks on graphs via curvature. In International Conference on Learning Representations, 2022. URL http::tttt/ttt http:ppss///ppeeeeiieiee nww. forum?
id= 70m RG
Arjan Van der Schaft. L--gain and passivity techniques nonlinear control. Springer, third edition, 2017.
14

---

Published as a conference paper at ICLR 2025
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, É ukasz Kaiser, and Illia Polosukhin. Attention is All you Need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph Attention Networks. In International Conference on Learning Representations, 2018. URL httpt:t:/ppeeeeeiiiiiiee
net/ www.moiioddiiiaoo Pi-
Yifei Wang, Yisen Wang, Jiansheng Yang, and Zhouchen Lin. Dissecting the Diffusion Process in Linear Graph Convolutional Networks. In Advances in Neural Information Processing Systems, volume 34, pp. 5758 5769.. Curran Associates, Inc., 2021.
Qitian Wu, Chenxiao Yang. Wentao Zhao, Yixuan He, David Wipf. and Junchi Yan. DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion. In The Eleventh International Conference on Learning Representations, 2023. URL
net net/ Eoruuoorumiiio (2d=]62002rppY3
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A Comprchensive Survey on Graph Neural Networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1):4-24, 2020.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How Powerful are Graph Neural Networks? In International Conference on Learning Representations, 2019. URL https //openrevien net./ f (tt.rrrmmmmdrrra111135
Chengxuan Ying. Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie- Yan Liu. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, 34:28877-28888, 2021.
Kai Zhao, Qiyu Kang, Yang Song, Rui She, Sijie Wang, and Wee Peng Tay. Adversarial Robustness in Graph Neural Networks: A Hamiltonian Approach. In Thirty-sevnnth Conference on Neural Information Processing Systems, 2023.
A ADDITIONAL DETAILS OF THE (PORT-)AAMILTONIA) FRAMEWORK
A. SENSITIVITY UPPER BOUND
Although the sensitivity of node state after time t with respect to its previous state can be bounded from below, allowing effective conservative message passing in PH-DGN, we observe that it is possible to compute an upper bound on such a measure, which we provide in the following theorem. While the theorem shows that, theoretically, the sensitivity measure may grow (i.e., potentially causing gradient explosion), we emphasize that during our experiments we did not encounter such a problem.
Theorem A.1. Consider the continuous system defined by Eq. 5). if is non-linear function with bounded derivative, i.e. 3M 0, lo'(())) < M, and the neighborhood aggregation function of the form &g = EVEN. Vx,, the backward sensitivity matrix (BSM) is bounded from above:
(x)
vdexp(QTT)
Vt [0,
0x,
where Q = W + Vd Mmax,e INGIIIV
|
We give the proof of this theorem in Appendix B.4
A.2 ARCHITECTURAL DETAILS DISSIPATIVE COMPONENTS
As typically employed, we follow physics-informet approaches that learn how much dissipation and external control is necessary to model the observations (Desai al. (021). particular, we consider these (graph-) neural network architectures for the dampening term D(q) and externl force term F(q. 1), assuming for simplicity qu R
15

---

Published as a conference paper at ICLR 2025
Dampening D(q): it is square matrix block with only diagonal entries being non-zero and defined as:
param: a lcarnable vector w R
param+: a learnable vector followed by a ReLU activation, i.c., ReLU(w) E R
MI.P4-ReL.U: 4-layer MLP with ReLU activation and all layers of dimension
DGN-ReLU: a DGN node-wise aggregation layer from Eq. 6 with ReLU activation.
External forcing F(q,t), component of the system. Since it takes as input + components it is defined as:
dimensional vector where each component the force on
MLP4-Sim: linear layers of units with sin activation followed by a last layer with units.
DGN-tank: a single node-wise DGN aggregation from Eq. 6 followed by a tanh activation.
Note that dampening, i.e., energy loss, is only given when ((q) represents a positive semi-definite matrix. Hence, we used ReLU-activation, except for param, which offers flexible trade-off between dampening and acceleration learned by backpropagation.
A.3 DISCRETIZATION OF PORT-HAMILTONIAN DGNS
As for standard DE-DGNs numerical discretization method is needed solve Eq. (5 However, as observed in Haber & Ruthotto (2017); Galimberti et al. (2023), not all standard techniques can be employed for solving Hamiltonian systems. Indeed, symplectic integration methods need be used to preserve the conservative properties in the discretized system.
For the ease of simplicity, in the following we focus on the Symplectic Euler method, however, we observe that more complex methods such as Strömer- Verlet can be employed (Hairer et al. (006).
The Symplectic Euler scheme, applied our PH-DGN with null driving forces Eq. the node representation at the (e 1) step as
updates
(e+1)) Pu
Pu CT V Ho(p((+1)) (1)) V (((p((+1) (19)
x
(f+1) =
=
VuEN
(11)
qu ((+1)
(e)
with the step size of the numerical discretization. We note that Eq. relies on both the current
and future state of the system, hence marking an implicit scheme that would require solving
linear system of equations in each step. To obtain an explicit version of Eq. (TI, we consider the
neighborhood aggregation function Eq. (6) and impose structure assumption on W and V.
namely w -WW W and V
[w,
V, A comparable assumption can be made for
other neighborhood aggregation functions, such as GCN aggregation. Despite the necessary block diagonal structure assumption W and V to ensure the separation into the p and q components, we note that W, p. W,, V,. and V, are unconstrained learnable weight matrices.
Therefore, the gradients in Eq. T1 be rewritten in the explicit form
16

---

Published as a conference paper at ICLR 2025
((+1) = p P
w a ( q + $5(19 (0)
toEN.) (oeN.,)) + bq)
Σ oW + Bb(19 BEN,) + ( by)
(12)
vENu)(u)
qu ((+1)
qu te
W W oWW ((+1) + Dol(p! ((+1)) JvEN.)+ JveN.))) bb)
E (lW + Pol(p) vENu)(u)
(1 ((+1) "ie.), BieN,) + bp)
(13)
We observe that Eqs. (12) and (33) can he understood as coupling two DGN layers. This discretization mechanism visually summarized in the middle of Figure [ where message-passing step from layer to layer l + is performed.
In the of PH-DGN with driving forces in Eq. TO the discretization employs the same step for (++1) in Eq. 13 while Eq. 12) includes the dissipative components, thus it be rewritten
Pu ((+1) = Pa + V.. qu ( Du(q Ho(p'), " + Fu(q9) 1) (14)
Lastly, it is important to acknowledge that properties observed in the continuous domain may not necessarily hold in the discrete setting due to the limitations of the discretization method. In the following theorem, show that when the Symplectic Euler method is employed, then Theorem 2.3 holds.
Theorem A.2. Considering the discretized system in Eq. (II obtained by Symplectic Euler dis- cretization, the backward sensitivity matrix (BSM) is bounded from below:
0xu
(I)
21,
AP L].
0xu
(L-E)
provide proof in Appendix B.5 Again, this indicates that even the discretized version of PH-DGN with null driving forces enables for effective propagation and conservative message passing.
B PROOFS OF THE THEORETICAL STATEMENTS
In this section, we provide the proofs of the theoretical statements in the main text and in appendix A As for the rest of the paper, we will use the denominator notation, i.e., Jacobian matrices have output components columns and input components on the rows.
B. PROOF OF THEOREM 2.1
Proof. First, we note that
8 J. V V.V. Hg(y(()) = (i,
(15)
0x
where V2 Hg is the symmetric Hessian matrix. Hence, the Jacobian is shortly written AB, where A is symmetric and B is anti-symmerric. Consider an eigenpair of AB, where the eigenvector is denoted by and the eigenvalue by À * 0. Then:
v AB = Av"
A -
v Av X (v-BB-lv)
17

---

Published as a conference paper at ICLR 2025
where represents the conjugate transpose. On the left-hand side, it is noticed that the Av) term is a real number. Recalling that B remains symmmtriitiii and for any real mnti-symmetric matrix C it holds that C* C C, it follows that (v* Cv)* = v'C'v
-v* Cv. Hence, the v v*B B--v term on the right-hand side is an imaginary number. Thereby, À needs to be purely imaginary, and, result, all eigenvalues of AB are purely imaginary.
B.2 PROOF OF THEOREM 2.2
Our conservative PH-DGN has shared weights W, V across the layers of the DGN. This means that the Hamiltonian is autonomous and does not depend explicitly on time Hy)y(t), we can see from Eq. (4). In such case, the energy is naturally conserved in the system it represents: this is consequence of the Hamiltonian flow and dynamics, we show in this theorem. For more in-depth description and analysis of Hamiltonian dynamics in general we refer to Arnold et al. (2013).
Hy as
Proof. The time derivative of (y((t) is given by means of the chain-rule:
dH((()) (y( dy(() aH(((!))
dt dy(t) J (H(((()
(16)
dt
(t
0y(t)
where the last equality holds since I is anti-symmetrie. Having no change over time implies that (y((t) = Hy(()) = const for all t.
Since the Hessian V Hy(y(t)) is symmetric, it follows directly
+ H²H (²Ha(y(1)
Holy())
0
Oq Öp'
Öp', Oq:
B.3 PROOF OF THEOREM 2.3
In order to prove the lower bound on the BSM, we need a technical lemma that describes the time evolution of the BSM itself, which extends the result from Galimberti et al. (2023).
Lemma B.1. Given the system dynamics of the ODE in Eq.
(2) we have that
d 0y(T) T
OH
Ty(T)
(17)
dt OT - t) Öy -((
Õy -T)
as in (Galimberti et al. (2023). The same applies, with a slightly different formula, for each node u, that is the BSM satisfies
d (x)(T) dt 0x,, -T) Ox. (T-t) oy ((T-t) Ox, (T-)
0y
(x,
(x,(T) Fx, (T -t)
(18)
where fu is the restriction of f I 0H node u, which can be wrillen
to the components corresponding to Xu. that is the dynamics
fa= I.JJ = I 0H
(19)
where Ly is the readout matrix, of the form
T0 c (w-1)
T T
0 Dfx$(1-w)
0g 0g 0g 0g
0 0 (n=)()
L
(20)
0 (w-1)
0 0gxg
0 (gx((n-u)
0 (gx(((--1)
Ixx4
0 (n-
which is a projection on the coordinates of a single node Notice as well that, in denominator notation
dy 0x.
Lu
(21)
We only show Eq. (1)). it is related to graph networks and is actually harder version of Eq. (7) with the latter being already proven in Galimberti et al. (2023). We also show the last part of the proof in general sense, without using the specific matrices of the Hamiltonian used.
18

---

Published as a conference paper at ICLR 2025
Proof. Following Galimberti et al. (023), the solution to the ODE dx. in integral form as
fu(((t)) can be written
de
[" July()))dr xx(T [T - s)ds
Xu (T) = (T _T-
(22)
Differentiating by the solution at previous time x, (T' t) we obtain
(xx(() Ox (T -t)
Lu 0 S. fa(y(T s)jds
Ox, -
@Sa(y(T - + 8))
Lu +
(23)
Oxu (T
17-11 (T-
Lu + ( (T' dy ((T-+++)
θy(T -t -t 0fu
ds
where in the second equality we brought the derivative term under the integral sign and in the third used the chain rule the derivative (recall we are using denominator notation). Considering a slight
perturbation in time 6, we consider (x. (x-(T_ as this will be used to calculate the time derivative of
the BSM. Using again the chain rule for the derivative and the formula above with T - - instead of T --tt, t, we have that
(T) (T - -- (T -t 0x (T -t)
(T (T,
(24)
dy(T - --+ + lu + 1/² -T
05
8x (T)
ds
- 1- -)) Oy y(( -11--))+))
Ix., (T -1)
This way, we have expressed 8x., (T)
(T--A) in terms of (T.
To calculate our objective, we want
7x (T-
to differentiate with respect to ô. We first calculate the difference:
(T) 0x, (T - t- 5) (T -T
(T)
öy(T - Oy 8) 0fu
ds Õx, (T)
Ox, - - 6) θy y(T 0x,, (T - t)
(25)
We can now divide by and take the limit 8 +
0x, (T)
0x, (T')
lim 5-+0 0
(T - 1- 0x,(T -1)
δy(T - t- 8) 0fu
Fx,, (T)
lim
ds
(26)
6-+0
8 0
0x, - -1- θy
0x
Oy
afa
0x, (T)
Ix,
Oy (y(T+) Ix, -t)
(T-+)
Where in the final equality we used the fundamental theorem of calculus. Finally
d (T) Oy 0fu Ox,(T) Fu, Ox,, (T -t)
Ox.(T)
(27)
dt 0x,, (T-
Ox. (T-1) By y(T-t) Fx,, (T
giving us the final result.
L
We are now ready prove Theorem
2.3 First, we calculate that
af
211 Qy
01l
02 Ty²
L
-
(28)
Øy
Fy
that F, TT. T. This will be helpful in the following matrix calculations.
19

---

Published as a conference paper at ICLR 2025
Proof. For brevity, we call
8x
(T which will be indicated simply as Vu. When
ix., (3-
== O. wu is just the Jacobian of the identity map n/ (T, T) = Tu and the result v J Tn V. = TT is
true for = Calculating the time derivative V Ju Va have that
d dt
1
Ju I.J. + + Ju
(F.V.)) J.V. + V. J.P.V
(29)
V.L.JS In v InLSS L,
=v (L.JSL + IuLuST L.) Vn
where in the second equality we used the result from lemma in parentheses is zero so that the time derivative is zero. Using the relations J L, L,, 1 Tu and Ju I. InnJ we easily see that, finally
B.1
We just need show that the term
dt (1 0xx(1) 8)
(1ox.(1-1)
]
(T) Oxx (T (t) ( ()
Tu
VIJT (LuJSL) LLJS(-L)) V.
(30)
which means that
Gx ( (T)
Tu
8x, (T) (x.
constant and equal to Ju for all t, that is our thesis.
[xx, (T-
Now, the bound on the gradient follows by considering any sub-mut cab-multiplicative norm || ||:
||alll = | (T)
0x. 1x - 1)
0xn (T) (T -
Js
[
0x,, (T -t)
and simplifying by || = 1.
B.4 PROOF OF THEOREM
To prove the upper bound, we the following technical lemma:
Lemma B.2 (Galimberti et al. (203)).. Consider a matrix A E R with columns a, E R". i.e. an ), and assume that ||a: S x+ for all = 1...., n. Then, | S
a1
a2
72.
This lemma gives bound on the spectral norm of matrix when its columns are uniformly bounded norm. Therefore, our proof strategy for Theorem A.I bounding each column the BSM matrix.
(x_(T) Proof. Consider the ODE in Eq. 27 from Lemma B.1 and split dx. (T-) [ 2: (t) (1 zI( J. Then, Eq. (7) is equivalent to
into columns 8x,,
Ux-(7T-)
(1) AT - f)z; t 10,
(31)
subject to (0) = e, where e; the unit vector with single nonzero entry in position The solution of the linear system of ODEs in Eq. (31) is given by the integral equation
z, (t) (0) /
AT s));(s)ds,
E [0,
(32)
By assuming that [,A < Q for all 7 E [0.T], and applying the triangular incquality in Eq. 32 is obtained that:
(2)22 < [2)(1)] + Q / || ds 11 Q //' [|x(())]2 ds,
where the last equality follows from ||2] ||e: = for all 1,2, ..... d. Then, applying the Gronwall inequality, it holds for all f [0.
||z, (t)ll - eXp(QT).
(33)
By applying Lemma B.2 the general bound follows.
20

---

Published as a conference paper at ICLR 2025
Lastly, characterize Q by bounding the norm [TA Vr [0. From Lemma B.1 Ag can
be expressed as A. = L.S.J L,, which is equivalently A_ = V2, H since J L, = L J. The Hessian V2, (y) is of the form:
Vz. Hy(y) = w diag(o'
[lag(o'(Wx,,, (Wx, + 2a + b)) +
Σ v diag(o (o (Wx, + 22 +)
[olaglo'Wxxx
sEN
After noting that V [y) 12 S V2 (12y)12211 1l,. the only varying part is the Hessian
V Hg(y) (y) since || 1| By Lemma B.2 diag(o (o' (x)) I vd M and noting that X ||
|X|
for any square matrix X, then
V Ho())l2 S W + VdM max [M V Q.
This also justifies our previous assumption that ||A (T) ll is bounded.
B.5 PROOF OF THEOREM A.2
Proof. discrete case, since the semi-implicit Euler integration scheme a symplectic method, it holds that:
Oxu
Ox
ix
(t-1)
Tu
0x
(f-1)
=
Ta
(34)
Further, by using the chain rule and applying Eq.
(34) iteratively we get:
(L)
0xn
(L)
Ix
((+1)
T
Oxn (i+1)
Ju
II
Jn
II
Tu
0x
(2-E)
Ox!
6
Ox
Ox
Li=L-
Li=L-
Hence, the BSM is symplectic at arbitrary depth and we can conclude the proof with:
Ox!
0x! (b)
Ox!
(L)
2
||
Ju
S
||Jul
ll
0x! ((-
ix!!
Ox!!
B.6 PROOF OF THEOREM 2.4
Proof. We employ a similar proof in Di Giovanni et al. (2023) and redo some calculations based
model and the Symplectic Euler scheme used to obtain Eqs. 12 and (13). To calculate 8x (x((+1)
for two consecutive nodes u, we consider the 4 sub-blocks as per
Tap((+1)
0q (C+1)
0x1(+1)
((+1)
0p (c)
(35)
0x(0)
(
!p!
0q (++))
0qu
0q (e)
We proceed to calculate the 4 blocks independently, stopping at first-order terms in for simplicity reasons:
0p e+1
dq E+1 0p,. f-1
= O(e2)
0p.
Op. Oqg.
WEN,CN.
0p.+1
Σ (V)))o((w))
An.
W. (u) W V (+)+
oqp
WEN.NA.
(36)
0q
Optt1 6+1 dq 1+1
= 0
Qqg
dqf (+1
WEN.NN.
0a
eAn
V. ( w. V.V[((() WEA,
Σ (VT))o,()))
Op..
21

---

Published as a conference paper at ICLR 2025
where compressed op(u) (W pPu Pp
(++1)
(f+1))
)=eN.) and analogously for y for length
reasons. Now, the norm of a block matrix is less or equal to the sum of the norms of the sub-blocks, so that
0x(+1))
((+1)
Opu (
(e+1)
((+1) Ö
aqu
(e+1)
iqq
(e+1)
+
Ox
(4)
0p (A)
dqu
op.
aqu
(E)
L1
L1
L1
L1
11
(37)
0pu 1++1)
Qqu (1+1)
O
0q (A)
Op. (2)
11
Now, using the Eq. 36
and setting Co = max lo' (zx)), we can estimate
Ox
(++1) 1)
S eAuv
2dc W V + Σ
A A
V
(e)
11
(38)
E un
2dca
V + Σ
A A Au mn V
Vilde de,
Recalling the block structure of V, we can say that [V| V VV and obtain this final form of the one-step neighbor sensitivity:
Oxu
(E+1)
S c
2/V/| +
Σ Aew V
(39)
0x (
L,
Repeating the same process for the node update, one has that
0x! (++1)
dl
(1
1 1 ECo (
Σ
A A Aun V9)
(40)
0x! (8)
Now, we can show the inductive step:
0x(++1) 11+1)
Ox
Ox
(F+1)
Ox
Ox.
(1+1)
(41)
Ox!
0x
Ox. (E)
Ox!0)
2x
WEN,
We now evaluate the norm of this object and, by using the triangular inequality and the two relations above for same-node and neighboring updates, we obtain
Oxn
(e+1)
(x
(E)
Oxn xx(++1)
(E+1)
dx
Oxn
(x(e+1)
(e+1)
+ Σ
Ox!
xx
Ox!!
xx
Ox!
L1
L1
WEN,
Li
L1
0x
dl
1 |
+ cCo Σ
A Aua A V21
(42)
xx
Ox!
2/ AusAAw V2
dA up
CCo
Auw
Ox
(0)
L,
we now consider the upper bound to be independent of the specific nodes and the maximum degree of node be N, have that
Ox
(E+1)
Ox!
dl 1+ eCol
W |1ec//
+ eCo
V
Ox! (0)
xx
I.j
(x
(E)
+
Au
2/V + Ans
ix
(0)
L1
(43)
0x
<
Id (1 + cCo | -N 11)(V2)
0x
(e) (2)
Andd (Neco ( + -N 11)
xx (0)
11
22

---

Published as a conference paper at ICLR 2025
which is in the form of theorem 3.2 of Di Giovanni et al. 2023). Calling max([W],.V]] obtain that
Üx 0x
(f+1)
OxL (E)
(Id(1 + EC, Nww) Auddw (N2 + N))
(44)
L we can collect a term WC, Nd and, following proof in Di Giovanni et al. (2023), we finally arrive at
We now sacrifice the term and set e
0x (xx+t1)
1)+1)
S (dw w(N + 1)A)+t)
(45)
0xi
Without the terms we sacrificed along the proof, our upper bound is at least No greater than the one in Di Giovanni et al. (2023).
B.7 ADDITIONAL THEOREMS FOR PH-DGN WITH DRIVING FORCES
In this section, consider the case of being bounded activation function [xxxx) S be with bounded derivative |o (x) I Cy. c.g. tanh. Further, we suppose the driving forces are bounded and Lipschitz continuous with common constant Bạ- In formulas, this means that (g) I Ba,
(D. 0q
< Ba, [q. t) s Ba and @F(q.t) < BN- Under these hypotheses, the self-influence of
node after one step is bounded from below by a constant. An upper bound on the backward sensitivity matrix (BSM) norm can also be derived for the same node influence and neighboring node influence after one step.
Theorem B.1 (Lower bound, port-Hamiltonian case). Consider the full nort-Hamiltonian update in Eqs. 11) and (14). Then, with the above hypotheses, the following lower bound for the BSM holds
0xu
(1+1)
0x (e+1)
DII
Ox (0)
Ox
(46)
2 cw Ba c (N
w)
where w = maa[[ [V] as before. In this equation, 10x (e+1) 18x! (8) represents the BSM for the
full nort-Hamiltonian update as in Eqs. 13 and 14). while 10% (0x((+1) 10x0 (0x
((+1)
is the BSM for the pure
Hamiltonian case in Eqs. 112) and (13) and D is defined as
(11)) + (F
D
(ul(q
Vo. Ho(p (0.00)))))))))
(47)
Ox
0x!,
This theorem shows that, with these hypotheses, the effect of the driving forces on the BSM norm can be controlled with small enough step size.
Proof. We start by providing the same calculations as Theorem 2.4 this time with port- Hamiltonian components included, for the one-step update in both the cross-node case
0p
-cAuy Du(qw)
V WT..oo + W. VJ (p)++
Σ (VT)) +0(E),
Op.
WEN.NN.
0p
e+1 -EAmv
V V. (w) + W
Σ (V Vo(()())))
dqg
wEN.NW.
@D,(q) Ōq.
( p (u(u)= + Σ V)T((w)) + OF(q.t)
eAuv
oq.
mEN,
0q 1+1
0ptt1 f+1 0g +1
O(E2)=
T-T O(E2),
dqf Op.
m.N.
0g
<Awe
V W.T. p (()))
W V.V. (1)
Σ (V))8
+ +0((2)
0p.
WEN.ON.
(48)
23

---

Published as a conference paper at ICLR 2025
and same node case
0p ++1
1
(Du O(e2),
0p
0p ++1
-d
(WT))or((u)
+
Σ (V.))
0q
WENu
(Dn(q)
V )z())) + 0F(9.1)
d
pp))
0((2)
(49)
dqu
dqu
WEN.
0qh 4+1
-el
(W,))o%(u)+ E (V))))(())) +0(E))
0q
wEN
0qh
0p..
we consider the one-step update for the same node, we can divide it into the non-dissipative component, called 0x(++1) and the dissipative component given by
0x
D (oppl) a (DD((a) (W op))+ [V)o((w))
(F t)
0qu
0qu
WEN,
(50)
Now, to estimate lower bound on the self-influence norm, we can use the triangular inequality
Oxu (E+1)
(f+1)
Ox.
D
(51)
Ox (2)
ox!0 2
T.
Since the first term is the update for the non-dissipative case, Theorem A.2 holds and then the right-hand side is greater or equal to
(-||DD)
(52)
The driving forces effect can then be estimated as
DI s <(I) (([[]]) + | + V]V]))
(53)
x dewLdd (N +3 +3++) w)
as per the definitions above, which leads to Eq. 446
We now calculate upper bound the one-step BSM node
Theorem B.2 (BSM upper bound, same-node update in port-Hamiltnnin case). Consider again the full port- art-Hamiltonian update in Eqs. (13) and (14) with the same hypothesis above. Then, the following upper bound for the BSM holds:
0x((+1) (1+1)
0x! (1+1)
+ +IDI
Ox!! (2)
Ox!! (P)
(54)
< d(1 + cd w(1 + BN) + w + 3))),
where the individual terms defined as in theorem B.I
Proof. We start by recalling the calculations for the same-node and cross-node update gradients from Theorem B.I. Then, we employ again the triangular inequality to obtain
0x
(E+1)
Ox!
(8+1)
<
+ D
(55)
Ox
(2)
xx(1)
(8)
L1
L1
24

---

Published as a conference paper at ICLR 2025
Since the first term involves the non-dissipative components only, the calculations in Theorem 2.44 We use the same calculations for the second term as in Theorem B.1 Finally, we can calculate
0xu (e+1)
(
+ D|I,,
0xa
0xu
(8)
- Id (1 + ECo | + (N - 12 - dew +3+w
(56)
2 d(1 + ed, + dew (N + +3+w) w)
S d(1 + edg w( + (N + ++ 3)))
Theorem B.3 (BSM upper bound, cross-node update in the Hart-tamilnnnian case). Consider again the full hort-Hamiltonian update in Eqs. (13) and (14) with the same hypothesis above. Then, the following upper bound for the cross-node BSM holds:
0xa (E+1)
0x (e-1)
+ [ emess
0x
0x (2)
(57)
S d(ec (1 + L)(N ++++3) +3)).
where
[8xa
/xxe represents the cross-node BSM for the full port-Hamiltonian update in and (4), while [0x(++1) 10x!"] the cross-node BSM for the pure Hamiltonian case as in and Demos is defined as
(E+1)
(0)
Eqs.
Eqs.
and 13
Ho(p ((1)) + (F.(q
D cross
(58)
Ox
0x!
Proof. We start by recalling the calculations for the same-node and cross-node update gradients from Theorem B.I. Then, we employ again the triangular inequality to obtain
Ox
(f+1)
0x(
(++1)
Deross
L1
(59)
0x
Ox!
L1
L1
Since the first term involves the non-dissipative components only, we the same calculations as in Theorem 2.4. For the driving forces component, similar calculations to the ones Theorem B.J result in
I Dress || S d(ec, w(1 + LN + ww- +++3))
(60)
from which our bound follows.
C EXPERIMENTAL DETAILS
C.I EMPLOYED BASELINES
In our experiments, the performance of our port-Hamiltniinn DGNs is compared with various -fate-of-the-at DGN baselines from the literature. Specifically, we consider:
classical MPNN-based methods, i.c., GCN (Kipl & Welling 2017), GraphSAGE (Hamilton
et al.
2017). GAT (Veličković al. 2018), GatedGCN (Bresson & Laurent 2017 GIN (Xu
et al. 2019). GINE (Hu et al. 2020). and GCNII (Chen et al. 2020);
DE-DGNs, i.e., DGC (Wagg al. 2021). GRAND (Chamberlain al. 2021a
Graph-
CON (Rusch et al.) 2022), A-DGN (Gravina et al. 2023), HANG (Zhao et al
2023
HamGNN (Han al. 2024) and SWAN (Gravina al.
2025a):
GGaahh Transformers, i.c.. Transformer (Vaswani et al. 2017 Dwivedi & Bresson 202
SAN (Kreuzer al. 202T), and GraphGPS (Rampášek et al.
2022):
Higher-Order DGNs, i.e., DIGL. (Gasteiger et al., 2019), MixHop (Abu-El-Haija et al. 2019) and DRew (Gutteridge et al. (023)).
Note that since DE- DE-DGNs belong to the same family as our proposed method, they are a direct competitor Hort-Hamiltonian DGNs.
25

---

Published as a conference paper at ICLR 2025
C.2 GRAPH TRANSFER TASK
The graph transfer task consists of the problem of propagating a label from a source node to a target node located at increased distance k in the graph. other words, we set up node-level regression task that measures how much of the source node information has reached the target node. The task was first proposed in the work of (Di Giovanni et al. 2023), , but here we follow the data generation and more challenging experimental setting of Gravina et al. (2025a). The task employ the same graph distributions as in Di Giovanni et al. (023), i.c. line, ring, and crossed ring graphs (see Figure | for visual exemplification of the three types of graphs when the distance between the source and target nodes is 5). However, to make the task more challenging we follow Gravina et al. (2025a), thus, we initialize node input features with random values uniformly sampled in the range [0, 0.5). In each graph, the source node is assigned with label "1", and a target node is assigned with label "0". We use an input dimension of 1, and source-target distance equal to 3, 5, 10, and 50. The target output is constructed by modifying the input features: the source node's s target is the label "0", the target node's 'ss target the label "1", and the intermediate nodes retain their original random values from the input. In other words, the ground truth values are the switched node labels of source and target nodes. We generate 1000 graphs for training. 100 for validation, and 100 for testing.
Figure 4: Three topologies for Graph Transfer. Left) Line. Center) Ring. Right) Crossed-Ring The distance between source and target nodes equal to Nodes marked with S are source nodes, while the nodes with 7' are target nodes.
Following Gravina al. (2025a), We design each model using three main components. First is the encoder, which maps the node input features into a latent hidden space. Second is the graph convolution component (i.e., PH-DGN or other baselines). Third is the readout, which maps the convolution's 's output into the output space. The encoder and readout have the same architecture across all models in the experiments.
We perform hyperparameter tuning via grid search, optimizing the Mean Squared Error (MSE) computed on the node features of the whole graph. In other words, the model is trained to predict the target values for all nodes, and the loss is computed as the MSE between the predicted values and the corresponding target values across the entire graph. We train the models using Adam optimizer (Kingma & Ba 2013) for a maximum of 2000 epochs and early stopping with patience of 100 epochs on the validation loss. For each model configuration, we perform training runs with different weight initialization and report the average and standard deviation of the results. We report in Table 3 the grid of hyperparameter employed by each model in our experiment.
C.3
GRAPH PROPERTY PREDICTION
The tasks consist of predicting two node-level (i.e., single source shortest path and eccentricity) and one graph-level (i.e., graph diameter) properties on synthetic graphs generated by diffferent graph distributions. In our experiments, we follow the data generation and experimental procedure outlined in Gravina al. (2023). Therefore, graphs contains between 25 and 35 nodes and are randomly generated from multiple distributions. Nodes are randomly initialized with value uniformly sampled the range (0, 1), while target values represent single source shortest path, eccentricity, and graph diameter depending on the selected task. 5120 graphs are used as the training set, 640 as the validation set, and the rest as the test set.
As in the original work, each model is designed as three components, i.e., encoder, graph convolution component (PH-DGN or baselines), and readout. We perform hyperparameter tuning via grid search, optimizing the Mean Square Error (MSE), training the models using Adam optimizer for a maximum of 1500 epochs, and early stopping with patience of 100 epochs on the validation error. For each model configuration,
perform 4 training runs with different weight initialization and report the average of the results. We report in Appendix C.5 the grid of hyperparameters employed by each model in our experiment.
26

---

Published as a conference paper at ICLR 2025
C.4 LONG-RANGE GRAPH BENCHMARK
We consider the Peptide-func and Peptide-struct tasks from the Long-Range Graph Benchmark (Dwivedi et al.) (022). Both tasks contain graphs corresponding to peptide molecules for total of 15,535 graphs, 2.3 million nodes and 4.7 million edges. Each graph has an average of 150 nodes, 307 edges, and an average diameter of 57. Peptide-func is a multi-label graph classification task with the aim of predicting the peptide function. Peptide-struct a multi-din caalti-dmmessoonal graph regression task, whose objective is to predict structural properties of the peptides.
We follow the experimental setting of Dwivedi et (202)) thus we consider a stratified splitting of the data, with 70% of graphs the training set, 15% as the validation set, and 15% as the test set. We perform hyperparameter tuning via grid search, optimizing the Average Precision (AP) in the Peptide-fune task and the Mean Absolute Error (MAE) in the Peptide-struct task, training the models using Adam optimizer for a maximum of 300 epochs. For each model configuration, we perform training runs with different weight initialization and report the average of the results. Moreover, stay within 500K parameter budget as proposed by Dwivedi et al. (022)). also consider the setting of Tönshoff al. 2023) that propose to employ a 3-layer MLP readout. We report in Appendix C.5
our experiments, we
the grid of hyperparameters employed by each model our experiment.
C.5 EXPLORED HYPERPARAMETER SPACE
In Table 3 we report the grid of hyperparameters employed in experiments by each method, adhering to the established procedures for each task to ensure fair evaluation and reproducibility. We note that for hyperparameters specific of layers L, we selected values based on the specific benchmark protocol (whenever possible) and considering factors like the average graph diameter in the training set.
PH-DGN, such the step size and the number
Table 3: The grid of hyperparameters employed during model selection for the graph transfer tasks (Transfer), graph property prediction tasks (GraphProp), and Long Range Graph Benchmark (LRGB).. We refer to Appendix A.2 for more details about dampening and external force implementations.
Values
Hy Hyperparameters
Transfer
GraphProp
LRGB
Optimizer
Adam
Adam
AdamW 0.001, 0.0005 0 195, 300
Learning rate
0.001
0.003
Weight decay
10-6
embedding dim
64
10, 20, 30
N. layers (L)
3. 5, 10, 50,
10, 20, 30
32, 64
100, 150
Termination time (T)
Le
0.1, 1, 2. 3, T/L
3, 5, 6 T/ L
e
0.5, 0.2, 0.1,
0.05, 0.01, 10-4
2 4g
Eq. (6) GCN
Eq. (6 GCN
Eq. (6). GCN GCN
Eq. (6). GCN
Eq. (6) GCN
Readout input
p. q. pl|q
p. q. p.p.p tanh
p. q. pl/q tanh
tanh
Dampening
param, param+,
param
MLP4-ReLU, DGN-ReLU MLP4-Sin, DGN-tanh
External Force
DGN-tanh
N. readout layers
1, 3
27

---

Published as a conference paper at ICLR 2025
D ADDITIONAL EXPERIMENTAL RESULTS
D.1 GRAPI PROPERTY PREDICTION RUNTIMES
In Table 4 we report runtimes of PH-DGN with and without driving forces as well as baseline methods on the graph property prediction task in Section 3.3] To obtain fair comparison, we configured all models with 20 layers and embedding dimension of 30. We averaged the time (in seconds) over random weight initializations for complete epoch, which includes the forward and backward passes on the training data, well the forward pass on the validation and test data.
Table shows that PH-DGN with and without driving forces has improved or comparable runtimes compared MPNNs. Notably, PH-DGN without driving forces on average 5.92 seconds faster than GAT and 5.19 seconds faster than GCN, Compared to DE-DGN baselines, our methods show longer execution times, which are inherently caused by the sequential computation of both p and q explicitly given in Appendix A.3 and non con-conservative components (i.e., param and MLP4-Rel R----RRLUU in Table 4. Lastly. related Hamiltonian approaches require at least twice much computation (HANG (Zhao et al. )023) up until 126x more seconds per epoch (HamGNN (Kang al. (023). This underlines our key contribution of MPNN-complexity model adhering to Hamiltonian laws, i.e.,
linear in number of edges and nodes.
Table 4: Average time per epoch (measured in seconds) and std, averaged over random weight initializations. Each time is obtained by employing 20 layers and an embedding dimension equal to 30. Our PH-DGN employs param and MLP4-ReLU as dampening and external force, respectively. The evaluation was carried out on an AMD EPYC 7543 CPU @ 2.80GHz. First, second, and third best results.
Model
Diameter
SSSP
Eccentricity
MPNNs
GCN
32.45 2.54
17.44
11.78 +2.43
GAT
20.20
26.41
17.28
GraphSAGE
13.12_2.99
13. 13.12=2.99
8.20.40.5
GIN
6.63 L0.28
21.16
14.22 13.17
GCNII
2..15
14.9647.17
15.70
DE-DGNs
DGC
8.97 8.97
12.54
1.21
GRAND
133.84 133.84442.57
109.15
202.46485.00
GraphCON
9.26
7.76
7.80.L0.05
A-DGN
8.42
7.86-2.11
13.18
HamGNN
1097.904379.56
1897
1897.04+200
027.04222.8
1773,43454.37
HANG
28.92
34.24 11.08
34.63
Ours
PH-DGN
15.28 20.02
15.34
PH-DGN
17.18
17.12
17.13
D.2 COMPLETE LRGB RESULTS
In Table we report the complete results for the LRGB tasks including more multi-hop DGNs and ablating the scores obtained with the original setting from Dwivedi et al. (2022) and the one proposed by Tönshoff et al. (2023), which leverage 3-layers MLP as a decoder to map the DGN output into the final prediction. Since directly, we cannot determine the best method between (Dwivedi et al. 2022) and Tönshoff et al. (2023). Therefore, in 5 present all the results.
unable to compare the validation scores of the baselines
28

---

Published as a conference paper at ICLR 2025
Table 5: Results for Peptides-fune and Peptides-struct averaged over 3 training seeds. Baseline results are taken from Dwivedi et al. (2022), Tönshoff et al. (2023), Gutteridge et al. (023). Gravina et al. (2025a). indictess the use positional or structural encoding. We have detailed the type of encoding wherever the original source explicitly specifies it. "*" means that re-computed the performance of the method strictly following the experimental protocol from Tönshoff et al. (2023). As proposed in Tönshoff et al. (2023), re-evaluated methods employ 3-layer MLP readout and (Laplacian) positional or (random walk) structural encoding. The first, second, and third best scores colored.
indicates
Peptides-func AP
Peptides-struct MAE
Model
MPNNs, Dwivedi al. (2022)
GCN
0.5930 10.0003
0.3496.10.0013
GINE
0,5498+0.000 .079
0.3547=0.0045
GCNII
0.5543 00078
0.3471 +0.0010
GatedGCN
0.5864:0000777
0.3420
Multi-hop DGNs, Gutteridge et al. (2023)
DIGL+MPNN
0.646940.0019
0.3173+0.000
DIGL+MPNN+LapPE
0.6830 10.0006
0.2616 10.0018
MixHop-GCN
0.6592 10.0000
0.2921 10.0023
MixHop-GGCN+LapppP
0.6843 10.0049
0.261 14 1..0023
DRew-GCN
0.6996 0076
0.278 I =0.0028
DRew-GCN+LapPE
0.7150 0044
0.2536+0 10.0015
DRew-GIN
0.6940 #0.0074
0,2799 0.2799 #0.0016
DRew-GIN+LapPE
0.7126 20.0045
0.2606 00.0014
DRew-GatedGGN
0.6733+0.000
0.2699 +0.0018
0,2699
DRew-GatedGCN+LappE
0.6977 10.0026
0.2539
4000000000007 +0.0007
Transformers, Gutteridge et al. (2023)
Transformer+
0.6326.1..01
0.2529 10.0016
SAN+LapPE
0,6384 20.0121
0.2683 00.003
GraphGPS+LapPE
0.6535 40.0041
0.2500
Moditied experimental protocol, Tönshoff et al. (2023)
GCN+PE/SE
0.6860
0.2460 +0.0007
GINE+PE/SE
0.662 +0.0067
0.2473 #0.0017
GCNII+PE/SE*
0,6444100.001
0.2507 +0.0012
GatedGCN+PE/SE
0.6765 10.0047
0.2477 10.0009
DRew-GCN++PE/SE
0.6945 00.45.0.000 10.0021
0.2517 10.0011
GraphGPS+PE/SE
0.653440.000
0.2509
DE-DGNs
GRAND
0,5789+0 +07894000022
0.34 +00000
GraphCON
0,6022 10.0068 0.0068
0.2778 0.2778+000012 #0.0018
A-DGN
0.5975 #0.0044
0.2874
20.0021
SWAN
0.6751 1 10.0039
0.2485 +0.0009
Ours
PH-DGNe
0.6961 10.0070
0,2581 +0.0020
PH-DGN
0.7012 10.0045
0.2465 10.0020
D.3 COMPARISON TO HAMGNN AND HANG
In this section, we discuss on the differences with related works in the line of Hamiltonian systems for DGNs, such as HamGNN (Kang et al. 2023) and HANG (Zhao et al. 2023).
At first, we highlight that HamGNN and HANG are both instances of auto dito-differentiaeee models that solved with neuralODE solver, while our PH-DGN provides exact equations for the layer-wise
29

---

Published as a conference paper at ICLR 2025
information aggregation, revealing generalized formulation that allows to turn any MPNN- model into its energy preservin version, thus remaining efficient and general.
From architectural point of view, HamGNN uses black-box Hamiltonian functions that are agnostic to the graph structure, thus limiting its interpretation as a DGN, while HANG explicitly couples the graph structure with the energy function, without making the connection design choice in relation to (long-range) message passing propagation, which is the main focus of our work. In contrast, our PH-DGN uses a port-Hamiltonial function that has clear interpretation and is directly translated into the nonlinear messagg--psssing framework, aiming to preserve information in the evolution and aggregation process. Thus, the Hort-tamilnn function drives the aggregation process based on the graph topology, allowing information to be propagated for an arbitrary number of layers. Additionally, our approach extends and generalizes Hamiltonian-inssiee introducing the ability to adaptively balance between conservative and dissipative behaviors, thus incorporating mort-Hamiltoniar dynamies. demonstrated theoretically in Section and empirically in Section 3
investigate this
DGNs by
PH-DGN achieves long-range propagation through intrinsic graph coupling combined with conservative dynamics, which can be modulated by learnable components from the port- Hamiltonian framework, such as internal damping and external forces. These properties, crucial for adding generality to the approach and for effective information flow, are absent in existing models like HamGNN and HANG. We summarize the above comparison in Table T in Table we report an empirical assessment between PH-DGN, HamGNN, and HANG to evaluate the computational requirements of the three methods on the Eccentricity task (Section 3.3). To ensure a fair evaluation. we conducted model selection given officially released implementations of HamGNN and HANG on grid of total number of layers L {1, 5, 10, 20, 30). embedding dimension d {20, 3. step size E [1.0, 0.5, 0.1 and in case of HamGNN we selected the Hamiltonian fuction to be H (H2, H3}, which are the top performing Hamiltonian functions in Kang et al. (2023). Note that for HamGNN, L = 30 was infeasible due to memory constraints of 40GB. Our results in Table 7 show that our PH-DGN not only outperforms HamGNN and HANG, but also operates at lower computational cost, proving be more efficient in terms of both speed and memory usage.
Table 6: Key properties of DE-DGN models inspired by Hamiltonian and port- Hot--Hamltonian dynamics.
Hamiltonian Conservation
Graph Coupling
Learnable Driving Forces
Ham.-insp. DGNs
HamGNN
HANG
Our PH-DGN
PH-DGN
Table 7: Mean test logio (ogno(MSh) (MS) and std average over 4 training seeds the Eccentricity task from Section 3.3 along with the measured memory consumption and average time per epoch (in seconds) of HamGNN, HANG, and our PH-DGN with L 20 and d 30. Best result is color coded.
Eccentricity logia (MSB) 4
Memory GB 4
Time per epoch
Model
sec, 4
Ham.-insp. DGNs HamGNN
0.7851 10.0110 0.8302+0.0051
26.8 GB
1773.43
HANG
3.2 GB
34.63-0.96
Our
PH-DGN
-0.9348-0.2000
1.3 GB
17.13
30

---

Published as a conference paper at ICLR 2025
D.4 ABLATION ON DRIVING FORCES
In order to study the different dampening and external forces we designed for the full port-Hamiltonian setup (i.e., including driving forces), provide an ablation of model the Minesweeper task
(Platonov et al. 2023), employing the most recent training protocol Lu et al. 2024). The model selection is split into two stages. First, the best purely conservative PH-DGN (i.e., PH-DGNc) is selected from a grid with total number of layers L E 51,5, 5,8J. embedding dimension d E {256, 512], step size e {0.1, 1.0) according to the validation ROC-AUC. Then, the best selected configuration is tested with different driving forces. We refer the reader to Appendix A.2 for the details on the tested architectures, We report in Table the results on Minesweeper task alongside with the top-6 models from Luo et al. 2024). We observe that purely conservative PH-DGNc show strong improvement with respect to baseline methods, achieving new state-of-the of-the-art performance. As evidenced by ablation in Table g the inclusion of driving forces could not improve this task, thus there is no advantage in deviating from a purely conservative regime. Our intuition that counting-bass based tasks, like Minesweeper, greatly benefit from purely conservative bias, such that used by PH-DGNc since they require preserving all the information.
To complete the pieture of the empirical performance differences between PH-DGN with and without driving forces, we report in Fig. the ablation on the Graph Transfer task.
We observe that the purely conservative PH-DGNc achieves better performance than the full PH- DGN on two out of three tasks in the long-range regime. This finding aligns with our intuition that purely conservative bias is essential for problems requiring the preservation of all information. Indeed, solving the graph transfer task effectively demands retaining the source node label without dissipation.
These results, together with those reported in Section 3 suggest that model selection should determine the optimal configuration of driving forces depending on the specific data setting.
Table Ablation results on Minesweeper for different architectures this work. We report ROC-AUC scores averaged over the 10 official tr/vl/ts splits of Platonov et al. (2023). The first, second, and third best scores are colored.
driving forces proposed in
Train Score ROC-AUC
Val Score
Test Score
Model
ROC-AUC f
ROC-AUC 1
Top-6 models from Luo et al. (2024)
GraphGPS
90.75 10.89
SGFormer
91 91.42_0.41
Polynormer
97.49 =0.48
GAT
97.73
GraphSAGE
97.77 =0.62
GCN
97.86
Our no driving forces
PH-DGN
99.78
98.37
98.45
Our with driving forces
PH-DGN
Dampening
External Force
MLP4-Sin
99.37
96.66+0.35
96.61
DGN-tanh
99.28
97.2940.28
97.20
param
99.79-0.u5
98.3640.22
98.42
param
MLP4-Sin
99.55
96.8640.33
96.86
param
DGN-tanh
99.30
97.35
97.27
ML.P4-ReLU
99.62
95.33
95.33 10.65
MI.P4-ReL.U
MI.P4-Sin
99.93
95.7940.57
95.67
MLP4-ReLU
DGN-tam tan-
98.79=0.21
95.42+0.45
95.41
DGN-ReLU
94.96
93.22+0.81
93.42-00:1
DGN-ReLU
MLP4-Sin
95.61 20.48
93.80
93.87
DGN-ReLU
DGN-tanh
95.01 -0.47
93.21 +0.72
93.32-0084
31

---

Published as a conference paper at ICLR 2025
GCN
GAT
GraphSAGE
ADGN
PH-DGNc
PHI-DGN
Mean Squared Error
Mean Squared Error
Mean Squared Error
10
-4
10
10-4
10-5
10"3
10-1
10-4
10
10-5
10
35 10
50
35 10
50
35 10
50
Source-target distance (#hops)
Source-target distance (#hops)
Source-target distance (#hops)
(a) Line
(b) Ring
Crossed-Ring
Figure 5: Information transfer performance on (a) Line, (b) Ring, and (c) Crossed-Ring graphs,
including the purely conservative PH-DGN (i.e., PH-DGNc) and PH-DGN with driving forces.
32


Reviews:
[{"review_id": "d6JJf0KjwN", "reviewer": "Reviewer_X6rz", "summary": "This paper introduces port-Hamiltonian Deep Graph Networks (PH-DGN), a new framework for graph neural networks that addresses the challenge of long-range information propagation. The approach embeds message passing within a port-Hamiltonian dynamical system framework, where node states are split into position (q) and momentum (p) components coupled through skew-symmetric matrices. The Hamiltonian function incorporates graph structure through neighborhood aggregation functions, and the port-Hamiltonian extension introduces learnable dissipative components (internal dampening and external forces) that allow the network to balance conservative information preservation with task-dependent information modification. The authors provide theoretical analysis of the framework's properties, including bounds on sensitivity and gradient behavior, and demonstrate empirically that their method outperforms existing approaches on several tasks requiring long-range information propagation, including synthetic graph property prediction tasks and real-world molecular property prediction. The framework can incorporate different message passing schemes and provides improved efficiency compared to previous Hamiltonian-based graph neural networks, with experimental results showing that while the purely conservative version performs well, including the dissipative components often leads to better task performance.", "questions": "1. While the paper shows good empirical performance, it's unclear what types of problems would theoretically benefit most from a port-Hamiltonian approach versus standard message passing. Could the authors provide analysis or insights about which properties of the underlying data generation process would suggest using their method?\n\n2. The authors demonstrate that adding dissipative components often improves performance, but how does the balance between conservative (Hamiltonian) and dissipative parts relate to the nature of the task? It would be valuable to see an analysis of when pure conservation might be preferable to including dissipation, and how practitioners should choose this balance for new problems.\n\n3. Given that the method is inspired by physical Hamiltonian systems, it's surprising that there are no experiments on problems with actual Hamiltonian dynamics (e.g., molecular dynamics simulations). Such experiments could help validate whether the method's conservation properties provide advantages for physically meaningful conservation laws, beyond just improving general information flow.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "Overall, the idea of using Hamilton's dynamics for GNN is attractive, though not entirely new. \nNevertheless, the paper is solid and is more general than existing approaches. \nThe improved efficiency over previous Hamiltonian GNN approaches and the practical benefits for long-range propagation make it a useful contribution to the field.\nThe experimental results are also interesting. \n\n1. Technical soundness:\n- Clear theoretical analysis of conservation properties\n- Explicit connection between Hamiltonian dynamics and message passing\n- Thorough experimental validation\n\n2. Practical value:\n- More efficient than previous Hamiltonian GNN approaches\n- Good performance on long-range tasks\n- Can incorporate different message passing schemes\n\n3. Clarity:\n- Well-structured presentation\n- Good balance of theory and empirics\n- Clear comparisons to prior work", "weaknesses": "1. Novelty is incremental:\n- Builds on existing ideas (Hamiltonian GNNs, message passing)\n- Main contribution is combining these effectively rather than fundamentally new concepts\n\n2. Technical questions:\n- The derivation of the discretization scheme could use more justification\n- Some assumptions about the structure of $W$ and $V$ matrices for explicit updates feel restrictive\n- Could better explain why port-Hamiltonian framework is more appropriate than simpler alternatives\n\n3. Empirical:\n- Some ablation studies could be stronger (e.g., analyzing impact of different dissipative terms)\n- Could better justify hyperparameter choices", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nvIbV12O4i", "reviewer": "Reviewer_XxSd", "summary": "This paper proposes PH-DGN, a new GNN based on the port-Hamiltonian system to develop GNN that can solve graph learning tasks that require long-range dependencies. Two variants of PH-DGN are proposed: a conservative PH-DGN based on the Hamiltonian system and a PH-DGN based on the port-Hamiltonian system by introducing learnable dissipative terms. \nThe theoretical analyses show that the conservative PH-DGN is stable and energy-preserving as a dynamical system, and derive a lower bound for the sensitivity, implying the possibility of long-range interaction.\nNumerical experiments show that the conservative PH-DGN is energy-preserving without gradient vanishing using synthesis datasets (Section 3.1) and that the long-range interaction can be achieved in graph learning tasks that require long-range interactions (Sections 3.2, 3.3). Also, the usefulness of two variants of PH-DGN is evaluated on long-range graph benchmarks of real datasets.", "questions": "**Questions**\n\n- l.161: What is the definition of anti-derivative of an activation function $\\sigma$?\n- l.223, l.228: Which norm is used in $\\| \\partial \\mathbf{b}_u(T) / \\partial \\mathbf{b}_u(T-t) \\|$? \n- l.259: *Therefore, [...]. , it holds the capability of PH-GDN to perform long-range propagation effectively*: the authors compare the upper bounds of sensitivity for the existing model (Theorem 2.3) and the proposed model (Theorem 2.4), then argue that since the latter is larger than the former, the proposed model is more effective in performing long-range propagation. However, it is difficult to claim so because there is the possibility that Theorem 2.4 only shows a looser bound than Theorem 2.3, which does not exclude the possibility that the upper bound does not properly reflect the sensitivity of the PH-GDN. It should be shown theoretically or experimentally that this upper bound adequately reflects the sensitivity of the PH-GDN.\n- l.1129: I want to clarify the setup of the Graph Transfer Task: If I understand correctly, the feature vectors of all nodes are 1-dimensional and randomly sampled from $\\mathrm{Unif}([0, 0.5))$. The target value is 0 for the source node and 1 for the target node. Assuming that this problem setup is correct, I need help understanding why this task is solvable because the model cannot distinguish source or target nodes from other nodes from feature vectors.\n\n\n**Minor Comments**\n\n- l.196: *non-dissipative (i.e., long-range)*: I think this is a slightly misleading expression. My understanding is that this paper uses non-dissipative in the sense of energy-perserving. Although non-dissipative implies long-range propagation (Theorem 2.3), they are not equivalent. In fact, this paper argues that non-dissipative PH-DGN performs better than conservative PH-DGN in predicting the LRGB dataset in some numerical experiments.\n- l.436: The use of position encoding is only mentioned in the caption of Table 2 and should be explicitly stated in the text, specifically in the setup of Section 3.4.\n- l.436: The correspondence between Table 2 and Table 5 needs to be clarified. For example, the method titled *MPNNs* in Table 2 is titled *re-evaluated* in Table 5. However, GCNII is not listed as *re-evaluated*, but as *MPNNs*. Furthermore, it is difficult to tell from the captions of Table 5 whether the authors test each listed method by themselves or is a citation of existing research. The reference should be indicated for each method in Table 5, for example, by adding a column such as *reference* to Table 5.\n- l.443: *Overall, our port-Hamiltonian framework [...] shows great benefit [...] without requiring additional strategies such as global position encoding, global attention mechanism, or rewiring techniques [...].*: I suggest clarifying how the usefulness of each strategy is shown by comparing it with existing methods. More specifically:\n  - The superiority of the proposed method over global position encoding is justified by the comparison with MPNN-based models using position encoding.    　　\n  - The superiority over the global attention mechanism is by comparison with the Transformer-based method.\n  - The superiority of rewiring methods by comparison with Drew.\n- l.1153: The reference to Adam (Kingma & Ba, 2015) should be cited.\n- l.1343: n. layers -> the number of layers\n- l.1353: Table 6 claims that PH-DGN achieves both Hamiltonian conservation and Learnable driving forces. However, I think this is misleading because to achieve Hamiltonian conservation, the dissipative component must be removed, resulting in $\\mathrm{PH-DGN}_{\\mathrm{C}}$. It is not possible to achieve both characteristics at the same time in one architecture. Instead, two variants that achieve only one of the two are proposed.", "limitations": "", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The background knowledge of the port-Hamiltonian system is explained carefully, increasing accessibility for readers unfamiliar with this topic.\n- For conservative PH-DGN, the ability of long-range interaction is theoretically shown by giving the lower bounds of sensitivity.\n- The validity of the proposed methods for synthesis datasets is properly demonstrated.", "weaknesses": "- As the title indicates, the theme of this paper is the relationship between GNN and long-range propagation based on the port-Hamiltonian system. However, no theoretical guarantees on long-range propagation are given for general PH-DGNs with dissipative components.\n- The tables of experimental results could be clearer (in particular Tables 2 and 5).\n- For $\\mathrm{PH-GDN}_{\\mathrm{C}}$, which has theoretical guarantees, the prediction performance on the real dataset (long-range graph benchmark) does not outperform existing methods, and hence its practical usefulness is limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Fe0GVofRt9", "reviewer": "Reviewer_7PXw", "summary": "This work provides a novel methodology for the incorporation of port-Hamiltonian dynamics in graph representation learning. The central model, called a port-Hamiltonian Deep Graph Network (PH-DGN), is introduced first in a purely conservative setting using only the Hamiltonian and no non-conservative terms. Several theorems are developed to show that this conservative case leads to long-range data propagation between graph nodes, where the graph dynamics exhibit no energy loss and gradients do not vanish as the backward sensitivity matrix is bounded below. Dissipitive forces are then added to the full port-Hamiltonian model, which are two additional networks that may be trained to capture non-conservative forces. Several experiments follow, including a showcase of energy conservation and sensitivity to empirically verify theoretical work, and a graph transfer problem and graph property prediction problem to compare performance on benchmark tasks against other graph models in settings which require long-range information propagation.", "questions": "- How would classical GCN aggregation interact with Theorem 2.4? Can the bound be easily extended for that case?\n- In Section 3.1, you mention that the growing behavior can be controlled by regularizing the weight matrices or using normalized aggregation functions. Did you try this? How are the empirical results?\n- Have you examined the interpretability of a trained PH-DGN? In particular, do the learned non-conservative forces make sense for the associated problem?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- There is very clear motivation to this work, and it builds nicely upon other references.\n- The proposed port-Hamiltonian approach is an original and clever way to allow for non-conservative dynamics in a graph network while still maintaining long-range message passing.\n- The theoretical results for the conservative case are strong, and the motivation and interpretation for these are presented nicely.\n- The experimental setup is superb; the care taken to ensure ease of replication is applauded. Model details are presented very clearly and choices are explained well for each step of the setup.\n- A strong suite of models are compared against, with many different competitors and different approaches. The consistently favorable results provide a great deal of strength to claims of the proposed method's performance.\n- The appendices are comprehensive for both proofs and experimental setup, and made clear many of the questions I had on an initial read.", "weaknesses": "- The majority of the theoretical results are developed for the conservative case. This makes sense in the context, as conservative long-range message passing is stated as a goal, but I would also be quite interested to see what could be proven for the fully general port-Hamiltonian case.\n- In the explanation of Theorem 2.3, the statement that \"the final representation of each node retains its complete past\" seems somewhat strong. While I understand that the BSM result shows the influence of the entire past history on the current state, this statement as written seems to imply something stronger, and perhaps could be made more clear.\n- The dissipitive force terms are added in some experiments to great success, but the explanations of their performance are more intuitive and are not supported by hard data in the paper. There may be a great opportunity here for visualization to support the intuitive claims.\n\nThere are two very minor typos:\n- In the first paragraph of Section 2, \"node states' in graph\" has an unnecessary apostrophe.\n- In Appendix D.3, \"on a grid of n. layers\" has an unnecessary period.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)