ODNet: Opinion Dynamics-Inspired Neural Message Passing for Graphs and Hypergraphs
Authors: Bingxin Zhou
Outongyi Lv
Jing Wang
Xiang Xiao
Weishu Zhao
Venue: Accepted by TMLR ()

Abstract: Neural message passing serves as a cornerstone framework in graph neural networks, providing a clear and intuitive mathematical guideline for the propagation and aggregation of information among interconnected nodes within graphs. Throughout this process, node representations undergo dynamic updates, considering both the individual states and connections of neighboring nodes. Concurrently, social networks, as prominent forms of interconnected data, form dynamic systems that achieve stability through continuous internal communications and opinion exchanges among social actors along their social ties. Drawing upon the shared concepts between these two domains, our study establishes an explicit connection between message passing and opinion dynamics in sociology. Moreover, we introduce a novel continuous message passing scheme termed ODNet, which integrates bounded confidence to refine the influence weight of local nodes for message propagation. By adjusting the similarity cutoffs of bounded confidence and influence weights within ODNet, we define opinion exchange rules that align with the characteristics of neural message passing and can effectively mitigate the oversmoothing issue. We extend the framework to hypergraphs and formulate corresponding continuous message passing rules, which reveal a close association with particle dynamics. Empirically, we showcase that ODNet enhances prediction performance across various social networks presented as homophilic graphs, heterophilic graphs, and hypergraphs. Notably, our proposed ODNet outperforms existing GNNs with its straightforward construction and robust theoretical foundation.

Published in Transactions on Machine Learning Research (12/2024)
ODNet: Opinion Dynamics-Inspired Neural Message Passing for Graphs and Hypergraphs
Bingxin Zhou
bingzin.zhouu@kjtuu.dd..c
Institute of Natural Sciences Shanghwi Jiao Tong Universily
Outongyi Lv
edu.ccm
Institute of Natural Sciences Shanghai Jiao Tong University
Jing Wang
shenjjaofeenjcesssjuuud..rr
School of Oceanography
Shanghai Jiao Tong University
Xiang Xiao
gjriao20188ssjh.eduuu edu.co
School Life Sciences and Biotechnology Shanghai Jiao Tong University
Weishu Zhao
swshh88@sjtu..dd edu.c...eeuuu.c...
School of Life Sciences and Biotechnology Shanghai Jiao Tong University
Reviewed OpenReview: /ttps://openreviiee net/ forum? id=ytXFKoCpyk
Abstract
Neural message passing serves as cornerstone framework in graph neural networks, pro- viding clear and intuitive mathematical guideline for the propagation and aggregation of information among interconnected nodes within graphs. Throughout this process, node representations undergo dyuamic updates, considering both the individual states and con- nections of neighboring nodes. Concurrently, social networks, as prominent forms of inter- connected data, form dynamic systems that achieve stability through continuous internal communications and opinion exchanges among social actors along their social tics. Draw- ing upon the shared concepts between these two domains, our study establishes an explicit connection between message passing and opinion dynamics in sociology. Moreover, we intro- duce novel continuons message passing schome termed ODNET, which integrates bounded confidence refine the influence weight. of local nodes for message propagation. By adjust- ing the similarity cutoffs of bounded confidence and influence weights within ODNET, we define opinion exchange rules that align with the characteristics of neural message passing and can effectively mitigate the oversmoothing issue. We extend the framework to hyper- graphs and formulate corresponding continuous message passing rules, which reveal a close association with particle dynamics. Empirically, we showcase that ODNET enhances predic- lion performance across various social networks presented as homophilic graphs, heterophilic graphs, and hypergraphs. Notably, our proposed ODNET outperforms existing GNNs with its straightforward construction and robust thooretical foundation.
1 Introduction
Graph neural networks have emerged as a powerful deep learning framework for analyzing the intercommu- nication of entities graph, offering versatile tools for node, edge, and graph-level representation learning

---

Published in Transactions on Machine Learning Research (12/2024)
boookd sediaaa
C2
aneval
bounded confidence
sepport
uppe0c
OOCIER spectren
strong
upinion
uptitti
Figure 1: An illustration of the propagation rule in ODNET with bounded confidence. Colored nodes represent individuals with different opinions. The central node (dark blue) interacts with light blue nodes sharing similar opinions and red node with siguificantly divergent thoughts. During message passing, ODNET enhances the aggregation weight of similar nodes (light blue) and adjusts the weights of conflict nodes (red). Bounded confidence defines the range of Similar/coffiii nodes and the extent of adjustments, and it is influenced by the graph's position on the opinion spectrum. See Section 5 for modeling details.
(carselli et al. 2008 Bronstein et al. 2017 The primary challenge in designing GNN models lies effec-
tively aggregating information based on local interactions for efficiently extracting hidden representations. This design philosophy has been generalized as newral message possing (Gilmer al. 2017 and later became fundamental feature extraction unit of graph for aggregating features of neighbors.
Concurrently, opinion dynamics studies the evolving social relations with the opinions dynamically changed over time through interactions (Acemoglu & Ozdaglar 2011). A social network establishes a system with individual social actors and the social ties among them. It uses a graph as an abstract representation of a group's structure (Proskurnikov & Tempo 2017 To analyze the formation, evolution, and spread of opinions within social networks communities, statistical physics has contributed by introducing hods and tools from dynamical systems theory (Helbing 2010 Weidlich 2006
Despite originating from different disciplines, neural message passing and opinion dynamics share core princi- ples of information propagation and aggregation within interconnected systems. Both fields focus on under- standing how local interactions among individuals (nodes) contribute to the emergence of global behaviors in social networks (graphs). This inherent connection motivates our exploration of an efficient alternative to neural message passing inspired by popular models in opinion dynamies. We aim address three key research questions: (1) How to integrate models of opinion dynamics into the newral message passing frame- work? (2) What are the implications of opinion dynamics models' properties for mitigating the oversmoothing (3) How can we apply opinion dynamnics-inspired neurul message passing rules to typical graphs encountered in social networks, such as heterophilic gruphs and hypergraphs?
issue in neural message pussing?
To address the first two questions, we analyze the formulation and dynamic behavior of two representative models in opinion dynamics: the Frencl-DeGrool (FD) model (DeGrool 1974: French Jr. 1956) and the Hegselmann-Krannn (HK) model (Rainer & Krause 2002). We reformulate the FD and HK models to estab- lish explicit connections with the message passing framework. Based on the unified formulation, we analyze the convergence behavior of the two models and provide novel explanation of the oversmoothing issue in GNNs from the perspective of opinion dynamics. We demonstrate that the FD message passing converges exponentially to a stable state when exhibiting strong local connectivities. This property is frequently ob- served in hypergraphs and significantly contributes to the oversmoothing issue. Alternatively, the HK model mitigates the issue by introducing heterophilious dynamics to avoid consensus or environmental averaging.
Based on this connection, we answer the third question by integrating the concept of bounded confidence from the HK model and introduce opinion dynamics-inspine message passing formulation named ODNET. A few recent studies combine opinion dynamics to define propagation rules on graphs. However, they are based on position-based first-order message passing (Okawa & Iwata 2022 or ignore data-driven propaga- tion rules GGiráldez-Cru et al., 2022), resulting in limited expressivity. This framework, instead, incorporates a confidence- based filtration mechanism on initial edge connections and incorporates second-ored order diffusion processes. By considering the similarity of node pairs and connection proximity, ODNET aggrogates neigh- boring information through automatic adjustment to edge weights. Through its piecewise message passing schemes, the model can strenthhen, weaken, or remove initial links within the graph. Furthermore, it al-
2

---

Published in Transactions on Machine Learning Research (12/2024)
lows for the assigmment of negative weights to capture conflicting viewpoints from neighboring nodes with substantial disparities. This capability proves essential especially when examining heterophilic networks, where connected entities exhibit dissimilar characteristics. The propagation scheme mirrors the dynamic spreading of opinions within a community, where effective communication channels converge individuals or agents towards consensus or a few dominant viewpoints (Figure D When exchanging information, individ- uals tend to support opinions aligned with themselves. However, when encountering significantly divergent thoughts, individuals may choose to disregard or oppose them, depending on position of the social network the opinion spectrum. For example, researchers (on the right side of the spectrum) typically concentrate studies within their expertise but are open to learning new perspectives from other domains, whereas politicians (on the left side of the spectrum) often encounter strong conflicts and resist propositions from competing parties.
We evaluate the versatile ODNET across three categories of graphs: homophilic graphs, heterophilic graphs, and hypergraphs, each characterized by unique properties resembling real-world social networks such as the small-world effect (Girvan & Newman 2002) and scale-free networks (Barabasi & Oltvai 2004). Empiri- cally, the introduced piecewise aggregation behavior enhances the performance of well-established message passing methods. In summary, our work has three main contributions: (1) We demonstrate an explicit connection between opinion dynamics and the neural message passing framework, which used to be two parallel research fields in modeling social networks (Section f By formulating opinion dynamics models under neural message passing frameworks, we offer a new interpretation of oversmoothing from the perspective of opinion dynamics. (2) We propose ODNet, a novel message passing scheme inspired by opinion dynamics (Section 5.1 ODNET provides a promising alternative to traditional neural message passing methods. It incorporates bounded confidence to refine the inffuence weight of local node features, boasting clean formulation, solid theoretical support, and effective miligation of the oversmoothing issue. Notably, the introduction of bounded confidence does not introduce additional computational cost to the propagation. (3) We defines the continuous ODNet for hypergraphs and provide thcoretical analysis (Section 5.2 By extensive experiments, we validate its superior performance on different types of graphs and compare with both classic and SOTA discrete or continuous message passing schemes.
2
Related Work
Neural Mcssage Passing on Graphs and Hypergraphs Neural message passing establishes a gen-
eral computational rule for updating node representations in attributed graphs [Gimmer el. al. 2017 By implementing different types of propagation rules, information at the node and edge levels is communicated among neighborhoods. In parallel, Feng et al. 2019 and Gao et al. (022bb) established a general convolutionn framework employing the incidence matrix for hypergraph learning. Various techniques have also undergone
expansion, such as the attention mechanism Bai et al. 2021 and spectral theory (addai et al., 2019
Continuous Message Passing and the Oversmoothing Issue The concept of contimuous GNN dy- namics was first introduced by Poli et al. (2020) and Chamberlain et al. (2021) which explicitly generalizes discrete propagation frameworks such as GAT
(Velikkovć et al.] 2018). This diffusion framework has seen active extensions into more continuous graph convolutions with additional considerations of graph rewiring [Brandstetter et al. 2021 Bodnar et al. 2022 Meanwhile, the energy conservation of second-order diffusion processes circunvents the oversmoothing issue and training instability by capturing long-ron mangrange interactions
Waan et al. 2023 Han et al. 2024
Opinion Dynamics for Social Networks and Graph Neural Networks The natural connection
between graphs and social networks has inspired pioneering works exploring the possibility of extending the analysis of social networks to graph neural networks. Sheaf networks build connections with socialdynamics
by separating messages from nodes into public and private opinions (Caralt el al.) Duta el al. 2024 Zaghen
lt al.) 2024 Hansen & Ghrist, 2021). SINN (Okawa & Iwata 2022) follows a first-order formulation of information exchange to capture the positions of individuals without considcring mechanisms to prevent excessive attraction or repulsion from neighbors, thus ignoring the inherent oversmoothing issue. To extend to the second-order formulation with Newtonian dynamics, social networks analysis defines collective dynamics
3

---

Published in Transactions on Machine Learning Research (12/2024)
Carrillo et al. 2010 Motsch & Tadmor 2014) that depicts the phenomenon of agents moving together based
attraction and repulsion forces (Cucker & Smale 2007 Fang et al. 2019 Jin & Shu 2021b). ATBCR GGiáldez--rre et al.) 2022) followed opinion dynamic rules and introduced the concept of attraction and
repulsion between nodes. However, the explicit propagation rule excludes learnable message passing layers,
which restricts its performance on more sophisticated networks such as heterophilic graphs.
3 Graphs from the View of Message Passing and Opinion Dynamics
3.1 Graphs and Hypergraphs
A graph g[W] = (V,E(W]) of N nodes can be associated with any square non-negative weight matrix W E RNNN where V represents the node set and & is the edge set. We denote 2; as the feature of node U; or the opinion of individual G is strongly connected (Godsil & Royle 2001 if there exists a path from every node to every other node. A cycle is a directed path that both begins and ends at the same node with no repeated nodes except for the initial/final one. The length of cycle is defined by the number of edges in the cyclic path. The periodicity of graph is defined as the smallest integer k that divides the length of every cycle in the graph. When k 1, G is termed aperiodic (Bullo et al. 2009
A hypergraph is a generalization of a graph in which an edge can connect any number of vertices. It can be denoted by triple H[W*] &V,E, J To avoid notation confusion, we still use V for the set of nodes and & for the set of hyperedges. We set [V] = N and |E| = M, and E(i) denotes a set containing all the least one hyperedge with node i. Usually, W is a diagonal matrix for hyperedges, where Wh represents the weight of the hyperedge In this paper, we extend the weight representation a triple tensor Wh E R NxNM where uf,,, designates an element in W The incidence matrix H E RN defines Hie 1 if the node belongs to the hyperedge e, otherwise H O. We generalize the indicator within Wh by setting wf, nonzero if node i are connected by hyperedge e, otherwise O.
nodes sharing
3.2 Neural Message Passing on Graphs and Hypergraphs
Neural Message Passing Giimer et al. 2017) stands as the prevailing propagator for updating node repre- sentations in GNNs. We denote a;; ( as the features of node in layer (A: 1) and ag, E Rdd the edge features from node j to node i. A message passing layer reads as follows:
x (k) (K) (xx( (x( DEEN, (()) (x. (k-1) (k-1) x) .a,)).
where O denotes differentiable, node permutation-inwant
function, such as summation, mean, max- imization. The and á denote differentiable functions such as MLPs (Multi-Layer Perceptrons), and M represents the set of one-hop neighbors of node i. The message passing mechanism updates the feature of each node by aggregating their self-features with neighbors' features. On hypergraphs, the message passing scheme considers interactions among multiple nodes reflected in a hyperedge. At the kth layer:
(k) (k+1) = (x) (x: PleEE1 (e, ).(.,)))))))))) (cf,)))
where (-) denotes a differentiable, hyperedge permutation-invarian function, and T0 is a differen- tiable, node permutation invariant. function. The (+)( denotes another differentiable function of propaga- tion, and j e e implies H₁ or af, # O.
4 Opinion Dynamics
This section introduces the FD and HK models, two iconic models in opinion dynamics, following discussion the interconnection of their formulation with message passing and the oversmoothing issue in GNNs.
4

---

Published in Transactions on Machine Learning Research (12/2024)
4.1 French-DeGrood Model
is groundbreaking agent-based model that simulates the evolution of opinions. In a given population of N individuals, each individual holds an opinion x;(k) at discrete time instances k = 0, 1,... An individual's opinion is evolved by
The Prench-DeGrool (FD) model DDerroo 1974
< 1) Σ (k)
(3)
where the non-negative influence weight way satisfies EØ=1 myy = 1. If way > 0, individuals and j are considcred neighbors. The influence weight signifies the relative impact that individual j exerts on during each opinion update. Importantly, all individuals concurrently update their opinions at each time step. The FD model captures how individual opinions converge within group that potentially leading a consensus. This opinion pooling process could also be interpreted a message passing scheme, which emulates how information is exchanged within a specific type of neural network.
Connection to Neural Message Passing The FD model is often regarded as a micro-level model based on individuals simulating the evolution of individual opinions. We show that it shares similaritics with GRAND Chamberlain et al. 2021), which describes a diffusion process on graphs by connecting heat conduction with neural message passing. The connection between the two models is established through the discretization of partial differential equation on graphs:
0 (1) (A(x(() IN)
(4)
Ot
where A ((x())) denotes the N X atlention matrix on nodes and IN is an identity matrix. GRAND coincides with the FD model when (A(x(t)) Ix) satisfies the row-stochastic property and a simple forward-Eie Euler method is applied with a time step of one. This intriguing parallel between the two models highlights the interoonnection of designs from the two different domains of social network modeling.
Opinion Consensus and the Oversmoothing Issue A fundamental result regarding the convergence of the FD model is well-ostablished, demonstrating that consensus is achieved exponentially fast for a strongly connected and aperiodic graph (Bullo et al. 2009 Proskurnikov & Tempo 2017 Ye 2019
Proposition 1. Consider the evolution of opinions æ.(k) for each individual i within the network G[W)
according to 5) Assuming that G[W) is strongly connected and aperiodic, and that W is row-stochastic. Define Ç as the dominant left eigenvector of W, then limg- x(k) = (S" x(0))IN exponentially fast.
The proposition above shows that any graph with self-loop is considered aperiodic, implying that ex- ponential decay often occurs in graphs with strong connectivity. In GNNs, this phenomenon is known oversmoothing NNt & Machara 2019 Oono & Suzuki 2019), which has been widely studied for its association with the exponential decay of the Dirichlet eneryy a measure of feature convergence weighted by graph structure. Despite their distinct origins, these phenomena describe similar processes.
4.2 Breselmann-Krause Model
In the FD model, each agent could interact with any other agent, regardless of their opinions. However, the previous analysis shows that such trivial aggregation behavior inevitably leads to oversmoothing the signals in the system. In real-life scenarios, individuals typically engage in conversations primarily with those who share similar viewpoints. In sociodynamics, this phenomenon is termed as collective behaviors, where a group of individuals collectively exhibit coordinated patterns. The Hegselmann-
(HK) model (Holm & Putkaradze 2006; Kolokolnikov et al. 2013 Motsch & Tadmor 2014 characterizes this behavior by
< (k + [Bi,
Σ
(,)k),
(E
where the bounded confidence 2i,, {j | - < e) includes all the peers j associated with the individual whose opinions diverges within a confined region 6; E R. This parameter defines the degree of
5

---

Published in Transactions on Machine Learning Research (12/2024)
uncertainty tolerance for opinion exchanges. It restricts that individual i with opinion æ, will only engage in interactions with those of their peers whose opinions fall within confined region.
Consensus versus Opinion Polarization The HK model is first-order opinion dynamics model that directly incorporates the influence of neighboring opinions on individual updates through bounded confidence. Nevertheless, bounded confidence docs not inhorently prevent the oversmoothing issuc. In scenarios of significant. heterophily, agents preferentially connect with dissimilar others. In this case, the model often leads to consensus and mirrors the oversmoothing problem in GNNs. It thus becomes pivotal to guide the HK model to capture opinion polarization and form multiple distinct and stable clusters. One way to approach this target involves employing strategies like bi-clustering with repulsion mechanisms (Fang et al. 2019 Jin
& Shu 2021b
Wang et. al. 2023
which ensures a lower bound for the Dirichlet energy of the input signal.
5 Opinion Jymamics-Inspinng Message Passing
This section defines ODNET, novel message passing framework that employs the influence function ((s) with bounded confidences. We offer comprehensive interpretation of each component in ODNET. We introduce the discrete and continuous forms of the proposed model on both graphs and hypergraphs.
5.1 ODNet Graphs
Discrete Formulation Denote ((-) non-decreasing function of the similarity measure stj to node and node j. The update rule for ODNET reads:
-z) a, + Σ =00) (t) (t).
(6)
The matrix Sij has different definitions, such as the normalized adjacency matrix [Kipf & Welling, 2017 and attention coefficients (Veličković et al. 2018). The control term u(x,) is deployed for stability. One ((x)))==;; with potential function P(x), such that 2011 It acts as a moral constraint preventing individuals from P((x) + co as x + с (Kolokolnikov et al.
straightforward choice of formulation is u(z,)
resorting to extreme violence in confiitt situations in sociodynamics.
Continuous Formulation In opinion dynamics, individual perspectives tend to evolve gradually rather than through sudden changes. This observation suggests enhancing the discrete message passing model into a continuous framework. This adaptation can be understood as transforming traditional message passing approach into a numerical approximation of the continuous model by rewriting 4 into:
(t) = Σ Øt
(t) - xx(()) + u(x;).
(7)
The continuous formulation 7 facilitates the application of numerous numerical approximation methods, transforming it back into a discrete model equipped with a specific scheme for residual adjustment. This adaptability allows applying diverse Ordinary Differential Equation (ODE) solvers ODNET, including
Neural ODEs (Chen al. 2018 The associated explorations can be found in Section
Choice of Influence Function The monotonic influence function Φ(-) characterizes the influence weight associated with the similarity of node pairs. It describes the diverse behaviors of opinion exchange in system, which is determined by the systems' pasitions along the opinion spectrum (Figure I. We define a piecewise d(-) to delineate influence regions akin to bounded confidence. For instance, with
us
if <
((s) =
8,
if El
S otherwise,
(8)
0,

---

Published in Transactions on Machine Learning Research (12/2024)
the propagation rule
B can be rewritten
(t + =
(t)(a)(t) xz(()) + 62
E
E -1,((,(()=z1(1)) + (t).
(9)
Se
This formulation amplifies the influence weight for highly similar node pairs with coefficient. > 0 while cutting connections for node pairs with low similarity, resembling how individuals tend to ignore opinions beyond their bounded confidence.
The previous choice corresponds to when individuals maintain neutral opinions toward each other. Con- versely, when individuals holding significantly divergent opinions and the network leans towards the 'strong opinion' side of the spectrum, interactions may become hostile. In such cases, Φ(.) can be defined
uss 8,
if 62 <8
((s)
if s otherwise,
(10)
v0 ((1ss) s),
where μ > and < 0. This configuration allows the neural network learn from positive neighbors with high similarities and to extract negative information from nodes with discrepancies. Specifically, the negative coefficient indicates that certain node pairs consistently repel each other, potentially leading to undesirable system expansion. Consequently, the previously introduced control term u(z;) is essential for the model towards enhanced system stability.
Remark 1. The choice of (6-) can be guided by the graph's 's intrinsic characteristics or by using the homophily level Zhhou el al. 2023) as quanlilalive measure. We recommend using 6 for homophilic graphs and 10 for heterophilic gruphs. The empirical analysis is in Section 0
5.2 ODNet on Hypergraphs
It conccivable to extend graph diffusion framework, e.g., GRAND, to hypergraphs with macroscopic in- terpretations. However, these diffusion-type dynamics the macro level are susceptible to oversmoothing of feature evolution, similar to traditional GNNs. Especially hypergraphs, since generally characterized with denser local connections than traditional graphs, addressing the oversmoothing issue becomes partic- ularly important when constructing message passing propagation rules for hypergraphs. In this section, we will first analyze the oversmoothing issue of diffusion-based hypergraph message passing, then propose the continuous update rule of ODNET for hypergraphs that circumvent the oversmoothing issuc.
Diffusion-based Hypergraph Message Passing When incorporated with macroscopic interpretations, diffusion-basss based hypergraph message passing propagates the attractions of individuals or agents. Consider the node feature space = Rd and the tangent vector field space TΩ Rdd For x, E and F E 7Ω where Fi,j = - Fj.i, we adopt the following inner products:
(x,y)
Σ x,y,, bg
[F,
+.0==E>
ΣΣ i>J EEE
Mi Fi.
(11)
Here hi, represents tuple related to node i,j and the associated hyperedge e, where he, 0 if H. 0.
We set hi to satisfy Σ Σ hig For any u E TΩ, by the adjoint relation [u, Vx] = (x, div u) with
3EE
Vx= Vx=X; X;, derive:
(div u)5
ΣΣ M.u.i. Ui.j-
3EE
This leads a formal diffusion process of hypergraph:
dx;
div Vx,
hi hig x.). Σee
(12)
dl

---

Published in Transactions on Machine Learning Research (12/2024)
For simplicity, define the hypergraph operator L= - (E (( and rewrite [2] aEE
dx -x.
When is semi-positive definite (s.p.d), we define different choices of h, lead to diverse diffusion-type equations. For example, when take forward Euler discretization on 12 and define
12 as a diffusion-type propagation hypergraph. The
Σ
Mas MMg 337
D HWD=
we have simplified HGNN
Gao et al. 2022a
without channel mixing.
Oversmoothing Problem of Hypergraph Message Passing To understand this claim, we first define the Dirichlet energy of a hypergraph H of vector field x E RRxd for diffusion-type hypergraph networks:
N E(x) = Σ H. - xx]1
(13)
When formulating the Dirichlet energy on hypergraph, rather than following the conventional formulation of E(x) := tr(x Lx), we adopt simplified definition by Rusch et al. (2022) due to the non-deterministic nature of C, which also facilitates a straightforward analysis of the variations in node features.
Definition 1. Let x! represent the hidden features at the t layer. Oversmoolhing in a hypergraph neural network is defined as the exponential decay of layer-wied wise Dirichlet energy as a function of layer depth I, i.e,,
E(x²)= S -ie-
(14)
where Cj and C2 are positive constants.
Given that [x] & Cie with y being the smallest positive eigenvalue of C, it is easy to see that oversmoothing is a common issue in general hypergraph diffusion networks. This observation comes from the diffusion structure, where the node features x decay exponentially zero with the s.p.d kernel L.
Continuous Formulation of ODNet Hypergraphs ODNET offers comprehensive message pass- ing framework that can be directly extended to hypergraphs. The primary distinction between hypergraphs over graphs lies in how hyperedges facilitate connectivity beyond traditional pairwise edges. Following the continuous representation of 12 we define:
dr: ΣEMS(n)a - -x;) + u(x;).
(15)
Öt
EW JEC
This model formulates collective behaviors-based diffusion-type dynamics for hypergraphs. It suggests that in a large community, information spreads through subgroups (nodes connected by a hyperedge) rather than solely through direct pairwise exchanges. As with graphs, the selection of s can include different metrics, such attention coefficients (Bai al. 2021 convolution coefficients (Gao et al. 2022al
The HK model can be interpreted diffusion process on graphs featuring piecewise al traction and repulsion behaviors, as outlined in 7 and 15 When devising message passing aggregation rules, employing a comprehensive framework for microscopic models based on collective behaviors within a complete interaction system proves effective in mitigating the oversmoothing issuc. The complete proof is in Appendix A
Remark 2.
The diffusion process of a hypergraph by 12 is also closely related to self-organized dynamics in particle systems [Mtsch & Tadmo1, 2014), where 12 represents a particle dynamics scenario with hi, signifying the interactive force between nodes i,j under a specific field e. Note that 12) corresponds to particular case of [5) where only aftractive forces influence the message evolution. However, this assumption is not universal for particle systems, and as demonstrated above, it can lead to the oversmoothing issue.
8

---

Published in Transactions on Machine Learning Research (12/2024)
Table 1: Average test accuracy on homophilic graphs over 10 random splits.
Cora 0.83
CiteSeer
PubMed
Coauthor CS
Computer
Photo 0.83
ogb-arXiv
homophily level
0.71
0.79
0.80
0.77
0.61
GCN
8..
71.9+1.9
77.8+2.9
91.1+0.5
82.6+2.44
91.2+1.2
72.2+0033
MONET
81.3113
71.242.0
78.612.3
90.810.6
83.5.12.2
91.2.12.3
N/A
AGE-HS
79.217.7
71.611.9
7..412.2
91.342.8
82.411.8
91.411.3
N/A
GGE-HXAGE-Max
76.611.9
67.542.3
76.1122.3
85.011.1
N/A
90.41:1.
N/A
GAT
81.11:3
7..41.99
78.712.3
90.510.6
78.01:1.
85.742.0
73.610.1
APT-PPR
81.6+0.3
68.5+0.2
76.7+0.3
91.3+0.1
85.4+0.3
90.9+0.3
N/A
CGNN
81.4+1.6
66.9+1.8
66.6+4.4
92.3+0.2
80.3+-200
91.4+1.55
58.7+2.5
GDE
78.7+2.2
71.8+1.1
73.9+3.7
91.6+0.1
82.9+0066
92.4+2.0
56.7+1009
GRAND-L.
83.6+1.0
73.++0.5
78.8+1.7
92.9++04
83.7+-1.2
92.3+-0.9
71.8+0.2
SINN
83.210.4
73.9
80.0
93.010.5
84.611.1
90.4 #2.4
N/A
ODNET (ours)
85.710.33
75.5+1.2
80.611.11
93.1
83.911.5
92.7110.6
74.65=10.16
6
Empirical Analysis
We validate the performance of ODNET through classic node-level represcntation learning tasks variety of social networks described by homophilie/hettrrphiiic graphs or hypergraphs.
6.1 Experimental Protocol
Training Setup We compare model to diverse set of top-performing baseline GNN models, including classic message passings with discrete continuous updating schemes, well as the latest hypergraph models. For ODNET, trained the model using neural ODE solver with Dormand Prince adaptive step size scheme (DOPRI5). For homophilic datasets, we utilized 10 random weight initializations and random splits, with each combination randomly selecting 20 instances from each class. In heterophilic and hypergraph datasets, we used the fixed 10 training/validation validation splits in Pei et al. (2020) and Yadati et al. (version 2.0.1) (Fey & Lenssen 2019 and PyTorch (version 1.7.0) and run on NVIDIA Tesla A100 GPU with 912 CUDA cores and 80GB HBM2 mounted
(2199. All implementations are programmed with PyTorch-Geomett
HPC cluster. All the details reproduce our results have been included in the submission. The prograin will be publicly available upon acceplance. The searching space of hypergraphs is reported in Table 7 We compare ODNET with a variety of previous methods on graphs and hypergraphs (soe Appendix B.
6.2 Node Classification on Graphs
Benchmark Datasets
To evaluate a diverse set of scenarios, we encompass seven homophilic graphs
(Cora McCallum et al.
2000 Citeseer (Sen et al. 2008 Pubmed (Namata et al. 2012 Coauthor
CS Shehur et al. 2018), Computer (Naata et al. 20121. Photo (Namata et al. 2012). ) and ogb-arXiv
Hu et al. (2020)), three heterophilic graphs (Texas, Wisconsin, and Cornell from the WebKB dataset (Garcia-Pa Plaza al. 2016)). An overview of the statistical information for the six homophilic graphs and three heterophilic graphs is provided in Table 4 of Appendix B The benchmarks used for evaluation
wide range of homophily level (hou et al., 2023 A low homophily level indicates that the dataset leans more lowards being helerophilie, where most neighbors do not share the same class as the central node. Conversely, a high homophily level signifies that the dataset tends towards homophily, with similar nodes more likely to be interconnected.
Result Analysis Tables 2 present the average accuracy for predicting node labels in homophilic and het- crophilic graphs, respectively. ODNET consistently ranks among the top-performing methods with minimal variance. The performance results for baseline methods are sourced from prior studies (Chamberlain al.,
2021 Chien al. 2021 Wang al. 2022
Notably, our model outperforms other continuous message pass-
9

---

Published in Transactions on Machine Learning Research (12/2024)
Texas
Texas
Cora
Cora
90
90
86
86
aceurac
80
80
82
82
70
70
78
78
-1000
100050
-10
-10
12
10
50
100
1000
-1
1.0
1.2
1.4
1.6
1.8
2.0
1
Figure 2: Impact of scaling factors and j on Texas and Cora.
Table Average test accuracy on heterophilic graphs over fixed split datasets.
Texas
Cornell Wisconsin
homophily level
0.11
0.21
0.30
MLP
80.8+4.88
85.3+3.3
81.9+6.4
GCN
55.115.2
51.843.1
60.5+5.3
GAT
52.246.6
49.4+4.1
61.9+5.1
GPRGNN
78.4144
82.914.2
80.318.1
H2GCN
84.9+7.2
87.7+ 5.0
82.7+5.3
GCNII
77.613..8
80.443.4
77.9
GEOM-GCN
66.812.7
64.513.7
60.5+3.7
PAIRNORM
60.3+43.
48.4+6.1
58.9+3.2
GRAPHSAGE
82.4+6.1
81.2+5.6
76.0=5.0
GRAPHCON
85.4+44.2
87.8+-3.3
84.3++4.8
ODNET
88.2+3.2
89.1 2.9
8..5
ing techniques, such as GRAND, by introducing the potential term and the bounded confidence mechanism. This superiority is particularly evident on heterophilic graphs, where the repulsive force among dissimilar node pairs significantly enhances prediction accuracy. Furthermore, Figure 2 illustrates the distinct pref- erences the influence function for homophilic and heterophilic graphs. Based on the theoretical analysis and empirical evidence, we recommend following for the former and 1C for the latter in general. The similarity cutoff also exhibits differing preferences. In homophilic graphs, nodes tend to amplify attraction among similar entities, while in heterophilic graphs, dissimilar nodes benefit more from emphasizing repul- sion. Additional evidence is provided in Section 6.5 Furthermore, the introduced bounded confidence in ODNET, similar to am activation function, does not increase the computational cost. For instance, training one round of CORA (on single A100 card) requires approximately 120 seconds for ODNET, 135 seconds for GRAND, and 164 seconds for SINN.
6.3 Node Classification on Hypergrpahs
Benchmark Datasets The hypergraph variant of ODNET undergoes an evaluation through semi- supervised node classification tasks conducted four benchmark hypergraphs extracted from Yadatí et al. 2019)) For co-citation networks (Cora-cocitation, CiteSeer-cociiation, and PubMMed-cociittion)) documents cited by a given document are intercomnected by hyperedge. Similarly, the co-authorship net- works (Cora-coauthor) aggregates all documents co-authored by an individual into a single hyperedge. A statistical summary of the four benchmarks provided in Table 5 in Appendix
In contrast to graph data with relatively sparse connections, hypergraphs utilize few hyperedges and establish densely connected local communities. As reported in Table g ODNET consistently outperforms baseline methods with a significant improvement. It is worth noting that our ODNET adopts the hypergraph weights at from HGNN (Feng et al. 2019) with a simple Euler scheme of first-order forward dif- ferences. In this sense, our method demonstrates great potential for significantly enhancing the performance of basic method with minimal additional complexity, surpassing even advanced methods.
Result Analysis
10

---

Published in Transactions on Machine Learning Research (12/2024)
Table 3: Average test accuracy on hypergraphs over 10 random splits.
Cora-coauthor
Cora-cocitation
CiteSeeer-ooiiaatio
PubMed-cocitatioo
HGNN
82.6-1.7
79.411.4
72.511.2
86.410.4
HYPERGCN
79.5-2.1
78.5+1.3
71.3++0.8
82.8+8.7
HCHA
82.6=1.0
79.1#1.0
72.411.4
86.4+0.4
HNHN
1.5
76.1+1.9
72.6+1166
86.9++0.3
UNIGCNII
83.6=1.1
78.8#1.1
73.0#2.2
88.3#0.4
HYPERND
80.6=1.3
79.2+1.1
72.6+1.5
86.7#0.4
ALLDEEPSETS
82.0=1.5
76.911.8
70.8
88.810.3
AU.SETTRANS
83.6-1.5
78.6+1.5
73.1+1.2
88.7+0.4
SheafHGCN
83.2=1.2
80.1
73.240.5
87.1
ED-HNN
84.0=1.6
80.3+1.4
73.7411.4
89.0++0.5
ODNET (Our)
84.5+1.6
80.710.9
74.0+0.9
89.0=1.44
300
GCN
E00
GAT
GRAND
100
ODNet
0
10
20
30
layer
Figure 3: Decays of Dirichlet energy with layers on Texas.
6.4 Dirichlet Energy and Community Consensus
Many message passing methods encounter the issue of oversmoothing. limiting their ability to enable deep networks to achieve expressive propagation. A GNN model is considered to alleviate the oversmoothing problem its Dirichlet energy rapidly approaches a lower bound as the number of network layers increases (Cai & Wang 2020). Figure illustrates the decay Dirichlet energy on Texas with all network parameters randomly initialized. The two conventional message passings, GCN and GAT, exhibit sudden progression of Dirichlet energy with exponential decay. In contrast, GRAND employs small multiplier to delay all nodes' features to collapse to the same value. ODNET stabilizes the energy decay with bounded confidence and the influence weights, offering simple and efficient solution to alleviate the oversmoothing issue. Since stable Dirichlet energy reflects the disparity of feature clusters, the observation that a decreasing profile of Φ reduces Dirichlet energy under stable conditions consistent with simulation results in opinion dynamics that heterophily dynamics enhances consensus (Motsch Tadmor 2014). Note that the 'heterophily' in opinion dynamics refers to the tendency of graph to form stronger connections with those who are different rather than those who are similar, which is different term from the 'heterophilic graph' in GNNs.
6.5 Additional Investigation
Neural ODE Solvers The Dormand Prince adaplive step size scheme (DOPRI5) served as the neural ODE solver for ODNET. Additionally, we evaluated the performance of two other solvers across various datascts: the Rungo-Kutta method (rk4) and the first-order Euler scheme (Euler). The outcomes are presented in Figure g Although different solvers did not consistently demonstrate a significant and sustained advantage of one the others, our selection of DOPRI5 yielded the overall best results.
11

---

Published in Transactions on Machine Learning Research (12/2024)
ODE Solver
90
DOPRIS
Enter
70
Cora
CiteSeee
PuhMel
Texas
Wismonsin
Comell
Figure Prediction performance with different neural ODE solvers.
T-1
T=10
T=50
climension
dimmession
diuueasion
dimension
dimension
dimension
Figure 5: t-SNE visualization on node embeddings for Texas.
Embedding Dynamics We employ the (-SNE algorithm to visualize the embedded features in two- dimensional plane for the Texas dataset. We choose the output embeddings from the last layer at different epochs. Figure S demonstrates an evident clustering trend the number of training epochs increases. By the 50th epoch, nodes with different labels are distinctly separable in the reduced two-dimensional space.
Influence Function We next dolve into the impact of different selections of the influence function on the performance of ODNET. This investigation encompasses various aspects, including the choice of scaling factors (y and v) and the definition of the similarity cutoffs (E1 and E2). Our findings are meticulously detailed in Tables in Appendix
B with particular focus the homophilic graph (Cora) and the heterophilic graph (eexas), respectively. An interesting trend emerges concerning the parameter v, indicating clear preference. Specifically, it is advisable to incorporate a repulsive effect on heterophilic graphs by assigning a negative value to μ. Conversely, for homophilic graphs, where similarity plays a pivotal role, setting v = 0 is more appropriate. To provide a direct comparison, Figure D showcases ODNET's performance under different similarity cutoffs, i.e., 61 and 62- For Cora, we maintain μ = 1.4 and set === 1.0 and -50. Generally, in the context of homophilic graphs, sctting a relatively small value for 52 tends to expand the region of nodes considered similar. This approach may be beneficial in mitigating the oversmoothing issue. Conversely, for heterophilic graphs, it is advisable to reverse information from only highly similar nodes, reflected in the choice of larger 62 value (up to 0.8). However, it is crucial to exercise cantion when pushing Eg towards 1.0, as discernible reduction in performance bocomes evident.
0, while for Texas, we
Texas
Texas
Cora
Cora
aucurey
90
90
90
80
80
80
80
1.2
3.0
4.0
2.0
3.0
1.04.5.00
0.3
0.5
0.7
0.6
0.7
0.8
0.9
1.0
6x(11022)
6x(1001)
F1
62
Figure 6: The impact of different e and e2 on ODNET.
6.6 Case Study: Biological Social Network Architecture Simplification
Microorganisms are the most extensively distributed and numerous group on Earth, which thrive in wide array of moderate and extreme environments, such deep-sea hydrothermal vents, ocean trenches, and
12

---

Published in Transactions on Machine Learning Research (12/2024)
(a) MT- original
(b) MT simplified
(+) ME- original
(d) ME simplified
nox
not
Corple
Figure 7: Co-occurrence network of selected metabolic genes in ME or MT before (a-b) and after (c-d) being simplified by ODNET. Connections are considered strong with weights> 0.05, and genes without a strong connection with any other peers are removed.
They present the greatest diversity of life on Earth. The remarkable diversity among microorganisms finds its primary expression through their intricate metabolic pathways (Louca al., 2018 Coelho et al., 2022 Consequently, investigating the connections between microbial metabolism in distinct environments carries profound significance in unraveling the intricate interplay between Earth's diverse ecosystems and the lives inhabiting them. Metagenomic analyses have revealed the remarkable complexity inherent to metabolic gene networks, due to the diversity and richness of functional genes and their interconnections. It thus becomes a necessity to simplify metabolic gene networks for investigating relationships among functional genes and key genes. Presently, the provailing approach involves adjusting connection weights to streamline the network, often relying on biological expertise. However, the absence of standardized simplification criterion results in heavy bias in network structures influenced by the subjective opinions of biologists.
platcaus (hhu & Huang 2022
Problem Formulation and Training Setup As an example of the environmental microbiome analysis, the co-oecurrence network is challenging to interpret due to the massive and complicated characteristics of genes and the unclear standard for assessment. The target here is to learn meaningful influence weights between gene pairs that simplify the co-occurrence network with effective biological justification. To this end, two networks originated from the microbial comparison between the Mariana Trench (MT) and Mount Everest (ME) networks (Liu et al. 2022) are utilized, where nodes are functional genes and edges are weighted by the probability of two key functional gene sets simultaneously occurring in the same species. Edges with exceptionally small weights will be discarded as noisy observations. As we are cager to identify the key genes and gene clusters from gene interactions, we construct the graph with initial connectivily (edges and edge weights), leaving any node attributes (e.g., function and pathway) unobserved. We define a node-level classification task for predicting whether node is 'strong', 'medium', or 'weak' inffuencer to its community, where the three levels are cut by their degree. Further details are provided in Appendix D
Result Analysis We trained two independent ODNETS on ME and MT networks, which achieved predic- tion accuracy as high as 96.9% and 75.0%, respectively. For both networks, melabolic genes were classified based solely on topological information, without the introduction of any a priori node features. Figure 7 visu- alizes the two networks in their original and the simplified appearance, respectively. For all the networks, an edge weight cutoff of 0.05 was applied to eliminate weak connections that could not be distinguishable from background noise. The original network without any simplification appeared cluttered and difficult to inter- pret (Figure bb). In contrast, the simplified networks greatly enhanced the readability of the co-occurrence network while retaining reasonable biological significance (Figure -dd. Furthermore, the simplified network was able to identify the biologically key genes that acted as "opinion leaders", serving bridges connecting different metabolic pathways. For example, in the MT network, the key gene nitrous oxide reductase (nos) bridged the carbon (Alkane, Aromatic, Complex sugar, D-AA and L-sugar) and nitrogen metabolism (N) in Figure 77 whereas in the ME network, the key genes of sulfate reduction (sat and aprA) coupled the carbon and sulfur metabolism (S) (Figure l). Thus, ODNET could be employed to present more discernible networks in environmental microbiome studies, and aid in comprehending key metabolic functions within microbiomes from diverse environments.
13

---

Published in Transactions on Machine Learning Research (12/2024)
7 Conclusion
This study establishes intriguing connections between sociodynamics and graph neural networks, two distinct fields that both actively investigate social networks from different perspectives. By bridging concepts from these two fields, we introduce bounded confidence for neural message passing, a novel mechanism inspired by opinion dynamics. The proposed ODNET effectively addressos the oversmoothing issue and consistently achieves top-notch performance in node prediction tasks across graphs with diverse local connectivity pat- terns. This success is attributed to the simplicity and efficacy of our piecewise message propagation rule.
In addition to providing strong thcoretical support and empirical performance, ODNET integrates theories from social networks, offering intuitive explanations for hyperparameter settings and the significance of net- work parameter learning. This integration showcases ODNET's significant potential in simplifying complex real-world social networks, presenting a fresh analytical approach that does not rely on existing attributive classification conventions. The robust performance of ODNET extends its applicability to simplifying in- tricate networks containing wealth of biological information, such as genes, gene-gene interactions, and metabolic pathways. This method's exceptional capacity to extract accurate insights and unveil the intrinsic mechanisms of cellular physiology provides invaluable support to biologists in dociphering the mechanisms of adaptation and pathway functions in the microbial realm. These findings are significant in understanding the interactions between Earth's 's environments and the metabolism of life.
Acknowledgments
The authors acknowledge support from the National Natural Science Foundation of China (62302291, 92451301).
References
Daron Acemoglu and Asuman Ozdaglar. Opinion dynamics and learning in social networks. Dynamic Games and Applications, 1:3-49, 2011.
Song Bai, Feihu Zhang, and Philip HS Torr. Hypergraph convolution and hypergraph attention. Pallern
Recognition, 110:107637, 2021.
Albert-Laszlo Barabasi and Zoltan N Oltvai. Network biology: understanding the cell's functional organi- zation. Nature reviews genetics, 5(2):101-113, 2004.
Ayodeji Bello, Yue Han, Haifeng Zhu, Liting Deng, Wei Yang, Qingxin Meng, Yu Sun, Ugochi Uzoamaka Egbeagu, Siyuan Sheng, Xiaotong Wu, et al. Microbial community composition, co-occurrence network pattern and nitrogen transformation genera response to biochar addition in cattle manuro-maize straw composting. Science of the Tolal Environment, 721:137759, 2020.
Cristian Bodnar, Francesco Di Giovanni, Benjamin Chamberlain, Pictro Lio, and Michael Bronstein. Neural sheaf diffusion: A lopological perspective on heterophily and oversmoothing in gnns. Advances in Newral Information Processing Systems, 35:18527-18541, 2022.
Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik J Bekkers, and Max Welling. Geometric and physical quantities improve (3) equivariant message passing. In ICLR, 2021.
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond Fuclidean data. IEFE Signal Processing Magazine, 34(4):18 42, 2017.
Francesco Bullo, Jorge Cortés, and Sonia Martinez. Distributed control of robotic networks: mathematical approach to motion coordination algorithms, volume 27. Princeton University Press, 2009.
Chen Cai and Yusu Wang. A note on over-smoothing for graph neural networks. arXiv:2006. 133118 2020.
Ferran Hernandez Caralt, Guillermo Bernardez, Iulia Duta, Eduard Alarcon, and Pictro Lio. Joint diffusion processes as an inductive bias in sheaf neural networks. In ICML 2024 Workshop on Khometry-groundde Representation Learning and Generative Modeling.
14

---

Published in Transactions on Machine Learning Research (12/2024)
José A Carrillo, Axel Klar, Stephan Martin, and Sudarshan Tiwari. Self-propellle interacting particle systems with roosting force. Mathemalical Models and Methods in Applied Sciences, 20(supp01))155 ::1mpp33)):133---- 1533-1552, 2010.
Benjamin Paul Chamberlain, James Rowbottom, Maria I. Gorinova, Stefan D Webb, Emanuele Rossi, and Michael M. Bronstein. GRAND: Graph neural diffusion. In ICML, 2021.
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In ICML, pp. 1725-1735. PMLR, 2020.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential oquations. In NewrIPS, volume 31, 2018.
Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph neural network. In ICLR, 2021.
Eli Chien, Chao Pan, Jianhao Peng, and Olgica Milenkovic. You are allset: A multiset function framework for hypergraph neural networks. In ICLR, 2022.
Luis Pedro Coelho, Renato Alves, Álvaro Rodríguez Del Río, Permille Neve Myers, Carlos P Cantalapiedra, Joaquín Giner-Lamia, Thomas Sebastian Schmidt, Damiel R Mende, Askarbek Orakov, Ivica Letunic, al. Towards the biogeography of prokaryotic genes. Nalure, 601(2)9)):252 2262, 2022.
Felipe Cucker and Steve Smale. Emergent behavior in flocks. IEEE Transactions on Automatic Control, 52 (5):852-8622 2007.
Morris H. DeGroot. Reaching a consensus. Journal of the American Statistical Association, 69(345):118- 121, 1974. ISSN 01621459.
Yihe Dong, Will Sawin, and Yoshua Bengio. Hnhn: Hypergraph networks with hyperedge neurons.
TVf:2000612274 2020.
Iulia Duta Giulia Cassarà, Fabrizio Silvestri, and Pietro Liò. Sheaf hypergraph networks. Advances in Newral Information Processing Systems, 36, 2024.
Di Fang, Seung- Yeal Ha, and Shi Jin. Emergent behaviors of the Sncker-Smale ensemble under attractive- repulsive couplings and rayleigh frictions. Mathematical Models and Methods in Applied Sciences, (97) 1949-1385 2019.
Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. AAAI, volume 33, pp. 3558 3565, 2019.
Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
John RP French Jr. A formal theory of social power. Psychological revieu, 63(3):181, 1956.
Yue Gao, Yifan Feng, Shuyi Ji, and Rongrong Ji. HGNN + General hypergraph neural networks. IEEE TPAMI, 2022a.
Yue Gao, Zizhao Zhang, Haojie Lin, Xibin Zhao, Shaoyi Du, and Changqing Zou. Hypergraph Learning: Methods and Practicos. IEEE TPAMI, 44(5), May 2022b. ISSN 1939-3539.
Alberto P García-Plaza, Victor Fresno, Raquel Martínez Unanue, and Arkaitz Zubiaga. Using fuzzy logic to leverage html markup for web page representation. IEEE Transactions on Fuzzy Systems, (514)9119- 2016.
Justin Gilmer, Samuel S Schocnholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In ICML, 2017.
Jesús Giráldez- Cru, Carmen Zarco, and Oscar Cordón. Analyzing the extremization of opinions in a general framework of bounded confidence and repulsion. Information Sciences, 609:1256-128 1270, 2022.
15

---

Published in Transactions on Machine Learning Research (12/2024)
Michelle Girvan and Mark EJ Newman. Community structure in social and biological networks. PNAS, 99 (12):7821-7826, 2002.
Chris Godsil and Gordon F' Royle. Algebraic graph theory, volume 207. Springer Science & Business Media, 2001.
William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. NewrIPS, 2017.
Andi Han, Dai Shi, Lequan Lin, and Junbin Gao. From continuous dynamics graph neural networks: Neural diffusion and beyond. Transactions Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreevie set//forum/idd1PPSxjjeee Survey Certification.
Jakob Hansen and Robert Ghrist. Opinion dynamics discourse sheaves. SIAM Journal Applied
Mathematics, 81(5):2033 2000)20000,, 2021.
Dirk Helbing. Quantitative sociodynamics: stochastic methods and models of social interaction processes. Springer Science & Business Modia, 2010.
Darryl D Holm and Vakhtang Putkaradze. Formation of clumps and patches in self-aggregation of finite-size particles. Physica D: Nonlinear Phenomena, 220(2):183-15) 11111188-96611111 2006.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning graphs. Advances in neural information processing systems, 33:22118 22133, 2020.
Jing Huang and Jie Yang. Unignn: a unified framework for graph and hypergraph neural networks. arXiv:2105.0000 00956, 2021.
Shi Jin and Ruiwen Shu. Collective dynamics of opposing groups with stochastic communcctiio. Vietnam Journal of Mathematics, 49:619 636, 2021a.
Shi Jin and Ruiwen Shu. Collective Dynamics of Opposing Groups with Stochastic Communcattion, Vietnam
Journal of Mathematics, 49(3):619-66 636, September 2021b. ISSN 2305-221X, 2305-2228. doi: 10.1007/ -10013-20000040--2
Thomas N. Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
Theodore Kolokolnikov, Hui Sun, David Uminsky, and Andrea L Bertozzi. Stability of ring patterns arising from two-dimensional particle interactions. Physical Review E, 84(1):015203, 2011.
Theodore Kolokolnikov, José A Carrillo, Andrea Bertozzi, Razvan Fetecau, and Mark Lewis. Emergent behaviour in multi-particle systems with non-local interactions, 2013.
Roie Levy and Elhanan Borenstein. Metabolic modeling of species interaction in the human microbiome elucidates community-leve assembly rules. PNAS, 110031):12804-1228) 2013.
Yongqin Liu, Zhihao Zhang, Mukan Ji, Aoran Hu, Jing Wang, Hongmei Jing, Keshao Liu, Xiang Xiao, and Weishu Zhao. Comparison of prokaryotes between mount everest and the mariana trench. Microbiome, 10(1):215, 2022.
Stilianos Louca, Martin Polz, Florent Mazel, Michaeline BN Albright, Julie A Huber, Mary O'Connor, Martin Ackermann, Aria S Hahn, Diane S Srivastava, Sean Crowe, et al. Function and functional rodundancy in microbial systems. Nature ecology evolution, 2(6):936 943, 2018.
Andrew Kachites McCallum, Kamal Nigam, Jason Rennic, and Kristie Scymore. Automating the construc- tion of internet portals with machine learning. Information Retrieval, 3(2):127-163, 2000.
16

---

Published in Transactions on Machine Learning Research (12/2024)
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bron- stein. Geometric deep learning graphs and manifolds using mixture model CNNs. In CVPR, pp- 5115 5124, 2017.
Sebastien Motsch and Eitan Tadmor. Heterophilious dynamics enhances consensus. SIAM Review, 56(4): 577-621, 2014.
Galilco Namata, Ben London, Lise Getoor, Bert Huang, and U Edu. Query-driven active surveying for collective classification. In 10th International Workshop on Mining and Learning with Graphs, volume 8, pp. 1, 2012.
Hoang Nt and Takanori Maehara. Revisiting graph neural networks: All we have is low-pass filters. arXiv:1905.00 09550, 2019.
Maya Okawa and Tomoharu Iwata. Predicting opinion dynamics via sociologically-infrmeed neural networks. In ACM SIGKDD, pp. 1306-1316, 2022.
Kenta Oono and Taiji Suzuki. Graph neural networks exponentially lose expressive power for node classifi- cation. In ICLR, 2019.
Hongbin Pei, Bingzhe Wei, Kevin Chen Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-GCN: geometric graph convolutional networks. In ICLR, 2020.
Michael Poli, Stefano Massaroli, Junyoung Park, Atsushi Yamashita, Hajime Asama, and Jinkyoo Park. Graph neural ordinary differential equations. In AAAI Workshop on Deep Learning on Graphs: Method- ologies and Applications, 2020.
Anton Proskurnikov and Roberto Tempo. tutorial on modcling and analysis of dynamic social networks. part i. Annual Reviews in Control, 43:65 79, 2017.
Hegselmann Rainer and Ulrich Krause. Opinion dynamics and bounded confidence: models, analysis and simulation, 2002.
T Konstantin Rusch, Ben Chamberlain, James Rowbottom, Siddhartha Mishra, and Michael Bronstein. Graph-coupled oscillator networks. In ICML, pp. 18888-18909. PMLR, 2022.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE transactions neural networks, 20(1):61 80, 2008.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collec- tive classification in network data. Al Magazine, 29(3):93 93, 2008.
Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXim:1811.05865 2018.
Wen-Sheng Shu and Li-Nan Huang. Microbial diversity in extreme environments. Nature Reviews Microbi- ology, 20(4):219-233, 2022.
Francesco Tudisco, Austin Benson, and Konstantin Prokopchik. Nonlinear higher-order label spreading. Tn WWW, pp. 2402-22 2213,, 2021.
Petar Veličkovié, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.
Peihao Wang, Shenghao Yang, Yunyu Liu, Zhangyang Wang, and Pan Li. Equivariant hypergraph diffusson neural operators. Xxx::2207.06630, 2022.
Yuelin Wang, Kai Yi, Xinliang Liu, Yu Guang Wang, and Shi Jin. ACMP: Allen-cahn message passing with attractive and repulsive forces for graph neural networks. In ICLR, 2023.
17

---

Published in Transactions on Machine Learning Research (12/2024)
Wolfgang Weidlich. Sociodynamics: A systematic approach to mathematical modelling in the social sciences. Courier Corporation, 2006.
Louis-Pascal Xhonneux, Meng Qu, and Jian Tang. Continuous graph neural networks, In 1CML, pp. 10432- 10441, 2020.
Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. Hypergen: A new method for training graph convolutional networks on hypergraphs. NeurIPS, 32, 2019.
Menghin Ye. Opinion dynamics and the evolution of social power in social networks. Springer, 2019.
Olga Zaghen, Antonio Longa, Steve Azzolin, Lev Telyatnikov, Andrea Passerini, and Pietro Lio. Sheaf diffusion goes nonlinear: Enhancing gmns with adaptive sheaf laplacians. In 1CML 2024 Workshop on Geometry-grouunded Representation Learning and Generative Modeling, 2024.
Lingxiao Zhao and Leman Akoglu. PairNorm: Tackling oversmoothing in GNNs. In ICLR, 2020.
Bingxin Zhou, Yuanhong Jiang, Yuguang Wang, Jingwei Liang, Junbin Gao, Shirui Pan, and Xiaoqun Zhang. Robust graph representation learning for local corruption recovery. In Proceedings of the ACM Web Conference 2023, pp. 438 448, 2023.
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. In NewrIPS, volume 33, pp. 7793-7804, 2020.
18


Reviews:
[{"review_id": "M7qwogk2tZ", "reviewer": "Reviewer_h3KK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1. The application of opinion dynamics principles to neural message passing is creative and bridges distinct research fields, offering a fresh perspective on mitigating oversmoothing.\n2. The paper provides solid theoretical support for ODNet’s mechanisms, alongside extensive experiments that demonstrate its competitive performance across multiple datasets.\n\n**Weaknesses:**\n1. Potential Generalizability Limitations: ODNet’s performance is validated on benchmark datasets, but its generalizability to other domains (e.g., biological networks) remains speculative.\n2. While theoretically well-founded, the application of ODNet to very large graphs may raise scalability concerns that are not fully addressed in this paper. \n3. Some citations are missing.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces ODNet, a novel neural message-passing framework for graph and hypergraph data inspired by sociological opinion dynamics models. ODNet applies concepts from these models to mitigate oversmoothing in Graph Neural Networks (GNNs) by incorporating bounded confidence into message propagation. The authors test the proposed model in both homophilic and heterophilic graphs and hypergraphs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. How might ODNet’s mechanisms adapt to extremely large-scale graphs or real-time applications, and what optimizations could address potential scalability issues?\n2. Would it be feasible to conduct a preliminary exploration of ODNet’s applicability to biological networks or other non-social domains to assess its broader utility?\n\n3. Bellow I'm listing some missing citations:  \n\n*GNN*  \n[1] SCARSELLI, Franco, et al. The graph neural network model. IEEE transactions on neural networks, 2008, 20.1: 61-80.  \n[2] Kipf, Thomas N., and Max Welling. \"Semi-supervised classification with graph convolutional networks.\" arXiv preprint arXiv:1609.02907 (2016).  \n\n\n*Opinion dynamics GNN related*  \n[1] Caralt FH, Gil GB, Duta I, Liò P, Cot EA. Joint Diffusion Processes as an Inductive Bias in Sheaf Neural Networks. arXiv preprint arXiv:2407.20597. 2024 Jul 30.  \n[2] Duta I, Cassarà G, Silvestri F, Liò P. Sheaf hypergraph networks. Advances in Neural Information Processing Systems. 2024 Feb 13;36 \n \n[3] Zaghen O, Longa A, Azzolin S, Telyatnikov L, Passerini A, Lio P. Sheaf Diffusion Goes Nonlinear: Enhancing GNNs with Adaptive Sheaf Laplacians. InICML 2024 Workshop on Geometry-grounded Representation Learning and Generative Modeling.  \n[4] Hansen, J. and Ghrist, R. Opinion dynamics on discourse sheaves. SIAM Journal on Applied Mathematics, 81(5): 2033–2060, 2021.", "broader_impact_concerns": "No ethical issue.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "EZ7pwD0nsi", "reviewer": "Reviewer_k7Qs", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The paper is based on an interesting connection between neural message passing mechanisms in GNNs and opinion dynamics in social networks. To my knowledge, although it is a very natural connection, it has not been made explicit in this form before, and provides a number of new, interesting insights. In particular, this connection enables leveraging many insights developed in the line of work in opinion dynamics for a different problem, as the authors showcase extensively. I expect that more interesting results will follow from this connection in future work. The paper does a great job at providing sufficient background on key concepts in the literature on opinion dynamics, and how those translate in the context of message passing in GNNs, without assuming much familiarity with either area. They are also able to provide an intuitive and informative explanation regarding oversmoothing, a well-known problem in the literature. The main contribution of the paper is to introduce a new mechanism for message passing, which has a strong theoretical foundation as it is inspired by well-established work in opinion dynamics. The experimental results are overall very convincing and thorough, and demonstrate that the proposed mechanism outperforms prior approaches in the literature. The writing overall is of high quality, and it was a joy to read the paper. All prior work closely related to the paper appears to been discussed in detail. Moreover, all claims appear to be sound; I did not detect any notable issues. For those reasons, I believe merits acceptance. \n\nIn terms of weaknesses, the connection between message passing and opinion dynamics appears to have been studied before, albeit in a different form, as the authors discuss in the related work; in that sense, the novelty of the paper is somewhat limited. Further, from a technical standpoint, the results appear to be straightforward extensions of existing results that follow directly from the aforementioned connection. But I do not believe that the above caveats are a basis for rejection.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper explores connections between opinion dynamics in social networks and neural message passing in GNNs. By leveraging insights from social networks, the paper introduces a new mechanism for neural message passing that has a strong theoretical foundation. It effectively addresses the well-known problem of oversmoothing and achieves top performance in node prediction tasks across a variety of connectivity patterns.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would recommend to highlight more explicitly in the introduction the fact that connections between message passing in GNNs and opinion dynamics have been explored before, so that the contribution of the paper is more accurately placed into the existing literature.", "broader_impact_concerns": "No concerns on the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "qh9WySjWc7", "reviewer": "Reviewer_YSQv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The connection between opinion dynamics and message-passing is interesting.\n- Promising improvements.\n\nWeaknesses\n- I recommend adding an additional paragraph in the experiment section to introduce all the baselines. In the current version, it’s challenging to link the baselines to their references and to clearly understand the key differences between the proposed method and the baselines.\n- Most of the baselines are relatively outdated. It would be beneficial to consider more advanced methods, such as ED-HNN [1]. I recommend conducting a more comprehensive literature review and including newer approaches for comparison.\n\n[1] Equivariant Hypergraph Diffusion Neural Operators, ICLR 2023.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper adopts concepts from opinion dynamics models, such as the French-DeGroot Model and the Hegselmann-Krause Model, to propose a new message-passing method for graph neural networks (GNNs). The main objective is to address the oversmoothing issue in GNNs. Experimental results demonstrate that the proposed ODNet outperforms baseline methods on several datasets specific to node classification.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see above.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewer comments have been sufficiently addressed during revision period.
Decision: Accept (poster)