ACMP: A LLEN -C AHN M ESSAGE P ASSING WITH A TTRACTIVE AND R EPULSIVE F ORCES FOR G RAPH N EURAL N ETWORKS
A BSTRACT
Neural message passing is a basic feature extraction unit for graph-structured data considering neighboring node features in network propagation from one layer to the next.
We model such process by an interacting particle system with attractive and repulsive forces and the Allen-Cahn force arising in the modeling of phase transition.
The dynamics of the system is a reaction-diffusion process which can separate particles without blowing up.
This induces an Allen-Cahn message passing (ACMP) for graph neural networks where the numerical iteration for the particle system solution constitutes the message passing propagation.
ACMP which has a simple implementation with a neural ODE solver can propel the network depth up to one hundred of layers with theoretically proven strictly positive lower bound of the Dirichlet energy.
It thus provides a deep model of GNNs circumventing the common GNN problem of oversmoothing.
GNNs with ACMP achieve state of the art performance for real-world node classiﬁcation tasks on both homophilic and heterophilic datasets.
Codes are available at https://github.com/ykiiiiii/ACMP .
I NTRODUCTION
Graph neural networks (GNNs) have received a great attention in the past ﬁve years due to its powerful expressiveness for learning graph structured data, with broad applications from recommendation systems to drug and protein designs (Atz et al., 2021;
Baek et al., 2021; Bronstein et al., 2021; 2017;
Gainza et al., 2020; Wu et al., 2020).
Neural message passing (Gilmer et al., 2017)
serves as a fundamental feature extraction unit for graph-structured data that aggregates the features of neighbors in network propagation.
We develop a GNN message passing, called the Allen-Cahn message passing (ACMP) , using interacting particle dynamics, where nodes are particles and edges representing the interactions of particles.
The system is driven by both attractive and repulsive forces, plus the
Allen-Cahn double-well potential from phase transition modeling.
This model is motivated by the behavior of the particle system of collective behaviors common in nature and human society, for example, insects forming swarms to work; birds forming ﬂocks to immigrate; humans forming parties to express public opinions.
Various mathematical models have been proposed to model these behaviors (Albi et al., 2019; Motsch & Tadmor, 2011; Castellano et al., 2009; Proskurnikov & Tempo, 2017; Degond & Motsch, 2008).
There are two major components in this model.
First, while the attractive force forces all particles into one cluster, the repulsive forces allow particles to separate into two different clusters, which is essential to avoid oversmoothing.
However, repulsive forces could make the Dirichlet energy diverge.
We augment the model with the Allen-Cahn (Allen &
Cahn, 1979) term (or Rayleigh friction (Rayleigh, 1894)), which is crucial in preventing the Dirichlet energy in the evolution from becoming unbounded, allowing us to prove mathematically that the lower bound of the Dirichlet energy is strictly bigger than zero, hence avoiding oversmoothing.
Speciﬁcally, we will prove that under suitable conditions on the parameters, the dynamics of the
ACMP particle system will time-asymptotically form 2 d different clusters and the Dirichlet energy has a strictly positive lower bound.
The structure of ACMP can handle two problems in GNNs: oversmoothing and heterophily.
Oversmoothing (Nt & Maehara, 2019; Oono & Suzuki, 2019; Konstantin Rusch et al., 2022) means that all node features become undistinguishable, and equivalently, in the formulation of particle systems, features form only one consensus.
Heterophily problems means GNNs perform worse in heterophilic graphs (Lim et al., 2021; Yan et al., 2021).
It is due to the neighboring nodes of different classes are mistaken for the same class in GNNs like GCN and GAT.
However, the presence of repulsion in ACMP makes particles separate into two different clusters, hence provides a simple and neat solution for prediction tasks on both two problems.
Overall, the beneﬁt of the Allen-Cahn message passing with repulsion is manifold.
1) It circumvents oversmoothing issue, namely the Dirichlet energy is bounded from below.
The network is stable in the sense that features and Dirichlet energy are bounded from above.
Feature smoothness (energy decreasing) and the balance between nodes features and edge features can be adjusted by network parameters that control the attraction, repulsion and phase transition.
The model can then reach an acceptable trade-off on self-features and neighbor effect, as shown in Figure 1.
Our model can thus handle node classiﬁcation tasks for both homophilic and heterophilic datasets by using only one-hop neighbour information.
The proposed model can be implemented by neural ODE solvers for the system with attractive and repulsive forces.
In theory, we prove that Dirichlet energy of GNNs with ACMP has a lower bound above zero (limiting oversmoothing), as well as an upper bound (circumventing blow-up) under speciﬁc conditions.
This agrees with the experimental results (Section 6).
We also prove that ACMP is a process for the features to generate clusters thanks to the double-well potential, which provides an interpretable theory for node classiﬁcation.
B ACKGROUND
Message Passing in GNNs Graph neural networks are a kind of deep neural networks which take graph data as input.
Neural Message Passing (MP) (Gilmer et al., 2017; Battaglia et al., 2018) is a most widely used propagator for node feature update in GNNs, which takes the following form: for the undirected graph G =
( V , E ) is with sets of nodes V and edges E , with x ( k  1)
i 2 R d denoting features of node i in layer ( k  1) and a j,i 2 R D edge features from node j to node i,
[Equation]
where ⇤ denotes a differentiable, (node) permutation invariant function, e.g., sum, mean or max, and  and  denote differentiable functions such as MLPs (MultiLayer Perceptrons), and N i is the set of one-hop neighbors of node i.
The message passing updates the feature of each node by aggregating the self-feature with neighbors’ features.
Many GNN feature extraction modules such as GCN (Kipf & Welling, 2017), GAT (Veliˇckovi´c et al., 2018) and GIN (Xu et al., 2018) can be written as message passing.
For example, the MP of GCNs reads, with learnable parameter matrix ⇥ , x 0 i = ⇥ >
P j 2 N i [ { i } a j,i p ˆ d j ˆ d i x j , where ˆ d i = 1 + P j 2 N ( i ) a j,i and ˆ D = diag( ˆ d 1 , . .
, ˆ d N ) is the degree matrix for A
+ I .
Graph attention network (GAT) uses attention coefﬁcients ↵ i,j as similarity information between nodes in the MP update x 0 i
= ↵ i,i ⇥ x
i +
P j 2 N i ↵ i,j ⇥ x j , with ↵ i,j = exp  LeakyReLU [Equation]
P
[Equation]
The MP framework was also developed as PDE solvers in Brandstetter et al.
(2022b)
by embedding differential equations as a parameter into message passing like Brandstetter et al.
(2022a).
This paper regards particle system evolution (ODE) as message passing propagation, and the appropriate design of the particle system offers desired properties for the resulting GNN.
Graph neural diffusion Neural diffusion equations on graphs (GRAND) are proposed by Chamberlain et al.
(2021), which provides a uniﬁed mathematical framework for some message passings:
[Equation]
where G = diag( a ( x i ( t ) , x j ( t ) , t )) where a is a function reﬂecting similarity between nodes i and j, and x i is the scale-valued feature for node i, and x =  x i .
M OTIVATIONS 3. A TTRACTIVE AND REPULSIVE FORCES
The equation (2) itself can be interpreted in a formulation different from diffusion.
In this paper, we study the neural equations of interacting particle system , which has a similar structure to (2).
We rewrite (2) into a component-wise version and obtain a particle system
[Equation]
In the formulation of particle systems, one can easily discover the evolution trend of the features.
If a ( x i , x j ) > 0 , the direction of x i ’s velocity is towards x j , which means that x i is attracted by x j .
In the contrast, if a ( x i , x j ) < 0 , x i has a trend to move away from x j .
Hence, a ( x i , x j ) serves as the attractiveness or repulsiveness of the force between x i and x j .
In the diffusion model above, all a ( x i , x j ) ’s are positive, therefore all the node features in one connected component attract each other.
If the weight matrix ( a ( x i
However, the message propagation does not limit to consensus (corresponding to diffusion).
Information interaction can derive polarization of ﬁnal judgement when negative message matters in some problems rather than positive message.
For instance, in a node classiﬁcation task on a bipartite, the neighbour message is negative since connected nodes belong to different classes.
In the formulation of particle systems, the mechanism of positive and negative messages can be modelled by adding bias  i,j into (3)
[Equation]
The coefﬁcient term a ( x i , x j )   i,j corresponds to the interactive force.
By adjusting  i,j , both in the system attractive and repulsive forces co-exist.
If a ( x i , x j )   i,j > 0 , x i is attracted by x j .
While if a ( x i , x j )   i,j < 0 , x i is repelled by x j .
If the coefﬁcient equates zero, there is no interaction between x i and x j .
Then, the dynamics is enabled to adapt both positive and negative message passing.
In this way, the neural message passing can handle either homophilic or heterophilic datasets (see Section 6 for detailed discussion).
3. P SEUDO -G INZBURG -L ANDAU ENERGY
However, adding the repulsion term may cause the particles being separated away inﬁnitely, thus the Dirichlet energy becomes unbounded.
To avoid this problem, we add a damping term  x i (1  x 2 i ) , which we call an Allen-Cahn term .
The coefﬁcient ↵ > 0 is multiplied just for technical convenience.
[Equation]
Gradient Flow
The variational principle governing many PDE models states that the equilibrium state is actually the minimizer of one speciﬁc energy.
We ﬁrst introduce the Dirichlet energy and show that (3) can be characterized by looking into the corresponding Euler-Lagrange equation of the Dirichlet energy.
Let adjacent matrix
A represent the undirected connectivity between nodes x i and x j , with a i,j = 1 for ( i, j ) 2 E and a i,j = 0 for ( i, j ) 62 E .
The Dirichlet energy E in terms of G =
( V , E ) and node features x 2 R N ⇥ d takes the form
[Equation]
By calculus of variation, we can formulate the corresponding particle equation
[Equation]
On the RHS of (7), the summation takes over the one-hop neighbors N i of node i, which aggregates the impact from the neighboring nodes.
Equation (7) is (5) when one takes adjacent matrix A as the weight matrix ( a ( x i , x j ))
N ⇥ N .
Particle equation with the double-well potential To avoid blowing-up of the solution, one can design an external potential to control the solutions so they are bounded.
Here, we deﬁne the pseudoGinzburg-Landau energy on graph G denoted by  : L 2 ( V ) ! R , as a combination of the interacting energy and double-well potential W : R ! R + , with
[Equation]
where parameters ↵ ,  > 0 are used to balance the two types of energy.
From now on, we denote a ( x i , x j ) by a i,j for simplicity.
The pseudo-Ginzburg-Landau energy is not a true energy because the matrix ( a i,j   i,j ) N ⇥ N can be non-positive deﬁnite.
If  i,j ’s all equate zero, it then becomes the
Ginzburg-Landau energy deﬁned in Bertozzi & Flenner (2012);
Luo & Bertozzi (2017).
Using this combined energy, we can obtain the Allen-Cahn equation with repulsion on graph as @ x @ t = r x  , which is equivalent to (5).
A LLEN -C AHN M ESSAGE P ASSING
We propose the Allen-Cahn Message Passing (ACMP) neural network based on equation (5), where the message is updated by the evolution of the equation via a neural ODE solver.
To our best knowledge, this is the ﬁrst time to introduce a type of message passing to amplify the difference between connected nodes by repulsive force.
Network Architecture Suppose d -dimensional node-wise features represented by a matrix x in where row i represents feature of node i. Our scheme ﬁrst embeds the node feature x (0) =
MLP( x in ) by a simple multi-layer perceptron (MLP), which is treated as an input for ACMP propagation A : R d ! R d , by x (0) 7! x ( T ) , where x ( T ) =
x (0)+
R T 0 @ x ( t ) @
t dt, x (0) =
MLP  x in  , where @ X ( t ) @ t is estimated by ACMP deﬁned on G based on (5).
The node features x ( T ) at the ending time are fed into an MLP based classiﬁer.
Then, we deﬁne the Allen-Cahn message passing by @
[Equation]
i
[Equation] = [Equation] Here ↵ ,  2 R d are learnable vectors of the same length as the node feature x i .
While we can use a more general case when each edge ( i, j ) uses different trainable  i,j , we have simplied to single hyper-parameter  2 R + [ { 0 } , which makes the network and optimization easier.
The  in our model is a crucial parameter, which can be adjusted such that the attractive and repulsive forces both present to enrich the message passing effect.
If one chooses  = 0 ,  = 0 , our model is reduced to the graph neural diffusion network (GRAND) in Chamberlain et al.
In experiments, we would make signiﬁcant use of nontrivial  and  .
The operations of all terms are channel-wise, involving d channels, except a ( x i ( t ) , x j ( t )) , and  represents channel-wise multiplication for d feature channels.
Figure 1 illustrates the one-step
ACMP mechanism (9): Nodes with close colors attracts each other otherwise repel.
Nodes in the same block tend to attract each other and both colors are strengthened during message passing propagation.
The double-well potential prevents the features and Dirichlet energy from blowup.
In this process, node feature x ( t ) is updated to x ( t +  t ) for a time increment  t .
Ultimately, a bi-cluster ﬂock is formed for node classiﬁcation.
In the propagation of ACMP in (9), we need to specify how the neighbors are interacted, that is how the a ( x i ( t ) , x j ( t )) is evolved with time.
There are many kinds of methods to update the edge weights.
The two typical types are GCN (Kipf & Welling, 2017) and GAT (Veliˇckovi´c et al., 2018).
ACMP-GCN : this model uses deterministic a ( x i ( t ) ,
x j ( t )) , which is given by the adacency matrix
A = ( a i,j ) of the original input graph G and does not change with time.
That is, the coefﬁcients in
GCNs a GCN i,j := a i,j /
[Equation]
ˆ d i ˆ d j .
The message passing of (9) is reduced to
[Equation]
ACMP-GAT : we can replace a GCN i,j in (10) by the attention coefﬁcients (1) of GAT, which with extra trainable parameters measures the similarity between two nodes by taking account of both node and structure features.
The system then drives edges to update in each iteration of message passing.
Neural ODE Solver Our method uses an ODE solver to numerically solving the equation ((9) and (10)).
To obtain the node features x ( T ) , we need a stable numerical integrator for solving the ODE efﬁciently and backpropagation of gradients.
Since our model is stable in terms of evolution time, most explicit and implicit numerical methods such as explicit Euler, Runge-Kutta 4th-order, midpoint,
Dormand-Prince
(Chen et al., 2018;
Lu et al., 2018; Norcliffe et al., 2020;
Chamberlain et al., 2021) work well as long as the step size ⌧ is small enough.
In experiments, we implement ACMP using Dormand-Prince5 method which provides a fast and stable numerical solver.
The network depth of ACMP-GNN is equal to the numerical iteration number n t set in the solver.
Computational Complexity
The computational complexity of the ACMP is O ( NEdn t ) , where n t , N, E and d are number of time steps in time interval [0 , T ] , number of nodes, number of edges and number of feature dimension, respectively.
Since our model only considers nearest (one-hop) neighbors, E is signiﬁcantly smaller than that of graph rewiring (Gasteiger et al., 2019;
Alon &
Yahav, 2021) and multi-hop (Zhu et al., 2020) methods.
Channel Mixer Channel mixing can be spontaneously introduced from the perspective of diffusion coefﬁcients though our model is previously written in the channel-wise form.
Whether channel mixing happens depends on the speciﬁc GNN driver we choose for ACMP.
When the coefﬁcients a ( x i ( t ) , x j ( t )) in (9) that do not update with time are a scalar or vector, like in ACMP-GCN, the operations of the message passing propagator are channel-wise and channel mixing is not incorporated.
On the other hand, the ACMP-GAT with graph attention driver incorporates a learnable channel mixing when the coefﬁcients are tensors.
The channel mixer can be introduced by generalizing the Dirichlet energy to high dimension, for example, E ( x ) :
= 1 N [Equation] when a i,j 2 R d ⇥ d are connectivity tensors.
D IRICHLET E NERGY
The dynamics (5) can circumvent the oversmoothing issue of GNNs (Nt & Maehara, 2019; Oono & Suzuki, 2019; Konstantin Rusch et al., 2022).
Oversmoothing phenomenon means that all node features converge to the same constant – consensus forms – as the network deepens, and equivalently, the Dirichlet energy will decay to zero exponentially.
This idea was ﬁrst introduced in Cai & Wang (2020).
Konstantin Rusch et al.
(2022) gives an explicit form for oversmoothing.
In our model, as we will show below, the node features in each channel tend to evolve into two clusters departing from each other under certain conditions.
This implies a strictly positive lower bound of the Dirichlet energy.
In addition, the system will not blow up thanks to the Allen-Cahn term.
We put all the proofs and some related supplementary results in the appendix.
Proposition 1.
If  > 0 , the node features x i in (5) is bounded in terms of k · k and energy for all t > 0 , i.e., E ( x ( t ))  C, and k x k  C, where the constant C only depends on N and  max .
In the following propositions, we imitate the emergent behavior analysis in Fang et al.
(2019) (see
Appendix for details).
For a graph G with N nodes, its vertices are said to form bi-cluster ﬂocking if there exist two disjoint sets of vertex subsets { x (1) i } N 1 i =1 and { x (2) i } N 2 j =1 satisfying ( i ) sup [Equation]
t
[Equation]
( ii ) 9 C 0 , T ⇤⇤ > {+
+} such that
[Equation]
where x (1) i , x (2) i denote any component of x (1) i , x (2) i .
We now show the long-time behaviour of model (5) following the analysis of Fang et al.
for strength coupling ( ↵ ,  ) that satisﬁes the following condition: there exists {  i,j } such that I :=
, N } can be divided into two disjoint groups I 1 , I 2 with N 1 and N 2 particles respectively:
[Equation]
where S, D are independent of time t.
The S and D in (12) are the repulsive and attractive forces.
We prove that if the repulsive force between the particles is stronger than the attractive force, that is, S > D, the system is guaranteed to have bi-cluster ﬂocking, as shown in Proposition 2 below.
For time t  0 , suppose x (1) c ( t ) and x (2)
c
( t ) are the feature centers of the two groups of the particles { x (1) i ( t ) }
N 1 i =1 and { x (2) j ( t ) } N 2 j =1 which are partitioned as above from the whole vertex set V , given by
[Equation]
Suppose x ( s ) c ( t ) has the d -dimensional feature, and let x ( s ) c,k ( t ) , k = 1 , . . .
, d, be the k th (dimension) component of the feature x
( s ) c ( t ) , s = 1 , 2 .
Proposition 2.
The system (5) has a bi-cluster ﬂocking if for each k = 1 , . . .
, d, the initial | x (1) c,k (0)  x (2) c,k (0) |  1 , and if there exists a positive constant ⌘ such that
[Equation]
where the  is the weight factor for the double-well potential in the equation (5).
Proposition 3.
For system (5) with bi-cluster ﬂocking, there exists a constant C >
and some time
T ⇤ such that 8 t  T ⇤ ,
[Equation]
Thus, if the non-zero a i,j are all positive, the Dirichlet energy for ACMP is lower bounded by a positive constant.
E XPERIMENTS
Dirichlet Energy We ﬁrst illustrate the evolution of the Dirichlet energy of ACMP by an undirected synthetic random graph.
The synthetic graph has 100 nodes with two classes and 2D feature which is sampled from the normal distribution with the same standard deviation  = 2 and two means µ 1 =  0 .
5 , µ 2 = 0 .
The nodes are connected randomly with probability p = 0 .
9 if they are in the same class, otherwise nodes in different classes are connected with probability p = 0 .
We compare the performance of GNN models with four message passing propagators: GCNs (Kipf & Welling, 2017), GAT (Veliˇckovi´c et al., 2018), GRAND (Chamberlain et al., 2021) and ACMPGCN.
In Figure 2, we visualize how the node features evolve from their initial state to their ﬁnal steady state when 50 layers of GNN are applied.
Additionally, in Figure 3, we show the Dirichlet energy of each layer’s output in logarithm scales.
Traditional GNNs such as GCNs and GAT suffer oversmoothing as the Dirichlet energy exponentially decays to zero in the ﬁrst ten layers.
GRAND relieves this problem by multiplying a small constant which can delay all nodes’ features to collapse to the same value.
For ACMP, the energy stabilizes at the level that relies upon the roots of the double-well potential in (9) after slightly decaying in the ﬁrst two layers.
Node Classiﬁcation We compare the performance of ACMP with several popular GNN model architectures on various node classiﬁcation benchmarks, containing both homophilic and heterophilic datasets.
Graph data is considered as homophilic (Pei et al., 2020) if similar nodes in the graph tend to connect together.
Conversely, the graph data is said heterophilic if it has a small homophily level, when most neighbors do not have the same label with source nodes.
We aim to demonstrate that ACMP is a ﬂexible GNN model which can learn well both kinds of datasets by balancing the attractive and repulsive forces.
The GCN for examples cannot perform well for heterophilic dataset as its message passing aggregates only the neighbor (1-hop) nodes.
The neural ODE is solved by
Torchdiffeq package with Dormand–Prince adaptive step size scheme.
Only few hyperparameters are needed to be tuned in our model.
For all the experiments, we ﬁne tune the learning rate, weight decay, dropout, hidden dimensional of the  which controls the repulsive force between nodes.
We outline the details of hyperparameter search space in Appendix D. of layer-wise node propagated
Homophilic datasets Our results are presented for the most widely used citation networks: Cora (McCallum et al., 2000), Citeseer (Sen et al., 2008) and Pubmed (Namata et al., 2012).
Moreover, we evaluate our model on the Amazon co-purchasing graphs Computer and Photo (Namata et al., 2012), and CoauthorCS (Shchur et al., 2018).
We compare our model with traditional GNN models:
Graph Convolutional Network (GCN) (Kipf & Welling, 2017), Graph Attention Network (GAT) (Veliˇckovi´c et al., 2018), Mixture Model Networks (Monti et al., 2017) and GraphSage (Hamilton et al., 2017).
We also compare our results with recent ODE-based GNNs, Continuous Graph Neural Networks (CGNN) (Xhonneux et al., 2020), Graph Neural Ordinary Differential Equations (GDE) (Poli et al., 2020) and Graph Neural Diffusion (GRAND) (Chamberlain et al., 2021).
To address the limitations of this evaluation methodology proposed by Shchur et al.
(2018), we report results for all datasets using 100 random splits with 10 random initialization’s, and show the results in Table 1.
[Table]
Heterophilic datasets We evaluate ACMP-GCN on the heterophilic graphs; Cornell, Texas and
Wisconsin from the WebKB dataset 1 .
In this case, the assumption of common neighbors does not hold.
The poor performance of GCN and GAT models shown in Table 2 indicates that many GNN models struggle in this setting.
Introducing repulsion can improve the performance of GNNs on heteroplilic datasets signiﬁcantly.
ACMP-GCN scores 30% higher than the original GCN for the
Texas dataset which has the smallest homophily level among the datasets in the table.
Attractive and Repulsive interpretation As shown in Table 2 and Table 1, ACMP-GCN and
ACMP-GAT achieve better performance than GCN and GAT on both homophilic and heterophilic datasets.
The majority of a i,j   in the homophilic are positive, which means most nodes are attracted to each other.
Conversely, most a i,j   for the heterophilic are negative, which means that most nodes are repelled by their neighbors.
Several GNNs exploiting multi-hop information can [Table] achieve high performance in node classiﬁcation (Zhu et al., 2020; Luan et al., 2021).
However, highorder neighbor information will make the adjacency matrix dense and therefore can not be extended to large graphs, due to heavier computational cost.
In our model, we take only one-hop information into account and add repulsive force (   0 ) to message passing, which has achieved the same or higher level of accuracy as multi-hop models in heterophilic datasets.
Performance of ACMP to
Hyperparameter  is a signal of the repulsive force, meaning that when a ij   is negative, the two nodes repel one another.
To illustrate  ’s impact, we use GCN as a diffusion term as a ij do not change during the ODE process and all the changes are related to  .
As shown by Figure 4, ACMP performs best in Cora (orange curve) when all nodes are attracted to one another i.e., all a ij   is positive.
As the beta increases, the performance of the model degrades.
In contrast, for the Texas dataset, when all force is attractive, ACMP achieves only 70% accuracy (blue curve).
As  increases, most a ij   is negative, and the model’s performance gets better.
When all the force is repulsive, ACMP achieves highest accuracy on Texas datasets, which is in accordance with our claim that repulsive force is important for heterophilic datasets.
R ELATED WORK
Neural differential equations The topic of neural ODEs becomes an emerging ﬁeld since E (2017) and Chen et al.
(2018), with many follow-up works in the GNN ﬁeld (Avelar et al., 2019;
Poli et al., 2020; Sanchez-Gonzalez et al., 2019).
GRAND (Chamberlain et al., 2021) propagated GNNs by the graph diffusion equation and Wu et al.
(2023) developed an energy-constrained diffusion transformer. GraphCON (Konstantin Rusch et al., 2022) employed a second-order system to conquer oversmoothing of deep graph neural networks.
By exploiting the ﬁxed point of the dynamical system, Gallicchio & Micheli (2020) proposed FDGNN as an approach to graph classiﬁcation.
Allen-Cahn based variational graph models In Bertozzi & Flenner (2012);
Luo & Bertozzi (2017); Merkurjev et al.
and references therein, authors extended Allen-Cahn related potential to graphical framework and developed a class of variational algorithms to solve the clustering, semisupervised learning and graph cutting problems.
The new ingredient of graph neural network which enables us to combine learnable attraction and repulsion separates our method from the classical variational graph models.
C ONCLUSION
We develop a new message passing method with simple implementation.
The method is based on the Allen-Cahn particle system with repulsive force.
The proposed ACMP inherits the characteristic dynamics of the particle system and thus shows adaption for node classiﬁcation tasks with high homophily difﬁculty.
Also, it propels networks to dozens of layers without getting oversmoothing.
A strictly positive lower bound of the Dirichlet energy is shown by theoretical and experimental oversmoothing model for various real datasets