Genetic-Evolutionary Graph Neural Networks: A Paradigm for Improved Graph Representation Learning
Authors: Haimin ZHANG
Min Xu
Venue: Accepted by TMLR ()

Abstract: Message-passing graph neural networks have become the dominant framework for learning over graphs. However, empirical studies continually show that message-passing graph neural networks tend to generate over-smoothed representations for nodes after iteratively applying message passing. This over-smoothing problem is a core issue that limits the representational capacity of message-passing graph neural networks. We argue that the fundamental problem with over-smoothing is a lack of diversity in the generated embeddings, and the problem could be reduced by enhancing the embedding diversity in the embedding generation process. To this end, we propose genetic-evolutionary graph neural networks, a new paradigm for graph representation learning inspired by genetic algorithms. We view each layer of a graph neural network as an evolutionary process and develop operations based on crossover and mutation to prevent embeddings from becoming similar to one another, thus enabling the model to generate improved graph representations. The proposed framework has good interpretablility, as it directly draws inspiration from genetic algorithms for preserving population diversity. We experimentally validate the proposed framework on six benchmark datasets on different tasks. The results show that our method significant advances the performance of current graph neural networks, resulting in new state-of-the-art results for graph representation learning on these datasets.

Published in Transactions on Machine Learning Research (08/2025)
Genetic-Evolutionary Graph Neural Networks: A Paradigm for Improved Graph Representation Learning
Haimin Zhang
haimin.zhong@@uts. edu..u
School of Electrical and Data Engineering University of Technology Sydney
Min Xu
min. Cutts edu. on
School of Electrical and Data Engineering University of Technology Sydney
Reviewed on OpenReview: https: optpreviee.r...eee.em net/ porm/f id= qEYTkLIVAB
Abstract
Messagc-passing graph noural networks have become the dominant framework for learning over graphs. However, empirical studies contimally show that message- paasagg-passing graph neural networks tend to generate over-smoothed representations for nodes after iteratively applying message passing. This over-smoothing problem core issue that limits the representa- tional capacity of message-passinng graph neural networks. We argue that the fundamental problem with over-smoothing is a lack of diversity in the generated embeddings, and the problem could be reduced by enhancing the embedding diversity in the embedding gener- ation process. To this end, we propose sevouliayvoy graph neural networks, a new paradigm for graph representation learning inspired by genetic algorithms. We view each layer graph neural network as an evolutionary process and develop operations basod on crossover and mutation to prevent embeddings from becoming similar to one another, thus enabling the model to generate improved graph representations. The proposed framework is well-motivated, it directly draws inspiration from genetic algorithms for preserving population diversity. We experimentally validate the proposed framework on six benchmark datasets on different lasks. The results show that our method significantly advances the performance of current graph neural networks, resulting new state-of-the- the-art results for graph representation learning on these datasets.
1 Introduction
Graphs general data structure for representing and analyzing complex relationships among entities. Many real-world systems, such social networks, molecular structures, communication networks, can be modeled using graphs. It is essential to develop intelligent models for uncovering the underlying patterns and interactions within these graph-structured systems. Recent years have seen an enormous body of studies on learning over graphs. The studies include graph foundation models, geometry processing and deep graph embedding. These advances have produced new state-of-tho-art or human-level results in various domains, including recommender systems Zhang al. 2024), chemical synthesis (Xie al.) 2024), and 2D and 3D vision tasks (Chen et al.) 2024 Kim et al. 2023
Graph neural networks have emerged as dominant framework for lcarning from graph-structured data. The development of graph neural network models can be motivated from different. approaches. The fundamental graph neural networks have been derived as a generalization of convolutions to non- Euclidean data (Bruna ct al. 2014), as well as by analogy to classic graph isomorphism tests (Hamilton al.) (017). Regardless of the motivations, the defining feature of the graph neural network framework is that it utilizes form of message passing wherein messages exchanged between nodes and updated using neural networks

---

Published in Transactions on Machine Learning Research (08/2025)
[Hamilton| 2020). During each graph neural network layer, the model aggregates features from a node's local neighbourhood and then updates the node's representation according to the aggregated information.
Message passing is at the heart of current graph neural networks. However, this paradigm of message passing also has major limitations. Theoretically, it is connected to the Weisfeiler-Lehman (WL) isomorphism test as well as to simple graph convolutions. The representational capacity of messagg--pssing graph neural networks is inherently bounded by the WL isomorphism test (Xu et al., 2019). Empirical studies continually find that gasssige-passing graph neural networks suffer from the problem of over-smoothing (Wu et al. 2023; Hamilton 2020 That is, the representations for all nodes can become very similar to one another after too many message passing iterations. These core limitations prevent graph neural networks from learning more meaningful representations from graphs. In recent years, an increasing number of studies have been devoted to addressing the bottlenecks, such as normalization and regularization techniques (Zhao & Akoolu 2020b), and combining the global self-attention mechanism (Rampasek et al. 2022). exploring generalized message passing (Barceló et al.] (020). Despite these advances, improving the capability of graph neural network models still remains a fundamental challenge in learning from graph--ttruluured data.
To learn meaningful graph representations, it is crucial to generate embeddings for all nodes that depend on both the graph structure and node attributes. However, when the over-smoothing phenomenon occurs, the representations for all nodes begin to look icentical to each other. The consequence is that node-specific feature information becomes lost. To prevent this issue, it is important to preserve the diversity of generated embeddings throughout their layer-wise generation process. In this paper, we propose genetic-evolutionrry graph newral networks, a new paradigm for graph representation learning that integrates the idea from genetic algorithms for maintaining population diversity into the message-passiigg graph neural network framework.
Genetic algorithms, inspired by Charles Darwin's 's theory of natural evolution, emulate the process of natural selection, wherein the fittest individuals are selected to reproduce and generate the next generation of offspring. Over successive generations, biological organisms evolve using biologically inspired operators, such as selection, crossover and mutation, towards "survival of the fittest". This approach has become a powerful tool for generating high-quality solutions for complex optimization and search problems.
In genetic algorithms, crossover and mutation play a key role in generating diverse individuals for selection, preventing the algorithms from premature convergence (Gupta & Ghafir 2012). Crossover introduces variety by combining genetic information from different parents, and mutation introduces small random changes in genetic information. In this work, we view the iterative node embedding process as an evolutionary process, in which each layer of message passing produces a new generation of embeddings. We introduce two crossover operations, iee, cross-generatie
crossover and sibling crossover, and a mutation operation, and we develop two graph neural network building blocks based on the operations. At each layer of graph neural network, we first use message passing to update node representations and then apply crossover and mutation to prevent embeddings from becoming similar to one another, thus enabling the model to learn improved graph representations.
Unlike previous methods, such as residual connections (He et al. 2016), SSFG (Zhang et al. 2022) and PairNorm (Zhao & Akoglu 2020a), this work proposes operations by drawing inspiration from genetic algorithms for addressing the over-smoothing problem in graph neural networks. Our framework is well- motivated as it views the layer-wise node embodding process as analogous to the genetic evolutionary process. It is a general paradigm that can be integrated into different graph neural network models. We conduct experiments on six benchmark datasets on different graph tasks. We show that the use of our framework significantly improves the performance of the bascline graph neural networks, advancing the statc-of-the-an results for graph representation learning on the datasets.
The main contributions of this paper can be summarized as follows. (1) This paper proposes a new framework named genetic-evolutionary graph neural norks for learning from graph-structured data. The core idea behind the proposed framework is to model each layer of a graph neural network as an evolutionary process. We develop three key operations inspired by crossover and mutation from genetic algorithms to enhance the diversity of generated embeddings at each layer. (2) The proposed framework is well-motivated, as it is directly inspired by genetics. It is a general paradigm which can be integrated into current message-passing graph neural networks. Empirical evaluations are conducted on six popular datasets on different graph
2

---

Published in Transactions on Machine Learning Research (08/2025)
tasks, and the results demonstrate that the proposed framework significantly improves the performance of the baseline graph neural networks.
2 Related Work
2.1 Graph Neural Networks
Most current graph neural networks can be categorized into spectral approaches and spatial approaches (Veličković et al.] 2018). The spectral approaches are developed based on spectral graph theory. The key idea of spectral graph neural networks is that convolutions are defined in the spectral domain through an extension of the Fourier transform to graphs. In contrast, spatial graph neural networks define convolutions in spatially localized neighbourhoods. The behaviour of the convolutions analogous to that of kernels convolutional neural networks which aggregate features from spatially-defined patches in an image.
Gilmer et al. [2017] reformulated common models a peessage-passingg neural network framework, wherein node features are exchanged between nodes and updated using neural networks. A common issue with messago-passing graph neural networks is known as the over-smoothing problem. This issue of over- smoothing was first identified by Li et al. (2018). It can also be viewed as a consequence of the neigh- bourhood aggregation operation in the message messagg-passing update (Hamilton) 2020). The follow-up studies for limiting over-smoothing include graph normalization and regularization techniques (Zhao & Akoglu 2020aa] Chen al. 2022), combing the global self-attention with local message passing (Rampasek et al.) 2022). and improved graph attention approaches (Wu et al., 2024). Additionally, there have been studies on uncovering over-smoothing in basic graph neural network models from theoretical analysis (Oono & Suzuki 2020 Luan
ct al. 2024) investigated the impact of the homophily principle, i.e., nodes with the same labels are more likely to be connected, on node distinguishability and and showed that graph neural network is capable of generating meaningful representations regardless of homophily levels. Kelesis et al. (2023) introduced a sim- ple modification to the slope of the ReLU activation function to mitigate the issue of oversmoothing. [Rog et al. (2020) introduces a method for reducing oversmoothing by randomly removing edges during training for node classification tasks. [elesis et al. (2025) proposed weight initialization method that takes into account the graph topology for reducing oversmoothing in graph neural networks.
2.2 Genetic Algorithms
Genetic algorithms, first introduced by John Holland in the 1960s (Holland 1992 are a hcuristic optimiza- tion method inspired by the principles of nature selection and evolutionary biology. Genetic algorithms are powerful method for solving complex constrained and unconstrained optimization problems. Tn the past decades, genetic algorithms have emerged as a widely adopted tool for addressing optimization and search problems across varions fields including scheduling, mathematics, robotics and bioinformatics (Alhijawi &
Awajan 2023 Davidor 1991
McCall 2005).
In machine learning, genetic algorithms have been successfully applied for feature selection (Babatunde al. 2014), neural network optimization (Miller et al. 1989) and hyperparameter tuning for models like neural networks and support vector machines (Alibrahim & Ludwig 2021 In object detection, hyperparameter cvolution which uses a genetic algorithm was applied for optimizing hyperparamcters in YOLO models [eemmon] 2016). Sehgal el. al.] (2019) showed that evolving the weights of deep neural network using a genetic algorithm was competitive approach for training reinforcement learning models. [Shi et al. (2022) used the genetic approach for graph neural network architccture search. Unlike previous work, to the best of our knowledge, this is the first work that introduces operations inspired by genetic algorithms for regularizing graph neural networks through enhancing embedding diversity.
3

---

Published in Transactions on Machine Learning Research (08/2025)
1
1
1
0
0
0
0
1
1
1
|
0
O
1
0
Crossover
Mutation
Figure Crossover recombines of the genetic information of parents produce an offspring. introduces small random changes in genetic information.
Mutation
3 Methodology
3.1 Graph Neural Networks
A graph G (V,E) can be defined through of nodes V and a set of edges & between pairs of these nodes. Each node E V is associated with a node-level feature X Graph neural networks are a general framework for representation learning over the graph and {x,, Vu E V}. At its core, the graph neural network framework ileralively updates the represenlation for every node using form of message passing, During each message message-passing iteration, each node E V aggregates the representations of the nodes in its neighborhood, and the representation for node u is then updated according to the aggregated representation. Following Hamilton 2020).. this message-passing framework can be expressed as follows:
h (K)
(pdate (R) (h(t-1), ((h (oW)))))))
where Update and Aggregate are neural networks, and (u)) is the set nodes u's neighbourhood. The superscripls used for distinguishing the embeddings and functions al different ilerations. At each ileration k, the Aggregate function takes the set of embeddings of nodes in (u)) as input and generates aggregated
message m ((u)' The Udatt function then generates the updated embedding for node u based on the message
(k)
(Mu)) and u's previous embedding h (k-1) The embeddings at k 0 are init ialized to the node-level features. m (k)
i... h Xu, Vu After K iterations of message passing, every node embedding contains information from its K-hop neighborhood.
This message passing formalism is currently the dominant framework for learning over graphs. However, common issue with message-passing graph neural networks is over-smoothing. The idea of over-smoothing is that the embeddings for all nodes begin to become similar and are relatively uninformative after too many rounds message passing. This issue of over-smoothing can be viewed as a consequence the neighborhood aggregation operation. [i al. (2018) showed that the graph convolution of the basic graph convolutional network model (Kipf & Welling, 2016) can be seen a special form of Laplacian smoothing that generates the representation for every node using the weighted average of a node's itself and its neighbours' embeddings. But after applying too many rounds of Laplacian smoothing, the representations for all nodes will become indistinguishable from each other. From the graph signal processing perspective, multiplying a signal by high powers of the symmetric normalized adjacency matrix Agym = D AD?, which corresponds to a convolutional filter the lowest eigenvalues, or frequencies, of the symmetric normalized Laplacian Laym = I- A-Asym- Thus, the simple graph neural network that stacks multiple rounds of graph convolution converges all the node representations to constant values within connected components on the graph, i.c., the "zero- frequency" of the Laplacian (Hamilton 2020
4

---

Published in Transactions on Machine Learning Research (08/2025)
3.2
Genetic- RevoutiiRe Graph Neural Networks
3.2.1 Motivation
In the above, we discussed the over-smoothing problem message- passing graph neural network. We see that the fundamental issue is the loss of embedding diversity at. each layer throughout the generation process. Thus, we can view the trade-off between model performance and depth of popular graph neural network models from this perspective. Graph neural networks need to model complex relationships and long-term dependencies using more layers to improve the performance. However, using too many layers will eliminate node-specific features, which leads to significantly reduced model performance.
Graph neural networks generate embeddings for nodes through an iterative cresagg-passinng process. At each message-passinng iteration, the representation for every node is updated according to the information aggregated from the node's graph neighbourhood. We can view this iterative process as an genetic evolution- ary process, wherein graph nodes individuals of population, and the model is to evolve population of nodes over multiple generations to obtain their expressive representations for graph tasks.
In genetic algorithms, a very homogencous population, i.c., little population diversity, is considered as the major reason for premature convergence to suboptimal solutions (Whitley) 2001]. Therefore, it is crucial to preserve the diversity of population during the evolutionary process. Similarly, we need to maintain the diversity of generated embedding in their generation to prevent the model from converging to a local optimum in optimization.
To preserve the population diversity, genetic algorithms the operators of crossover and mutation to generate diverse individuals and select those best fit the environment to evolve over successive generations. The crossover operation recombines of the characteristics of each ancestor of an offspring, and the mutation operation randomly changes the genetic information to increase the variability (see Figure D In similar manner, we can generalize the mechanisms to the embedding generation process. By integrating crossover and mutation methods within the message-passing framework, we can prevent generated embeddings from becoming too similar to each other. This ultimately would enhance the model representational capacity.
3.2.2 Improving Graph Neural Networks with Genetic Operations
We view each layer of a graph neural network as a genetic evolution process, in which the nodes represent individuals of population and their embeddings represent chromosomes that store genetic information. During each graph neural network layer, we first use message passing to update the embeddings for all nodes and then use genetic operations to increase the diversity of generated embeckingg.. We propose three operations inspired by genetic algorithms: (1) cross-generation crossover, (2) sibling crossover, and (3) mutation.
Genetically, crossover is a process in which the genelic information of two parents is recombined to produce new offspring, resulting in the exchange of genetic material between parental chromosomes. This mechanism forms the basis for driving biological variation, shaping differences in traits within species and introducing novel traits previously unseen in a population. enabling novel gene combinations to emerge and spread across generations. Fundamentally, this process creates diversity at the level of genes that reflects difference in chromosomes of different individuals.
basically helps promote the evolutionary process by
Cross-generation crossover. Similar to crossover in genetics, the cross- gosseeeneration operation in our frame- work recombines the embedding for node generated by message-pan saaogggg-assnngggg and the node's previous layer (k), I ha,d) and h (k-1) = (h (k-1) "...,,. (((--)) which represent the embedding for node u generated by message passing and u's previous layer embedding, cross- gessraeenertionn crossover can expressed as follows:
embedding. For [A = (bu,)- ...,
(k) 6
-Crossover (K) 1h (k-1))
where h (k)
(k-1)
if <p 1 = 1,..,, d,
(2)
u,s
h
else
5

---

Published in Transactions on Machine Learning Research (08/2025)
Algorithm Pseudocode for gross-generation crossover in a Py PyTorcchllik style.
h, h,in: representaton generated by nessage and the previous layer enbedding f_prob: probabilty of reconbining information from self.diats Bernoulli distribation defined by torch, distributions Bernoulli (wwacchteeeoorreaete (f...ooroo)
def forsard(self, h, b_in):
if self. training == True: _aaovver_aakk self dist.sanplee... www.saaaapppphaaeeeeeeeeeeeeeeee (h. shape) generate
mask
crossover from and h_in b_in crossover mask h (1 crossover_naa _mask)
else:
b_in self.f_pree f_probb h self.f self.f.prob)
Algorithm Pscudocode for mutation in PyTorch-like style.
self- running mean: nean of h over the training set self wruniingggeeee var: the variance of h over the training set self welf.nutation. www.atiin...rbb......... probility nutation
def forvard(self, h):
self. training True:
mean h.mean(((o)
var [ar
b.numel() () h.size(1)
with torch torch.no_grad():
momentun update of running nean and running. var nonentun
rumning_eeg self self wwwningnnvnsrs....nn var self monentun var / 1) + (1 self momentuu) self, running.vaa var
nonentun) self, self, wwwningnnneeenee nean
generate
gaussien_noise torch. randu shape)
training catation_nasi Bernoulli. sanple shape) # generate nutation mask h (gaussian_noiie noise self. self. running. var self runninggnnenn nean) mutation, nask h (1 mutation_nask nacki) else:
self. runninggnenn nean mutamtmtnnntnto
return
and A (0, 1) and is probability indicating information from the previous layer embodding. At cach dimension, the feature is randomly selected from the embedding generated using message passing or from the embeding inputted to this layer. Because each round of message passing generates smoothed version of the imput, recombining information from a node's previous layer embedding reduces the smoothness of the generated embeddings. This operation is a parameter- free method and can integrated into current graph nerual networks.
Sibling crossover is an operation that randomly selects information from siblings. In our impelmentation, generate multi-head outputs using message passing as siblings and update the embedding for node by randomly selecting information from the multi-head outputs.
(k) Crossover [ (k.he)d))
(k.h.,) h (khhedss)
where h bu = hug
hi ~ Categoricall) }), and is the number of heads. Each h (k.bead,) in the multi-head outputs represents a sibling generated using the same input. This operation also increases individual diversity by randomly combining information from different siblings.
Mutation the process which some genes individuals are randomly changed. In our framework, the feature each dimension randomly replaced by value sampled from Gaussian distribution, wherein the statistics calculated using batches. For batch of vectors (hl, h.,,.. hma, calculate the

---

Published in Transactions on Machine Learning Research (08/2025)
Chu 1)
{h 1)
Multi-head Message-passiing
Message-passing Layer
MP head,
MP head
MP heady
Cross-gnnrraton Crossover
Sibling Crossover
Mutation
Mutation
(a)
(b)
Figure 2: Building block architectures: Block (a) applics cross-gencration to a node's embedding generated using message passing and the node's previous layer embedding, and Block (b) applies sibling crossover to a set of outputs generated using multi-head message passing,
4 and variance of the feature the training set follows.
# - (HB) - m
(4)
m (g(
where us and og are the mean and variance of the batch B. Here we use the unbiased variance estimate. Then we randomly sample a vector from a multivariate Gaussian distribution NO,,1) and update the feature as follows:
h = (78 + 4)mask + h2 (1 mask)
(5)
where the mask ~ Bernoolli(mutttiinn_ _attiatttiiorrt...aaeaaaaa The mutation operation is also a parameter-free method. basically introduces randomness features as a regularization method, enabling the model explore new space for optimization.
3.3 Model Architecture
Algorithm and Algorithmm2e show our Pytorch-style pseudo-code for the cross-generation crossover operation and mutation operation respectively. The code for sibling crossover can be easily adapted from Algorithm We design two building blocks based on the cross-generation crossover operaton and sibling crossover operation (sce Figure The first building block applics the cross-gencration crossover after message passing, followed by the mutation operation. Note that this building block compatible with different graph neural network models and it does not introduce additional trainable parameters. The other building block applies sibling crossover to set of multi-hoad outputs, followed by the mutation operation. This method requires the model to generate multiple siblings using a multi-head message passing.
The embedding generation process takes the graph g = (V,E) and features for all nodes x,, Vu e V, as input. This is followed by K building blocks that generate hidden embeddings. Finally, readout function is applied the output of the last block to generate the graph representation. For nodo-level tasks, the embeddings generated by the last. block directly used.

---

Published in Transactions on Machine Learning Research (08/2025)
Table 1: Classification accuracy (%) on MNIST and CTFAR10 on the superpixel graph classification task.
The cross-generation crossover and mutation operations are applied the base graph neural network model.
Model
MNIST
CIFAR10
MoNet (Monti et al.
2017)
90.805++0.032
54.6555+0.518
GraphSACA AGE (Hamltonn et al.
2017
97.312+00099
65.767+0.308
GIN (Xu et al.
2019
96.485-1..255
15.255=1.522
GCNI Chen et al.
2020
90.667+10.143
56.081+1.119
PNA Corso et al. 2020)
97. 97.14+0.12
70.35 63
DGN Beaini et al.
2021
72.838+0.44
CRa WT (Toenshoff et al.
2021
97.94410.000
69.013 259
GIN-AK+ ZZaao al.
2021
72.19:0.11
3WLGNN MMrron et al.
2019
95.075-10.961
59.175
EGT (Hussain et al.
2022
98.173++0.087
68.702 0.09
GatedGCN + SSFG /Zhang et al.
2022
97.985+10.032
71.938
EdgeGCN Zhang et al.
2023)
98.4322:0.055
76.127::0.45
Exphormer (Shirzad et al. 2023
98.550+0.000
74.754
TIGT CCoi et al.) 2024)
98. 08.230+0..0
73.955#0.360
Rand Align GatedGCN
(Zhang & Xu 2024
98.512++0003
76.395
GCN
Kipf & Welling 2016
90.70511.2218
55.710:10.381
GCN + Ours
95.926:10.031
59.15711.130
GAT (Veličković et al. 2018
95.535-10.200
64.223
GAT
Ours
97.643++0.10
69.247
GatedGCN
[Bresson & Laurent 2017 Ours
97.340++0.0
67.312:10.3
GatedGCN
98.526110.031
78.242:10.22
GPS Rampasek et al. 2022
98.051+0.112
12.298-10.355
Finctuned GPS Tönshoff et al. 2024
98.186++0.100
75.680+t0.
Finetuned GPS Ours
98.685++0.029
80.636++0.195
4 Empirical Evaluation
4.1 Datasets and Setup
The experiments are conducted on six benchmark datasets, i.e., MNIST, CIFARIO, PascalVOC-SP, COCO- SP, Peptides-fune and Peptides-struct (Dwivedi al.) 2020 2022) on three graph tasks, graph classification, node classification, and graph regression. We closely follow the setup Dwivedi al. (020 2022 for training and evaluating the models. The details of the datasets and evaluation metrics provided in the appendix section.
4.2 Results
CIFAR10 and MNIST. Table I reports the results the two datasets on the superpixel classification task. We use the GPS (Rampasek et al. 2022 as the base model. The GPS model is hybrid of local aggregation and global aggregation architecture. It uses GatedGCN for local aggregation and uses Transformer for global aggregation. We apply cross-gencration and mutation (i.c., block (a) in Figure 2 to the base GatedGCN model. The crossover rate is set to 0.5 and mutation rate set to 0.1. We see from Table D that our met improves the performance
the base Finetuned GPS by a large margin, with a relative improvement of 0.44% and 6.55% on MNIST and CIFAR10 respectively. It simultancously outperforms both Exphormer (Shirzad et al. 2023) and RandAlign (Zhang Xu 2024), which previously achieved the best performance MNIST and CIFAR10 respectively.

---

Published in Transactions on Machine Learning Research (08/2025)
Table 2: Results on PasCalVOC-SP and COCO-SP on the node classification task. The cross-generation crossover and mutation operations applied to the base graph neural network model.
PascalVOC-SP (F1 f)
COCO-SP
Model
(F1 f)
GCN
(Kipf & Welling 2016
0.1268=:00006
0.0841=0.0010
GINE
Hu et al. 2019
0.1265++0.0076
0. 0.1339=+0.004
GCNII
CChen et al. 2020)
0.1698:10.0080
0.14041:00011
GatedG
Bresson Laurent
2017
0.2873=0.0219
0.2641=0.0045
GatedGCN RWSE (Rampasel) et a..
2022
0.2860+000008
0.2574=0.0004
Transformer LapPE Dwivedi ct al. 2022
0.2694
0.2618=000031
SAN + LapPE Dwivedi et al.
2022
0.3230+10.0033
0. 0.2592-+0.0158
SAN + RWSE Dwivedi et al.
2022
0.3216
0.2434=0.0156
Exphormer Shirzad ct al. (2023)
0.3975++0.037
0.3455==0.0009
RandAlign GPS [Zhang Xu
2024
0.4042+10.001
0.35671000026
Finetuned GCN
Tönshoff al.
2024)
0.2078
0. 0.1338=0.0007
Finetuned GCN
Ours
0.2445+10.0008
0.1485=0.0004
Finetuned CatedGCN
Tönshoff et al. 2024
0.3850+10.0040
0.2922:10.0018
Finetuned GCNedGCN Ours
0.417610.0013
0.3226=0.0010
GPS Rampasek et al. 2022) Finctuned GPS Tönshoff ct al. 2024
0.3748+:0.019
0.3412=0.0044
0.4440==0.0060
0.355
Finetuned GPS Ours
0.4832+000001
0.4002==0.0019
Table 3: Results on Pepti-func and Pepti-struct. The sibling crossover and mutation operations are applied to the base graph nerual network model.
Poptides-func (AP +)
Peptidos-struct
Model
(MAE J)
GCN
0.5930=0.0023
0.3496++0.0013
GINE
0.5498=0.0079
0.3547:10.005
GCNII Chen al. 2020
0.5543=0.0078
GatedGCN
0.5864=0.0077
0.3420+0.0013
Gated + RWSE
0.606900.0035
0.3357+000006
Trams
0.6326=0.0126
0.252910.0016
SAN+LapPE
0.6384=0.0121
0.268310.0043
SAN+RWSE
0.6439=0.0075
0.2545++0.0012
Exphormer (Shirzad
2023)
0.6527=0.0043
0.2481+000007
GPS (Rampasek et al. 2022)
0.6535=0.0041
0.2500++00005
Finetuned GPS
Tönshoff et al.
2023
0.6534=0.0091
0.2509-10.0014
Finctuned GPS + Ours
0.6730=0.0053
0.2486:10.0009
Finetuned GCN
Tönshoff et al.
2023
0.6860=0.0050
0.2460+00.007
Finetuned GCN
Ours
0.7021 -0.003
0.2426+0.0014
PascalVOCCSP and COCO-SP. The two dalasets are long-range prediction datasets compared to MNIST and CIFAR10. The task is to predict if a node corresponds to a region of an image which belongs to a particular class. We use Finctuned GPS (Tönshoff et al. 2023) as the base model. The Finetuned GPS is also a hybrid of GatedGCN and Transformer architecture. We apply cross-generation and mutation to the base GatedGCN model. The crossover rate is set to 0.9 and mutation rate is set to 0.05. The results are

---

Published in Transactions on Machine Learning Research (08/2025)
Table 4: Ablation study: Importance of crossover and mutation on the model performance on CIFAR10 and PascalVOC-SP.
Base Model
Crossover Mutation
CIFAR10
PascalVOC-SP
X
75.680=+0.188
0.4440=000005
Finetuned GPS
X
79.434+0.2228 0.4952+10.009
Tönshoff et al.
2023
V
77.029
0.4554+0.0077
80.6 30.636+00.155 0.4832++0.0031
Accuracy
0.78
0.78
0.7
0.25
Croesover rate
Crossover rate
Figure 3: Impact of the crossover rate p on the performance of Finetuned GPS on CIFAR10 and PascalVOC- SP.
CIFAR10
PasculVGc-8P
0.8
90
0.7
85
Acournyy
GPS WD method Training
GPS mehhod -raniing
0PS our metifdd Traning
(FS our mxtfod Taaning
0P8 nur rrsftod Tost
GPS oz method
65
Layer number
Layer number
Figure 4: Results of our method on the base Finetuned GPS model with different layers on CIFARIO and PascalVOC-SP.
reported in Table 2 Previously, Finetuned GPS achieved the best performance among the baseline models on the two datasets. As compared to Finetuned GPS, the use of our method results in a relative improvement of 8.83% and 3.04% respectively without using additional model parameters. Once again, our framework achieves new state-of-the-art performance on the two datasets.
Peptides-func and Peptides-struct. We Finetuned GCN (Tönshoff et al. 2023 as the base model on the two datasets. We use sibling crossover and mutation to the base model. The number of siblings is set to 2 and mutation rate is set to 0.1. The results are reported in Table 6 Finetuned GCN is a strong baseline model in previous work. We see from Table t that the use of framework further improve the model performance.
A blation Study. We conduct an ablation study on CIFAR10 and PascalVOC-SP analyse the importance of crossover and mutation the model performance. Table shows the ablation study results. Tt can be
10

---

Published in Transactions on Machine Learning Research (08/2025)
CIFAR10
PASCALVOC-SP
1.05
Average Cosine Simiarby
0.95
Average Cosino Simiaaryy
0.9
0.85
0.85
0.75
0.8
0.75
0.68
0.65
GDN16
our mwfroc
0.56
OCNIS our melhod
0.8
Layer
Layer
Figure 5: Average cosine similarity at different model layers on CIFAR10 and PascalVOC-SP
seen from Table 5 that the crossover operation plays a major role improving the model performance. The mutation operation helps further improve the model performance as a regularization method.
We further analyzed the impact of the crossover rate p model performance CIFAR10 and PascalVOC-
SP. Figure 3 shows the experimental results. We see that the best performance is achieved when is set to different values on the two datasets. When is set to O, it is equivalent to not using crossover. A recommended strategy for tuning p is starting from a small value, e.g., 0.05, and then gradually increasing it to find the optimal value. The strategy also used for tuning the mutation rate in our evaluations.
We conducted experiments to analyze the performance of our method on the base Finetuned GPS model with different layers CIFARIO and PascalVOC-SP. The results are shown in Figure 4 We also analyzed the performance of our method the base Finetuned GPS model with different layers CIFAR10, and the results are reported in Figure 7 in the appendix section. It can be seen from Figure 3 and Figure 7 that the use of our method improves the model generalization performance on the base models.
We compared our method with the basic GCN which residual connections and batch normalizations are not used on MNIST and CIFAR10. The results are shown in Table We see that the model performance drops without using these techniques and that the joint use of our method with residual connections and batch normalizations yields the best task performance.
We validated our method for few-shot learning on the Omniglot datasel (Lake et al. 2015). The results are reported in Table 6 The base GNN (Satorras & Estrach, 2018) model achieves competitive results while remaining simpler than other methods. The use of our mothod further improves the performance of GNN Satorras & Estrach 2018) for 1-shot 20-way and 5-shot 20-way experiments.
Figure 5 visualizes the average cosine similarity at different layers of GCN model on CIFAR10 and PascalVOC-SP. On the CIFAR10 dataset, early stage for the base GCN model, which indicates that the oversmoothing problem occurs. With our method, it can be seen that the average cosine similarity is reduced. On the PascalVOC dataset, the average cosine similarity is also reduced when out method is applied to the base GCN model. Figure 6 shows the results of our method on vanilla GCN with increased layers on MNIST and CIFAR10. We see that the results for the vanilla GCN drop significantly on the two datasets. In contrast, our method enables the model to maintain stable performance, cxhibiting only a slight drop with deeper layers. The results show that our method effectively prevents the model performance from severe drop with increased layers.
that the average cosine similarity converges to 1 at
The idea of genetic operations is to enhance embedding diversity during training to prevent the model from premature convergence. The genetic operations fundamentally work as regularization approach by introducing randomness. The crossover operation can be understood as a random pooling method, which randomly recombines features of different input embeddings according to a predefined probability value. The mutation operation can be seen as a generalized Dropout method that randomly replaces the feature point at dimension with value sampled from a Gaussian distribution, wherein the mean and variance of the Gaussian distribution calculated using batch data. The Dropout method, in contrast, randomly replace
Discussion.
11

---

Published in Transactions on Machine Learning Research (08/2025)
Table 5: Comparison of our method with the basic GCN, wherein residual connections and batch normal-
izations (BN) are not used.
Model
MNIST
CIFAR10
GCN (w/o residual connections and BN)
87.590
48.810#1.045
GCN (with residual connections and BN)
90.705++0.218
55.710=0.388
GCN (with residual connections and BN) Ours
95.926:40.031
19.15710.130
Table 6: Results on the base GNN model (Satorras Estrach
2018 for few-shot learning on Omniglot.
20 Way
Model
1-shot
5-shot
Siamese Net (Koch al. 2015
88.2%
97.0%
Matching Networks [Vinyals et al. 2016
93.8%
98.5%
Prototypical Networks (nell et al. 2017
95.4%
98.8%
ConvNet with Memory Kaissr et a
2011
95.0%
98.6%
Meta Networks (Munkhdalai & Yu
2017
97.0%
R2-D2 [Bertinetto al. 2019)
96.24%
99.20%
GNN
Satorras & Estrach 2018
97.4%
99.0%
GNN
+ Ours
98.03%
99.47%
the feature point at dimension with the value of 0. The fundamental idea of genetic operations provides new perspective on machine learning tricks. It could be potentially applied for regularizing other machine learning models, such as multilayer perceptrons and Transformers.
In this work, the hyperparameters, i.c., crossover rate and mutation rate, are determined by empirical tuning through experiments. We found that of the crossover rate and the mutation rate were set to different values to achieve the best performance different tasks. In future work, automatic methods could be explored for searching the optimal crossover and mutation rates. This would help save the extensive hyperparameter tuning procedure.
The proposed genetic operations introduce randomness in the training procedure. Therefore, the model requires more training epochs to optimize, which increases the training time. Additionally, the optimal combination of genetic operations and the hyperparameters need be tuned experimentally for different tasks and different base models, which further increases the training time.
5
Conclusions
This paper presents a new framework called cnnutic-vvoluionnayy graph neural networks for graph represen- tation learning. The key idea of our approach to view each layer of a graph neural network as a genetic evolutionary process and neural networks. We developed three operations, i.e., cross- gesss-ennrrttinn crossover, sibling crossover and muta- tion, inspired by genetic algorithms and presented two building blocks based on the the operations for graph representation learning. An important advantage of the proposed framework lics in its interpretability, as it frames layerwisely graph representation learning as an evolutionary process. The experimental evaluations were conducted on six popular datasets on different graph tasks. The results showed that the use of our framework significantly improves the performance of the base graph neural networks, achieving new state- of- the-art performance for graph representation learning on these datasets. We also presented ablations of our framework, showing the importance of each operation on the overall model performance.
operations to prevent the over-smoothing problem in graph
12

---

Published in Transactions on Machine Learning Research (08/2025)
MNIST
CIFAR10
100
70
60
80
00
Nocuracy
30
80
20
20
10
-0- Vedla OCN Sel)
Vall GCN (ua
Vella GEN Quas
Layers
Layers
Figure Performance of our method on the vanilla GCN with increased layers.
6 Acknowledgments
This work was supported by the Australian Research Council under the Linkage Projects Grant LP210100129.
References
Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süsstrunk. Slic superpixels compared to stato-of-thc-art superpixel methods. IEEE transactions on pottern analysis and machine intelligence, 34(11):2274 2282, 2012.
B Alhijawi and A Awajan. Genetic algorithms: theory, genetic operators, solutions, and applications, evol. intel., 2023, 2023.
Hussain Alibrahim and Simone A Ludwig. Hyperparameter optimization: Comparing genetic algorithm against grid search and bayesian optimization. In 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 1559. IEEE, 2021.
Oluleye H Babatunde, Leisa Armsstron, Jinsong Leng, and Dean Diepeveen. A genetic algorithm-based feature selection. 2014.
Pablo Barceló, Egor V Kostylev, Mikael Monet, Jorge Pérez, Juan Reutter, and Juan- Silva. The logical expressiveness of graph neural networks. In 8th. International Conference on Learning Representattons (ICLR 2020), 2020.
Dominique Beaini, Saro Passaro, Vincent Létourneau, Will Hamilton, Gabriele Corso, and Pietro Liò. Di- rectional graph networks. In International Conference on Machine Learning, pp. 748-758. PMLR, 2021.
Luca Bertinetto, Joao F Henriques, Philip HS Torr, and Andrea Vedaldi. Meta-learning with differentiable closed-form solvers. In International conference on learning representations, 2019.
Xavier Bresson and Thomas Laurent. Residual gated graph convnets. arXiv preprint arXiu:1111.0755 2017.
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks graphs. International Conference Learning Representations, 2014.
Chaoqi Chen, Yushuang Wu, Qiyuan Dai, Hong-Yu Yu Zhou, Mutian Xu, Sibci Yang, Xiaoguang Han, and Yizhou Yu. A survey on graph neural networks and graph transformers in computer vision: A task- oriented perspective. IEEE Transactions Pattern Anolysis and Machine Intelligence, 2024.
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In International Conference on Machine Learning, pp. 1725-1735. PMLR, 2020.
13

---

Published in Transactions on Machine Learning Research (08/2025)
Yihao Chen, Xin Tang, Xianbiao Qi, Chun-Guang Li, and Rong Xiao. Learning graph normalization for graph neural networks. Neurocompulingg 493:613-624 2022.
Yun Young Choi, Sun Woo Park, Minho Lee, and Youngho Woo. Topology-informed graph transformer. arXiv preprint 2xxt::2422.0005,, 2024.
Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liô, and Petar Veličkovié. Principal neighbourhood
aggregation for graph nets. Advances in Neural Information Processing Systems, 33: 33:13260 13271, 2020.
Yuval Davidor. Genetic Algorithms and Robotics: A heuristic strategy for optimization, volume 1. World Scientific Publishing Company, 1991.
Vijay Prakash Dwivedi, Chaitanya K Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Bench- marking graph neural networks. arXie preprint 0rX0:2203.00082, 2020.
Vijay Prakash Dwivedi, Ladislav Rampasek, Mikhail Galkin, Ali Parviz, Guy Wolf, Anh Tuan Lau, and Dominique Beaini. Long range graph benchmark. In Thirty-siath Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
Justin Gilmer, Samuel S Schocnholz, Patrick F Riley Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp. 1263 1272. PMLR, 2017.
Deepli Gupta and Shabina Ghafir. An overview of methods maintaining diversity in genetic algorithms. International journal of emerying technology and advanced engineering, 2(5):56-66)) 60, 2012.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning large graphs. In Advances in neural information processing systems, pp. 1024-1034, 2017.
William L Hamilton.
Graph representation learning. Synthesis Lectures on Artifical Intelligence and Machine Learning, 14(3):1-159, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Tn Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 778, 2016.
John H Holland. Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT press, 1992.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategics for pro-training graph neural networks. arXiu preprint 1xXx:19005.1263 2019.
Md Shamim Hussain, Mohammed J Zaki, and Dharmashankar Subramanian. Global self-attention replacement for graph convolution. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp- 655-665, 2022.
Lukasz Kaiser, Ofir Nachum, Aurko Roy, and Samy Bengio. Learning to remember rare events. ar preprint 1xXte:1703.031299 2017.
Dimitrios Kelesis, Dimitrios Vogiatzis, Georgios Katsimpras, Dimitris Fotakis, and Georgios Paliouras. Re- ducing oversmoothing in graph neural networks by changing the activation function. In ECAT 2023, pp. 1231-1238. IOS Press, 2023.
Dimitrios Kelesis, Dimitris Fotakis, and Georgios Paliouras. Reducing oversmoothing through informed weight initialization in graph neural networks. Applied Intelligence, 55(7):632, 2025.
Sangwon Kim, Dasom Ahn, and Byoung Chul Ko. Cross-modal learning with 3d deformable attention for action recognition. In Proceedings of the IEEE/CVF international conference computer vision, pp.
2023.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint Xiv: 109. 02907, 2016.
14

---

Published in Transactions on Machine Learning Research (08/2025)
Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. Siamese neural networks for one-shot image recognition. In TCML deep learning workshop, volume 2, pp. 1-30. Lille, 2015.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Brenden M Lake, Ruslan Salakhutdinov, and Joshua. B Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 350(6266):1332 1338, 2015.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- based learning applied to docu- ment recognition. Proceedings of the IEEE, 86(11):2278-2322 2324, 1998.
Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi- supervised learning. AAAT Conference on Artificial Intelligence, 2018.
Sitao Luan, Chenqing Hua, Minkai Xu, Qincheng Lu, Jiaqi Zhu, Wen Chang, Jie Fu, Jure Leskovec, and Doina Precup. When do graph neural networks help with node classification? investigating the homophily principle on node distinguishability. Advances in Neural Information Processing Systems, 36, 2024.
Haggai Maron, Heli Ben-Hamu, Hadar Serviansky, and Yaron Lipman. Provably powerful graph networks. arXiv preprint Xim: 1905 1 11136, 2019.
John McCall. Genetic algorithms for modelling and optimisation. Journal of compulational and Applied Mathematics, 184(1):205 222, 2005.
Geoffrey F Miller, Peter M Todd, and Shailesh U Hegde. Designing neural networks using genetic algorithms. In ICGA, volume 89, pp. 379-384, 1989.
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bron- stein. Geometric deep learning on graphs and manifolds using mixture model enns. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5115-5124, 2017.
Tsendsuren Munkhdalai and Hong Yu. Meta networks. In International conference on machine learning, pp. 2554 2563. PMLR, 2017.
Kenla Oono and Taiji Suzuki. Graph neural networks exponentially lose expressive power for node classifi- cation. International Conference on Learning Representation (ICLR), 2020.
Ladislav Rampasek, Mikhail Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. Recipe for a general, powerful, scalable graph transformer. In Alice H. Oh, Alckh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Redmon. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pottern recognition, 2016.
Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. Dropedge: Towards deep graph convolutional networks on node classification. In International Conference on Learning Representations, 2020.
Victor Garcia Satorras and Joan Bruna Estrach. Few-shot learning with graph neural networks. In Interna- lional conference on learning representations, 2018.
Adarsh Sehgal, Hung La, Sushil Louis, and Hai Nguyen. Deep reinforcement learning using genetic algorithm for parameter optimization. In 2019 Third IEFE International Conference on Robotic Compuling (TRC), pp. 596 601. IEEE, 2019.
Min Shi, Yufei Tang, Xingquan Zhu, Yu Huang, David Wilson, Yuan Zhuang, and Jianxun Liu, Genetic-gun: Evolutionary architecture search for graph neural networks. Knowledge-baat based systems, 247:108752, 2022.
Hamed Shirzad, Ameya Velingker, Balaji Venkatachalam, Danica Sutherland, and Ali Kemal Sinop. Ex- phormer: Sparse transformers for graphs. In International Conference on Machine Learning, 2023.
15

---

Published in Transactions on Machine Learning Research (08/2025)
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems, 30, 2017.
Jan Toenshoff, Martin Ritzert, Hinrikus Wolf, and Martin Grohe. Graph learning with 1d convolutions on random walks. arXiv preprint or 11::2102.08786, 2021.
Jan Tönshoff, Martin Ritzert, Eran Roscnbluth, and Martin Grohe. Where did the gap go? reassessing the long-range graph benchmark. The Second Learning on Graphs Conference (LoG 2023), 2023.
Jan Tönshoff, Martin Ritzert, Fran Rosenbluth, and Martin Grohe Where did the gap go? reassessing the long-range graph benchmark. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL
https://openreviev.
Petar Velíčković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liø, and Yoshua Bengio. Graph Attention Networks. International Conference on Learning Representations, 2018.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016.
Darrell Whitley. An overview of evolutionary algorithms: practical issues and common pitsalls. Information and software technology, 43(14):817 831, 2001.
Xinyi Wu, Amir Ajorlou, Zihui Wu, and Ali Jadbabaie. Demystifying oversmoothing attention- graph neural networks. Advances Neural Information Processing Systems, 36:35084-35106, 2023.
Xinyi Wu, Amir Ajorlou, Zihui Wu, and Ali Jadbabaie. Demystifying oversmoothing in iitention-based graph neural networks. Aduances in Newral Information Processing Systems, 36, 2024.
Jiancong Xie, Yi Wang, Jiahua Rao, Shuangjia Zheng, and Yuedong Yang. Self-supervised contrastive molec- ular representation learning with chemical synthesis knowledge graph. Journal of Chemical Information and Modeling, 64(():1945-199) 2024.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? 2019.
Haimin Zhang and Min Xu. Randalign: parameter-free method for regularizing graph convolutional networks. Xiv preprint arXio:2404, 09774, 2024.
Haimin Zhang, Min Xu, Guoqiang Zhang, and Kenta Niwa. Ssfg: Stochastically scaling features and gradients for regularizing graph convolutional networks. IEEE Tronsactions on Neural Networks and Learning Systems, 2022.
Haimin Zhang, Jiahao Xia, Guoqiang Zhang, and Min Xu. Learning graph representations through learning and propagating edge features. IEFE Transactions on Newral Networks and Learning Systems, 2023.
Jiahao Zhang, Rui Xue, Wenqi Fan, Xin Xu, Qing Li, Jian Pei, and Xiaorui Liu. Linear-time graph neural networks for scalable recommendations. In Proceedings of the ACM on Web Conference 2024, pp. 3533- 3544, 2024.
Lingxiao Zhao and Leman Akoglu. Pairnorm: Tackling oversmoothing in gnns. International Conference on Learning Representations, 2020a.
Lingxiao Zhao and Leman Akoglu. Pairnorm: Tackling oversmoothing in gnns. International Conference on Learning Representations, 2020b.
Lingxiao Zhao, Wei Jin, Leman Akoglu, and Neil Shah. From stars to subgraphs: Uplifting any gnn with local structure awareness. arXiv preprint Xrxiv:1 10. 03753, 2021.
16

---

Published in Transactions on Machine Learning Research (08/2025)
A
Appendix
Datasets.
The experiments were conducted on the following six benchmark datascts.
MNIST and CIFAR10 two datasets for superpixel graph classification (wiveddi al. 2020
The superpixels are converted from original images in MNIST (LeCun et al. 1998 and CIFAR10
(Krizhevsky et al. 2009 using the SLIC algorithm AAchant et al. 2012).
PascalVOC-SP and COCO-SP are two datasets of superpiexels (Dwivedi el 2022), which are converted from images in original PascalVOC and COCO datasets. The task on the two datasets is to predict if a node corresponds to region of an image which belongs to particular class.
Peptidcs-func and Peptidcs-Struct Dwivedi et al. 2022 are two datasets of peptides molccular graphs. The nodes in the graphs represent heavy (non-hydrooeen) atoms of the peptides, and the edges represent the bonds between these atoms. The graphs are categorized into 10 classes based on the peptide functions, e-g., antibacterial, antiviral, cell-cell communication. The two datasets used for evaluating the model's performance for multi-label graph classification and multi-label graph regression.
The statistics of the benchmark datasets used in the experiments are shown in below Table
Table 7: Statistics of the six benchmark datasets used in the experiments.
Graphs
Nodes
Arg. nodes/graph
# #Training
#Vaiddttio
# #Testt
# Categories
MNIST
70K 60K
40-75
55,000
5000
10,000
10
CIFARIO
85-150
45,000
5000
10,000
10
PoscalVOC-SP
5,443.545
479.40
20
COCO-SP
123,286
58,793,216
476.88
113,286
5,000
5,000
81
Peptides-fune
15,535
2,344,839
150.94
70%
15%
15%
10
Peptides-struct
15,535
2,344,859
150.94
70%
15%
15
Evaluation Metrics. Following Dwivedi et al. (2020) and Rampasek et al. (0022 the following metrics are used evaluation on different lasks. The performance MNIST and CIFAR10 on graph classification is evaluated using the classification accuracy. The performance on PascalVOC-SP and COCO-SP on node classification is evaluated using the macro weighted F1 score. The performance on Peptides-func on multi- label graph classification is evaluated using average precision (AP) across the categories. The performance on septides-struct on multi-label graph regression evaluated using mean absolute error (MAF).
17

---

Published in Transactions on Machine Learning Research (08/2025)
Algorithm cross-generation Crossover, Sibling Crossover, and Mutation
function
h in, crossover prob)
// current layer embedeing h, previous layer embedding h in, crossover probability crossover_ prob if model.training = True then // training phase srrnoolllscmmlleppooooooooo
3:
4: crossover mask =
probp // each value in
crossover mask is sampled from the Bernoulli distribution with probability p
h crossover in crossover mask * (1 crossover (omooerr__ss))
else // inference phase
h crossover h in * + h (1 pp)
8: end if
return h crossover
10: end function
11: function ShoLiNChRooov sibling list)
12:
// list of sibling representations h sibling list.
13:
num hum__sbblinge = len(h sibling list)
14:
if model. training
Truc then // training phase
15:
sibling mask one hol (ww.alioooiaal.saalleeeeeiiooepe h crossover for E (1,..., hwi_nuummmmmnnnn siblings) do
nam siblings size num siblings))
16:
17:
h crossoover = h lisibbiing list[i] * sibling mask[i] end for 20: else // inferennce phase
18:
19:
21: h crossover mean bhinling list)
22: end if
23:
return h crossover
24:
end function
25:
function MUTATION(h, Cartoonhh_babhhhheeeee mutation_prob
26:
mean = h batch.mean()
27: var = h batch. batch.car()
28:
batch_size size = h batchhbatch__siee
29:
running_ _eanin rwnning var = UPDATE _ MEAN (eeeaaAARammaammma var)
30: if modei.training Truc then // training phase
31:
gaussian noise Gaussian.sampple(h batch.shape) mutation mask Bernoulli batch.shape) btt.aamappppabaaahhhhhhppe h (gaussian noise running vor + running (anning_mean) mutation mask + h batch (1
32:
33:
mutation mask)
34:
else // inference phase
35: h = rrwnnnng_meett mean * mutation _actii__rbbbb_b + batch (1 - mutationn
36: end if
37: return
38: end function
18

---

8: end for Published in Transactions on Machine Learning Research (08/2025)
Algorithm 4 The embedding generalion process with cross-generation crossover and mutation.
Input: Graph (V, E); number of graph neural layers K; input node fcatures (x,, Vu E V); crossover
probability p
Output: Node embeddings (K) for all u E V
1: h - Xx,
V for k .,... do
for V do
ha Update hu (k-1) (0 D -,VV N(w))) // generate embedding for
using Equation CIoSSCCNERATOOONOOSSVVnllll
h (N) +
he (x-1) .p)
MUTATION(h," (R), mutaaioon_prol hg
6:
7:
end for
Algorithm The embedding generation process with sibling crossover and mutation.
Input: Graph G (V, E); number of graph neural layers K; input node features {x., Vu VJ; number of siblings num hamm_siblings mutation probability mutation_prol
Output: Node embeddings ((k) for all h xx,, Yu V
for = 1, K do
for e V do
4:
for E num num_mimilll siblings do // generate multiple sibling representations using multi-head message
passing.
h = Uplate! (n.* (n( Aggregate!" ((h! (k-1) Ve E N(())))
end for (k)
SUBLINGCROSSOVER(MM (k) h (A)
ha
+- MUTATION(h," (k) ha
(k-1) mutation prob)
end for
10: end for
19

---

Published in Transactions on Machine Learning Research (08/2025)
CIFAR10
95
GCN w/o our method Training
90
GCN w/o our method -Test
GCN our method -Training
85
GCN method -Test
80
75
Accuracy
70
65
60
55
45
0
4
8
12
16
Layer number
Figure 7: Results of our method on the base Finetuned GCN model with different layers on CIFAR10.
20


Reviews:
[{"review_id": "WHQPufw2wO", "reviewer": "Reviewer_eHe7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n- Oversmoothing is an interesting problem in GNNs.\n- Bringing intuition from genetics to the GNN field is an innovative idea that can advance our understanding of GNNs.\n- The formulation of the proposed method is clear and easy to follow.\n\nWeaknesses\n\n- Introduction mixes background on genetic algorithms that belongs in Related Work and overlaps with introductory material.\n- Treating each layer as an evolutionary process while it actually brings distant information from multi-hop neighborhoods is confusing. Can the authors further elaborate on the intuition behind that claim?\n- Related Work on oversmoothing mitigation is incomplete, it omits methods related to oversmoothing reduction, such as activation function effects, edge removal strategies, and weight initialization techniques.\n- I think there is a missing \"of\" in the last sentence of the Abstract. Also, in Section 3.2.1 contains typos (\"using using”, \"information information”) and Section 3.2.2 contatins notation errors (e.g., Equation 2’s superscript should be $k‑1$ not $k$).\n- Cross-generation crossover mechanics lack detail, i.e., probability schedules and random draws are not specified, and non-learnable operations raise questions about model training.\n- In \"sibling crossover”, initialization seeds and their effect on head representations are not explained. If all heads are initialized with the same seeds, then one would expect to create the same representations, hence leading to no embedding variety. Can the authors further elaborate on this?\n- Mutation design choices (e.g., Gaussian distribution, small mutation rates $\\leq$ 0.1) are not well-explained. Additionally, in general Mutation seems to offer little benefit and incur extra training cost, this warrants further justification.\n- Extremely high crossover rates (e.g., 0.9 on PascalVOC-SP and COCO-SP) may nullify message passing by preserving initial values across layers.\n- Figure 4 shows that GPS does not suffer from oversmoothing even without the proposed method, which undermines the motivation of the paper.\n- Experimental setup lacks deep model scenarios and explicit comparisons of vanilla vs. genetically enhanced GNNs as depth increases.\n- Hyperparameter selection procedure (validation set tuning) is not explicitly described.\n- The claim of interpretability is not well-supported, since stochastic operations driven by random sampling obscure per‑layer behavior.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a framework, i.e., genetic-evolutionary graph neural networks, which regards each layer of a message-passing GNN as an evolutionary process inspired by genetic algorithms. By developing three operations (two forms of crossover and mutation), the framework enhances embedding diversity at each layer, thereby claiming to reduce oversmoothing which shrinks deep GNNs performance. The authors claim that the proposed method is both interpretable, owing to its direct analogy with biogenetic processes, and broadly applicable, as it can be seamlessly integrated into existing message-passing architectures. Empirical evaluation on six benchmark datasets across various graph learning tasks demonstrates that the proposed method outperforms baseline models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please kindly refer to the Weaknesses.", "broader_impact_concerns": "N/A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mp8DGtMlHX", "reviewer": "Reviewer_YiwB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+ The proposed methods are simple and effective. While sibling crossover requires additional training and is only applicable to certain GNN architectures, the other two methods -- cross-generation crossover and mutation -- are more broadly applicable.\n+ Experiments are performed on diverse tasks, including node classification, graph classification and regression, and the results demonstrate the effectiveness of the proposed method\n+ The paper is mostly well written, and the ideas are clearly presented\n\nWeaknesses:\n+ While the proposed methods are motivated primarily as a means to reduce oversmoothing, there are no ablation studies included showing that oversmoothing is indeed reduced. \n+ It is unclear how sensitive the proposed method is to hyperparameters. The authors mention that different crossover and mutation rates are used for different tasks and models, but it is unclear how these rates were chosen. \n+ The need and scope of the sibling crossover method are not well defined.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes incorporating genetic algorithms into the training of GNNs to reduce oversmoothing. Two types of genetic algorithms are introduced for GNNs: (1) crossovers are performed between embeddings in the current and previous layers, or between messages received from siblings, and (2) mutations are applied to embeddings by randomly masking out some entries in the vector.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Critical for acceptance:\n+ While accuracy is indeed improved by incorporating the proposed methods into training, there is no evidence presented to show that the accuracy improvements came from reduced oversmoothing. In fact, crossover and mutation can be seen as a form of regularization, where some intermediate activations are randomly masked out and modified, and regularization is a well-known method of improving accuracy on different classes of neural networks. A study of inter- and intra-class node embedding similarities with and without the proposed methods would validate the hypothesis of reduced oversmoothing.\n+ Describe how mutation and crossover rates were chosen for different tasks and models. (1) Did the authors tried out different hyperparameter values and report only the best results? (2) Is there a way to automate this search for optimal crossover and mutation rates?\n+ What is the need for sibling crossover? (1) Is it meant to be used along with cross-generation crossover, or should only one of them be used at a time? (2) When is sibling crossover meant to be used? In particular, why is not used for experiments on MNIST/CIFAR? (3) What is the additional overhead of training the multiple message passing heads?\n+ Clarify whether the proposed methods are used only during training, or if they are used during inference also. It seems like inference can also benefit from the proposed methods, especially in tasks such as graph classification where training and inference are performed on different graphs.\n+ Add a discussion on limitations of the proposed method. In particular, can this be applied to few-shot learning? When only few training samples are available, the regularization effect of the proposed methods could destabilize training.\n\nWould strengthen the paper:\n+ Add results on one few-shot learning task\n+ Release of source code would help reproducibility", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Pf469E78oy", "reviewer": "Reviewer_8ebW", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n       Idea of interfusing ideas from genetic algorithms into GNNs is an interesting approach to tackling the oversmoothing problem\n       \n       Mostly well motivated and well explained\n\n      Show the addition of genetic steps improves performance on interesting data sets, beyond the overstudied node-classification data sets (e.g., CORA)\n\nWeaknesses:\n\n      Sibling crossover (as well as the concept of siblings) is not well explained and I am still not exactly sure what it is\n\n      Experiments seem to focus on using Genetic Algorithms to improve GPS and related. It would be interesting to see how they work with simpler MPNNs, e.g., GCN or SAGE", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper aims to mix ideas from genetic algorithms into message passing neural networks to avoid the well-known oversmoothing problem. Specifically, each block consists of one message passing operation (choice can vary) and then genetic-inspired operations (crossover and mutation). They then demonstrate that the addition of genetic steps improves network performance on tasks such as super-pixel classification  and long-range-prediction-node-classification", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Section 2.1: It is not true that spectral GNNs are message-passing operations since they do not, generically speaking, have localized receptive fields. (Exception is low-order polynomials of the graph laplacian.) Please revise accordingly\n\nLast sentence of 2.1 is unclear. Please revise.\n\nSome of the genetic steps seem related to standard GNN tricks. For instance, cross-generation crossover seems like it could be understood as  a randomized gating mechanism and mutation seems like a form of regularization via noise. Note, I think that this is a strength, not a weakness as it allows a new perspective on \"ML tricks.\" However, discussion of this point would be helpful and significantly improve the paper\n\nPlease add pseudo code for the cross-sibling step. Also, a \"big algorithm\" that ties the full block together.\n\nPlease run some experiments with simpler GNNs as a the \"base model\" rather than GPS\n\nInclude more discussion of other work comparing evolutionary algorithms and GNNs\n\nMinor:\n\n      Abstract: \"significant\" -> \"significantly\"\n\n      First paragraph of intro: Make it clear which references are for which topic\n\n      Second paragraph of intro: \"network was been derived\" -> \"networks have been derived\"\n\n      Third paragraph of intro: Add citation for GNNs being bounded by WL (I think Xu et al is the standard here)\n\n      Same paragraph, add more citations for oversmoothing problem\n\n     Section 3.1: Change 1 to I in the definition of L_{sym}\n\n    Section 3.2.1: \"algorhtms\"\n\n    Table 1: It appears you write GCN when you mean GPS for Rampasek et al 2022. Please revise and double check throughout for similar issues\n\n    Discussion of Table 1: The relative improvement of 11.53% seems wrong. It appears to be 80%-75% for fine-tunded GPS and 59%-55.7% for GPS. Please double check all discussion for correct summarization", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)