Self-Assembling Graph Perceptrons
Authors: Jialong Chen
Tong Wang
Bowen Deng
Luonan Chen
Zibin Zheng
Chuan Chen
Venue: NeurIPS 2025 spotlight ()
Keywords: Self-Assembling Neural Network
Plastic Neural Network
Primary area: neuroscience_and_cognitive_science
TL;DR: A graph-structured perceptron model with self-assembly capabilities.

Abstract: Inspired by the workings of biological brains, humans have designed artificial neural networks (ANNs), sparking profound advancements across various fields. However, the biological brain possesses high plasticity, enabling it to develop simple, efficient, and powerful structures to cope with complex external environments. In contrast, the superior performance of ANNs often relies on meticulously crafted architectures, which can make them vulnerable when handling complex inputs. Moreover, overparameterization often characterizes the most advanced ANNs. This paper explores the path toward building streamlined and plastic ANNs. Firstly, we introduce the Graph Perceptron (GP), which extends the most fundamental ANN, the Multi-Layer Perceptron (MLP). Subsequently, we incorporate a self-assembly mechanism on top of GP called Self-Assembling Graph Perceptron (SAGP). During training, SAGP can autonomously adjust the network's number of neurons and synapses and their connectivity. SAGP achieves comparable or even superior performance with only about 5% of the size of an MLP. We also demonstrate the SAGP's advantages in enhancing model interpretability and feature selection.

Self-Assembling Graph Perceptrons
Jialong Chen 8, Tong Wang's 2 Bowen Deng' E
Luonan Chen
Zibin Zheng'
Chuan Chen E
Sun Yat-sen University, Texas A&M University,
Shanghai Jiao Tong University, *University of Chinese Academy of Sciences
Abstract
Inspired by the workings of biological brains, humans have designed artificial neural networks (ANNs), sparking profound advancements across various fields. However, the biological brain possesses high plasticity, enabling it to develop sim- ple, efficient, and powerful structures to cope with complex external environments. In contrast, the superior performance of ANNs often relies meticulously crafted architectures, which can make them vulnerable when handling complex inputs. Moreover, overparameteriiztion often characterizes the most advanced ANNs. This paper explores the path toward building streamlined and plastic ANNs. Firstly, we introduce the Graph Perceptron (GP), which extends the most fundamental ANN, the Multi-Layer Perceptron (MLP), Subsequently, we incorporate a self-assembly mechanism on top of GP called Self-Assembling Graph Perceptron (SAGP). Dur- ing training, SAGP can autonomously adjust the network's 's number of neurons and synapses and their connectivity. SAGP achieves comparable or even superior performance with only about 5% of the size of an MLP. We also demonstrate the SAGP's advantages in enhancing model interpretability and feature selection.
Introduction
With the exceptional intelligence of the brain, humanity has created a remarkable modern civilization. This intelligence stems from the brain's intricate structure, resulting from long-term environmental adaptation and natural selection. Research shows that the survival environment profoundly impacts shaping the brain, which in turn indirectly shapes human cognitive and emotional abilities [1].
the other hand, since the 1940s, researchers have been exploring how to simulate the behavior of the biological brain using computers to build artificial intelligence agents. It was only in recent decades that a framework known as artificial neural networks (ANNs) has indeed demonstrated significant potential in this field. ANNs simulate neurons and their synaptic connections in the brain, determining each neuron's 's output by the strength of the input signals they receive.
Despite the powerful capabilities of ANNs. their design seems to deviate from the initial intent of mimicking the biological brain. In ANNs, the number of neurons and synaptic connection patterns are predefined, resulting in noticeable vulnerabilities when the network faces complex environments. [2] and [3] have demonstrated, from theoretical and experimental perspectives, respectively, the significant impact of the number of hidden layers and neurons on the ability of Multi- Layer Perceptrons (MLPs). Moreover, modern deep models are often parmperameterized with the most advanced large language models reaching parameter scale of trillions. In contrast, biological brains continually self-assemble throughout their lifecycle, developing remarkable capabilities, For instance, nematode manage all its behaviors with fewer than 500 neurons [4].
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Recently, some studies have focused on creating ANNs that can assemble themselves based on input without relying on prior knowledge. [5] and [6] proposed learning genomes that regulate neuronal behavior, enabling self-adjustment of synaptic connection rules without altering the number of neurons. The neural development program (NDP, [7, 8)) first suggests regulating neuron growth through genomes, allowing the network to develop from a single neuron and incrementally add new neurons and synapses in response to imputs, eventually growing into a network of a predefined size. However, NDP does not fully realize biological self-assembly. It neglects the most important reason biological neural systems maintain their efficiency and compactness: neuronal apoptosis [9]. Furthermore, since NDP explicitly encodes genome for generating the network structure and optimizes by reinforcement learning. the high computational cost prevents it from running on even typical-seale dataset.
In this paper, we first introduce a generalized version of the Multi-Layer Perceptron (MLP), namely the Graph Perceptron (GP). Building upon this, we present the Self-Assembling Graph Perceptrons (SAGP) -the first model with full self-assembly capability. SAGP begins from the simplest form and autonomously determines when to grow or undergo apoptosis during its developmental cycle. Synaptic connections between neurons are dynamically adjusted. Unlike previous works, SAGP does not explicitly model the genome that controls neuronal behavior and topology. Instead, it achieves self-assembly by establishing assembly rules and simulating the competitive pressures found in nature. This makes SAGP more aligned with the general paradigm of modern deep networks and results in a significant improvement in the assembly speed by more than 10,000 times. We demonstrate that SAGP can achieve more streamlined perceptron topology than MLP while highlighting the potential of the self-assembly mechanism in enhancing model interpretability and feature selection.
2 Related work
Neural network bionics
Scientists have long sought inspiration from the biological world to develop algorithms for solv- ing real-world problems. Early bio-inspired strategies, such as evolutionary and genetic al- gorithms [10, 11], solve complex optimization problems by simulating the process of biological evolution. Meanwhile, artificial neural networks, which simulate the process of neuronal interac- tions through synaptic connections [12), have achieved great success in numerous applications. However, ANNs, represented by MLP, are pre- defined as layered structures with fixed sizes and usually require a large number of neurons and dense synaptic connections for optimal performance, which is contrary to the properties of biological neural networks (Fig. 1). Consequently, a growing body of research is focused on designing net- works with more bio-inspired features. Spiking Neural Networks (SNNs) mimic the mechanism of neurons transmitting information through action potentials, enabling asynchronous and low-power information processing [13, 14]; Liquid Neural Networks (LNNs) simulate the neural connection of the nematodes, using a small number of neurons to generate continuous- -time outputs [15]. Plastic Neural Networks (PNNs) emulate the developmental processes of organisms, dynamically adjusting their structures based on environmental states [16].
Strueture
Assembly
X
Connection
Netwrrkd
Self Scsemassnilingg
Sparse
Layered
Pre-defined
Dense
Figure 1: Biological NN v.s. Artificial NN.
Synaptic-level plasticity The characteristic of plastic ANNs lies in their ability to adjust their structure in response to environmental changes. Initially, this research area focused on synaptic-level plasticity; inspired by carly neurobiological theories, researchers developed mechanisms such as the Hebbian rule [17] and Spike-Timing Dependent Plasticity (STDP, [18]) to enable ANNs to self- regulate synaptic strength. These classical methods are unsupervised, while more modern approaches employ backpropagation to learn synaptic rules [19, 20] or use meta-learning to obtain genomes that control synaptic behavior [5, 6].
Neuron-level plasticity Some methods allow neurons to adjust their state based on the environ- ment, such as by modifying weights [21], changing activation functions [22]. or learning rates [23]. Neuroevolution [24, 25] encodes the network topology as individuals in population and finds the optimal ones by evolution. [26] considers parameter pruning based on plastic neurons. Only recently

---

has the concept of self-assembly - the dynamic increase of neurons within single network (as opposed a population) - been introduced by Neural Developmental Program (NDP, [7. 8J). We achieve fully self-assembling model with neural competition and apoptosis mechanisms. Compared NDP, our model improves the assembly speed by over 10 times per epoch and shows the ability to integrate with modern deep models. See Appendix A for more related works.
3 Self-Assembling Graph Perceptrons
This section introduces approach, the Self-A Seif-Asse Graph Perceptrons (SAGP), in detail. Without additional constraints, the free connections between neurons may exhibit more general connection pattern than the layered connections, namely, graph-structured connection. Section 3.1, we explore how information is updated in a graph-structured perceptron model (i.e., GP), which forms the foundation for achieving the network's self-assembly capability. Section 3.2 introduces the approach by which GP achieves self-assembly through establishing assembly rules and simulating competitive pressures.
3.1 From MLPs to graph perceptrons
Since the inception of perceptron models, they have been conventionally regarded as layered structures [12, 27, 28, 29]. Although fully connected perceptron models like Hopfield Networks [30] and Boltzmann Machines [31] emerged in the past, they gradually became marginalized due to practical limitations. How- ever, we re-examine this concept inspired by the message- message-passing passing mechanism [32].
Mgo
who
wit
w
Wio
Figure 2: A simple example of Multi-Layer Per- ceptrons, where a represents the activation func- tion.
Take simple MLP with 2 imput neurons, 2 hid-
Zo
den neurons in 1 hidden layer, and output neuron as an example (Fig. 2). Let x
and
[20] represent the model's 's input and output, respectively, then we have:
wT o (Wo and W
x + bo),
(I)
where Wo
0 ooo
wi
W
01
bo
w
Wio
Now, we number all neurons sequentially in the order of input, hidden, and output layers. In this way, the topology of the MLP and the weights of its edges can be represented by a 5 adjacency matrix A. If the i-th neuron has synaptic connection to the j-th neuron, then A, is the weight of that synapse; otherwise, Aij = 0. Input neurons carry self-loops, meaning they continuously send information the network. We also construct vector b of length 5 to represent the biases of all the neurons, where the biases of input and output neurons are set to 0, as follows:
22x2
Wo o
02x1
[02
A
O2x2
O2x2
W1
b =
bu
01x2
01x2
01x1
0
Here, O represents zero matrix vector. If we input the input neurons and use zero inputs for
other neurons, then, after one step of message-passing [32] on the graph described by A, we obtain the same intermediate results at the hidden neurons would in MLPs:
(xx)
A
02 0
+
b
o + bb)
0
By repeating this process once again, the output neuron produces the same result the output of the MLPs:
a(x)
(xx) W Wo + bo
AT.
a (W. x+ + bo)
+ b
(4)
0j
z
3

---

Hidden
Inpur
Output
Competition
Agoplosiis (if (fpossille)
Self-assembling cycle
Figure schematic of our model (SAGP). assembles itself by simulating neuron growth, competition, and apoptosis.
Thus, obtain alternative representation of where denote taking the last two elements of a vector to form a new one:
Python-style indexing [-2:] to
A
A
+ b
+
(5)
03
[-2]
is easy to show that similar conclusions hold for MLPs of any number of hidden layers and sizes (See appendix B).
Based on this observation, we now define a generalized perceptron model, referred to as the Graph Perceptrons (GP). Let the sets of input neurons, hidden neurons, and output neurons be denoted as I, H. and O, respectively. GP has the following adjacency matrix and biases:
Z
AI-
AI-O
[Z] bx
A
0 x
z
AR-H
Ax
0
OIxiZ
010) x(H|
x
0
GP allows direct connections from input neurons to output neurons and interconnections between any two hidden neurons. In GP, the feature update process is described
VI E [0, L - 1), h²t ++1 = At' + + 241 = (hh- where x
0 0p1 and z = h
When Ar-0 0, Ax-I = I, and L 2, the GP degenerate to MLPs with 1 hidden layer. L represents the number of message- meessage-passing steps. It is worth noting that when a GP has the same topology the MLP shown in Fig. 2, it can perform arbitrary L 2) pesssge-ppsssing steps, not only 2. When > 2, this can be interpreted as the output neurons not producing an output immediately upon receiving the first message but rather waiting for further ones. We also provide neuron-level equivalent representation of equation (7), which is useful in subsequent discussions:
VI [0, L 1], vi E HUO,
Iz if (a 0, clsc.
h 1++1 = b, Σ Ayra a/t1 =hh(t)) where 2/ jEN
and [h/1,E0.
Where N = -UHUOO The strength of GP lies in its ability to enable perceptrons to work under any topology. crucial for building aele-asss neural networks, as neurons' dynamic growth and apoptosis lead to complex connectivity patterns.
3.2 Growth and apoptosis: Let GP assemble itself
This section introduces how to implement GP with full self-assembly capabilities, namely SAGP, SAGP is the first to realize a fully self-assembling neural network, capable of growing new neurons and achieving neuronal apoptosis and synaptic pruning. This functionality is realized by setting assembly rules and simulating competitive pressures, thus avoiding direct encoding of the genome.
Initial state The SAGP begins its development from the simplest state. GP always has fixed number of input and output neurons, but at this stage, no hidden neuron, i.e., [ 0. also includes all possible synapses from I to O, totaling [Z] x (0).
Neuron competition & synaptic competition During the development of the biological brain, neuron competition [33] and synaptic competition [34] play crucial role in acquiring cognitive and memory

---

774
SparseLoss, (10)
2.0
0:56
0.8
2 0.5
my
m
0.6
0.
m2
0.2
0.4
Strengthen
(b) Figure 4(b): The competition loss weakens most masks while only enhancing those with competitive advantage.
(a) Figure 4(a): Synapse-level mask (left) and neuron- level mask (right).
abilities. We simulate this process by learnable masks with competition loss. Two types of masks introduced in SAGP: neuron-level masks (denoted m²) and synapse-level masks (denoted as Maj. Fig. 4a). Noting that if hidden neuron has no outgoing synapses, will never affect the output neurons. Therefore, the node-level mask acts all the neuron's outgoing edges. The feature update of the GP can be rewritten as:
h = = + Σ JEN
my maj A.
All masks are learnable and can only take values of 0 or 1. It is achieved through two-step process: first, for learnable real number E R, we generate a soft mask m [0, 1] by Gumbel reparameterization [35]; then, we use the Straight-Throuuh Estimator (STE, [36]) to produce a hard mask m = m E {0,1) that suitable for back-propagation:
m Gumbel_Sigmold(in)
(10)
H Stop ( 0.5) - m m
(11)
m" !1
(12)
However, what truly enables the masks to make a competitive effect is what we refer to as the "Competition Loss":
CompLoss, (m²) = C(a)) -m² -m
(13)
Here, E (0, 1) is a temperature parameter, and C(a) ) is scaling factor designed ensure that CompLoss, has a maximum value of exactly over [0, 1]. As shown in Fig. 4b, SparseLoss weakens the less advantageous soft masks, driving them towards In contrast, only the masks that dominate in competition- i.c., those with larger values-ant enhanced, tending towards 1. By summing up the SparseLoss of all masks, we obtain the auxiliary loss:
AuxLoss
71 [74]
Σ CompLoss, (m,) iE
72
CompLoss, (m
I
174
(14)
VETUM
JEHUO
Yi and 7/2 are both weighting parameters. We achieve neuron competition and synaptic competition mechanism via proppaptpaaanionn by adding AuxLoss to downstream task loss.
Neuronal apoptosis Biological organisms streamline the structure of their nervous systems through synaptic pruning [37] and apoptosis [9]. Similarly, when training SAGP, we introduce the following rules to remove neurons from the network to ensure the efficiency of structure:
Apoptosis Rule: Remove hidden neuron i from set H and remove all synapses connected to i if m. < 8 MeanyeH
Here, 3 (0, 1). The soft mask of hidden neuron reflects its status in the competition. The Apoptosis Rule removes neurons that are relatively weaker within the overall population, as their influence the outcome is negligible, thereby reducing computational overhead.
Neuronal growth Organisms tend reproduce faster under reduced competitive pressure [38]. In SAGP, if neurons have been pruned over N consecutive training epochs, this may indicate that competition has sufficiently stabilized, with each neuron occupying a corresponding niche. At this point, add new hidden neurons increase competitive pressure:
5

---

Growth Rule: Add new neuron to H. and add all possible synapses connected to this new neuron to the network if no neurons are pruned for N consecutive epochs.
As the neural network trains, the number of hidden neurons dynamically increases or decreases based rules. Synapses neurons in the network may also become temporarily ineffective due to insufficient competitiveness (corresponding to a small mask value). See Appendix C.3 for implementation details.
Assembling dynamies We visualized the self- assembly process of SAGP (Fig. 5a and Fig. 5b) and its performance variations (Fig. 5c). The results show that the self-assembly undergoes three distinct stages: First, the network rapidly expands
Spiich 3.30
Spach -0003
its maximum size within short period (approximately 0 to 1000 epochs); next, intense competition occurs between neurons and synapses, with the majority of neuronal apopto- sis and synaptic pruning happening during this stage (approximately 1000 to 10000 epochs); finally, the network topology stabilizes, with the number of neurons remaining almost constant while the number of synapses slowly decreases (approximately 10000 to 50000 epochs). Even as the network size decreases, its performance keeps improving, indicating that the informa- tion density in the parameters is increasing. In- terestingly, this trend is similar to the changes in human cortical volume: the cortex rapidly reaches its maximum volume during childhood. then gradually decreases in size over a long maturation period, enhancing its functionality [39].
(a) Figure 5(a) We visualize the perceptron topology several time points, where only the hidden neurons and their connections are depicted, while input and output neurons are omitted.
Hideers
Actove vntteeen
Edges
15000
Acive Soges
10000
5000
10
20
10"
Epoch
(b) Figure 5(b) Results of training SAGP on the FSDD dataset. We first present the change in the number of active neurons and synapses during training, with apparent three-stage characteristic observed
101
10*
Irain Looa
Loos
202
Vol
Hyperparametes Several hyperparameters are introduced SAGP, including L, o, 71, 72, 3, and N. We provide set of empirical hyperpa- rameters, which are consistently applied across all experiments discussed in this paper (See Ap- pendix C.2). While hyperparameter tuning typi- cally yields better performance for specific tasks
10
10
Epoch
(c) Figure 5(c) Trends in the training and validation losses during the training of SAGP on the FSDD dataset.
or datasets, by fixing these parameters, we demonstrate the strong adaptability of SAGP in tackling complex environments.
Complexity Let the batch size be The computational complexity of one forward propagation in SAGP is O (BLS), where L and S represent the number of message-passing steps and the number of synapses, respectively. S can be further expressed as + + where n,, n22 and ng represent the numbers of input, hidden, and output neurons, respectively, and X denotes the density of the adjacency matrix. Therefore, the complexity of SAGP also expressed as O(ALL(m) + + n²)
Experiment
Overview In the experiment, we answer three questions: first, the impact of topological structure on the perceptron model; second, the performance of SAGP on deep learning tasks; and third, the inspiration that self-assembling neural networks provide for modern deep learning. Three domains of the dataset are used: text, audio, and images. We also conducted experiments on deep graph models and temporal models. See Appendix C.I for more dataset details.
4.1 Perceptron topology
Question 1: Is multi- multi--layered connection like MLP always the best perceptron topology?

---

Existing perceptron topology
(A) Multi-layer comnection (MLC)
(B) MLC + residual connection
(C) MLC Jordan's
(D) MLC + Ellan's s
Input neurons
connection
connection
New perceptron topology
Hidden neurons
Output neurons
(E) Complete connection
(F) Restricted complete
(G) MLC complete
(IT) MLC self connection
connection
connection
We conduct experiments 8 different topologies of graph perceptrons investigate the impact of neuron connectivity patterns. Topologies (A) to (D) are reported in existing literature, while (E) (H) new topologies designed by us.
Figure 7:
With fixed budget of 128 hidden neu- rons, investigated the performance of
Metric
(A)
(B)
(C)
(E)
(F)
(G)
Fl-mi.
.877
.870
.865
866
0OM
.876
.861
.873
8 different topologies for graph pereep- trons (Table 1). The results show that no single topology consistently outperforms others across different datasets and met- rics. This suggests that mechanisms like self-assembly, which dynamically adjust the perceptron topology, have the poten- tial to deliver better performance than fixed multi-layer topology. We select the FSDD dataset with the smallest perfor- mance gap for further investigation. We report the convergence speed of the GP (Fig. 6 left) and the impact of hidden neu- ron count on performance (Fig. 6 right). We found that the eight topologies can be divided into two categories: The first cat- egory includes topologies with strict hi- erarchical connection structures like (A), (C), (D), and (H), which converge faster but suffer from significant performance degradation low budgets. The second category consists of topologies with non- strict hierarchical connections like (B),
Pho.
Fl-ma.
843
.836
.28
.31
OOM
.833
.821
.838
AUC-mi.
9985
.986
.985
9985
0OM
987
.985
.985
AUC-ma.
.88
.988
.987
.977
0OM
.987
.987
Fi-mi.
7800
.780
.787
.787
0OM
780
.779
784
Com.
Lma.
.714
.710
.710
.155
OOM
.713
.714
.700
AUC-mi
1972
.971
973
9971
OOM
.971
.94
.973
AUC-ma.
978
.977
978
.976
OOM
.976
978
.976
Fi-mi.
.300
.348
.343
.338
.368
338
.322
ESC.
Fi-ma.
.317
.338
.327
324
.323
.352
.320
304
AUC-mi.
.886
.882
.878
886
.873
.877
.879
.892
AUC-ma.
.880
.875
.871
.877
.867
.873
.873
885
.934
.933
FSD.
Fi-ma.
.388
.936
.931
.344
.366
938
.39
.33
AUC-mi.
.995
.996
.995
.997
.997
.966
.997
.996
AUC-ma.
.966
.997
.996
.996
.996
.966
.997
.996
.825
855
Fas.
Fi-ma.
.800
.824
.849
.844
848
.817
.845
854
AUC-mi.
.889
.984
.988
.984
.989
.990
AtCC-ma
986
.979
.985
984
.985
.999
.985
.987
CIF.
Fl ma.
.461
.370
.456
.412
0OM
.365
.453
.446
AUC-mi.
.867
.677
.860
AUC-ma.
8665
.796
.866
.829
OOM
.793
8864
.858
Table 1: The performance of graph perceptrons with differ- ent topologies when the number of hidden neurons is fixed 128. OOM indicates out of memory. The best results are highlighted.
(E), (F), and (G), which converge slower but perform well even with small number of hidden neurons. Thus, another potential advantage of general-topology perceptrons is that they may achieve performance comparable to multilayer-comnected ones with much smaller size. However, multi- layer perceptrons have computational efficiency advantage since they can be implemented using straightforward matrix multiplications rather than message-passingg mechanisms.
10
10
124757
100.
Answer 1: MLPs strike excellent balance between performance and cost, but GPs can achieve the same even higher performance with more streamlined topology.
Doech 10
107
102
182
850
Figure The graph perceptron's convergence speed (left) and its sensitivity to the number of hidden neurons (right).
7

---

Scale
Metrie
[74]
#Synapse
Fl-micro (Accuracy)
F1-macro
AUC-mi
AUC-macr
MLP-b
512
585536
8687
6314
9651
9867,
Photo
SAGP-
0.20(0.0453)
4196(1 (0%)
870 eo 200r
(000.00)
8354, mas (800448))
9656 ( 0530)
9881 (00.00000.555
SAGP-]
0.00(0.0055)
4040[1 (55%)
La (90. (90.82%)
8335 wur ( (300.258)
9855 noau (100.045))
9882 s000 (100.15%)
MLP-b
1024
659968
7520 cocs
7073, o02s
9696 2000
9769 oonc
Computers
SAGP.b
0.006
4376000060%
7G57 beca (98, 085305
1986 ous (98, 77%)
9657 a.ooaa (99)60)))
4F122 ooss (50 85%)
SAGP4
0.0060.005))
3810(0
aocu 1981155)
(98 80%)
soast (99.6)%)
9752 wo00 (59
MLP
1024
3257 tomo
2993 0103
0036
50
SAGPb
98.20(9.509%)
20874/6 189))
3150
o150 (96.770) 76676
3029
(10 54%%
8716 oven (98, 06%)
8060
oean 84%)
SAGPJ
106.0(00-118)
11251/2900))
2438 ossal (104.28%)
3306 orse (110 (1.042%) 42%)
S711 pooss (98.01%)
8019 oap (97 94%)
MLP-h
2048
883712
9467, 1003
9473 usss
tosss
1976 0084
SAGP-b
55. 65.1012.6000 60%)
6246(0.7096
9142g 011 5756)
a1 40 (966))9))
9025 coief 46%)
8823,
oma 48%)
SAGP4
66.00(2.60%)
4312().47%)
9120
onca (6..2 99563
3132 man (9G-40%)
0017 pour (90. (99.38%)
2019 om (00 44%)
Fasion
MLP-11
1024
6686772
8968 1000
8651
9894
MNIST
SAGP-]
2.00().265))
3247/0 49%%
was (9s 02%)
1000 (99.15%)
ond 23%)
CIFAR-10
SAGP-1
MLP-1
768
65.6018.5-9))
90574/9.865)
920064
5157
5201
poss
5168 0017
5005
(98. 50%)
8018 (00.0000000000000000
a
(90.
8941 0013
8880 ooss (90 32%)
Table 2: A comprehensive comparison between SAGP and MLP. We report the network scale after training and model performance under 4 metrics. "-I" refers to the performance on the last epoch and "-b" refers to the performance on the best epoch.
4.2 Self-assembling graph perceptrons
Question 2: What are the advantages of a graph perceptron with full self-assembly capacity?
We compared two perceptron models: SAGP and MLP (Table 2). The only previous self-assembling model NDP was excluded from the comparison due to resource constraints (a single train- ing run exceeding 12 hours on a single Nvidia RTX 4090). The suffix "-b" in- dicates the epoch with the best perfor- mance during training, i.e., the epoch that achieves the lowest loss on the val- idation set. The suffix "-I" refers to the last training epoch when convergence is reached. MIPs report results from the best epoch by default unless the dataset lacks a validation set. SAGP-b exhibits better performance, while SAGP-I has a smaller topology size. Compared to MLP, SAGP uses only 0% ~ 10.41% of hidden neurons and 0.49% 9.86% of
[H]]
Average time per epoch (s) Accuracy (%)
NDP
48
1.58
93.0 2.9
SAGP
1.36 10
99.8
Table 3: Following the same settings as in [7], we fairly compare NDP and SAGP on the toy dataset Digit [7].
[H]
#Synapse
Fl-micro
GAMLP-b
512
385536
.9284
Pho.
GAMLP4SAGP-b
33.6018.52%0
23991 (6.0))66622 (6.22%)
9236 (9. 48%)
GAMI.P-SAGPI
25.50(4.98%)
538 5381(1140%%
91164 (1%))
GAMLP-b
1024
659968
85516
Com.
PSAAPPb
19.30(1.886)
13259/2.01 %%)
.6200 (00..9%)
GAMLP4SAGPM
0.70(0.07%)
526 526.3(0.08%)
.8527 (99.78%)
L.TM-h
1024
379904
.3321
ESC.
LSTM+SAGP.b
117.2(11.4%)
28624
3350(100.9%%
LSTM+SAGP-1
117.2(11.4%)
(8.39)7
31115(103.7%)
LSTM-b
2018
883712
.9513
FSD.
LSTM+SAGP.h
81.4(3.97%)
(1.81().64%))
.9885
LSTM+SAGP-I
68.7(3.35%)
(0652(..21%)
9200596.46%%
TeNet-1
1024
398336
.8979
synapses, achieving performance rang- ing from 96% ~ 110.42%. Following the setup in [7]. we compared SAGP and NDP on toy dataset, Digit (Table
Fas.
LeNe++AGP-1
25.5(2.49%)
5853(1.47%)
88633 (96.5)))
LeNet-l
768
438784
6521
CIF.
LeNe+SAGP1
25.8(3.36%)
26362(6.05%%)
6650 (10.00))
Table 4: We replaced the MLP as the nonlinear transfor- mation layers of several classic models with SAGP and evaluated its performance.
3). We found that SAGP's average run- time per epoch improved by over 10,000 times. Due to neuronal apoptosis, SAGP eventually removed all hidden neurons, achieving significantly better performance with only the synapses between input and output neurons. We also experimented with the potential of SAGP as a submodule in other models (Table 4). Specifically, we replaced the MLP used for generating classification outputs in GAMLP [40], LSTM [41], and LeNet [42] with SAGP. The results show that SAGP can achieve comparable performance while maintaining a significantly simplified topology. Notably, reinforcement learning-trained NDP cannot be integrated into these models.
Answer SAGP achieves performance comparable to MLP with a smaller topology,
while being far more efficient and flexible than NDP.
8

---

4.3 Inspiration for deep learning
Question 3: In what fields SAGP show its potential?
Model Interpretability A significant challenge of modern deep learning models is the lack of interpretability. We visualize the out-degree of each pixel (input neuron) in the SAGP for CIFAR- 10. We found that the pixels with high degrees are concentrated in the image's 's central region, where the image's 's main subject typically occupies. When selecting the top 50% of pixels by degree, the central semantics of the image are still preserved (fig. 8). This suggests that even purely perceptron-basee based model like SAGP might exhibit some clues about the reasoning behind its judgments, similar to how humans respond.
Figure 8: In the CIFAR-10 dataset, each pixel is treated as an input neuron. We visualize the out- degree of all pixels (left) in SAGP and display the image in which only the top 50% of the pixels with the largest out-degrees are retained (right).
Backbone
Ratio
XGBoost
LassoNet
GradEnFSS
Ours
5%
.7794 0064
8800 +.00+.0002
.841 2.000
.83455.0000
GCN
10%
.7879 .120
8435
82362.000
8450
20%
.270 0043
8543 0054
8451 0085
8529 0059
5%
8137
0094
8473 .007
8233 0070
84100.0002
GAT
10%
8305
8180
8839
.8577 +.007
20%
8407
.86 186194.00
.8627
.8500
Table 5: We selected the top 5%, 10%, and 20% of features based on the out-degree as subsets, and compared them with state-of-the-art feature selection methods on the Computers dataset. The numbers in the table represent the accuracy of the semi-supervised classification task based on the selected features.
Feature selection Sometimes we want to deter- mine which data preprocessing method is more effective or select the most critical subset from many features to reduce computation. These issues are collectively referred to as feature se- lection problems. Our experiments show that after training with SAGP, the out-degree of in- put features (input neurons) can indicate feature importance. In the FSDD audio dataset, find that Mel-Frequency Cepstral Coefficients (MFCC) and Mel frequency spectrogram (Mel) features are much more effective than chroma features (Fig. 9). On the Computers datasets, selected the top 5%, 10%, and 20% of features based on their out-degree, and tested them with backbone models GCN [43] and GAT [44]. The classical non-deep eue-suupee feature selection method XGBoost [45], well as the state-of- the-art deep semi-supervised feature selection methods LassoNet [46] and GradEnFS [47], were used for comparison (Table 5). The results show that SAGP significantly outperforms XGBoost and GradEnFS, and is also competitive with LassoNet, despite SAGP not being specifically designed for feature selection tasks.
26400
When training SAGP on the FSDD dataset, we present the out-degree statisties for three types of input features (left), and the results of training an MI. with different combinations of input features (right).
Figure
Answer 3: The out-degree of input neurons of a well-trained SAGP can be used in areas such as model interpretability and feature selection.
5 Conclusion
We introduced SAGP, a graph-structun perceptron model with full self-assembly capabilities inspired by the growth process of the human brain. SAGP optimizes its topology and enhances its functionality through neuron growth, competition, and apoptosis. We highlighted the advantages of SAGP in terms of structural simplification and assembly speed and explored its potential in interpretability and feature selection.

---

Acknowledgments
The research supported by the National Key R&D Program of China (2023YFB2703700, 1012YFA0004800 2025YFF1207900. 2025YFC3409300), and the National Natural Science Foun- dation China (62176269, T2350003, T2341007, 12131020, 42450084, 42450135, 12326614, and 12426310)
References
[1] Mualem, R., L. Morales-Quezatee R. H. Farraj, et al. Econeurobiology and brain development in children: key factors affecting development, behavioral outcomes, and school interventions. Frontiers in Public Health, 12:13 12:1376075, 2024.
[2] Hong, R., A. Kratsios. Bridging the gap between approximation and learning via optimal approximation relu mips of maximal regularity. arXiv preprint arXiv:2409.22335, 2024.
[3] Yu, Y., Zhang. Multi-layer perceptron trainability explained via variability. arXiv preprint arXiv:2105.08911, 2021.
[4] Cook, S. T. Jarrell, C. A. Brittin, et al. Whole-animal connectomes both caenorhabditis elegans sexes. Nature, 571(7763):63-71,, 2019.
[5] Sandler, M., M. Vladymyrov, A. Zhmoginov, al. Meta-learning bidirectional update rules. In International Conference on Machine Learning, pages 9288-9300. PMLR, 2021.
[6] Kirsch, L., Schmidhuber. Meta learning backpropagation and improving it. Advances in Neural Information Processing Systems, 34:14122-14134, 2021.
[7] Najarro, E., S. Sudhakaran, S. Risi. Towards self-assembling artificial neural networks through neural developmental programs. In Artificial Life Conference Proceedings 35, vol. 2023, page 80, MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journalsiiffo ... 2023.
[8] Plantec, E., J. W. Pedersen, M. L. Montero, et al. Evolving self-assembling neural networks: From spontaneous activity to experience-dependee learning. In ALIFE 2024: Proceedings of the 2024 Artificial Life Conference. MIT Press, 2024.
[9] Yuan, J., B. A. Yankner. Apoptosis in the nervous system. Nature, 407 809, 2000.
[10] Dorigo, M., V. Maniezzo, A. Colori.. Ant system: optimization by a colony of cooperating agents. IEEE transactions systems, man, and cybernetics, part (cybernetics), 26(1):29-41, 1996.
[11] Katoch, S., S. Chauhan, V. Kumar. A review genetic algorithm: past, present, and future. Multimedia tools and applications, 80:8091-8126, 2021.
[12] Rosenblatt, F. The perceptron: probabilistic model for information storage and organization the brain. Psychological review, 65(6))386, 1958.
[13] Nunes. J. D., M. Carvalho, D. Carneiro, et al. Spiking neural networks: A survey. IEEE Access, 10:60738-60764, 2022.
[14] Yi, 7., J. Lian, Q. Liu, et al. Learning rules in spiking neural networks: A survey. Neurocomput- ing, 531:163-179, 2023.
[15] Hasani, R., M. Lechner, A. Amini, et al. Liquid time-constant networks. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pages 7657-7666. 2021.
[16] Mouret, J.-B., P. Tonelli. Artificial evolution plastic neural networks: a few key concepts. In Growing Adaptive Machines: combining Development and Learning in Artificial Neural Networks, pages 251-261. Springer, 2014.
[17] Hebb, D. The organization of behavior. neuropsychological theory. 1949.
10

---

[18] Markram, H., J. Lübke, M. Frotscher, et al. Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps. Science, 275 1997.
[19] Miconi, T., K. Stanley, J. Clune. Differentiable plasticity: training plastic neural networks with backpropagation. In International Conference on Machine Learning pages 3559- PMLR, 2018.
[20] Miconi, T. A. Rawal, J. Clune, et al. Backpropamine: training self modifying neural networks with differentiable neuromodulated plasticity. arXiv preprint arXiv:2002.10588 2020.
[21] Schmidhuber, J. A 'self-referential weiggt matrix. In ICANN'93: Proceedings of the Interna- tional Conference Artificial Neural Networks Amsterdam, The Netherlands 13-16 September 1993 3, pages 446-450. Springer, 1993.
[22] Li, Y. S. Ji. Neural plasticity networks. In 202/ International Joint Conference on Neural Networks (IJCNN), pages 1 1-9. IEEE, 2021.
[23] Paik, I., S. Oh, T. Kwak, et al. Overcoming catastrophic forgetting by neuron- plasticity control. In Proceedings of the AAAI Conference Artificial Intelligence, vol. 34, pages 3339-5346. 2020.
[24] Stanley, K. O., R. Mikkkulaienn Evolving neural networks through augmenting topologies. Evolutionary computation, 10(2):99-177, 2002.
[25] Stanley, K. O., D. B. D' D'Ambosii,, J. Gauci. A hypercube- based encoding for evolving large- scale neural networks. Artificial life, 15(2):1552112, 2009.
[26] Han, B., F. Zhao, Y. Zeng, al. Developmental cussicity-inssiiee adaptive pruning for deep spiking and artificial neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[27] Smolensky. P., et al. Information processing in dynamical systems: Foundations of harmony theory. 1986.
[28] Werbos, P. Beyond regression: New tools for prediction and analysis in the behavioral sciences. PhD thesis, Committee on Applied Mathematics, Harvard University, Cambridge, MA, 1974.
[29] Rumelhart, D. E., G. E. Hinton, R. J. Williams. Learning representations by back pack-propagating errors. nature, (288))8 1986.
[30] Hopfield, J. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 79(8):2554-2558, 1982.
[31] Hinton, G. E., J. Sejnowski, et al. Learning and relearning in boltzmann machines. Parallel distributed processing: Explorations in the microstructure cognition, 1(282-117)2, 1986.
[32] Gilmer, J., S. S. Schoenholz, P. F. Riley, et al. Neural message passing for quantum chemistry. In International conference on machine learning, pages 1263-1272. PMLR, 2017.
[33] Han, J.-H., S. A. Kushner, A. P. Yiu, et al. Neuronal competition and selection during memory formation. science, 31 31665823)457-4660, 2007.
[34] Ramiro-Cortés, Y., A. F. Hobbiss, I. Israely. Synaptic competition in structural plasticity and cognitive function. Philosophical Transactions of the Royal Society B: Biological Sciences, (B0(1633):20130555 2014.
[35] Jang, E., S. Gu, B. Poole. Categorical reparameteriization with gumbel-softmax. arXiv preprint arXiv:1611.01144. 2016.
[36] Bengio, Y., N. Léonard, A. Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.344. 2013.
[37] Paolicelli, R. C., G. Bolasco, F. Pagani, et Synaptic pruning by microglia is necessary for normal brain development. science, 14046))))144614444 2011.

---

[38] Cuda, J., H. Skálová, Z. Janovský, et al. Competition among native and invasive Impatiens species: the roles of environmental factors, population density and life stage. AoB PLANTS, 7:plv033, 2015.
[39] Bethlehem, R. A., J. Seidlitz, S. R. White, et al. Brain charts for the human lifespan. Nature, 604(7906):525-5555 553, 2022.
[40] Chen, L.,, Z. Chen, J. Bruna. On graph neural networks versus graph-augmented mlps. CoRR, 1Ds/2000.15116, 2020.
[41] Hochreiter, S., Schmidhuber. Long short-term memory. Neural Computation, 9 1997.
[42] LeCun, Y., L. Bottou. Y. Bengio, et al. Gradient- based learning applied to document recognition. Proc. IEEE, 86(1 1):22)8-2322 1998.
[43] Kipt, T., M. Welling. Semi-supervised classification with graph convolutional networks. ArXiv, abs/1609.02907, 2016.
[44] Velickovic, P., G. Cucurull, A. Casanova, et al. Graph attention networks. ArXiv, abs/1710.10903, 2017.
[45] Chen, T., C. Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785-794. 2016.
[46] Lemhadri, I., F. Ruan, L. Abraham, et al. Lassonet: A neural network with feature sparsity. Journal of Machine Learning Research, 22(127):1-29, 2021.
[47] Liu, K., Atashgahi, G. Sokar, et al. Supervised feature selection via ensemble gradient information from sparse neural networks. In International Conference on Artificial Intelligence and Statistics, pages 3952-3960. PMLR, 2024.
12

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
We made our main claims in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations are provided in Appendix D.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
13

---

Justification:
Proofs are provided in Appendix A.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All important implementation details have been presented the main text and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer: [Yes]
Justification:
Appendix contains links to the code repository.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ //nips. wttpccccpppccccceeececceccccccct /ttpa//ggiiieeeeeeeuuuubiiiiiiii
/CodeSubmie ssionnollll onPolicy))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Appendix contains experimental setting and details. Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:[Yes)
Justification:
We repeated the experiment several times.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
15

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Main text discuss compute resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttp///eerrp//e//ee//eee/ htcppullic/tthiccuiieeeeeeo
Answer: [Yes]
Justification: This paper complies with the NeurlPS Code Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Appendix contains broader impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
16

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA].
Justification:
This article does not contain any data with high risk of abuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: All assets in this article are original.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
17

---

Answer: [Yes]
Justification:
The new dataset introduced in this article has detailed instructions for use.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper did not use human crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects were involved in this article.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
18

---

Justification:
This article only performs question answering and fine-tuning on LLM.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy (https://neuul httrrrpp...cc/ooneeeeeee//////// for what should or should not be described.
:
19


Reviews:
[{"review_id": "wcbLa4BizH", "reviewer": "Reviewer_v1ys", "summary": "This paper introduces SAGP, a bio-inspired architecture that generalizes the classic MLP into a graph-structured neural network with dynamic topology. Unlike traditional MLPs with fixed layers and widths, SAGP features neuron growth, synaptic pruning, and apoptosis, enabling it to self-organize from a minimal initial state.", "questions": "1. Can you provide more comprehensive or systematic analysis of training cost?\n2. How does SAGP compared to spiking neural network (SNN) in interms of efficiency? SAGP must be trained from scratch, while SNN, another brain-inspired efficient model can be distilled or converted from pretrained ANNs, enabling much faster convergence and practical reuse.", "limitations": "I would suggest to conduct more applicable experimental settings on larger datasets with training cost comparison.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Pros:\n1. The idea of reformulation of MLPs as graph perceptrons is intersting. \n2. The authors derive a clean algebraic formulation for MLPs using message passing on adjacency matrices. This lays a principled foundation for extending the MLP to non-layered, dynamic topologies.\n3. The paper provides quantitative evidence of efficiency during inference. For example, on the FSDD dataset, SAGP achieves >99% of MLP performance using only ~0.7% of the synapses.\n\nCons:\n1. All benchmarks are limited to small or medium datasets. The model is not tested on large-scale tasks (e.g., ImageNet), so scalability and training stability in such settings remain unverified.\n2. SAGP lacks mechanisms to initialize or transfer topology across tasks (for the potential of building foundation model). Each run builds a new network from a minimal seed, limiting its applicability in real-world multitask or continual learning scenarios, and making it vulnerable to instability or variance across runs.", "was_revised": true, "final_justification": "I have raised my confidence as I gave positive score in the review.", "rebuttal": "We appreciate the reviewers' diligent efforts in providing thoughtful comments. Below is our response.\n\n# Cons 1 & Cons 2\nWe would like to answer both cons simultaneously. First, we apologize that, given our limited time and resources, it is difficult for us to train a SAGP (or any other model containing SAGP) from scratch on the ImageNet dataset of 10 million images. However, we would like to emphasize that **SAGP can actually be initialized in non-minimal seed topologies.**  Note that the growth/death mechanism of neurons and synapses described in the paper is actually completely independent of the initialization. Therefore, we can initialize SAGP with any topology and weights, which is no difference from initializing with a minimum seed. Furthermore, we conducted the following experiments in the hope of addressing your concerns about both cons 1 and 2.\n\nFirst, we used the pre-trained weights of MobileNetV2 on ImageNet-1K, available in PyTorch. We then froze the network's backbone weights and used the weights of the final fully-connected layer as the initialization for the SAGP. (Specifically, the SAGP was initialized to a network with 1280 input neurons, 1000 output neurons, and 0 hidden neurons. The synaptic weights between each input-output neuron pair remained the same as those in the corresponding fully-connected layer of the original network.) We fixed the batch size to 128 and continued training for 100 full epochs (which required about one million backpropagations). The results are as follows:\n\n|                  | #Synapse | ACC   |\n| ---------------- | -------- | ----- |\n| MobileNetV2      | 1280000  | 71.8% |\n| MobileNetV2+SAGP | 435024   | 68.2% |\n\nDue to time constraints, we did not perform more epochs. However, according to our observations, performing more rounds would further reduce the number of synapse in SAGP. which is consistent with our conclusions in the original paper.\n\n# Question 1\nOf course, we now provide a theoretical and experimental analysis of the training cost of SAGP, and a detailed comparison with MLP.\n\nFor simplicity, we ignore the activation function and bias in MLP/SAGP because they are not the main factors affecting time/memory cost. At this point, a two-layer MLP can be expressed as:\n\n$$\n\\boldsymbol{z}=\\boldsymbol{W}^T_2\\boldsymbol{W}^T_1\\boldsymbol{x}\n$$\n\nWhere $\\boldsymbol{x} \\in \\mathbb R^{|\\mathcal I|}$,  $\\boldsymbol{W}_1^T \\in \\mathbb R^{|\\mathcal H| \\times |\\mathcal I|}$,$\\boldsymbol{W}_2^T \\in \\mathbb R^{|\\mathcal O| \\times |\\mathcal H|}$ and $\\boldsymbol{z} \\in \\mathbb R^{|\\mathcal O|}$。Recall that the time complexity of multiplying a $(n\\times m)$ matrix by a $(m \\times p)$matrix is $\\Theta(nmp)$ and the space complexity is $\\Theta(1)$. Now, we multiply $\\boldsymbol{W}_1^T$ by $\\boldsymbol{x}$, and then multiply the result by $\\boldsymbol{W}_2^T$. \n\nAfter these two steps, the time complexity is $\\Theta(|\\mathcal H|\\times|\\mathcal O| + |\\mathcal H|\\times|\\mathcal I|)$, and the space complexity is $\\Theta(1)$ (We only consider the overhead beyond input and output). Note that the total number of synapses $S_{mlp}$ in this MLP is also $|\\mathcal H|\\times|\\mathcal O| + |\\mathcal H|\\times|\\mathcal I|$, so the time complexity can also be expressed as $\\Theta(S_{mlp})$.\n\nThen we consider GP. For a GP that has undergone two message-passing, it can be simplified as:\n\n$$\n\\boldsymbol{z} = \\left((\\boldsymbol{A}^T)^2[ \\boldsymbol{x} \\\\ 0_{|\\mathcal H| + |\\mathcal O|}]^T  \\right)_{[-|\\mathcal O|:]}\n$$\n\n\n\nThe definition of $\\boldsymbol A$ is consistent with that of Formula 6 in the paper. GP use sparse matrix multiplication in its implementation, which in **once sparse matrix multiplication**, the time complexity and space complexity are both $\\Theta(S)$, where $S$ is the number of non-zero elements in $\\boldsymbol{A}$. Note that the number of non-zero elements of $\\boldsymbol{A}$ is also equal to the number of synapses in the GP ($S_{gp}$), so for a GP with two message-passings, the time complexity and space complexity are $\\Theta(2S_{gp})$ and $\\Theta(S_{gp})$ respectively.\n\nThe above conclusions can be extended to any layer of $L$-layer MLP or any $L$-times message-passing GP. Now, we Let the batch size is $B$, and the MLP and GP training epochs are $k_{mlp}$ and $k_{gp}$ respectively, then the total training complexity comparison is:\n\n|      | Time                        | Space               |\n| ---- | --------------------------- | ------------------- |\n| MLP  | $$\\Theta(k_{mlp}BS_{mlp})$$ | $$\\Theta(B)$$       |\n| GP   | $$\\Theta(k_{gp}BLS_{gp})$$  | $$\\Theta(BS_{gp})$$ |\n\n**Here, we would like to ask you to pay special attention to one detail**: in the table, it seems that the time complexity of GP is one factor $L$ higher than that of MLP, but in fact, when $\\Theta(k_{mlp})=\\Theta(k_{gp})$, the complexity of the two is equivalent. This is because $S_{mlp}$ increases linearly with $L$, but $S_{gp}$ is constant with $L$. In other words, $\\Theta(S_{mlp})=\\Theta(LS_{gp})$.\n\nFor the complexity of SAGP, we only need to replace $S_{gp}$ in SP with $\\max S_{sagp}$.\n\nWe then give the following observation in our experiment:\n\n|                   | Photo  | Computers | ESC-50 | FSDD   | Fashion-MNIST | CIFAR-10 |\n| ----------------- | ------ | --------- | ------ | ------ | ------------- | -------- |\n| $S_{mlp}$         | 385536 | 659968    | 379904 | 883712 | 668672        | 920064   |\n| $$L\\times\\max S_{sagp}$$ | 30126  | 31257     | 66519  | 47565  | 184572        | 625923   |\n\nBased on the above analysis, we believe that, **theoretically, the (average) single-epoch training time complexity of SAGP is consistent with that of MLP. However, in practice, SAGP's higher time cost stems from the use of more epochs, while its higher space cost stems from the use of sparse matrix multiplication, which increases linearly with $S_{gp}$.**\n\nNext, we report the experimental results\n\n|                             | Photo         | Computers     | ESC-50        | FSDD          | Fashion-MNIST(batch_size=128) | CIFAR-10 (batch_size=128) |\n| --------------------------- | ------------- | ------------- | ------------- | ------------- | ----------------------------- | ------------------------- |\n| Training epoch (MLP)        | 1000          | 1000          | 1000          | 1000          | 1000                          | 1000                      |\n| Training epoch (SAGP)       | 50000         | 50000         | 50000         | 50000         | 100000                        | 100000                    |\n| Ratio                       | $\\times$50    | $\\times$50    | $\\times$50    | $\\times$50    | $\\times$100                   | $\\times$100               |\n| Training Time (MLP)         | 11s           | 16s           | 12s           | 13s           | 39s                           | 43s                       |\n| Training Time (SAGP)        | 754s          | 748s          | 634s          | 674s          | 3705s                         | 4031s                     |\n| Ratio                       | $\\times$68.54 | $\\times$46.75 | $\\times$52.87 | $\\times$56.24 | $\\times$95.00                 | $\\times$93.74             |\n| Training Peak memory (MLP)  | 734 Mb        | 636 Mb        | 651 Mb        | 898 Mb        | 512 Mb                        | 510 Mb                    |\n| Training Peak memory (SAGP) | 7020 Mb       | 14610 Mb      | 4994Mb        | 7833Mb        | 14189 Mb                      | 15530 Mb                  |\n| Ratio                       | $\\times$9.56  | $\\times$22.97 | $\\times$7.67  | $\\times$8.27  | $\\times$27.71                 | $\\times$30.45             |\n\nPlease note that the number of SAGP training epochs shown in this table is not strictly necessary. In fact, reducing this number of epochs to 20% does not significantly reduce the model accuracy. We set this large number of epochs to explore the limits of synapse compression.\n\nWe also acknowledge in the limitation section in the appendix that improving training efficiency is one of the core directions for optimizing SAGP. In addition, we discussed some methods for reducing the training complexity of SAGP in **sections 1.1 to 1.3 of our response to reviewer 4y96**. If you are interested, you are welcome to read them further.\n\n# Question 2\n\nSNN and self-assembling neural networks are both biologically inspired architectures, but their efficiency advantages are actually reflected in different aspects. SNN focuses on reducing energy consumption. Its core idea is to reduce unnecessary neuronal activities by realizing asynchronous triggering of neurons based on events, thereby reducing power consumption. The core of self-assembling networks lies in optimizing the connections between neurons to achieve a sparser topology. In other words, the efficiency of SAGP is reflected in the significant reduction in the number of parameters. Therefore, there is no strong comparison between the two.\n\nIn addition, as we mentioned above, SAGP can actually use any MLP module in the trained model as its initialization and further train it. And this can indeed accelerate the convergence of the model. In particular, SAGP may be more suitable for continuous learning than MLP or SNN, because SAGP can learn new content by growing new neurons, thereby reducing the modification of the original weights.\n\nTo verify this, we also added the following experiments: First, we split the 10 classes in the Fashion-Mnist dataset into two parts. We initially trained SAGP on data from only the first five classes. After the topology stabilized, we restarted the self-assembly process on data from the last five classes. At this point, we have the following results:\n\n|                           | MLP            | SAGP            |\n| ------------------------- | -------------- | --------------- |\n| ACC on first five classes | 90.7% -> 43.6% | 89.2% -> %61.7% |"}, {"review_id": "8JeWdvMtfl", "reviewer": "Reviewer_4y96", "summary": "Inspired by flexible and evolving structures in biological neural network, the authors propose a novel perceptron model, SAGP. Proposed architecture GP generalizes perceptron models, and during training, SAGP optimizes proposed loss function to learn perceptron structure. In multiple classification datasets, SAGP achieves similar accuracies to those achieved by MLP, but with much sparser neural connections. Authors further demonstrate SAGP is useful for interpretability and feature selection.", "questions": "See weaknesses.", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "The **key strengths** include:\n\n- [S1 originality]. The proposed method SAGP is original. The choice to generalize perceptron models is innovative, and the architecture design is simple and neat.\n- [S2 comprehensive exp.]. Analysis and experiments were comprehensive. Some results are reported in Appendix to demonstrate the consistency of the findings across different datasets. Feature selection and interpretability experiments further demonstrate usefulness of the proposed method.\n- [S3 theoretical property]. Proving that MLP is a special case of GP is important (Theorem 1). Besides, mathematical derivation of GP from MLP is creative.\n- [S4 clear presentation]. The presentation is highly effective. Figures clearly convey their messages. Mathematics are well explained. The argument structure in Experiment section is clearly guided.\n\nThese strengths clearly set up the paper for a top-tier conference. However, I lean toward only borderline accept since SAGP seems to have **very low applicability with two primary limitations**. \n\n- [W1 complexity]. While SAGP’s ‘inference’ speed and memory may be comparable to those in MLP due to learned sparsity (Appendix Sec 5.1), they are substantially larger for model training. Since the complexity is square of input, hidden, and output neurons, SAGP cannot scale to larger dimensions. Indeed, SAGP reportedly becomes very dense during training (Fig. 5a), which would limit its application.\n- [W2 accuracy]. Since the reported SAGP accuracies are comparable to those of MLP, they do not justify SAGP’s complexity handicap. Thus, as a user, the present work does not convincingly demonstrate empirical advantages of using SAGP.\n\nI am willing to raise my score to accept, if either W1 or W2 is somewhat resolved during rebuttal. In light of that, I would like to make few **suggestions for future reference**.\n\n- [suggestion 1 - faster variant of SAGP]. Have the authors considered designing a faster variant of SAGP to mitigate the complexity issue? Naively, I would think restricting synapse space may significantly reduce complexity.\n- [suggestion 2 - self-supervision tasks]. Have the authors tested SAGP for self-supervision tasks? Current performance metrics are all based on classification. Since SAGP generalizes MLP, I speculate that the autoencoder with SAGP encoder and decoder may substantially outperform an autoencoder with MLPs. Even if SAGP does not outperform, given the importance of self-supervision tasks in the deep learning today, testing SAGP for self-supervision may improve the present work.", "was_revised": true, "final_justification": "This is a strong paper with clear presentation, rigor, originality, and remarkable performance. \n\nConcerns have been raised primarily about the method's complexity. However, the authors satisfactorily resolved them with theoretical proofs, empirical demonstration, and an updated version of the method that reduces complexity. \n\nIn my view, the only small weakness of the paper is the use of only classification benchmarks to evaluate the proposed method's performance. This weakness is understandable, given the other advantages that the proposed method has.", "rebuttal": "We are grateful for your professional comments and recognition of our work. We would like to respond respectfully to the weaknesses you pointed out as follows.\n\n# 1. [Weakness1: complexity]\nTo ensure a proper understanding of this weakness, we would like to reanalyze the theoretical complexity of MLP and SAGP before formally answering it. \n\nFor simplicity, we ignore the activation function and bias in MLP/SAGP because they are not the main factors affecting time/memory cost. At this point, a two-layer MLP can be expressed as:\n\n$$\n\\boldsymbol{z}=\\boldsymbol{W}^T_2\\boldsymbol{W}^T_1\\boldsymbol{x}\n$$\n\nWhere $\\boldsymbol{x} \\in \\mathbb R^{|\\mathcal I|}$,  $\\boldsymbol{W}_1^T \\in \\mathbb R^{|\\mathcal H| \\times |\\mathcal I|}$,$\\boldsymbol{W}_2^T \\in \\mathbb R^{|\\mathcal O| \\times |\\mathcal H|}$ and $\\boldsymbol{z} \\in \\mathbb R^{|\\mathcal O|}$。Recall that the time complexity of multiplying a $(n\\times m)$ matrix by a $(m \\times p)$matrix is $\\Theta(nmp)$ and the space complexity is $\\Theta(1)$. Now, we multiply $\\boldsymbol{W}_1^T$ by $\\boldsymbol{x}$, and then multiply the result by $\\boldsymbol{W}_2^T$. \n\nAfter these two steps, the time complexity is $\\Theta(|\\mathcal H|\\times|\\mathcal O| + |\\mathcal H|\\times|\\mathcal I|)$, and the space complexity is $\\Theta(1)$ (We only consider the overhead beyond input and output). Note that the total number of synapses $S_{mlp}$ in this MLP is also $|\\mathcal H|\\times|\\mathcal O| + |\\mathcal H|\\times|\\mathcal I|$, so the time complexity can also be expressed as $\\Theta(S_{mlp})$.\n\nThen we consider GP. For a GP that has undergone two message-passing, it can be simplified as:\n\n$$\n\\boldsymbol{z} = \\left((\\boldsymbol{A}^T)^2[ \\boldsymbol{x} \\\\ 0_{|\\mathcal H| + |\\mathcal O|}]^T  \\right)_{[-|\\mathcal O|:]}\n$$\n\n\n\nThe definition of $\\boldsymbol A$ is consistent with that of Formula 6 in the paper. GP use sparse matrix multiplication in its implementation, which in **once sparse matrix multiplication**, the time complexity and space complexity are both $\\Theta(S)$, where $S$ is the number of non-zero elements in $\\boldsymbol{A}$. Note that the number of non-zero elements of $\\boldsymbol{A}$ is also equal to the number of synapses in the GP ($S_{gp}$), so for a GP with two message-passings, the time complexity and space complexity are $\\Theta(2S_{gp})$ and $\\Theta(S_{gp})$ respectively.\n\nThe above conclusions can be extended to any layer of $L$-layer MLP or any $L$-times message-passing GP. Now, we Let the batch size is $B$, and the MLP and GP training epochs are $k_{mlp}$ and $k_{gp}$ respectively, then the total training complexity comparison is:\n\n|      | Time                        | Space               |\n| ---- | --------------------------- | ------------------- |\n| MLP  | $$\\Theta(k_{mlp}BS_{mlp})$$ | $$\\Theta(B)$$       |\n| GP   | $$\\Theta(k_{gp}BLS_{gp})$$  | $$\\Theta(BS_{gp})$$ |\n\n**Here, we would like to ask you to pay special attention to one detail**: in the table, it seems that the time complexity of GP is one factor $L$ higher than that of MLP, but in fact, when $\\Theta(k_{mlp})=\\Theta(k_{gp})$, the complexity of the two is equivalent. This is because $S_{mlp}$ increases linearly with $L$, but $S_{gp}$ is constant with $L$. In other words, $\\Theta(S_{mlp})=\\Theta(LS_{gp})$.\n\nFor the complexity of SAGP, we only need to replace $S_{gp}$ in SP with $\\max S_{sagp}$.\n\nWe then give the following observation in our experiment:\n\n|                   | Photo  | Computers | ESC-50 | FSDD   | Fashion-MNIST | CIFAR-10 |\n| ----------------- | ------ | --------- | ------ | ------ | ------------- | -------- |\n| $S_{mlp}$         | 385536 | 659968    | 379904 | 883712 | 668672        | 920064   |\n| $$L\\times\\max S_{sagp}$$ | 30126  | 31257     | 66519  | 47565  | 184572        | 625923   |\n\nBased on the above analysis, we believe that, **theoretically, the (average) single-epoch training time complexity of SAGP is consistent with that of MLP. However, in practice, SAGP's higher time cost stems from the use of more epochs, while its higher space cost stems from the use of sparse matrix multiplication, which increases linearly with $S_{gp}$.**\n\nNext, we discuss how to improve SAGP from the two aspects of reducing time and space complexity.\n\n## 1.1 Improve the time complexity of SAGP\n\n### 1.1.1 Reducing the training epochs\nThe key challenge in improving time complexity is reducing the number of training epochs. As shown in the results presented in Section 3.2, “Assembly Dynamics,” the number of hidden neurons and validation loss of SAGP on the FSDD dataset remain nearly constant around epoch 10,000. At this point, the ratio of the number of synapses in SAGP to the number of synapses in the MLP is 0.96%. It took an additional 40,000 epochs to reduce this ratio to 0.49%. This is actually a trade-off that depends on user preference: if users are willing to spend more time, they can obtain a smaller model. Therefore, if you want to reduce training time, directly reducing the number of training epochs is the simplest approach, which generally does not affect accuracy.\n\n### 1.1.2 Initializing SAGP using MLP\nAlthough we describe SAGP as developing from a minimal topology in the main text, please note that the neuron/synapse growth and apoptosis process in SAGP is actually independent of initialization. Therefore, we can first train an MLP on the same dataset and then use it as the initialization for SAGP. This effectively reduces the optimization difficulty and the number of required epochs.\n\n## 1.2 Improve the space complexity of SAGP\n\n### 1.2.1 Limiting the in-degree of neurons.\nThe key to improving the space complexity of SAGP is to reduce $\\max S_{sagp}$. As you mentioned in [suggestion 1], limiting the synaptic space is a great approach. From a biological perspective, it is obviously unlikely that a neuron would need to receive input from all the other neurons in the brain. From a computational complexity perspective, if we limit the maximum in-degree of a neuron to $K$, the relationship between $\\max S_{sagp}$ and the number of neurons will decrease from quadratic to linear.\n\n## 1.3 Additional experimental results\nWe use the number of neurons remaining unchanged in 100 epochs as the stopping condition and report the computational cost and accuracy of the two methods mentioned in 1.1.2 and 1.2.1. Among them, SAGP+I means using MLP as initialization. SAGP+L means limiting in-degree. SAGP+I+L means applying both at the same time. When +I is applied, we first construct and train a three-layer MLP with 50% of the budgeted hidden neurons as the initialization of SAGP, and then use the remaining 50% to grow according to the rules of SAGP. When “+L” is applied, only the top 100 weighted incoming edges in each neuron are used for message-passing.\n\nWe apologize for the time and character limitations, so we only report the results on the dataset Pohto as follows:\n\n|          | Epoch       | Training Time | Max Memory     | ACC         | #Synapse  |\n| -------- | ----------- | ------------- | -------------- | ----------- | --------- |\n| SAGP     | 7150        | 267s          | 7020Mb         | .8687       | 5316      |\n| SAGP+I   | 5681 (-21%) | 220s (-17%)   | 10633Mb (+15%) | .8549 (-2%) | 4871(-8%) |\n| SAGP+L   | 6977 (-2%)  | 251s (-6%)    | 6459Mb (-8%)   | .8574 (-1%) | 5301(-0%) |\n| SAGP+I+L | 6430 (-10%) | 231s (-13%)   | 6391Mb (-9%)   | .8594 (-1%) | 5514(+4%) |\n\nWe found that using the \"MLP initialization\" and \"limiting in-degree\" strategies played a positive role in optimizing the time and space complexity of SAGP, respectively, and the two strategies can work simultaneously while keeping the accuracy (ACC) and network size (#Synapse) approximately unchanged. Given that we have only conducted preliminary explorations of these two strategies, we believe they could perform even better with more thoughtful design.\n\n# 2. [W2 accuracy]\n\nWe fully understand your concerns about the applicability of SAGP. We acknowledge that SAGP still has some gaps in efficiency compared to MLP, which is almost the simplest neural network. However, SAGP is already over 10,000 times faster than the NDP[7], which was previously the only self-assembling neural network. On the other hand, self-assembly also gives SAGP unique advantages that MLP does not have. For example, as we mentioned in Section 4.3 of the main text, we found that when SAGP is used for feature selection, it can achieve comparable or even better performance than the state-of-the-art models.This means we can leverage SAGP to accelerate the training of all other models on the same dataset.\n\nFurthermore, we have found that SAGP is particularly well-suited for **continuous learning**. We conduct an additional experiment to verify this. First, we split the 10 classes in the Fashion-Mnist dataset into two parts. We initially trained SAGP on data from only the first five classes. After the topology stabilized, we restarted the self-assembly process on data from the last five classes. At this point, we get the following results:\n\n|                           | MLP            | SAGP            |\n| ------------------------- | -------------- | --------------- |\n| ACC on first five classes | 90.7% -> 43.6% | 89.2% -> %61.7% |\n\nWe found that SAGP forgets much slower than MLP on the first five classes. We hypothesize that this is due to the unique mechanism by which SAGP can learn new knowledge by generating new neuron. This may also constitute one of the unique advantages of SAGP.\n\nFinally, we apologize that due to time constraints, we were unable to report results on the self-supervised task, but this is indeed a promising idea because SAGP may learn task-specific structural information better than MLP. We will include this experiment in the future improvements of the paper."}, {"review_id": "QVSljUdjAT", "reviewer": "Reviewer_rmb6", "summary": "Inspired by the function of the human brain, this paper introduces the graph perceptron (GP) together with an assembling mechanism which allows for neuron growth and apoptosis. The authors investigate the training process of the GP, finding similarities to how the human brain functions. In competition with standard MLPs, the GP obtains similar performance with a small fraction of the MLP's size. Additionally, the GPs abilities for feature selection is studied, finding it to match or outperform state of the feature selection algorithms.", "questions": "Major questions/suggestions:\n1. As explained in the introduction, the GP architecture is inspired by the human brain, and the authors find the training dynamics to be similar to that of the human brain (ll. 210 - 214). This suggests to me that it might be possible to explain some aspects of the human brain with this model, extending the impact of this model to the neuroscience community (although I understand that this is not the scope of the current paper). This seems like a missed opportunity to me, adding the authors' thoughts on this to the discussion would be interesting.\n2. (similar to weakness 1) For the experiments in table 2, being the performance comparison between MLP and the GP, additionally stating the runtime and maximum memory would be interesting and relevant for practitioners.\n3. (figure 6 left plot) I find it interesting that the different models seemingly have very different train loss, even after one epoch, being almost at random initialization, for which I would expect similar loss for all models. Adding the authors' thoughts to this would be valuable.\n4. The appendix offers a lot of helpful information, it would be useful to point to this at relevant points in the manuscript.\n\n\nMinor questions/suggestions:\n1. What is the toy dataset Digit used for table 3? I can not find an explanation.\n2. What do the numbers stated in table 5 represent?\n3. l. 142 defines $\\mathcal{N}$, however this is used neither in the text nor in eq.8?\n4. After reading only question 3 and answer 3 they do not fit well to each other. After reading the main text the authors' thoughts become clear, but I would suggest to reformulate answer 3 such that it is understandable withou fully reading the main text.\n5. l.352 typo slection\n6. figure 5c: using a single y-axis would increase the readability of the plot, at first glance it looks as if the train loss is higher than the validation loss\n7. Pointing to the dataset explanation in the appendix directly in the caption of table 1 would make it easier to find.", "limitations": "I do not see any additional limitations to those the authors already discuss in the appendix. However, I would encourage the authors to make these more prominent in the main text, at least by pointing to the appendix in the conclusion.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. The graph perceptron is well motivated by the functioning of the human brain. During the training the authors find similarities to how humans learn, without building this explicitly into the model. This gives the model relevance beyond its application in machine learning tasks.\n2. The graph perceptron is thoroughly studied over multiple aspects of its behaviour (training dynamics, performance comparison against MLP and state of the art, feature selection), the experiments are well designed, performed and presented.\n3. Matching the MLP performance with only 5% of its size is an impressive result.\n4. The paper is a pleasure to read. The authors' thoughts are explained well by both text and figures.\n\nWeaknesses:\n1. For the experiments only the size of the final GP in comparison to the MLP is given. For practiticioners the runtime as well as maximum used memory would be as relevent. (This is discussed shortly in the appendix, but nowhere pointed to in the main text as far as I see)", "was_revised": true, "final_justification": "This is a strong paper with clear presentation and many helpful figures, making it a pleasure to read. It is easy to follow the authors' thoughts even when coming from a different field. Additionally, the proposed model is rigorously investigated with thoughtful experiments, which moreover demonstrate the superior performance of the proposed model.\n\nTherefore I strongly recommend accepting this paper.", "rebuttal": "We deeply appreciate your hard work in providing a professional and thorough review. Below is our response.\n\n# Responses to weaknesses:\n\n+ [W1]:\n  We provide the statistics of runtime costs that you are concerned about in the following two parts, along with a detailed comparison with MLP:\n\n ## Training cost (Table A)\n\n|                             | Photo         | Computers     | ESC-50        | FSDD          | Fashion-MNIST(batch_size=128) | CIFAR-10 (batch_size=128) |\n| --------------------------- | ------------- | ------------- | ------------- | ------------- | ----------------------------- | ------------------------- |\n| Training epoch (MLP)        | 1000          | 1000          | 1000          | 1000          | 1000                          | 1000                      |\n| Training epoch (SAGP)       | 50000         | 50000         | 50000         | 50000         | 100000                        | 100000                    |\n| Ratio                       | $\\times$50    | $\\times$50    | $\\times$50    | $\\times$50    | $\\times$100                   | $\\times$100               |\n| Training Time (MLP)         | 11s           | 16s           | 12s           | 13s           | 39s                           | 43s                       |\n| Training Time (SAGP)        | 754s          | 748s          | 634s          | 674s          | 3705s                         | 4031s                     |\n| Ratio                       | $\\times$68.54 | $\\times$46.75 | $\\times$52.87 | $\\times$56.24 | $\\times$95.00                 | $\\times$93.74             |\n| Training Peak memory (MLP)  | 734 Mb        | 636 Mb        | 651 Mb        | 898 Mb        | 512 Mb                        | 510 Mb                    |\n| Training Peak memory (SAGP) | 7020 Mb       | 14610 Mb      | 4994Mb        | 7833Mb        | 14189 Mb                      | 15530 Mb                  |\n| Ratio                       | $\\times$9.56  | $\\times$22.97 | $\\times$7.67  | $\\times$8.27  | $\\times$27.71                 | $\\times$30.45             |\n\n## Inference cost (Table B)\n\n\n|                             | Photo        | Computers    | ESC-50       | FSDD         | Fashion-MNIST(batch_size=128) | CIFAR-10 (batch_size=128) |\n| --------------------------- | ------------ | ------------ | ------------ | ------------ | ----------------------------- | ------------------------- |\n| Training Time (MLP)         | 0.03s        | 0.09s        | 0.03s        | 0.02s        | 1.23s                         | 1.98s                     |\n| Training Time (SAGP)        | 0.11s        | 0.16s        | 0.09s        | 0.08s        | 1.29s                         | 2.53s                     |\n| Ratio                       | $\\times$3.66 | $\\times$1.78 | $\\times$3.00 | $\\times$4.00 | $\\times$1.04                  | $\\times$1.28              |\n| Training Peak memory (MLP)  | 90 Mb        | 156 Mb       | 132 Mb       | 322 Mb       | 30 Mb                         | 78 Mb                     |\n| Training Peak memory (SAGP) | 291 Mb       | 401 Mb       | 821Mb        | 768Mb        | 39 Mb                         | 186 Mb                    |\n| Ratio                       | $\\times$3.23 | $\\times$2.57 | $\\times$6.22 | $\\times$2.38 | $\\times$1.30                  | $\\times$2.38              |\n\nAs shown in Table A, SAGP takes roughly the same amount of time to train per round as MLP, but the greater number of rounds results in a longer training time. Furthermore, SAGP training memory overhead is approximately 20 times higher. In terms of inference cost, both time and space consumption are roughly 1-6 times that of MLP.\n\nAs we mentioned in the limiation section of the appendix, we clearly understand that improving the efficiency of SAGP is a key area of future research. However, SAGP is already over 10,000 times faster than NDP, the previously considered sole self-assembling network. Furthermore, self-assembly also brings several unique benefits to the model. For example, as we mentioned in the main text, when used for feature selection, SAGP can reach or even surpass the state-of-the-art methods.\n\nFinally, our response to reviewer **4y96** discusses in detail the efficiency improvements to SAGP. If you are interested in this topic, you are welcome to read them further. Due to space limitations, we will not elaborate on these details here. We sincerely hope you will understand.\n\n# Responses to major questions/suggestions:\n\n+ [Major Q1]: Explaining the behavior of biological brains with (self-assembling) neural networks is indeed a very interesting question. We would like to share some naive opinions on this. Why is the increase in cortical volume in biological brains mainly concentrated in the early stages of biological development, rather than evenly distributed throughout the whole life cycle? This question has been extensively studied in biology. But if we only explain it from the experimental results of SAGP, we think it is very similar to the ``coordinate descent method'' in optimization. Coordinate descent is a strategy whereby for a multi-parameter optimization problem, we select only one variable at a time for optimization, while keeping the other variables unchanged. MLP only optimizes the weights between neurons, while SAGP needs to optimize weights and topology. In the early stages of network optimization, since all weights are randomly initialized, there are no absolutely dominant neurons at this time. Therefore, the growth rule will play a major rule, causing the network size to expand rapidly. After a period of training, dominant neurons begin to appear, and at this time the apoptosis rule becomes dominant. The network size gradually decreases. It is as if the model first focuses on optimizing the topological structure and then focuses on optimizing the weights. We speculate that organisms have somehow learned this \"coordinate descent\" strategy in long-term evolution, and this strategy may be more stable for the development of a single organism than strategies that optimize both topology and weights at the same time (such as traditional random gradient descent).\n\n+ [Major Q2]: Pleas see our response to Weaknesses 1 on this issue.\n\n+ [Major Q3]: To explain this problem, we first repeat formula (5) in the main text, which shows how a GP after two massage-passing obtains its logits:\n\n  $$\n  \\mathcal z = (A^T\\sigma(A^T\\begin{bmatrix}x \\\\ 0_{|\\mathcal H| + |\\mathcal O|}\\end{bmatrix}^T+b)+b)_{[-|\\mathcal O|:]}\n  $$\n\n  Where $|\\mathcal H|$ and $|\\mathcal O|$ represent the number of hidden neurons and output neurons, respectively. To simplify the situation, we now remove all activation functions and biases from the above formula:\n  $$\n  \\mathcal z = \\left((A^T)^2\\begin{bmatrix}x \\\\ 0_{|\\mathcal H| + |\\mathcal O|}\\end{bmatrix}^T\\right)_{[-|\\mathcal O|:]}\n  $$\n  If all elements in $A$ are randomly initialized, then the loss calculated using $z$ and the label should indeed have the same expectation. However, not all elements in $A$ are learnable. In fact, in a topology, if two neurons are connected, then the weights between them need to be randomly initialized (and learned). Otherwise, the weights are fixed to zero. Therefore, for different topologies, the non-zero positions are also different, so the obtained $z$ will also be very different, resulting in different initial losses. However, it must be warned that this is only a very simplified analysis. We ignore the activation function, bias, and do not consider other parts of the specific implementation (such as normalization). Therefore, it is difficult to have an exact relationship here that allows us to get its initial loss from the topology of the GP.\n\n+ [Major Q4]: Thank you for your suggestion, which is very important for us to further improve the clarity of the paper. Now we have mentioned everything related to the experiments in the appendix in more detail in Section 4 (Experiments) in the main text: including dataset description, hyperparameters, pseudocode, ablation experiments, parameter experiments, etc. At the same time, we mentioned the section ``Differences between self-assembling networks and other topics'' in the appendix at the end of Section 2 (Related Work) in the main text. Finally, we also mentioned the limitations of SAGP (Appendix Section 5.1) in Section 5 (Conclusion) in the main text to help readers understand the model more comprehensively and objectively.\n\n# Responses to minor questions/suggestions:\n\n+ [Minor Q1]: The toy dataset Digit is the dataset used in the previous self-assembling neural network NDP [7] (this dataset is actually an 8*8 pixel version of the famous MNIST dataset). We have now added additional explanations and references in the Caption of table 3.\n+ [Minor Q2]: The data in Table 5 represents the accuracy of the selected feature subsets in the downstream classification task after using different feature selection methods/ ratios. The higher the number, the better the method can select the most important feature components. We have now added additional instructions in the table caption.\n+ [Minor Q3]: $\\mathcal N$ is explicitly used as the subscript of the summation sign in Equation 8 and Equation 9.\n+ [Minor Q4]: This is a great suggestion. We have now modified Question 3 to “In what fields can SAGP show its potential ?” and Answer 3 to “The out-degree of input neurons of a well-trained SAGP can be used in areas such as model interpretability and feature selection.” to ensure that the question-answer pairs are more clearly aligned.\n+ [Minor Q5]: We have fixed this typo in the latest version.\n+ [Minor Q6]: In Figure 5C, since the magnitudes of the two curves are not exactly the same, using a single y-axis will make one of the trends unclear. If space permits, we will consider replacing it with two independent images in the final version."}, {"review_id": "9azC64OtFS", "reviewer": "Reviewer_6vGG", "summary": "The paper introduces a genuinely innovative approach combining Graph Perceptrons (GP) as a generalization of MLPs with self-assembly mechanisms inspired by biological neural development.", "questions": "Q1: Comparison against state-of-the-art architectures should also be provided for completeness (not only MLP). For instance, CNN results for image datasets.\n\nQ2: How would the self-assembly principles scale to modern large-scale architectures and datasets? The computational complexity O(λBL(n₁ + n₂ + n₃)²) raises concerns about practical scalability beyond the relatively simple experimental settings presented.", "limitations": "Missing limitations section in the main text.", "rating": 4, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper presents a novel bio-inspired self-assembly mechanism with strong theoretical foundations and impressive efficiency gains, achieving comparable performance with only 0.49%-9.86% of synapses compared to MLPs while demonstrating over 10,000x speedup compared to previous methods. The comprehensive experimental evaluation across multiple domains and integration with existing architectures shows practical potential.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful review, we now respond to your question below.\n\n# Regarding Q1\n\nWe understand your concerns regarding the baselines in Table 2. However, our intended conclusion in Table 2 is that \"SAGP can achieve nearly identical performance to MLP using a much sparser topology than MLP,\" not that \"SAGP surpasses MLP in accuracy.\" Adding more powerful comparison methods to Table 2 would be easy, but it wouldn't add any useful conclusions. On the one hand, MLP is almost the simplest neural network, so more powerful methods (such as CNNs) clearly achieve superior performance and are therefore likely to outperform SAGP. On the other hand, these methods include not only the MLP as a module but also many other modules, such as convolutional layers and regularization layers. It's difficult to say how many \"neurons\" or \"synapses\" a convolutional layer or regularization layer has. We typically use these terms only in relation to MLPs, so including other models in Table 1 would obscure the comparison of topology size.\n\nWe specifically explore other applications of SAGP besides reducing topology size in Section 5.3 of the main paper, such as feature selection. At this point, we compare SAGP with the most advanced methods in this field and find that SAGP remains highly competitive.\n\n\n\n# Regarding Q2\n\nIn fact, the computational complexity of SAGP ($O(\\lambda BL(n_1+n_2+n_3)^2)$) is equivalent to that of MLP in some sense. We consider a two-layer MLP and ignore the activation function and bias (because they are not the main factors affecting the complexity), which can be expressed as:\n\n$$\n\\boldsymbol{z}=\\boldsymbol{W}^T_2\\boldsymbol{W}^T_1\\boldsymbol{x}\n$$\n\nWhere $\\boldsymbol{x} \\in \\mathbb R^{|\\mathcal I|}$, $\\boldsymbol{W}_1^T \\in \\mathbb R^{|\\mathcal H| \\times |\\mathcal I|}$,$\\boldsymbol{W}_2^T \\in \\mathbb R^{|\\mathcal O| \\times |\\mathcal H|}$ and $\\boldsymbol{z} \\in \\mathbb R^{|\\mathcal O|}$\n. \n\nRecall that the time complexity of multiplying a $(n\\times m)$ matrix by a $(m \\times p)$matrix is $\\Theta(nmp)$ and the space complexity is $\\Theta(1)$. Therefore, the time and space complexities of computing $z$ are $\\Theta(|\\mathcal H|\\times|\\mathcal O| + |\\mathcal H|\\times|\\mathcal I|)$ and $\\Theta(1)$, respectively. Note that the total number of synapses $S_{mlp}$ in this MLP is also $|\\mathcal H|\\times|\\mathcal O| + |\\mathcal H|\\times|\\mathcal I|$, so the time complexity can also be expressed as $\\Theta(S_{mlp})$.\n\nA simplified GP can be expressed as:\n\n$$\n\\boldsymbol{z} = \\left((\\boldsymbol{A}^T)^2\\begin{bmatrix}\\boldsymbol{x} \\\\ 0_{|\\mathcal H| + |\\mathcal O|}\\end{bmatrix}^T\\right)_{[-|\\mathcal O|:]}\n$$\n\nThe definition of $\\boldsymbol A$ is consistent with that of Formula 6 in the paper. GP use sparse matrix multiplication in its implementation, which in **once sparse matrix multiplication**, the time complexity and space complexity are both $\\Theta(S_{gp})$ , Here, $S_{gp}$ represents the number of synapses in GP, which is equivalent to the number of non-zero elements in $\\boldsymbol A$. So for a GP with two message-passing, the time complexity and space complexity are $\\Theta(2S_{gp})$ and $\\Theta(S_{gp})$ respectively.\n\nThe above conclusions can be extended to any layer of $L$-layer MLP or any $L$-times message-passing GP. Now, we Let the batch size is $B$, and the MLP and GP training rounds are $k_{mlp}$ and $k_{gp}$ respectively, then the total training complexity comparison is:\n\n|      | Time                        | Space               |\n| ---- | --------------------------- | ------------------- |\n| MLP  | $$\\Theta(k_{mlp}BS_{mlp})$$ | $$\\Theta(B)$$       |\n| GP   | $$\\Theta(k_{gp}BLS_{gp})$$  | $$\\Theta(BS_{gp})$$ |\n\n$\\lambda(n_1+n_2+n_3)^2$ is actually the expansion of $S_{gp}$.\n\n**Here, we would like to ask you to pay special attention to one detail**: in the table, it seems that the time complexity of GP is one factor $L$ higher than that of MLP, but in fact, when $\\Theta(k_{mlp})=\\Theta(k_{gp})$, the complexity of the two is equivalent. This is because $S_{mlp}$ increases linearly with $L$, but $S_{mlp}$ is constant with $L$. In other words, $\\Theta(S_{mlp})=\\Theta(LS_{gp})$.\n\nFor the complexity of SAGP, we only need to replace $S_{gp}$ with $\\max S_{sagp}$.\n\nIn particular, in experimental observations, we found that:\n\n|                            | Photo  | Computers | ESC-50 | FSDD   | Fashion-MNIST | CIFAR-10 |\n| -------------------------- | ------ | --------- | ------ | ------ | ------------- | -------- |\n| $S_{mlp}$                  | 385536 | 659968    | 379904 | 883712 | 668672        | 920064   |\n| $$L \\times \\max S_{sagp}$$ | 30126  | 31257     | 66519  | 47565  | 184572        | 625923   |\n\nThis further illustrates that $\\Theta(S_{mlp})=\\Theta(L\\max S_{sagp})$. Therefore, at least in theory, the time cost of a single round of inference for SAGP is no more than that of MLP. But in practice, SAGP does cost more than MLP (we clearly acknowledge this in the \"Limitations\" section of the Appendix). Based on the complexity comparison table above, we can reduce the cost of SAGP in two ways: reducing $k_{gp}$ or reducing $\\max S_{sagp}$. Here, we offer two strategies to achieve this.\n\n1. **Initializing SAGP using MLP.** Although we describe SAGP as developing from a minimal topology in the main text, please note that the neuron/synapse growth and apoptosis process in SAGP is actually independent of initialization. Therefore, we can first train an MLP on the same dataset and then use it as the initialization for SAGP. This effectively reduces the optimization difficulty and the number of required epochs.\n2. **Limiting the in-degree of neurons.** Thanks to the reviewer **4y96** for the great suggestion. We note that directly limiting the maximum degree of neurons is a simple and reasonable approach - after all, it is unlikely that a neuron in a biological brain is connected to all other neurons.\n\nWe verified these two ideas on the Photo dataset as follows:\n\n|          | Epoch($k_{sagp}$) | Training Time | Max Memory     | ACC         | #Synapse  |\n| -------- | ----------------- | ------------- | -------------- | ----------- | --------- |\n| SAGP     | 7150              | 267s          | 7020Mb         | .8687       | 5316      |\n| SAGP+I   | 5681 (-21%)       | 220s (-17%)   | 10633Mb (+15%) | .8549 (-2%) | 4871(-8%) |\n| SAGP+L   | 6977 (-2%)        | 251s (-6%)    | 6459Mb (-8%)   | .8574 (-1%) | 5301(-0%) |\n| SAGP+I+L | 6430 (-10%)       | 231s (-13%)   | 6391Mb (-9%)   | .8594 (-1%) | 5514(+4%) |\n\nWhere SAGP+I means using MLP as initialization. SAGP+L means limiting in-degree. SAGP+I+L means applying both at the same time. When +I is applied, we first construct and train a three-layer MLP with 50% of the budgeted hidden neurons as the initialization of SAGP, and then use the remaining 50% to grow according to the rules of SAGP. When “+L” is applied, only the top 100 weighted incoming edges in each neuron are used for message-passing. We use the number of neurons remaining unchanged in 100 epochs as the stopping condition.\n\nWe found that using the \"Initializing SAGP using MLP\" and \"Limiting the in-degree of neurons\" strategies played a positive role in optimizing the time and space complexity of SAGP, respectively, and the two strategies can work simultaneously while keeping the accuracy (ACC) and network size (#Synapse) approximately unchanged. Given that we have only conducted preliminary explorations of these two strategies, we believe that after further detailed design and optimization, it is entirely possible to apply the self-assembly mechanism to larger models and datasets."}]

Author rebuttal:

Meta-review: This paper proposes Self-Assembling Graph Perceptrons (SAGP), a novel architecture that generalizes MLPs into graph-structured networks with neuron growth, synaptic pruning, and apoptosis. The work is both theoretically rigorous by showing that MLPs are a special case of SAGP and empirically supported, with experiments demonstrating that SAGP achieves comparable accuracy to MLPs using only a fraction of the synapses. The approach also highlights advantages for interpretability and continual learning, making it a timely and creative contribution to biologically inspired architectures.

The main concerns raised by reviewers centered on training efficiency and scalability. While per-epoch complexity is theoretically comparable to MLPs, SAGP requires many more epochs and significantly more memory during training, raising questions about its applicability to large-scale datasets. Reviewers also noted the limited evaluation scope (mostly small- and medium-scale classification benchmarks) and the absence of comparisons to state-of-the-art architectures such as CNNs. These limitations prevent the work from being viewed as fully established for large-scale use cases.

The authors provided a thorough and convincing rebuttal, including new results on ImageNet-1K with MobileNetV2 initialization, detailed training/inference cost tables, and strategies to reduce complexity (e.g., MLP initialization, limiting neuron in-degree). Reviewers found these clarifications strong, with two upgrading their recommendations to strong accept. While efficiency and evaluation scope remain open challenges, the paper introduces a principled and impactful idea that will spark further research. Therefore, I recommend acceptance.
Decision: Accept (spotlight)