NeurIPS 2025

Brain network science modelling of sparse neural
networks enables Transformers and LLMs to perform
as fully connected
Yingtao Zhang1,2, Diego Cerretti1,2, Jialin Zhao1,2, Ziheng Liao1,2, Wenjing Wu1,2
Umberto Michieli4,5 & Carlo Vittorio Cannistraci1,2,3∗
1Center for Complex Network Intelligence (CCNI)†
2Dept. of Computer Science & Technology, 3School of Biomedical Engineering, Tsinghua University
4University of Padova, 5Canva Research
Abstract
This study aims to enlarge our current knowledge on the application of brain-
inspired network science principles for training artificial neural networks (ANNs)
with sparse connectivity. Dynamic sparse training (DST) emulates the synaptic
turnover of real brain networks, reducing the computational demands of train-
ing and inference in ANNs. However, existing DST methods face difficulties in
maintaining peak performance at high connectivity sparsity levels. The Cannistraci-
Hebb training (CHT) is a brain-inspired method that is used in DST for growing
synaptic connectivity in sparse neural networks. CHT leverages a gradient-free,
topology-driven link regrowth mechanism, which has been shown to achieve ultra-
sparse (1% connectivity or lower) advantage across various tasks compared to
fully connected networks. Yet, CHT suffers two main drawbacks: (i) its time
complexity is O(N · d3)- N node network size, d node degree - hence it can be
efficiently applied only to ultra-sparse networks. (ii) it rigidly selects top link
prediction scores, which is inappropriate for the early training epochs, when the
network topology presents many unreliable connections. Here, we design the first
brain-inspired network model - termed bipartite receptive field (BRF) - to initialize
the connectivity of sparse artificial neural networks. Then, we propose a matrix
multiplication GPU-friendly approximation of the CH link predictor, which reduces
the computational complexity to O(N 3), enabling a fast implementation of link
prediction in large-scale m...