  Expressivity plays a fundamental role in evaluating deep neural networks, and
it is closely related to understanding the limit of performance improvement. In
this paper, we propose a three-pipeline training framework based on critical
expressivity, including global model contraction, weight evolution, and link's
weight rewiring. Specifically, we propose a pyramidal-like skeleton to overcome
the saddle points that affect information transfer. Then we analyze the reason
for the modularity (clustering) phenomenon in network topology and use it to
rewire potential erroneous weighted links. We conduct numerical experiments on
node classification and the results confirm that the proposed training
framework leads to a significantly improved performance in terms of fast
convergence and robustness to potential erroneous weighted links. The
architecture design on GNNs, in turn, verifies the expressivity of GNNs from
dynamics and topological space aspects and provides useful guidelines in
designing more efficient neural networks.
