  In recent studies, neural message passing has proved to be an effective way
to design graph neural networks (GNNs), which have achieved state-of-the-art
performance in many graph-based tasks. However, current neural-message passing
architectures typically need to perform an expensive recursive neighborhood
expansion in multiple rounds and consequently suffer from a scalability issue.
Moreover, most existing neural-message passing schemes are inflexible since
they are restricted to fixed-hop neighborhoods and insensitive to the actual
demands of different nodes. We circumvent these limitations by a novel
feature-message passing framework, called Graph Multi-layer Perceptron (GMLP),
which separates the neural update from the message passing. With such
separation, GMLP significantly improves the scalability and efficiency by
performing the message passing procedure in a pre-compute manner, and is
flexible and adaptive in leveraging node feature messages over various levels
of localities. We further derive novel variants of scalable GNNs under this
framework to achieve the best of both worlds in terms of performance and
efficiency. We conduct extensive evaluations on 11 benchmark datasets,
including large-scale datasets like ogbn-products and an industrial dataset,
demonstrating that GMLP achieves not only the state-of-art performance, but
also high training scalability and efficiency.
