Title: Caterpillar GNN: Replacing Message Passing with Efficient Aggregation
Primary area: learning on graphs and other geometries & topologies
Keywords: graph neural networks, expressivity, homomorphism counts, graph theory, theoretical machine learning, learning theory

Abstract: Message-passing graph neural networks (MPGNNs) dominate modern graph learning.
Typical efforts enhance MPGNN's expressive power by enriching the adjacency-based aggregation.
In contrast, we introduce an efficient aggregation over walk incidence-based matrices that are constructed to deliberately trade off some expressivity for stronger and more structured inductive bias.
Our approach allows for seamless scaling between classical message-passing and simpler methods based on walks. 
We rigorously characterize the expressive power at each intermediate step using homomorphism counts over a hierarchy of generalized caterpillar graphs. Based on this foundation, we propose Caterpillar GNNs, whose robust graph-level aggregation successfully tackles a benchmark specifically designed to challenge MPGNNs.
Moreover, we demonstrate that, on real-world datasets, Caterpillar GNNs achieve comparable predictive performance while significantly reducing the number of nodes in the hidden layers of the computational graph.

TL;DR: This paper introduces and theoretically characterizes a less expressive but more structured alternative to message passing, grounded in graph theory and automata theory.