ICLR 2025

Published as a conference paper at ICLR 2025
LEARNING EFFICIENT POSITIONAL ENCODINGS WITH
GRAPH NEURAL NETWORKS
Charilaos I. Kanatsoulis1∗, Evelyn Choi1, Stephanie Jegelka2,3, Jure Leskovec1†,
Alejandro Ribeiro4†
1Stanford University
2MIT
3Technical University of Munich
4 University of Pennsylvania
ABSTRACT
Positional encodings (PEs) are essential for effective graph representation learning
because they provide position awareness in inherently position-agnostic trans-
former architectures and increase the expressive capacity of Graph Neural Net-
works (GNNs). However, designing powerful and efficient PEs for graphs poses
significant challenges due to the absence of canonical node ordering and the scale
of the graph. In this work, we identify four key properties that graph PEs should
satisfy: stability, expressive power, scalability, and genericness. We find that exist-
ing eigenvector-based PE methods often fall short of jointly satisfying these criteria.
To address this gap, we introduce PEARL, a novel framework of learnable PEs for
graphs. Our primary insight is that message-passing GNNs function as nonlinear
mappings of eigenvectors, enabling the design of GNN architectures for generating
powerful and efficient PEs. A crucial challenge lies in initializing node features
in a manner that is both expressive and permutation equivariant. We tackle this
by initializing GNNs with random node inputs or standard basis vectors, thereby
unlocking the expressive power of message-passing operations, while employing
statistical pooling functions to maintain permutation equivariance. Our analysis
demonstrates that PEARL approximates equivariant functions of eigenvectors
with linear complexity, while rigorously establishing its stability and high expres-
sive power. Experimental evaluations show that PEARL outperforms lightweight
versions of eigenvector-based PEs and achieves comparable performance to full
eigenvector-based PEs, but with one or two orders of magnitude lower com...