Title: Variational Graph Structure Learning for GNNs by Using Marginal Likelihood
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords: Graph Structure Learning, Graph Neural Networks, Marginal Likelihood, Laplace Approximation

Abstract: Learning graph structures for Graph Neural Networks (GNNs) can improve their performance, but it involves a challenging search over the large discrete space of all possible graphs. Prior works often enforce fixed constraints on the graph structure to induce properties such as sparsity, but such rigidity can be overly restrictive and harm performance. Here, we propose a simpler alternative to use the marginal likelihood which naturally favors such properties. We show that a variational formulation with Laplace's method automatically leads to a marginal likelihood based objective over discrete graph structures, which can be optimized efficiently using the Gumbel-Softmax trick. We call this approach the Laplace Approximation-based Graph Structure (LAGS) method, and show empirically that it improves the performance of different base GNNs, including recent state-of-the-art GNNs that outperform graph transformers.