Learning Graph Normalization for Graph Neural Networks
Authors: Yihao Chen, Xin Tang, Xianbiao Qi, Chun-Guang Li, Rong Xiao
Keywords: Graph Neural Network, Normalization, Graph Normalization
Abstract: Graph Neural Networks (GNNs) have emerged as a useful paradigm to process graph-structured data. Usually, GNNs are stacked to multiple layers and the node representations in each layer are computed through propagating and aggre- gating the neighboring node features with respect to the graph. To effectively train a GNN with multiple layers, some normalization techniques are necessary. Though the existing normalization techniques have been shown to accelerate train- ing GNNs, the structure information on the graph is ignored yet. In this paper, we propose two graph-aware normalization methods to effectively train GNNs. Then, by taking into account that normalization methods for GNNs are highly task-relevant and it is hard to know in advance which normalization method is the best, we propose to learn attentive graph normalization by optimizing a weighted combination of multiple graph normalization methods at different scales. By op- timizing the combination weights, we can automatically select the best or the best combination of multiple normalization methods for a specific task. We con- duct extensive experiments on benchmark datasets for different tasks and confirm that the graph-aware normalization methods lead to promising results and that the learned weights suggest the more appropriate normalization methods for specific task