Paper:
Graph Neural Networks (GNNs) are based on repeated aggregations of information across nodes' neighbors in a graph. However, because common neighbors are shared between different nodes, this leads to repeated and inefficient computations. We propose Hierarchically Aggregated computation Graphs (HAGs), a new GNN graph representation that explicitly avoids redundancy by managing intermediate aggregation results hierarchically, and eliminating repeated computations and unnecessary data transfers in GNN training and inference. We introduce an accurate cost function to quantitatively evaluate the runtime performance of different HAGs and use a novel search algorithm to find optimized HAGs. Experiments show that the HAG representation significantly outperforms the standard GNN graph representation by increasing the end-to-end training throughput by up to 2.8× and reducing the aggregations and data transfers in GNN training by up to 6.3× and 5.6×. Meanwhile, HAGs improve runtime performance by preserving GNN computation, and maintain the original model accuracy for arbitrary GNNs.

Review:
This paper aims to propose a speeding-up strategy to reduce the training time for existing GNN models by reducing the redundant neighbor pairs. The idea is simple and clear. The paper is well-written. However, major concerns are:
1. This strategy is only for equal contribution models (e.g., GCN, GraphSAGE), not for methods which consider distinct contribution weights for individual node neighbor (e.g. GAT). However, in my opinion, for one target node, different neighbors should be assigned with different contributions instead of equal contributions.
2. What kind of graphs can the proposed model be applied to? This paper seems to only consider unweighted undirected graphs. How about directed and weighted graphs? Even for an unweighted undirected graph, the symmetric information may also be redundant for further elimination. Then can HAG reduce this symmetric redundancy?
3. There is no effectiveness evaluation comparing the original GNN models with the versions with HAG. The authors claim that, with the HAG process, the efficiency could be improved without losing accuracy. But there are no experimental results verifying that the effectiveness of the HAG-versions which could obtain comparable performance with the original GNN models for some downstream applications (e.g., node classification).
--------------------------------------------------Update------------------------------------------------
Thanks very much for the authors' feedback. The revised version has clarified some of my concerns. However, the equal-contribution (in Comment 1) is still a big one that the authors should pay attention. I increase my score to 3.