Introduction
Graph-structured data are ubiquitous in the real world.
To model and learn from such data, graph representation learning aims to produce meaningful node representations by simultaneously considering the graph topology and node attributes.
It has attracted growing interest in recent years, as well as numerous real-world applications [ 32 ] .

In particular, graph neural networks (GNNs) are a widely used approach for node, edge, and graph prediction tasks.
Recently, many GNN models have been proposed (e.g., graph convolutional network [ 14 ] , graph attention network [ 26 ] , simple graph convolution [ 30 ] ).
Most modern GNN models follow a “message passing” scheme:
they iteratively aggregate the hidden representations of every node with those of the adjacent nodes to generate new hidden representations, where each iteration is parameterized as a neural network layer with learnable weights.

Despite the success existing GNN models achieve on many different scenarios, they lack the ability to capture long-range dependencies.
Specifically, for a predefined number of layers T , these models cannot capture dependencies with a range longer than T -hops away from any given node.
A straightforward strategy to capture long-range dependencies is to stack a large number of GNN layers for receiving “messages” from distant nodes.
However, existing work has observed poor empirical performance when stacking more than a few layers [ 16 ] , which has been referred to as oversmoothing. This has been attributed to various reasons, including node representations becoming indistinguishable as depth increases.
Besides oversmoothing, GNN models with numerous layers require excessive computational cost in practice since they need to repeatedly propagate representations across many layers.
For these two reasons, simply stacking many layers for GNNs is not a suitable way to capture long-range dependencies.

Recently, instead of stacking many layers, several works have been proposed to capture long-range dependencies in graphs. Pei et al. [ 21 ] propose a global graph method Geom-GCN which builds structural neighborhoods based on the graph and the embedding space, and uses a bi-level aggregation to capture long-range dependencies.
However, as Geom-GCN still only has finite layers, it still fails to capture very long range dependencies.

To model longer range dependencies, Gu et al. [ 10 ] propose the implicit graph neural network (IGNN), which can be considered as a GNN model with infinite layers.
Thus, IGNN does not suffer from any a priori limitation on the range of information it can capture.
To achieve this, IGNN generates predictions as the solution to a fixed-point equilibrium equation.
Specifically, the solution is obtained by an iterative solver. Moreover, they require additional conditions to ensure well-posedness of the model and use projected gradient descent method to train the model.
However, the practical limitations of iterative solvers are widely recognized: they can lack robustness; the generated solution is approximated; and the number of iterations cannot be known in advance [ 23 ] .
In our experiments, we found that IGNN sometimes experiences non-convergence in its iterative solver.
Besides the non-convergence issue, the iterative solver of IGNN can be inefficient as IGNN needs to run it once per forward or backward pass.

Motivated by these limitations, we propose our Efficient Infinite-Depth Graph Neural Network (EIGNN) approach, which can effectively capture very long range dependencies in graphs.
Instead of relying on iterative solvers to generate the solution like in IGNN [ 10 ] , we derive a closed-form solution for EIGNN without additional conditions, avoiding the need for projected gradient descent to train the model.
Furthermore, we propose to use eigendecomposition to improve the efficiency of EIGNN, without affecting its accuracy.

The contributions of this work are summarized as follows:

To capture long-range dependencies, we propose our infinite-depth EIGNN model. To do this, we first define our model as the limit of an infinite sequence of graph convolutions, and theoretically prove its convergence. Then, we derive tractable forward and backward computations for EIGNN.

We then further derive an eigendecomposition-based approach to improve the computational/memory complexity of our approach, without affecting its accuracy.

We empirically compare our model to recent baseline GNN models on synthetic and real-world graph datasets. The results show that EIGNN has a better ability to capture very long range dependencies and provides better performance compared with other baselines. Moreover, the empirical results of noise sensitivity experiments demonstrate that EIGNN is more robust against both noise and adversarial perturbations.