user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Predict then Propagate: Graph Neural Networks meet Personalized PageRank

Paper content:
Neural message passing algorithms for semi-supervised classification on graphs have recently achieved great success.
However, for classifying a node these methods only consider nodes that are a few propagation steps away and the size of this utilized neighborhood is hard to extend.
In this paper, we use the relationship between graph convolutional networks (GCN) and PageRank to derive an improved propagation scheme based on personalized PageRank.
We utilize this propagation procedure to construct a simple model, personalized propagation of neural predictions (PPNP), and its fast approximation, APPNP.
Our model's training time is on par or faster and its number of parameters on par or lower than previous models.
It leverages a large, adjustable neighborhood for classification and can be easily combined with any neural network.
We show that this model outperforms several recently proposed methods for semi-supervised classification in the most thorough study done so far for GCN-like models.
Our implementation is available online.
Graphs are ubiquitous in the real world and its description through scientific models.
They are used to study the spread of information, to optimize delivery, to recommend new books, to suggest friends, or to find a party's potential voters.
Deep learning approaches have achieved great success on many important graph problems such as link prediction BID15 , graph classification BID12 BID31 BID13 and semi-supervised node classification BID43 BID21 .There
are many approaches for leveraging deep learning algorithms on graphs. Node
embedding methods use random walks or matrix factorization to directly train individual node embeddings, often without using node features and usually in an unsupervised manner, i.e. without leveraging node classes BID33 BID40 BID30 BID15 BID35 . Many
other approaches use both graph structure and node features in a supervised setting. Examples
for these include spectral graph convolutional neural networks BID6 BID11 , message passing (or neighbor aggregation) algorithms BID19 BID21 BID16 BID34 BID28 BID13 , and neighbor aggregation via recurrent neural networks BID36 BID24 BID10 . Among these
categories, the class of message passing algorithms has garnered particular attention recently due to its flexibility and good performance.Several works have been aimed at improving the basic neighborhood aggregation scheme by using attention mechanisms BID19 BID16 BID41 , random walks BID0 BID44 , edge features BID19 BID13 BID37 and making it more scalable on large graphs BID44 . However, all
of these methods only use the information of a very limited neighborhood for each node. A larger neighborhood
would be desirable to provide the model with more information, especially for nodes in the periphery or in a sparsely labelled setting.Increasing the size of the neighborhood used by these algorithms, i.e. their range, is not trivial since neighborhood aggregation in this scheme is essentially a type of Laplacian smoothing and too many layers lead to oversmoothing . BID42 highlighted the
same problem by establishing a relationship between the message passing algorithm termed Graph Convolutional Network (GCN) by BID21 and a random walk. Using this relationship
we see that GCN converges to this random walk's limit distribution as the number of layers increases. The limit distribution
is a property of the graph as a whole and does not take the random walk's starting (root) node into account. As such it is unsuited
to describe the root node's neighborhood. Hence, GCN's performance
necessarily deteriorates for a high number of layers (or aggregation/propagation steps).To solve this issue, in this
paper, we first highlight the inherent connection between the limit distribution and PageRank BID32 . We then propose an algorithm
that utilizes a propagation scheme derived from personalized PageRank instead. This algorithm adds a chance
of teleporting back to the root node, which ensures that the PageRank score encodes the local neighborhood for every root node BID32 . The teleport probability allows
us to balance the needs of preserving locality (i.e. staying close to the root node to avoid oversmoothing) and leveraging the information from a large neighborhood. We show that this propagation scheme
permits the use of far more (in fact, infinitely many) propagation steps without leading to oversmoothing.Moreover, while propagation and classification are inherently intertwined in message passing, our proposed algorithm separates the neural network from the propagation scheme. This allows us to achieve a much higher
range without changing the neural network, whereas in the message passing scheme every additional propagation step would require an additional layer. It also permits the independent development
of the propagation algorithm and the neural network generating predictions from node features. That is, we can combine any state-of-the-art
prediction method with our propagation scheme. We even found that adding our propagation scheme
during inference significantly improves the accuracy of networks that were trained without using any graph information.Our model achieves state-of-the-art results while requiring fewer parameters and less training time compared to most competing models, with a computational complexity that is linear in the number of edges. We show these results in the most thorough study
(including significance testing) of message passing models using graphs with text-based features that has been done so far.
In this paper we have introduced personalized propagation of neural predictions (PPNP) and its fast approximation, APPNP.
We derived this model by considering the relationship between GCN and PageRank and extending it to personalized PageRank.
This simple model decouples prediction and propagation and solves the limited range problem inherent in many message passing models without introducing any additional parameters.
It uses the information from a large, adjustable (via the teleport probability α) neighborhood for classifying each node.
The model is computationally efficient and outperforms several state-of-the-art methods for semi-supervised classification on multiple graphs in the most thorough study which has been done for GCN-like models so far.For future work it would be interesting to combine PPNP with more complex neural networks used e.g. in computer vision or natural language processing.
Furthermore, faster or incremental approximations of personalized PageRank BID2 BID3 BID25 and more sophisticated propagation schemes would also benefit the method.A EXISTENCE OF Π PPRThe matrix DISPLAYFORM0 exists iff the determinant det(I n − (1 − α)Â) = 0, which is the case iff det(Â − 1 1−α I n ) = 0, i.e. iff 1 1−α is not an eigenvalue ofÂ.
This value is always larger than 1 since the teleport probability α ∈ (0, 1].
Furthermore, the symmetrically normalized matrixÂ has the same eigenvalues as the row-stochastic matrixÃ rw .
This can be shown by multiplying the eigenvalue equationÂv = λv .
The largest eigenvalue of a row-stochastic matrix is 1, as can be proven using the Gershgorin circle theorem.
Hence, B CONVERGENCE OF APPNP APPNP uses the iterative equation DISPLAYFORM1 After the k-th propagation step, the resulting predictions are DISPLAYFORM2 H.If we take the limit k → ∞ the left term tends to 0 and the right term becomes a geometric series.
The series converges since α ∈ (0, 1] andÂ is symmetrically normalized and therefore det(Â) ≤ 1, resulting in The sampling procedure is illustrated in FIG3 .
The data is first split into a visible and a test set.
For the visible set 1500 nodes were sampled for the citation graphs and 5000 for MICROSOFT ACADEMIC.
The test set contains all remaining nodes.
We use three different label sets in each experiment: A training set of 20 nodes per class, an early stopping set of 500 nodes and either a validation or test set.
The validation set contains the remaining nodes of the visible set.
We use 20 random seeds for determining the splits.
These seeds are drawn once and fixed across runs to facilitate comparisons.
We use one set of seeds for the validation splits and a different set for the test splits.
Each experiment is run with 5 random initializations on each data split, leading to a total of 100 runs per experiment.
DISPLAYFORM3 The early stopping criterion uses a patience of p = 100 and an (unreachably high) maximum of n = 10 000 epochs.
The patience is reset whenever the accuracy increases or the loss decreases on the early stopping set.
We choose the parameter set achieving the highest accuracy and break ties by selecting the lowest loss on this set.
This criterion was inspired by GAT BID41 .We
used TensorFlow (Martín BID26 for all experiments except bootstrapped feature propagation. All
uncertainties and confidence intervals correspond to a confidence level of 95 % and were calculated by bootstrapping with 1000 samples.We use the Adam optimizer with a learning rate of l = 0.01 and cross-entropy loss for all models BID20 . Weights
are initialized as described in BID14 . The feature
matrix is L 1 normalized per row.

assistant: text: Personalized propagation of neural predictions (PPNP) improves graph neural networks by separating them into prediction and propagation via personalized PageRank.