Current graph neural networks (GNNs) are promising, especially when the entire graph is known for training . However, it is not yet clear how to efficiently train GNNs on temporal graphs , where new vertices, edges, and even classes appear over time. We face two challenges : First, shifts in the label distribution (including the appearance of new labels), which require adapting the model . Second, the growth of the graph, which makes it, at some point, infeasible to train over all vertices and edges. We address these issues by applying a sliding window technique, i.e., we incrementally train GNNs on limited window sizes and analyze their performance. For our experiments, we have compiled three new temporal graph datasets based on scientific publications and evaluate isotropic and anisotropic GNN architectures . Our results show that both GNN types provide good results even for a window size of just 1 time step. With window sizes of 3 to 4 time steps, GNNs achieve at least 95\% accuracy compared to using the entire timeline of the graph . With window sizes of 6 or 8, at least 99\% accuracy could be retained. These discoveries have direct consequences for training GNNs over temporal graphs. We provide the code URL and the newly compiled datasets URL for reproducibility and reuse .
Graph neural networks (GNNs) have emerged as the standard method for numerous tasks on graph-structured data such as node classification . However, real-world graphs are often evolving over time and even new classes may arise. We model these challenges as an instance of lifelong learning, in which a learner faces a sequence of tasks and may take over knowledge acquired in past tasks. Such knowledge may be stored explicitly as historic data or implicitly within model parameters. In this work, we systematically analyze the influence of implicit and explicit knowledge. Therefore, we present an incremental training method for lifelong learning on graphs and introduce a new measure based on k-neighborhood time differences to address variances in the historic data. We apply our training method to five representative GNN architectures and evaluate them on three new lifelong node classification datasets . Our results show that no more than 50\% of the GNN's receptive field is necessary to retain at least 95\% accuracy compared to training over the complete history of the graph data. Furthermore, our experiments confirm that implicit knowledge becomes more important when fewer explicit knowledge is available .
R
Current graph
Graph
R
are promising, especially when the entire graph is known for training
have emerged as the standard method for numerous tasks on graph-structured data such as node classification
R
it is not yet clear how to efficiently train GNNs on temporal graphs , where new vertices, edges, and even classes appear over time. We face two challenges : First, shifts in the label distribution (including the appearance of new labels), which require adapting the model
real-world graphs are often evolving over time and even new classes may arise. We model these challenges as an instance of lifelong learning, in which a learner faces a sequence of tasks and may take over knowledge acquired in past tasks. Such knowledge may be stored explicitly as historic data or implicitly within model parameters. In this work, we systematically analyze the influence of implicit and explicit knowledge. Therefore, we present an incremental training method for lifelong learning on graphs and introduce a new measure based on k-neighborhood time differences to address variances in the historic data. We apply our training method to five representative GNN architectures and evaluate them on three new lifelong node classification datasets
D
Second, the growth of the graph, which makes it, at some point, infeasible to train over all vertices and edges. We address these issues by applying a sliding window technique, i.e., we incrementally train GNNs on limited window sizes and analyze their performance. For our experiments, we have compiled three new temporal graph datasets based on scientific publications and evaluate isotropic and anisotropic GNN architectures .
R
both GNN types provide good results even for a window size of just 1 time step. With window sizes of 3 to 4 time steps, GNNs achieve
no more than 50\% of the GNN's receptive field is necessary to retain
R
using the entire timeline
training over the complete history
R
. With window sizes of 6 or 8, at least 99\% accuracy could be retained. These discoveries have direct consequences for training GNNs over temporal graphs. We provide the code URL and the newly compiled datasets URL for reproducibility and reuse
data. Furthermore, our experiments confirm that implicit knowledge becomes more important when fewer explicit knowledge is available