  In this work, we study the phenomenon of catastrophic forgetting in the graph
representation learning scenario. The primary objective of the analysis is to
understand whether classical continual learning techniques for flat and
sequential data have a tangible impact on performances when applied to graph
data. To do so, we experiment with a structure-agnostic model and a deep graph
network in a robust and controlled environment on three different datasets. The
benchmark is complemented by an investigation on the effect of
structure-preserving regularization techniques on catastrophic forgetting. We
find that replay is the most effective strategy in so far, which also benefits
the most from the use of regularization. Our findings suggest interesting
future research at the intersection of the continual and graph representation
learning fields. Finally, we provide researchers with a flexible software
framework to reproduce our results and carry out further experiments.
