Hierarchical Prototype Network for Continual Graph Representation Learning

1 Introduction21 Graph representation learning aims to pursue a meaningful vector representation of each node so as22 to facilitate downstream applications such as node classification , link prediction , etc . Traditional23 methods are developed based on graph statistics [ 23 ] or hand-crafted features [ 3 , 16 ] . Recently,24 a great amount of attention has been paid to graph neural networks ( GNNs ) , such as graph con-25 volutional network ( GCNs ) [ 12 ] , GraphSAGE [ 10 ] , Graph Attention Networks ( GATs ) [ 31 ] , and26 their extensions [ 34 , 6 , 41 , 14 , 7 , 24 , 38 ] . This is because they can jointly consider the feature and27 topological information of each node . Most of these approaches , however , focus on static graphs and28 can not generalize to the case when new categories of nodes are emerging.29 In many real world applications , different categories of nodes and their associated edges ( in the form30 of subgraphs ) are often continuously emerging in existing graphs . For instance , in a citation network31 [ 27 , 32 , 20 ] , papers describing new research areas will gradually appear in the citation graph ; in a32 co-purchasing network such as Amazon [ 4 ] , new types of products will continuously be updated to33 the graph . Given these facts , how to incorporate the feature and topological information of new nodes34 in a continuous and effective manner such that performance over existing nodes is uninterrupted is a35 critical problem to investigate.36 To address this issue , various types of continual learning approaches can be considered . Existing37 continual learning techniques fall into three main categories , i.e. , regularization-based methods that38 penalize ( or reward ) their model objectives so as to maintain satisfactory performance on previous39 tasks [ 11 , 9 , 26 ] , e.g. , Learning without Forgetting ( LwF ) [ 15 ] and Elastic Weight Consolidation40 ( EWC ) [ 13 ] ; memory-replay based methods that constantly feed a model with representative data41 Submitted to 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Do not distribute . or exemplars of previous tasks to prevent them from being forgotten [ 18 , 28 , 2 , 5 , 8 ] , e.g. , Gradient42 Episodic Memory ( GEM ) [ 18 ] ; and parametric isolation based methods that adaptively introduce43 new parameters for new tasks and avoid the existing parameters of previous tasks being drastically44 changed [ 25 , 36 , 35 , 33 ] . Although these approaches exhibited promising performance in mitigating45 the problem of catastrophic forgetting in different applications , e.g. , image classification , action46 recognition , and reinforcement learning , they are not suitable for continual graph representation47 learning since both the feature information and topological structure of the target node need to be48 considered appropriately.49 More recently , Zhou et al . [ 39 ] proposed to store a set of representative experience nodes in a buffer50 and replay them along with new tasks ( categories ) to prevent forgetting existing tasks ( categories ) .51 The buffer , however , only stores node features and ignores the topological information of graphs.52 Liu et al . [ 17 ] developed topology-aware weight preserving ( TWP ) that can preserve the topological53 information of existing graphs . However , its design hinders the capability of learning topology on54 new tasks ( categories ) . Note that continual graph representation learning is essentially different from55 dynamic graph works which mainly concern time dependent graphs in which nodes and ( or ) edges56 change over time [ 37 , 21 , 40 , 19 ] . Therefore , the methods developed for dynamic graphs can not be57 directly applied to this task.58 A desired learning system for continual graph representation learning is to continuously grasp59 knowledge from new categories of emerging nodes and capture their topological structures without60 interfering with the learned knowledge over existing graphs . To this end , we present a completely61 novel framework , i.e. , Hierarchical Prototype Networks ( HPNs ) , to continuously extract different62 levels of abstract knowledge ( in the form of prototypes ) from graph data such that new knowledge63 will be accommodated while earlier experience can still be well retained . Within this framework,64 representation learning is simultaneously conducted to avoid catastrophic forgetting , instead of65 considering these two objectives separately . Specifically , based on the assumption that each node66 can be decomposed into basic atomic characteristics belonging to a set of attributes ( e.g. , gender,67 nationality , hobby , etc . ) and the relationship between a pair of nodes can be categorized into different68 types ( e.g. , trust or distrust in a social network ) , we develop the Atomic Feature Extractors ( AFEs ) 69 to decompose each node into two sets of atomic embeddings , i.e. , atomic node embeddings which70 encode the node feature information and atomic structure embeddings which encode its relations to71 neighboring nodes within multi-hop . Next , we present Hierarchical Prototype Networks to adaptively72 select , compose , and store representative embeddings with three levels of prototypes , i.e. , atomic-73 level , node-level , and class-level . Given a new node , only the relevant AFEs and prototypes in each74 level will be activated and refined , while others are uninterrupted . Eventually , each node can be75 represented with a tri-level prototypes which encode its feature as well as structure information from76 different abstract levels and can be used for downstream tasks such as node classification . Finally , we77 provide the theoretical analysis for the memory consumption upper bound of HPNs and its continual78 learning capability . To summarize , the main contributions of our work include:79 • We present a novel framework , i.e. , Hierarchical Prototype Networks ( HPNs ) , to contin-80 uously extract different levels of abstract knowledge ( in the form of prototypes ) from the81 graph data such that new knowledge will be accommodated while earlier experience can be82 well retained.83 • We provide the theoretical analysis for the memory consumption upper bound of HPNs and84 its continual learning capability.85 • Our experiment results on eight different public datasets demonstrate that the proposed86 HPNs not only achieve state-of-the-art performance , exhibiting good continual learning87 capability , but also use less parameters ( more efficient ) . For instance , on OGB-Products88 dataset that contains more than 2 million nodes and 47 categories of nodes , HPNs achieves89 around 80 % accuracy with only thousands of parameters.90 2 Hierarchical Prototype Networks91 In this section , we first state the problem we aim to study and the notations . Then we present92 Hierarchical Prototype Networks ( HPNs ) that consist of two core modules , i.e. , Atomic Feature93 Extractor ( AFEs ) and Hierarchical Prototype Networks ( HPNs ) , as shown in Figure 1 . AFEs serve to94 extract a set of atomic features from the given graph , and the HPNs aim to select , compose , and store95 the representative features in the form of different levels of prototypes . During the training stage,96 each node will only refine the relevant AFEs and prototypes of the model without interfering with the97 irrelevant parts ( i.e. , to avoid catastrophic forgetting ) . In the test stage , the model will activate the98 relevant AFEs and prototypes to perform the inference.99 2.1 Problem Statement and Notations100 We study continual learning on graphs that have new categories of nodes and associated edges ( in the101 form of subgraphs ) emerging in a continuous manner . In the context of continual learning , assuming102 we have a sequence of p tasks { T i|i = 1 , ... , p } , in which each task T i aims to learn a satisfied103 representation for a new subgraph Gi consisting of nodes belonging to some new categories . A104 desired model should maintain its performance on all previous tasks after being successively trained105 on the sequence of p tasks from T 1 to T p.106 For simplicity , we omit the subscripts in this section . Full notations will be used in the theoretical107 analysis . Each graph G consists of a node set V = { vi|i = 1 , ... , N } with N nodes and an edge set108 E = { ( vi , vj ) } denoting the connections of nodes in V. Each node vi can be represented as a feature109 vector x ( vi ) ∈ Rdv that encodes node attributes , e.g. , gender , nationality , hobby , etc . The set of l-hop110 neighboring nodes of vi is defined as N l ( vi ) , with N 0 ( vi ) = { vi } .111 2.2 Atomic Feature Extractors112 Based on the assumption that different nodes can be decomposed into basic atomic characteristics113 belonging to a set of attributes ( e.g. , gender , nationality , hobby , etc . ) and the relations between a114 pair of nodes can also be categorized into different types ( e.g. , trust or distrust in a social network ) ,115 we develop Atomic Feature Extractors ( AFEs ) to consider two different sets of atomic embeddings,116 i.e. , atomic node embeddings which encode the node features and atomic structure embeddings117 that encode its relations to neighbors within multi-hop . Specifically , to ensure that each node can118 be represented as different combinations of a subset of atomic features , AFEs are designed as119 learnable linear transformations AFEnode = { Ai ∈ Rdv×da |i ∈ { 1 , ... , la } } and AFEstruct = { Rj ∈120 Rdv×dr |j ∈ { 1 , ... , lr } } where Ai and Rj are real matrices to encode atomic node and structure121 information , respectively . la and lr denotes the cardinality of AFEnode and AFEstruct , respectively.122 Given a node v , a set of atomic node embeddings is obtained by applying AFEnode to the feature123 vector x ( v ) :124 EnodeA ( v ) = { xT ( v ) Ai|Ai ∈ AFEnode } . ( 1 ) To obtain atomic structure embeddings , the multi-hop neighboring nodes of v have to be considered.125 We first uniformly sample a fixed number of vertices from 1-hop up to h-hop neighborhood , i.e.,126 Nsub ( v ) ⊆ ⋃ l∈ { 1 , ... , h } N l ( v ) . Then these selected nodes are embedded via projection matrices in127 AFEstruct to encode different types of interactions with the target node v:128 EstructA ( v ) = { xT ( u ) Ri|Ri ∈ AFEstruct , u ∈ Nsub } . ( 2 ) Finally , the complete atomic feature set of target node v is:129 EA ( v ) = EnodeA ( v ) ∪ EstructA ( v ) . ( 3 ) Note that Ai and Ri are designed to generate different types of atomic features . To ensure that , we130 impose a divergence loss on AFEs to ensure they are be uncorrelated with each other and thus can131 map features to different subspaces:132 Ldiv = ∑ i 6=j ATi Aj + ∑ i 6=j RTi Rj . ( 4 ) 2.3 Hierarchical Prototype Networks133 With the atomic features extracted based on AFEs , hierarchical prototype networks ( HPNs ) will select,134 compose , and store representative features in the form of different levels of prototypes as shown in135 Figure 1 . This is mainly achieved by refining existing prototypes and creating new prototypes only136 when necessary . Specifically , HPNs will produce three different levels of prototypes , i.e. , atomic-137 level prototypes ( A-prototypes ) , node-level prototypes ( N-prototypes ) , and class-level prototypes138 ( C-prototypes ) . From atomic-level to class-level , the prototypes denote abstract knowledge of the139 graph at different scales which is analog to the feature maps of convolutional neural networks at140 different layers.141 We first introduce how HPNs can refine existing prototypes . For each task that contains certain142 categories of nodes , instead of using all atomic embeddings generated by existing AFEs , HPNs only143 select a small and fixed number of AFEs from both AFEnode and AFEstruct which are more relevant144 to the given task . In this way , only the relevant AFEs are refined while others remain uninterrupted.145 Specifically , as shown in Figure 1 , given a node from an incoming subgraph , each AFE is used to146 generate an embedding . Those AFEs with embeddings that are closer to existing A-prototypes are147 deemed as more confident ones and chosen . Formally , we first obtain EnodeA ( v ) and EstructA ( v ) via Eq.148 ( 1 ) and Eq . ( 2 ) , respectively . Then , we calculate the maximum cosine similarity between atomic149 embeddings of each AFE ( ei ) and the A-prototypes as:150 SimMAXidi = max p ( eTi p ‖ei‖2‖p‖2 ) , ei ∈ EidA ( v ) , p ∈ PA , ( 5 ) where id ∈ { node , struct } , i ranges from 1 to la ( or lr ) , and PA is the atomic prototype set containing151 all A-prototypes . After that , we sort the AFEs in a descending order according to SimMAXidi as152 AFEsortnode = { Ai′ ∈ Rdv×da |i′ ∈ { 1 , ... , la } } and AFE sort struct = { Rj′ ∈ Rdv×dr |j′ ∈ { 1 , ... , lr } } .153 Finally , we select the top l′a and top l ′ r ranked AFEs from these two sets as AFE select node and AFE select struct ,154 respectively . l′a and l ′ r are fixed hyperparameters with l ′ a ≤ la and l′r ≤ lr . The atomic embeddings155 generated by these selected AFEs are denoted as EselectA ( v ) .156 Based on EselectA ( v ) , HPNs then starts to distill representative features , which is conducted by refining157 existing prototypes and creating new prototypes simultaneously . A matching process is first conducted158 between the EselectA ( v ) and PA to recognize the atomic features that are compatible with exiting A-159 prototypes and those ones to be accommodated with new A-prototypes . Formally , we measure the160 cosine similarity between elements in EselectA ( v ) and elements in PA as161 SimE→A ( v ) = { eTi p ‖ei‖2‖p‖2 |ei ∈ EselectA ( v ) , p ∈ PA } . ( 6 ) The atomic embeddings that are compatible with existing A-prototypes are these ones with cosine162 similarity not less than a certain threshold tA to have at least one existing A-prototype , i.e.,163 Eold ( v ) = { ei| ∃p ∈ PA s.t . eTi p ‖ei‖2‖p‖2 > tA } . ( 7 ) Eold ( v ) collects a set of atomic embeddings satisfying the previous condition and can be used to164 refine PA. To this end , a distance loss Ldis is computed to enhance the cosine similarity between165 each ei ∈ Eold ( v ) and its corresponding A-prototype pi ∈ PA , i.e.,166 Ldis = − ∑ ei∈Eold ( v ) eTi pi ‖ei‖2‖pi‖2 ( 8 ) By minimizing Ldis , not only the existing A-prototypes in PA will get refined , the atomic embeddings167 will also be closer to ‘ standard ’ A-prototypes.168 Algorithm 1 : Learning Procedure for HPNs . Input : Task sequence : { T1 , ... , Tp } , HPNs 1 for T ← 1 to p do 2 Get the data of the current task : V , E , X ( V ) = { x ( v ) |v ∈ V } . 3 Select AFEselectnode and AFE select struct . 4 Compute L = HPNs ( V , X ( V ) , E ) . 5 L = HPNs ( V , X ( V ) , E ) . 6 Optimize L. Output : updated HPNs Next , we discuss how to deal with the atomic embeddings that are not close to any existing prototypes,169 i.e. , Enew ( v ) = EselectA ( v ) \Eold ( v ) or Enew ( v ) = { ei| ∀p ∈ PA , eTi p ‖ei‖2‖p‖2 < tA } .170 Contrary to Eold ( v ) , atomic embeddings in Enew ( v ) are regarded as new atomic features of the171 corresponding AFEs . In this case , new prototypes should be generated to accommodate them.172 Considering that very similar embeddings may exist in Enew ( v ) and cause HPNs to create redundant173 prototypes , we first filter Enew ( v ) into E′new ( v ) to keep only the representative ones such that174 ∀ei , ej ∈ E′new ( v ) , eTi ej ‖ei‖2‖ej‖2 < tA . ( 9 ) Then , E′new ( v ) is included into PA as new A-prototypes , which will be further refined in the future.175 PA = PA ∪ E′new ( v ) . ( 10 ) After generating new prototypes , the matching will be conducted to get a new SimE→A ( v ) in which176 each element is not less than tA . Then each element in EselectA ( v ) is assigned a closest A-prototype177 according to SimE→A ( v ) , and each node is associated with a set of atomic prototypes A ( v ) .178 To map A ( v ) to high level prototypes so as to obtain hierarchical prototype representations . A ( v ) is179 firstly mapped to a N-prototype denoting the overall features of v. We assume that N-prototypes lie180 in a dn dimensional space and a fully connected layer is applied to transform A ( v ) into the new space181 EN ( v ) = FCA→N ( a1 ⊕ · · · ⊕ al′a+l′r ) , ∀ai ∈ A ( v ) , where ⊕ denotes the concatenation operator.182 With EN ( v ) , we then find a matching N-prototype or establish a new one , which is similar to the183 process at atomic level except that the threshold is set as tN , instead of tA . Learning class-level184 prototypes from node-level prototypes is same except that we set the matching threshold as tC .185 Finally , the hierarchical prototype representations of the target node is contained in the following set186 PH ( v ) = A ( v ) ∪ N ( v ) ∪ C ( v ) . ( 11 ) Note that A ( v ) contains multiple A-prototypes denoting atomic features of v from different aspects.187 N ( v ) and C ( v ) only contain one N-prototype and one C-prototype , representing the overall character-188 istics of v and the common characteristics shared by the community containing v , respectively.189 2.4 Learning Objective190 The obtained hierarchical prototypes for each node are first concatenated into a unified vector and191 then pass through a fully connected layer FC to obtain a c ( the number of classes ) dimensional192 feature vector , i.e. , FC ( h1 ⊕ · · · ⊕ hl′a+l′r+2 ) , ∀hi ∈ PH ( v ) . In this paper , we aim to perform node193 classification . Therefore , based on the c dimensional feature vector and the softmax function σ ( · ) ,194 we can estimate the label with ŷi = σ ( FC ( h1 ⊕ · · · ⊕ hl′a+l′r+2 ) ) i where i is the index of class . To195 perform node classification , with the output predictions ŷi and the target label yi ∈ { 1 , 2 , ... , c } , the196 corresponding classification loss is given by197 Lcls = c∑ i=1 −yi log ( ŷi ) , ( 12 ) 198 which is essentially the cross entropy loss function . Note that besides node classification , PH ( v ) may199 also be used for other tasks based on different objective functions . In this paper , we focus on node200 classification and the overall loss of HPNs is:201 L = Ldis + Ldiv + Lcls . ( 13 ) During the training stage , subgraphs with different tasks ( containing different categories of nodes ) are202 continuously fed to HPNs . Note that unlike topology-aware weight preserving ( TWP ) method [ 17 ] ,203 HPNs do not require task indicator for training and test , and therefore is more practical for real-world204 continual graph representation learning applications.205 2.5 Theoretical Analysis206 In this subsection , we provide the theoretical upper bound for the memory consumption and analyze207 how the model configuration would affect HPNs ’ capacity in dealing with different tasks . Both208 theoretical results are justified and analyzed in the experiments . Only the main results are provided209 here , while the detailed proof and analysis are given in Appendix.210 We first show that the numbers of different prototypes are upper bounded by the number of atomic211 feature extractors and the dimension of the prototypes . Specifically , we have:212 Theorem 1 ( Upper bounds for numbers of prototypes ) . Given the notations defined in HPNs , the213 upper bound for the number of A-prototypes na can be given by214 nA 6 ( la + lr ) max N S ( da , N , 1− tA ) , ( 14 ) and the upper bounds for the number of N-prototypes and the C-prototypes are:215 nN 6 max N S ( dn , N , 1− tN ) and nC 6 max N S ( dc , N , 1− tC ) ( 15 ) where S ( n , N , t ) is the spherical code defined on a n dimensional hypersphere ( details in Appendix ) .216 Theorem 1 provides an upper bound for the memory consumption of HPNs . In our experiments , we217 show that the number of parameters for most baseline methods are even higher than this upper bound.218 Besides memory consumption , the more important problem for a continual learning model is the219 capability to maintain memory on previously learned tasks . Based on our model design , we formulate220 this as : whether learning new tasks affect the representations the model generates for old task data.221 We give explicit definitions on tasks and task distances based on set theory ( in Appendix ) , then222 construct a bound to indicate what configuration would the model have to ensure this capability.223 Theorem 2 ( Task distance preserving ) . For HPNs trained on consecutive tasks T p and T p+1.224 If lada + lrdr > ( lr + 1 ) dv and W is column full rank , then as long as tA < λmin ( lr +225 1 ) dist ( Vp , Vp+1 ) , learning on T p+1 will not modify representations HPNs generate for data from226 T p , i.e . catastrophic forgetting is avoided.227 In Theorem 2 , λi is eigenvalues of the WTW , where W is a matrix constructed via AFEs ( details in228 Appendix ) . dv , da and dr are dimensions of data and two kinds of atomic embeddings . The bound in229 this theorem is not tight , as the tight bound would be dependant on the specific dataset properties.230 But this informs us that either the number of AFEs or the dimension of the prototypes has to be large231 enough to ensure that data from two tasks can be well separated in the representation space.232 According to Theorem 1 , the upper bound of the memory consumption is dependent on S ( da , N , tA ) ,233 S ( dn , N , tN ) , and S ( dc , N , tC ) . As S ( n , N , t ) grows fast with n , we prefer larger number of AFEs234 with smaller prototype dimensions . We also empirically demonstrate this in Section 3.6 . Besides , the235 upper bound proposed in Theorem 1 is explicitly computed and compared to experimental results.For236 both theorems , proofs and detailed explanations are included in Appendix.237 3 Experiments238 In the experiments , we answer the following six questions : ( 1 ) Whether HPNs can outperform239 state-of-the-art approaches ? ( 2 ) How does each component of HPNs contribute to its performance ? 240 ( 3 ) Whether HPNs can memorize previous tasks after learning each new task ? ( 4 ) Are HPNs sensitive241 to the hyperparameters ? ( 5 ) Whether the theoretical results can be empirically verified ? ( 6 ) Whether242 the learned prototypes can be interpreted via visualization ? 243 3.1 Datasets244 To assess the effectiveness of the proposed HPNs , we consider 8 datasets which include 3 citation245 networks ( Cora [ 27 ] , Citeseer [ 27 ] , OGB-Arxiv [ 32 , 20 ] ) , 3 web page networks ( Wisconsin , Cornell,246 Texas ) [ 22 ] , 1 actor co-occurence network ( Actor ) [ 22 ] , and 1 product co-purchasing networks247 ( OGB-Products [ 4 ] ) . Detailed statistics about these datasets are provided in the Appendix.248 Among these datasets , the results of 4 datasets , i.e. , Cora , Citeseer , OGB-Arxiv ( 169,343 nodes,249 1,166,243 edges ) , and OGB-Products ( 2,449,029 nodes , 61,859,140 edges ) , are reported in the paper250 and the results of other 4 datasets are available in the Appendix.251 3.2 Experimental Setup and Evaluation Metrics252 To perform continual graph representation learning with new categories of nodes continuously253 emerging , we adopt a class-incremental scheme for all datasets . Each new task brings a subgraph with254 new categories of nodes and associated edges , e.g. , task 1 contains classes 1 and 2 , task 2 contains255 new classes 3 and 4 , etc . Each model is trained on a sequence of tasks , and the performance will be256 evaluated on all previous tasks . Specifically , we adopt accuracy mean ( AM ) and forgetting mean ( FM ) 257 as metrics for evaluation . After learning on all tasks , the AM and FM are computed as the average258 accuracy and the average accuracy decrease on all previous tasks . Negative FM indicates the existence259 of forgetting , zero FM denotes no forgetting and positive FM denotes positive knowledge transfer260 between tasks . For HPNs , we set da = dn = dc = 16 , la = lr = 22 , and h = 2 . The threshold tA,261 tN , and tC are selected by cross validation on { 0.01 , 0.05 , 0.1 , 0.15 , 0.2 , 0.25 , 0.3 , 0.35 , 0.4 } . The262 experiments on the important hyperparameters are provided in Section 3.6 . All experiments are run263 on an Nvidia Titan Xp GPU . Full implementation details are in Appendix , and the code is available264 in supplementary materials.265 3.3 Comparisons with Baseline Methods266 We compare HPNs with various baseline methods . Experience Replay based GNN ( ERGNN ) [ 39 ] 267 and Topology-aware Weight Preserving ( TWP ) [ 17 ] are developed for continual graph representation268 learning . The others approaches , including Elastic Weight Consolidation ( EWC ) [ 13 ] , Learning with-269 out Forgetting ( LwF ) [ 15 ] , Gradient Episodic Memory ( GEM ) [ 18 ] , and Memory Aware Synapses270 ( MAS ) [ 1 ] ) are popular continual learning methods for Euclidean data . All the baselines are imple-271 mented based on three popular backbone models , i.e. , Graph Convolutional Networks ( GCNs ) [ 12 ] ,272 Graph Attentional Networks ( GATs ) [ 31 ] , and Graph Isomorphism Network ( GIN ) [ 34 ] .273 Note that Joint training ( Join . ) in Table 1 does not represent continual learning . It allows a model to274 access data of all tasks at any time and thus is often used as an upper bound for continual learning . [ 29 ] .275 In Table 1 , we observe that regularization based approaches , e.g. , EWC and TWP , generally obtain276 lower forgetting , but the accuracy ( AM ) is limited by the constraints . However , the forgetting277 problem of regularization based methods will become increasingly severe when the number of278 tasks is relatively large , as shown in Section 3.5 . Memory replay based methods such as GEM279 achieve better performance without using any constraint . However , the memory consumption is280 higher ( Section 3.7 ) . HPNs significantly outperform all baselines without inheriting their limitations.281 Compared to regularization based methods , HPNs do not impose constraints to limit the model ’ s282 expressiveness , therefore the performance is much better . Compared to memory replay based methods,283 HPNs do not only perform better but also are memory efficient as shown in Section 3.7 . Joint training284 ( Join . ) achieves comparable performance to HPNs on small datasets but is significantly worse on large285 OGB datasets . This is because joint training ( Join . ) is a multi-task setting , inter-task interference286 may cause negative transfer , which is not obvious on small datasets with only a few tasks but287 becomes prominent on large datasets with tens of tasks . In HPNs , different tasks can choose different288 combinations of the parameters and thus task interference is dramatically alleviated.289 3.4 Ablation Study290 We conduct ablation studies on different levels of prototypes and different combinations of three loss291 terms . In Table 2 , we show the performance of HPNs when A- , N- , and C-Prototypes are gradually292 added ( Cora dataset ) . We notice both AM and FM of HPNs increase when higher level prototypes293 are considered . This suggests that high level prototypes can enhance the model ’ s performance and294 robustness against forgetting.The effect of different combinations of loss terms are shown in Table 3.295 The first three rows show that adding Ldiv or Ldis with Lcls may slightly improve the performance.296 By jointly considering these three terms , the performance ( AM ) can be further improved . This is297 because Ldiv pushes different AFEs away from each other and Ldis makes the prototypes of each298 AFE be more close to its output . Jointly considering Ldiv and Ldis with Lcls can make the prototype299 space better separated as shown in Section 3.8.300 3.5 Learning Dynamics301 For continual learning , it is important to memorize previous tasks after learning each new task . To302 measure this , instead of directly measuring the average accuracy on previous tasks which may mix up303 the accuracy change caused by forgetting and task differences , we develop a new metric , i.e. , average304 retaining score ( ARS ) , to address this problem . Specifically , after learning on a task T i , the ratio305 between the model ’ s accuracy on a previous task T i−m and its accuracy on T i−m after it had been306 just learned on T i−m is defined as the retaining ratio . Then the ARS is the average retaining ratio of307 all previous tasks after learning a new task.308 Figure 3 ( left ) shows the ARS change of HPNs and two baselines . GAT represents the models without309 continual learning techniques . TWP+GAT is the best baseline in terms of forgetting . GAT forgets310 quickly , while TWP significantly alleviates the forgetting problem for GAT . But as more tasks come311 in , the forgetting of TWP+GAT increases . As different tasks require different parameters , TWP+GAT312 ( regularization based ) is seeking a trade off between old and new tasks . With more new tasks,313 TWP+GAT tends to gradually adapt to new tasks and forget old ones . On contrary , HPNs maintain314 the ARS very well . This is because HPNs learn prototypes to denote the common basic features and315 learning new tasks does not hurt the parameters for old tasks . New tasks can be handled with new316 combinations of the existing basic prototypes . If necessary , new prototypes can be established for317 more expressiveness.318 3.6 Parameter Sensitivity319 As discussed in Section 2.5 , the number of AFEs and the prototype dimensions are key factors320 in determining the continual learning capability and memory consumption . Here , we conduct321 experiments with different number of AFEs and prototype dimensions to justify the theoretical322 results . We keep the dimensions of different prototypes equal and the number of two types of AFEs323 equal for simplicity.324 As shown in Figure 2 ( a ) and ( b ) , larger dimensions and the number of AFEs yield better AM and325 FM , which is consistent with Theorem 2 . Besides , AM is mostly determined by the number of AFEs326 since HPNs compose prototypes with different AFEs to represent each target node . The number of327 possible combinations determines its expressiveness . Considering the above results and the bound328 ( Theorem 1 ) for the number of prototypes , using large number of AFEs and small dimension can329 ensure both high performance and low memory usage , as verified in Section 3.7.330 We also evaluate the effectiveness of HPNs when prototype thresholds vary from 0.01 to 0.4 . Here,331 we set tA = tN = tC for simplicity . In Figure 2 ( c ) and ( d ) , we observe that the performance332 ( AM and FM ) of HPNs are generally stable when tA varies and slightly better when tA is between333 0.2 and 0.3 . This is because when tA is too small or too large , we will have too many or too less334 prototypes ( consistent with Theorem 1 ) as shown in Figure 3 ( middle ) , which may cause the problem335 of overfitting or underfiting.336 3.7 Memory Consumption337 We compare memory consumption of different methods , as well as a explicitly theoretical memory338 upper bound , with the baselines on OGB-Products ( the largest dataset ) . We also show the actual339 memory consumption of HPNs in the process of continual learning.340 In Table 4 , even on the dataset with millions of nodes and 23 tasks , HPNs can accommodate all tasks341 with a small amount of parameters . Besides , the dynamic change of parameter amount is shown in342 Figure 3 ( right ) . The red dashed line denotes the theoretical upper bound ( 6,163 ) , and the computation343 details are included in Appendix . In Figure 3 ( right ) , we notice the actual memory usage of HPNs is344 much lower than the upper bound . Moreover , even the upper bound is among the lowest for memory345 consumption compared to baselines . The model we use here is the same as the one in Section 3.3346 3.8 Visualization347 To show that HPNs can generate interpretable prototype representations , we apply t-SNE [ 30 ] to348 visualize the node representations of the Cora dataset ( test set ) after learning each task . As shown in349 Figure 4 , each task contains two classes corresponding to ( red , blue ) , ( green , salmon ) , and ( purple,350 orange ) , as new tasks come in gradually , the representations are consistently well separated , which351 will be beneficial for downstream tasks.352 4 Conclusion353 In this paper , we proposed Hierarchical Prototype Networks ( HPNs ) , to continuously extract different354 levels of abstract knowledge ( in the form of prototypes ) from streams of tasks on graph representation355 learning . The performance of HPNs is both theoretically and experimentally justified . In the future,356 we will apply HPNs to more application scenarios like link prediction , multi-label classification,357 anomaly detection , etc.358 References359 [ 1 ] Rahaf Aljundi , Francesca Babiloni , Mohamed Elhoseiny , Marcus Rohrbach , and Tinne Tuyte-360 laars . Memory aware synapses : Learning what ( not ) to forget . In Proceedings of the European361 Conference on Computer Vision ( ECCV ) , pages 139–154 , 2018.362 [ 2 ] Rahaf Aljundi , Min Lin , Baptiste Goujaud , and Yoshua Bengio . Gradient based sample selection363 for online continual learning . In Advances in Neural Information Processing Systems , pages364 11816–11825 , 2019.365 [ 3 ] Smriti Bhagat , Graham Cormode , and S Muthukrishnan . Node classification in social networks.366 In Social network data analytics , pages 115–148 . Springer , 2011.367 [ 4 ] K. Bhatia , K. Dahiya , H. Jain , P. Kar , A. Mittal , Y. Prabhu , and M. Varma . The extreme368 classification repository : Multi-label datasets and code , 2016.369 [ 5 ] Lucas Caccia , Eugene Belilovsky , Massimo Caccia , and Joelle Pineau . Online learned continual370 compression with adaptive quantization modules . In International Conference on Machine371 Learning , pages 1240–1250 . PMLR , 2020.372 [ 6 ] Jie Chen , Tengfei Ma , and Cao Xiao . Fastgcn : fast learning with graph convolutional networks373 via importance sampling . arXiv preprint arXiv:1801.10247 , 2018.374 [ 7 ] Ming Chen , Zhewei Wei , Zengfeng Huang , Bolin Ding , and Yaliang Li . Simple and deep graph375 convolutional networks . In International Conference on Machine Learning , pages 1725–1735.376 PMLR , 2020.377 [ 8 ] Aristotelis Chrysakis and Marie-Francine Moens . Online continual learning from imbalanced378 data . In International Conference on Machine Learning , pages 1952–1961 . PMLR , 2020.379 [ 9 ] Mehrdad Farajtabar , Navid Azizan , Alex Mott , and Ang Li . Orthogonal gradient descent for380 continual learning . In International Conference on Artificial Intelligence and Statistics , pages381 3762–3773 . PMLR , 2020.382 [ 10 ] Will Hamilton , Zhitao Ying , and Jure Leskovec . Inductive representation learning on large383 graphs . In Advances in neural information processing systems , pages 1024–1034 , 2017.384 [ 11 ] Heechul Jung , Jeongwoo Ju , Minju Jung , and Junmo Kim . Less-forgetting learning in deep385 neural networks . arXiv preprint arXiv:1607.00122 , 2016.386 [ 12 ] Thomas N Kipf and Max Welling . Semi-supervised classification with graph convolutional387 networks . arXiv preprint arXiv:1609.02907 , 2016.388 [ 13 ] James Kirkpatrick , Razvan Pascanu , Neil Rabinowitz , Joel Veness , Guillaume Desjardins,389 Andrei A Rusu , Kieran Milan , John Quan , Tiago Ramalho , Agnieszka Grabska-Barwinska , et al.390 Overcoming catastrophic forgetting in neural networks . Proceedings of the national academy of391 sciences , 114 ( 13 ) :3521–3526 , 2017.392 [ 14 ] Guohao Li , Matthias Muller , Ali Thabet , and Bernard Ghanem . Deepgcns : Can gcns go as393 deep as cnns ? In Proceedings of the IEEE International Conference on Computer Vision , pages394 9267–9276 , 2019.395 [ 15 ] Zhizhong Li and Derek Hoiem . Learning without forgetting . IEEE transactions on pattern396 analysis and machine intelligence , 40 ( 12 ) :2935–2947 , 2017.397 [ 16 ] David Liben-Nowell and Jon Kleinberg . The link-prediction problem for social networks.398 Journal of the American society for information science and technology , 58 ( 7 ) :1019–1031,399 2007.400 [ 17 ] Huihui Liu , Yiding Yang , and Xinchao Wang . Overcoming catastrophic forgetting in graph401 neural networks . arXiv preprint arXiv:2012.06002 , 2020.402 [ 18 ] David Lopez-Paz and Marc ’ Aurelio Ranzato . Gradient episodic memory for continual learning.403 In Advances in neural information processing systems , pages 6467–6476 , 2017.404 [ 19 ] Yao Ma , Ziyi Guo , Zhaocun Ren , Jiliang Tang , and Dawei Yin . Streaming graph neural405 networks . In Proceedings of the 43rd International ACM SIGIR Conference on Research and406 Development in Information Retrieval , pages 719–728 , 2020.407 [ 20 ] Tomas Mikolov , Ilya Sutskever , Kai Chen , Greg Corrado , and Jeffrey Dean . Distributed408 representations of words and phrases and their compositionality . arXiv preprint arXiv:1310.4546,409 2013.410 [ 21 ] Giang Hoang Nguyen , John Boaz Lee , Ryan A Rossi , Nesreen K Ahmed , Eunyee Koh , and411 Sungchul Kim . Continuous-time dynamic network embeddings . In Companion Proceedings of412 the The Web Conference 2018 , pages 969–976 , 2018.413 [ 22 ] Hongbin Pei , Bingzhe Wei , Kevin Chen-Chuan Chang , Yu Lei , and Bo Yang . Geom-gcn:414 Geometric graph convolutional networks . arXiv preprint arXiv:2002.05287 , 2020.415 [ 23 ] Bryan Perozzi , Rami Al-Rfou , and Steven Skiena . Deepwalk : Online learning of social repre-416 sentations . In Proceedings of the 20th ACM SIGKDD international conference on Knowledge417 discovery and data mining , pages 701–710 , 2014.418 [ 24 ] Yu Rong , Wenbing Huang , Tingyang Xu , and Junzhou Huang . Dropedge : Towards deep419 graph convolutional networks on node classification . In International Conference on Learning420 Representations , 2019.421 [ 25 ] Andrei A Rusu , Neil C Rabinowitz , Guillaume Desjardins , Hubert Soyer , James Kirkpatrick,422 Koray Kavukcuoglu , Razvan Pascanu , and Raia Hadsell . Progressive neural networks . arXiv423 preprint arXiv:1606.04671 , 2016.424 [ 26 ] Gobinda Saha and Kaushik Roy . Gradient projection memory for continual learning . In425 International Conference on Learning Representation , 2021.426 [ 27 ] Prithviraj Sen , Galileo Namata , Mustafa Bilgic , Lise Getoor , Brian Galligher , and Tina Eliassi-427 Rad . Collective classification in network data . AI magazine , 29 ( 3 ) :93–93 , 2008.428 [ 28 ] Hanul Shin , Jung Kwon Lee , Jaehong Kim , and Jiwon Kim . Continual learning with deep429 generative replay . In Advances in neural information processing systems , pages 2990–2999,430 2017.431 [ 29 ] Gido M Van de Ven and Andreas S Tolias . Three scenarios for continual learning . arXiv preprint432 arXiv:1904.07734 , 2019.433 [ 30 ] Laurens Van der Maaten and Geoffrey Hinton . Visualizing data using t-sne . Journal of machine434 learning research , 9 ( 11 ) , 2008.435 [ 31 ] Petar Veličković , Guillem Cucurull , Arantxa Casanova , Adriana Romero , Pietro Lio , and Yoshua436 Bengio . Graph attention networks . arXiv preprint arXiv:1710.10903 , 2017.437 [ 32 ] Kuansan Wang , Zhihong Shen , Chiyuan Huang , Chieh-Han Wu , Yuxiao Dong , and Anshul438 Kanakia . Microsoft academic graph : When experts are not enough . Quantitative Science439 Studies , 1 ( 1 ) :396–413 , 2020.440 [ 33 ] Mitchell Wortsman , Vivek Ramanujan , Rosanne Liu , Aniruddha Kembhavi , Mohammad441 Rastegari , Jason Yosinski , and Ali Farhadi . Supermasks in superposition . arXiv preprint442 arXiv:2006.14769 , 2020.443 [ 34 ] Keyulu Xu , Weihua Hu , Jure Leskovec , and Stefanie Jegelka . How powerful are graph neural444 networks ? arXiv preprint arXiv:1810.00826 , 2018.445 [ 35 ] Jaehong Yoon , Saehoon Kim , Eunho Yang , and Sung Ju Hwang . Scalable and order-robust446 continual learning with additive parameter decomposition . In International Conference on447 Learning Representation , 2020.448 [ 36 ] Jaehong Yoon , Eunho Yang , Jeongtae Lee , and Sung Ju Hwang . Lifelong learning with449 dynamically expandable networks . arXiv preprint arXiv:1708.01547 , 2017.450 [ 37 ] Wenchao Yu , Wei Cheng , Charu C Aggarwal , Kai Zhang , Haifeng Chen , and Wei Wang.451 Netwalk : A flexible deep embedding approach for anomaly detection in dynamic networks . In452 Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & 453 Data Mining , pages 2672–2681 , 2018.454 [ 38 ] Xikun Zhang , Chang Xu , and Dacheng Tao . On dropping clusters to regularize graph convolu-455 tional neural networks . 2020.456 [ 39 ] Fan Zhou , Chengtai Cao , Ting Zhong , Kunpeng Zhang , Goce Trajcevski , and Ji Geng . Continual457 graph learning . arXiv preprint arXiv:2003.09908 , 2020.458 [ 40 ] Lekui Zhou , Yang Yang , Xiang Ren , Fei Wu , and Yueting Zhuang . Dynamic network embedding459 by modeling triadic closure process . In Proceedings of the AAAI Conference on Artificial460 Intelligence , volume 32 , 2018.461 [ 41 ] Difan Zou , Ziniu Hu , Yewen Wang , Song Jiang , Yizhou Sun , and Quanquan Gu . Layer-462 dependent importance sampling for training deep and large graph convolutional networks . In463 Advances in Neural Information Processing Systems , pages 11247–11256 , 2019.464 Checklist465 1 . For all authors ... 466 ( a ) Do the main claims made in the abstract and introduction accurately reflect the paper ’ s467 contributions and scope ? [ Yes ] 468 ( b ) Did you describe the limitations of your work ? [ Yes ] In Conclusion , and in the469 theoretical part of Appendix.470 ( c ) Did you discuss any potential negative societal impacts of your work ? [ No ] Our work471 solves the continual graph representation learning problem . As far as we know , there is472 no potential negative societal impacts of our work.473 ( d ) Have you read the ethics review guidelines and ensured that your paper conforms to474 them ? [ Yes ] 475 2 . If you are including theoretical results ... 476 ( a ) Did you state the full set of assumptions of all theoretical results ? [ Yes ] Details are477 included in Appendix.478 ( b ) Did you include complete proofs of all theoretical results ? [ Yes ] Proofs are in Appendix479 3 . If you ran experiments ... 480 ( a ) Did you include the code , data , and instructions needed to reproduce the main ex-481 perimental results ( either in the supplemental material or as a URL ) ? [ Yes ] Code is482 included in the supplementary materials.483 ( b ) Did you specify all the training details ( e.g. , data splits , hyperparameters , how they484 were chosen ) ? [ Yes ] Details are included in Appendix.485 ( c ) Did you report error bars ( e.g. , with respect to the random seed after running experi-486 ments multiple times ) ? [ Yes ] In all tables and in Figure 2487 ( d ) Did you include the total amount of compute and the type of resources used ( e.g. , type488 of GPUs , internal cluster , or cloud provider ) ? [ Yes ] Relevant details are included in489 Appendix490 4 . If you are using existing assets ( e.g. , code , data , models ) or curating/releasing new assets ... 491 ( a ) If your work uses existing assets , did you cite the creators ? [ Yes ] 492 ( b ) Did you mention the license of the assets ? [ Yes ] We mentioned this in the dataset493 detail part in Appendix.494 ( c ) Did you include any new assets either in the supplemental material or as a URL ? [ Yes ] 495 The code of our model is included in the supplementary materials.496 ( d ) Did you discuss whether and how consent was obtained from people whose data you ’ re497 using/curating ? [ Yes ] We mentioned this in the dataset detail part in Appendix.498 ( e ) Did you discuss whether the data you are using/curating contains personally identifiable499 information or offensive content ? [ Yes ] We mentioned this in the dataset detail part in500 Appendix.501 5 . If you used crowdsourcing or conducted research with human subjects ... 502 ( a ) Did you include the full text of instructions given to participants and screenshots , if503 applicable ? [ N/A ] 504 ( b ) Did you describe any potential participant risks , with links to Institutional Review505 Board ( IRB ) approvals , if applicable ? [ N/A ] 506 ( c ) Did you include the estimated hourly wage paid to participants and the total amount507 spent on participant compensation ? [ N/A ] 508

Summary: The paper addresses the continuous representation learning problem for graph data. The main goal is to learn emerging novel node categories, and simultaneously maintain the learned knowledge over existing graphs. In the proposed solution called Hierarchical Prototype Networks (HPNs), several levels of general knowledge are extracted, in the form of atomic, node and class-level prototypes. A theoretical analysis for the memory consumption and continual learning capability is presented. Empirical study is conducted  on eight different public datasets, including large-scale datasets with millions-scale nodes.