# MSEN: A Multi-Scale Evolutionary Network for Modeling the Evolution of Temporal Knowledge Graphs
## 3 The Proposed Method
### Local Memory Encoder
#### 3.3.1 HT-GNN  
Obtaining informative embedding representations for each temporal knowledge graph is critical for effective local encoding. To better capture semantic information, we propose a novel graph neural network-based knowledge graph embedding method called Hierarchical Transfer-Aware Graph Neural Network (HT-GNN). As illustrated in Figure 2, HT-GNN performs sequential aggregation at each layer: entity aggregation, relation aggregation, convolution aggregation, and sum aggregation. The blue nodes serve as aggregation centers. Different aggregation results are merged as input to the next layer. Finally, the outputs from the multi-layer aggregations are combined as the overall model output. Figure 1: Illustration of the proposed MSEN model. Specifically, entity aggregation allows nodes to incorporate semantics from adjacent entities based on co-occurrence, capturing semantic dependencies. The entity aggregation function and weight calculation are:  
\[\left(\mathrm{h}_{\mathrm{o,t}}^{\mathrm{ent}}\right)^{1}=\sigma\left(\sum _{(\mathrm{s},\mathrm{r})\in\mathrm{N}_{\mathrm{t}}}\alpha_{\mathrm{ent}}^{1}w_{ \mathrm{ent}}^{1}\mathrm{h}_{\mathrm{s,t}}^{1}\right), \tag{1}\]  
where \(\mathrm{h}_{\mathrm{o,t}}^{\mathrm{ent}}\) is embedding representation of entity \(\mathrm{o}\). \(\mathrm{h}_{\mathrm{s,t}}^{1}\) is the embedding representation of entity \(\mathrm{s}\) which is a neighboring node of entity \(\mathrm{o}\). \(w_{\mathrm{ent}}^{1}\) is a learnable weight matrix. \(\mathrm{N}_{\mathrm{t}}\) represents the neighboring entities and relations of entity \(\mathrm{o}\). \(\sigma\) is a nonlinear activation function. \(\alpha_{\mathrm{ent}}^{1}\) is the aggregation weight, calculated as follows:  
\[\alpha_{\mathrm{ent}}^{1}=\frac{\exp\left(\mathrm{h}\left(\mathrm{L}\left( \left[\mathrm{h}_{\mathrm{s,t}}^{1}\left\|\mathrm{h}_{\mathrm{o,t}}^{1}\right. \right\|\right)\right)\right)\right)}{\sum_{(\mathrm{s},\mathrm{r})\in\mathrm{ N}_{\mathrm{t}}}\exp\left(\mathrm{h}\left(\mathrm{L}\left(\left[\mathrm{h}_{ \mathrm{s,t}}^{1}\left\|\mathrm{h}_{\mathrm{o,t}}^{1}\right.\right]\right) \right)\right)}, \tag{2}\]  
where \(\mathrm{h}(\cdot)\) is the LeakyRelu activation function. \(\mathrm{L}(\cdot)\) represents the fully connected function. \(\parallel\) represents the concatenation operation. The result of entity aggregation is used as input to relation aggregation. The relation aggregation infuses relation semantics into the entity representations, thereby obtaining the co-occurrence of each node and relation in the knowledge graph. The relation aggregation function and corresponding weight calculation method used in the paper are as follows:  
\[\left(\mathrm{h}_{\mathrm{o,t}}^{\mathrm{rel}}\right)^{1}=\sigma\left(\sum _{(\mathrm{s},\mathrm{r})\in\mathrm{N}_{\mathrm{t}}}\alpha_{\mathrm{rel}}^{1}w_ {\mathrm{rel}}^{1}\mathrm{h}_{\mathrm{r,t}}^{1}\right), \tag{3}\]  
where \(\mathrm{h}_{\mathrm{r,t}}^{1}\) is the embedding representation of the neighboring relation \(\mathrm{r}\) of entity \(\mathrm{o}\). \(w_{\mathrm{rel}}^{1}\) is a learnable weight matrix. \(\alpha_{\mathrm{rel}}^{1}\) is the relation aggregation weight, calculated as follows:  
\[\alpha_{\mathrm{rel}}^{1}=\frac{\exp\left(\mathrm{h}\left(\mathrm{L}\left( \left[\mathrm{h}_{\mathrm{r,t}}^{1}\left\|\mathrm{h}_{\mathrm{o,t}}^{1}\right. \right]\right)\right)\right)}{\sum_{(\mathrm{s},\mathrm{r})\in\mathrm{N}_{ \mathrm{t}}}\exp\left(\mathrm{h}\left(\mathrm{L}\left(\left[\mathrm{h}_{ \mathrm{r,t}}^{1}\left\|\mathrm{h}_{\mathrm{o,t}}^{1}\right.\right]\right) \right)\right)} \tag{4}\]  
The result of relation aggregation is used as input to convolution aggregation. For convolution aggregation, it obtains the hidden relation between entity \(\mathrm{s}\), relation \(\mathrm{r}\), and  
Figure 2: Architecture of the proposed HT-GNN model. entity o, which enriches the semantic information of entities.