## 3 Proposed Model
### Embedding Propagation Layer  
This layer is primarily based on the GCN architecture for recursive embedding propagation, including attention embedding propagation and embedding aggregation. Specifically, the graph attention mechanism is employed to learn the attention weights of neighboring entities for each entity, enabling selective cascading embedding propagation. The entity embeddings are then recursively propagated and aggregated onto the central entity in the KG, ultimately achieving entity representation learning. The detailed structure of the embedding propagation layer is illustrated in Figure 4.  
**Attention Embedding Propagation.** Its inputs are entities, relations, and attribute embeddings. The embedding representation of entity \(s_{i}\) as \(\mathbf{h}_{i,t}\in\mathbb{R}^{d}\) at timestamp \(t\) and the set of its neighbors as \(\mathcal{N}_{i,t}=\{o,v|(s_{i},r,o,t)\in T_{R}\cup(s_{i},r,o,t)\in T_{A}\}\), aims to obtain an updated embedded representation \(\overrightarrow{h}_{i,t}\) of entity \(s_{i}\) by encoding neighbor entity information and attribute information. In this paper, we take the quadruple \((s_{i},r,o,t)\) as an example:  
\[\overrightarrow{h}_{i,t}=\sum_{o\in\mathcal{N}_{i,t}}\pi(s_{i},r,o,t)\mathbf{W}( \mathbf{r}+\mathbf{o}), \tag{2}\]  
where \(W\in\mathbb{R}^{d\times d}\) is a learnable linear transformation that transforms the input embedding into a high-order feature space. \(\pi(s_{i},r,o,t)\) is an attention coefficient representing the importance of entity \(O\) to entity \(s_{i}\).  
Figure 3: Attribute embedding layer detailed process.  
The attention coefficient is used to control the amount of information propagation between the entities, and is computed by the _softmax function_ for all quadruple connected to entity \(s_{i}\):  
\[\pi(s_{i},r,o,t)=\frac{exp(\pi(s_{i},r,o,t))}{\sum\limits_{o^{\prime}\in\mathcal{ N}_{i,t}}exp(\pi(s_{i},r^{\prime},o^{\prime},t))}, \tag{3}\]  
In this paper, the attention coefficients are realized by a simple single-layer feedforward neural network:  
\[\pi(s_{i},r,o,t)=LeakyRelu\left((\mathbf{Wr})^{T}\mathbf{W}(\mathbf{r}+\mathbf{o})\right), \tag{4}\]  
where \(LeakyRelu(\cdot)\) denotes the activation function.  
**Embedding Aggregation.** To learn more stable entity representations and obtain richer semantic information, the last layer of HCAE uses a multi-head attention mechanism. The role of embedding aggregation is to integrate the embedding representations of each head in order to obtain the final embedding representation of the center entity. In this paper, the average aggregation method is used to extend Equation (2):  
\[\overset{\rightarrow}{\mathbf{h}}_{i,t}=LeakyRelu\left(\frac{1}{m}\sum\limits_{j =1}^{m}\sum\limits_{o\in\mathcal{N}_{i,t}}\pi(s_{i},r,o,t)^{j}\mathbf{W}^{j}(\mathbf{ r}+\mathbf{o})\right), \tag{5}\]  
where \(m\) denotes the number of attention heads, \(\pi(s_{i},r,o,t)^{j}\) indicates the \(j\) normalized attention coefficient, and \(\mathbf{W}^{j}\) is a linear transformation.  
Next, multiple layers of embedding propagation are stacked to aggregate features from multi-hop neighboring entities, encoding deeper-level connectivity information to capture semantic information within the high-order connectivity structures in the TKG. The embedding representation of the \(l\) layer of entity \(s_{i}\) is:  
\[\overset{\rightarrow}{\mathbf{h}}_{i,t}(l)=\sum\limits_{o\in\mathcal{N}_{i,t}} \pi(s_{i},r,o,t)\mathbf{W}\Big{(}\mathbf{r}^{(l-1)}+\mathbf{o}^{(l-1)}\Big{)}, \tag{6}\]  
Figure 4: The structure of the embedding propagation layer.