## 3 Cakgc
### Encoder  
The CAKGC model employs a dual encoder architecture. The two encoders are initialized using the same pre-trained language model, BERT, but they do not share parameters. Given a triplet \((h,r,t)\), we consider the case of predicting the tail entity. Specifically, the input of the first encoder is defined as follows in Equation (1):  
\[x_{1}=hh_{des}[CLS]h_{type}r_{des}[MASK] \tag{1}\]  
where \(h\) and \(r\) represent the head entity and relation of a triplet and \(h_{des}\), \(r_{des}\), \(h_{type}\) denote the textual descriptions of the head entity, textual description of the relation, and the category information of the head entity, respectively. Compared to the textual description sequences, category information sequences are typically much shorter. In the context of modeling long sequences, the shorter category information sequences may not provide sufficient contextual information as the model needs to process a large amount of information. In such cases, the model may tend to prioritize processing the textual description information while neglecting the category information, leading to a performance decline.  
To assist the model in better understanding long sequences and guide the model to pay more attention to the category information, we prepend a special token \([CLS]\) before the category information. Additionally, we adopt the Masked Entity Modeling (MEM) approach by replacing the tail entity with a special token \([MASK]\). The input sequence x1 is then fed into the pre-trained language model BERT to obtain the output \(y_{1}\in\mathbb{R}^{H}\) for the first encoder, which represents the embedding of the masked tail entity. This is formally represented as Equation (2).  
\[y_{1}=BERT_{1}(x_{1})_{mask} \tag{2}\]  
Figure 1: Overview of CAKGC.  
Similarly, for the second encoder, we introduce the input of entity category information, which is represented as Equation (3):  
\[x_{2}=tt_{des}[CLS]t_{type} \tag{3}\]  
where \(t\), \(t_{des}\), and \(t_{type}\) represent the tail entity, textual description of the tail entity, and the category of the tail entity, respectively. To emphasize the importance of category information, we also prepend a special token \([CLS]\) before the category information. The input sequence \(x_{2}\) is fed into BERT, and the output \(y_{2}\in\mathbb{R}^{H}\) from the second encoder is obtained. It represents the embedded representation of the tail entity encoded by the pre-trained language model BERT, as shown in Equation (4).  
\[y_{2}=BERT_{2}(x_{2})_{t} \tag{4}\]  
For a given triplet \((h,r,t)\), we utilize two encoders to obtain the embedding representations of the head entity and tail entity. The first encoder encodes the textual description of the head entity, its category information, and the textual description of the relation, resulting in the embedding representation, \(y_{1}\), for the head entity-relation pair. The second encoder jointly encodes the textual description and category information of the tail entity, obtaining the embedding representation, \(y_{2}\), for the tail entity. Thus, \(y_{1}\) and \(y_{2}\) can be regarded as positive sample pairs for the entities. Additionally, we can replace the tail entity, \(t\), with other entities from the same batch and encode them using the second encoder, obtaining multiple negative sample pairs. We aim to maximize the similarity between positive sample pairs and minimize the similarity between negative sample pairs. Through contrastive learning between positive and negative sample pairs, the model can learn more accurate representations of entities and relations. Furthermore, by jointly encoding the textual description and category information, we can significantly enrich the representation of entities, especially for entities with shorter textual descriptions.