## 2 Related Works
### Text-Description-Based Approach  
Research has shown that pre-trained language models, such as the Transformer [16]-based bidirectional encoder BERT [17], can significantly improve the performance of natural language processing tasks with limited data. [PERSON] [18] introduced a novel data augmentation method that utilizes lyrics to improve the generalization ability of the method and enhance its performance. For long-tail entities and relations in knowledge graphs, traditional methods that utilize structural information often yield randomly initialized entity and relation representations, which leads to a sharp decline in reasoning performance on long-tail entities. To better leverage the rich semantic knowledge in pre-trained language models to represent long-tail data, KG-BERT [3] was proposed as the first model to fine-tune pre-trained language models for knowledge graph completion tasks. This model treats triplets as textual sequences and uses the output probabilities from BERT to judge the truthfulness of triplets. Building upon KG-BERT, Pretrain-KGE [19] introduces a general training framework consisting of three stages: semantic-based fine-tuning, knowledge extraction, and KG training. It learns the structural information of triplets while retaining some of the knowledge from BERT to learn better knowledge graph embeddings. Inspired by masked language modeling (MLM), MEM-KGC [20] is proposed by [PERSON] et al. This method first masks the tail entity and treats the head entity and relation as the context for the tail entity. Then, it predicts the masked entity from all entities, effectively capturing the characteristics of unseen entities. KG-BERT only encodes the textual descriptions of entities and relations, overlooking the structural information in the knowledge graph. To address this, StAR [4] by [PERSON] et al. combines text and structure information. This model adopts a Siamese network structure with a pre-trained language model as the encoder to fully utilize the textual information of entities and relations. To overcome the limited number of negative samples and the issue of some entities lacking textual descriptions, LMKE [5] by [PERSON] et al. introduces contrastive learning into text-based reasoning methods. The embeddings of entities within a batch are used as negative samples for each other, avoiding the extra cost of encoding negative samples. For entities without textual descriptions, LMKE jointly uses entity names, relation names, relation textual descriptions, and other information to represent their embeddings, enriching the representation of entities with missing information. [PERSON] [21] argues that the key to model performance lies in an efficient negative sampling method for contrastive learning. Therefore, this method designs three types of negative sampling: within-batch negative sampling, batch-level negative sampling, and a simplified form of self-negative sampling for hard negatives. These methods have brought new ideas and performance improvements to knowledge graph embedding and completion tasks.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Method Category** & **Key Innovations** \\ \hline \multirow{5}{*}{Distance-based methods} & Introduce various improvements to manage the semantic representation of relations, including translation operations, relation hyperplanes, relation-specific mapping matrices, and complex rotational operations to better handle symmetrical relationships. \\ \cline{2-3}  & Focus on matrix factorization techniques that optimize computational efficiency and parameter count, addressing asymmetry in relations and automating the design of scoring functions. \\ \cline{2-3}  & \\ \cline{2-3}  & Focus on matrix factorization techniques that optimize computational efficiency and parameter count, addressing asymmetry in relations and automating the design of scoring functions. \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison of distance-based and tensor-decomposition-based methods.