Enhanced Knowledge Graph Reasoning with LLM as a Discriminator: Overcoming Sparsity through Augmentation
Authors: Tingting Yi, Jiang Zhong, Yinhong, Qizhu Dai


Reviewer paper summaries: This paper addresses the issue of knowledge graph sparsity which hampers the performance of knowledge graph embedding models in knowledge graph reasoning tasks. The authors propose a novel framework that generates auxiliary triples for sparse entities based on their structural and textual descriptions. These triples are then filtered by a fine-tuned large language model acting as a discriminator, ensuring only high-confidence triples are used to augment the KG. Additionally, the framework employs an attention-based message aggregation mechanism to prioritize important neighbor information and a dynamic loss modification strategy to adjust the intensity of KG augmentation during training. The proposed method is evaluated on the FB15k-237N and CoDEx-S datasets and shows improvements over baselines.
Summary of strengths: 1. The paper introduces a novel framework that combines auxiliary triples construction and a large language model as a discriminator to enriches the structural information of sparse entities.

2. The paper introduces a dynamic loss modification strategy that adjusts the intensity of KG augmentation during training. It allows the model to concentrate on more relevant information, thereby improving the overall learning process.
Summary of weaknesses: 1. Although the integration of large language models (LLMs) enhances the model's performance in KGR tasks, the improvement is not significantly greater than that achieved by some existing baselines. Additionally, the computational cost associated with fine-tuning LLMs appears to be substantially higher.

2. The approach involves fine-tuning large language models, which can be computationally expensive and may not scale well to very large KGs. It would be beneficial for the authors to provide more analysis and discussion on this.
Comments, suggestions and typos: 1. While the LLM discriminator aims to filter out low-confidence triples, there is still a risk of introducing noise into the KG. More analysis and discussion on this might be helpful to better demonstrate the effectiveness of the proposed method.
Overall assessment scores: 2