LLMRank: Enhancing Large Language Models for Unsupervised Keyphrase Extraction with a Candidate Graph Approach
Authors: Haoran Ding, Xiao Luo


Reviewer paper summaries: This paper proposes a novel unsupervised keyphrase extraction method called LLMRank. The work combines Large Language Models (LLMs) with a graph-based approach to improve keyphrase extraction performance. Specifically, LLMRank first uses LLMs to generate many candidate keyphrases, then represents these candidates as nodes in a graph, establishes edges between nodes based on co-occurrence relationships, and finally ranks keyphrases using the graph structure. The authors evaluated the method on four public datasets and compared it with 17 baseline models.
Summary of strengths: 1. The method innovatively combines LLMs and graph methods, showing the potential to overcome LLMs' limitations in processing long texts.
2. As an unsupervised method, LLMRank doesn't require labeled data, reducing application costs.
3. It fully utilizes LLMs' semantic understanding and text generation strengths.
4. Comparisons with numerous baselines across multiple datasets demonstrate a thorough experimental design.
5. The authors have made the source code publicly available, facilitating research reproducibility and further improvements.
Summary of weaknesses: 1. The paper lacks a profound explanation of LLMRank's theoretical basis and doesn't provide a theoretical justification for the method's effectiveness.
2. Using LLMs to generate numerous candidate phrases may incur high computational costs. The paper doesn't discuss the method's efficiency.
3. There's no exploration of how key parameters (e.g., number of candidate phrases, graph construction method) affect the results.
4. Compared with multiple baselines, it lacks comparison with the latest LLM-based keyphrase extraction methods.
5. The paper doesn't detail how the method performs on documents of different types and lengths.
Comments, suggestions and typos: see Weaknesses
Overall assessment scores: 2.5