Related Work and Citation Text Generation: A Survey (2024)

Abstract: To convince readers of the novelty of their research paper, authors must perform a literature review and compose a coherent story that connects and relates prior works to the current work. This challenging nature of literature review writing makes automatic related work generation (RWG) academically and computationally interesting, and also makes it an excellent test bed for examining the capability of SOTA natural language processing (NLP) models. Since the initial proposal of the RWG task, its popularity has waxed and waned, following the capabilities of mainstream NLP approaches. In this work, we survey the zoo of RWG historical works, summarizing the key approaches and task definitions and discussing the ongoing challenges of RWG.

## introduction academic research is an exploratory activity to solve problems that have never been resolved before. each academic research paper must sit at the frontier of the field and present novelties that have not been addressed by prior work; to convince readers of the novelty of the current work, the authors must perform a literature review to compare their work with the prior work. in natural language processing (nlp), a short literature review is usually conducted under the "related work" section (rws). writing an rws is non-trivial; it is insufficient to simply concatenate generic summaries of prior works. instead, composing a coherent story that connects each related work and the current (citing) work, reflecting the author's understanding of their field, is preferred @xcite . the challenging nature of rws writing makes automatic related work generation (rwg) an academically and computationally interesting problem. rwg is a complex task that involves multiple nlp subtasks, such as retrieval-augmented generation, long document understanding, and queryfocused multi-document summarization. moreover, since most nlp papers have an rws and nlp researchers are natural domain experts for evaluating these rws, the rwg task is an excellent test bed for examining the capability of sota nlp models. rwg also fills a practical need. due to the rapid pace of research publications, including pre-prints that have not yet been peer-reviewed, keeping up to date with the latest work in a research area is very time-consuming. even with daily feed tools, like the semantic scholar research feed foot_0 , researchers still have to curate, read, and digest all the new papers in their feed. thus, there is a need for concise, automatically generated literature reviews that regularly summarize the papers in a user's feed. since @xcite initially proposed the task, the popularity of rwg has waxed and waned, following the capabilities of mainstream nlp approaches: from rule-based to extractive summarization, then to abstractive summarization on the sentence level, and finally to abstractive section-level rwg. currently there is a surge of renewed interest in rwg due to the recent success of large language models (llms). in this work, we survey the zoo of rwg historical works. we find that, surprisingly, most rwg works are not directly comparable because they vary drastically in task definition and simplifying assumptions (section 2), as well as using different input features and representations (section 3). there is no standard benchmark dataset for rwg (section 4), as most works apply custom preprocessing to extract rws or individual citations, reflecting differences in their task definitions. further, many works do not release their models or generated outputs, so it is often impossible for later works to compare against earlier approaches (section 5). finally, we discuss ethical concerns related to rwg, such as plagiarism and non-factual statements, and the potential consequences of fully automatic rwg on the human process of scientific thinking and writing (section 6.3). ## task definition the task definition for rwg has varied as the sota text summarization approach has evolved over time. even where the overall approach is similar (e.g. extractive or abstractive approaches), different assumptions are made with respect to the availability of system inputs and the unit at which an rws is generated ( ## overview of approaches when hoang and kan (2010) proposed the rwg task, they identified three main steps: (1) finding relevant documents, (2) identifying the salient aspects of these documents with respect to the current work; (3) generating a topic-biased summary. in practice, all existing works skip the document retrieval step by using the gold cited paper list in the target rws. at a high level, the methodologies of most extractive, citation-level and section-level abstractive rwg approaches are similar within their respective categories: extractive approaches focus on the salience step and simply concatenate the extracted sentences to form the summary, while abstractive approaches focus on directly generating the summary, often without explicitly modeling salience. in this section, we do not give an exhaustive description of all methodologies, but highlight some common features and design perspectives from the overall body of rwg work (summarized in ## datasets despite the twenty published works on rwg, there is no standard benchmark dataset for the task. as we discussed in section 2, most rwg works define their own version of the task; they also create their own datasets, adapted to their particular task definition. in this section, we describe the most commonly used sources of scientific articles ( ## conclusions and discussion having surveyed the field of rwg from the perspectives of task definition, approach, datasets, and evaluation methods, we conclude by identifying three main challenges in modern rwg and make recommendations for future work in this area.