Task Definition and Integration For Scientific-Document Writing Support (2021)

Abstract: With the increase in the number of published academic papers, growing expectations have been placed on research related to supporting the writing process of scientific papers. Recently, research has been conducted on various tasks such as citation worthiness (judging whether a sentence requires citation), citation recommendation, and citation-text generation. However, since each task has been studied and evaluated using data that has been independently developed, it is currently impossible to verify whether such tasks can be successfully pipelined to effective use in scientific-document writing. In this paper, we first define a series of tasks related to scientific-document writing that can be pipelined. Then, we create a dataset of academic papers that can be used for the evaluation of each task as well as a series of these tasks. Finally, using the dataset, we evaluate the tasks of citation worthiness and citation recommendation as well as both of these tasks integrated. The results of our evaluations show that the proposed approach is promising.

## introduction when writing a scientific paper, it is important to search for relevant papers and cite them appropriately. however, despite the importance of this requirement, the recent sharp increase in published scientific papers is making it difficult for researchers to comprehensively carry out this process. consequently, much work has been devoted to developing systems that support the writing of scientific papers. for example, some studies have attempted to summarize papers on a particular subject @xcite @xcite . the creation of knowledge graphs of scientific papers has also been pro-posed @xcite , and @xcite proposed an automatic content-analysis method by extracting the semantic relations of entities in abstracts. other studies have focused on citation recommendation @xcite and generation of citation text @xcite . using the database of pubmed foot_0 papers, @xcite proposed recommending citations on the basis of keywords as well as the contents of a paper. @xcite proposed the generation of citation text, and @xcite proposed a classification model for the task of judging whether a sentence requires citation (citation worthiness). although many reports have been presented and an abundance of effort has been expended on data creation @xcite @xcite , each previous study has focused on a particular problem in scientificwriting support and has been performed independently using its own specific dataset. therefore, we do not yet know whether these investigations can be successfully pipelined nor how to ascertain the overall performance of a system that can comprehensively recommend citations. consequently, it is currently impossible to verify that the technologies centered around scientific-paper writing are actually helpful in comprehensively supporting real-world scientific-paper writing. in this paper, we first define a series of tasks related to scientific-paper writing that can be pipelined. then, we create a dataset 2 of academic papers that can be used for the evaluation of each task in scientific-paper writing as well as a series of these tasks. finally, using the dataset, we evalu-ate the individual tasks of citation worthiness and citation recommendation as well as the integrated task composed of these two individual tasks. experimental results show that our task setting and the dataset can be successfully used for scientificpaper writing support. ## handling "related work" section in a scientific paper, the section generally called "related work" is important for situating one's research in the field and clarifying the new contribution of the proposed work. however, the task of writing the related work section is timeconsuming because one needs to read through many papers in related areas and carefully cite them. due to this cost, much work has been directed to improving the efficiency of this process. at the beginning stages of this line of research, we saw many studies aimed at helping authors understand the gist of a paper, that is, preparing a summary of the paper highlighting important points such as objective, problem, and methods @xcite . there have also been studies that consider how a paper is cited in summarizing the paper in question @xcite . the summarization of scientific papers continues to be an important research focus @xcite . however, capturing the summarization of a particular paper in isolation would obviously not produce a universal solution when facing the abundance of papers that are available to readers. recent years have seen an increase in work related to citation recommendation, and this work has been greatly aided by the availability of largescale article data in electronic form. such studies have mainly focused on the papers that one should cite due to their authority and relevance based on keywords @xcite . recently, some studies have focused on recommending papers that might be overlooked by limiting the scope to authority and relevance. such methods utilize a citation network and more fine-grained content similarity, making it possible to identify specific papers that should be cited @xcite . moreover, @xcite proposed a method for citation recommendation by categorizing relevant papers on the basis of their data, methods, and problems. in our approach, we list tasks related to scientific-paper writing and include the task of citation recom- ## listing of tasks we listed tasks related to citation that can cover the main phases in scientific-paper writing: (i) when we conceive an idea, (ii) when we obtain research results and are ready to situate the work, and (iii) when we finalize the related work section. figure 1 shows how scientific-paper writing can be supported in each phase. at (i), which happens at the beginning of research, one is apprehensive that the conceived idea may not be original and thus feels the need to perform a survey of related work. in this situation, it is desired that the scientific-paper writing support system recommends relevant papers (fig. 1 (1)) using as input the research problem and its approach, which are typically written in the paper's abstract. at (ii), which is done in the mid-point stage of paper writing, there may be cases when citations are not appropriate or missing. therefore, it is necessary to provide support for the detection of missing references (fig. 1 (2)), the detection of inappropriate citations (fig. 1 (3)), and the recommendation of suitable citations (fig. 1 (4 )). at (iii), which is when the author applies finishing touches to the paper, support for tailoring the related work section would be appropriate (fig. 1 (5 )), such as how the references should be categorized and how they should be presented. tasks ( 1 )-( 5 ) in the figure can be broken down into more fine-grained tasks as follows: as can be seen in the above listing, the tasks follow the chronological order of how a paper is written in its research phases. they can be pipelined. these tasks have mostly been identified and tackled in previous studies, but they have been researched separately. the list of tasks includes (3)-2, which we newly conceived in this work; in pipelining the paper-writing process, we considered this a useful sub-task for citation allocation. ## data creation after having defined the tasks, we created a dataset for the evaluation of the individual tasks and, moreover, the integrated (pipelined) tasks. for this purpose, we use the same data as source. ## experiment using the task data, we evaluated baseline performance for these tasks. the aim of the experiment is to show the feasibility of our approach, that is, to test the performance of individual tasks and the integrated task using the same dataset. if this were successful, it would mean that our approach is effective for supporting various phases in scientificpaper writing. ## summary and future work in this paper, to achieve better support of scientific-paper writing, we first defined a series of tasks that can be pipelined. then, focusing on the tasks of citation worthiness, citation allocation, and sentence-citation pair classification, we created a dataset of academic papers that could be used for the evaluation of each task as well as an integrated series of the tasks. we showed experimental results for citation worthiness, citation allocation, and sentence-citation pair classification for individual tasks as well as the case when these tasks are combined. our series of experimental results shows the feasibility of our approach. we also showed the current performance using the same dataset. future work includes creating data for other tasks and performing experiments with them as well as their combinations in pipelined tasks. we will also consider the use of domain-specific pretrained language models, such as scibert @xcite , in order to improve performance. furthermore, we plan to perform a human-in-theloop evaluation in which a system supports researchers in their various writing phases. finally, it would also be useful to improve the accuracy of the tasks we tackled in this paper.