Autodive: An Integrated Onsite Scientific Literature Annotation Tool (2023)

Abstract: Scientific literature is always available in Adobe’s Portable Document Format (PDF), which is friendly for scientists to read. Compared with raw text, annotating directly on PDF documents can greatly improve the labeling efficiency of scientists whose annotation costs are very high. In this paper, we present Autodive, an integrated onsite scientific literature annotation tool for natural scientists and Natural Language Processing (NLP) researchers. This tool provides six core functions of annotation that support the whole lifecycle of corpus generation including i)annotation project management, ii)resource management, iii)ontology management, iv)manual annotation, v)onsite auto annotation, and vi)annotation task statistic. Two experiments are carried out to verify efficiency of the presented tool. A live demo of Autodive is available at http://autodive.sciwiki.cn . The source code is available at https://github.com/Autodive .

## introduction influential applications such as alphafold2 @xcite , and mobile robotic chemist @xcite rely on high-quality databases and domain knowledge, some of which are constructed from scientific literature. the traditional method of building such databases still needs the manual annotation of numerous professionals. with the development of scientific research and the enhancement of interdisciplinary integration, the number of scientific articles has increased explosively, which also requires the annotators who are assigned to build the domain-specific database to have a suitable background and are familiar with annotation of literature data. some research attempts to assist the manual annotation process with the * correspond author intelligent natural language model, such as named entity recognition (ner) and relation identification (ri). the intelligent method can automatically extract knowledge from articles, and form highquality databases after expert proofreading @xcite . however, the performance of the general model in specific fields is not satisfactory, and the construction of a specific intelligent model needs high-quality databases. thus, an easy-to-use annotation tool with a graphical user interface that allows the labeling of text efficiently and consistently is crucial and necessary. annotation tools play a crucial role during the database-making process in the field of biology @xcite , material @xcite , and chemistry @xcite . although the majority of released annotation tools mainly focus on the annotation of multimedia such as image and video, there are still text annotation tools such as annie @xcite , ts-anno @xcite and doccano @xcite . however, most of the tools need to convert the input file from pdf format to plain text, which may cause additional labor costs for resource preparation. moreover, the annotation of specific data required professionals with knowledge of different fields. compared with raw text, annotating directly on pdf documents conforms to the reading habits of the professionals with the original images, tables and layout information and can greatly improve the labeling efficiency of them whose annotation costs are very high. we refer to this need as onsite annotation, which includes the abilities to display and direct annotate on the original pdf documents. to meet the needs of professionals on direct annotation, the new tool should support direct scientific literature annotation in pdf format. that means this tool should display the original literature in pdf format directly, so users can read the complete pdf content in the annotation inter-face, where they annotate the entity and relation of scientific literature through scheduled annotation operation. in this way, this tool retains the layout information of the original pdf literature to fit the reading habits of annotators. different from traditional text annotation projects, scientific annotation projects require annotation tools to process long literature, annotate complex entities and export appropriate annotation results to a formed database. besides onsite annotation, integration is also a valuable factor during the design of the tool. first, the existing domain-specific ontology should be easily integrated which can reduce the time costs of ontology design. second, the tool should be well integrated with existing and new named-entity recognition (ner) models flexibly. third, integration with existing file systems or literature collections, such as pubmend central and self-organized document folders, should also be valued. by integrating with an existing data source, a researcher can deploy this tool locally without leakage of unpublished or copyrighted documents. hence, we propose an open-sourced annotation tool autodive, with its contributions summarized below: ## architecture the overall architecture of autodive is shown in fig. 1 with three layers which are data source, server layer, and frontend. the core component of autodive is the server layer. data source adapter can integrate specific data sources such as pubmed central or file system with autodive. three core server-side engines play important roles. the first is the regular expression parsing engine. it can help autodive extract entities by predefined regular expressions. the second is the ontology management engine. it can load and parse ontology files with owl format and save the designed ontology to owl or self-defined json files. the third engine is the ner model management engine. it is a model base that is extendable and friendly to newly trained ner models. the annotation engine is the bridge between the server-side and the frontend. this engine provides locating and feedback of entities. frontend is implemented mainly using a progressive javascript framework vue.js. as shown in fig. 2 , the complete literature in pdf format is shown on the right part of the annotation page of autodive, where annotators can read the literature and annotate the entity. users click the mouse to select words on the displayed pdf document, and click the right mouse to select the type of entities or relationships. when connecting two annotated entities, users can annotate relation of them. shortcut-key annotation is allowed for the increase of efficiency of the annotation project. all identified or annotated entities and relations are listed in the entity-label-list and relation-label-list with predefined ontology. ## modules autodive integrates management, annotation, and optimization through six modules and is specially built for scientific literature annotation projects. users use the project management module to develop and manage personalized annotation projects. the resource management module allows users to manage their own literature resources pool. the ontology management module pre-defines the knowledge ontology, which is defined and standardized by the annotation project administrator. the basic components for annotating are the manual annotation module and the auto annotation module. trained automatic annotation models are saved in the background in order to improve the productivity of manual annotation work. finally, we created a statistic view module for autodive to represent the overall progress of the annotation project. ## discussion we created autodive, a collaborative scientific literature annotation tool that offers a comprehensive solution for the whole lifecycle of annotation, especially for scientific literature annotation. in addition, we provide automated annotation for annotators and can integrated with a variety of ner models. we found that autodive can not only be utilized for scientific literature, but also for any editable pdf file. also, autodive is released as an open source project under apache 2.0 license. autodive also has some limitations. first, accuracy of ner models. for text annotation projects, a more accurate ner model can ensure the accuracy of auto annotation. second, collision in teamwork. reduce annotation disputes in annotation projects caused by diverse knowledge perspectives between annotators and obtain more accurate annotation data. third, there are pdf files that cannot be changed. some obsolete or illegible pdf documents are difficult to process and cannot be used in the current autodive version. in our future project, we intend to expend the scope of literature annotation to include graphics and tables in addition to text. additionally, autodive will support more file types, including the ability to download in json/conll format and to upload plain text and pictures.