G en GO : ACL Paper Explorer with Semantic Features (2024)

Abstract: We present GenGO, a system for exploring papers published in ACL conferences. Paper data stored in our database is enriched with multi-aspect summaries, extracted named entities, a field of study label, and text embeddings by our data processing pipeline. These metadata are used in our web-based user interface to enable researchers to quickly find papers relevant to their interests, and grasp an overview of papers without reading full-text of papers. To make GenGO to be available online as long as possible, we design GenGO to be simple and efficient to reduce maintenance and financial costs. In addition, the modularity of our data processing pipeline lets developers easily extend it to add new features. We make our code available to foster open development and transparency: https://gengo.sotaro.io.

## introduction the rapidly growing number of scientific papers makes it difficult for researchers to keep up-todate with the state of the literature @xcite . scholarly document processing (sdp) aims to support researchers with this challenge by building tools to process knowledge stored in research papers @xcite and has been increasingly drawing attention from academic and industrial communities. together with the recent developments in natural language processing (nlp), a number of powerful technologies are now available in sdp. for instance, automatic scientific paper summarization systems provide short summaries encapsulating the essential points in a paper so that researchers can grasp the overview without reading its abstract or even fulltext @xcite . a series of works on paper representation learning aims to obtain numerical representations of papers that can be used for information retrieval or recommendation @xcite . automatic information extraction systems allow repositories of papers to be organized in a structured manner @xcite . there are also system demonstrations that implement user interfaces to the sdp technologies to make the aforementioned models available to scientists. @xcite introduce a system that lets users consume papers with automatically generated summaries. @xcite is a web-based tool where users can explore different complex concepts and their relations by exploiting scientific knowledge graphs. as the nlp community also faces a large number of publications @xcite with dynamically changing trends @xcite , there are systems which explicitly target this domain. the acl anthology @xcite serves as an essential resource for the community by providing a repository of papers published in acl conferences. @xcite introduce the acl anthology searchbench that implements acl anthology a more fine-grained structured search. ding et al. (2020) device a model-based semantic search in addition to the lexical search to provide a powerful literature search infrastructure. however, currently, there are no works which integrate different types of sdp technologies (e.g., summarization, information retrieval and extraction) in one system for nlp papers. in this paper, we describe our system, dubbed gengo, equipped with various semantic features to help researchers consume a large number of papers efficiently. specifically, gengo combines three types of sdp technologies. (1) multi-aspect summarization: one paper in gengo is coupled with four one-sentence summaries that summarize the overview, challenge, approach, and outcome of a paper. this enables researchers to quickly understand the overview of a paper from different viewpoints. (2) semantic search: our system can retrieve papers that are semantically relevant given user-provided queries. (3) information extraction: gengo automatically extracts technical terms and predicts the topic of a paper to enable fine-grained filtering and search. we design gengo to be efficient and simple using cloud-based software so it requires low maintenance and financial cost, and currently achieves to index 30k+ papers. ## developments in sdp in this section, we review existing papers and commercial products that target processing scholarly documents. the models and datasets used in the development of gengo are marked with * . ## system requirements one pragmatic challenge for a system demonstration project is to keep the system up and running, especially when it is maintained by a few developers with a limited budget. indeed, 10 out of 27 web-based system demonstrations presented at acl 2023 (held in july) are offline after less than one year (at the time of writing, march 2024). we speculate that this is due to (1) maintenance effort and (2) financial costs as mentioned by @xcite . to achieve a long lifespan of our system, we design gengo to minimize the aforementioned two factors. (1) maintenance effort: we minimize the number of servers to be taken care of. specifically, our system does not rely on custom servers that often demand intensive system maintenance. instead, we opt for a server-less architecture by making use of managed cloud-based services. to reduce (2) financial costs: we designed our pipeline not to make any online inferences that require gpus. our data processing pipeline can run fully offline, process each document once, and store the results in a database. the only online inference gengo makes is when to compute a text embedding of a user-provided query. instead of hosting a server to compute embeddings for each user request, we use a lightweight text embedding model and perform the inference on the user's device. because of these design decisions, gengo is currently running without any financial cost from our disposal while indexing more than 30k papers. by using the cost estimator provided by qdrant 4 , we estimate that when the number of indexed papers increases by ten times, it would require around 10 euro per month. ## gengo in this section, we describe the main features of gengo ( §4.1) and a system overview ( §4.2). ## limitation of current system because the metadata used to build the gengo is from the acl anthology project, our system inherits its challenges such as the disambiguation of author names @xcite . in addition, there are several limitations unique to our system. (1) speed and the number of indexed papers: compared to similar services developed by large institutions such as semantic scholar, our system relies on limited computational resources. this currently results in slower response time especially when showing a long list of papers, and also to compensate for the limited size of the data storage, we only index approximately 30% of papers from the whole acl anthology repository. we are also hindered in applying the most powerful models available in our data factory due to the lack of sufficient computational resources. (2) summarization quality: hallucination in text generation remains an open challenge @xcite @xcite . during the development, we observed cases where the generated summaries do not convey the full information of a paper or contain information inconsistent with the corresponding paper. (3) retrieval bias: while the semantic search approach used in our system can work with paraphrases or synonyms, due to the 'blackboxness' of model-based encoders, it remains unclear whether such models have a bias towards retrieving certain styles of texts @xcite . ## conclusion in this paper, we described gengo, a system to explore acl papers with various types of semantically empowered functionalities. our system enables nlp researchers to quickly find relevant papers using semantic search and various fine-grained filters, and grasp paper overviews by reading multiaspect summaries. gengo also provides utility features such as paper recommendations or 'read it later' to further enhance user experience.