What shall we read : the article or the citations? - A case study on scientific language understanding (2023)

Abstract: The number of scientific articles is increasing tremendously across all domains to such an extent that it has become hard for researchers to remain up-to-date. Evidently, scientific language understanding systems and Information Extraction (IE) systems, with the advancement of Natural Language Processing (NLP) techniques, are benefiting the needs of users. Although the majority of the practices for building such systems are data-driven, advocating the idea of “The more, the better”. In this work, we revisit the paradigm - questioning what type of data : text (title, abstract) or citations, can have more impact on the performance of scientific language understanding systems.

## introduction scientific databases 1 are accumulating a large amount of literature to such an extent that it is getting overwhelming @xcite and practically impossible to be up-to-date for researchers. several recent works have looked into building intelligent systems for tasks such as ad-hoc based retrieval, conversational-agents, recommendation, summarization, document search, and re-ranking, as shown by the advances in the area of neural information retrieval(ir) and biomedical text mining @xcite @xcite . given the unstructured nature, the lengthiness and other metadata (such as citations information), in the scientific documents, the key question that arises when dealing with scientific information extraction is what type of data can be helpful to build high-performance models : text and/or citation information ? we investigate the informativeness of different type of features : text-only features, graph-only features and text-graph features via the document classification task. additionally, we further explore impact of increasing the amount of text features. the main contribution of our work is to benchmark the effectiveness of the three type of features via various machine learning and deep learning based models for document classification task. ## related work this work investigates the impact of different text and graph-based features for document classification task. we briefly discuss the works which are related to our study by grouping them into the following categories : neural ie & transfer learning. the effectiveness of more text for text-ranking and text-retrieval tasks has been studied previously @xcite . several works @xcite @xcite have also investigated at the importance of features from different sections in scientific articles using methods such as traditional @xcite and weighted word-count @xcite . in parallel, transfer learning @xcite @xcite has received a lot of attention with domain specific pre-trained language models (plms) for various downstream tasks such ner, relation extraction, question answering, document classification for scientific and biomedical text mining. citation graphs. viewing data as a graph has been positively explored for various nlp tasks @xcite involving knowledge graphs @xcite and ontologyintegrated models @xcite , showcasing the existing informativeness in graphs. we explore graph representation learning @xcite @xcite combining the structural information of graph with text-features in order to enhance the learning of information extraction models. ## results & discussion random vs features comparing with random features, we notice the impact of text-only features (tfidf-c/m, plm), graph-only features (dw) and combination of text+graph features (tfidf+dw, gs). ## conclusion in this work, we provide a benchmark to study the informativeness of different features : text-based and citation information-based for document classification. we show that the models perform better with the abstract text and we show that the information contained in the citation graph can be useful in addition to the text based features. in future, we would like to extend our study to investigate the relevance of other sections in scientific articles by combining graph models with plms. additionally, the dynamic nature of the citation graphs can be interesting to understand the evolution and change points phenomena to study novelty and discovery in scientific citation graphs.