The ACL OCL Corpus: Advancing Open Science in Computational Linguistics (2023)

Abstract: We present ACL OCL, a scholarly corpus derived from the ACL Anthology to assist Open scientific research in the Computational Linguistics domain. Integrating and enhancing the previous versions of the ACL Anthology, the ACL OCL contributes metadata, PDF files, citation graphs and additional structured full texts with sections, figures, and links to a large knowledge resource (Semantic Scholar). The ACL OCL spans seven decades, containing 73K papers, alongside 210K figures. We spotlight how ACL OCL applies to observe trends in computational linguistics. By detecting paper topics with a supervised neural model, we note that interest in “Syntax: Tagging, Chunking and Parsing” is waning and “Natural Language Generation” is resurging. Our dataset is available from HuggingFace (https://huggingface.co/datasets/WINGNUS/ACL-OCL).

## introduction building scholarly corpora for open research accelerates scientific progress and promotes reproducibility in research by providing researchers with accessible and standardized data resources. driven by advancements in natural language processing and machine learning technologies, the computational linguistics (cl) discipline has experienced rapid growth in recent years. this rapid growth underpins the importance of having a scholarly corpus in the cl domain for ensuring sustainable progress. the acl anthology 2 is an important resource that digitally archives all scientific papers in the cl domain, including metadata, pdf files, and supplementary materials. previous scholarly corpora built on it, such as the anthology reference corpus (arc; @xcite and the anthology author network (aan; @xcite , extend its utility by providing full texts, citation and collaboration networks. however, both are becoming obsolete due to their outdated text extraction methods and insufficient updates. we present the acl ocl (or ocl for short), an enriched and contemporary scholarly corpus that builds upon the strengths of its predecessors while addressing their limitations. the ocl corpus includes 73,285 papers hosted by the acl anthology published from 1952 to september 2022. the ocl further provides higher-quality structured full texts for all papers, instead of previous stringformatted ones, enabling richer textual analyses. for instance, higher-quality full texts better foster the development of document-level information extraction tasks @xcite . the structured information in full texts, such as sections and paragraphs, facilitates section-wise tasks such as related work generation @xcite and enables fine-grained linguistic analyses name #doc. text type linked kg fig. peer source domain refseer @xcite 1.0m string citeseerx × partial www multi s2orc @xcite 8.1m structured s2ag × partial multi multi csl @xcite 396k -self × all ccj multi unarxive @xcite 1.9m structured mag × partial arxiv multi acl arc @xcite . in addition, to advance multimodal research such as figure caption generation @xcite , ocl extracts 210k figures from its source documents. furthermore, we link ocl to large-scale scientific knowledge graphs to enrich ocl with external information. in particular, we consider information such as citation data from a larger scholarly database (e.g., semantic scholar) and linkage to other platforms (e.g., arxiv). to showcase the scientific value of the ocl corpus, we illustrate its utility through a downstream application of temporal topic trends @xcite within the cl domain. we first train a pre-trained language model (plm) based topic classification method on a subset of 2,545 scientific papers with ground truth topic labels. we then extrapolate and integrate the model predictions as silver-labeled topic information in the ocl. the contributions of this paper are as follows: ## related work scholarly datasets typically fall into two categories: task-specific and open research-oriented. the former, designed to serve one task, includes selective information of scientific papers such as abstract and citation strings, paired with task-specific outputs such as entities @xcite , summaries @xcite and citation intent labels @xcite . in contrast, open research-oriented scholarly datasets aim to provide comprehensive and fundamental information about scientific papers, such as metadata and full text. the open scholarly datasets not only facilitate researchers in refining their task-specific data but also aid in analyzing the characteristics of scientific papers or groups of them. our work is in line with the open scholarly dataset construction to serve a wide range of applications. similar to other datasets, the ocl features publication metadata, a staple in open scholarly datasets. this can enhance metadata analysis and bibliographic research within the cl domain. a comparison of the ocl with existing datasets, excluding metadata aspects, is presented in inspired by and following s2orc, the ocl provides structured full texts with the scientific documents' discourse structure (i.e., sections), which enables more extensive textual analysis. in contrast to corpora that rely solely on internal papers for cita-tion networks and thus limit their completeness, the ocl is linked to a large knowledge graph to overcome the constraints. furthermore, multi-modal features such as figures are extracted to foster research in document layout and multi-modality. ## the acl ocl corpus we start by crawling the pdf files and metadata from the source acl anthology. we then pass the pdf files through the full-text extraction process. we enhance this data using semantic scholar's api to fetch additional information about ocl papers. ## dataset analysis we present statistical analyses of the ocl corpus, including the distribution of papers across years and linguistic distribution. we further highlight the quality of full texts and citation graph analysis. ## objective topic classification topic information serves as a crucial attribute in retrieving scientific papers. we focus on objective topics @xcite , which are used to denote specific research tasks such as machine translation or text generation. notably, even though authors submit topics during manuscript submission, this information remains invisible on the acl anthology website. we aim to assign the most appropriate objective topic to each cl paper and further explore how this topic information can benefit the development of the cl community. the single-label topic classification setting is adopted in this paper for simplicity; multi-label classification is left to future work. given a scientific document d, with its textual information such as title, abstract, and full text, objective topic classification aims to assign a task topic label @xmath0. l is the topic label set taken from the submission topics (e.g., "generation", "question answering") of the acl confer-ences. the complete list of 21 topics is presented in appendix b. based on the amount of supervised information used for training, we explore three classes of methods for topic classification: unsupervised, semisupervised, and supervised methods. ## downstream applications we previously highlighted one example application of acl ocl, topic trend analysis. by nature, acl ocl is a scholarly corpus aiming to enable and benefit a wide range of research. we now further depict a few existing research directions that could benefit from acl ocl, predicting opportunities enabled by its characteristics. high-quality texts are more suited for taskspecific supervised data. for example, the abstracts and related work texts can be used as direct references for summarization and related work generation (hoang and kan, 2010; hu and wan, 2014) tasks, respectively. ## conclusion we introduce acl ocl, a scholarly corpus aiming to advance open research in the computational linguistics domain. the structured full texts, enriched metadata from semantic scholar academic graph, and figures provided by ocl can benefit existing research tasks as well as enable more opportunities. we highlight the utility of ocl in temporal topic trends analysis in the cl domain. the topics are generated by a trained neural model with a small yet effective scientific topic dataset. by analyzing the topics' popularity, we ask for more attention on the emerging new topics such as "ethics of nlp" and the declining underrepresented topics like "discourse and pragmatics" and "phonology, morphology and word segmentation". in the future, we will work on the data currency of ocl, which aims to keep ocl up-to-date with the acl anthology data. we plan to update ocl by year to keep it alive. the ultimate solution is to provide full-text extraction and information extraction apis to acl anthology, thus hosting the ocl data on acl anthology itself.