M ulti C ite: Modeling realistic citations requires moving beyond the single-sentence single-label setting (2022)

Abstract: Citation context analysis (CCA) is an important task in natural language processing that studies how and why scholars discuss each others’ work. Despite decades of study, computational methods for CCA have largely relied on overly-simplistic assumptions of how authors cite, which ignore several important phenomena. For instance, scholarly papers often contain rich discussions of cited work that span multiple sentences and express multiple intents concurrently. Yet, recent work in CCA is often approached as a single-sentence, single-label classification task, and thus many datasets used to develop modern computational approaches fail to capture this interesting discourse. To address this research gap, we highlight three understudied phenomena for CCA and release MULTICITE, a new dataset of 12.6K citation contexts from 1.2K computational linguistics papers that fully models these phenomena. Not only is it the largest collection of expert-annotated citation contexts to-date, MULTICITE contains multi-sentence, multi-label citation contexts annotated through-out entire full paper texts. We demonstrate how MULTICITE can enable the development of new computational methods on three important CCA tasks. We release our code and dataset at https://github.com/allenai/multicite .

## introduction citations connect the current paper to the broader discourse of science (e.g., @xcite , help signal future impact and uses (e.g., @xcite , and, in downstream applications, can aid in summarizing a work's contributions (e.g., @xcite @xcite . the study of the role and purpose of citations, known as citation context analysis (cca @xcite , has uncovered traces of how ideas have influenced a field, collaboration and competition among peers, and trends in scientific fields. computational approaches to cca have largely focused on classifying the intent or purpose of a particular citation. with the advent of deep neural models, recent cca research efforts have focused on increasing the size of the published resources @xcite @xcite . however, larger data sets have come at the expense of oversimplifying the rich discourse patterns surrounding citations, especially as complexity of annotation is often traded for large-scale data collection. the aforementioned recent large-scale resources, for instance, have approached cca as a single-sentence, single-label classification task, overlooking the richness and nuance with which citations are used in discourse. contributions. in this work, we aim to fuel and inspire research into new or understudied tasks in cca and their respective computational methods motivated by complex phenomena in scholarly citations. our contributions are three-fold. ## understudied phenomena in cca using examples, we highlight and motivate three understudied phenomena in cca, showing that each reflects natural ways in which authors cite. multi-sentence contexts. while a citation appears in a particular sentence, its discussion may span that sentence and beyond. for example, consider the sentence: this sentence alone provides background information about a previous approach and consequently, one could describe its function as background from that text. however, the subsequent sentence continues the discussion of the citation: only by including this sentence can we identify the underlying intent of the authors: the cited publication is used as motivation for the presented research. through our annotation process ( §4) we find that 17.1% of citation contexts involve multiple sentences. thus, we argue that correctly modeling the precise scope of a citation -that is, its associated context -is necessary for understanding the discourse role it plays in the citing paper. ## multicite: a new resource for cca author/ year concept † size context? multi-label? dense sampling? pride and knoth (2020) purpose & influence 11,233 single sentence ✗ ✗ cohan et al. (2019) intent 11,020 single sentence ✗ ✗ ravi et al. (2018) sentiment 8,925 single sentence ✗ ✗ tuarob et al. (2019) algorithm's function 8,796 3 sentences ✗ ✗ athar (2011) sentiment 8,736 single sentence ✗ ✗ abu-jbara et al. (2013) purpose & polarity 3,271 variable @xmath0. (2006) function 2,829 surrounding paragraph ✗ ✗ jochim and schütze (2012) citation facets 2,008 did not annotate contexts ✗ ✗ jurgens et al. (2018) function 1,969 surrounding 300 chars ✗ ✗ athar and teufel (2012) sentiment 1,741 variable up to 4 sents 1 label/sent. ✗ multicite (this work) function 12,653 variable, per-label ✓ ✓ ## citation context classification the first task we consider makes use of both the multi-sentence and multi-label phenomena described in §2: multi-label classification. one can view this as the multi-label extension to the traditional multi-class classification task common in prior work (see figure 2 ), but we show experimentally that models using variable-length gold contexts yield superior results. this motivates (1) a shift in computational research away from assuming fixed-window (often 1-sentence) inputs, and (2) additional investment in understudied tasks like citation context identification ( §6). ## identifying citation contexts motivated by the need for variable-length gold contexts, the second task we consider is citation context identification @xcite , which has seen little research since the advent of neural models. we demonstrate experimentally how multicite can be used to train modern neural models for this understudied task, thereby setting first transformer baselines to support future study. ## evidence-based assessment of citing-cited paper relationships dense citations to a single reference reveal a multifaceted relationship between the citing and cited papers. the dense annotations in multicite allow development of cca methods that model these document-level relationships, e.g., assessing holistically how the cited work has influenced the design or outcome of the citing work. consider an application scenario where a user wants to know why a given paper cites another. a hypothetical system might provide a paper-level assessment of their relationship and citation contexts as supporting evidence from the citing paper's full text. motivated by this aspirational application, we propose a cca task requiring document-level understanding of dense citations: answering questions about citing-cited paper relationships by operating across multiple mentions within paper full-text. figure 3 provides a schematic of this task, which demonstrates the novel research potential of multicite at supporting higher-level cca, and by casting cca as a form of question answering (qa), highlights compatibility with modern attempts on scientific qa @xcite . ## conclusion aiming to inspire novel research in cca, we have acknowledged the existence of three understudied phenomena, and present multicite, a novel resource that is both the largest corpus of citation contexts to-date and captures all three phenomena. we employ multicite in three experiments demonstrating the importance of these phenomena, establishing strong baselines, and showcasing ways of conducting novel cca. in the future, we intend to increase diversity of the dataset by extending it to cover more disciplines of science. we make all code and data publicly available at https://github.com/allenai/multicite .