Predicting Long-Term Citations from Short-Term Linguistic Influence (2022)

Abstract: A standard measure of the influence of a research paper is the number of times it is cited. However, papers may be cited for many reasons, and citation count is not informative about the extent to which a paper affected the content of subsequent publications. We therefore propose a novel method to quantify linguistic influence in timestamped document collections. There are two main steps: first, identify lexical and semantic changes using contextual embeddings and word frequencies; second, aggregate information about these changes into per-document influence parameters by estimating a high-dimensional Hawkes process with a low-rank parameter matrix. The resulting measures of linguistic influence are predictive of future citations. Specifically, the estimate of linguistic influence from the two years after a paper’s publication is correlated with and predictive of its citation count in the following three years. This is demonstrated using an online evaluation with incremental temporal training/test splits, in comparison with a strong baseline that includes predictors for initial citation counts, topics, and lexical features.

## introduction the citation count of a paper is a standard, easily measurable proxy for its influence @xcite . researchers have shown that citation count is strongly correlated with the quality of scientific work (e.g., @xcite , the recognition that a paper or an author gets (e.g., @xcite , or in policy decisions such as assessment of scientific performance (e.g., @xcite . consequently, citation count is a ubiquitously deployed and important measure of a paper with whole subfields of research dedicated to its analysis @xcite . however, papers may be cited (or not cited) for many reasons, and citation count alone is insufficient to explain the emergence and the spread of @xcite [75, 90) 90 percentile bin 0.05 0.10 0.15 extra future log citations ## methodology this section describes our method for estimating the linguistic influence of each document in a times-tamped collection. our work builds on the theory of point process models @xcite , in which the basic unit of data is a set of marked event timestamps. in our case, the events correspond to the use of an innovative word or usage; the mark corresponds to the document in which word or usage appears. to estimate linguistic influence of individual documents, we fit a parametric model in which per-document influence parameters explain the density of events in subsequent documents. we first describe the modeling framework in which these influence parameters are estimated ( § 2.1) and then describe how event cascades are constructed ( § 2.2) from semantic changes ( § 2.2.1) and lexical innovations ( § 2.2.2). ## experimental setup for this study, multilingual bert is used as the contextualizing model even though our data is english papers. this is to handle even those english language papers that have foreign language tokens. specifically, the bert-base-multilingual-uncased model from the hugging face @xcite library is used. 5 the size of the contextualized embeddings is 3144 dimensions after concatenating the final four layers. continued pretraining previous work has shown that the quality of the contextual embed-dings improves when the pretrained bert is further trained on domain-specific text (e.g., @xcite . for this study, we continued to pretrain bert model for 3 epochs to optimize the masked language modeling objective. the probability of masking is set to 15 %. wordpiece aggregation since bert learns subword embeddings by breaking tokens into wordpieces, the embeddings of the wordpieces need to be aggregated to get a representation of a token. this aggregation is done by taking the average of the wordpiece embeddings. 6 data preprocessing non-english papers in the corpus are ignored from the analysis by identifying the language of the papers using langid @xcite . the vocabulary v is constructed by retaining words that appear at least 10 times in the abstracts and do not appear in more than 90 % abstracts. each paper is first segmented by whitespace and then broken into chunks of 200 tokens. only alphabetic tokens are retained. ## conclusion we have presented a new technique for quantifying semantic influence in time-stamped documents. quantitative analysis demonstrates that this measure of semantic influence is strongly correlated with long-term citations a paper receives, and leads to improvement in the prediction of future citations. our tool offers additional granularity in terms of linguistic influence, which can supplement structural measures of influence based on citation counts. though we present quantitative analyses for scholarly documents in computational linguistics, our tool could be applied to scholarly documents in other research areas or to documents such as patents or court opinions where citation counts are considered structural measures of influence. we plan to focus on these applications in the future.