The Normalized Impact Index for Keywords in Scholarly Papers to Detect Subtle Research Topics (2020)

Abstract: Mainly due to the open access movement, the number of scholarly papers we can freely access is drastically increasing. A huge amount of papers is a promising resource for text mining and machine learning. Given a set of papers, for example, we can grasp past or current trends in a research community. Compared to the trend detection, it is more difficult to forecast trends in the near future, since the number of occurrences of some features, which are major cues for automatic detection, such as the word frequency, is quite small before such a trend will emerge. As a first step toward trend forecasting, this paper is devoted to finding subtle trends. To do this, the authors propose an index for keywords, called normalized impact index, and visualize keywords and their indices as a heat map. The authors have conducted case studies using some keywords already known as popular, and we found some keywords whose frequencies are not so large but whose indices are large.

## introduction thanks to the recent open access movement, we can freely access to a huge amount of papers on scholarly repositories, such as institutional repositories maintained by academic institutions. according to irus-uk, foot_0 there exits about 2m items on more than 200 repositories in the uk, as of may 2020. according to nii, foot_1 there exist more than 2.4m full-text papers on 734 institutional repositories in japan, as of march 2020. in addition to institutional repositories, we also have disciplinary repositories, such as arxiv. 3 we can also use a global aggregation servie, which collects papers on repositories. for exam-ple, core foot_3 collects papers from more than one thousand data providers in about 150 countries, and provides search apis, dump files, and search facility for collected papers @xcite . the latest dump file provided by core contains 123m metadata items, 85.6m abstracts, and 9.8m full text papers. some commercial publishers also began to provide apis for automatic processing. 5 basically, items on scholarly repositories are readable pdf files. when research results were published on paper, research papers were final outcomes of the researches. in case of digital media, however, contents of the papers can be an input for automatic processing. we can find many researches which use scholarly papers as input for computer algorithms. for example, some entities, like dataset names, used in papers are automatically extracted @xcite , and papers are used to predict research impacts of a new given paper @xcite and to predict new materials @xcite . the final goal of our research is to forecast popular trends in the near future. a typical method for this is to use a clustering algorithm, which is unsupervised learning, and divides target items into groups based on a predefined distance metric. some approaches use clustering algorithms to divide words in papers into groups, such as the topic model @xcite . once we introduce a distance metric to data, a target data item is defined as a point in the space defined by the metric, and thus we can compare similarities between any two points. in this sense, this approach uses an absolute distance. there also exit relative approaches, like network structures, in which we know that two items are adjancent. in par-ticular, we can naturally construct multiple network structures from papers, like networks of authors, citations, words, and their combinations @xcite . however, these researches assume that there are already a number of publications @xcite . in this sense, these approaches are for topic detection, not for topic forecast. in this paper, we try to find small topics as a first step toward forecasting future topics. to this end, we propose an index for keywords to measure their impact, assuming a keyword denotes a research topic. we use a relative frequency in the definition of the index to find small topics. as far as the authors know, the frequency of keywords is not directly used to detect topics in research papers, unlike topic or trend detection in general text data. the authors think that this is because a frequency based method requires a list of stop words to remove unnecessary keywords, but it is too costly to construct it for each discipline in case of research papers. to evaluate the proposed index, we use some popular keywords in one discipline, and we check if the proposed indices for them can grasp their popularity. using this approach, we do not have to consider the issue of stop words. in other words, we try to find some properties among popular topics with the proposed index. for comparison, we also show topic detection by absolute frequency and a standard clustering algorithm. ## normalized impact index we assume the range of publication years, y 1 , y 2 , . . . , y n , and let @xmath0, y 2 , . . . , y n }. for @xmath1, d(y) denotes the set of papers published in y. for a word w and a year @xmath2, the normalized impact index, denoted by h(w, y), is defined as as follows: the proposed index for w and y is a relative frequency, normalized by both the number of publications in y and the total frequency of w among all years. therefore, we can compare h(w 1 , y 1 ) and h(w 2 , y 2 ). to understand the meaning of the index, let us assume that |d(y)| = 1 tentatively. then we can treat h(w, y) as a probability since we have y h(w, y) = 1. so, when we depict this index as a bar chart for some w whose height is h(w, y i ), the total area of the bars for w is normalized to 1. therefore, we can compare any two words w 1 and w 2 , in the view point of their trends. when we consider trends of keywords, it is natural to see temporal changes of the index from some reference year y 1 , that is, h(w, y) -h(w, y 1 ), ## case study in this section, we apply the proposed index to a real dataset to confirm its efficacy. as described in section 1, a frequency based method suffers from the issue of stop words. to avoid the issue, we check the values of the proposed index for some keywords the authors selected from some specific field. these keywords are already known as popular topics. therefore, it means that we only check positive examples. since the proposed index is defined with relative frequencies, we show the result of topic detection with absolute frequencies for comparison (see section 3.2). then, we apply a clustering algorithm to our dataset in section 3.3, to confirm that a clustering algorithm for keywords can find large topics, not small ones as described in section 1. ## conclusion in this paper, we have introduced an index to find keywords, which express small topics, using relative frequencies. as visualization, the difference of the proposed index from the reference year, 2000 in this paper, is depicted as a heat map. therefore, we can easily find subtle topics even if their absolute frequencies are not so large. we have conducted case studies using the proposed index, and confirmed that some keywords, which are already known as popular, show sharp rises of the proposed index. as described in section 3, we have only checked popular keywords. so it is an important future work to check all keywords whose values of the proposed index. even if we find some keywords with high values of the proposed index, you might want to check their absolute frequencies. therefore, it is also important to develop a visualization tool which enables to check both the absolute frequency and the proposed index. similarly, it is an important future work for the tool to introduce a grouping facility, which groups a different keywords in a hierarchical way, and then we can grasp transitions of topics with flexible granularity with the tool. to do so, we can use some vocabulary system, like one in @xcite , or word embeddings to measure the distances between two keywords.