Multi-label Classification of Scientific Research Documents Across Domains and Languages (2022)

Abstract: Automatically organizing scholarly literature is a necessary and challenging task. By assigning scientific research publications key concepts, researchers, policymakers, and the general public are able to search for and discover relevant research literature. The organization of scientific research evolves with new discoveries and publications, requiring an up-to-date and scalable text classification model. Additionally, scientific research publications benefit from multi-label classification, particularly with more fine-grained sub-domains. Prior work has focused on classifying scientific publications from one research area (e.g., computer science), referencing static concept descriptions, and implementing an English-only classification model. We propose a multi-label classification model that can be implemented in non-English languages, across all of scientific literature, with updatable concept descriptions.

## introduction maintaining an up-to-date organization of scientific literature in any domain requires an automated approach-a comprehensive and real-time solution for a constant influx of text data. specifically, research publications require characterization or indexing in order to be searchable and accessible to researchers, policymakers, and the public. many academic databases and publishers maintain a taxonomy that authors or editors reference in order to manually assign topics, research fields, or concepts to scientific publications. yet, manual labeling is notoriously laborious and error-prone. automation is necessary to accurately label documents with taxonomy concepts in a timely manner. here, we focus on scientific publication classification based on microsoft academic graph's field of study taxonomy @xcite . this taxonomy contains a hierarchy of scientific concepts (fields of study) to organize scholarly litera-ture. our objective is to design an updatable and scalable multi-label classification model that is independent of manual annotation or input language. we experiment with scientific research documents in english and chinese, as these are by far the two most frequent languages for publications in our database. our work leverages a multi-lingual knowledge base, wikipedia, in order to obtain up-to-date concept descriptions in english and other languages. using mediawiki's api, we first locate an english concept's wikipedia page and are then able to find the corresponding page in other languages @xcite . hence, a multi-lingual knowledge base provides multi-lingual concept descriptions without requiring any direct translating of the concept taxonomy or concept descriptions. we represent both the concept descriptions and research publications text data in embedding form. by using vector space representations of text (word embeddings) we can compute the cosine similarities between concept embeddings and publication embeddings, with the cosine similarity score indicating the relevance of a concept to a publication. in this way, we are able to compute either one top field (most similar) or multiple fields of study that are relevant (determined by a similarity score threshold for the task at hand) to a given publication. a multi-label classification model is a practical approach to scientific publication classification, as most scientific research publications are relevant to more than one field of study, particularly at the more granular level of fields. for example, a publication can be relevant to natural language processing and machine learning. we implement our multi-label classification model in english and chinese, generating field descriptions, embeddings, and field-to-publication similarity scores in each language. our database of scholarly literature contains more than 184 million documents in english and more than 44 million documents in chinese, which serve both as input text for word embeddings and as target publications for classification. applying our scientific publication word embeddings and field of study descriptions from wikipedia, we compute field embeddings for 313 different fields of study, and publication embeddings for the scientific research publications in english and chinese. because we do not have a manually annotated, ground-truth dataset with field labels assigned to publications, we provide extensive evaluations of our results and include a case study on artificial intelligence and machine learning publications. the contributions of the paper are summarized as follows: 1) word embeddings in english and chinese, trained on a comprehensive set of scholarly literature, 2) a scientific text classification model not restricted to the english language, and 3) a python library for updating field embeddings and models in sync with changes to underlying field definitions (from wikipedia articles and the sources they cite), to address conceptual drift. all results and code will be made public in our github repository foot_0 . ## related work classifying text according to a defined taxonomy is applied across a wide range of domains, such as patents, news articles, and scientific literature, using numerous machine learning approaches. text classification for scientific literature typically involves text extraction, topic modeling, or citation graphs to cluster related documents @xcite @xcite . prior research that uses a predefined taxonomy for multi-label classification is generally limited to one broad area of research, and selecting a dataset with annotated publication data (i.e., a dataset limited to a classification scheme). santos and rodrigues reference the association for computing machinery (acm) concept classification system (ccs) to assign multiple concept labels to computer science papers @xcite . the authors crawl relevant web pages to identify concept-related descriptive text and implement three different classification models: binary relevance, label powerset, and multi-label k-nearest neighbors @xcite . similarly, mustafa et al. reference the acm ## field of study classification model our field of study multi-label classification model is adapted from mag's scientific publication classification scheme, with key design modifications. in shen et al.'s model, the descriptive text used to generate l0 and l1 fos embeddings are titles and abstracts from sets of scientific publications for each field, in which the publications are selected from a sample of unknown journals and conferences @xcite . for one of their embeddings set the authors generate word2vec vectors. we use wikipedia article text and reference publications for l0 and l1 fos descriptive text. in this way, field descriptions can be replicated, extended to languages other than english, and updated as the fields evolve. we describe in this section the project workflow to process our data and design our field of study classification model. figure 3 shows the high-level pipeline to produce the field geography (displaying 6 of 11 l1) sociology (displaying 5 of 13 l1) archaeology, cartography, forestry, geodesy, meteorology, regional science anthropology, demography, ethnology, gender studies, media studies, political economy geology (displaying 6 of 18 l1) and document embedding outputs necessary for our classification model. we describe each step in our classification model pipeline as follows: step 1: to normalize the scientific publication text, we remove all punctuation and numeric tokens. for languages that are case-sensitive, we set all text to lowercase. for example "covid-19" is transformed to "covid19" in english. the normalized texts are used as inputs for the tf-idf and fasttext embeddings in step 2. step 2: with the normalized scientific publication text (from step 1) as input, we produce tf-idf embeddings using tfidftransformer from gensim and 250-dimensional fasttext word embeddings using the skipgram model @xcite . tf-idf pro-vides a measurement of how important a word is to a document based on the word's occurrences in the entire document. fasttext word embeddings encode n-grams in a vector space that represents semantics. since both vector representations of words (tf-idf and fasttext) are determined by the input corpus, it is necessary to use a representative corpus for the task at hand. step 3: for each of the 19 l0 and 294 l1 fos, we retrieve the corresponding associated text (page content and reference publications) in wikipedia, which we refer to as descriptive field text. combining the wikipedia page text and scientific publication text we aim to capture both definitions and exemplar research for a given field. 2 and the descriptive field text from step 3. we follow the procedure in "algorithm 1" to generate tf-idf and fasttext embeddings for each fos. to single embedding input: word embedding dictionary, e text, t output: single text vector, t 1: procedure embed_text(t, e) 2: @xmath0. empty array to store word vectors 3: for word in t do 4: if word in e.keys() then 5: @xmath1: v.append( w ) 7: end if 8: end for 9: @xmath2, @xmath3: @xmath4.norm( t, 2, @xmath5: if @xmath6: else @xmath7: end if 14: return t . the text vector is t 15: end procedure step 5: separate from fos embeddings in step 4, we compute entity embeddings. we generate these for a fos or publication as the average over the embeddings of each fos mention in its text. ## experiments we perform steps 1-7 on en-publications and zh-publications. text normalization and embedding generation (steps 1-2) may require different tools and packages depending on the choice of non-english languages; we use jieba for chinese text processing. ## results and evaluation evaluating our results is particularly challenging without a ground-truth dataset that contains publications and their corresponding field of study labels. because of this limitation, we offer several methods of evaluation that do not require annotation (to limit human bias and error) and can be replicated. our evaluation methods compare results at the fos level and the publication level in order to measure our taxonomy representation results (fos embeddings) and our publication classification results. ## conclusion and future work organizing scholarly literature is necessary for accessibility and usefulness of scientific research publications. prior work has focused on a few broad areas of research, english-only research publications and taxonomies, and static taxonomy descriptions. in this paper, we implement a multi-label classification model that encompasses research fields from all of science, can be updated using a comprehen-sive, online knowledge base, and is not restricted to the english language. in future work, we plan to expand to additional languages and explore the longitudinal dynamics of fields: how their relative positions have shifted, within and between languages, as wikipedia article text and references have changed.