Measuring Fine-Grained Domain Relevance of Terms: A Hierarchical Core-Fringe Approach (2021)

Abstract: We propose to measure fine-grained domain relevance– the degree that a term is relevant to a broad (e.g., computer science) or narrow (e.g., deep learning) domain. Such measurement is crucial for many downstream tasks in natural language processing. To handle long-tail terms, we build a core-anchored semantic graph, which uses core terms with rich description information to bridge the vast remaining fringe terms semantically. To support a fine-grained domain without relying on a matching corpus for supervision, we develop hierarchical core-fringe learning, which learns core and fringe terms jointly in a semi-supervised manner contextualized in the hierarchy of the domain. To reduce expensive human efforts, we employ automatic annotation and hierarchical positive-unlabeled learning. Our approach applies to big or small domains, covers head or tail terms, and requires little human effort. Extensive experiments demonstrate that our methods outperform strong baselines and even surpass professional human performance.

## introduction with countless terms in human languages, no one can know all terms, especially those belonging to a technical domain. even for domain experts, it is quite challenging to identify all terms in the domains they are specialized in. however, recognizing and understanding domain-relevant terms is the basis to master domain knowledge. and having a sense of domains that terms are relevant to is an initial and crucial step for term understanding. in this paper, as our problem, we propose to measure fine-grained domain relevance, which is defined as the degree that a term is relevant to a given domain, and the given domain can be broad or narrow-an important property of terms that has not been carefully studied before. e.g., deep learning is a term relevant to the domains of computer science and, more specifically, machine learning, but not so much to others like database or compiler. thus, it has a high domain relevance for the former domains but a low one for the latter. from another perspective, we propose to decouple extraction and evaluation in automatic term extraction that aims to extract domain-specific terms from texts @xcite . this decoupling setting is novel and useful because it is not limited to broad domains where a domain-specific corpus is available, and also does not require terms must appear in the corpus. a good command of domain relevance of terms will facilitate many downstream applications. e.g., to build a domain taxonomy or ontology, a crucial step is to acquire relevant terms @xcite . also, it can provide or filter necessary candidate terms for domain-focused natural language tasks @xcite . in addition, for text classification and recommendation, the domain relevance of a document can be measured by that of its terms. we aim to measure fine-grained domain relevance as a semantic property of any term in human languages. therefore, to be practical, the proposed model for domain relevance measuring must meet the following requirements: 1) covering almost all terms in human languages; 2) applying to a wide range of broad and narrow domains; and 3) relying on little or no human annotation. however, among countless terms, only some of them are popular ones organized and associated with rich information on the web, e.g., wikipedia pages, which we can leverage to characterize the domain relevance of such "head terms." in contrast, there are numerous "long-tail terms"-those not as frequently used-which lack descriptive information. as challenge 1, how to measure the domain relevance for such long-tail terms? on the other hand, among possible domains of interest, only those broad ones (e.g., physics, computer science) naturally have domain-specific corpora. many existing works @xcite @xcite have relied on such domain-specific corpora to identify domain-specific terms by contrasting their distributions to general ones. in contrast, those fine-grained domains (e.g., quantum mechanics, deep learning)which can be any topics of interest-do not usually have a matching corpus. as challenge 2, how to achieve good performance for a fine-grained domain without assuming a domain-specific corpus? finally, automatic learning usually requires large amounts of training data. since there are countless terms and plentiful domains, human annotation is very time-consuming and laborious. as challenge 3, how to reduce expensive human efforts when applying machine learning methods to our problem? as our solutions, we propose a hierarchical corefringe domain relevance learning approach that addresses these challenges. first, to deal with longtail terms, we design the core-anchored semantic graph, which includes core terms which have rich description and fringe terms without that information. based on this graph, we can bridge the domain relevance through term relevance and include any term in evaluation. second, to leverage the graph and support fine-grained domains without relying on domain-specific corpora, we propose hierarchical core-fringe learning, which learns the domain relevance of core and fringe terms jointly in a semi-supervised manner contextualized in the hierarchy of the domain. third, to reduce human effort, we employ automatic annotation and hierarchical positive-unlabeled learning, which allow to train our model with little even no human effort. overall, our framework consists of two processes: 1) the offline construction process, where a domain relevance measuring model is trained by taking a large set of seed terms and their features as input; 2) the online query process, where the trained model can return the domain relevance of query terms by including them in the core-anchored semantic graph. our approach applies to a wide range of domains and can handle any query, while nearly no human effort is required. to validate the effectiveness of our proposed methods, we conduct extensive experiments on various domains with different settings. results show our methods significantly outperform well-designed baselines and even surpass human performance by professionals. ## related work the problem of domain relevance of terms is related to automatic term extraction, which aims to extract domain-specific terms from texts automatically. compared to our task, automatic term extraction, where extraction and evaluation are combined, possesses a limited application and has a relatively large dependence on corpora and human annotation, so it is limited to several broad domains and may only cover a small number of terms. existing approaches for automatic term extraction can be roughly divided into three categories: linguistic, statistical, and machine learning methods. linguistic methods apply human-designed rules to identify technical/legal terms in a target corpus @xcite . statistical methods use statistical information, e.g., frequency of terms, to identify terms from a corpus @xcite @xcite @xcite . machine learning methods learn a classifier, e.g., logistic regression classifier, with manually labeled data @xcite @xcite . there also exists some work on automatic term extraction with wikipedia @xcite . however, terms studied there are restricted to terms associated with a wikipedia page. recently, inspired by distributed representations of words @xcite , methods based on deep learning are proposed and achieve state-ofthe-art performance. @xcite design supervised learning methods by taking the concatenation of domain-specific and general word embeddings as input. @xcite propose a multi-channel neural network model that leverages domain-specific and general word embeddings. the techniques behind our hierarchical corefringe learning methods are related to research on graph neural networks (gnns) @xcite ; hierarchical text classification @xcite @xcite ; and positive-unlabeled learning @xcite @xcite . ## methodology we study the fine-grained domain relevance of terms, which is defined as follows: definition 1 (fine-grained domain relevance) the fine-grained domain relevance of a term is the degree that the term is relevant to a given domain, and the given domain can be broad or narrow. the domain relevance of terms depends on many factors. in general, a term with higher semantic relevance, broader meaning scope, and better usage possesses a higher domain relevance regarding the target domain. to measure the fine-grained domain relevance of terms, we propose a hierarchical corefringe approach, which includes an offline training process and can handle any query term in evaluation. the overview of the framework is illustrated in figure 1 . ## experiments in this section, we evaluate our model from different perspectives. 1) we compare with baselines by treating some labeled terms as queries. 2) we compare with human professionals by letting humans and machines judge which term in a query pair is more relevant to a target domain. 3) we conduct intuitive case studies by ranking terms according to their domain relevance. ## conclusion we introduce and study the fine-grained domain relevance of terms-an important property of terms that has not been carefully studied before. we propose a hierarchical core-fringe domain relevance learning approach, which can cover almost all terms in human languages and various domains, while requires little or even no human annotation. we believe this work will inspire an automated solution for knowledge management and help a wide range of downstream applications in natural language processing. it is also interesting to integrate our methods to more challenging tasks, for example, to characterize more complex properties of terms even understand terms.