Domain Representative Keywords Selection: A Probabilistic Approach (2022)

Abstract: We propose a probabilistic approach to select a subset of a target domain representative keywords from a candidate set, contrasting with a context domain. Such a task is crucial for many downstream tasks in natural language processing. To contrast the target domain and the context domain, we adapt the two-component mixture model concept to generate a distribution of candidate keywords. It provides more importance to the distinctive keywords of the target domain than common keywords contrasting with the context domain. To support the representativeness of the selected keywords towards the target domain, we introduce an optimization algorithm for selecting the subset from the generated candidate distribution. We have shown that the optimization algorithm can be efficiently implemented with a near-optimal approximation guarantee. Finally, extensive experiments on multiple domains demonstrate the superiority of our approach over other baselines for the tasks of keyword summary generation and trending keywords selection.

## introduction domain representative keywords are the core knowledge of a target domain of interest. a target domain can be a broad area of science like computer science (cs) or its sub-field artificial intelligence (ai). acquiring domain representative keywords benefits various natural language processing (nlp) tasks such as information summarization, organization, and extraction. for instance, acquiring a set of domain representative keywords is an important first step in organizing domain knowledge with a taxonomy of keywords @xcite . moreover, tagging documents @xcite with domain representative keywords helps to facilitate search or recommendation in a domain. for another example, summarizing a domain using its trending keywords for a specific time frame helps researchers get a snapshot of research trends or emerging areas of interest, e.g., new emerging security vulnerabilities. in reality, while representing a domain, the desired keywords often depend on a given context domain. e.g., if we are interested in representing the cs domain with the context of general knowledge (all areas of knowledge), keywords like model, data, information make sense in distinguishing cs from general knowledge. however, if the context is general science, those keywords are not distinctive enough to distinguish cs from other areas like mathematics or physics. instead, the keywords machine learning, data mining, deep learning make more sense in this case. therefore, it is important to contrast with a known context domain while representing a target domain, but most of the existing work ignored this. an application of this is shown in fig. 1 (a ) from the arxiv category taxonomy 2 . we can see a shift of categories from general to more specific subcategories of research areas: @xmath0. knowing cs categories (partially shown in (1)) as the context, the keywords specified in (2) (i.e., speech, text retrieval) are more appropriate to represent cl than keywords overlapped with other cs categories. moreover, the number of keywords that need to be selected depends on the nature of the applications. e.g., users are interested in a quick, highlevel overview with fewer keywords while summarizing a particular domain. on the other hand, while building a controlled vocabulary representative of a certain domain, the number of keywords is naturally large. an application of the controlled vocabulary is illustrated in figure 1 @xcite . it shows the fingerprint visualization of a cs researcher generated by elsevier fingerprint engine foot_1 , a system for research profiling. the researcher's profile is summarized using keywords in engineering & material science domain. however, we can see that some non-representative keywords like labels and merging are used in the summarization. therefore, having a representative controlled vocabulary for each domain will facilitate this application for expressively representing a researcher profile. we thus propose the problem of domain representative keywords selection. as input, we are given a set of candidate keywords, a target and a context domain represented by their corresponding corpora, and a size k. as output, we aim to select a subset consisting of k keywords from the given candidate set such that the subset best represents the target domain contrasting with the context domain. here, we assume that the candidate keywords are from the target domain and can be implicitly extracted from the given target domain corpus or externally given keywords for that domain. from the above problem and discussion, we have identified that the solution for the problem needs to meet the following two requirements: (1) the selected keywords should be distinctive to the target domain contrastive with a context domain; (2) the selected keywords should represent the target domain as a whole within the specific size constraint. none of the existing work satisfies all of them. previously, research has been conducted on automatic keyword extraction @xcite @xcite and phrase mining @xcite . however, their main focus is to extract terms from single/multiple documents without considering whether the extracted terms are distinctive to a target domain contrastive with a context. there is also some previous research @xcite @xcite that tries to find fine-grained domain-specific keywords from the text. however, these approaches mostly rank keywords based on their specificity to a corpus (or domain) rather than selecting a predefined number of keywords with a global objective of representing the target domain. therefore, in this work, we propose a solution to satisfy all the specified requirements. the first challenge on fulfilling the requirements is contrasting the target and context domains. among candidate keywords, the distinctive keywords may have similar corpus statistics (i.e., frequency from target domain corpus) with many nondistinctive popular keywords. therefore, simply filtering out highly frequent keywords may lose many distinctive keywords for a target domain. instead, it is more intuitive to say that the keywords that frequently appear in both target and context corpora are often not distinctive keywords for the target domain. it inspires us to leverage the two-component mixture model (mm) @xcite concept to generate the candidate keywords distribution contrasting with the context domain. as far as we know, this is the first work to utilize a mixture model mechanism for keywords selection. the second challenge is the representation under a size constraint. if we simply select the top distinctive keywords based on the mm-generated distribution, we may end up with redundant keywords that may fall short in representing the target domain as a whole. hence, it is more intuitive to consider selecting keywords with a domain representation objective. therefore, we cast this as an optimization problem of selecting k keywords that coarsen the candidate distribution adapting the concept of statistical machine translation @xcite with the objective of minimizing the divergence between the initial and coarsened distributions of candidate keywords. in summary, as our contributions in this paper, firstly, we propose a new problem formulation named domain representative keywords selection. secondly, we propose a framework for solving the problem consisting of two steps: (1) generating candidate keywords distribution using a twocomponent mixture model mechanism and (2) selecting a subset of keywords utilizing the generated distribution with an introduced optimization algorithm. thirdly, we prove that our proposed optimization problem can be efficiently solved with a near-optimal approximation ratio. finally, to validate the effectiveness of our approaches, we con-duct extensive experiments on multiple domains for different tasks demonstrating the superiority of our framework against strongly designed baselines. ## related work the problem of domain representative keywords selection is related to the automatic keyword extraction (ake) problem. ake focuses on extracting or generating the most prominent keywords from single/multiple documents. existing methods for ake can be classified into two categories: supervised and unsupervised keyword extraction. early supervised methods consider ake as a binary classification problem @xcite by learning a classifier from annotated documents to predict whether a candidate phrase is a keyword or not. recently, deep learning has been used for the supervised ake. e.g., @xcite uses an encoder-decoder-based framework to generate keywords where @xcite addresses ake as a sequence labeling problem. unsupervised ake methods mostly apply graphbased ranking mechanisms utilizing semantic relatedness measure between keywords @xcite . besides, linguistic @xcite and semantic @xcite approaches have also been used for unsupervised ake. however, the main focus of the above studies is to describe single/multiple documents rather than domain-specific keywords extraction. to solve this problem, several researches @xcite @xcite @xcite have been conducted on domain-specific fine-grained keyword extraction. e.g., @xcite propose an algorithm for measuring the relevance of a keyword in a particular domain. however, this approach requires a user to provide some seed domain-relevant terms for supervising the algorithm. moreover, the above approaches only consider ranking keywords based on their domain specificity (or relevance). none of them deals with the problem of domain representative keyword selection with a specific size constraint. the mixture model used for generating keywords distribution in our approach is related to the research on probabilistic topic models @xcite and comparative text mining @xcite . however, the difference between our approach and these studies is that rather than finding multiple latent topics or themes from a collection or multiple collections of documents, we model a target domain corpus as a distribution of unigram language model contrastive with a context model. ## proposed methodology our proposed framework consists of two steps: (1) generating distribution for the candidate keywords and (2) selecting a subset that best represents the target domain utilizing the generated distribution. ## experiments this section evaluates our models from different perspectives: (1) the ability to select representative summary keywords for a target domain; (2) the performance for trending keywords selection task in a domain for different time frames. ## conclusion this paper proposes an approach for solving an important but understudied problem of a domain representative keywords selection from candidates contrasting with a context domain. our approach utilizes a two-component mixture model mechanism followed by a novel subset selection optimization algorithm to tackle the problem. we believe this work will encourage the automated text structuring problem and help a wide range of downstream applications in nlp. for future research direction, we want to focus on adapting the proposed approach in a more challenging task like single document summarization where the scope of information is limited. besides, our proposed techniques are general and thus can be used in many applications such as information extraction, topic modeling, and concept indexing. exploration of those applications is an interesting future direction.