Topic Taxonomy Expansion via Hierarchy-Aware Topic Phrase Generation (2022)

Abstract: Topic taxonomies display hierarchical topic structures of a text corpus and provide topical knowledge to enhance various NLP applications. To dynamically incorporate new topic information, several recent studies have tried to expand (or complete) a topic taxonomy by inserting emerging topics identified in a set of new documents. However, existing methods focus only on frequent terms in documents and the local topic-subtopic relations in a taxonomy, which leads to limited topic term coverage and fails to model the global taxonomy structure. In this work, we propose a novel framework for topic taxonomy expansion, named TopicExpan, which directly generates topic-related terms belonging to new topics. Specifically, TopicExpan leverages the hierarchical relation structure surrounding a new topic and the textual content of an input document for topic term generation. This approach encourages newly-inserted topics to further cover important but less frequent terms as well as to keep their relation consistency within the taxonomy. Experimental results on two real-world text corpora show that TopicExpan significantly outperforms other baseline methods in terms of the quality of output taxonomies.

## introduction topic taxonomy is a tree-structured representation of hierarchical relationship among multiple topics found in a text corpus @xcite @xcite . each topic node is defined by a set of semantically coherent terms related to a specific topic (i.e., topic term cluster), and each edge implies the "general-specific" relation between two topics (i.e., topic-subtopic). with the knowledge of hierarchical topic structures, topic taxonomies have been successfully utilized in many text mining applications, such as text summarization @xcite and categorization @xcite . recently, automated expansion (or completion) of an existing topic taxonomy has been studied @xcite , which helps people to incrementally manage the topic knowledge within fast-growing document collections. this task has two technical challenges: (1) identifying new topics by collecting topic-related terms that have novel semantics, and (2) inserting the new topics at the right position in the hierarchy. in figure 1 , for example, a new topic node painter that consists of its topic-related terms [baroque painter, realist painter, portraitist, ...] is inserted at the child position (i.e., subtopic) of the existing topic node artist, without breaking the consistency of topic relations with the neighbor nodes. the existing methods for topic taxonomy expansion, however, suffer from two major limitations: (1) limited term coverage -they identify new topics from a set of candidate terms, while relying on entity extraction tools @xcite or phrase mining techniques @xcite @xcite to obtain the high-frequency candidate terms in a corpus. such extraction techniques will miss a lot of topic-related terms that have low frequency, and thus lead to an incomplete set of candidate terms @xcite . (2) inconsistent topic relation -as they insert new topics by considering only the first-order relation between two topics (i.e., a topic and its subtopic), the newly-inserted topics are likely to have inconsistent relations with other existing topics. the expansion strategy based on the first-order topic relation is inadequate to capture the holistic structure information of the existing topic taxonomy. as a solution to both challenges, we present top-icexpan, a new framework that expands the topic taxonomy via hierarchy-aware topic term generation. the key idea is to directly generate topicrelated terms from documents by taking the topic hierarchy into consideration. from the perspective of term coverage, this generation-based approach can identify more multi-word terms even if they have low frequency in the given corpus @xcite , compared to the extraction-based approach only working on the extracted candidate terms that frequently appear in the corpus. to combat the challenge of relation inconsistency, we utilize graph neural networks (gnns) to encode the relation structure surrounding each topic @xcite and generate topic-related terms conditioned on these relation structure encodings. this allows us to accurately capture a hierarchical structure beyond the first-order relation between two topics. to be specific, topicexpan consists of the training step and the expansion step. the training step is for optimizing a neural model that topicconditionally generates a term from an input document. technically, for topic-conditional term generation, the model utilizes the relation structure of a topic node as well as the textual content of an input document. the expansion step is for discovering novel topics and inserting them into the topic taxonomy. to this end, topicexpan places a virtual topic node underneath each existing topic node, and then it generates the topic terms conditioned on the virtual topic by utilizing the trained model. in the end, it performs clustering on the generated terms to identify multiple novel topics, which are inserted at the position of the virtual topic node. contributions. the main contributions of this paper can be summarized as follows: (1) we propose a novel framework for topic taxonomy expansion, which tackles the challenges in topic term coverage and topic relation consistency via hierarchyaware topic term generation. (2) we present a neural model to generate a topic-related term from an input document topic-conditionally by capturing the hierarchical relation structure surrounding each topic based on gnns. (3) our comprehensive evaluation on two real-world datasets demonstrates that output taxonomies of topicexpan show better relation consistency as well as term coverage, compared to that of other baseline methods. ## related work topic taxonomy construction. to build a topic taxonomy of a given corpus from scratch, the stateof-the-art methods have focused on finding out discriminative term clusters in a hierarchical manner @xcite @xcite . several recent studies have started to enrich and expand an existing topic taxonomy by discovering novel topics from a corpus and inserting them into the taxonomy @xcite . they leverage the initial topic taxonomy as supervision for learning the hierarchical relation among topics. to be specific, they discover new subtopics that should be inserted at the child of each topic, by using a relation classifier trained on (parent, child) topic pairs @xcite or performing novel subtopic clustering @xcite . however, all the methods rely on candidate terms extracted from a corpus and also consider only the first-order relation between two topics, which degrades the term coverage and relation consistency of output topic taxonomies. ## problem formulation notations. a topic taxonomy @xmath0, r) is a tree structure about topics, where each node (∈ c) represents a single conceptual topic and each edge (∈ r) implies the hierarchical relation between a topic and its subtopic. a topic node c @xmath1, denoted by p j (i.e., term cluster for the topic c j ), where the most representative term (i.e., center term) serves as the topic name. each document d @xmath2, . . . , v il ] and each term foot_0 p @xmath3, . . . , v kt ] in a given corpus d is the sequence of l and t word tokens in the vocabulary set @xmath4, respectively. here, each term is regarded as a phrase that consists of one or more word tokens, so the terms "phrase" and "term" are used interchangeably in this paper. ## conclusion in this paper, we study the problem of topic taxonomy expansion, pointing out that the existing approach has shown limited term coverage and inconsistent topic relation. our topicexpan framework introduces hierarchy-aware topic term generation, which generates a topic-related term by using both the textual content of an input document and the relation structure of a topic as the condition for generation. the quantitative and qualitative evaluation demonstrates that our framework successfully obtains much higher-quality topic taxonomy in various aspects, compared to other baseline methods. for future work, it would be promising to incorporate an effective measure for the topic relevance of multi-word terms (i.e., phrases) into our framework. since learning and utilizing the representation of multi-word terms remains challenging and worth exploring, it can be widely applied to many other text mining tasks.