Enhancing Hierarchical Text Classification through Knowledge Graph Integration (2023)

Abstract: Hierarchical Text Classification (HTC) is an essential and challenging subtask of multi-label text classification with a taxonomic hierarchy. Recent advances in deep learning and pre-trained language models have led to significant breakthroughs in the HTC problem. However, despite their effectiveness, these methods are often restricted by a lack of domain knowledge, which leads them to make mistakes in a variety of situations. Generally, when manually classifying a specific document to the taxonomic hierarchy, experts make inference based on their prior knowledge and experience. For machines to achieve this capability, we propose a novel Knowledge-enabled Hierarchical Text Classification model (K-HTC), which incorporates knowledge graphs into HTC. Specifically, K-HTC innovatively integrates knowledge into both the text representation and hierarchical label learning process, addressing the knowledge limitations of traditional methods. Additionally, a novel knowledge-aware contrastive learning strategy is proposed to further exploit the information inherent in the data. Extensive experiments on two publicly available HTC datasets show the efficacy of our proposed method, and indicate the necessity of incorporating knowledge graphs in HTC tasks.

## introduction hierarchical text classification (htc), as a particular multi-label text classification problem, has been extensively applied in many real-world applications, such as book categorization @xcite and scientific paper classification @xcite . in htc, documents are tagged with multiple categories that can be structured as a tree or an acyclic graph (e.g., the taxonomic hierarchy illustrated in the bottom left of figure 1 ), which poses a higher challenge than the ordinary text classification problems @xcite . the existing state-of-the-art approaches for htc @xcite @xcite ) mainly focus on the representation learning from the input text and hierarchical label structure, most of which rely on the pre-trained language models (e.g., bert @xcite ). specifically, @xcite adopted bert as the encoder and proposed a matching network to mine the relative distance between texts and labels. @xcite proposed a novel contrastive learning method to embed the hierarchy into bert encoder. despite the success of this paradigm, approaches without domain knowledge have significant limitations and may lead to mistakes in many cases. an example of this can be observed in figure 1 , where machines may classify a document as belonging to the category travel: usa & canada simply based on the presence of the phrase the usa in the document. however, if machines are equipped with a relevant knowledge graph, they can mine more information from other concepts, such as sahara and algiers. specifically, sahara is part of africa and algiers is the capital of algeria in africa. further, sahara and algiers are both tourist attractions. with the above relevant knowledge, machines will be more facilitated to make the correct inference, i.e., travel and travel: africa in the taxonomic hierarchy. nevertheless, to the best of our knowledge, few works focused on incorporating knowledge graphs into htc. indeed, many technical challenges are inherent in designing effective solutions to incorporate knowledge graphs (kgs) into htc. first, the text and kg are organized quite differently. text is organized as a sequence of tokens, whereas the kg is organized as a graph. how to effectively integrate kgs into popular text representation models (e.g., bert) is an open issue. second, compared with ordinary text classification, htc has a more complex label structure, which provides additional prior knowledge but also poses a significant challenge for label learning and the interaction between labels and documents. third, documents within the same category may contain more common concepts in the knowledge graph because they describe similar entities or topics, while documents in different categories do not. this provides a new entry point on how we can further leverage kgs in htc. in this paper, we propose a knowledge-enabled hierarchical text classification model (k-htc) to incorporate knowledge graphs into htc process. specifically, we first design a knowledge-aware text encoder (kte), which can fuse the text representation and its corresponding concept representation learned from kgs at the word granularity, thereby obtaining a more comprehensive and effective representation. subsequently, to perform label learning more effectively, we create a knowledgeaware hierarchical label attention (khla) module. it employs external knowledge from kgs for label representation and optimizes it based on the hierarchical structure, which further enhances the document representation via a label attention mechanism. after that, we propose a knowledge-aware contrastive learning (kcl) strategy. it employs the shared knowledge concepts and hierarchical labels to learn the relationships between different documents, which can further exploit the information inherent in the data. finally, extensive experiments on two publicly available datasets demonstrate the effectiveness of our proposed method, and further indicate the necessity to incorporate knowledge graphs, especially for the classification on deeper and more difficult levels. ## preliminaries in this section, we first give the problem statement of incorporating kgs into htc, and then introduce the knowledge preparation for k-htc model. ## k-htc model in this section, we will introduce the technical details of k-htc model. as figure 2 shows, k-htc consists of three components: 1) knowledge-aware text encoder (kte); 2) knowledge-aware hierarchical label attention (khla); 3) knowledgeaware contrastive learning (kcl). ## conclusions in this paper, we explored a motivated direction for incorporating the knowledge graph into hierarchical text classification. we first analyzed the necessity to integrate knowledge graphs and further proposed a knowledge-enabled hierarchical text classification model (k-htc). specifically, we designed a knowledge-aware text encoder, which could fuse the text representation and its corresponding concept representation learned from kgs. subsequently, a knowledge-aware hierarchical label attention module was designed to model the interaction between the documents and hierarchical labels. more importantly, we proposed a knowledge-aware contrastive learning strategy, which could further boost the classification performance by exploiting the information inherent in the data. finally, extensive experiments on two publicly available htc datasets demonstrated the effectiveness of our proposed method. we hope our work will lead to more future studies.