Hierarchy-aware Label Semantics Matching Network for Hierarchical Text Classification (2021)

Abstract: Hierarchical text classification is an important yet challenging task due to the complex structure of the label hierarchy. Existing methods ignore the semantic relationship between text and labels, so they cannot make full use of the hierarchical information. To this end, we formulate the text-label semantics relationship as a semantic matching problem and thus propose a hierarchy-aware label semantics matching network (HiMatch). First, we project text semantics and label semantics into a joint embedding space. We then introduce a joint embedding loss and a matching learning loss to model the matching relationship between the text semantics and the label semantics. Our model captures the text-label semantics matching relationship among coarse-grained labels and fine-grained labels in a hierarchy-aware manner. The experimental results on various benchmark datasets verify that our model achieves state-of-the-art results.

## introduction hierarchical text classification (htc) is widely used in natural language processing (nlp), such as news categorization @xcite and scientific paper classification @xcite . htc is a particular multi-label text classification problem, which introduces hierarchies to organize label structure. as depicted in figure 1 , htc models predict multiple labels in a given label hierarchy, which generally construct one or multiple paths from coarse-grained labels to fine-grained labels in a top-down manner (aixin @xcite . generally speaking, fine-grained labels are the most appropriate labels for describing the input text. coarse-grained labels are generally the parent nodes of coarse-or fine-grained labels, expressing a more general concept. the key challenges of * *corresponding author htc are to model the large-scale, imbalanced, and structured label hierarchy @xcite . existing work in htc has introduced various methods to use hierarchical information in a holistic way. to capture the holistic label correlation features, some researchers proposed a hierarchyaware global model to exploit the prior probability of label dependencies through graph convolution networks (gcn) and treelstm @xcite . some researchers also introduced more label correlation features such as label semantic similarity and label co-occurrence @xcite . they followed the traditional way to transform htc into multiple binary classifiers for every label @xcite . however, they ignored the interaction between text semantics and label semantics @xcite , which is highly useful for classification @xcite . hence, their models may not be sufficient to model complex label dependencies and provide comparable text-label classification scores @xcite . a natural strategy for modeling the interaction between text semantics and label semantics is to introduce a text-label joint embedding by label attention @xcite or autoencoders @xcite . label attention-based methods adopted a self-attention mechanism to identify label-specific information @xcite . autoencoder-based methods extended the vanilla canonical correlated autoencoder @xcite to a ranking-based autoencoder architecture to produce comparable text-label scores @xcite . however, these methods assume all the labels are independent without fully considering the correlation between coarse-grained labels and fine-grained labels, which cannot be simply transferred to htc models @xcite . in this paper, we formulate the interaction between text and label as a semantic matching problem and propose a hierarchy-aware label semantics matching network (himatch). the principal idea is that the text representations should be semantically similar to the target label representations (especially fine-grained labels), while they should be semantically far away from the incorrect label representations. first, we adopt a text encoder and a label encoder (shown in figure 2 ) to extract textual semantics and label semantics, respectively. second, inspired by the methods of learning common embeddings @xcite , we project both textual semantics and label semantics into a text-label joint embedding space where correlations between text and labels are exploited. in this joint embedding space, we introduce a joint embedding loss between text semantics and target label semantics to learn a text-label joint embedding. after that, we apply a matching learning loss to capture text-label matching relationships in a hierarchy-aware manner. in this way, the finegrained labels are semantically closest to the text semantics, followed by the coarse-grained labels, while the incorrect labels should be semantically far away from the text semantics. hence, we propose a hierarchy-aware matching learning method to capture different matching relationships through different penalty margins on semantic distances. finally, we employ the textual representations guided by the joint embedding loss and matching learning loss to perform the hierarchical text classification. the major contributions of this paper are: ## proposed method in this section, we will describe the details about our hierarchy-aware label semantics matching network. figure 2 shows the overall architecture of our proposed model. ## conclusion here we present a novel hierarchical text classification model called himatch that can capture semantic relationships among texts and labels at different abstraction levels. instead of treating htc as multiple binary classification tasks, we consider the text-label semantics matching relationship and formulate it as a semantic matching problem. we learn a joint semantic embedding between text and labels. finally, we propose a hierarchy-aware matching strategy to model different matching relationships among coarse-grained labels, fine-grained labels and incorrect labels. in future work, we plan to extend our model to the zero-shot learning scenario.