Enhancing Hierarchical Text Classification through Knowledge Graph Integration
Year: 2023

Abstract:
Hierarchical Text Classification (HTC) is an essential and challenging subtask of multi-label text classification with a taxonomic hierarchy. Recent advances in deep learning and pretrained language models have led to significant breakthroughs in the HTC problem. However, despite their effectiveness, these methods are often restricted by a lack of domain knowledge, which leads them to make mistakes in a variety of situations. Generally, when manually classifying a specific document to the taxonomic hierarchy, experts make inference based on their prior knowledge and experience. For machines to achieve this capability, we propose a novel Knowledge-enabled Hierarchical Text Classification model (K-HTC), which incorporates knowledge graphs into HTC. Specifically, KHTC innovatively integrates knowledge into both the text representation and hierarchical label learning process, addressing the knowledge limitations of traditional methods. Additionally, a novel knowledge-aware contrastive learning strategy is proposed to further exploit the information inherent in the data. Extensive experiments on two publicly available HTC datasets show the efficacy of our proposed method, and indicate the necessity of incorporating knowledge graphs in HTC tasks.

heading: 1 Introduction
text: Hierarchical Text Classification (HTC), as a particular multi-label text classification problem, has been extensively applied in many real-world applications, such as book categorization (Remus et al., 2019) and scientific paper classification (Kowsari et al., 2017). In HTC, documents are tagged with multiple categories that can be structured as a tree or an acyclic graph (e.g., the taxonomic hierarchy illustrated in the bottom left of Figure 1), which poses a higher challenge than the ordinary text classification problems (Sun and Lim, 2001).
∗Corresponding author.
The existing state-of-the-art approaches for HTC (Zhou et al., 2020; Deng et al., 2021; Chen et al., 2021; Wang et al., 2022b,c) mainly focus on the representation learning from the input text and hierarchical label structure, most of which rely on the pre-trained language models (e.g., BERT (Devlin et al., 2018)). Specifically, Chen et al. (2021) adopted BERT as the encoder and proposed a matching network to mine the relative distance between texts and labels. Wang et al. (2022b) proposed a novel contrastive learning method to embed the hierarchy into BERT encoder.
Despite the success of this paradigm, approaches without domain knowledge have significant limitations and may lead to mistakes in many cases. An example of this can be observed in Figure 1, where machines may classify a document as belonging to the category Travel: USA & Canada simply based on the presence of the phrase The USA in the document. However, if machines are equipped with a relevant knowledge graph, they can mine more information from other concepts, such as Sahara and Algiers. Specifically, Sahara is part of Africa and Algiers is the capital of Algeria in Africa. Further, Sahara and Algiers are both Tourist Attractions. With the above relevant knowledge, machines will be more facilitated to make the correct inference, i.e., Travel and Travel: Africa in the taxonomic hierarchy. Nevertheless, to
5797
the best of our knowledge, few works focused on incorporating knowledge graphs into HTC.
Indeed, many technical challenges are inherent in designing effective solutions to incorporate knowledge graphs (KGs) into HTC. First, the text and KG are organized quite differently. Text is organized as a sequence of tokens, whereas the KG is organized as a graph. How to effectively integrate KGs into popular text representation models (e.g., BERT) is an open issue. Second, compared with ordinary text classification, HTC has a more complex label structure, which provides additional prior knowledge but also poses a significant challenge for label learning and the interaction between labels and documents. Third, documents within the same category may contain more common concepts in the knowledge graph because they describe similar entities or topics, while documents in different categories do not. This provides a new entry point on how we can further leverage KGs in HTC.
In this paper, we propose a Knowledge-enabled Hierarchical Text Classification model (K-HTC) to incorporate knowledge graphs into HTC process. Specifically, we first design a Knowledge-aware Text Encoder (KTE), which can fuse the text representation and its corresponding concept representation learned from KGs at the word granularity, thereby obtaining a more comprehensive and effective representation. Subsequently, to perform label learning more effectively, we create a Knowledgeaware Hierarchical Label Attention (KHLA) module. It employs external knowledge from KGs for label representation and optimizes it based on the hierarchical structure, which further enhances the document representation via a label attention mechanism. After that, we propose a Knowledge-aware Contrastive Learning (KCL) strategy. It employs the shared knowledge concepts and hierarchical labels to learn the relationships between different documents, which can further exploit the information inherent in the data. Finally, extensive experiments on two publicly available datasets demonstrate the effectiveness of our proposed method, and further indicate the necessity to incorporate knowledge graphs, especially for the classification on deeper and more difficult levels.

heading: 2 Related Work

heading: 2.1 Hierarchical Text Classification
text: Hierarchical text classification is a particular multilabel text classification problem, where the docu-
ments are assigned to one or more nodes of a taxonomic hierarchy (Wehrmann et al., 2018). Existing works for HTC could be categorized into local and global approaches according to their exploration strategies. The local approaches train multiple classifiers, each responsible for the corresponding local region (e.g., each label or level). For instance, Banerjee et al. (2019) trained a classifier for each label and proposed a strategy to transfer parameters of parent models to its child models. Shimura et al. (2018) designed a CNN-based method to use data in the upper levels to contribute to the categorization in the lower levels.
As for global methods, they build a single classifier for all classes, which will take the class hierarchy as a whole into account. For example, Cai and Hofmann (2004) proposed a hierarchical Support Vector Machine (SVM) algorithm based on discriminant functions. In recent years, with the rapid development of deep neural networks, many deep learning algorithms, such as Attention and Pre-trained Language Models, have been employed in HTC. Huang et al. (2019) designed an attention-based recurrent network to mine the textclass associations. Zhou et al. (2020) adopted a typical structure encoder for modeling label dependencies in both top-down and bottom-up manners. Chen et al. (2021) adopted BERT as encoder and proposed a matching network to mine the relative distance between texts and labels. Wang et al. (2022b) suggested a contrastive learning method to embed the hierarchy into BERT encoder. Wang et al. (2022c) introduced prompt learning into HTC and proposed a novel multi-label MLM perspective. Nevertheless, most of these methods ignore the relevant knowledge in the modeling process and have significant limitations in many cases.

heading: 2.2 Knowledge Graph
text: Knowledge Graph (KG) has millions of entries that describe real-world concepts (entities) like people, places and organizations. In a KG, concepts (entities) are represented as nodes, while the relations between concepts are described as edges. Recently, many knowledge graphs have been established in both academia and industry, such as ConceptNet (Speer et al., 2017), DBpedia (Lehmann et al., 2015) and Freebase (Bollacker et al., 2008).
On the basis of KGs, researchers attempt to incorporate them into many downstream application tasks and obtain significant improvements. For in-
stance, Wang et al. (2017) proposed a CNN-based text classification method, which combined internal representation and external knowledge representation from KGs. Jang et al. (2021) presented a novel knowledge-infused attention mechanism to incorporate high-level concepts into Neural Network models, achieving accurate and interpretable text classification. Lin et al. (2019) proposed a textual inference framework for question answering, which effectively utilized external structured knowledge graphs to perform explainable inferences. As far as we know, there are very few works that have attempted to incorporate knowledge graphs into HTC, making our K-HTC model a pioneering approach in this field.

heading: 3 Preliminaries
text: In this section, we first give the problem statement of incorporating KGs into HTC, and then introduce the knowledge preparation for K-HTC model.

heading: 3.1 Problem Statement
text: Given the input document D and an external knowledge graph G1 = (E,R, T ), HTC aims to predict a subset y of label set Y . The size of label set Y is K. In the knowledge graph G1, E is the set of concepts, R is the set of relations, and T = E×R×E is the set of triples.
It is notable that the label set Y is organized as an acyclic graph: G2 = (Y,A), where A is the adjacency matrix of Y . Besides, each label yi ∈ Y corresponds to a label name Li, which can be seen as a short text description.

heading: 3.2 Knowledge Preparation
text: In this subsection, we first identify the concepts mentioned in the input documents and label names
(Concept Recognition), and then pre-train the concept embedding (Concept Pre-training).
Concept Recognition. Given the text x = {x1, x2, ..., xN}, we are expected to match its tokens to the concepts from the given knowledge graph G1 (in this paper, we adopt the advanced KG named ConceptNet (Speer et al., 2017)). Following the strategy proposed by (Lin et al., 2019), we set rules like soft matching with lemmatization and filtering of stop words to enhance the n-gram matching performance. After that, we can obtain two sequences:
x = {x1, x2, ..., xN}, c = {c1, c2, ..., cN},
(1)
where x is the original text sequence. c is matched concept sequence, which means that ci is the matched concept of xi. For n-gram concepts, we align them to the first token in its corresponding phrases in x (Zhang et al., 2019). If there is no matched concept for token xi, we set ci = [PAD].
Concept Pre-training. After the concept recognition process, we can obtain the set of concepts mentioned in the whole dataset. We retain these mentioned concepts and their related concepts (first-order neighbors) in the original knowledge graph G1, thus yielding a new pruned knowledge graph G′1. Subsequently, we utilize the TransE (Bordes et al., 2013) model on G′1 to pretrain concept embedding U ∈ RNc×v, where Nc is the number of concepts, v indicates the embedding size. This pre-trained concept embedding will be used as initialization in KTE module (Section 4.1).

heading: 4 K-HTC Model
text: In this section, we will introduce the technical details of K-HTC model. As Figure 2 shows, K-HTC consists of three components: 1) Knowledge-aware Text Encoder (KTE); 2) Knowledge-aware Hierarchical Label Attention (KHLA); 3) Knowledgeaware Contrastive Learning (KCL).

heading: 4.1 Knowledge-aware Text Encoder
text: In this part, we aim to obtain the knowledgeaware representation of the given text by integrating external knowledge from KGs. As illustrated in Figure 3, given a token sequence x = {x1, x2, ..., xN} and its corresponding concept sequence c = {c1, c2, ..., cN}, we first apply the pretrained language encoder (i.e., BERT) to compute its word semantic embedding:
{w1, ..., wN} = BERT ({x1, ..., xN}). (2)
Regarding the concept sequence c, we map each concept into the embedding space via the pretrained TransE embedding U :
{u1, ..., uN} = U({c1, ..., cN}). (3)
Subsequently, for each concept ci, we randomly select k neighbors in the pruned knowledge graph G′1 to conduct the GraphSAGE algorithm (Hamilton et al., 2017), which can aggregate its context information in the KG:
u′i = GraphSAGE(ui, Gk), (4)
where Gk is the context graph composed of ci and its k neighbors, u′i ∈ Rv is the aggregated representation of concept ci. After that, we fuse the word semantic representation wi and its corresponding concept representation u′i:
{m1, ...,mN} = {w1 + u′1, ..., wN + u′N}, (5)
where + refers to the point-wise addition. We call {m1, ...,mN} as knowledge-aware representation.

heading: 4.2 Knowledge-aware Hierarchical Label Attention
text: In this part, we first learn the label representation via external knowledge and taxonomic hierarchy, and then conduct label attention to obtain the classenhanced document representation.
Label Representation Learning. With the Knowledge-aware Text Encoder (KTE), we can obtain the knowledge-aware representation of hierarchical labels via their label names:
Ril =mean(KTE(Li)), i = 1, ...,K,
Rl = [R 1 l , R 2 l , ..., R K l ],
(6)
where Li is the name of label i, Ril ∈ Rv is the representation of label i, while Rl ∈ RK×v indicates the representation of all labels.
Then, we adopt GCN layer to propagate the representation of labels on the label hierarchy graph G2. Specifically, it takes the feature matrix H(l) and the matrix Ã as input, and updates the embedding of the labels by utilizing the information of adjacent labels:
H(l+1) = σ(D̃− 1 2 ÃD̃− 1 2H(l)W (l)), (7)
where Ã = A + I , A is the adjacency matrix of G2, I is the identity matrix, D̃ = ∑ i Ãij , and W (l) is a layer-specific trainable weight matrix. σ denotes a non-linear activation function (e.g., ReLU). We set H(0) = Rl, and the last hidden layer is used as the propagated label representation, i.e., H = H(l+1) ∈ RK×v. Label Attention. After that, we apply the propagated label representation H to perform K different classes of attention to the input document:
Rd = KTE(D),
O = tanh(Wo ·RTd ), Watt = softmax(H ·O),
(8)
where D is the input document, Rd ∈ RN×v is the knowledge-aware representation of D. Wo ∈ Rv×v is a randomly initialized weight matrix, and the softmax() ensures all the computed weights sum up to 1 for each category. Watt ∈ RK×N denotes the attention matrix.
Subsequently, we compute weighted sums by multiplying the attention matrix Watt and the document representation Rd:
M1 = mean(Watt ·Rd), (9)
where M1 ∈ Rv represents the class-enhanced representation for the document.
Furthermore, inspired by (Wang et al., 2022b), we utilize another randomly initialized label embedding H2 ∈ RK×v to perform the same operation in Eq.(8-9) and obtain another class-enhanced document representation M2. Finally, we concat the M1, M2 and the [CLS] representation from BERT encoder as the final representation:
Rcat = concat(M1,M2, H[CLS]),
Rf = Wf ·Rcat + bf , (10)
where Wf ∈ Rv×3v is a randomly initialized weight matrix, bf ∈ Rv is corresponding bias vector, Rf ∈ Rv is the final document representation.

heading: 4.3 Knowledge-aware Contrastive Learning
text: As we discussed in Section 1, the documents in the same category may share more concepts in the knowledge graph, while documents in different categories do not (more analysis about this phenomenon can be found in Appendix A). Therefore, we propose a contrastive learning strategy to further exploit the information inherent in the data. Specifically, we design this from both knowledge-driven and hierarchy-driven perspectives.
Knowledge-driven CL. In this part, we aim to close the distance between documents that share more concepts in the knowledge graph. Specifically, inspired by (Wang et al., 2022a), in a minibatch of size b, we define a function to output all other instances for a specific instance i: g(i) = {k|k ∈ {1, 2, ..., b}, k ̸= i}. Then the knowledgedriven contrastive loss for each instance pair (i, j) can be calculated as:
Lijc = −βij log e−d(zi,zj)/τ∑
k∈g(i) e −d(zi,zk)/τ , (11)
cij = |Ci ∩ Cj |, βij = cij∑
k∈g(i) cik , (12)
where τ is the temperature of contrastive learning, d(·, ·) is the euclidean distance and zi represents the final representation Rf of document i. Ci is the concept set in document i, cij indicates the number of shared concepts in document i and j, and βij is the normalization of cij .
The contrastive loss for the whole mini-batch is the sum of all the instance pairs: Lc =∑
i ∑ j∈g(i) L ij c . With this contrastive loss, for an instance pair (i, j), the more concepts they share,
the larger the weight βij will become, thus increasing the value of their loss term Lijc . In consequence, their distance d(zi, zj) will become closer. On the contrary, if they share fewer concepts, their distance d(zi, zj) will be optimized relatively farther.
Hierarchy-driven CL. In addition to the knowledge-driven CL, we can optimize the document representation via hierarchical label structure. As illustrated in Figure 4, document D1 and D2 share two labels in the hierarchy, while D1 and D3 only share one. Naturally, the distance between D1 and D2 should be closer than that between D1 and D3. From this perspective, in a mini-batch, we calculate the number of shared labels between document i and j:
lij = |Yi ∩ Yj |, (13)
where Yi means the label set of document i. Then, we use lij to replace cij in Eq.(12), and further calculate another contrastive loss Lijh following Eq.(11). After that, we sum this loss across the whole mini-batch and obtain the hierarchy-driven contrastive loss Lh = ∑ i ∑ j∈g(i) L ij h .

heading: 4.4 Output Layer
text: Output Classifier. Following the previous work (Zhou et al., 2020), in the output layer, we flatten the hierarchy for multi-label classification. We feed the final document representation Rf in Eq.(10) to a two-layer classifier:
Q = φ(Wq ·Rf + bq), P = σ(Wp ·Q+ bp),
(14)
where Wq ∈ Rv×v, Wp ∈ RK×v are ramdomly initialized weight matries, bq ∈ Rv, bp ∈ RK are corresponding bias vectors, φ is a non-linear activation function (e.g., ReLU), while σ is the sigmoid activation. P is a continuous vector and each element in this vector Pi denotes the probability that the document belongs to category i.
Training. For multi-label classification, we choose the binary cross-entropy loss function for document i on label j:
Lijbce = −yij log(pij)−(1−yij) log(1−pij), (15)
Lbce = ∑
i
K∑
j=1
Lijbce, (16)
where pij is the prediction score, yij is the ground truth. The final loss is the combination of the classification loss and the two constrastive losses:
L = Lbce + λcLc + λhLh, (17)
where λc and λh are hyperparameters that control the weights of two contrastive losses.

heading: 5 Experiment

heading: 5.1 Experiment Setup
text: Datasets and Evaluation Metrics. We conduct experiments on the BlurbGenreCollection-EN (BGC)1 and Web-of-Science (WOS)2 (Kowsari et al., 2017) datasets. BGC consists of advertising descriptions of books, while WOS contains abstracts of published papers from Web of Science. More statistics about the datasets are illustrated in Table 1. As for the knowledge graph, we adopt the advanced knowledge graph named ConceptNet (Speer et al., 2017).
We measure the experimental results with standard evaluation metrics (Gopal and Yang, 2013; Liu et al., 2020; Zhang et al., 2022), including MacroPrecision, Macro-Recall, Macro-F1 and Micro-F1.
Implementation Details. In the Knowledge Pretraining part, we utilize OpenKE (Han et al., 2018)
1https://www.inf.uni-hamburg.de/en/inst/ab/lt/ resources/data/blurb-genre-collection.html
2https://data.mendeley.com/datasets/ 9rw3vkcfy4/2
to train concept embedding via TransE. The dimension of TransE embedding is set to 768.
We adopt bert-base-uncased from Transformers (Wolf et al., 2020) as the base architecture. In KTE module, when we conduct GraphSAGE to aggregate the neighbor information to concepts, we set the neighbor num k = 3 for each concept. We choose the mean aggregator as the aggregation function of GraphSAGE and the layer is set to 1. In KHLA module, the layer of GCN is set to 1. In KCL module, we set the contrastive learning temperature τ = 10 for knowledge-driven CL, while τ = 1 for hierarchy-driven CL. The dimension of hidden states is set to v = 768 in this paper. As for the loss weight in Eq.(17), λh is set to 1e − 4 on both BGC and WOS, while λc is set to 1e− 3 on BGC and 1e− 2 on WOS 3.
The batch size is set to 16, and our model is optimized by Adam (Kingma and Ba, 2014) with a learning rate of 2e − 5. We train the model with train set and evaluate on development set after every epoch, and stop training if the Macro-F1 does not increase for 10 epochs. We run all experiments on a Linux server with two 3.00GHz Intel Xeon Gold 5317 CPUs and one Tesla A100 GPU 4.
Benchmark Methods. We compare K-HTC with the state-of-the-art HTC methods.
• HiAGM5 (Zhou et al., 2020) exploits the prior probability of label dependencies through a GCN-based structure encoder.
• HTCInfoMax6 (Deng et al., 2021) considers the text-label mutual information maximization and label prior matching in HTC.
• HiMatch7 (Chen et al., 2021) mines the relative distance between texts and labels, which also provides a plus version based on BERT.
• HGCLR8 (Wang et al., 2022b) designs a contrastive learning method to embed the hierarchy into the BERT encoder.
• HPT9 (Wang et al., 2022c) introduces prompt learning into HTC problem, which proposes a novel multi-label MLM perspective.
3We tune these hyperparameters on the development set to obtain the best hyperparameter settings.
4Our code is available via https://github.com/ liuyeah/K-HTC.
5https://github.com/Alibaba-NLP/HiAGM 6https://github.com/RingBDStack/HTCInfoMax 7https://github.com/qianlima-lab/HiMatch 8https://github.com/wzh9969/contrastive-htc 9https://github.com/wzh9969/HPT
• KW-BERT (Jang et al., 2021) is the advanced text classification method that incorporates knowledge graphs, which also adopts BERT as the text encoder.
Among these baselines, only HiAGM and HTCInfoMax do not adopt the BERT encoder. For fair comparison, we implement them with BERT encoder, and denote them as HiAGM+BERT and HTCInfoMax+BERT.

heading: 5.2 Experimental Result
text: The main results are shown in Table 2. Our proposed K-HTC method outperforms all baselines in all metrics, except for HPT in Micro-F1 on the BGC dataset, which proves the effectiveness of our method and the necessity to incorporate knowledge graphs. Moreover, there are also some interesting phenomena from these results:
First, the differences between the hierarchyaware methods (i.e., HiAGM, HTCInfoMax and HiMatch) and their BERT-variants are more pronounced on BGC than on WOS. In detail, The depth of WOS is 2, and each document is labeled with one label on each level. However, the depth of BGC is 4, and the number of labels per document is unfixed10. As a result, the BGC dataset is more difficult than WOS, and it may be more conducive to the role of BERT. Another consideration is the pre-trained corpora of BERT. One of the pre-trained datasets of BERT is BookCorpus (Zhu
10The average number is 3.01. Please recall Table 1 for more details.
et al., 2015), which is the same document type as BGC. This also plays a great role in improving the model’s effectiveness. Second, with the help of the external KG and the proposed knowledge-infused attention mechanism, KW-BERT achieves good results on both two datasets as well. However, it performs relatively poorly compared with K-HTC, which demonstrates the effectiveness of our model design from another perspective. Third, although HPT achieves a slight ahead over K-HTC in MicroF1 on the BGC dataset, it regresses obviously on other metrics. In detail, Micro-F1 directly takes all the instances into account, while Macro-F1 gives equal weight to each class in the averaging process. For the multi-label classification with complex label structures, Macro-F1 is harder and more differentiated, which can better reflect the model capability. We further conduct the significance test in Appendix B.

heading: 5.3 Ablation Study
text: In this subsection, we conduct ablation experiments to prove the effectiveness of different components of K-HTC model. We disassemble K-HTC by removing the KTE, KHLA, and KCL modules in turn. In particular, removing KTE indicates that the text encoder degenerates to the traditional BERT encoder. After removing KHLA, K-HTC pays little attention on the interaction between documents and labels, and thus we directly conduct mean pooling on the output of KTE to obtain the final representation Rf of the document. Finally, omitting KCL means that we directly omit two contrastive losses
in Eq.(17) in the training process.
The results on the BGC and WOS datasets are listed in Table 3 and Table 4, respectively. From these statistics, we can find that there are obvious decreases in all ablation variants, which thoroughly demonstrates the validity and non-redundancy of our K-HTC method. Additionally, on the BGC dataset, the importance of KHLA module is relatively stronger than other modules. It is reasonable as BGC has a more complicated label hierarchy, which puts higher demands for label learning and the interaction between the documents and labels.

heading: 5.4 Effect of Knowledge on Different Levels
text: To further verify the effect of incorporating knowledge, we analyze the performance of K-HTC and its ablation variants on different levels of the BGC dataset. Specifically, BGC has four levels of labels, with the granularity of classification getting finer from top to bottom. Figure 5 deposits the performance comparison on different levels. It is clear that as the level deepens, the performance of all methods decreases, indicating the classification difficulty increases significantly. At the same time, the gap between K-HTC and its ablation variants widens as the depth increases. This suggests that incorporating knowledge can help improve the classification effectively, especially for these deeper and more difficult levels. Furthermore, the situation is more evident in the comparison between K-HTC and its variant -w/o KHLA, which is consistent with the analysis in Section 5.3.
No. λc λh Macro-F1 Micro-F1
K-HTC

heading: 5.5 Parameter Sensitivity
text: To study the influence of the loss hyperparameters λc and λh in K-HTC, we conduct comprehensive parameter sensitivity experiments on the WOS dataset. The results are reported in Table 5.
The first experiment is the best hyperparameters of our model. In experiment ② ∼ ④, we fix λh and fine-tune λc; in experiment ⑤ ∼ ⑧, λc is fixed and λh is fine-tuned. From the results, we find that the larger or smaller λc will lead to an obvious decrease on the classification performance. The same situation happens to λh. It is reasonable as these two hyperparameters control the weights of two contrastive losses. Too large weight will affect the original BCE classification loss, while too small weight will restrict its own effect.

heading: 5.6 Case Study
text: To further illustrate the effect of incorporating knowledge graphs in the K-HTC model, we conduct case study on both WOS and BGC datasets. Specifically, in Figure 6 and 7, we present the in-
put document, the knowledge retrieved from KG, the ground truth and the prediction of K-HTC, respectively. As shown in Figure 6, with the help of the knowledge (Neural_Network, is_a, Machine_Learning) and (Neural_Network, related_to, Computer), K-HTC reasonably makes the correct inference, i.e., Computer Science and Machine Learning in the taxonomic hierarchy. A similar situation can be found in the case of Figure 7 as well. These intuitively demonstrate the great role of knowledge and further verify the validity of our K-HTC method.
More experimental analyses, such as Visualization and Bad Case Analysis, can be found in Appendix C and D.

heading: 6 Conclusions
text: In this paper, we explored a motivated direction for incorporating the knowledge graph into hierarchical text classification. We first analyzed the necessity to integrate knowledge graphs and further proposed a Knowledge-enabled Hierarchical Text Classification model (K-HTC). Specifically, we designed a knowledge-aware text encoder, which could fuse the text representation and its corresponding concept representation learned from KGs. Subsequently, a knowledge-aware hierarchical label attention module was designed to model the interaction between the documents and hierarchical labels. More importantly, we proposed a knowledge-aware contrastive learning strategy, which could further boost the classification performance by exploiting the information inherent in the data. Finally, extensive experiments on two publicly available HTC datasets demonstrated the effectiveness of our proposed method. We hope our work will lead to more future studies.
Limitations
In our proposed K-HTC method, incorporating the knowledge graph requires the concept recognition and pre-training process, as we introduced in Section 3.2. This process may consume additional time compared with other HTC methods, but it can be done in advance and does not need to be repeated, making it suitable for both research and industrial settings. Besides, due to the errors of concept recognition algorithms, this process may introduce some noisy information in reality. This will interfere with the use of knowledge. In future work, we will attempt to utilize entity linking algorithms (Wang et al., 2023) to further guarantee the quality of recognized knowledge.
Another limitation is that we utilize the label name in the KHLA module. It may not be available for some datasets with only label ids. In response to this, we can select high-frequency keywords from documents in each category, which play the same role as the label name.

heading: Acknowledgements
text: This research was partially supported by grants from the National Natural Science Foundation of China (Grants No. U20A20229, No. 62106244), and the National Education Examinations Authority (Grant No. GJK2021009).

heading: A Data Analysis
text: We calculate the average number of shared concepts between two arbitrary documents in the same category. Table 6 illustrates this situation on different levels. The "Total" line reports the average number of shared concepts between two arbitrary documents across the whole dataset, which can be adopted as the comparison standard.
We could find that the shared concepts increase as the depth deepens, except for a slight fluctuation on the fourth level of BGC. Besides, the results on different levels are all significantly larger than the "Total" line. These findings provide valid support for the Knowledge-aware Contrastive Learning (KCL) module in Section 4.3.

heading: B Significance Analysis
text: In Table 2, the experimental results of our KHTC model and HPT are relatively close. To better demonstrate the superiority of K-HTC, we do the Student t-test to clarify whether K-HTC performs better than HPT. Specifically, we repeat the experiment five times with different seeds on both BGC and WOS datasets, and report the p-value results on Macro-F1. From the results in Table 7, we find that both two results are smaller than the significance level 0.05. Therefore, we reject the hypothesis that the performances between K-HTC and HPT are approximate. It suggests that K-HTC is more effective than HPT in most circumstances.
C Visualization
In K-HTC, we design a Knowledge-aware Hierarchical Label Attention (KHLA) module to learn the label representation, which can further mine the interaction between documents and labels. In the hierarchical label structure, it is expected that labels with the same parent have more similar representations than those with different parents. To verify this, we plot the T-SNE projections of the learned label embedding (i.e., H learned from Eq.(7)) on the WOS dataset. Specifically, the depth of the WOS label hierarchy is 2. In Figure 8, the left part plots child labels on the second level, while the right part indicates the parent labels on the first level. From this figure, we can find that labels with the same parent are clearly clustered together, while labels with different parents are significantly farther apart from each other. This thoroughly demonstrates the effectiveness of the KHLA module.

heading: D Bad Case Analysis
text: As we discussed in the Limitations section, we incorporate the knowledge graph via the concept
recognition process. This may introduce some noise due to the inevitable errors of the recognition algorithm. More importantly, even with accurate concept recognition results, how to ensure the effectiveness of external knowledge is still a challenge. An example of this can be observed in Figure 9, although our method accurately recognizes the mentioned concepts Clark and Scott, it still introduces the noisy knowledge (Clark, related_to, USA) and (Scott, related_to, USA). As a result, K-HTC makes a wrong prediction, i.e., Travel: USA & Canada. This indicates that we need to focus on the quality of relevant knowledge rather than roughly introducing all of them.
In future work, we will attempt to design a knowledge filtering module to ensure the quality of introduced knowledge, which can further improve the performance of K-HTC.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section 7 (after the Conclusions)
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 5.1

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 5.1
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 5.1
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 5.1
C 3 Did you run computational experiments? Section 5
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Not applicable. The efficiency is not our focus or goal in this work.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 5.1 and Section 5.5

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 5 and Appendix B

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 5.1
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: In future work, we will attempt to design a knowledge filtering module to ensure the quality of introduced knowledge, which can further improve the performance of K-HTC. ACL 2023 Responsible NLP Checklist