Peer-Label Assisted Hierarchical Text Classification
Year: 2023

Abstract:
Hierarchical text classification (HTC) is a challenging task, in which the labels of texts can be organized into a category hierarchy. To deal with the HTC problem, many existing works focus on utilizing the parent-child relationships that are explicitly shown in the hierarchy. However, texts with a category hierarchy also have some latent relevancy among labels in the same level of the hierarchy. We refer to these labels as peer labels, from which the peer effects are originally utilized in our work to improve the classification performance. To fully explore the peer-label relationship, we develop a PeerHTC method. This method innovatively measures the latent relevancy of peer labels through several metrics and then encodes the relevancy with a Graph Convolutional Neural Network. We also propose a sample importance learning method to ameliorate the side effects raised by modelling the peer label relevancy. Our experiments on several standard datasets demonstrate the evidence of peer labels and the superiority of PeerHTC over other state-of-the-art HTC methods in terms of classification accuracy.

heading: 1 Introduction
text: Hierarchical text classification (HTC) is a multilabel text classification problem which aims to classify texts into categories that can be organized into a taxonomic hierarchy. It is an important problem in natural language processing and has attracted increasing attention in both industrial and academic fields. Typical HTC problems include patent categorization (Gomez and Moens, 2014), medical record coding (Cao et al., 2020), and product categorization (Cevahir and Murakami, 2016).
Due to the complexity of category hierarchy, the problem of hierarchical text classification is more challenging than plain text classification. The parent-child relationships between categories in adjacent levels of the hierarchy are usually defined
∗ Corresponding author.
in advance. Then a natural way to solve the HTC problem is to incorporate this prior knowledge into the model, i.e., making the model aware of the hierarchy. Building "hierarchy-aware" models is beneficial for HTC, which is particularly true for categories with few samples. Therefore, it has long been the main focus in HTC to figure out the most effective way of utilizing the category hierarchy to improve the classification performance.
In the past literature, existing approaches for HTC can be generally categorized into three groups: local approaches, global approaches, and local-global-combined ones (also known as hybrid approaches). The local approaches train local classifiers for every child label, every parent label or every level in the hierarchy (Shimura et al., 2018; Banerjee et al., 2019). The parameters of local classifiers are initialized in a top-down fashion according to the category hierarchy. However, these approaches usually contain a large number of parameters, and the whole hierarchy cannot be fully captured merely by parameter initialization. Global approaches, which are popular in recent years, aim to flatten HTC into a multi-label classification problem, and then incorporate the information of category hierarchy in various ways, such as using regularization terms (Gopal and Yang, 2013), modeling the architecture of category hierarchy (Zhou et al., 2020), and using contrastive learning (Wang et al., 2022). The local-global-combined approaches can be seen as an improvement on local approaches, which construct the information flow between local classifiers in more effective ways, and meanwhile utilize a global classifier to coordinate local ones (Wehrmann et al., 2018; Rojas et al., 2020). However, these models might still suffer from error propagation (Rojas et al., 2020), because the classification of child layers are dependent on that of their parents.
To the best of our knowledge, existing methods only exploit category relevancy that is explicitly
3747
reflected in the hierarchy. For example, Gopal and Yang (2013) used a recursive regularization term in which the parameters of parent labels are expected to be similar to those of their children. Zhou et al. (2020) used a structure encoder for labels, in which the information of parent and child labels are integrated into each label’s representation. However, there could still exist some latent relevancy among the labels in the same level, which could also be beneficial to the HTC problem. Take the BlurbGenreCollection dataset (Aly et al., 2019) as an example, which consists of descriptions and genres of books. In this dataset, two third-level book categories "World History" and "Travel: Asia" belong to different second-level categories "History" and "Travel", respectively. However, these two thirdlevel categories both involve geographical and cultural contents. Therefore, intuitively they should share some common characteristics, and the classification of one category could benefit that of the other. The phenomenon that labels in the same level possess latent relevancy is similar to the "peer effect" existing among peer friends. Thus we call these labels as "peer labels" in this work.
To utilize the latent relevancy of peer labels to improve the HTC performance, we develop a PeerHTC method. It incorporates two types of label relationships: the parent-child relationship explicitly reflected in the hierarchy, and the peer-label relationship implicitly hidden in the hierarchy. We propose several measures to learn the relevancy structure among peer labels, and then utilize the Graph Convolutional Neural Network (GCN) to realize "feature sharing" among peer labels. To address the possible side effect caused by modeling peer labels, we also develop a measure to evaluate the degree of confusion between labels in the same level, and then assign different weights to training samples according to their contribution in alleviating label confusion. The PeerHTC method is realized through an embedded two-stage training approach, in which valuable information about latent relevancy of peer labels and the label confusion can be harvested from the first round of warm-up training and then enhances the second round for final classification.
The rest of this article is organized as follows: Section 2 introduces related works. Section 3 defines the HTC problem. Section 4 introduces the PeerHTC method in detail. Section 5 presents the experimental results on three datasets. Section
6 concludes the article. We share our codes on GitHub1 for reproducibility.

heading: 2 Related Work

heading: 2.1 Local Approaches
text: The local approaches train local classifiers for each category or each level in the hierarchy. These local classifiers are initialized in a top-down fashion according to the category hierarchy so that knowledge learned by each parent classifier can be transferred to their children. For example, the method HTrans (Hierarchical Transfer Learning) (Banerjee et al., 2019) trained a binary classifier for each label, and then initialized the classifiers according to their parents. The method HFTCNN (Hierarchical Finetuning Based CNN) (Shimura et al., 2018) trained a multi-label classifier for each level in the category hierarchy, and then followed a similar approach for parameter initialization. However, these models usually have a large number of parameters to estimate and also suffer from insufficient use of the category hierarchy.

heading: 2.2 Global Approaches
text: Global approaches flatten HTC into a simple multilabel classification problem, and seek to incorporate the information of category hierarchy in various ways. For example, Gopal and Yang (2013) imposed recursive regularization on parameters of parent and child nodes. The method HiAGM (Zhou et al., 2020) includes two variants, i.e., HiAGM-LA and HiAGM-TP. In HiAGM-LA, texts and labels are encoded separately, and multi-label attention mechanism is used to extract label-wise features. A structure encoder is also used to aggregate prior category hierarchy information into label embeddings. In HiAGM-TP, label embeddings are not used and text features are directly propagated through the structure encoder. The method HTCInfoMax (Hierarchical Text Classification via Information Maximization) (Deng et al., 2021) seeks to improve HiAGM-LA with mutual information maximization that constrains text and label representations. The method HiMatch (Hierarchy-aware Label Semantics Matching Network) (Chen et al., 2021) projects the representations of words and labels into a common latent space and utilizes hierarchyaware matching learning. The method HGCLR (Hierarchy-Guided Contrastive Learning) (Wang et al., 2022) models texts and labels separately only
1https://github.com/WoodySJR/PeerHTC
in the training process, and then incorporates the information of category hierarchy into the text encoder via contrastive learning.

heading: 2.3 Hybrid Approaches
text: Local-global-combined approaches(or hybrid approaches) can be seen as an improvement on local ones. The method HMCN (Hierarchical Multilabel Classification Networks) (Wehrmann et al., 2018) is probably the first hybrid model. In HMCN, local classifiers are arranged in series and global classification is conducted to coordinate these local classifiers. The method HARNN (Hierarchical Attention-based Recurrent Neural Network) (Huang et al., 2019) is another typical hybrid model. It shares a similar architecture with HMCN, but uses the multi-label attention mechanism to extract label-wise text features. However, since errors in the prediction of higher-level categories may provide misleading information for lower levels, these hybrid approaches might suffer from error propagation (Rojas et al., 2020).

heading: 3 Problem Formulation
text: We define the HTC problem in this section. Specifically, we first give the definition of a category hierarchy and its properties, and then define the HTC problem mathematically.
Definition 1. (Category Hierarchy) Assume there exists an H-level category hierarchy γ. All possible labels in γ are denoted by C = {C1, C2, · · · , CH}, where Ci = {c1, c2, · · · } ∈ {0, 1}|Ci| is the label set in the i-th level, and |Ci| is the total number of labels in Ci. Consequently, the total number of labels in C is K = ∑H i=1 |Ci|. The category hierarchy γ is then defined to be a partially order set (C,≺), where ≺ represents the superior-subordinate relationship between labels and it satisfies the following three properties:
• asymmetry: ∀cx ∈ Ci and cy ∈ Cj , if cx ≺ cy, then we have cy ⊀ cx.
• anti-reflexivity: ∀cx ∈ Ci, we have cx ⊀ cx.
• transitivity: ∀cx ∈ Ci, cy ∈ Cj and cz ∈ Ck, if cx ≺ cy and cy ≺ cz , then we have cx ≺ cz.
Definition 2. (Hierarchical Text Classification, HTC) Given a category hierarchy γ = (C,≺), assume there exist a total number of M documents denoted by D = {(D1, L1), (D2, L2), · · · , (DM , LM )}. Here Dd
denotes the dth text document, which is typically a sequence of words, i.e., Dd = {wd1, wd2, · · · , wdNd}, where Nd is the total number of words in document Dd. Define Ld = {ld1, ld2, · · · , ldH} to be the label set of the dth document with the i-th level label set ldi ⊂ Ci. Then the goal of HTC is to train a classification model Ω based on γ and D. Specifically, for an arbitrary text document D∗, we can predict its label set L∗ through the classification model, i.e.,
Ω(D∗, γ,Θ) → L∗,
where Θ is the parameters in the model Ω.

heading: 4 Methodology
text: In this section, we introduce the PeerHTC method in detail. We first introduce peer label learning and sample importance learning, and then propose a two-stage training procedure. The overall architecture of PeerHTC is illustrated in Figure 1.

heading: 4.1 Peer Label Learning

heading: 4.1.1 Latent relevancy encoding by GCN
text: As we mentioned before, there exist latent relationships among peer labels, which are not explicitly expressed in the category hierarchy. Incorporating
the latent relevancy structure among peer labels could also benefit HTC. Motivated by this idea, we consider learning label relevancy from two perspectives. First, we follow the practice of HiAGM (Zhou et al., 2020) to learn the parent-child relationships between labels in adjacent levels. Second, we utilize GCN to incorporate the latent relevancy among peer labels into the label encoder.
We encode labels and texts separately in PeerHTC. For labels, let V ∈ Rdv×K denote the initial label embeddings, where dv is the embedding dimension and K is the total number of labels. Then following HiAGM (Zhou et al., 2020), we feed the initial embeddings V into a TreeLSTM encoder to learn the hierarchy-aware embeddings H↕, where the symbol "↕" stands for the parent-child relationships. Actually, H↕ is the concatenation of two sets of embeddings derived in top-down and bottom-up fashions, i.e., H↕ = H↑⊕H↓; please refer to Zhou et al. (2020) for more details.
To characterize the latent relationships of peer labels, we use H↔ = GCN(V ), which is derived from latent label connections enabled by GCN. We refer to H↔as peer-aware embeddings in the subsequent analysis. To fully explore the latent relevancy of peer labels, we propose GCN methods using two strategies. The first one is level-wise GCN, which only incorporates connections of labels in the same level. Specifically, define A to be the adjacent matrix that tells how labels should be connected. Define W and b to represent the weight matrix and bias term, which are both trainable. Let σ(·) denote ReLU non-linear activation function. Then in level-wise GCN, we first compute H↔(h) for levels 1 ≤ h ≤ H , and then concatenate them together. The detailed computation is shown below.
H↔(h) = σ(A(h)V ⊤ (h)W(h) + b(h)) ⊤,
H↔ = concat{H↔(1), H↔(2), · · · , H↔(H)}. (1)
The second method is to use whole-hierarchy GCN, which is a single GCN for labels in the whole hierarchy. This strategy allows for label connections throughout the whole hierarchy. The peeraware embeddings are then computed as follows:
H↔ = σ(AV ⊤W + b)⊤. (2)
After computing the hierarchy-aware embeddings H↕ and peer-aware embeddings H↔, we concatenate them together. Specifically, we have H↔ concatenated column-wise with H↕, the result
of which is then put through a non-linear projection. This leads to the final label embeddings H∗, which is computed as follows:
H∗ = σ{W ∗ · (H↕ ⊕H↔)},

heading: 4.1.2 Initialization of the adjacent matrix A
text: By using GCN as the latent relevancy encoder to model peer labels, we need to specify the adjacent matrix A in advance, i.e., to tell how labels should be associated with graph edges. To this end, we propose a data-driven approach to initialize A. Inspired by the idea of knowledge distillation (Hinton et al., 2015), the estimated label probabilities (also called soft labels in knowledge distillation) contain extra knowledge on the relationships among different labels. Therefore, the estimated label probabilities can be regarded as a good source to learn the latent relevancy among peer labels. Specifically, if two labels are closely related with each other, their estimated label probabilities should tend to be correlated on the same sample. Therefore, a similarity measure between the estimated probabilities of two labels can reflect how closely they are related. Besides, recall that we have computed the label embeddings H∗, the similarity among which could also reflect label relevancy.
Based on the above considerations, we propose two methods for initializing A. In the first method, we adopt the non-parametric Spearman Rank Correlation Coefficient (SRCC) to measure the similarity between estimated probabilities. Let pdk be the estimated probability of the dth document associated with the kth label. Recall there are a total of M documents. Hence we can compute the rank of pdk among the estimated probabilities of the M documents (i.e., p1k, ..., pMk), which is denoted by rdk. We then compute r̄k, which is the average of rdk among M documents. Then, we can compute absolute SRCC for any two labels k and j as follows
ρrankkj = ∣∣∣∣∣∣ ∑M d=1(rdk − r̄k)(rdj − r̄j)√∑M
d=1(rdk − r̄k)2 ∑M d=1(rdj − r̄j)2
∣∣∣∣∣∣ .
(3) The SRCC measure can be computed on either training samples or test samples, since it does not require true labels.
In the second method, we measure label relevancy based on label embeddings. Specifically, let hk and hj be the embeddings for labels k and j, which are extracted from H∗. Then we can use the
absolute cosine similarity between them to measure their relevancy, i.e.,
ρembkj = ∣∣∣∣ h⊤k hj
∥hk∥ · ∥hj∥
∣∣∣∣ . (4)
After getting the similarity measures (ρrankkj or ρembkj ), they are aligned into a matrix and then normalized row-wise (except for the diagonal entries so they remain to be one). This leads to two matrices Arank and Aemb, which would then be used in the initialization of GCN. We empirically compare the performance of different initialization methods of A; see Section 5.2.3 for the detailed results.

heading: 4.1.3 Multi-label attention
text: We adopt the multi-label attention mechanism to extract label-wise text features (Huang et al., 2019; Zhou et al., 2020; Deng et al., 2021). For the dth document with Nd words, let s̃d = {sd1, ...sdNd} denote the word representations derived from a text encoder. Recall that hk is the embedding of label k, which is extracted from H∗. Then within the dth document, we can compute the attention score α(d)kn between the representation of the nth word and the embedding of label k, i.e.,
α (d) kn = exp{s⊤dnhk}∑Nd g=1 exp{s⊤dghk} .
The value α(d)kn indicates how informative the nth word is to a certain label k within one document.
Note that in PeerHTC, label embeddings have now included two parts of information, i.e., the hierarchical relationship between parent and child labels and the latent relationship between peer labels. Hence, the attention score α(d)kn is also equipped with the ability to identify text features favoring labels closely related to label k. This leads to reinforced feature sharing. Finally, we calculate a weighted average u(d)k = ∑Nd g=1 α (d) kn sdg for label k. These features are then flattened and fed into a fully-connected network for final classification.

heading: 4.2 Sample Importance Learning

heading: 4.2.1 A metric for label confusion
text: Assisted by GCN to model the latent relevancy of peer labels, we achieve reinforced feature sharing that would enhance the classification of one category with the help of text features extracted by other closely related categories. However, a side effect emerges when we strengthen the similarity
between the embeddings of peer labels by GCN. That is, it would make easily confused labels become even less distinguishable. To characterize this phenomenon, we first formalize a new concept called "label confusion". Specifically, we say there is confusion between two labels k and j, when one document belongs to label k but gets a high probability in another label j, or the other way around. Take two book categories named "Classics" and "Poetry" for example. They are prone to confusion since both of them involve some genteel expression. More intuitively, label confusion is pretty much like the case where a person gets confused when distinguishing between very similar objects.
To tackle this potential side effect, we first propose a metric to evaluate how easily any two labels can be confused. Let L(d) denote the true label set of the dth document. Assume we have label k ∈ L(d) but label j /∈ L(d). Then the estimated probability of label j measures the confusion between these two labels on this document. To formulate this idea mathematically, let pdk be the estimated probability that the dth document belongs to label k. Let ckj denote the degree of confusion between labels k and j. Denote the index set Dkj = {d : 1 ≤ d ≤ M,k ∈ L(d), j /∈ L(d)}. Then we can compute ckj as follows,
ckj = 1 |Dkj | ∑
d∈Dkj pdj . (5)

heading: 4.2.2 Training with sample weighting
text: A document sample is said to be important in distinguishing labels k from j, if its label set contains k but not j. With the metric of label confusion ckj , we can evaluate the importance of each training sample. Specifically, define βdk to be the importance of the dth document with respect to a label k. Then in the case k /∈ L(d), we set βdk = 1. In the case k ∈ L(d), we specify βdk as follows:
βdk = 1 + ∑
j /∈L(d) {exp(τckj)− 1}, (6)
where τ is a temperature hyperparameter controlling how radical we are in assigning sample weights. We then plug βdk into the binary cross entropy loss (BCE) function, which is popularly used in HTC (Nam et al., 2014), i.e.,
L = − ∑
d∈D
∑ k∈C βdk{yk log(pdk)
+ (1− yk) log(1− pdk)},
where yk is either 1 or 0 depending on whether k ∈ L(d) or not.

heading: 4.3 A Two-Stage Training Approach
text: As mentioned in Section 4.1 and Section 4.2, we use a data-driven approach to identify the adjacent matrix A and sample importance weights βdk’s, which all rely on the estimated label probabilities and label embeddings. To obtain the adjacent matrix and sample importance weights, we develop a two-stage training approach. The first round is a warm-up training stage. We randomly initialize the adjacent matrix A in GCN, and assign equal weights to all training samples. Then we train the PeerHTC model for the first time. The obtained estimated label probabilities and label embeddings from the warm-up training are then used to compute the adjacent matrix and sample importance weights. Then we re-train the PeerHTC model for the second time with the updated adjacent matrices and sample weights. This leads to the final classification model. The whole procedure is illustrated in Algorithm 1.
Algorithm 1: The Two-Stage Training Approach of PeerHTC
Input: Training documents D and validation/test documents D∗; label hierarchy γ; Output: A hierarchical text classifier Ω. for Warm-up training: do
Step 1. Randomly initialize the adjacent matrix A in GCN, and assign equal weights to training samples; Step 2. Train the PeerHTC model and then save the estimated label probabilities on training and test samples, together with the label embeddings H∗; Step 3. Compute the adjacent matrices Arank or Aemb; Step 4. Compute the label confusion measure cjk and the sample weight βdk.
for Second-round training: do Re-initialize the adjacent matrix in GCN with Arank or Aemb, and plug sample weights βdk into the loss function. Then re-train the PeerHTC model to get the final classifier.

heading: 5 Experiments

heading: 5.1 Experimental Setup
text: Datasets. We use three datasets to explore the classification performance. The first one is Webof-Science (WOS) dataset (Kowsari et al., 2017), which consists of abstracts of published papers from journals in Web of Science. The disciplines
that each paper belongs to are regarded as the classification labels. The second dataset is BlurbGenreCollection (BGC) (Aly et al., 2019), which consists of advertising descriptions of books. The genres of books are regarded as classification labels, while the advertising descriptions are regarded as text documents. The last dataset consists of the textual names of retailing products (we refer to as Goods), which are collected by ourselves from a Chinese retailing company. In this dataset, each product belongs to a three-level product hierarchy. Among the three datasets, WOS and Goods are both for singlepath HTC, i.e., each sample only has one single label in each level, whereas samples in BGC have multi-path labels, i.e., each sample is allowed to have multiple labels in the same level. Each dataset is randomly split into the training set (70%), validation set (15%) and test set (15%). The descriptive statistics of the three datasets are listed in Table 1. In addition, the intended use of public datasets and pre-trained models, as specified in their license or terms, was strictly obeyed in our work.
Baselines. In order to demonstrate the effectiveness of PeerHTC, we compare it with three naive approaches that treat HTC as a simple multi-label classification problem, along with four state-of-theart HTC models. The three naive approaches are LSTM (Hochreiter and Schmidhuber, 1997), TextRCNN (Lai et al., 2015), and BERT (Devlin et al., 2018). The four state-of-the-art HTC models are briefly introduced as follows.
(1) HMCN (Wehrmann et al., 2018). It is probably the first hybrid approach that combines a sequence of local classifiers with global optimization.
(2) HARNN (Huang et al., 2019). It is also a hybrid approach, but utilizes attention mechanism and refines how information flows between levels.
(3) HiAGM-LA (Zhou et al., 2020). It encodes labels and documents separately, and utilizes multilabel attention mechanism to extract hierarchy-
aware text features.
(4) HTCInfomax (Deng et al., 2021). It is basically an improvement on top of HiAGM-LA via mutual information maximization.
Evaluation metrics. To measure the classification performance, we apply two standard evaluation metrics, i.e., the Micro-F1 and Macro-F1 (Gopal and Yang, 2013). Micro-F1 computes the overall precision and recall of all the labels, while Macro-F1 computes the average of F1 scores of all labels. As a result, Micro-F1 assigns greater weights to more frequent labels, while Macro-F1 treats all the labels equally.
Implementation details. We use BERT as the text encoder in PeerHTC, and set the dimension of label embeddings as 256. The BERT models are pretrained on "book_corpus_wiki_en_uncased" and "wiki_cn_cased" for English and Chinese datasets respectively, both of which have 12 hidden layers and 768 hidden units. The vocabulary is created with words that appear no less than 5 times. We set the maximum length of token inputs as 100. The threshold for tagging is chosen to be 0.5. Model parameters are initialized according to the Xavier uniform (Glorot and Bengio, 2010) when random initialization is needed. We use the Adam optimizer (Kingma and Ba, 2014) with momentum parameters β1 = 0.9 and β2 = 0.999, a learning rate α = 1× 10−5, and a mini-batch size of 64. To prevent overfitting, we also use dropout (Srivastava et al., 2014) with the rate of 0.1, and weight decay (Loshchilov and Hutter, 2017) with the tuning weight equal to 1×10−7. The parameter τ in equation (6) is set as 1.2.
For HTC competitors, the same parameter settings are adopted. We follow their original practices to use simple text encoders, but also replace them with BERT for a fair comparison. Specifically, in HARNN, HiAGM-LA and HTCInfoMax, we use a single-layer bidirectional LSTM as the text encoder; in HMCN, we use three parallel CNN layers with filter sizes {3, 4, 5} and numbers of channel {100, 70, 70} as the text encoder. For simple text encoders, 300-dimensional pretrained word embeddings GloVe (Pennington et al., 2014) and Fasttext (Bojanowski et al., 2017) are used on English and Chinese datasets respectively. Our hyperparameters are tuned on the validation set, taking both classification performance and the computation resources available into consideration, and the classification performances reported in our exper-
imental results are evaluated on the test set. Our models are trained on two Tesla P100 GPUs.

heading: 5.2 Experimental Results

heading: 5.2.1 Evidence of peer labels
text: In order to demonstrate the existence of peer labels, we take the BGC and WOS datasets as examples and show the adjacent matrices of their first-level labels, which are computed using estimated label probabilities on training samples according to equation (3). As shown by Figure 2, some labels have higher degrees of relevancy with others, which can serve as useful prior knowledge for classification. For example, in the BGC dataset, "Classics" and "Poetry", corresponding to the intersection of the third row and fourth column in Figure 2(a), are closely related. In the WOS dataset, "Mechanical Aerospace Engineering (MAE)" and "Civil Engineering" (in the fourth row and fifth column) show extremely high relevancy. These findings verify the existence of peer label relevancy. However, compared with BGC and WOS datasets, the latent relevancy of peer labels is relatively weak for the Goods dataset, which is not shown to save space. This is largely due to the fact that the Goods dataset has a larger number of categories, which are more fine-grained and thus less relevant with each another.

heading: 5.2.2 Comparison results
text: To explore the classification performance of PeerHTC, we compare this model with (1) naive classification approaches (i.e., LSTM, TextRCNN, BERT), and (2) state-of-the-art HTC approaches (i.e., HMCN, HARNN, HiAGM-LA, HTCInfoMax). For the HTC approaches, we also replace their original text encoders with BERT for better classification performance and a fair comparison with PeerHTC. To characterize the latent relevancy
of peer labels in PeerHTC, the GCN methods with the "level-wise" and "whole-hierarchy" strategies are considered; see Equations (1) and (2) for details. To decide the adjacent matrix A in GCN, we consider three methods: (1) Arank-train, computed by the estimated label probabilities on training samples; (2) Arank-test, computed by the estimated label probabilities on test samples; and (3) Aemb, computed by label embeddings. This results in a total of 2 × 3 = 6 choices for A. We report the performances of all these different choices for a comparison.
Table 2 reports the classification results of different models, from which we can draw the following conclusions. First, our method PeerHTC has achieved better classification performance than all naive approaches and HTC approaches, when evaluated by both Micro-F1 and Macro-F1 in the three datasets. Second, when replacing the traditional text encoders in HTC approaches by BERT, their classification performances have been largely improved. This reveals the great potential of pretrained models in HTC problems. Even so, our method PeerHTC still outperforms HTC approaches with BERT on the BGC and WOS datasets. On the Goods dataset, however, the method HTCInfoMax achieves the best performance while PeerHTC ranks second with comparable results. We believe that this slightly poorer performance of PeerHTC mainly results from the weak latent relevancy among peer labels, as remarked in Section 5.2.1.

heading: 5.2.3 Ablation study
text: To further demonstrate the advantages of incorporating peer labels and using sample weights, we
conduct the following ablation study. Specifically, we compare the following three models. The first one, denoted by "NA", is a naive HTC model without considering peer labels or sample weights. The second one, denoted by "OPL", is a variant of PeerHTC that only considers the latent relevancy among peer labels, but does not utilize the sample weights. The last one is our proposed PeerHTC, which considers both peer labels and sample weights. In addition, to explore the performances of using different adjacent matrices A, we report OPL with six different adjacent matrices, as described in Section 5.2.2. The detailed results are shown in Table 3. It is obvious that, nearly all OPL methods, as well as the PeerHTC method, have obtained better classification performance than the NA method. These results suggest that leveraging peer effect is beneficial to hierarchical text classification.
We then compare the performances of using different adjacent matrices in OPL. As shown by Table 3, on the dataset BGC, the "whole-hierarchy" strategy works better in most cases. On the dataset WOS, the "level-wise" strategy generally works better. On the Goods dataset, the performances of "level wise" and "whole hierarchy" are rather comparable. When it comes to adjacent matrices computed using either sample probabilities or label embeddings, there is no obvious distinction between their performances, indicating all these methods can be helpful in revealing latent relationships among peer labels.
Finally, we focus on the effect of using sample weights. As we mentioned in Section 4.2, characterizing the latent relevancy of peer labels would create shortcuts between labels and may have potential side effect of label confusion. To cope with this problem, we first measure the degree of label confusion, then identify the importance of different training samples in alleviating label confusion, and finally plug these weights into the BCE loss func-
tion. As shown by Table 3, the PeerHTC method can improve the classification performance consistently on the three datasets, when compared with the OPL method. These results demonstrate the usefulness of sample weights in PeerHTC.

heading: 6 Conclusion
text: In this work, we originally propose the concept of "peer labels" to characterize the phenomenon that labels in the same level have latent relevancy. To utilize these peer labels to enhance HTC, we develop the PeerHTC method. We exploit GCN as an encoder for latent relevancy among peer labels, and reinforce feature sharing among these closely related peer labels. We also use a novel technique to assign training sample weights based on their importance in alleviating label confusion. The above procedures are embedded in a two-stage training approach. Our experimental results demonstrate the evidence of peer labels in real datasets and the generally better performance of PeerHTC against other state-of-the-art HTC methods. In terms of application, one would expect a higher lift in performance from PeerHTC when the granularity of categorization is relatively low, as demonstrated by our experimental results. We also suggest that one should carry out exploratory analysis or take into account domain knowledge, in order to decide the extent of peer-label relevancy for a specific dataset.
Limitations
In this work, we adopt a data-driven approach to identifying latent relevancy. However, we believe that external knowledge such as knowledge graphs could also be of great help for this purpose, and is thus one of the directions of our future work.

heading: Acknowledgements
text: Feifei Wang is the corresponding author. This work is supported by National Natural Science Foundation of China (No.72001205), the Fundamental Research Funds for the Central Universities and the Research Funds of Renmin University of China (21XNA026). Besides, we thank all the anonymous reviewers for their valuable suggestions.

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Limitations are discussed in Section 7.
7 A2. Did you discuss any potential risks of your work? To the best of our knowledge, our work does not involve any potential risk.
3 A3. Do the abstract and introduction summarize the paper’s main claims? The abstract is at the very beginning of the paper, while the introduction is in section 1.
7 A4. Have you used AI writing assistants when working on this paper? We did not use any AI writing assistant.
B 3 Did you use or create scientific artifacts? Some scientific artifacts(eg. datasets and pre-trained models) were used in our experiments in Section 5.
3 B1. Did you cite the creators of artifacts you used? They are cited in Section 5.
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Section 5.1.
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 5.1.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Data used in our work do not involve such problems.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Introduction to the datasets is included in Section 5.1.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Yes. In Section 5.
C 3 Did you run computational experiments? Section 5.
7 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Our computing infrastructure is introduced in Section 5. The number of parameters and GPU hours involved in our experiments are on a reasonable scale and we believe they will not cause any difficulty to reproduction. Thereby, they are omitted to save space.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 5.
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Our experimental results are derived on the test set in a single run, which is pointed out in Section 5.
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Such implementation details are included in Section 5.1.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: However, we believe that external knowledge such as knowledge graphs could also be of great help for this purpose, and is thus one of the directions of our future work