Towards Better Hierarchical Text Classification with Data Generation
Year: 2023

Abstract:
Hierarchical text classification (HTC) focuses on classifying one text into multiple labels, which are organized as a hierarchical taxonomy. Due to its wide involution in realistic scenarios, HTC attracts long-term attention from both industry and academia. However, the high cost of hierarchical multi-label annotation makes HTC suffer from the data scarcity problem. In view of the difficulty in balancing the controllability of multiple structural labels and text diversity, automatically generating high-quality data for HTC is challenging and under-explored. To fill this blank, we propose a novel data generation framework tailored for HTC, which can achieve both label controllability and text diversity by extracting high-quality semanticlevel and phrase-level hierarchical label information. Experimental results on three benchmarks demonstrate that, compared with existing data augmentation methods, the data generated from our method can bring the most significant performance improvements of several strong HTC models. Extensive analysis confirms that the improvements yielded by our proposed method do correlate to the enhancement of label controllability and text diversity. 1

heading: 1 Introduction
text: Hierarchical Text Classification (HTC) is a representative multi-label text classification problem, aiming to assign one text with multiple labels in a given label hierarchical taxonomy. HTC is widely involved in realistic scenarios, e.g., news classification (Lewis et al., 2004; Sandhaus, 2008), science paper classification (Kowsari et al., 2017), Ecommerce (Gao, 2020). To solve such an important and challenging task, adequate high-quality labeled data is indispensable. However, multi-labeled data
∗Equal contribution. Work is done during the internship of Yue Wang at Ant Group.
†Corresponding author 1Our codes are publicly available at https://github.
com/wangyuenlp/Data-Generation-for-HTC.
annotation is usually expensive, and the hierarchical structure of multi-label further makes such annotation unaffordable. Thus, the primary goal of our approach is to deal with the data scarcity problem. Existing works (Kowsari et al., 2017; Shimura et al., 2018; Banerjee et al., 2019; Zhou et al., 2020; Wang et al., 2022c,d; Jiang et al., 2022) focus on enhancing the model ability to relieve the need for annotated data, but few of them address this problem by automatically generating high-quality data.
Recently, since generative Pre-trained Language Models (PLMs) have achieved surprising performance (Brown et al., 2020; Lewis et al., 2020; Raffel et al., 2020), generative data augmentation has drawn more and more attention (Anaby-Tavor et al., 2020; Schick and Schütze, 2021; Wang et al., 2022a; Meng et al., 2022). With the help of the rich knowledge obtained from the large-scale data in the pre-training stage, generative data augmentation can further improve the quality of the generated data, which can better alleviate the data scarcity problem. However, generating high-quality data for HTC is still an under-explored problem. We argue that two main challenges exist for this problem: the need for label controllability and text diversity. If the meaning of the generated data is out of the control of the given labels, it is usually noisy data assigned with wrong labels, which does little help and even harms HTC models. Besides, the generated data may not improve the generalization ability of HTC models if the expression is similar to the original text. Intuitively, restricted label constraints can ensure label controllability but might impede the text diversity of the generated data, especially in HTC, which has multiple label constraints in a hierarchical structure (Kumar et al., 2021). Therefore, achieving a good balance between controllability and diversity is the key to generating high-quality data for HTC.
To improve label controllability and text diversity of generated data, we propose a novel data
7722
generation framework for HTC, which aims to enrich the input information of the data generation model. Specifically, we first design the semanticlevel and phrase-level label information enhanced prompt, consisting of label names and phrases extracted from origin training samples. Although label names can improve label controllability, to a certain extent, only leveraging label names makes the model inevitably generate similar data since the same input label name needs to create hundreds and even thousands of data samples. Thus, the phrases extracted from original train samples can be used as a supplement to label names. To ensure label controllability and text diversity, we need the phrases to represent the critical information that can infer the origin label correctly and simultaneously avoid common expressions. Therefore, to deal with these problems, we propose a hierarchical label information enhanced keyword extractor to extract the most relevant phrase to the label meaning. Finally, to further improve the label controllability, we introduce consistency filtering to filter out the low-quality generated data, which makes the HTC model predict different labels compared to the controlled labels. Experimental results on three benchmarks confirm the effectiveness of our proposed method. Furthermore, both quantitative and qualitative analyses show the superiority of our method over existing data augmentation methods in comparing label controllability and text diversity.
In a nutshell, our contributions are as follows:
• To the best of our knowledge, we are the first to explore the effectiveness of generative data augmentation methods on the HTC problem.
• We propose a data generation framework for HTC, which leverages both semantic-level and phrase-level hierarchical label information to enhance the label controllability and text diversity of the generated data.
• We confirm the effectiveness of our methods over several HTC models on three benchmarks, which can bring 1.39, 1.37, and 1.25 Macro-F1 improvements of BERT-base, RoBERTa-base, and state-of-the-art HTC model HGCLG on WOS, respectively.

heading: 2 Related Work
text: Hierarchical Text Classification Compared to multi-label text classification, due to the unique
label hierarchical taxonomy, the previous work on HTC focus on fully using the hierarchical information of the label taxonomy. Kowsari et al. (2017); Shimura et al. (2018); Wehrmann et al. (2018); Banerjee et al. (2019) train different classifiers for different nodes or levels and transfer the knowledge of the parent nodes’ classifiers to the child nodes, which are called local approaches (Zhou et al., 2020). Different from local approaches, global approaches treat HTC as a flat multi-label text classification problem. Due to the lack of hierarchical information in the classifier, the global approaches focus on utilizing hierarchical information to further improve performance. Mao et al. (2019) use hierarchical information to enhance the Label Assignment Policy and propose a deep reinforcement learning-based general framework for HTC; Wu et al. (2019) utilize a meta-learner to deal with the complexity and dependencies of different labels; Aly et al. (2019) introduce capsule networks for HTC and utilize label co-occurrence to initialize weight better; Deng et al. (2021) propose text-label mutual information maximization and label prior matching to filter out irrelevant information and learn better hierarchy-aware representations; Rojas et al. (2020) define HTC as a sequence-to-sequence problem and utilize an auxiliary synthetic task and external knowledge to improve performance further. Recently, there are more and more focus on leveraging a structure encoder to model the hierarchical information (Zhou et al., 2020; Chen et al., 2021; Wang et al., 2021a, 2022c,d; Jiang et al., 2022). Different from previous work, we focus on introducing generative data augmentation to address the data scarcity problem of HTC and use a structure encoder to model hierarchical label information to improve the quality of the generated data.
Generative Data Augmentation With the development of text generation models, generative data augmentation gains more and more attention from the community. Existing works apply generative data augmentation methods to various NLP downstream tasks, including multi-class text classification (Malandrakis et al., 2019; Liu et al., 2020; Anaby-Tavor et al., 2020), multi-label text classification (Zhang et al., 2020), text entailment (Vu et al., 2021; Wang et al., 2022b), relation extraction (Lee et al., 2021), sequence labeling (Wang et al., 2022a), intent classification and slot tagging (Lee et al., 2021; Rosenbaum et al., 2022), etc. Besides, there is also a line of work fo-
cused on generating new data without fine-tuning PLMs to solve various tasks on the zero-shot setting, also called dataset generation (Schick and Schütze, 2021; Wang et al., 2021b; Meng et al., 2022; Ye et al., 2022). However, the effectiveness of generative data augmentation methods on the HTC problem is under-explored. Apart from the difference in applied tasks, existing works also differ in model architecture, objective functions, and learning paradigms. Specifically, Malandrakis et al. (2019) propose a conditional variational autoencoders-based controlled text generation model for data augmentation; Liu et al. (2020) utilize reinforcement learning to guide the conditional generation; Vu et al. (2021) leverage data generation model to generate unlabeled synthetic data and conduct self-training; Lee et al. (2021) propose an example extrapolator to generate new labeled synthetic data from existing examples. Wang et al. (2022a) propose a soft prompt-based data generation model for low-resource scenarios. Wang et al. (2022b) unify diverse NLP tasks into a text-to-text format to pre-train a multi-task data generation PLM. In this work, we focus on using generative data augmentation to bring benefits to HTC models in the full-supervised setting and improve the qual-
ity of generated data by incorporating hierarchical label information.

heading: 3 Method
text: In this section, we first give the task formulation for HTC and the goal of data generation. Then, we introduce our framework consisting of the semanticlevel and phrase-level label information enhanced prompt, hierarchical label information enhanced keyword extractor and consistency filtering.

heading: 3.1 Task Formulation
text: To the HTC task, given the sample (𝑥, 𝑦) from the training set 𝐷𝑇𝑟𝑎𝑖𝑛, the HTC model 𝑀𝐻𝑇𝐶 is trained to predict 𝑦 according to the text 𝑥. Here 𝑦 is a subset of label set 𝑌 , which is organized as a directed acyclic graph 𝐺 = (𝑌, 𝐸), where the edge set 𝐸 denotes the hierarchical information of 𝑌 . The data generation model 𝑀𝐺𝐸𝑁 aims to make the final HTC model 𝑀𝐴𝑈𝐺 perform better than 𝑀𝐻𝑇𝐶 . To achieve this goal, 𝑀𝐺𝐸𝑁 can utilize 𝐷𝑇𝑟𝑎𝑖𝑛 to generate 𝐷𝐺𝐸𝑁 , whose number is usually larger than 𝐷𝑇𝑟𝑎𝑖𝑛. The training data of 𝑀𝐴𝑈𝐺 consists of both 𝐷𝑇𝑟𝑎𝑖𝑛 and 𝐷𝐺𝐸𝑁 .

heading: 3.2 Overview of Our Framework
text: Our framework consists of three models: the data generation model (Sec. 3.3), the keyword extractor (Sec. 3.4), and the filter model (Sec. 3.5). We show an illustration of our framework in Figure 1. In general, we first use 𝐷𝑇𝑟𝑎𝑖𝑛 to train the keyword extractor 𝑀𝐾𝐸 and the filter model 𝑀𝐹𝑖𝑙𝑡𝑒𝑟 . Then we use 𝑀𝐾𝐸 to extract keywords 𝑥 for each instance in 𝐷𝑇𝑟𝑎𝑖𝑛. Afterward, to train the data generation model 𝑀𝐺𝐸𝑁 , we organize the label names and extracted keywords 𝑥 of each instance in 𝐷𝑇𝑟𝑎𝑖𝑛 as the input sequence and the original text as the target sequence. In the inference stage of 𝑀𝐺𝐸𝑁 , we utilize the sampling-based decode strategy to generate different texts according to the label names and extracted keywords 𝑥 of instances in 𝐷𝑇𝑟𝑎𝑖𝑛. Finally, we use the filter model 𝑀𝐹𝑖𝑙𝑡𝑒𝑟 to remove low-quality generated data. In the following, we will describe our framework in detail.

heading: 3.3 Semantic-Level and Phrase-Level Label Information Enhanced Prompt
text: To improve label controllability and text diversity, we propose the semantic-level and phrase-level label information enhanced prompt consisting of label names and keywords extracted from the samples of 𝐷𝑇𝑟𝑎𝑖𝑛. Both label names and keywords can improve label controllability by providing important label information to 𝑀𝐺𝐸𝑁 from the semantic and phrase levels. Besides, due to the diversity of phrase expressions under the same semantics, providing keywords extracted from different samples with the same label can also help to generate diverse data. Specifically, given the sample (𝑥, 𝑦), we organize the input of the 𝑀𝐺𝐸𝑁 as ‘generate with label: 𝑦; generate with keywords: 𝑥’ and the target output as 𝑥, where 𝑥 refers the keywords extracted from the 𝑥. In the inference stage, we follow the former prompt format to generate candidate data 𝐷𝐶𝐴𝑁𝐷 , whose keywords are also extracted from samples of 𝐷𝑇𝑟𝑎𝑖𝑛. We shuffle the order of keywords at both the training and inference stage to avoid the over-fitting problem.

heading: 3.4 Hierarchical Label Information Enhanced Keyword Extractor
text: The quality of keywords is important to balance label controllability and text diversity. Biased phrases irrelevant to the label meaning may mislead the data generation model and harm the label controllability. Excessively unimportant phrases unre-
lated to the labels of samples may control 𝑀𝐺𝐸𝑁 to generate stern expression, which may do harm to diversity. To improve the quality of extracted keywords, we propose the hierarchical label information enhanced keyword extractor. For the sample (𝑥, 𝑦), we want to extract sub-sequence 𝑥, where 𝑥 consists of the most relevant keywords for the corresponding multi-label 𝑦. Specifically, we first use Graphormer (Ying et al., 2021) to model the hierarchical information 𝐺 = (𝑌, 𝐸) and get the label embedding 𝑙 𝑗 for each node 𝑗 ∈ 𝑌 . Formally:
{𝑙1, 𝑙2, . . . , 𝑙𝑛} = 𝐺𝑟𝑎𝑝ℎ𝑜𝑟𝑚𝑒𝑟 (𝑌, 𝐸).
Next, for the given input 𝑥, we get the text embedding of each token 𝑥𝑖 by:
{𝑡1, 𝑡2, . . . , 𝑡𝑛} = 𝐵𝐸𝑅𝑇𝑒𝑚𝑏 (𝑥),
where 𝐵𝐸𝑅𝑇𝑒𝑚𝑏 (𝑥) denotes using BERT (Devlin et al., 2019) to encode the given sentence 𝑥.
We then utilize the attention mechanism to catch the relevance between token 𝑥𝑖 and label 𝑦 𝑗 as:
𝑄𝑖 = 𝑡𝑖𝑊𝑄, 𝐾 𝑗 = 𝑙 𝑗𝑊𝐾 , 𝐴𝑖 𝑗 = 𝑄𝑖𝐾 𝑇 𝑗√
𝑑ℎ ,
where 𝑊𝑄 ∈ R𝑑ℎ×𝑑ℎ and 𝑊𝑄 ∈ R𝑑ℎ×𝑑ℎ are two weight matrices. Afterward, we use GumbelSoftmax (Jang et al., 2016) to calculate the probability that 𝑥𝑖 is the keyword of class 𝑦𝑖 by:
𝑃𝑖 𝑗 = gumbel_softmax (𝐴𝑖1, 𝐴𝑖2, . . . , 𝐴𝑖𝑘) 𝑗 ,
which satisfies ∑ 𝑗∈𝑌 𝑃𝑖 𝑗 = 1.
Therefore, the relevance score between token 𝑥𝑖 and a multi-label 𝑦 can be calculated as:
𝑃𝑖 = ∑︁ 𝑗∈𝑦 𝑃𝑖 𝑗 .
Intuitively, if 𝑥 contains the most relevant information, a classifier can infer the origin label correctly from 𝑥 and predict null labels from the rest of the sequence. This intuition guides us to design the final loss function L. Specifically, with the calculated 𝑃𝑖 for each token 𝑥𝑖, we first select the sequence of the most relevant keywords 𝑥𝑝 and the sequence of the least relevant keywords 𝑥𝑛. 𝑥𝑝 contains the top 15% tokens with the highest 𝑃𝑖 while 𝑥𝑛 contains 15% token with the lowest 𝑃𝑖. For better training, we generate 𝑥𝑝 and 𝑥𝑛 from 𝑥 by masking the rest of the tokens in practice. Finally, we encourage the classifier 𝑓 to assign 𝑥𝑛
with the original multi-label 𝑦 and 𝑥𝑛 with all-zero label 𝑦𝑛𝑜𝑛 by the final loss function:
L = ℓ( 𝑓 (𝑥𝑝), 𝑦) + ℓ( 𝑓 (𝑥𝑛), 𝑦𝑛𝑜𝑛),
where ℓ denotes the binary cross entropy loss for multi-label classification. We jointly update the parameter of the graph network 𝐺𝑟𝑎𝑝ℎ𝑜𝑟𝑚𝑒𝑟 and the text encoder 𝐵𝐸𝑅𝑇𝑒𝑚𝑏. After training, for each sentence 𝑥, we directly use 𝑥𝑝 as the extracted keywords 𝑥 for each sample at the inference stage.

heading: 3.5 Consistency Filtering
text: Although the above methods improve the label controllability as far as possible, they may still generate low-quality samples whose meaning is contradicted the assigned label. This phenomenon is inevitable due to the pursuit of text diversity. To filter out these samples, we introduce consistency filtering (Anaby-Tavor et al., 2020). Specifically, we first use the training set 𝐷𝑇𝑟𝑎𝑖𝑛 to train an HTC model 𝑀𝐹𝑖𝑙𝑡𝑒𝑟 . Then, we use 𝑀𝐹𝑖𝑙𝑡𝑒𝑟 to predict the labels of 𝐷𝐶𝐴𝑁𝐷 and compare the predicted labels with their assigned labels. Only the generated samples whose predicted labels by 𝑀𝐹𝑖𝑙𝑡𝑒𝑟 are consistent with assigned labels will be kept, constituting 𝐷𝐺𝐸𝑁 . We combine 𝐷𝐺𝐸𝑁 with 𝐷𝑇𝑟𝑎𝑖𝑛 to train the target HTC model 𝑀𝐴𝑢𝑔 from scratch.

heading: 4 Experiments

heading: 4.1 Datasets and Metrics
text: We conduct experiments on three HTC benchmarks: Web-of-Science (WOS) (Kowsari et al., 2017), The New York Times Annotated Corpus (NYT) (Sandhaus, 2008), and Reuters Corpus Volume I Version 2 (RCV1-V2) (Lewis et al., 2004). WOS is a science paper classification dataset, while NYT and RCV1-V2 are news classification datasets. We show the statistic of three benchmarks in Table 1. We follow Zhou et al. (2020); Chen et al. (2021); Wang et al. (2022c) to pre-process the data and report the Macro-F1 and Micro-F1 as the metrics of HTC models.

heading: 4.2 Baselines
text: To confirm the effectiveness of our methods, we compare their improvements with different data augmentation methods on multiple representative HTC models. Our baselines can be grouped into two types: HTC and data augmentation methods.
HTC. We choose three baselines: (1) BERT (Devlin et al., 2019), (2) RoBERTa (Liu et al., 2019), and (3) HGCLR (Wang et al., 2022c). BERT and RoBERTa are popular PLMs, which are widely used as text encoders. HGCLR is a state-of-the-art HTC model which utilizes contrastive learning to incorporate hierarchical label information.
Data Augmentation. We select five strong data augmentation methods for comparisons: (1) EDA (Wei and Zou, 2019) is a rule-based data augmentation method that uses four simple editing operations to disturb texts; (2) Back Translation (BT) (Sennrich et al., 2016) first translate the texts into other languages and then translate them to the original language; (3) LAMBADA (AnabyTavor et al., 2020) is a generative data generation method that generates new data according to label names; (4) GDA (Zhang et al., 2020) utilize generative PLMs to generate label-invariant perturbations on the texts; (5) PromDA (Wang et al., 2022a) is a state-of-the-art generative data augmentation method that uses an efficient fine-tuning technique to train the data generation model.

heading: 4.3 Implementation Details
text: We implement all the data generation and HTC models with the open-sourced toolkit Transformers2 (Wolf et al., 2020). For data generation models, we take the T5 (Raffel et al., 2020) as our backbone PLM and conduct fine-tuning from the publicly available checkpoint t5-large 3. In the fine-tuning stage, we use Adafactor (Shazeer and Stern, 2018) optimizer to update all parameters of T5 and set the learning rate as 1e-3 and batch size as 32. We fine-tune data generation models for 10 k steps, evaluate the perplexity on the development set every 1 k steps, and save the model with the lowest perplexity. In the inference stage,
2https://github.com/huggingface/transformers 3https://huggingface.co/t5-large
we use Nucleus Sampling (Holtzman et al., 2019) to generate diverse data. Specifically, we set Topp as 0.9 and get five independently sampled outputs for one input text. For all data augmentation methods, we augment data 5 times the number of samples in the original training set. For EDA, we use the official implentation 4. For BT, we use the open-sourced English to France machine translation model Helsinki-NLP/opus-mt-en-fr 5 and France to English model Helsinki-NLP/opus-mt-fren 6. For HTC models, we use bert-base-uncased7 and roberta-base8 as the initial checkpoints for BERT and RoBERTa respectively and completely follow Wang et al. (2022c) to set hyper-parameters. We report the average performance of HTC models using five different random seeds.

heading: 4.4 Main Results
text: We report the performance of all baselines and our proposed method in Table 2. From the results, we can find that EDA and BT fail to improve the performance of the HTC models significantly, which shows simple perturbations on the original texts cannot help to improve the generalization ability of HTC models in the full-supervised setting. Ex-
4https://github.com/jasonwei20/eda_nlp 5https://huggingface.co/Helsinki-NLP/
opus-mt-en-fr 6https://huggingface.co/Helsinki-NLP/ opus-mt-fr-en 7https://huggingface.co/bert-base-uncased 8https://huggingface.co/roberta-base
isting generative data augmentation methods can help some HTC models achieve better performance, but the improvements are marginal and fail to improve performance on NYT and RCV1-V2. This phenomenon shows that generating diverse data satisfied with multiple label constraints for HTC is still challenging. With the help of the data generated from our proposed method, HTC models can achieve better performance in almost all settings. The improvements are more significant than other data augmentation methods. Besides, we also find the improvements of Macro-F1 brought by generative data augmentation methods are better than the Micro-F1, which we speculate results from the long-tail labels suffering from a more severe data scarcity problem. All these results confirm the effectiveness of our proposed methods to deal with the data scarcity problem for HTC.

heading: 4.5 Model Ablation
text: We conduct a model ablation study and report the results in Table 3. We find that only providing label names or keywords to the data generation models harms the performance of HTC models. This result confirms our hypothesis that we need to deliver both semantic-level and phrase-level label information to enhance the ability of the data generation models. We also replace the proposed keyword extractor with yake (Campos et al., 2018a,b, 2020), a state-of-the-art unsupervised keyword extractor. The results show that without the help of the hier-
archical label information, the extracted keywords may not be the words most relevant to the assigned labels, which harms the quality of the generated data. Besides, we also replace t5-large (770M parameters) with t5-base (220M parameters) 9 to study the effect of the backbone generative PLMs. With the decrease in parameters, the performance of our proposed method drops slightly. Despite the drop, it still performs better than other data generation methods, which use t5-large as the backbone, compared to the results reported in Table 2. Furthermore, without the help of the unified data generation prompt format, Macro-F1 drops more significantly than Micro-F1, which shows our method can transfer the knowledge from head labels to longtail labels and thus can mitigate the sparse label distributions.

heading: 4.6 The Number of Generated Data
text: We also study the effect of the number of generated data. The results are shown in Figure 2. We can observe that when the number of generated data is smaller than 5 times the number of original train data, with the increase of the number of generated data, the performance of the HTC model improves. When bigger than 5 times the original training data, no steady growth trend is observed. Therefore, we generate 5 times the original training data in our experiments. Besides, we find that the change of
9https://huggingface.co/t5-base
Macro-F1 is more significant than Micro-F1. We speculate it also results from the generated data doing more help to the long-tailed data.

heading: 4.7 The Diversity of the Generated Data
text: We use Self-BLEU (Zhu et al., 2018) as the metric to conduct a quantitative analysis of the diversity of the generated data. Because we want the generated data to diversify as much as possible under the condition of meeting the need for label controllability, we report the Self-BLEU score of filtered data after using consistency filtering. From the results in Table 4, we can observe that the Self-BLEU score of EDA and BT is significantly higher than generative data augmentation methods, and our proposed method achieves the lowest Self-BLEU score. These results further confirm the effectiveness of our proposed method to improve text diversity while ensuring label controllability.

heading: 4.8 The Effect of Consistency Filtering
text: We also study the effect of consistent filtering and report the results in Table 5. EDA and BT achieve no improvements after consistency filtering, which
shows that simple perturbations may not make texts deviate from the original semantics. Therefore, few noise data with wrong labels exist in the data augmented from EDA and BT. Although generative data augmentation methods achieve significant improvements than EDA and BT after consistency filtering, they even perform worse before consistency filtering. The phenomenon may result from the noise data with wrong labels, which is inevitably due to the pursuit of diversity. These results show that consistency filtering is indispensable for generative data augmentation methods to balance label controllability and text diversity. Besides, compared to existing data generation methods, our proposed method can achieve the best performance with or without the help of consistency filtering, which further confirms its effectiveness.

heading: 4.9 Computation Cost Analysis
text: We conduct a computation cost analysis of different data augmentation methods, which is reported in Table 6. Despite the small extra computational cost, EDA and BT fail to improve the performance of the HTC models significantly. Furthermore, with similar computation costs, other generative data augmentation methods cannot bring as many improvements as our methods. The results show that increasing computational cost is not a sufficient condition for improving performance and further embodies the technical contribution of our method.

heading: 4.10 Case Analysis
text: In Figure 7, we conduct a case analysis on the extracted keywords and generated data. To the data augmented from EDA and BT, there is a lot of overlap with the original text, while generative data augmentation methods rarely. To the extracted keywords, we can observe that yake may draw some words that are irrelevant to the label information, e.g., ‘present study investigated, investigated the effects, days, present’. These excessively useless keywords can not improve label controllability and hinder the diversity of the generated data, which causes the generated data to have very similar semantics to the original sample. With the help of the hierarchical label information and the loss function we designed, the keywords extracted by our methods can pick up the most important information that causes the original text to be annotated as the original label. Therefore, the data generated by our proposed methods can satisfy the label constraints and have differences with the original texts from both semantic and grammatical levels. We show more generated data in appendix C.

heading: 5 Conclusion
text: HTC is an important and challenging task, which usually suffers from the data scarcity problem because of the high cost of hierarchical multi-label annotation. With the development of generative PLMs, generating high-quality data for HTC becomes possible. In this paper, to deal with the data scarcity problem, we explore the effect of generative augmentation methods on HTC models. In order to improve the label controllability and text diversity, we propose a novel data generation framework for HTC, which consists of semanticlevel and phrase-level label information enhanced prompt, hierarchical label information enhanced keyword extractor and consistency filtering. Despite the challenge of generating high-quality data for HTC, our proposed framework can achieve a balance between the controllability of multiple hierarchical labels and text diversity and improve the performance of several strong HTC models, which is demonstrated by experimental results on three HTC benchmarks and comprehensive analysis.

heading: 6 Limitations
text: Despite the effectiveness of our proposed method, it still has two main limitations: (1). Generative data augmentation methods need to use the original
HTC training set to fine-tune the backbone generative PLMs. Then, they need to go through an inference stage to generate data. Both the training and inference stage need more GPU resources, which increase carbon emissions. Although the data generation is usually complete offline and does not improve the time cost of online progress, we leave how to relieve the need for GPU resources as future directions. (2). Although we conduct experiments on three widely used HTC benchmarks, the language of all these benchmarks is English, which has limited morphology. The effectiveness of our proposed method on language with varied morphology needs to be further confirmed.

heading: 7 Ethics Statement
text: The cases shown in this paper are generated automatically from different data augmentation methods, which do not represent the viewpoints of the authors. Due to social bias and lack of professional knowledge, the data generated from generative PLMs may contain misleading and toxic information, which needs to be addressed before being applied to realistic scenarios. Besides, all data generated from our proposed methods are only for scientific research. Finally, we provide comprehensive details of our model implementation and upload the source code, which guarantees the reproducibility of our experimental results.

heading: Acknowledgements
text: This work is supported by the National Science Foundation of China (NSFC No. 62206194), the Natural Science Foundation of Jiangsu Province, China (Grant No. BK20220488), and JSSCBS20210661. This work is also supported by Beijing Academy of Artificial Intelligence (BAAI) and Ant Group through Ant Innovative Research Program.

heading: A Human Evaluation for Filter Model
text: To further confirm the performance of the 𝑀𝐹𝑖𝑙𝑡𝑒𝑟 model, we conduct a human evaluation of 200 sampled generated data and compare the consistency between human judgments and model predictions. According to the human annotation, 200 sampled generated data consist of 191 correct samples and 9 noisy samples. Our filter model keeps 187 out of 191 correct samples and only 2 noisy samples, which confirms the effectiveness of the 𝑀𝐹𝑖𝑙𝑡𝑒𝑟 model.

heading: B Noisy Label
text: To examine the performance when there are noisy labels in the original data, we add 10% noisy data in the original training set of WOS and conduct experiments on BERT-base. The results show that the performance keeps almost unchanged with our proposed method but drops significantly without our method. We hypothesize the stability comes from the "Semantic-Level and Phrase-Level Label Information Enhanced Prompt," which can unify the format of model inputs to keep the generalization ability obtained in the pre-training stage and utilize both the information of label names and keywords to avoid disturbing from noisy label names as much as possible.

heading: C Case Analysis
text: We show more data generated by our proposed methods in Table 9, 10 and 11. We can observe that the data generated by our proposed method is diverse at both semantic and grammatical levels and satisfies the meaning of the original labels.
Original Label: CS, Cryptography Original Text: a new method for obtaining random bijective s-boxes based on discrete chaotic map is presented. the proposed method uses a
discrete chaotic map based on the composition of permutations. the obtained s-boxes have been tested on the number of criteria, such as bijection, nonlinearity, strict avalanche criterion, output bits independence criterion, equiprobable input/output xor distribution and maximum expected linear probability. the results of performance test show that the s-box presented in this paper has good cryptographic properties. the advantage of the proposed method is the possibility to achieve large key space, which makes it suitable for generation of s-boxes for larger values of n. also, because this method uses discrete chaotic map based on the composition of permutations which has finite space domain, there is no need for discretization of continuous values of chaotic map, so the process of generation of s-boxes is not affected by approximations of any kind.
Original Label: Medical, Ankylosing Spondylitis Original Text: ankylosing spondylitis (as) is a chronic systemic inflammatory disease. via autoimmune mediators, as can damage the auditory
system similar to other systems. otoacoustic emission studies in as patients showed that the damage that causes hearing loss was in the outer hair cells. the medial olivocochlear (moc) reflex is used to evaluate the moc efferent system (moes), which includes the outer hair cells. the aim of this study was to evaluate the presence of subclinical damage in the inner ear with the aid of the moc reflex test in as patients with no hearing complaints. thirty-four patients with as and a control group of 30 healthy volunteers with similar demographic characteristics were evaluated in the study. otoacoustic emission responses, moc reflex results, and frequency-specific and total suppression findings were compared between the groups. the relationship between clinical and laboratory findings for the as patients, and the moc reflex data were also investigated. reduced moc reflex response (p = 0.04) and suppression (p = 0.019) were detected in as patients. when the clinical and laboratory findings for the as patients and the moc reflex test results were compared, a significant correlation was found only between the moc reflex and the erythrocyte sedimentation rate. the results showed that as can damage the inner ear, especially the moes, and can reduce the moc reflex response without clinical hearing loss.
Original Label: CS, Cryptography Original Text: with ever increasing load and insufficient growth of generation and transmission capacities, the operating constraints of modern
integrated power system are increasing. it has led to ever increasing dynamic problems of low frequency oscillations in the system that needs to be detected and damped out quickly and efficiently. in electrical power networks small oscillations appear from time to time. these oscillations concern the quantities determining the equilibrium point of the system, and following which, system stability and system behaviors are influenced. the objective of our study is to check the static stability of the high voltage power to small perturbations of electrical network. in this context, after linearizing the system, power system stabilizer (pss) has so far been extensively utilized to mitigate these problems. this paper presents a novel and efficient approach for the optimal tuning of power system stabilizer parameters (pss) using a genetic algorithm (ga) with the eigenvalue-based objective function. the proposed approach is implemented and examined in a system with a single machine connected to an infinite bus via a transmission line. the results of this technique have been verified by eigenvalue analysis and time-domain simulations. the obtained results were evaluated and compared with ones obtained by ordinary gas. eigenvalue analysis and nonlinear system simulations demonstrated the effectiveness of the proposed approach in damping the electromechanical oscillations and enhancing the system dynamic stability.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
6
3 A2. Did you discuss any potential risks of your work? 7
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? No response.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? No response.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No response.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No response.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
C 3 Did you run computational experiments? 4
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? 4
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? 4
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 4
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 4
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Although the data generation is usually complete offline and does not improve the time cost of online progress, we leave how to relieve the need for GPU resources as future directions. (2). Although we conduct experiments on three widely used HTC benchmarks, the language of all these benchmarks is English, which has limited morphology. The effectiveness of our proposed method on language with varied morphology needs to be further confirmed