Towards Better Hierarchical Text Classification with Data Generation (2023)

Abstract: Hierarchical text classification (HTC) focuses on classifying one text into multiple labels, which are organized as a hierarchical taxonomy. Due to its wide involution in realistic scenarios, HTC attracts long-term attention from both industry and academia. However, the high cost of hierarchical multi-label annotation makes HTC suffer from the data scarcity problem. In view of the difficulty in balancing the controllability of multiple structural labels and text diversity, automatically generating high-quality data for HTC is challenging and under-explored. To fill this blank, we propose a novel data generation framework tailored for HTC, which can achieve both label controllability and text diversity by extracting high-quality semantic-level and phrase-level hierarchical label information. Experimental results on three benchmarks demonstrate that, compared with existing data augmentation methods, the data generated from our method can bring the most significant performance improvements of several strong HTC models. Extensive analysis confirms that the improvements yielded by our proposed method do correlate to the enhancement of label controllability and text diversity.

## introduction hierarchical text classification (htc) is a representative multi-label text classification problem, aiming to assign one text with multiple labels in a given label hierarchical taxonomy. htc is widely involved in realistic scenarios, e.g., news classification @xcite , science paper classification @xcite , ecommerce @xcite . to solve such an important and challenging task, adequate high-quality labeled data is indispensable. however, multi-labeled data annotation is usually expensive, and the hierarchical structure of multi-label further makes such annotation unaffordable. thus, the primary goal of our approach is to deal with the data scarcity problem. existing works @xcite @xcite @xcite focus on enhancing the model ability to relieve the need for annotated data, but few of them address this problem by automatically generating high-quality data. recently, since generative pre-trained language models (plms) have achieved surprising performance @xcite @xcite , generative data augmentation has drawn more and more attention @xcite @xcite . with the help of the rich knowledge obtained from the large-scale data in the pre-training stage, generative data augmentation can further improve the quality of the generated data, which can better alleviate the data scarcity problem. however, generating high-quality data for htc is still an under-explored problem. we argue that two main challenges exist for this problem: the need for label controllability and text diversity. if the meaning of the generated data is out of the control of the given labels, it is usually noisy data assigned with wrong labels, which does little help and even harms htc models. besides, the generated data may not improve the generalization ability of htc models if the expression is similar to the original text. intuitively, restricted label constraints can ensure label controllability but might impede the text diversity of the generated data, especially in htc, which has multiple label constraints in a hierarchical structure @xcite . therefore, achieving a good balance between controllability and diversity is the key to generating high-quality data for htc. to improve label controllability and text diversity of generated data, we propose a novel data generation framework for htc, which aims to enrich the input information of the data generation model. specifically, we first design the semanticlevel and phrase-level label information enhanced prompt, consisting of label names and phrases extracted from origin training samples. although label names can improve label controllability, to a certain extent, only leveraging label names makes the model inevitably generate similar data since the same input label name needs to create hundreds and even thousands of data samples. thus, the phrases extracted from original train samples can be used as a supplement to label names. to ensure label controllability and text diversity, we need the phrases to represent the critical information that can infer the origin label correctly and simultaneously avoid common expressions. therefore, to deal with these problems, we propose a hierarchical label information enhanced keyword extractor to extract the most relevant phrase to the label meaning. finally, to further improve the label controllability, we introduce consistency filtering to filter out the low-quality generated data, which makes the htc model predict different labels compared to the controlled labels. experimental results on three benchmarks confirm the effectiveness of our proposed method. furthermore, both quantitative and qualitative analyses show the superiority of our method over existing data augmentation methods in comparing label controllability and text diversity. in a nutshell, our contributions are as follows: mao et al. (2019) use hierarchical information to enhance the label assignment policy and propose a deep reinforcement learning-based general framework for htc; wu et al. (2019) utilize a meta-learner to deal with the complexity and dependencies of different labels; aly et al. (2019) introduce capsule networks for htc and utilize label co-occurrence to initialize weight better; deng et al. (2021) propose text-label mutual information maximization and label prior matching to filter out irrelevant information and learn better hierarchy-aware representations; rojas et al. (2020) define htc as a sequence-to-sequence problem and utilize an auxiliary synthetic task and external knowledge to improve performance further. recently, there are more and more focus on leveraging a structure encoder to model the hierarchical information @xcite @xcite @xcite . different from previous work, we focus on introducing generative data augmentation to address the data scarcity problem of htc and use a structure encoder to model hierarchical label information to improve the quality of the generated data. ## method in this section, we first give the task formulation for htc and the goal of data generation. then, we introduce our framework consisting of the semanticlevel and phrase-level label information enhanced prompt, hierarchical label information enhanced keyword extractor and consistency filtering. ## conclusion htc is an important and challenging task, which usually suffers from the data scarcity problem because of the high cost of hierarchical multi-label annotation. with the development of generative plms, generating high-quality data for htc becomes possible. in this paper, to deal with the data scarcity problem, we explore the effect of generative augmentation methods on htc models. in order to improve the label controllability and text diversity, we propose a novel data generation framework for htc, which consists of semanticlevel and phrase-level label information enhanced prompt, hierarchical label information enhanced keyword extractor and consistency filtering. despite the challenge of generating high-quality data for htc, our proposed framework can achieve a balance between the controllability of multiple hierarchical labels and text diversity and improve the performance of several strong htc models, which is demonstrated by experimental results on three htc benchmarks and comprehensive analysis.