Improving Pretrained Models for Zero-shot Multi-label Text Classification through Reinforced Label Hierarchy Reasoning (2021)

Abstract: Exploiting label hierarchies has become a promising approach to tackling the zero-shot multi-label text classification (ZS-MTC) problem. Conventional methods aim to learn a matching model between text and labels, using a graph encoder to incorporate label hierarchies to obtain effective label representations (Rios and Kavuluru, 2018). More recently, pretrained models like BERT (Devlin et al., 2018) have been used to convert classification tasks into a textual entailment task (Yin et al., 2019). This approach is naturally suitable for the ZS-MTC task. However, pretrained models are underexplored in the existing work because they do not generate individual vector representations for text or labels, making it unintuitive to combine them with conventional graph encoding methods. In this paper, we explore to improve pretrained models with label hierarchies on the ZS-MTC task. We propose a Reinforced Label Hierarchy Reasoning (RLHR) approach to encourage interdependence among labels in the hierarchies during training. Meanwhile, to overcome the weakness of flat predictions, we design a rollback algorithm that can remove logical errors from predictions during inference. Experimental results on three real-life datasets show that our approach achieves better performance and outperforms previous non-pretrained methods on the ZS-MTC task.

## introduction multi-label text classification (mtc) is a basic nlp problem that underlies many real-life applications like product categorization @xcite and medical records coding @xcite . the labels in the output space are often interdependent and in many applications organized in a hierarchy, as shown in the example in figure 1 . a significant challenge for real-life development of mtc applications is severe deficiencies of annotated data for each label in the hierarchy, which demands better solutions for zero-shot learning. the existing zero-shot learning for multi-label text classification (zs-mtc) mostly learns a matching model between the feature space of text and the label space @xcite . in order to learn effective representations for labels, a majority of existing work incorporates label hierarchies via a label encoder designed as graph neural networks (gnns) that can aggregate the neighboring information for labels @xcite . recently, pretrained models like bert @xcite have been widely used as strong matching models due to their superior representation ability @xcite . they have been applied to convert a classification task to a textual entailment task, by treating the text to be classified as the premise, and its label as the hypothesis, which is naturally suitable for the zs-mtc study @xcite . however, the problem of this approach is that pretrained models cannot generate individual vector representations for labels-a label is coupled with the corresponding text in learning joint representation-thus conventional methods, like gnns which utilize the label hierarchy to obtain better label representations, cannot be directly applied to pretrained models, making them underexplored in the existing research. although pretrained models have shown potential on zs-mtc, as discussed above, it is not intuitive to introduce structural information of label hierarchies to the learning procedure. flattening all the labels without considering their hierarchical structures, however, will result in predictions that contain logical errors, which are known as the class-membership inconsistency @xcite . the problem will be even more salient for pretrained models because they only take the literal tokens of the labels as input. an example with logical errors is shown in figure 1 . without label hierarchy information, the model correctly predicts bikes as a true label, but fails to predict its parent label, sporting goods. meanwhile, the model does not choose the label local services while predicting its child label bike repair due to the fact that bike repair has tokens similar to those in the input text. to overcome the forementioned weakness, we propose a reinforced label hierarchy reasoning (rlhr) approach to introduce label structure information to pretrained models. instead of regarding labels to be independent, we cast zs-mtc as a deterministic markov decision process (mdp) over the label hierarchy. an agent starts from the root label and learns to navigate to the potential labels by hierarchical deduction in the label hierarchy. the reward is based on the correctness of the deduction paths, not simply on the correctness of each label. thus the reward received by one predicted label will be determined by both the label itself and other labels on the same path, which can help to strengthen the interconnections among labels. meanwhile, we find that the hierarchical inference method @xcite will broadcast the errors arising at the higher levels of label hierarchies. thus we further design a rollback algorithm based on the predicted matching scores of labels to reduce the logical errors in the flat prediction mode during inference. we apply our approach to different pretrained models and conduct experiments on three real-life datasets. results demonstrate that pretrained models outperform conventional nonpretrained methods by a substantial margin. after being combined with our approach, pretrained models can attain further improvement on both the classification metrics and logical error metrics foot_0 . we summarize our contributions as follows: ## related work exploiting the prior distribution of the label space has proven to be an effective method to tackle the multi-label text classification problem because it can provide the model with information about the label structure. mao et al. (2019 leveraging the label hierarchy to tackle zs-mtc has shown to be promising in previous work, which mostly aimed to learn a matching model between texts and labels. @xcite ; xie et al. (2019) adopted label-wise attention networks to encourage interactions between text and labels. @xcite ; @xcite used graph neural networks to capture the structural information in the label hierarchy. however, few existing works investigate the effectiveness of pretrained models on the zs-mtc task, despite pretrained models being effective as matching models for many natural language processing tasks @xcite @xcite . the logical error problem in flat predictions has been widely discussed in previous mtc work @xcite @xcite , which is mostly solved through a hierarchical procedure during inference. in our work, we will investigate such a method and see that the hierarchical inference method is not optimal for pretrained models on the zs-mtc task because it broadcasts errors top-down in the label hierarchy. path reasoning is effective for exploiting explicit relationships in structured data, which can be combined with reinforcement learning, e.g., knowledge graph reasoning @xcite @xcite . we propose to introduce the label hierarchy to pretrained models through path reasoning, with the aim to strengthen the intercon-nections between labels. to the best of our knowledge, our work is the first to improve pretrained models through label hierarchies for zs-mtc. ## methodology the goal of our rlhr approach is to learn a policy p that can make more consistent predictions by traversing the label hierarchy g to generate deduction paths. given a training instance x, an agent will start from the root r and follow p at each time step to extend the deduction paths by navigating to the children labels at the next level. by measuring the correctness of the generated deduction paths with reinforcement learning (rl), the label hierarchy is introduced to the model during the training time and the interconnections of labels will hence be strengthened, which can help to reduce logical errors in prediction. as we will show in our experiments, hierarchical inference, which is used in previous work @xcite , will propagate the errors occurring at the high levels of hierarchies during inference, resulting in inferior performance. thus we still adopt the flat prediction during inference, but further design a rollback algorithm based on the structure of g and the predicted matching scores. we will introduce the details of our proposed rlhr and the rollback algorithm in the following subsections. ## conclusion we propose a reinforced label hierarchy reasoning approach to incorporate label hierarchies into pretrained models in order to better solve the zeroshot multi-label text classification tasks. we train an agent that starts from the root label, navigates to potential labels in the label hierarchies and generates multiple deduction paths. by rewarding based on the sampled deduction paths, our approach can strengthen the interconnections among the labels during the training stage. to overcome the weakness of hierarchical inference methods, we further design a rollback algorithm that can remove the logical errors in flat predictions. experiments on the three datasets demonstrate that our proposed approach improves the performance of pretrained models and enable the models to make more consistent predictions.