Venue: NAACL 2021

Improving Pretrained Models for Zero-shot Multi-label Text Classification through Reinforced Label Hierarchy Reasoning

Hui Liu\({}^{1}\) Danqing Zhang\({}^{2}\) Bing Yin\({}^{2}\) Xiaodan Zhu\({}^{1}\)

\({}^{1}\)Ingenuity Labs Research Institute & ECE, Queen's University, Canada

{hui.liu, xiaodan.zhu}@queensu.ca

\({}^{2}\)Amazon.com Inc, Palo Alto, CA, USA

{danginz, alexbyin}@amazon.com

###### Abstract

Explotiing label hierarchies has become a promising approach to tackling the zero-shot multi-label text classification (ZS-MTC) problem. Conventional methods aim to learn a matching model between text and labels, using a graph encoder to incorporate label hierarchies to obtain effective label representations Rios and Kavuluru (2018). More recently, pretrained models like BERT Devlin et al. (2018) have been used to convert classification tasks into a textual entailment task Yin et al. (2019). This approach is naturally suitable for the ZS-MTC task. However, pretrained models are underexplored in the existing work because they do not generate individual vector representations for text or labels, making it unintuitive to combine them with conventional graph encoding methods. In this paper, we explore to improve pretrained models with label hierarchies on the ZS-MTC task. We propose a Reinforced Label Hierarchy Reasoning (RLHR) approach to encourage interdependence among labels in the hierarchies during training. Meanwhile, to overcome the weakness of flat predictions, we design a rollback algorithm that can remove logical errors from predictions during inference. Experimental results on three real-life datasets show that our approach achieves better performance and outperforms previous non-pretrained methods on the ZS-MTC task.

+
Footnote †: _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 1051–1062 June 6–11, 2021. ©2021 Association for Computational Linguistics

## 1 Introduction

Multi-label text classification (MTC) is a basic NLP problem that underlies many real-life applications like product categorization Partalas et al. (2015) and medical records coding Du et al. (2019). The labels in the output space are often interdependent and in many applications organized in a hierarchy, as shown in the example in Figure 1. A significant challenge for real-life development of MTC applications is severe deficiencies of annotated data for each label in the hierarchy, which demands better solutions for zero-shot learning. The existing zero-shot learning for multi-label text classification (ZS-MTC) mostly learns a matching model between the feature space of text and the label space Ye et al. (2020). In order to learn effective representations for labels, a majority of existing work incorporates label hierarchies via a label encoder designed as Graph Neural Networks (GNNs) that can aggregate the neighboring information for labels Chalkidis et al. (2020); Lu et al. (2020).

Recently, pretrained models like BERT Devlin et al. (2018) have been widely used as strong matching models due to their superior representation ability Qiao et al. (2019). They have been applied to convert a classification task to a textual entailment task, by treating the text to be classified as the premise, and its label as the hypothesis, which is naturally suitable for the ZS-MTC study Yin et al. (2019). However, the problem of this approach is that pretrained models cannot generate individual vector representations for labels--a label is coupled with the corresponding text in learning joint representation--thus conventional methods, like GNNs which utilize the label hierarchy to obtain better label representations, cannot be directly applied to pretrained models, making them underexplored in the existing research.

Although pretrained models have shown potential on ZS-MTC, as discussed above, it is not intuitive to introduce structural information of label hierarchies to the learning procedure. Flattening

Figure 1: An example of label hierarchy and predictions with logical errors. Circled labels are model predictions without incorporating label hierarchy.

 

all the labels without considering their hierarchical structures, however, will result in predictions that contain logical errors, which are known as the class-membership inconsistency Silla and Freitas (2011). The problem will be even more salient for pretrained models because they only take the literal tokens of the labels as input. An example with logical errors is shown in Figure 1. Without label hierarchy information, the model correctly predicts _Bikes_ as a true label, but fails to predict its parent label, _Sporting Goods_. Meanwhile, the model does not choose the label _Local Services_ while predicting its child label _Bike Repair_ due to the fact that _Bike Repair_ has tokens similar to those in the input text.

To overcome the forementioned weakness, we propose a Reinforced Label Hierarchy Reasoning (RLHR) approach to introduce label structure information to pretrained models. Instead of regarding labels to be independent, we cast ZS-MTC as a deterministic Markov Decision Process (MDP) over the label hierarchy. An agent starts from the root label and learns to navigate to the potential labels by hierarchical deduction in the label hierarchy. The reward is based on the correctness of the deduction paths, not simply on the correctness of each label. Thus the reward received by one predicted label will be determined by both the label itself and other labels on the same path, which can help to strengthen the interconnections among labels. Meanwhile, we find that the hierarchical inference method Huang et al. (2019) will broadcast the errors arising at the higher levels of label hierarchies. Thus we further design a rollback algorithm based on the predicted matching scores of labels to reduce the logical errors in the flat prediction mode during inference. We apply our approach to different pretrained models and conduct experiments on three real-life datasets. Results demonstrate that pretrained models outperform conventional non-pretrained methods by a substantial margin. After being combined with our approach, pretrained models can attain further improvement on both the classification metrics and logical error metrics1. We summarize our contributions as follows: Footnote 1: Code and data available at [https://github.com/layneins/Zero-shot-RLHR](https://github.com/layneins/Zero-shot-RLHR)

* We demonstrate that pretrained models outperform conventional methods on ZS-MTC.
* We design a novel Reinforced Label Hierarchy Reasoning (RLHR) approach and a matching-score-based rollback algorithm to introduce the structural information of label hierarchies to pretrained models in both the training and inference stage.
* Experiments with different pretrained models are performed on three real-life datasets. We show the effectiveness of our proposed approach and provide detailed analyses.

## 2 Related Work

Exploiting the prior distribution of the label space has proven to be an effective method to tackle the multi-label text classification problem because it can provide the model with information about the label structure. Mao et al. (2019); Huang et al. (2019) took the explicitly represented label hierarchy as the structural information, while Wu et al. (2019) assumed the prior distribution to be implicit and trained their model to learn the distribution during learning.

Leveraging the label hierarchy to tackle ZS-MTC has shown to be promising in previous work, which mostly aimed to learn a matching model between texts and labels. Chalkidis et al. (2020, 2019); Xie et al. (2019) adopted Label-Wise Attention Networks to encourage interactions between text and labels. Rios and Kavuluru (2018); Lu et al. (2020) used Graph Neural Networks to capture the structural information in the label hierarchy. However, few existing works investigate the effectiveness of pretrained models on the ZS-MTC task, despite pretrained models being effective as matching models for many natural language processing tasks Ma et al. (2019); Qiao et al. (2019); Nogueira et al. (2019).

The logical error problem in flat predictions has been widely discussed in previous MTC work Silla and Freitas (2011); Wehrmann et al. (2018); Mao et al. (2019), which is mostly solved through a hierarchical procedure during inference. In our work, we will investigate such a method and see that the hierarchical inference method is not optimal for pretrained models on the ZS-MTC task because it broadcasts errors top-down in the label hierarchy.

Path reasoning is effective for exploiting explicit relationships in structured data, which can be combined with reinforcement learning, e.g., knowledge graph reasoning Wan et al. (2020); Xian et al. (2019); Xiong et al. (2017). We propose to introduce the label hierarchy to pretrained models through path reasoning, with the aim to strengthen the intercon 

nections between labels. To the best of our knowledge, our work is the first to improve pretrained models through label hierarchies for ZS-MTC.

## 3 Problem Formulation

### Label Hierarchy Reasoning

In general, a label hierarchy is defined as \(\bar{G}=(\mathcal{L},\mathscr{E})\), where \(\mathcal{L}\) and \(\mathscr{E}\) are a set of labels and relations, respectively. The latter represent parent-child relations between labels. The root of \(\mathcal{G}\) is a special label \(\mathbb{R}\). A data instance \(\mathbf{x}\) is defined as a tuple \((T,P)\) with \(T\) as the input text and \(P=\{p_{1},p_{2},\cdots,p_{N}\}\) as deduction paths, and a path \(p_{i}=\{\mathbb{R},l^{1}_{i},\cdots,l^{K-1}_{i},l^{K}_{i}\}\) where \(l^{k}_{i}\in\mathcal{L}\) is at the \(k^{th}\) layer of \(\bar{G}\) and \(l^{k-1}_{i}\) is the parent of \(l^{k}_{i}\). A deduction path must be contiguous, starting with \(\mathbb{R}\), and is not required to terminate at a leaf label.

### Zero-shot Multi-label Text Classification

Let \(\mathcal{L}_{s}\) and \(\mathcal{L}_{u}\) denote the seen and unseen labels, respectively, where \(\mathcal{L}_{s}\cup\mathcal{L}_{u}=\mathcal{L}\). Given a training set \(\mathcal{D}^{s}=\{\mathbf{x}^{s}_{i}\}_{i=1}^{N_{s}}\) where the labels of \(\mathbf{x}^{s}_{i}\) are all seen labels, we aim to learn a matching model \(f(\mathcal{D}^{s};\theta)\) and make prediction on \(\mathcal{D}^{u}=\{\mathbf{x}^{u}_{i}\}_{i=1}^{N_{2}}\). Some deduction paths of \(\mathbf{x}^{u}_{i}\) consist of seen labels while some contain both seen and unseen labels. Notice that the children of an unseen label are also unseen labels. Evaluations on \(\mathcal{D}^{u}\) will be conducted in two settings: (1) evaluate the performance on \(\mathcal{L}_{u}\), which is known as the zero-shot (ZS) setting, and (2) evaluate the performance on \(\mathcal{L}_{s}\cup\mathcal{L}_{u}\), which is the generalized zero-shot (GZS) setting Huynh and Elhamifar (2020).

## 4 Methodology

The goal of our RLHR approach is to learn a policy \(\mathcal{D}\) that can make more consistent predictions by traversing the label hierarchy \(\bar{G}\) to generate deduction paths. Given a training instance \(x\), an agent will start from the root \(\mathbb{R}\) and follow \(\mathcal{D}\) at each time step to extend the deduction paths by navigating to the children labels at the next level. By measuring the correctness of the generated deduction paths with reinforcement learning (RL), the label hierarchy is introduced to the model during the training time and the interconnections of labels will hence be strengthened, which can help to reduce logical errors in prediction. As we will show in our experiments, hierarchical inference, which is used in previous work Mao et al. (2019), will propagate the errors occurring at the high levels of hierarchies during inference, resulting in inferior performance. Thus we still adopt the flat prediction during inference, but further design a rollback algorithm based on the structure of \(\bar{G}\) and the predicted matching scores. We will introduce the details of our proposed RLHR and the rollback algorithm in the following subsections.

### Base Model

Our base model adopts pretrained models \(\Pi l\), e.g., BERT Devlin et al. (2018), which have proven to be effective in matching modelling. Given the input text \(T\) and the label \(l\), we follow Yin et al. (2019) by transforming the text-label pair into textual entailment representation as "[CLS] \(T\) [SEP] hypothesis of \(l\)". The hidden vector \(v_{cls}\) of [CLS] is regarded as the aggregate representation and will be used in the classification layer to calculate the matching score \(ms\). The overall calculation process of \(ms\) is abbreviated as:

\[ms=\mathcal{I}l(T,l) \tag{1}\]

If \(ms\geq\gamma\) where \(\gamma\) is a threshold, we then say \(T\) belongs to label \(l\). In experiments \(\gamma\) is set to be 0.5.

### Reinforced Label Hierarchy Reasoning (RLHR)

Different from vanilla pretrained models that rely on flat prediction during training, we propose to formulate the ZS-MTC task as a deterministic Markov Decision Process (MDP) over label hierarchies. For the input text, the agent trained by RLHR will predict \(M\) deduction paths from the root label \(\mathbb{R}\). When all deduction paths are generated, the rewards will be received, which are determined by the correctness of the paths. An overall illustration of the RLHR approach is shown in Figure 2. We introduce the details of the RL modules in this subsection.

#### 4.2.1 States

Maintaining just one deduction path for one data instance will result in an inefficient learning process. However, the number of potential deduction paths will increase exponentially as the model goes deeper into the lower levels of the hierarchies. To maintain a good trade-off between computational resources and time efficiency, we keep the beam of deduction paths to be \(M\). Thus for a data instance \(x\), the global state \(S^{k}\) at step \(k\) is composed of the sub-states of \(M\) deduction paths:

\[S^{k}=\{s^{k}_{i}\}_{i=1}^{M}\] 

The sub-state \(s_{i}^{k}\) for deduction path \(p_{i}\) at step \(k\) is defined as a tuple \((T,l_{i}^{k})\), where \(T\) is input text and \(l_{i}^{k}\) is the label.

#### 4.2.2 Actions

The complete action space \(A_{i}^{k}\) of sub-state \(s_{i}^{k}\) is defined as all possible child labels of label \(l_{i}^{k}\):

\[A_{i}^{k}=\{l|l\in C(l_{i}^{k})\} \tag{3}\]

where \(C(l_{i}^{k})\) denotes the child labels of \(l_{i}^{k}\). For the deduction path \(p_{i}\) at the time step \(k\), an action \(a_{i}^{k}\) is to select one label \(l_{i}^{k+1}\) from \(A_{i}^{k}\). Notice that the agent may not select any labels from \(A_{i}^{k}\), which means path \(p_{i}\) ends before it arrives at a leaf label and a "stop" action is taken. By adding this "early stop" mechanism, we can make the agent automatically learn when to stop assigning new labels to the deduction paths.

#### 4.2.3 Policy

We parameterize the action \(a_{i}^{k}\) by a policy network \(\pi(\cdot|s,A;\theta)\) where \(\theta\) is parameters. For deduction path \(p_{i}\) at time step \(k\), the policy network takes as input the state \(s_{i}^{k}\) and the corresponding action space \(A_{i}^{k}\), emitting the matching score of each action in \(A_{i}^{k}\), which is calculated by the base pretrained model \(\mathcal{H}\). Finally an action \(a_{k}\) is sampled based on the matching score distribution of the actions in \(A_{i}^{k}\). The calculation is formulated as follows:

\[\pi(\mathbf{a_{i}^{k}}|s_{i}^{k},A_{i}^{k};\theta)=\{\mathcal{H}(T,l) |l\in A_{i}^{k}\} \tag{4}\] \[a_{i}^{k}\sim\pi(\mathbf{a_{i}^{k}}|s_{i}^{k},A_{i}^{k};\theta) \tag{5}\]

#### 4.2.4 Reward

In our approach, the reward is based on the correctness of a complete deduction path. Instead of treating all labels to be flat, our approach encourages the interdependence among the labels. The reward received by a label \(l_{i}^{k}\) is not only decided by the correctness of itself but also the correctness of other labels on the same deduction path \(p_{i}\). Given the golden deduction paths \(\hat{P}=\{\hat{p}_{1},\hat{p}_{2},\cdots,\hat{p}_{N}\}\), \(p_{i}\) will obtain a positive reward if \(p_{i}\) is in \(\hat{P}\) or \(p_{i}\) is a sub-path of a path in \(\hat{P}\). Formally the reward of path \(p_{i}\) is defined as:

\[r_{i}=\begin{cases}\lambda\cdot 1,&\text{if $p_{i}\subseteq\hat{p}_{j}$ where $\hat{p}_{ j}\in\hat{P}$}\\ -1,&\text{otherwise,}\end{cases} \tag{6}\]

where \(\lambda\) is a hyper-parameter for scaling. Under most circumstances, the number of wrong deduction paths will be greater than the correct ones. The problem will be even more severe for the MTC tasks because the distribution of positive labels and negative labels is usually imbalanced given a data instance \(x\). A larger \(\lambda\) can encourage the model to focus more on the correct paths.

Notice that our approach differs from existing methods which adopt hierarchical classification Sun and Lim (2001); Peng et al. (2018). A hierarchical classification method based on the label hierarchy can only cast the influence from parent label to child label, while in our approach the influence is mutual between parent label and child label, which can hence strengthen the reasoning ability of the models.

#### 4.2.5 Optimization

Our goal is to learn a stochastic policy \(\pi\) that maximize the expected total reward \(J(\theta)\) of the M sampled deduction paths, which can be formulated as:

\[J(\theta)=E_{\pi(a|s)}[\sum_{i=1}^{M}r_{i}(\mathbf{s},\mathbf{a})] \tag{7}\]

where \(\theta\) is the parameter of policy network. We adopt policy gradient Sutton et al. (2000) as the optimization algorithm which updates \(\theta\) as:

\[\theta\leftarrow\theta+\eta\nabla_{\theta}\tilde{J}(\theta) \tag{8}\]

Figure 2: An example of our RLHR approach with \(M=4\). Green circles are the ground truth labels. \(p_{1},p_{2},p_{3}\), and \(p_{4}\) are four sampled deduction paths, where \(p_{3}\) ends before it arrives at a leaf label.

 

where \(\eta\) is the discount learning rate. Since there are multiple deduction paths for one data instance, the gradient can be approximated by

\[\tilde{J}(\theta)=\frac{1}{M}\sum_{i=1}^{M}\sum_{k}\log\pi(a_{i}^{ k}|s_{i}^{k};\theta)\cdot(r_{i}-r_{b}) \tag{9}\]

\(r_{b}\) is a constant for the stabilization of the training procedure, for which we use the average reward of the last training epoch in our experiments.

### Inference Rollback

Existing methods mostly adopt the hierarchical inference method Mao et al. (2019), which will avoid logical errors, i.e., class-membership inconsistency Silla and Freitas (2011), but bring a serious problem: the prediction errors made at the high levels of a hierarchy are often severely propagated to the lower levels. For instance, if a correct label at the first layer is missing, then all the descendant labels will not be considered during inference. This will no doubt harm the performance. On the contrary, if the model still makes flat prediction, all labels will be visited during inference, while more logical errors will probably arise.

To overcome the forementioned weaknesses, we propose a rollback algorithm during the inference stage based on the predicted matching scores of all labels. For a data instance \(x\), we obtain the predicted labels in flat prediction mode as \(P\), which consists of two parts: (1) labels that can form complete deduction paths, and (2) labels with logical errors, which we denote as \(P_{e}=\{l_{1}^{k_{1}},l_{2}^{k_{2}},\cdots,l_{N}^{k_{N}}\}\). For a label \(l_{i}^{k_{i}}\in P_{e}\), we extract its deduction path from \(\mathbb{G}\) as \(p_{i}=\{\mathbb{R},l_{i}^{1},\cdots,l_{i}^{k_{i}-1},l_{i}^{k_{i}}\}\) and their corresponding predicted matching scores \(\{1,ms_{i}^{1},\cdots,ms_{i}^{k_{i}-1},ms_{i}^{k_{i}}\}\)2. Meanwhile we set a rollback threshold \(\mu^{k}\) for the labels in the \(k^{th}\) layer of \(\mathbb{G}\), where \(\{\mu_{k}\}\) are hyper-parameters tuned on the development set. As long as the matching scores meet the requirements

Footnote 2: Root label \(\mathbb{R}\) always has a matching score 1.

\[\{ms_{i}^{j}\geq\mu^{j}\}_{j=1}^{k_{i}-1},\]

we add the labels in \(p_{i}\) back to \(P\). Otherwise label \(l_{i}^{k_{i}}\) will be removed from \(P\).

The motivation behind this matching-score-based rollback algorithm is that for a label hierarchy \(\mathbb{G}\), the labels at higher-level hierarchy contain more training instances but their meaning are more abstract, while the labels at lower levels are more specific such as the labels "Active Life" and "Bike Rentals" in Figure 1. Pretrained models just take as input the literal tokens of a label and thus are possible to obtain a better performance on certain labels at the lower levels than those at higher levels.

## 5 Experiments

### Experimental Setup

#### 5.1.1 Datasets

We conduct experiments on three real-life datasets from different domains; the details are provided in Table 1. Yelp3 is a customer review dataset, in which we need to classify customer reviews into correct business categories. WOS Kowsari et al. (2017) is a scientific paper dataset which provides the abstracts of published papers and the corresponding topics. QCD is a query classification dataset we create for the ZS-MTC task. It is composed of search queries and target product types, which is collected from e-commerce websites. The layer numbers of the label hierarchies in Yelp, WOS and QCD are 4, 2, and 3, respectively. For examples of the three datasets, please refer to Appendix A.1.

Footnote 3: [https://www.yelp.com/dataset](https://www.yelp.com/dataset)

#### 5.1.2 Implementation Details

We test our proposed approach with two pretrained models, BERT Devlin et al. (2018) and DistilBERT Sanh et al. (2019). For BERT, we use the uncased base version, which is of 12-layer transformer blocks, 768-dimension hidden state, 12 attention heads and 110M parameters in total. For DistilBERT, it contains 6-layers transformer blocks, 768-dimension hidden state and 12 attention heads, totally 66M parameters. For training, we use Adam Kingma and Ba (2014) for optimization and learning rate is set to 1e-6. Meanwhile we adopt early stopping to avoid overfitting on the training data. \(\lambda\) is set to 30 on Yelp, 20 on QCD, and 5 on WOS,

\begin{table}
\begin{tabular}{c|c c c c|c c} \hline \multirow{2}{*}{Dataset} & \multicolumn{4}{c|}{Docs} & \multicolumn{2}{c}{Labels} \\  & \#Train & \#Dev & \#Test & Avg(\(|L|\)) & seen unseen \\ \hline Yelp & 187153 & 10858 & 10858 & 3.80 & 466 & 71 \\ WOS & 36397 & 5294 & 5294 & 2.00 & 122 & 28 \\ QCD & 177423 & 12277 & 12277 & 4.69 & 243 & 93 \\ \hline \end{tabular}
\end{table}
Table 1: Dataset Statistics. Avg(\(|L|\)) denotes the average number of labels in one data instance.

 

which we will discuss more in Section 5.3.4. We set \(M\) to 5 with DistilBERT and 3 with BERT by trading off between training time and GPU memory usage.

The RL training procedure is unstable and slow if the agent is trained from scratch Silver et al. (2016). So with both BERT and DistilBERT, we pretrain the policy network in flat prediction mode on the training data with the learning rate of 1e-5.

#### 5.1.3 Evaluation Metrics

In our experiments, we use standard metrics MicroF1 and Macro-F1 to evaluate the classification performance for both the zero-shot and generalized zero-shot setting. Meanwhile, we also adopt Example-based F1 Peng et al. (2016) to measure the performance from the instance level, which is different from Micro/Macro-F1 measuring from the label level. Though some previous works adopted ranking based metrics Rios and Kavuluru (2018) for large-scale MTC, they are not appropriate in our settings because the datasets used in this work contain smaller label space.

For logical errors, we report the _logical error rate_, which is defined as the average number of logical errors in one data instance. We take the number of logical errors in one data instance as the number of labels that cannot form a complete deduction path.

Evaluation is conducted in two settings: (1) evaluate the performance on unseen labels only, which is the zero-shot (ZS) setting, and (2) evaluate the performance on both seen labels and unseen labels, i.e., the generalized zero-shot (GZS) setting Huynh and Elhamifar (2020).

### Baselines

We use two different types of baselines. (1) The type of models where label hierarchy is not utilized, and we use **CNN** and CNN with Label-Wise Attention Networks (**CNN+LWAN**) Chalkidis et al. (2019) in our experiments. (2) The type of models where GNNs are utilized to encode the label hierarchy to capture the label structure information. Specifically we use **ZAGCNN** proposed by Rios and Kavuluru (2018).

### Results

Table 2 shows the experimental results of the baseline models and our proposed RLHR approach on three real-life datasets in both the zero-shot and generalized zero-shot setting.

#### 5.3.1 Classification Performance

As we can see in Table 2 that CNN and CNN+LWAN have poor performance under the ZS

\begin{table}
\begin{tabular}{c|c|c c|c|c c|c c|c c|c} \hline \multirow{2}{*}{Method} & \multirow{2}{*}{Setting} & \multicolumn{3}{c|}{Yelp} & \multicolumn{3}{c|}{WOS} & \multicolumn{3}{c}{QCD} \\ \cline{3-17}  & & Ma-F & Mi-F & EBF & Err↓ & Ma-F & Mi-F & EBF & Err↓ & Ma-F & Mi-F & EBF & Err↓ \\ \hline CNN & ZS & 0.33 & 2.02 & 16.35 & 0.3211 & 0.36 & 4.43 & 7.00 & 29.58 & 28.22 & 0.2977 & 5.02 & 6.58 & 26.94 & 2.9386 \\  & GZS & 1.31 & 14.97 & 16.35 & 0.3211 & 0.36 & 4.43 & 7.00 & 29.58 & 28.22 & 0.2977 & 5.02 & 6.58 & 26.94 & 2.9386 \\ \hline CNN & ZS & 4.24 & 7.15 & 19.38 & 0.9303 & 0.54 & 4.53 & 4.53 & 4.53 & 4.52 & 0.3079 & 5.02 & 7.09 & & \\   +LWAN & GZS & 4.67 & 19.26 & & & 6.81 & 29.00 & & 10.03 & 28.86 & 28.24 & 4.3923 \\ \hline ZAGCNN & ZS & 17.94 & 18.75 & 28.24 & 1.3136 & 12.02 & 17.17 & 24.72 & 2.5827 & 5.22 & 10.01 & & \\  & GZS & 16.30 & 25.97 & 28.24 & 1.3136 & 19.59 & 36.37 & 24.72 & 2.5827 & 5.22 & 10.01 & & \\ \hline DistilBERT & ZS & 41.42 & 40.33 & 30.44 & 0.4039 & 70.69 & 65.19 & 55.18 & 0.5178 & 23.68 & 24.95 & & \\  & GZS & 21.29 & 28.18 & & 68.03 & 63.64 & & 24.43 & 34.29 & & \\ \hline \multirow{2}{*}{+RLHR} & ZS & 42.16 & 43.87 & 40.85 & 0.3347 & 74.56 & 72.44 & & 24.58 & 27.79 & & \\  & GZS & 26.95 & 40.43 & & 71.65 & 68.05 & 61.06 & 0.4732 & 24.58 & 27.79 & 37.46 & **0.8389** \\ \hline BERT & ZS & 44.49 & 42.61 & 34.59 & 0.3755 & 77.87 & 77.27 & & & & 28.18 & 27.45 & 36.88 & 1.2497 \\  & GZS & 23.38 & 31.53 & 34.59 & 0.3755 & 74.69 & 70.56 & & & & 27.04 & 37.20 & & \\ \hline \multirow{2}{*}{+RLHR} & ZS & **45.46** & **48.26** & & & **78.46** & **79.19** & & & & **28.32** & **28.80** & & \\  & GZS & **32.09** & **49.75** & **49.52** & **0.2952** & **75.51** & **72.62** & **64.43** & 0.2488 & **28.67** & 41.08 & 39.99 & 1.1984 \\ \hline \end{tabular}
\end{table}
Table 2: Results of different methods on the three datasets under two settings. Ma-F, Mi-F, EBF, and Err denote Macro-F1, Micro-F1, Example-based F1, and logical error rate, respectively. ZS and GZS denote the zero-shot and generalized zero-shot setting. ↓ means the lower the better. Bold numbers indicate the best results for each metric. All the results are acquired under the flat prediction.

 

setting while the performance under GZS setting is better, which suggests CNN and CNN+LWAN cannot provide accurate predictions for unseen labels due to the lack of label structure information. In contrast, ZAGCNN, which utilizes the label hierarchy, performs better, particularly on unseen labels, which demonstrates the importance of label hierarchy for ZS-MTC.

On the other hand, pretrained models, including DistilBERT and BERT, both outperform conventional non-pretrained methods with substantial improvements on three datasets, though ZAGCNN shows slight advantages on Micro-F1 and Example-based F1 on the QCD dataset under the GZS setting. When incorporated with RLHR, the performance of pretrained models can be further improved by a relatively large margin. We notice that the improvement under GZS setting is more significant than in the ZS setting, suggesting that seen labels benefit more from our RLHR than unseen labels.

#### 5.3.2 Logical Errors

As shown in Table 2, utilizing label hierarchies does not necessarily reduce the logical error rate for conventional methods, though it can improve the classification performance. For example, the logical error rate of ZAGCNN is higher than CNN and CNN+LWAN on Yelp and WOS. The logical error rate of pretrained models is generally lower than the conventional methods. However, pretrained models still face the logical error problem though they perform well on the classification metrics. We can also see that our RLHR can help reduce the logical error rate for DistilBERT and BERT under most circumstances.

Note that better classification performance does not necessarily lead to a lower logical error rate. From Table 2, we can see although CNN and CNN+LWAN perform poorly on classification metrics, they achieve a better logical error rate than ZAGCNN and DistilBERT on the WOS dataset. Similarly, the logical error rate of BERT is higher than DistilBERT on QCD even though BERT has a better classification performance. Our proposed RLHR approach can improve both the classification performance and logical error performance, which demonstrates the effectiveness of RLHR.

#### 5.3.3 Analyses on Rollback Algorithm

Due to the limit of space, we only report the results of our proposed rollback algorithm based on BERT and put the results on DistilBERT in Appendix A.2. As shown in Table 3, we can see that when being combined with our proposed rollback algorithm, the performance of BERT+RLHR can be further improved, raising Example-based F1 on Yelp, WOS, and QCD from 49.52%, 64.43%, 39.99% to 50.01%, 69.32% and 40.13%, respec

\begin{table}
\begin{tabular}{c|c|c c c|c c c|c c c} \hline \multirow{2}{*}{Method} & \multirow{2}{*}{Setting} & \multicolumn{3}{c|}{Yelp} & \multicolumn{3}{c|}{WOS} & \multicolumn{3}{c}{QCD} \\ \cline{3-11}  & & Ma-F & Mi-F & EBF & Ma-F & Mi-F & EBF & Ma-F & Mi-F & EBF \\ \hline \multirow{2}{*}{BERT} & ZS & 44.49 & 42.61 & \multirow{2}{*}{34.59} & 77.87 & 77.27 & \multirow{2}{*}{56.69} & 28.18 & 27.45 & \multirow{2}{*}{36.88} \\  & GZS & 23.38 & 31.53 & & & & & & & & \\ \hline BERT & ZS & 45.11 & 43.46 & \multirow{2}{*}{34.79} & 73.68 & 74.14 & \multirow{2}{*}{54.52} & 26.67 & 31.33 & \multirow{2}{*}{37.76} \\   +Hie-Infe & GZS & 23.58 & 31.72 & & & & & & & & \\ \hline BERT & ZS & 44.46 & 42.57 & \multirow{2}{*}{34.65} & 77.87 & 77.27 & \multirow{2}{*}{58.28} & 28.15 & 27.57 & \multirow{2}{*}{36.69} \\   +Rollback & GZS & 23.35 & 31.56 & & & & & & & & \\ \hline \hline \multirow{2}{*}{BERT+RLHR} & ZS & 45.46 & 48.26 & \multirow{2}{*}{49.52} & 78.46 & 79.19 & \multirow{2}{*}{64.43} & **28.32** & 28.80 & \multirow{2}{*}{39.99} \\   & GZS & 32.09 & 49.75 & & & & & & & & \\ \hline BERT+RLHR & ZS & 39.57 & 42.91 & \multirow{2}{*}{48.53} & 65.82 & 67.93 & \multirow{2}{*}{56.34} & 25.34 & **32.46** & **42.23** & **40.97** \\   +Hie-Infe & GZS & 31.22 & 49.2 & & & & & & & & \\ \hline BERT+RLHR & ZS & **45.57** & **48.32** & \multirow{2}{*}{**50.01**} & **78.46** & **79.19** & \multirow{2}{*}{69.32} & 28.03 & 29.71 & \multirow{2}{*}{40.13} \\   +Rollback & GZS & **32.17** & **50.18** & & & & & & & & \\ \hline \end{tabular}
\end{table}
Table 3: Performance of our matching-score-based rollback algorithm and the comparison to the hierarchical inference method. Ma-F, Mi-F, EBF, and Err denote Macro-F1, Micro-F1, Example-based F1, and logical error rate, respectively. ZS and GZS denote the zero-shot and generalized zero-shot setting. Bold numbers indicate the best results for each metric. “BERT+Hie-Info” in the last row means BERT with the hierarchical inference method, which is used in previous work Huang et al. (2019).

 

tively. Our proposed rollback algorithm can also be combined with BERT only, while the gain is relatively marginal. We further investigate this and observe that at the same level of the label hierarchy, the matching scores obtained in RLHR is more polarized, compared to those obtained with BERT, suggesting RLHR is more confident about the predictions when the label hierarchy is provided. This yields a better prediction performance of RLHR when the rollback algorithm is adopted.

Meanwhile, we compare the hierarchical inference method Huang et al. (2019) with our rollback algorithm. Both methods can completely remove logical errors from the predicted results. However, as we can see in the table, the performance of the hierarchical inference method is not consistent on the three datasets, with either BERT or BERT+RLHR. When conducting hierarchical inference, BERT+RLHR achieves the best Micro-F1 and Example-based F1 on QCD dataset, while the performance is harmed with a significant gap on the WOS dataset. Similarly, the performance of hierarchical inference with BERT achieves minor improvement on the QCD dataset, while on WOS and Yelp, the performance is sometimes improved marginally or sometimes worse. The effectiveness of hierarchical inference method depends mainly on the classification difficulty of labels at the higher levels of label hierarchies. As we know, such labels are usually more abstract and general, thus making the performance of hierarchical inference susceptible.

#### 5.3.4 Influence of \(\lambda\)

We discuss the influence of the parameter \(\lambda\) on logical error rates and unseen label classification in this section. Due to the limit of space, we only represent the results with BERT and put the results based on DistilBERT in Appendix A.3. As shown in Figure 3, for datasets with large hierarchy, like Yelp and QCD, a larger \(\lambda\) helps achieve better classification performance on unseen labels, while it will bring more logical errors. On the contrary, a relatively small \(\lambda\) yields better classification performance and lower logical error rates on datasets with small hierarchies like WOS, as shown in Figure 2(b). The reason is that for a large hierarchy, the number of sampled correct deduction paths will be much less than that of the wrong paths which is common in the ZS-MTC task because the positive labels are usually much less than negative labels, while for a small label hierarchy, the number of sampled correct paths are close to the false ones. A large \(\lambda\) will encourage a model to focus more on sampled correct paths, which will hence improve the classification performance. Meanwhile, if \(\lambda\) is too large, it will bring a bias to the dominating labels which appear more in the datasets. Thus it will reduce the generalization ability of the model, which will harm the performance.

## 6 Conclusion

We propose a Reinforced Label Hierarchy Reasoning approach to incorporate label hierarchies into pretrained models in order to better solve the zero-shot multi-label text classification tasks. We train an agent that starts from the root label, navigates to potential labels in the label hierarchies and generates multiple deduction paths. By rewarding based on the sampled deduction paths, our approach can strengthen the interconnections among the labels during the training stage. To overcome the weakness of hierarchical inference methods, we further design a rollback algorithm that can remove the logical errors in flat predictions. Experiments on the three datasets demonstrate that our proposed approach improves the performance of pretrained models and enable the models to make more consistent predictions.

Figure 3: Influence of \(\lambda\) on RLHR approach with BERT. Err, Ma-F and Mi-F denote logical error rate, Macro-F1 and Micro-F1 respectively.

 



## References

* C. M. Aitken, K. R. Hinton, and J. L. Friedman (2017)The machine learning approach to boosting neural networks. In Proceedings of the Thirtieth International Conference on Machine Learning and Knowledge Discovery, pp. 345-368. Cited by: SS2.
* I. Chalkidis, S. Fergadiotis, P. Malakasiotis, and I. Androutsopoulos (2019)Large-scale multi-label text classification on EU legislation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 6314-6322. External Links: Link, Document Cited by: SS2.
* I. Chalkidis, Z. Li, Y. Chen, Z. Huang, X. Liu, D. Zhang, and S. Wang (2019)Hierarchical multi-label text classification: an attention-based recurrent network approach. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1051-1060. Cited by: SS2.
* T. G. Chan, H. Lee, and F. Toutanova (2018)BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. Cited by: SS2.
* J. Du, Q. Chen, Y. Peng, Y. Xiang, C. Tao, and Z. Lu (2019)ML-net: multi-label classification of biomedical texts with deep neural networks. Journal of the American Medical Informatics Association26 (11), pp. 1279-1285. Cited by: SS2.
* W. Huang, E. Chen, Q. Liu, Y. Chen, Z. Huang, Y. Liu, Z. Zhao, D. Zhang, and S. Wang (2019)Hierarchical multi-label text classification: An attention-based recurrent network approach. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1051-1060. Cited by: SS2.
* D. Huynh and E. Elhamifar (2020)A shared multi-attention framework for multi-label zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8776-8786. Cited by: SS2.
* D. Kingma and J. Ba (2014)Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. Cited by: SS2.
* K. Kowsari, D. E Brown, M. Heidarysafa, K. Jafari Meimandi, M. S Gerber, and L. E. Barnes (2017)Holtex: hierarchical deep learning for text classification. In 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 364-371. Cited by: SS2.
* J. Lu, L. Du, M. Liu, and J. Dimnall (2020)Multi-label few/zero-shot learning with knowledge aggregated from multiple label graphs. arXiv preprint arXiv:2010.07459. Cited by: SS2.
* X. Ma, P. Xu, Z. Wang, R. Nallapati, and B. Xiang (2019)Universal text representation from bert: an empirical study. arXiv preprint arXiv:1910.07973. Cited by: SS2.
* Y. Mao, J. Tian, J. Han, and X. Ren (2019)Hierarchical text classification with reinforced label assignment. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 445-455. Cited by: SS2.
* R. Nogueira, W. Yang, K. Cho, and J. Lin (2019)Multi-stage document ranking with bert. arXiv preprint arXiv:1910.14424. Cited by: SS2.
* I. Pantalas, A. Kosmopoulos, N. Baskiotiis, T. Artieres, G. Paliouras, E. Gaussier, I. Androutsopoulos, M. Amini, and P. Galinari (2015)Lshtc: a benchmark for large-scale text classification. arXiv preprint arXiv:1503.08581. Cited by: SS2.
* H. Peng, J. Li, Y. He, Y. Liu, M. Bao, L. Wang, Y. Song, and Q. Yang (2018)Large-scale hierarchical text classification with recursively regularized deep graph-cnn. In Proceedings of the 2018 World Wide Web Conference, pp. 1063-1072. Cited by: SS2.
* S. Peng, R. You, H. Wang, C. Zhai, H. Mamitsuka, and S. Zhu (2016)Deepmesh: deep semantic representation for improving large-scale mesh indexing. Bioinformatics32 (12), pp. i70-i79. Cited by: SS2.
* Y. Qiao, C. Xiong, Z. Liu, and Z. Liu (2019)Understanding the behaviors of bert in ranking. arXiv preprint arXiv:1904.07531. Cited by: SS2.
* A. Rios and R. Kavuluru (2018)Fewshot and zero-shot multi-label learning for structured label spaces. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Online 2018, pp. 3132. Note: NIH Public Access Cited by: SS2.
* V. Sanh, L. Debut, J. Chaumond, and T. Wolf (2019)Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. Cited by: SS2.
* C. N. Silla and A. A. Freitas (2011)A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery22 (1-2), pp. 31-72. Cited by: SS2.
* D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al. (2016)Mastering the game of go with deep neural networks and tree search. nature529 (7587), pp. 484-489. Cited by: SS2.
* A. Sun and E. Lim (2001)Hierarchical text classification and evaluation. In Proceedings 2001 IEEE International Conference on Data Mining, pp. 521-528. Cited by: SS2.
* R. S. Sutton, D. A. McAllester, S. P. Singh, and Y. Mansour (2000)Policy gradient

methods for reinforcement learning with function approximation. In _Advances in neural information processing systems_, pages 1057-1063.
* Wan et al. (2020) Guojia Wan, Shirui Pan, Chen Gong, Chuan Zhou, and Gholamreza Haffari. 2020. Reasoning like human: Hierarchical reinforcement learning for knowledge graph reasoning. In _International Joint Conference on Artificial Intelligence 2020_, pages 1926-1932. Association for the Advancement of Artificial Intelligence (AAAI).
* Wehrmann et al. (2018) Jonatas Wehrmann, Ricardo Cerri, and Rodrigo Barros. 2018. Hierarchical multi-label classification networks. In _International Conference on Machine Learning_, pages 5075-5084.
* Wu et al. (2019) Jiawei Wu, Wenhan Xiong, and William Yang Wang. 2019. Learning to learn and predict: A meta-learning approach for multi-label classification. _arXiv preprint arXiv:1909.04176_.
* Xian et al. (2019) Yikun Xian, Zuohui Fu, S Muthukrishnan, Gerard De Melo, and Yongfeng Zhang. 2019. Reinforcement knowledge graph reasoning for explainable recommendation. In _Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval_, pages 285-294.
* Xie et al. (2019) Xiancheng Xie, Yun Xiong, Philip S Yu, and Yangyong Zhu. 2019. Ehr coding with multi-scale feature attention and structured knowledge graph propagation. In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_, pages 649-658.
* Xiong et al. (2017) Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. DeepPath: A reinforcement learning method for knowledge graph reasoning. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_, pages 564-573, Copenhagen, Denmark. Association for Computational Linguistics.
* Ye et al. (2020) Zhiquan Ye, Yukia Geng, Jiasoyan Chen, Jingmin Chen, Xiaoxiao Xu, SuHang Zheng, Feng Wang, Jun Zhang, and Huajun Chen. 2020. Zero-shot text classification via reinforced self-training. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 3014-3024.
* Yin et al. (2019) Wenpeng Yin, Jamaal Hay, and Dan Roth. 2019. Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 3914-3923, Hong Kong, China. Association for Computational Linguistics.

 

Appendix

### Dataset preparation

We split the labels in the label space as seen labels and unseen labels. Unseen labels do not necessarily need to be leaf labels, and if an intermediate label is chosen as unseen, then all its descendant labels will be set as unseen. Meanwhile, each data instance in dev/test sets will contain at least one unseen label.

Table 5 shows the example instances of Yelp, WOS and QCD datasets used in this work.

### Rollback Results with DistilBERT

As shown in Table 6, DistilBERT+RLHR with Rollback algorithm can achieve the best performance on most evaluation metrics. Although the hierarchical inference method can improve DistilBERT on QCD dataset, its performance is not consistent. It lowers the performance by large margins on WOS with both DistilBERT and DistilBERT+RLHR. In contrast, the rollback algorithm has consistent performance on all the three datasets, especially when combined with our proposed RLHR approach.

### Influence of \(\lambda\) with DistilBERT

As shown in Figure 4, the influence of parameter \(\lambda\) on three datasets with DistilBERT is similar to that with BERT. For Yelp and QCD datasets, a larger \(\lambda\) helps achieve better classification performance on unseen labels, while it will bring more logical errors. On the contrary, a relatively small \(\lambda\) yields both better classification performance and lower logical error rates on WOS dataset, as shown in Figure 3(b). The results support our analyses in Section 5.3.4.

### Deduction Path Analysis

We represent the results of deduction paths in this section, which is an important evaluation of if the model captures the interdependencies of labels. A path is considered as correct when it equals to or belongs to a golden deduction path, and we report Example-based Precision, Recall and F1 based on BERT. As shown in Table 4, BERT can achieve high recall but low precision on the deduction paths, which means that it tends to predict more labels as correct. This is because pretrained models only take the literal tokens of labels as input without any label structure information. On the contrary, RLHR, which incorporates the label hierarchy, can provide more accurate predictions of deduction paths with much higher precision on all the three datasets.

\begin{table}
\begin{tabular}{c|c c c|c c c} \hline \multirow{2}{*}{Dataset} & \multicolumn{3}{c|}{BERT} & \multicolumn{3}{c}{BERT+RLHR} \\ \cline{2-7}  & P & R & F1 & P & R & F1 \\ \hline Yelp & 17.17 & 72.54 & 26.03 & 38.04 & 52.61 & 40.27 \\ WOS & 33.25 & 77.57 & 44.35 & 47.34 & 66.51 & 53.28 \\ QCD & 18.43 & 58.37 & 26.68 & 22.55 & 57.11 & 30.71 \\ \hline \end{tabular}
\end{table}
Table 4: Performance on deduction paths. P, R, F1 denote Example-based Precision, Recall and F1.

 



\begin{table}
\begin{tabular}{c|l|r r l|r l|r l|r l} \hline \multirow{2}{*}{Method} & \multirow{2}{*}{Setting} & \multicolumn{4}{c|}{Yelp} & \multicolumn{3}{c|}{WOS} & \multicolumn{3}{c}{QCD} \\ \cline{5-10}  & & Ma-F & Mi-F & EBF & Ma-F & Mi-F & EBF & Ma-F & Mi-F & EBF \\ \hline DistilBERT & ZS & 41.42 & 40.33 & \multirow{2}{*}{30.44} & 70.69 & 65.19 & \multirow{2}{*}{55.18} & 23.68 & 24.95 & \multirow{2}{*}{33.57} \\  & GZS & 21.29 & 28.18 & & 68.03 & 63.64 & & 24.43 & 34.29 & \\ \hline DistilBERT & ZS & 41.88 & 41.00 & \multirow{2}{*}{30.61} & 67.81 & 66.45 & \multirow{2}{*}{53.13} & 21.13 & 29.29 & \multirow{2}{*}{34.35} \\   +Hie-Infe & GZS & 21.49 & 28.36 & & 65.65 & 64.05 & & 23.91 & 35.12 & \\ \hline DistilBERT & ZS & 41.49 & 40.32 & \multirow{2}{*}{30.47} & 70.69 & 65.19 & \multirow{2}{*}{56.54} & 23.81 & 24.7 & \multirow{2}{*}{33.34} \\   +Rollback & GZS & 21.28 & 28.18 & & 68.44 & 63.31 & & 24.36 & 33.99 & \\ \hline \hline DistilBERT+RLHR & ZS & 42.16 & 43.87 & \multirow{2}{*}{40.85} & 74.56 & 72.44 & \multirow{2}{*}{61.06} & 24.58 & 27.79 & \multirow{2}{*}{37.46} \\  & GZS & 26.95 & 40.43 & & 71.65 & 68.05 & & 26.10 & 38.73 & \\ \hline DistilBERT+RLHR & ZS & 39.48 & 41.65 & \multirow{2}{*}{40.65} & 63.61 & 64.21 & \multirow{2}{*}{53.39} & 20.18 & **29.68** & \multirow{2}{*}{**38.13**} \\   +Hie-Infe & GZS & 26.79 & 40.44 & & 62.63 & 64.05 & & 24.98 & **39.44** & **38.13** \\ \hline DistilBERT+RLHR & ZS & **42.27** & **43.91** & \multirow{2}{*}{**41.03**} & **74.56** & **72.44** & \multirow{2}{*}{**65.64**} & **24.89** & 28.34 & \multirow{2}{*}{37.45} \\   +Rollback & GZS & **26.97** & **40.55** & & **73.14** & **71.48** & & **26.17** & 38.68 & \\ \hline \end{tabular}
\end{table}
Table 6: Results and comparisons of our matching-score-based rollback algorithm on DistilBERT. Ma-F, Mi-F, EBF, Err denote Macro-F1, Micro-F1, Example-based F1 and logical error rate respectively, and ZS, GZS denote zero-shot setting and generalized zero-shot setting. Bold figures indicate the best results for each metric.

Figure 4: Influence of \(\lambda\) on RLHR approach with DistilBERT. Err, Ma-F and Mi-F denote logical error rate, Macro-F1 and Micro-F1 respectively.

\begin{table}
\begin{tabular}{l p{142.3pt} p{142.3pt}} \hline Dataset & Text & Labels \\ \hline \multirow{4}{*}{Yelp} & Mini donuts at it’s finest. I was there on Saturday and it was absolutely delicious. I had a mini six pack of D O's. I would highly recommend this place for a sweet snack. Five thumbs up. & _Food_, _Restaurants_, _Dounts_, _Food Stands_ \\ \hline WOS & This paper presents the design and experimental evaluation of discrete time sliding mode controller using multirate output feedback to minimize structural vibration of a cantilever beam using shape memory alloy wires as control actuators and piezoercamics as sensor and disturbance actuator. Linear dynamic models of the smart cantilever beam are obtained using online recursive least square parameter estimation. A digital control system that consists of Simulink (TM) modeling software and dSPACE DS1104 controller board is used for identification and control. The effectiveness of the controller is shown through simulation and experimentation by exciting the structure at resonance.