Multi-Label Few-Shot Learning for Aspect Category Detection (2021)

Abstract: Aspect category detection (ACD) in sentiment analysis aims to identify the aspect categories mentioned in a sentence. In this paper, we formulate ACD in the few-shot learning scenario. However, existing few-shot learning approaches mainly focus on single-label predictions. These methods can not work well for the ACD task since a sentence may contain multiple aspect categories. Therefore, we propose a multi-label few-shot learning method based on the prototypical network. To alleviate the noise, we design two effective attention mechanisms. The support-set attention aims to extract better prototypes by removing irrelevant aspects. The query-set attention computes multiple prototype-specific representations for each query instance, which are then used to compute accurate distances with the corresponding prototypes. To achieve multi-label inference, we further learn a dynamic threshold per instance by a policy network. Extensive experimental results on three datasets demonstrate that the proposed method significantly outperforms strong baselines.

## introduction aspect category detection (acd) @xcite is an important task in sentiment analysis. it aims to identify the aspect categories mentioned in a given sentence from a predefined set of aspect categories. for example, in the sentence "the cheesecake is tasty and the staffs are friendly", two aspect categories, i.e. food and service, are mentioned. the performance of existing approaches for the acd task @xcite @xcite relies heavily on the scale of the labeled dataset. they usually suffer from limited data and fail to generalize well to novel aspect categories with only a few labeled * shiwan zhao is the corresponding author. † the work was (partially) done in ibm. 2) the owners are extremely smart and worldly 3) not a typical customer service response, especially from the owner ! 1) i think the salon has problems starting with the owner. 2) the owner is very nice. 1) cleanliness was great, and the food was really good. 2) people have mentioned, bed bugs on yelp !! figure 1 : example meta-task in a 3-way 2-shot scenario. the words in gray background describe the target aspects of interest, while the words marked by the rectangle are irrelevant aspects, which tend to be noise for this meta-task. few-shot learning (fsl) provides a solution to address the above challenges. fsl learns like a human, identifying novel classes with limited supervised information by exploiting prior knowledge. many efforts have been devoted to fsl @xcite @xcite @xcite . among these methods, the prototypical network @xcite is a promising approach, which is simple but effective. it follows the meta-learning paradigm by building a collection of n -way kshot meta-tasks. a meta-task aims to infer a query set with the help of a small labeled support set. it first learns a prototype for each class in the support set. then the query instance is predicted by measuring the distance with n prototypes in the embedding space. in this paper, we formulate acd in the fsl scenario, which aims to detect aspect categories accurately with limited training instances. however, acd is a multi-label classification problem since a sentence may contain multiple aspect categories. most fsl works learn a single-label classifier and can not work well to address the acd task. the reasons are two-fold. firstly, the sentences of each class (i.e., aspect category) in the support set are diverse and contain noise from irrelevant aspects. as displayed in figure 1 , there are three classes in the support set, and each class has two instances. the aspect categories food and salon tend to be noise for this meta-task, making it hard to learn a good prototype for each class in the support set. secondly, the query set is also noisy. figure 1 demonstrates three different cases. the first sentence mentions two aspects hotel and room cleanliness out of the support set. we need to detect both aspects accurately as multi-label classification. when detecting each of them, the other aspect acts as noise and makes the task hard. the second sentence is an easy case with a single aspect staff owner. the third sentence mentions the aspect staff owner out of the support set, while the aspect service is noise for this meta-task. in summary, the noise from both the support set and query set makes the few-shot acd a challenging task. to this end, we propose a multi-label fsl method based on the prototypical network @xcite . we alleviate the noise in the support set and query set by two effective attention mechanisms. concretely, the support-set attention tries to extract the common aspect of each class. by removing the noise (i.e., irrelevant aspects), the supportset attention can yield better prototypes. then for a query instance, the query-set attention utilizes the prototypes to compute multiple prototype-specific query representations, in which the irrelevant aspects are removed. given the better prototypes and the corresponding prototype-specific query representations, we can compute accurate distances between the query instance and the prototypes in the embedding space. we detect the aspect categories in the query instance by ranking the distances. to select the positive aspects from the ranking, we design a policy network @xcite to learn a dynamic threshold for each instance. the threshold is modeled as the action of the policy network with continuous action space. the main contributions of our work are as follows: few-shot learning few-shot learning (fsl) @xcite is close to real artificial intelligence, which borrows the learning process from the human. by incorporating the prior knowledge, it obtains new knowledge fast with limited supervised information. many works have been proposed for fsl, which can be mainly divided into four research directions. one promising direction is distance-based methods. these methods measure the distance between instances in the feature embedding space. the siamese network @xcite infers the similarity score between an instance pair. others compare the cosine similarity @xcite or euclidean distance @xcite . the relation network @xcite exploits a neural network to learn the distance metric. afterward, sa e query softmax label of query mse loss 1 1 0 0.5 0.5 0 normalize support-set attention sa e sa e e qa qa qa h i 1 support set aspect category 1 aspect category 2 aspect category 3 h q h i 1 h i 2 v i attention matrix w i aspect-wise attention r i 1 r i 2 e sa qa encoder support-set attention query-set attention euclidean distance prototype r i k-shot instances of aspect i common aspect vector garcia and bruna (2018) utilize graph convolution network to extract the structural information of classes. the second direction focuses on the optimization of networks. model-agnostic metalearning (maml) algorithm @xcite learns a good initialization of the model and updates the model by a few labeled examples. meta networks @xcite achieve rapid generalization via fast parameterization. the third type is based on hallucination @xcite . this research line directly deals with data deficiency by "learning to augment", which designs a generator on the base classes and then hallucinates novel class data to augment few-shot samples. the last direction introduces a weight generator to predict classification weight given a few novel class samples, either based on attention mechanism @xcite or gaussian distribution @xcite . a recent work proto-hatt @xcite is similar to ours. proto-hatt is based on the prototypical network @xcite , which deals with the text noise in the relation classification task by employing hybrid attention at both the instancelevel and the feature-level. this method is designed for single-label fsl. compared with it, our method designs two attention mechanisms to alleviate the noise on the support set and query set, respectively. the collaboration of two attentions helps compute accurate distances between the query instance and prototypes, and then improves multi-label fsl. multi-label few-shot learning compared with single-label fsl, the multi-label fsl has been underexplored. previous works focus on image synthesis @xcite and signal processing @xcite . @xcite develop few-shot and zero-shot methods for multilabel text classification when there is a known structure over the label space. their approach relies on label descriptors and the hierarchical structure of the label spaces, which limits its application in practice. @xcite propose to address the multi-label intent detection task in the fsl scenario. it calibrates the threshold by kernel regression. different from this work, we learn a dynamic threshold per instance in a reinforced manner. ## methodology in the few-shot acd scenario, each meta-task contains a support set s and a query set q. the metatask is to assign the query instance to the class(es) of the support set. an instance may be a multiaspect sentence. thus a query sentence may describe more than one class out of the support set foot_0 . therefore, we define the few-shot acd as a multilabel few-shot classification problem.