Zero-shot Text Classification via Reinforced Self-training (2020)

Abstract: Zero-shot learning has been a tough problem since no labeled data is available for unseen classes during training, especially for classes with low similarity. In this situation, transferring from seen classes to unseen classes is extremely hard. To tackle this problem, in this paper we propose a self-training based method to efficiently leverage unlabeled data. Traditional self-training methods use fixed heuristics to select instances from unlabeled data, whose performance varies among different datasets. We propose a reinforcement learning framework to learn data selection strategy automatically and provide more reliable selection. Experimental results on both benchmarks and a real-world e-commerce dataset show that our approach significantly outperforms previous methods in zero-shot text classification

## introduction zero-shot learning (zsl) is a challenging task as no labeled data is available for unseen classes during training. there are extensive works proposed in zero-shot image classification task. the main focus of these works is how to transfer knowledge from seen classes to unseen classes. to associate unseen classes with seen classes, they usually resort to semantic information such as visual attributes @xcite , word embeddings of class names @xcite and class hierarchy @xcite . for example, if the model has not seen any instances of "humpback whale" in the training stage, it could still make predictions at testing stage since "humpback whale" is semantically close to "killer whale" and "blue whale" in the seen class set * , so the model is capable of transferring knowledge from seen classes to unseen classes. these methods assume that semantically similar classes share similar image features, however, they may fail in the cases where classes share low similarities. this problem becomes even more salient in typical nlp tasks such as text classification. for example, let us consider a 10-class emotion classification task @xcite , in which the model is trained on class "sadness" while makes predictions on instances from class "joy". notice that most emotions are relatively independent, which means the way we express certain emotion is pretty different from other emotions. as a result, for an unseen class we can hardly find a similar class in the seen class set. transferring from seen classes to unseen classes can be extremely hard as matching patterns that can be shared among classes are rare. essentially, zsl methods aim to learn a matching model between feature space and semantic space, which refers to text and label in text classification task respectively. matching patterns between text and label can be roughly classified as class-invariant patterns and class-specific ones. the former refers to the patterns that are shared among classes, while the latter is dependent on a certain class. there are mainly two ways to deal with this troublesome zero-shot learning situation, including (1) integrating more external knowledge to label sentence fear 1. one day, when i realized that i was alone, i felt fear of loneliness. guilty 2. i felt guilty when i lied to my parents. guilty 3. i wished secretly and lied to a friend because i didn't want her to stay in my house. it is straightforward to consider that if we add sentence 2 to training set, the model is capable of learning class-specific pattern as sentence 2 and sentence 3 share the intra-class similarity. in this way, we can mine class-specific feature through class-invariant feature. however, directly applying traditional selftraining method to zero-shot learning may encounter some problems: (1) traditional selftraining methods use manually designed heuristics to select data, so manual adjustment of selection strategy is costly @xcite . (2) due to the severe domain shift @xcite , traditional self-training method may not provide reliable selection. to alleviate these problems, we present a reinforcement learning framework to learn data selection policy, which can select unlabeled data automatically and provide more reliable selection. the contributions of our work can be summarized as follows: ## conclusion in this paper, we propose a reinforced self-training framework for zero-shot text classification. to realize the transferring between classes with low similarity, our method essentially turns a zero-shot learning problem into a semi-supervised learning problem. in this way, our approach could leverage unlabeled data and alleviate the domain shift between seen classes and unseen classes. beyond that, we use reinforcement learning to learn data selection policy automatically, thus obviating the need to manual adjustment. experimental results on both benchmarks and real-world e-commerce dataset demonstrate the effectiveness of the integration of unlabeled data and the reinforced data selection policy.