HSCNN : A Hybrid- S iamese Convolutional Neural Network for Extremely Imbalanced Multi-label Text Classification (2020)

Abstract: The data imbalance problem is a crucial issue for the multi-label text classification. Some existing works tackle it by proposing imbalanced loss objectives instead of the vanilla cross-entropy loss, but their performances remain limited in the cases of extremely imbalanced data. We propose a hybrid solution which adapts general networks for the head categories, and few-shot techniques for the tail categories. We propose a Hybrid-Siamese Convolutional Neural Network (HSCNN) with additional technical attributes, i.e., a multi-task architecture based on Single and Siamese networks; a category-specific similarity in the Siamese structure; a specific sampling method for training HSCNN. The results using two benchmark datasets and three loss objectives show that our method can improve the performance of Single networks with diverse loss objectives on the tail or entire categories.

## introduction the data imbalance problem is a crucial issue for the multi-label text classification. in many corpora for the classification tasks, the number of instances of a category follows the long tail distribution, where many tail categories has only a small number of instances. to handle this problem, some works sample hard examples for training @xcite ; some works address the problem by proposing imbalance loss objectives, e.g., weighted cross-entropy loss and focal loss @xcite , in place of the vanilla cross-entropy loss @xcite . although the imbalanced loss objectives are better than the vanilla one, their performances remain limited in the cases of extremely imbalanced data because they are not designed for it, i.e., tail (head) categories have extremely small (large) numbers of instances. on the one hand, the recent few-shot learning techniques (e.g., optimization-based methods @xcite @xcite , metric-based methods @xcite @xcite ) have become popular for various nlp tasks @xcite . they have already shown the capability for few-shot classifications. they thus may also perform well for tail category classification (some of the tail categories are few-shot, all have relatively small numbers of instances). on the other hand, for the head categories with many instances, general approaches such as the single cnn model @xcite may be more effective in terms of performance and more efficient in terms of complexity. therefore, our basic idea for tackling the problem of extremely imbalanced multi-label text classification is a hybrid solution that adapts a general approach (i.e., a single network) for head categories and a few-shot approach for tail categories, so that we can take the advantages of both of them. for the few-shot approach, we select the siamese network @xcite because it is easier to integrate with different single networks. a naïve solution is training them separately and utilizing their results on head categories and tail categories respectively as the combined classification results. to make the hybrid solution effective, rather than a naïve combination on the results of two types of networks, we propose a hybrid-siamese convolutional neural network (hscnn) with additional technical properties. first, it is based on a multi-task architecture to deal with forgetting and overfitting problems when training the siamese network. second, the single similarity output of the vanilla siamese structure is limited for estimating the similarities for a large number of categories and multiple categories; we thus propose a categoryspecific similarity in the siamese structure. third, we propose a specific sampling method to train the hscnn. the results using two benchmark datasets and three loss objectives (including one vanilla crossentropy loss and two imbalanced losses) show that the proposed method can improve the performance of a single network with diverse loss objectives on the tail categories and the entire categories. the main contributions of this paper can be summarized as follows. (1). we propose a hybrid method based on general and few-shot techniques to mitigate the extremely imbalanced multi-label text classification problem. (2). we propose a novel hscnn model based on a multi-task architecture, a category-specific similarity, and a specific sampling method. (3). our approach can be integrated with the imbalanced loss objectives to improve the performance; the hybrid-siamese architecture can extend to incorporate with other types of single networks rather than the single cnn network. ## our approach we denote the data as @xmath0, the category set as c and a category as c. the number of training instances is n . the number of training instances of a category c is n c . ## conclusion in this paper, we propose a hybrid solution with a hscnn model for dealing with extremely imbalanced multi-label text classification. the proposed method can improve the performance of single networks with diverse loss objectives on the tail categories or entire categories. in future work, we will try other types of single networks (e.g., @xcite @xcite ).