Contrastive Learning-Enhanced Nearest Neighbor Mechanism for Multi-Label Text Classification (2022)

Abstract: Multi-Label Text Classification (MLTC) is a fundamental and challenging task in natural language processing. Previous studies mainly focus on learning text representation and modeling label correlation but neglect the rich knowledge from the existing similar instances when predicting labels of a specific text. To make up for this oversight, we propose a k nearest neighbor (kNN) mechanism which retrieves several neighbor instances and interpolates the model output with their labels. Moreover, we design a multi-label contrastive learning objective that makes the model aware of the kNN classification process and improves the quality of the retrieved neighbors while inference. Extensive experiments show that our method can bring consistent and significant performance improvement to multiple MLTC models including the state-of-the-art pretrained and non-pretrained ones.

## introduction multi-label text classification (mltc) is a fundamental task in natural language processing, which can be found in many real-world scenarios such as web page tagging @xcite , topic recognition @xcite , sentiment analysis @xcite and so on. different from multi-class classification where only one label is identified as positive, mltc aims to assign multiple labels from a predefined set to each text. till now, extensive research has been carried out to solve the mltc task. among them, some methods focus on learning enhanced text representation with deep neural networks @xcite or the label-wise attention mechanism @xcite . meanwhile, others try to model the label correlation by the sequential prediction @xcite , iterative reasoning @xcite , or graph neural networks @xcite . ## proposed method in this section, we introduce our proposed method in detail. as depicted in fig. 1 , we design a k nearest neighbor mechanism for mltc (step 2, 3) and enhance it by training the model with a multilabel contrastive learning objective (step 1). ## experiments in this section, we conduct multiple experiments to evaluate the efficacy of our method. implementa-tion details and the overhead of our method can be found in appendix a and b respectively. ## conclusion in this paper, we proposed a k nearest neighbor mechanism along with a multi-label contrastive learning objective for mltc. extensive experiments verified the effectiveness of our method and revealed the source of performance improvements our method brings. for future work, we will explore how to improve the performance of mltc models directly with contrastive learning. tab. 8. it's worth noting that the extra inference time brought by our method does not exceed 5ms in all cases.