Noisy Multi-Label Text Classification via Instance-Label Pair Correction (2024)

Abstract: In noisy label learning, instance selection based on small-loss criteria has been proven to be highly effective. However, in the case of noisy multi-label text classification (NMLTC), the presence of noise is not limited to the instance-level but extends to the (instance-label) pair-level.This gives rise to two main challenges.(1) The loss information at the pair-level fails to capture the variations between instances. (2) There are two types of noise at the pair-level: false positives and false negatives. Identifying false negatives from a large pool of negative pairs presents an exceedingly difficult task. To tackle these issues, we propose a novel approach called instance-label pair correction (iLaCo), which aims to address the problem of noisy pair selection and correction in NMLTC tasks.Specifically, we first introduce a holistic selection metric that identifies noisy pairs by simultaneously considering global loss information and instance-specific ranking information.Secondly, we employ a filter guided by label correlation to focus exclusively on negative pairs with label relevance. This filter significantly reduces the difficulty of identifying false negatives.Experimental analysis indicates that our framework effectively corrects noisy pairs in NMLTC datasets, leading to a significant improvement in model performance.

## introduction multi-label text classification (mltc) aims to predict the most relevant labels for each text from a label set. in real applications, noise is inevitably present in the data of mltc @xcite . it poses a significant challenge for machine learning models, particularly deep learning models @xcite . when dealing with learning from noisy labels (lnl), one effective approach to mitigate the impact of noisy data is to identify * corresponding author. a clean subset through sample (instance) selection @xcite . by removing data with noisy labels, we can reduce the influence of mislabeled data and improve the learning process. the small-loss criteria is a popular method @xcite @xcite , assuming that samples with lower loss values are likely to be clean. this is based on the observation that deep networks initially learn simple patterns and later tend to overfit to the noisy patterns @xcite @xcite . however, what distinguishes noisy multi-label text classification (nmltc) from typical lnl is that in nmltc, noise occurs not at the instancelevel but at the (instance-label) pair-level, as depicted in figure 1 . in this scenario, using the small-loss criterion for selection encounters two challenges. (1) the pair-level loss information is global and instance-independent, meaning it does not capture the distinctions between each individual instance. (2) nmltc exhibits two types of noise: false positives (fp) and false negatives (fn). specifically, due to the abundance of true negative (tn) pairs in nmltc, it becomes challenging to identify the fn noise from the large pool of negative pairs. to address these challenges, we propose a novel instance-label pair correction (ilaco) framework, aiming to achieve pair selection and correction for noisy multi-label text classification tasks. the framework introduces three key components: holistic selection metric, negative pair filter, and cocorrection. we first designed a holistic selection metric (hsm) to assess the learning difficulty of instancelabel pairs. our hsm consists of both a loss-based metric and a rank-based metric. the loss-based metric captures the global learning difficulty of instance-label pair by considering the associated loss values. a lower loss indicates a lower learning difficulty for the corresponding instance-label pair. on the other hand, the rank-based metric reflects the instance-specific learning difficulty of instance-label pair. in mltc, predictions rely heavily on the label rank, with lower ranks indicating better memorization of the instance-label pair by the model @xcite . by incorporating the rank-based metric, we can prevent some instance-label pairs with high loss values but correctly predicted pairs from being classified as corrupted pairs. to ensure stability and consistency, we adopt a time-consistent approach (historical average) @xcite , to reflect the learning difficulty of each instance-label pair throughout the entire training process. to address the challenge posed by the abundance of negative pairs, as depicted in figure 1 , we first observe a strong label correlation between fn labels (i.e., pluto) and true positive (tp) labels (i.e., astronomy and neptune) for the same instance. as a result, we adopt a filter guided by instancespecific label correlation, focusing on negative pairs that exhibit a certain level of label correlation with the positive pairs of each instance. by doing so, we significantly reduce the number of candidate negative pairs for each instance, thereby reducing the difficulty of identifying fn pairs within the negative pairs. in the final stage, we perform noise estimation and label correction simultaneously for both positive and negative pairs. after obtaining the hsm for positive and negative pairs, we utilize a gaussian mixture model (gmm) to estimate the noise rates. we then employ a pseudo-labeling approach that combines both soft and hard correction strategies for label correction. it is important to note that in multi-class tasks, after identifying noisy labels, the common practice is to utilize sample selection or sample weighting methods @xcite . however, in the case of instance-label pairs being a binary classification problem, we can directly flip the labels to correct them. our contributions can be summarized into four key aspects: (1) we propose the instance-label pair correction (ilaco) approach, which successfully applies the memorization effect to multi-label text classification for noise reduction. (2) we introduce a holistic selection metric (hsm) that combines the global information of the training process (loss) with the instance-specific information (rank). hsm provides a better reflection of the difficulty in memorizing instance-label pairs. (3) we devise a label correlation-based negative pair filter, which enhances the recognition of false negative pairs by removing most irrelevant true negative pairs through label correlation. (4) the superior performance of ilaco is validated through extensive experiments on three benchmark datasets. ## method the overall architecture of our model is illustrated in figure 2 . we first employ the standard architecture for multi-label text classification to train the model on the noisy dataset. subsequently, based on the historical information obtained during the training process, we utilize our proposed ilaco method to correct the labels and generate pseudo labels. finally, we retrain the multi-label text classification model using the corrected pseudo labels to obtain the final model. the ilaco framework consists of three components: the holistic selection metric (hsm), negative pair filter, and co-correction. we present more details of these components in the following sections. ## related work multi-label text classification the most common approach for addressing multi-label text classification (mltc) is to use the identical document representation to train classification models @xcite . consequently, label-specific feature learning @xcite @xcite , which focuses on capturing the unique characteristics of each label, has shown promise in enhancing label discrimination. some works @xcite have also explored transfer learning from head labels to tail labels to mitigate the adverse effects of label long-tail distribution. recently, there has been growing interest in mltc under noisy settings @xcite . the nem method @xcite ) models the transition process of noisy labels using latent variable models to achieve robust mltc. in this paper, we extend the memorization effect @xcite to noisy mltc for the first time, and propose an instance-label pair correction method. ## conclusions in this paper, we propose a method for instancelabel pair correction that combines the historical loss information and rank information from the training process to identify and correct positive and negative noise in noisy multi-label text classification tasks. our experiments yield compelling results, highlighting the superiority of our model compared to existing state-of-the-art (sota) baselines for multi-label text classification and noisy multi-label classification.