Enhancing Multi-Label Text Classification under Label-Dependent Noise: A Label-Specific Denoising Framework (2024)

Abstract: Recent advancements in noisy multi-label text classification have primarily relied on the class-conditional noise (CCN) assumption, which treats each label independently undergoing label flipping to generate noisy labels. However, in real-world scenarios, noisy labels often exhibit dependencies with true labels. In this study, we validate through hypothesis testing that real-world datasets are unlikely to adhere to the CCN assumption, indicating that label noise is dependent on the labels. To address this, we introduce a label-specific denoising framework designed to counteract label-dependent noise. The framework initially presents a holistic selection metric that evaluates noisy labels by concurrently considering loss information, ranking information, and feature centroid. Subsequently, it identifies and corrects noisy labels individually for each label category in a fine-grained manner. Extensive experiments on benchmark datasets demonstrate the effectiveness of our method under both synthetic and real-world noise conditions, significantly improving performance over existing state-of-the-art models.

## introduction multi-label text classification (mltc) aims to predict the most relevant labels for each text from a label set. in real applications, noise is inevitably present in the data of mltc @xcite . it poses a significant challenge for machine learning models, particularly deep learning models @xcite . in noisy multi-label classification, most existing methods rely on the class-conditional noise (ccn) assumption @xcite @xcite . this assumption posits that label noise originates from independent label flipping for each * corresponding author. category, with each category having a distinct flipping probability. however, in practice, noisy labels often exhibit a certain degree of correlation with the true labels @xcite . as shown in figure 1 , due to category ambiguity, lack of expert knowledge, or the influence of attention shift @xcite , annotators are more likely to mislabel the current document as "moon" or "earth" instead of unrelated labels such as "pytorch" or "tensorflow". in this paper, our first contribution (section 2) is to propose a theoretical hypothesis test on the realworld dataset riedel @xcite to demonstrate that real-world multi-label noise is less likely to be ccn, and more likely to be label-dependent noise (ldn), where the occurrence of label noise depends on the positive labels associated with the current sample. to mitigate noisy multi-label text classification under ldn, our second contribution (section 2) is to introduce for generating controllable ldn and analyze the characteristics of training under ldn. our third contribution (section 3) is the proposal of a label-specific denoising (led) framework to address ldn. led initially introduces a holistic selection metric (hsm), incor-porating ranking-enhanced loss (rel) and centroid distance (cd) to assess labels from both output and feature perspectives. following this, led employs the hsm to conduct label-specific noise identification and correction for each label category. the superior performance of led is verified in extensive experiments, under ldn with varying noise fractions, including on real-world benchmarks. ## method previous work @xcite @xcite has identified noisy labels based on the "memorization effect", using the loss values from early epochs of deep learning models. however, as shown in section 2, this approach faces challenges with ldn due to its similarity to the true labels. therefore, in this section, we propose a label-specific denoising (led) framework. this framework considers various aspects of neural net-work training, including loss, ranking, and feature space neighbors, which provide a more comprehensive reflection of the likelihood that a label is noisy. specifically, we introduce a holistic selection metric (hsm) that includes ranking-enhanced loss (rel) and centroid distance (cd). we then identify noisy labels for each label category from the perspective of each individual label category. we use a gaussian mixture model (gmm) to identify noisy labels among positive and negative labels for each category, resulting in a noise probability for each label. based on these noise probabilities, we refine the original labels in a fine-grained manner. the corrected labels are then used for retraining the model. the overall framework is shown in figure 4 . ## conclusions in this paper, we first verify that real-world datasets often deviate from the class-conditional noise assumption. based on this observation, we introduce label-dependent noise (ldn), revealing the characteristics of label-dependent noise and designing a method to generate controllable ldn. subsequently, we propose a novel label-specific denoising framework to enhance multi-label text classification under label-dependent noise. extensive experiments on benchmark datasets demonstrate that our method significantly improves performance under both synthetic and real-world noise conditions, outperforming existing state-of-the-art models.