Causal Denoising Prototypical Network for Few-Shot Multi-label Aspect Category Detection (2025)

Abstract: The multi-label aspect category detection (MACD) task has attracted great attention in sentiment analysis. Many recent methods have formulated the MACD task by learning robust prototypes to represent categories with limited support samples. However, few of them address the noise categories in the support set that hinder their models from effective prototype generations. To this end, we propose a causal denoising prototypical network (CDPN) for few-shot MACD. We reveal the underlying relation between causal inference and contrastive learning, and present causal contrastive learning (CCL) using discrete and continuous noise as negative samples. We empirically found that CCL can (1) prevent models from overly predicting more categories and (2) mitigate semantic ambiguity issues among categories. Experimental results show that CDPN outperforms competitive baselines. Our code is available online.

## introduction with the rapid growth of online services, multilabel aspect category detection (macd) that detects aspect categories within a given sentence assigned multiple categories @xcite @xcite , has gained considerable attention. due to the success of few-shot learning @xcite @xcite and metalearning (ml) @xcite , @xcite have formulated macd task with a meta-task which is illustrated in fig. 1 . it aims to predict the class label of the query set by leveraging the support set. thereafter, @xcite and @xcite propose labeldriven prototype denoising models. @xcite employ sample-set operations based on ml for macd tasks. @xcite utilize an ml-based framework with variational distribution figure 1 : an example of a 3-way 2-shot meta-task. the colored boxes denote the target aspect categories, and the gray boxes refer to words that cause noisy categories. the causal interference mechanism has been widely utilized to reduce the negative impact of intermediate factors (e.g., noise or spurious correlations) @xcite @xcite @xcite . as the casual graph shown in fig. 2 (a), the treatment variable x directly affects the response variable y and indirectly affects y through n , where n refers to the undesired factor. they mitigate the impact of n by using counterfactual intervention to force that @xmath0, where y and η are realizations of y and n . however, few approaches utilize causal interference to denoise the representations of support samples for macd tasks. several recent approaches have utilized contrastive learning (cl) @xcite to push prototypes of various categories away, achieving remarkable results. they regard the representations of other categories as negative pairs for contrast. however, they pay no attention to integrating causal inference and cl, although there exist underlying yet strong relations between them, in the sense that (1) the noise-free state of @xmath1; (2) the way to reinforce the cohesion between positive pairs and diversity between negative pairs in cl can be utilized to predict y for causal inference @xcite . motivated by the observations, we propose a causal denoising prototypical network (cdpn) for few-shot macd. specifically, we integrate causal inference with cl to effectively denoise the representation of support samples. different from related works that regard the representations of other categories as negative pairs @xcite , our approach utilizes the noise representations extracted from support samples for causal contrastive learning (ccl). moreover, we introduce discrete and continuous noise as negative samples for ccl. the discrete noise is generated by a negative-label-driven attention mechanism, and the continuous noise is obtained by applying dropout at a substantial rate on the representations of support samples. we empirically found that discrete noise representation facilitates better convergence by incorporating multiple negative labels. conversely, the advantage of continuous noise representation lies in its ability to be obtained without requiring negative labels. the main contributions of our work can be summarized as follows: (1) we highlight the underlying yet strong relations between causal inference and contrastive learning for macd tasks and thus propose a novel causal contrastive learning; (2) we explore discrete and continuous noise as negative samples for ccl to denoise the representations of support samples; this enables ccl to work well without negative labels and perform even better with them; (3) extensive experiments show that cdpn outperforms sota baselines in the macd task. we discuss its efficacy compared to some llms (i.e., llama2-7b, llama3-8b, gpt-3.5-turbo and gpt-4o-mini). we also found that these llms tend to overly infer more aspect categories, while ccl can prevent our cdpn from the issue. ## methodology fig. 3 illustrates the framework of our cdpn, which consists of (i) prototypical representation learning and (ii) causal representation denoising. ## conclusion we proposed a concise task-specific cdpn model for the few-shot macd tasks. we integrated causal inference and contrastive learning to eliminate the undesired impact of noise categories in the support set. we explored discrete and continuous noise to build negative pairs for ccl. extensive experiments have demonstrated that our cdpn achieves sota performance. in future work, we intend to (i) generate more counterfacts with llms for enhancing cdpn and (ii) examine our ccl aspect sentiment triplet extraction (aste) @xcite and opinion aspect target sentiment quadruple extraction (oats) @xcite .