Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discovery (2023)

Abstract: Generalized intent discovery aims to extend a closed-set in-domain intent classifier to an open-world intent set including in-domain and out-of-domain intents. The key challenges lie in pseudo label disambiguation and representation learning. Previous methods suffer from a coupling of pseudo label disambiguation and representation learning, that is, the reliability of pseudo labels relies on representation learning, and representation learning is restricted by pseudo labels in turn. In this paper, we propose a decoupled prototype learning framework (DPL) to decouple pseudo label disambiguation and representation learning. Specifically, we firstly introduce prototypical contrastive representation learning (PCL) to get discriminative representations. And then we adopt a prototype-based label disambiguation method (PLD) to obtain pseudo labels. We theoretically prove that PCL and PLD work in a collaborative fashion and facilitate pseudo label disambiguation. Experiments and analysis on three benchmark datasets show the effectiveness of our method.

## introduction intent classification (ic) is an important component of task-oriented dialogue (tod) systems. traditional intent classification models are based on a closed-set hypothesis @xcite . that is, they rely on a pre-defined intent set provided by domain experts and can only recognize limited in-domain (ind) intent categories. however, users may input out-of-domain (ood) queries in the real open world. ood intent detection @xcite @xcite ) identifies whether a user query falls outside the range of pre-defined ind intent set. further, ood intent discovery task @xcite ## theoretical analysis in this section, we provide a theoretical explanation of why prototypical contrastive representation learning can learn cluster-friendly intent representations and class prototypes that facilitate pseudo label disambiguation. pcl essentially draws similar samples towards the same prototype, and forms compact clusters in the representation space, which is consistent with the goal of clustering, so we will explain it from the perspective of em algorithm. as defined in section 2.1, we have n labeled ind samples and m unlabeled ood samples. in the gid task, our goal is to find suitable network parameters to maximize the log-likelihood function as follows: e-step in the supervised learning setting, it is easy to estimate the likelihood probability using ground-truth labels. however, in the gid task, we not only have labeled ind samples, but also have a large number of unlabeled ood samples, so we need to associate each sample with an implicit variable j, @xmath0, 2, ..., n + m (j represents the intent category). in addition, this likelihood function is hard to be directly optimized, so we need to introduce a probability density function q i (j) to represent the probability that sample x i belongs to intent category j. finally, we can use jensen's inequality to derive the lower bound of the maximum likelihood function as follows (we leave detailed derivation process in appendix c): ## related work generalized intent discovery existing intent classification models have little to offer in an openworld setting, in which many new intent categories are not pre-defined and no labeled data is available. these models can only recognize limited in-domain (ind) intent categories. lin and xu (2019); xu et al. (2020) propose the ood intent detection task to identify whether a user query falls outside the range of a pre-defined intent set. further, ood intent discovery task (also known as new intent discovery) @xcite ) is proposed to cluster unlabeled ood data. @xcite proposes the generalized intent discovery (gid) task, which aims to simultaneously classify a set of labeled ind intents while discovering and recognizing new unlabeled ood types incrementally. prototype-based learning prototype-based metric learning methods have been promising approaches in many applications. @xcite first proposes prototypical networks (protonet) which introduces prototypes into deep learning. specifically, protonet calculates prototype vectors by taking the average of instance vectors and makes predictions by metric-based comparisons between prototypes and query instances. @xcite proposes self-supervised prototype representation learning by using prototypes as latent variables. learning good representations also helps weakly supervised learning tasks, including noisy label learning @xcite , semi-supervised learning @xcite , partial label learning @xcite , etc. inspired by these methods, we propose a decoupled prototype learning framework (dpl) to decouple pseudo label disambiguation and representation learning for gid. ## conclusion in this paper, we propose a decoupled prototype learning (dpl) framework for generalized intent discovery. we introduce prototypical contrastive representation learning and prototype-based label disambiguation method to decouple representation learning and pseudo label disambiguation. theoretical analysis and extensive experiments prove that our method can learn discriminative intent representations and prototypes, which facilitates pseudo label disambiguation. we will explore broader applications of dpl method in the future.