C luster P rompt: Cluster Semantic Enhanced Prompt Learning for New Intent Discovery (2023)

Abstract: The discovery of new intent categories from user utterances is a crucial task in expanding agent skills. The key lies in how to efficiently solicit semantic evidence from utterances and properly transfer knowledge from existing intents to new intents. However, previous methods laid too much emphasis on relations among utterances or clusters for transfer learning, while paying less attention to the usage of semantics. As a result, these methods suffer from in-domain over-fitting and often generate meaningless new intent clusters due to data distortion. In this paper, we present a novel approach called Cluster Semantic Enhanced Prompt Learning (CsePL) for discovering new intents. Our method leverages two-level contrastive learning with label semantic alignment to learn meaningful representations of intent clusters. These learned intent representations are then utilized as soft prompt initializations for discriminating new intents, reducing the dominance of existing intents. Extensive experiments conducted on three public datasets demonstrate the superiority of our proposed method. It not only outperforms existing methods but also suggests meaningful intent labels and enables early detection of new intents.

## introduction new intent discovery (nid) aims to automatically identify novel intent categories that are not defined or observed beforehand. it plays a critical role in task-oriented dialogue systems with the ability to discern newly emerging user preferences @xcite , thereby providing high-quality services @xcite @xcite . different from the traditional intent classification @xcite @xcite , the key challenges of nid lie in how to properly transfer the prior knowledge from exist- representation space push away known intent unknown intent label semantic pull together in-domain overfitting unknown intent cluster known intent cluster known intent cluster ## 10471 aligning the known intent cluster representations to such stable label semantic features can protect the model from in-domain over-fitting problem and enable the model to give suggestions about intent labels. to achieve this, we also adopt a contrastive learning objective for label semantic alignment. specifically, for each known intent y @xmath0, we first use the bert-based model to get embeddings for tokens in intent label y i . as each intent label may contain multiple tokens, we directly apply mean pooling operation to obtain intent label representation as: as a result, the overall loss function in the icrl stage can be formulated as follows: ## conclusion in this paper, we reemphasized the importance of semantic knowledge in new intent discovery and proposed a cluster semantic enhanced prompt learning (csepl) method. specifically, we designed two-level contrastive learning with label semantic alignment for intent cluster representation learning, and a soft prompting method to leverage the learned intent cluster representation for nid. experimental results on three public datasets demonstrate the effectiveness of the csepl. extensive analyses further show that the csepl not only significantly outperforms the existing baselines, but also implies new intent labels and detects the appearance of the new intents at an early stage.