A Probabilistic Framework for Discovering New Intents
Year: 2023

Abstract:
Discovering new intents is of great significance for establishing the Task-Oriented Dialogue System. Most prevailing approaches either cannot transfer prior knowledge inherent in known intents or fall into the dilemma of forgetting prior knowledge in the follow-up. Furthermore, such approaches fail to thoroughly explore the inherent structure of unlabeled data, thereby failing to capture the fundamental characteristics that define an intent in general sense. In this paper, starting from the intuition that discovering intents should be beneficial for identifying known intents, we propose a probabilistic framework for discovering intents where intent assignments are treated as latent variables. We adopt the Expectation Maximization framework for optimization. Specifically, In the Estep, we conduct intent discovery and explore the intrinsic structure of unlabeled data by the posterior of intent assignments. In the M-step, we alleviate the forgetting of prior knowledge transferred from known intents by optimizing the discrimination of labeled data. Extensive experiments conducted on three challenging real-world datasets demonstrate the generality and effectiveness of the proposed framework and implementation. Codes is publicly available.1

heading: 1 Introduction
text: Unknown intent detection (Zhou et al., 2022) in the Task-Oriented Dialogue System (TODS) has gradually attracted more and more attention from researchers. However, detecting unknown intent is only the first step. For the TODS, intent discovery is crucial but also more challenging. Because the pre-defined intent set in the TODS is limited to cover all intents, the TODS should discover potential new intents automatically during interactions
∗Equal contribution. †Corresponding author.
1https://github.com/zyh190507/Probabilistic-discoverynew-intents
with the users. And as a practical matter, a large number of valuable unlabeled data will be generated within the interaction between users and the dialogue system. Considering the limited labeled corpus and time-consuming annotating, which also requires expertise, the TODS should adaptively discover intents from those unlabeled data with the aid of limited labeled data.
Just as discovering new intents plays a crucial role in establishing the TODS, discovering new intents has raised a lot of research interest just like unknown intent detection. Unsupervised cluster learning is a popular paradigm to solve this problem. Specifically, previous works (Hakkani-Tür et al., 2013, 2015; Shi et al., 2018; Padmasundari, 2018) formulate intent discovery as an unsupervised clustering process. However, these methods mainly focus on constructing pseudo-supervised signals to guide the clustering process while neglecting the prior knowledge embedded in the available labeled data. In real user-facing scenarios, we often possess a small amount of labeled data in advance, which contains prior knowledge that can guide the intent discovery process, and a substantial volume of unlabeled data generated in the interaction with the dialogue system mentioned above, which contains both known intents and unknown intents to be discovered.
How do discover intents in the unlabeled corpus using the labeled data? Recently, the semisupervised methods (Lin et al., 2020; Zhang et al., 2021) have become popular. DeepAligned (Zhang et al., 2021) is the most typical and has also inspired a series of effective works (Shen et al., 2021; Zhang et al., 2022) recently. DeepAligned first generalizes the prior knowledge into the semantic features of unlabeled data by pre-training. Then, to learn cluster-friendly representations, DeepAligned assigns a pseudo label to each unlabeled utterance and re-trains the model under the supervision of those pseudo labels.
3771
Nevertheless, DeepAligned suffers from many problems. Firstly, when the model is re-trained with the pseudo supervision signal, the model will forget the knowledge transferred in the transferring stage, which is demonstrated in Figure 1. Then, the model could be misled by inaccurate pseudo labels, particularly in large-sized intent space (Wang et al., 2021). More importantly, softmax loss formed by pseudo labels cannot explore the intrinsic structure of unlabeled data, so it can not provide accurate clustering supervised signals.
Different from the previous methods, we start from the intuition that the intent discovery should not damage the identification of the known intents. Ideally, the two processes should achieve a win-win situation. The knowledge contained in labeled data corpus (as known intents) can be used to guide the discovery, and the information learned from the unlabeled corpus during discovery could improve the identification of the known intents.
Therefore, with the help of optimizing the identification of labeled data given the whole data corpus, we propose a principled probabilistic framework for intent discovery, where intent assignments as a latent variable. We adopt Expectation Maximization as a principal template for optimizing this typical latent variable model. Specifically, in the E-step, we use the current model to discover intents and calculate a specified posterior probability of intent assignments to explore the intrinsic structure of data. In the M-step, the probability of identification of labeled data including those newly discovered from unlabeled data, and the posterior
probability of intent assignments, which is to help learn friendly-discovery features, are maximized simultaneously to optimize and update model parameters. Extensive experiments conducted in three benchmark datasets demonstrate our method can achieve substantial improvements over strong baselines. Our contributions are as follows:
(Theory) We introduce a principled probabilistic framework for discovering new intents and provide a learning algorithm based on Expectation Maximization. To the best of our knowledge, this is the first complete theoretical framework in this field and we hope it can inspire follow-up research.
(Methodology) We provide an efficient implementation based on the proposed probabilistic framework. After transferring prior knowledge, we use a simple yet effective method to alleviate forgetting. Furthermore, we propose a new contrastive paradigm to explore the intrinsic structure of unlabeled data, which avoids the model shift towards inaccurate pseudo labels but helps to better learn the friendly-discovery features.
(Experiments and Analysis) We conduct extensive experiments and detailed analyses on a suite of real-world datasets to demonstrate the generality and effectiveness of our proposed framework and implementation.

heading: 2 Related Work
text: Our work is mainly related to two lines of research: Unsupervised and Semi-supervised clustering.
Unsupervised Clustering Extracting meaningful information from unlabeled data has been studied for a long time. Traditional approaches like K-means (MacQueen et al., 1967) and Agglomerative Clustering (AC) (Gowda and Krishna, 1978) are seminal but hardly perform well in highdimensional space. Recent efforts are devoted to using the deep neural network to obtain good clustering representations. Xie et al. (2016) propose Deep Embedded Cluster (DEC) to learn and refine the features iteratively by optimizing a clustering objective based on an auxiliary distribution. Unlike DEC, Yang et al. (2017) propose Deep Clustering Network (DCN) that performs nonlinear dimensionality reduction and k-means clustering jointly to learn friendly representation. Chang et al. (2017) (DAC) apply unsupervised clustering to image clustering and proposes a binary-classification framework that uses adaptive learning for optimization. Then, DeepCluster (Caron et al., 2018) proposes
an end-to-end training method that performs cluster assignments and representation learning alternately. However, the key drawback of unsupervised methods is their incapability of taking advantage of prior knowledge to guide the clustering. Semi-supervised Clustering By virtue of a few labeled data, semi-supervised clustering usually produces better results compared with unsupervised counterparts. PCK-Means (Basu et al., 2004) proposes that the clustering can be supervised by pairwise constraints between samples in the dataset. KCL (Hsu et al., 2017) transfers knowledge in the form of pairwise similarity predictions first and learns a clustering network to transfer learning. Along this line, MCL (Hsu et al., 2019) further formulates multi-classification as meta classification that predicts pairwise similarity and generalizes the paradigm to various settings. DTC (Han et al., 2019) extends the DEC algorithm and proposes a mechanism to estimate the number of new images categories using labeled data. When it comes to the field of text clustering, CDAC+ (Lin et al., 2020) combines the pairwise constraints and target distribution to discover new intents while DeepAligned (Zhang et al., 2021) introduces an alignment strategy to improve the clustering consistency. Recently, SCL (Shen et al., 2021) incorporates a strong backbone MPNet in the Siamese Network structure with pairwise contrastive loss to learn the sentence representations. Similarly, MTP (Zhang et al., 2022) enhances sentence representation through multi-task pre-training strategy and extra data. Although these methods take known intents into account, they may suffer from knowledge forgetting during the training process. More importantly, these methods are insufficient in the probe into the intrinsic structure of unlabeled data, making it hard to distinguish the characteristics that form an intent.

heading: 3 Approach

heading: 3.1 Problem Definition
text: Given as input an labeled dataset Dl = {xli, i = 1, . . . , N} where intents Y l = {yli, i = 1, . . . , N} are known and an unlabeled dataset Du = {xui , i = 1, . . . ,M}. Our goal is to produce intent assignments as output by clustering (or partitioning) the whole dataset D, which denotes D = Dl ∪ Du. Directly optimizing the goal is intractable as the lack of knowledge about new intents and the intrinsic structure of unlabeled data. As analyzed
in Section 1, discovering intents should not damage but be beneficial for the identification of known intents, which can be formulated to optimize p(Y l|Dl, D; θ). Since Dl⊂D, the optimization objective can be written as: p(Y l|D; θ).
Denote our latent variable (representing intent assignments obtained by clustering on D) by Z and let ZD be a possible value of Z. Using Bayes rule, p(Y l|D; θ) can be calculated as:
p(Y l|D) = ∑
ZD∈Z p(Y l|ZD, D; θ)p(ZD|D; θ).
(1) Exactly optimizing Eq.(1) is intractable due to its combinatorial nature. Consider a specific value ZD, the log-likelihood can be simplified as:
Lobj = log p(Y l|ZD, D; θ) + log p(ZD|D; θ). (2)
Our goal is get better ZD (i.e.intent discovery) by optimizing Lobj , and a better ZD can also help optimize Lobj .

heading: 3.2 Intent Representation and Transferring Knowledge
text: Before optimizing Lobj , we want to transfer knowledge from the labeled corpus to initialize the model. Transferring knowledge has been widely studied and types of transferred knowledge have been proposed for a variety of circumstances. Considering the excellent generalization of the pre-trained model, we fine-tune BERT (Devlin et al., 2018) with labeled corpus under the supervision of cross entropy as suggested in (Zhang et al., 2021). Given the i-th labeled utterance xi, we first get its contextual embeddings by utilizing BERT and then perform mean-pooling to get sentence semantic representation zi. The objective of fine-tune Lce as:
Lce = − 1
N
N∑
i=1
log exp(ϕ(zi) yi) ∑Kl
j=1 exp(ϕ(zi) j) , (3)
where ϕ(·) represents a linear classifier and ϕ(zi)j denotes the logits of the j-th class, K l denotes the total number of known intents.

heading: 3.3 EM Framework for Optimization
text: Intent Assignments Z (In the following, we omit the subscript D of ZD for clarity.) Specific intent assignments Z involves two components: how to determine K representing how many intents in
dataset D and how to assign the utterance in the dataset to corresponding intent. Many methods (Han et al., 2019; Shen et al., 2021) have been proposed to estimate K. Considering the tradeoff between efficiency and effect, we follow Zhang et al. (2021) (see Appendix D for discussions on more accurate estimating K under our framework) and first set a rough value K (e.g., the multiple of the ground truth number) for K and then refine it by dropping clusters (formed by grouping the dataset D into K semantic clusters using k-means) whose size is less than a certain threshold. After estimating how many intents are contained in the dataset, we perform k-means to assign cluster assignments as (pseudo) intent to each utterance. Next, we discuss in detail how to further optimize Eq.(2) with Expectation-Maximization (EM) algorithm framework.
E-Step We have assigned a specific intent assignment Z to latent variable Z based on prior knowledge. We expect that the intent assignments Z should reflect what characteristics make a good intent in general rather than specific intents. Therefore, the standard cross entropy loss formed by specific pseudo labels adopted by Caron et al. (2018); Zhang et al. (2021) can not achieve this purpose, and even the model may be confused by the false pseudo labels according to Wang et al. (2021). To better reflect the intrinsic structure of dataset D and learn friendly features for intent assignments, we hope that intent assignments Z can make utterances with the same intent close enough and pull utterances with different intents far away in the semantic feature space. Inspired by contrastive learning paradigm, we estimate the posterior p(Z|D; θ):
p(Z|D; θ) = ∏
Ck∈Z p(Ck|D; θ) (4)
= ∏
Ck∈Z
∏
x∈Ck p(x ∈ Ck|D; θ) (5)
∝ ∏
Ck∈Z
∏
x∈Ck
∑ x+∈Ck exp(x · x
+)∑ xp∈D\{x} exp(x · xp) ,
(6)
where Ck is a cluster produced by Z , and x · x+ is calculated by consine between features. To optimize Eq.(2), we also need to compute p(Y l|Z, D; θ). Exactly computing is difficult as the label space in Z does not match that of Y l. Consider the catastrophic forgetting as in Deepaligned
mentioned above, we approximate p(Y l|Z, D; θ):
p(Y l|Z, D; θ) =p(Y l|Z, Dl, Du; θ) (7) ∝ p(Y l|Dl, D̂l(Du,Z); θ) (8)
∝ ∏
x∈Dl∪D̂l
exp(ϕ(x)y) ∑Kl
j=1 exp(ϕ(x) j) ,
(9)
where ϕ(·) denotes same linear classifier as Eq.(3), y denotes the label of x, K l denotes the total number of known intents and Dl denotes labeled data in D. D̂l(Du,Z) refers to the set of samples in Du that can be considered as known intents after the operation of Z .
D̂l = {(x, yl)|x ∈ NZ(xl), (xl, yl) ∈ Dl}, (10)
where xl is the sample from Dl, yl is the label of xl. NZ(xl) is the unlabeled nearest neighbor samples set that belongs to the same cluster (divided by Z) as xl. See Appendix E for specific benefits from D̂l. The labeled data is tailored to model training. On the one hand, the model will not lose the knowledge transferred from labeled data, on the other hand, the model can constantly explore the intrinsic structure of the dataset by utilizing it.
M-Step In the M-step, we update the θ in Eq. (2). In addition to bringing Eq. (4) and Eq. (7) into Eq. (2), we introduce two hyper-parameters to help optimize objectives. The overall loss L can be formulated as follows:
L = λ · ∑
Ck∈Z
∑
x∈Ck log
∑ x+∈Ck exp(
x·x+ τ )∑
xp∈D\{x} exp( x·xp τ )
(11)
+ (1− λ) · ∑
x∈Dl∪D̂l log
exp(ϕ(x)y) ∑Kl
j=1 exp(ϕ(x) j) ,
(12)
where λ is to balance the proportion of two loglikelihoods (discussed in Section 5.3) during training, τ is a hyper-parameter for temperature scaling which often appears in contrastive learning.
We summarize the whole training process of the EM framework in Algorithm 1 and the model architecture of our approach as shown in Figure 2.
It is worth noting that our method actually proposes a framework where probability estimation can flexibly adopt different ways for a variety of circumstances.
Algorithm 1 EM algorithm for optimization Input: Dl = {xli, i = 1, . . . , N}, Y l = {yli, i = 1, . . . , N}, Du = {xui , i = 1, . . . ,M}. Parameter: Model parameters θ.
1: Intialize θ by transferring knowledge. 2: while not converged do 3: Perform intent assignment (Z) using Kmeans; \\ E-Step 4: Compute P (Y l|Z, D; θ) and P (Z|D; θ) using current parameters θ; \\ E-Step 5: Update model parameters θ to maximize the log-likelihood L in Eq. (11). \\ M-Step 6: end while 7: return θ

heading: 4 Experiments

heading: 4.1 Datasets
text: We conduct experiments on three challenging datasets to verify the effectiveness of our proposed method. The detailed statistics are shown in Appendix A.
CLINC (Larson et al., 2019) is a popular intent dataset designed for out-of-domain intent detection, which contains 150 intents from 10 domains and 22500 utterances.
BANKING (Casanueva et al., 2020) is a banking dataset covering 77 intents and containing 13083 utterances.
StackOverflow represents a dataset dispersed
through Kaggle.com, encompassing 20 intents and 20000 utterances. We adopt the dataset processed by Xu et al. (2015).

heading: 4.2 Baseline and Evaluation Metrics
text: We follow Lin et al. (2020); Zhang et al. (2021) and divide the baselines to be compared into two categories: Unsupervised (Unsup.) and Semisupervised (Semi-sup.). All methods are introduced in Related Work (Section 2). For fairness, we uniformly use BERT as the backbone network when compared with the above methods. We also note that SCL (Shen et al., 2021) uses a stronger backbone network to obtain semantically meaningful sentence representations, and we also use the same backbone network in comparison with these methods. Similarly, when comparing with MTP-CLNN (Zhang et al., 2022), we use the same additional data and multi-task pre-training to enhance sentence representation.
To evaluate clustering results, we follow existing methods (Lin et al., 2020; Zhang et al., 2021) and adopt three widely recognized metrics: Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and clustering accuracy (ACC). It should be noted that when calculating ACC, the Hungarian algorithm is adopted to find the optimal alignment between the pseudo labels and the ground-truth labels as following Zhang et al. (2021).

heading: 4.3 Experimental Settings
text: For each dataset, 75% of all intents are randomly selected as known intent, with the remaining designated as unknown. Furthermore, 10% of the known intents data are chosen randomly as labeled data. We set the number of intents as ground truth in line with previous methods Lin et al. (2020); Zhang et al. (2021, 2022). Our other experimental settings are mostly the same as Lin et al. (2020); Zhang et al. (2021, 2022) for a fair comparison. We take different random seeds to run at least three rounds on the test set and report the final averaged results.
Our main experiments use pre-trained BERT, which is implemented in the Huggingface Transformers2, as the network backbone. We also replace the backbones of the compared baselines with the same BERT as ours. Only when comparing with SCL (Shen et al., 2021), which definitely point out that they use pre-trained MPNet (Reimers and Gurevych, 2019) as the backbone network, will we adopt the same backbone network for a fair comparison. Similarly, we will use the same additional data and the same pre-training strategy for fair comparison only when we compare with MTP (Zhang et al., 2022).
Moreover, considering the efficiency of the training process and the capacity of GPU, we only finetune the last transformer layer parameters during transferring knowledge and freeze all but the latter 6 transformer layers parameters during performing
2https://github.com/huggingface/transformers
the EM algorithm. See Appendix B for training details and parameters.

heading: 5 Results and Discussion

heading: 5.1 Main results
text: We present the main results in table 1, where the best results are highlighted in bold. It is clear from the results that our method achieves substantial improvements in all metrics and all datasets, especially in the BANKING dataset, where the number of samples in each class is imbalanced. These results illustrate the effectiveness and generalization of our method. At the same time, we note that most semi-supervised methods are better than unsupervised as a whole, which further verifies the importance of labeled data. From this perspective, we can explain why our method can be better than DeepAligned as it will constantly forget the knowledge existing in labeled data as shown in Section 1, and our method tailors the labeled data into model training to guide clustering so that our method can achieve better results.
To make a fair comparison with SCL (Shen et al., 2021), we also replace the backbone network in our method with the same MPNet as SCL, keeping other parts of our method unchanged. We present the results of our comparison with SCL and various variants (See Shen et al. (2021) for the calculation of specific strategies) on CLINC and BANKING in Table 2, where the best results are also highlighted in bold. Table 3 is the result of the comparison between our method and MTP, where DAC and CLNN are different strategies for intent discovery after pretraining. To make a fair comparison, we only adopt the same additional data and pre-training strategies (based on its released code) as MTP in the first step (Finetune stage in Figure 2), and the rest of the methods remain unchanged.

heading: 5.2 A Closer Look at Effectiveness
text: To better verify the effectiveness of our proposed method, we analyze the comparison results between our method and DeepAligned in a more finegrained way. We separate the known intents and the unknown intents from the test set and compare our method with DeepAligned on these two sub-datasets respectively (the experimental settings remain unchanged). The results are shown in Table 4, which demonstrates that our method can not only effectively apply to known intents, but also can more effectively discover new intents, and the effect of improvement is substantial. This also fully conforms to our expectations that the two processes of intent discovery and recognition of known intents can be “win-win”.

heading: 5.3 Effect of Exploration and Utilization
text: In objective function Eq. (11), we use λ to reconcile the effects of the two log-likelihoods. Intuitively, the first term is used to explore the intrinsic structure of unlabeled data, and the second term is used to strengthen the knowledge transferred from labeled data to utilize. We vary the value of λ and conduct experiments on CLINC and BANKING to explore the effect of λ, which also reflects the inference of exploration and utilization. As shown in Figure 3, only utilizing labeled data (λ = 0.0) or only exploring(λ = 1.0) the intrinsic structure will not achieve good results (below average). Interestingly, on all metrics and datasets, the effect of λ shows a similar trend (increase first and then
decrease), which indicates that we can adjust the value of λ to give full play to the role of both so that the model can make better use of known knowledge to discover intents accurately. This result shows that if the model wants to achieve good results, exploration and utilization are indispensable.

heading: 5.4 Effect of the Initial Number of Intents
text: Because we do not know the actual number of intents, we usually need to assign an initial number of intents (i.e., K) in advance as we do earlier. This also requires us to investigate the sensitivity of the model to the initial K. We investigate the performance of our method in the datasets by varying initial values (leaving others unchanged). As shown Figure 4, compared with others, our method can better adapt to different initial values.

heading: 5.5 Effect of the Known Intent Ratio
text: We also investigate the effect of known intent ratios on performance by adopting different known class ratios (25%, 50% and 75%). As shown in Figure 5, our method also shows better performance compared with other baselines. Interestingly, The advantage of our method in dataset BANKING is significant. We speculate that this may be related to the unbalanced number of samples in BANKING.
Although there are more known intents, it does not mean that enough labeled and balanced samples are provided. As a result, the previous methods (e.g. DeepAligned) not only failed to transfer more prior knowledge but also exacerbated the speed of forgetting in the follow-up process. This also provides room for future research.

heading: 5.6 More Than Remembering Knowledge
text: We show knowledge forgetting in DeepAligned in Section 1. After fine-tuning with labeled data, the prior knowledge is stored in the model in the form of model parameters. With the subsequent clustering steps, the parameters change gradually (the forgetting process is step by step from the forgetting curve in previous works).
However, as shown in Figure 6, we observe that our method does not have the catastrophic forgetting that occurs in DeepAligned. On the contrary, with the iteration (EM algorithm), our performance is better than that in the pre-transfer stage. We surmise that this improvement is brought by the sample set D̂l discovered in the unlabeled data (also can improve the intent discovery in Appendix E) corpus helps the identification of the known intents.

heading: 6 Conclusion
text: In this paper, we provide a probabilistic framework for intent discovery. This is the first complete theoretical framework for intent discovery. We also provide an efficient implementation based on this proposed framework. Compared with the existing methods, our method effectively alleviates the forgetting of prior knowledge transferred from known intents and provides intensive clustering supervised signals for discovering intents. Extensive experiments conducted in three challenging
datasets demonstrate our method can achieve substantial improvements. The subsequent analysis also shows that our method can better estimate the number of intents and adapt to various conditions. In the future, we will try different methods to perform intent assignments and explore more methods to approximate p(Y l|Z, D; θ) and p(Z|D; θ).
Limitations
To better inspire the follow-up work, we summarize the limitations of our method as follows: 1) From our experimental results the Appendix D, we can see that the estimation of the number of intents in our proposed can be further improved. 2) We do not try more means to prevent knowledge from forgetting. We can probe into the intrinsic structure of unlabeled data in a more fine-grained way by improving the posterior estimation. 3) According to Section 5.3, we have verified that both exploration and utilization are indispensable, but at the same time, we only empirically choose the specific proportion of both, without theoretical analysis of the most appropriate proportion for each dataset. We look forward to making progress in the follow-up research on the above limitations.

heading: Acknowledgements
text: We thank Dr.Wang yuxin and Dr.Liu Peiju for their patience and meaningful suggestions in this work early. This work was supported by the National Key Research and Development Program of China (No.2020AAA0108700), National Natural Science Foundation of China (No.62022027) and CAAIHuawei MindSpore Open Fund.

heading: A Statistics of Datasets
text: We present detailed statistics of datasets in our experiments in Table 6. CLINC (Larson et al., 2019) is a dataset designed for Out-of-domain intent detection, which contains
150 intents from 10 domains and 22500 utterances. BANKING (Casanueva et al., 2020) is a dataset covering 77 intents and containing 13083 utterances. StackOverflow (Xu et al., 2015) represents a dataset dispersed through Kaggle.com, encompassing 20 intents and 20000 utterances. We adopt the dataset processed by (Xu et al., 2015).

heading: B Experiment Details
text: Our main experiments use pre-trained BERT (bertuncased, with 12-layer transformer), which is implemented in the Huggingface Transformers3. We try learning rate in {1e − 5, 5e − 5} and λ in {0.5, 0.6}. The training batch size is 512, and the temperature scale τ is 0.1. All experiments were conducted in the Nvidia Ge-Force RTX-3090 Graphical Card with 24G graphical memory.

heading: C More Results on Effect of Exploration and Utilization
text: In this section, we detail the results of varying λ in the Table 5. This result can be used as a supplement to Section 5.3, which further proves that if the model wants to achieve better results, both exploration and utilization are indispensable.
D Estimate the Number of Intents (K)
A key point of intent discovery is whether the model can accurately predict the number of intents. DeepAligned proposes a simple yet effective estimation method. However, due to the alignment operation in the iterative process of clustering (see Zhang et al. (2021) for details), DeepAligned
3https://github.com/huggingface/transformers
needs to determine K in advance and only limited labeled data is used, while a large number of unlabeled data are ignored. Unlikely, our method does not directly rely on pseudo labels so that we can continue to refine K during subsequent clustering. We use the same settings as Zhang et al. (2021) and firstly assign the number of intents (i.e., K in intent assignments) as two times the ground truth number to investigate the ability to estimate K. In the process of executing the EM algorithm, we refine K per 10 epochs using the method as suggested in Section 3.3. To effectively demonstrate the impact and efficiency of our proposed framework on the estimation of K, we did not consider dataset D̂l in the experiment. We get the final performance of the model and the results are shown in Table 7 (Figure 7 shows the intermediate values of K per epoch.) shows that our method can predict the number of intents more accurately and achieve
better results at the same time. During the experiment, we observed that the performance of model exhibited fluctuations attributed to the setting of hyperparameters. A more comprehensive and indepth investigation of the estimation of K will be left for future research endeavors.

heading: E Effect of D̂l discovered in unlabeled data
text: In addition to the labeled data in hand, in Section 3.3, we also use the sample set D̂l predicted known intents in unlabeled data during discovery (See Section 3.3 for the specific construction of D̂l. The nearest neighbor measure is based on the cosine similarity of the sample representation in the semantic space). In this section, we will further analyze the benefits brought by these discovered sample set. We have compared the effects of adding D̂l
and not adding D̂l, and the comparison results are shown in Table 8. From Table 8, we can easily conclude that the added sample set D̂l can improve the effectiveness. This also proves the importance of exploring the intrinsic structure of unlabeled data, which can not only improve the effect of preventing knowledge forgetting (Section 5.6) to improve the identification of IND, but also improve the effect of intent discovery, which is completely in line with our expectations.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Section "Limitations" (7th Section)
3 A2. Did you discuss any potential risks of your work? Section "Limitations" (7th Section)
3 A3. Do the abstract and introduction summarize the paper’s main claims? "Abstract" and Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4.1 and Appendix A
3 B1. Did you cite the creators of artifacts you used? Section 4.1
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? These datasets are available for all researchers in the NLP community.
7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? These datasets are only for scientific research and are available for all members of the NLP research community. We have adhered to the typical method of utilizing these resources.
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? These datasets are only for scientific research and are available for all members of the NLP research community. We have adhered to the typical method of utilizing these resources.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Section 4.1
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Appendix A
C 3 Did you run computational experiments? Section 4 and Section 5
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Appendix B
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 4.3 and Appendix B
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4.3 and Section 5.1
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Appendix B
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: A more comprehensive and indepth investigation of the estimation of K will be left for future research endeavors