A Unified Supervised and Unsupervised Dialogue Topic Segmentation Framework Based on Utterance Pair Modeling (2025)

Abstract: The Dialogue Topic Segmentation task aims to divide a dialogue into different topic paragraphs in order to better understand the structure and content of the dialogue. Due to the short sentences, serious references and non-standard language in the dialogue, it is difficult to determine the boundaries of the topic. Although the unsupervised approaches based on LLMs performs well, it is still difficult to surpass the supervised methods based on classical models in specific domains. To this end, this paper proposes UPS (Utterance Pair Segment), a dialogue topic segmentation method based on utterance pair relationship modeling, unifying the supervised and unsupervised network architectures. For supervised pre-training, the model predicts the adjacency and topic affiliation of utterances in dialogues. For unsupervised pre-training, the dialogue-level and utterance-level relationship prediction tasks are used to train the model. The pre-training and fine-tuning strategies are carried out in different scenarios, such as supervised, few-shot, and unsupervised data. By adding a domain adapter and a task adapter to the Transformer, the model learns in the pre-training and fine-tuning stages, respectively, which significantly improves the segmentation effect. As the result, the proposed method has achieved the best results on multiple benchmark datasets across various scenarios.

## introduction the dialogue topic segmentation is an important task in natural language processing (nlp), which helps to better understand the structure and content of a dialogue by dividing it into multiple different "topics". it can be applied to a variety of application scenarios such as dialogue generation @xcite , dialogue summarization @xcite , and knowledge selection @xcite . dialogue topic segmentation methods are divided into supervised and unsupervised types. su-pervised methods typically model topic segmentation as an utterance sequence labeling problem @xcite . it predicts whether each sentence is beginning of a topic paragraph. this approach typically requires a large amount of labeled data to train the model and may perform better when dealing with specific domains or specific types of dialogue. large language models(llms) have performed well on many general-purpose tasks in recent years, but there is still a gap with supervised learning methods using small language models. recent research has demonstrated the greater potential and advantages of using small language models for supervised learning in dialogue topic segmentation tasks @xcite . therefore supervised learning remains the optimal choice when high-quality labeled data is available. however, in specialized domains lacking such data, supervised methods are not applicable. unsupervised methods @xcite do not rely on labeled datasets and train coherent models to evaluate the similarity of continuous discourse, and then employ global segmentation algorithms to compute topic segmentation points. unsupervised methods do not directly use segmentation annotations as supervisory signals but instead learn the similarity of continuous utterances to indirectly identify topic boundaries. although unsupervised methods @xcite can address the lack of training data in certain domains, but, due to the limitations of their network structures, existing unsupervised methods cannot utilize segmentation annotations as supervisory signals, making it difficult to extend them to broader data scenarios such as supervised and few-shot learning. the main contributions of this paper are listed as follows: ## related work dialogue topic segmentation divides a dialogue into sections by identifying different topics within the dialogue, including both unsupervised @xcite and supervised methods. previous studies have often adopted unsupervised methods due to the lack of dialogue topic segmentation datasets.unsupervised methods generally included two steps: assessing topic similarity and determining segmentation boundaries. popular unsupervised methods were texttiling @xcite and its improvements @xcite . discourse similarity was often taken into account when assessing topic similarity. texttiling (hearst, 1997) measured discourse similarity based on patterns of lexical co-occurrence and distribution. @xcite leveraged bert model embeddings to enhance texttiling, taking advantage of the rich semantic information contained in pre-trained models. @xcite proposed training a utterance pair coherence scoring model to measure topic relevance between utterances. @xcite introduced dialstart, which considered both utterance similarity and coherence. despite the advantages of unsupervised methods in terms of data cost, their performance was often difficult to match that of supervised methods @xcite . @xcite proposed the pen-ns method, which utilized a parallel extraction network for segment extraction and optimized the bipartite matching cost to capture inter-segment dependencies. @xcite proposed segment pooling lstm (s-lstm) model for joint topic segmentation and labeling. @xcite proposed sector, an end-to-end model that learned latent topic embeddings. relevant experiments have shown that using the entire text as input can lead to a loss of coherence in dialogue. @xcite demonstrated the importance of text pairs modeling for topic segmentation. they addressed the challenge of data scarcity by constructing a dataset with pairs labeled as internal paragraph or internal document. @xcite extended this to dialogue by building a corpus of consecutive and non-consecutive utterance pairs. @xcite proposed dialogue sentence embedding (dse), employing consecutive utterances within the same dialogue as positive pairs for contrastive learning. in order to decrease the cost of fine-tuning pretrained models to downstream tasks, prompt learning becomes a mainstream method. in topic segmentation tasks, when using llms for topic segmentation, prompt templates can be used to guide models such as gpt-3.5 @xcite and gpt-4 @xcite . topic shift is a task that detects if the topic changes in a dialogue and has similarities with topic segmentation. @xcite utilized label-level, topic-level, and turnlevel prompts to enhance a model's ability to understand and predict topic shift in a dialogue. in summary, this paper proposes a topic segmentation method based on utterance pair relationship modeling, which unifies the supervised and ## conclusion this paper proposes a novel approach for topic segmentation in dialogue texts based on "utterance pair" relationship modeling, unifying supervised and unsupervised network architectures. to address the availability of annotated data, both supervised and unsupervised pre-training strategies are employed. to facilitate knowledge transfer during the two-stage training process, the model incorporates domain and task adapters. experiments conducted across three data scenario settings validate the effectiveness of the proposed method. future work will explore more complex multi-modal sce-narios, and investigate the applicability across different languages and cultural contexts to enhance the method's generalizability and robustness.