Turn-Level Active Learning for Dialogue State Tracking (2023)

Abstract: Dialogue state tracking (DST) plays an important role in task-oriented dialogue systems. However, collecting a large amount of turn-by-turn annotated dialogue data is costly and inefficient. In this paper, we propose a novel turn-level active learning framework for DST to actively select turns in dialogues to annotate. Given the limited labelling budget, experimental results demonstrate the effectiveness of selective annotation of dialogue turns. Additionally, our approach can effectively achieve comparable DST performance to traditional training approaches with significantly less annotated data, which provides a more efficient way to annotate new dialogue data.

## introduction dialogue state tracking (dst) constitutes an essential component of task-oriented dialogue systems. the task of dst is to extract and keep track of the user's intentions and goals as the dialogue progresses @xcite . given the dialogue context, dst needs to predict all (domainslot, value) at each turn. since the subsequent system action and response rely on the predicted values of specified domain-slots, an accurate prediction of the dialogue state is vital. despite the importance of dst, collecting annotated dialogue data for training is expensive and time-consuming, and how to efficiently annotate dialogue is still challenging. it typically requires human workers to manually annotate dialogue states @xcite or uses a machines talking to machines (m2m) framework to simulate user and system conversations @xcite . either way, every turn in the conversation needs to be annotated because existing dst approaches are generally trained in a fully supervised manner, where turn-level annotations are re-quired. however, if it is possible to find the most informative and valuable turn in a dialogue to label, which enables the training of a dst model to achieve comparable performance, we could save the need to annotate the entire dialogue, and could efficiently utilize the large-scale dialogue data collected through api calls. active learning (al) aims to reduce annotation costs by choosing the most important samples to label @xcite @xcite . it iteratively uses an acquisition strategy to find samples that benefit model performance the most. thus, with fewer labelled data, it is possible to achieve the same or better performance. al has been successfully applied to many fields in natural language processing and computer vision @xcite @xcite . however, the adoption of al in dst has been studied very rarely. @xcite have studied to use al to reduce the labelling cost in dst, using a dialogue-level strategy. they select a batch of dialogues in each al iteration and label the entire dialogues (e.g., every turn of each dialogue), which is inefficient to scale to tremendous unlabelled data. to our knowledge, turn-level al remains unstudied for the task of dst. furthermore, existing dst approaches @xcite @xcite treat each dialogue turn as a single, independent training instance with no difference. in fact, in the real-world, utterances in a dialogue have different difficulty levels @xcite and do not share equal importance in a conversation. for example, in fig. 1 , turn-1 is simple and only contains a single domain-slot and value (i.e., hotel-@xmath0, while turn-2 is more complex and generates three new domain-slots, i.e., hotel-book day, hotel-book people, hotel-book stay. given the limited labelling budget, it is an obvious choice to label turn-2 instead of turn-1 since the former is ## preliminaries we formalize the notations and terminologies used in the paper as follows. active learning (al) al aims to strategically select informative unlabelled data to annotate while maximizing a model's training performance @xcite . this paper focuses on pool-based active learning, where an unlabelled data pool is available. suppose that we have a model m, a small seed set of labelled data l and a large pool of unlabelled data u. a typical iteration of al contains three steps: (1) train the model m using l; (2) apply an acquisition function a to select k instances from u and ask an oracle to annotate them; and (3) add the newly labelled data into l. ## active learning for dialogue state tracking in this section, we first define our turn-level albased dst framework, followed by the turn selection strategies. ## conclusion this paper tackles the practical dialogue annotation problem by proposing a novel turn-level al framework for dst, which strategically selects the most valuable turn from each dialogue for labelling and training. experiments show that our approach outperforms strong dst baselines in the weaklysupervised scenarios and achieves the same or better joint goal and slot accuracy with significantly less annotated data. further analysis are conducted to investigate the impact of al query sizes, base dst models and turn selection methods.