Diable: Efficient Dialogue State Tracking as Operations on Tables (2023)

Abstract: Sequence-to-sequence state-of-the-art systems for dialogue state tracking (DST) use the full dialogue history as input, represent the current state as a list with all the slots, and generate the entire state from scratch at each dialogue turn. This approach is inefficient, especially when the number of slots is large and the conversation is long. We propose Diable, a new task formalisation that simplifies the design and implementation of efficient DST systems and allows one to easily plug and play large language models. We represent the dialogue state as a table and formalise DST as a table manipulation task. At each turn, the system updates the previous state by generating table operations based on the dialogue context. Extensive experimentation on the MultiWoz datasets demonstrates that Diable (i) outperforms strong efficient DST baselines, (ii) is 2.4x more time efficient than current state-of-the-art methods while retaining competitive Joint Goal Accuracy, and (iii) is robust to noisy data annotations due to the table operations approach.

## introduction dialogue state tracking (dst; @xcite is the task of tracking user requests from the dialogue history in the form of slot-value pairs @xcite @xcite . the slots are defined in a domainspecific schema and represent the fields that need to be extracted from the dialogue to execute queries in the backend and generate responses. recent generative approaches to dst based on language models @xcite often use the entire dialogue history as input and represent the state, at each turn, as the concatenation of all the slots in the schema, where inactive when the conversation starts, the state table is empty. at each dialogue turn, the system outputs a table update operation (either insert or delete), and the state is modified accordingly. we propose efficient dialogue state tracking as operations on tables (diable, shown in figure 1 ), a novel task formulation and a new dst approach that better uses the generative capabilities of language models. our approach simplifies the design and implementation of dst systems and works with any sequence-to-sequence model. our intuition is that a dst system translates conversations into filters for database searches. inspired by formal languages for databases and the recent success in applying sequence-to-sequence models to text-to-sql tasks @xcite , we represent the dialogue state as an implicit table and frame dst as a table manipulation task. at each turn, the system updates the previous state by generating update operations expressed in a simplified formal language based on the current dialogue context (see figure 1 ). diable is the first end-to-end dst system that outputs state operations and values jointly while processing all slots simultaneously. based on extensive experimentation using the multiwoz benchmark @xcite , we show that diable can successfully and efficiently translate conversations into filters for database searches. our approach minimises the number of input and output tokens required resulting in a significant efficiency gain (2.4x reduction in inference time compared to state-of-the-art cumulative state systems). our main contributions are as follows: ## a taxonomy of dst approaches the goal of dst systems is to handle long, diverse conversations in multiple domains with large schemas and unrestricted vocabulary, potentially without extra supervision @xcite . achieving this goal has prompted the development of different dst approaches. ontology-based approaches treat dst as either a classification or a token classification task. they assume that all possible slot-value pairs are restricted to a fixed set, or ontology, either predefined or extracted from the training data. classificationbased approaches output a probability distribution over values given the dialogue context and a slot @xcite while token classification approaches output a probability distribution over slots for each token @xcite . the ontology-based formulation simplifies the dst task considerably, thus the performance of these systems is usually relatively high for specific datasets @xcite @xcite . complex dialogues with large schemas pose a significant challenge for traditional ontology-based approaches as they do not easily generalise to new domains nor scale to large ontologies @xcite @xcite . for this reason, ontology-based approaches are out of scope for our paper. open-vocabulary approaches address these limitations by formulating dst as either a reading comprehension task wherein for each slot a span is extracted from the dialogue context @xcite , or as a generation task wherein a value for each slot is generated based on the dialogue history @xcite . by leveraging sequence-to-sequence models @xcite @xcite , generative approaches have recently achieved states-of-the-art results @xcite @xcite @xcite @xcite @xcite @xcite . however, these methods predict the dialogue state from scratch at each turn and generate a value for each slot, even when a slot is not active (figure 2 ). we argue ( §6) that these are the main sources of inefficiencies of current dst systems. we compare diable with these methods in the "cumulative state models" section of efficient approaches seek efficiency by minimising the number of values to generate, thus decomposing dst into two successive sub-tasks: state operation prediction and value generation. in this way, only the slots that need to be changed are considered for value generation @xcite . these approaches are the most related to diable in that they target efficiency. we compare them against diable in section "efficient models" of ## diable: dialogue state tracking as operations on tables we introduce a novel efficient formulation of the dst task and a system, diable, specifically designed to enhance efficiency and optimise the capabilities of state-of-the-art sequence-to-sequence models. in this section, we describe our approach, formalise the dst problem, and introduce the concepts of state as a table and state operations. ## results in this section, we discuss our experimental results. in ## discussion our task formalisation is intuitively simple and is especially beneficial for large pre-trained sequenceto-sequence models. first, the state is expanded sequentially and thus only includes the necessary slots. this minimises the size of the input context, allowing the models to scale to larger schemas before reaching their maximum input length. second, since the model needs to focus on the state changes, the decoder only needs to generate operations for a limited number of slots (previous slots persist implicitly in the state, no need for explicit "carryover" operations). third, our system is general in that it deals with span-based and categorical slots in the same way, and outputs both the operations and the slot-value pairs in a single forward pass, without the need for specialised architectures. finally, since not all pre-defined slots are needed in the input, we do not have to access the schema beforehand, and thus it can be learned from the data directly. ## conclusions in this paper, we introduce a novel efficient formulation of the dst task and a new system, diable, specifically designed to enhance efficiency and optimise the capabilities of state-of-the-art sequenceto-sequence models. diable represents the dialogue state as an implicit table and updates it using a sequence-to-sequence model to generate table operations at each turn. our task formalisation provides a significant efficiency gain (up to 2.4x speed-up in inference time) compared to cumulative state approaches adopted by the current state-of-the-art dst systems. moreover, this sizeable improvement comes with a minimal efficiencyaccuracy trade-off. in fact, diable outperforms other efficient dst approaches in the literature by more than 3 absolute jga points on multiwoz 2.4 and shows a competitive performance with respect to current dst state-of-the-art systems. diable comes with other advantages: it is simple and general (it makes no assumptions about the schema and does not require any specialised architecture) and it is robust to noise. moreover, it allows to plug and play sequence-to-sequence models without any architectural modification easily. finally, our approach goes beyond the dialogue setting and can be adapted to the sequential processing of long documents for information extraction tasks with memory-constrained language models.