Efficient Dialog State Tracking Using Gated- Intent based Slot Operation Prediction for On-device Dialog Systems (2022)

Abstract: Conversational agents on smart devices need to be efficient concerning latency in responding, for enhanced user experience and real-time utility. This demands on-device processing (as on-device processing is quicker), which limits the availability of resources such as memory and processing. Most state-of-the-art Dialog State Tracking (DST) systems make use of large pre-trained language models that require high resource computation, typically available on high-end servers. Whereas, on-device systems are memory efficient, have reduced time/latency, preserve privacy, and don’t rely on network. A recent approach tries to reduce the latency by splitting the task of slot prediction into two subtasks of State Operation Prediction (SOP) to select an action for each slot, and Slot Value Generation (SVG) responsible for producing values for the identified slots. SVG being computationally expensive, is performed only for a small subset of actions predicted in the SOP. Motivated from this optimization technique, we build a similar system and work on multi-task learning to achieve significant improvements in DST performance, while optimizing the resource consumption. We propose a quadruplet (Domain, Intent, Slot, and Slot Value) based DST, which significantly boosts the performance. We experiment with different techniques to fuse different layers of representations from intent and slot prediction tasks. We obtain the best joint accuracy of 53.3% on the publicly available MultiWOZ 2.2 dataset, using BERT-medium along with a gating mechanism. We also compare the cost efficiency of our system with other large models and find that our system is best suited for an on-device based production environment.

## introduction with the rapid growth of internet and thus internet of things, smart devices including smartphones, tv, refrigerators, among others that can communicate with each other are being increasingly introduced in the market. smart devices come with processing power, which opens up the capability of deploying ai solutions @xcite . these solutions also include on-device conversational agents (ca) and thus its components such as intent detection @xcite . these cas such as alexa, bixby, and google home, tend to be taskoriented, and perform the device-specific tasks. a user of a smart device ca expects a quick action and response from the device, otherwise it's no better than manually performing the task. the low latency demands for on-device processing to reduce/remove network calls to a server. even though the smart devices come with processing capabilities, usually the processing power and memory are very limited. this makes it very difficult to deploy large and complex dnn models on the device. we are particularly interested in the task of dialog state tracking (dst), which is a crucial module of a ca. many state-of-the-art (sota) dst systems, such as @xcite , @xcite are based on large language models, which need high processing power and memory during inference, and thus suitable for server side processing. in this scenario, on-device systems can play a major role. they can operate on low resources and hence, can be run on mobile devices / edge processors. in addition to occupying lesser space and providing lower latency, they also require lesser ram. they are better than server based ## baseline system (som-dst) as discussed in the section 1, in this work, we base our experiments on improving the performance and optimizing the cost of the som-dst architecture (depicted in figure 1 ) by @xcite . to improve the latency, the dst system is divided into two modules: a. state operation predictor (sop): for each slot (defined in the ontology), classify it amongst a predefined set of labels (such as carryover, update, delete, don't-care). these label values help us identify which slot's value has to be generated/updated and which has to be modified, deleted, skipped etc. the input to the sop module is formed by concatenating the current dialogue context with the previous belief state (slots and corresponding values). the input is passed through a bert encoder to obtain encodings for each slot, which are further processed for operation classification. b. slot value generator (svg): this module generates value only for the slots in which update operation is predicted from sop. svg generates the slot values using a simple gru based model. sop gates the amount of svg requests made. this is a very efficient way of determining the dialog state. in this work, we first replicate the results on multiwoz 2.2 dataset using the same architecture. we then experiment with different fusion experiments for the multitask learning of slot and intent predictions. ## fusion experiments we are mainly trying to fuse information from intent classification into state operation prediction. introduction of the intent prediction into the som-dst architecture was designed in several ways as follows ## analysis in we also compare the sizes of the different pretrained models used by different systems, which gives a hint of the comparative memory efficiency of the models. compared to t5 and pegasus, our model makes use of bert-medium , which is 4 times and 10 times smaller, respectively. our model size is 202 mb which makes it feasible to deploy on-device. as presented in ## conclusion from our experiment results, we can conclude that using gating based self-attention on the intent logits for state operation prediction improves the accuracy. there is also a significant reduction in model size and latency when compared to other existing sota models which use large pre-trained language models. this makes our model more suitable for on-device based production environment.