A Task-Oriented Dialogue Architecture via Transformer Neural Language Models and Symbolic Injection (2021)

Abstract: Recently, transformer language models have been applied to build both task- and non-task-oriented dialogue systems. Although transformers perform well on most of the NLP tasks, they perform poorly on context retrieval and symbolic reasoning. Our work aims to address this limitation by embedding the model in an operational loop that blends both natural language generation and symbolic injection. We evaluated our system on the multi-domain DSTC8 data set and reported joint goal accuracy of 75.8% (ranked among the first half positions), intent accuracy of 97.4% (which is higher than the reported literature), and a 15% improvement for success rate compared to a baseline with no symbolic injection. These promising results suggest that transformer language models can not only generate proper system responses but also symbolic representations that can further be used to enhance the overall quality of the dialogue management as well as serving as scaffolding for complex conversational reasoning.

## introduction building task-oriented dialogue systems using a conventional pipeline approach, where modules are optimized separately, increases the fine control for dialogue management, but it does not necessarily improve overall performance @xcite . in contrast, end-to-end neural models employ a straightforward training approach to generating system responses; however, this approach is impractical for goal-oriented dialogues where the system needs to interact with external systems or generate an explanation that supports its decisions @xcite . recently, the use of transformer models for building end-to-end dialogue systems has attracted considerable attention @xcite ; however, as far as we know, current approaches operate solely at the text (word) level. we extend this approach to utilize transformer model's versatility to generate more complex constructs such as symbol representations. in this paper, we propose a hybrid approach. we first apply a fine-tuned, end-to-end transformer model for multi-domain task-oriented dialogue. then, during inference, we decouple the execution into expert modules that collaboratively process the content of a common knowledge base (resembling the blackboard architecture @xcite ). in our experiments, we empirically demonstrate that the transformer model can be fine-tuned to generate not only text from a given input but also symbolic representations (e.g., @xmath0, manipulate those symbolic representations to generate new ones (e.g., dialogue @xmath1, and generate natural language from symbols (e.g., system @xmath2. this work led us to a new generic reasoning architecture that leverages the ability of a transformer model to effectively manipulate representations that are mixtures of natural and symbolic language. the result is a simple architecture that uses a uniform representation to blend together dialogue aspects of interpretation, language understanding and generation, and behavior. blackboard: it provides a global memory where pieces of knowledge (history, user's intents and goals, system actions, etc.) are continuously updated by modules to maintain the dialogue context. forget: this module shortens sequence inputs that surpass the maximum limit of tokens that a transformer model can process at a time (in our case, 1,024 tokens for gpt-2). additional input tokens beyond this limit are truncated, potentially blackboard (dialogue context) word-level dst (gpt-2) dialogue history dialogue interface user utterance forget service call service results system response system actions history dst slot-values validator pol (gpt-2) service executor buses hotel flights train shared ride events… service call history modified slot values validator slot values history dst serv. results system actions decoder nlg (gpt-2) history dst results sys acts system response decoder decoder history history dst serv call h i s t o r y , d s t d s t serv. call serv. results s y s a c t io n s hist, dst res, sys act delexicalized sys resp. discarding relevant symbols needed for dialogue processing. instead, this module discards only the oldest (non-symbolic) elements in dialogue history to keep the input token size within the limit. a more sophisticated component that is more selective in discarding irrelevant information, chunking information, etc., is left as future work. ## experiment framework we used the open-source implementation of gpt-2-small transformer model foot_0 with values for adam learning rate (5.75e-5), epsilon for adam optimizer (1e-8), and batch size (4). to generate more coherent text as proposed by @xcite , we set parameters top-p nucleus sampling (0.95) and top-k sampling (50) using grid search. we utilized the schema-guided data set proposed at the dialogue system technology challenge dstc8-task4 foot_1 . we chose this data set due to: 1) its rich annotations across the whole dialogue pipeline; 2) its size that exceeds the existing dialogue corpora in scale (with over 20k multidomain, task-oriented dialogues spanning 45 apis over 20 domains); and 3) it contains a significant amount of dialogues for the transportation domain 3 . in this work, we only tested dialogues containing domains/services shown during training although unseen slot values were allowed (the evaluation of unseen domains is left as future work). since the dstc8 challenge does not provide sql scripts or equivalent, we reverse-engineered the database results from the data set and implemented our own services database. we carried out automatic evaluation of our system on 2,361 dialogues using diverse metrics. for dst, we used the metrics provided with the data set, measuring: average goal accuracy (accuracy of predicting the value of a slot correctly), joint goal accuracy (average accuracy of predicting all slot assignments for a turn correctly), active intent accuracy (a fraction of user turns for which the intent was rightly predicted), and requested slot f1 (the macro-averaged f1 score for requested slots over all eligible user turns). in addition, we extended these metrics to measure system actions in a similar way: service call accuracy, joint parameter accuracy, and joint system action accuracy. also, we used success rate for system performance, and bleu for fluency of the generated response. human evaluation is left as future work. finally, post hoc, we ran an error analysis that let us identify the kind of error that affected system performance the most, allowing us to build a simple heuristic-based expert that focused on measuring particular kinds of modeling errors to identify areas for improving overall performance. ## results error analysis: from all the reported metrics, we focused on the results obtained for the joint goal accuracy (jga) for two reasons: 1) jga is the primary evaluation metric used for ranking approaches submitted to dstc8-task4; and 2) this metric got the lowest scores for exp o and exp g among all the evaluation metrics (see from the error analysis, we found 3 main kinds of errors that affect jga: 1) slot names were correctly predicted but slot values were not (10.5% of errors); 2) slot names that appeared in the gold dst but were not predicted by the system (29.1%); and 3) slot names that were predicted but did not appear in the gold dst (60.4%). given its significant presence, we focus on the third kind of error. the main causes for this error to occur are: 1) the slot value is predicted but not mentioned by either the user or the system in the dialogue history (over-fitting); and 2) the slot value is mentioned/offered by the system but not accepted by the user (e.g., the system says "there is a direct bus that departs at 9:50 am and costs $36.", where the slot trip_fare was unsolicited by the user, and then the user says "hmm any buses departing in the we implemented a heuristic-based slot-value validator to mitigate the error above. first, we extracted and classified all the slot values from the training data set and store them in a dictionary. then, a set of heuristic rules based on fuzzy string matching determine whether a slot value is present in the dialogue history by calculating its similarity with the values in the dictionary, fixing the first cause of the error. next, if the slot value is mentioned only by the system, the value is retained only if the system offered the value at any prior turn (i.e., sys_act: ''offer'') and the user accepted the offered slot value in the next turn (i.e., ## related work in comparison with traditional pipelined dialogue architectures @xcite where nlu @xcite , dst @xcite , and pol/nlg @xcite modules are optimized separately; our architecture is simpler and less prone to cascading failures due to the folding of multiple nlp tasks into a single transformer model and the exposure of symbolic representation directly to the model. more recently, pre-trained language models similar to gpt-2 have been used for building end-to-end dialogue systems. our approach is similar in nature to the work proposed by @xcite in that we use a single causal language model to generate all outputs given a dialogue context. however, unlike these approaches, our model not only encodes dst and database results (which shows a labeling cost reduction) but also encodes dialogue policy and service call templates, allowing the system to be able to monitor errors and manipulate symbolic representations at different stages of turn processing. similar to other transformer dialog systems @xcite , our model learns from text; however, our model also learns and generates complex structures that intermix natural and symbolic language. in particular, the work described by @xcite and @xcite encodes both belief state and knowledge base constructs into simple text representations and generates text-only outputs. in contrast, our approach encodes, manipulates, and generates more sophisticated knowledge representations, roughly first-order logic constant terms that are implicitly learned and which could be used to communicate with external sources and expert components such as a symbolic reasoner. dialogue systems have many similarities to conversational workflow systems. the virtual information officer @xcite required more than thirty individual models performing task classification, entity resolution, and information extraction. moreover, @xcite discuss the challenges found when directly translating natural language inputs into symbolic api calls in a service composition system. both systems would benefit from the architecture and method presented here. ## conclusions in this paper, we empirically demonstrated that several capabilities of transformer language models can be leveraged to construct a new dialogue architecture that is more flexible and simpler (resulting in much lower engineering costs) and extensible (allowing symbolic injection and manipulation), while retaining reasonable performance. algorithm 1 main flow of control dialogue control(): input: @xmath3: while signal != end do 2: figure 2 : sample of a multi-domain dialogue. for each gold user utterance, we provide both the gold system utterance and the utterance generated by our system.