Enhancing Task-oriented Dialogue Systems with Generative Post-processing Networks (2023)

Abstract: Recently, post-processing networks (PPNs), which modify the outputs of arbitrary modules including non-differentiable ones in task-oriented dialogue systems, have been proposed. PPNs have successfully improved the dialogue performance by post-processing natural language understanding (NLU), dialogue state tracking (DST), and dialogue policy (Policy) modules with a classification-based approach. However, they cannot be applied to natural language generation (NLG) modules because the post-processing of the utterance output by the NLG module requires a generative approach. In this study, we propose a new post-processing component for NLG, generative post-processing networks (GenPPNs). For optimizing GenPPNs via reinforcement learning, the reward function incorporates dialogue act contribution, a new measure to evaluate the contribution of GenPPN-generated utterances with regard to task completion in dialogue. Through simulation and human evaluation experiments based on the MultiWOZ dataset, we confirmed that GenPPNs improve the task completion performance of task-oriented dialogue systems.

## introduction a typical task-oriented dialogue system has a pipelined structure consisting of four modules @xcite ): natural language understanding (nlu), dialogue state tracking (dst), dialogue policy (policy), and natural language generation (nlg). many studies have used reinforcement learning (rl) to improve the task completion performance of an entire pipelined dialogue system by fine-tuning modules directly @xcite @xcite . recently, @xcite proposed a novel method using post-processing networks (ppns) that can optimize pipelined dialogue systems that consist of arbitrary modules including non-differentiable ones (e.g., rule-based and web api-based). their method uses rl to optimize a neural-based component called a ppn, which modifies the output of each module. ppns have been applied to three modules, namely nlu, dst, and policy, and have been shown to improve the task completion performance of various pipelined systems. in ppns, the post-processing of module outputs is treated as a binary classification task, i.e., adding or removing slots in the output of each module, and this classification task is modeled with a multi-layer perceptron (mlp). this means that the current ppns could not be applied to nlgs, where the unit of output is not slots but a sequence of tokens, i.e., natural language. to overcome this limitation, we propose a pretrained language model (plm)-based generative post-processing network (genppn) that can postprocess the output of nlgs in a manner similar to conventional ppns (figure 1 ). to optimize genppn via commonly used rl frameworks for plms @xcite , a reward for each utterance at each turn (an utterancelevel reward) is required. in a task-oriented dialogue, however, a reward indicating success or failure is obtained only at the end of a multi-turn interaction (a dialogue-level reward). therefore, we introduce a dialogue act (da) contribution for distributing the dialogue-level reward to the utterance-level reward. here, a da is the meaning representation of the information that nlg converts into an utterance, and da contribution is a measure of how much the da of each utterance contributes to the final task completion of the dialogue. experiments on the multiwoz dataset @xcite confirm that genppns can improve the task performance of the entire dialogue system, regardless of the architecture of the nlg module. furthermore, an ablation study reveals that the introduction of da contribution is effective for learning genppns to improve task completion. the contributions of this study are threefold: ## experiments in our experiments, we first evaluated the effectiveness of genppns using a user simulator. then, using the optimized genppn, we conducted a dialogue evaluation experiment using human subjects. ## case study it is worth noting that the utterance generated by post-processing does not contain any information about the attraction. this is probably because genppn judged from the dialogue context that the information about the attraction was not what the user requested, and that conveying this piece of information would interfere with the task completion. these results indicate that genppn can correct errors propagated from preceding modules beyond the realm of nlg so that the entire system's performance improves. ## summary and future work post-processing networks (ppns), which modify the outputs of arbitrary modules including nondifferentiable ones in task-oriented dialogue systems, have been applied to nlu, dst, and policy modules. in this paper, we proposed generative post-processing networks (genppns) for nlg modules. we optimized genppn toward the task completion performance of the entire system by using da contribution-based rewards. experiments on the multiwoz dataset and dialogue simulations confirmed that our genppn could improve the task success rate of various dialogue systems, regardless of the nlg architecture. it was also confirmed that a genppn optimized with a dialogue simulation was effective in a human evaluation experiment. in the future, we would like to integrate our genppn with the existing ppns of nlu, dst, and policy to realize the post-processing of all modules and further improve the systems performance. since genppn is a generative model, we would like to extend it not only to text but also to the post-processing of speech recognition and speech synthesis modules in spoken dialogue systems.