Dynamic Stochastic Decoding Strategy for Open-Domain Dialogue Generation (2024)

Abstract: Stochastic sampling strategies such as top-k and top-p have been widely used in dialogue generation task. However, as an open-domain chatting system, there will be two different conversation scenarios, i.e. chit-chat and knowledge-based question answering. In the former situation, responses diversity is essential due to the one-to-many nature in dialogue. The latter, on the other hand, requires less randomness given that stochastic decoding strategy entails the risk of generating incorrect information. As a result, an adaptive and flexible decoding strategy is needed to cope with these two scenarios simultaneously. To this end, we propose the dynamic decoding strategy (DDS), which can adjust the decoding space w.r.t. different contexts. In DDS, both sequence-level and token-level adaptive search can be achieved to adjust the decoding process in a unified framework. Besides, our adaptive algorithm can not only be used during model inference, but it can also be applied during the model training stage to further enhance the performance. Comprehensive experiments indicate that the proposed decoding strategy can consistently improve the performance of pre-trained dialogue models when coupled with four well-used stochastic decoding algorithms.

## introduction building generative open-domain dialogue system is a significant yet challenging area of deep learning research. it has been widely recognized that the pre-training paradigm, in which large-scale transformer-based models are trained with massive amounts of conversational data, is an effective and promising approach. some of the more notable works in english include dialogpt @xcite , lamda @xcite , blender @xcite , and lately, chatgpt has attracted great attention and interest from researchers and the industry. for chinese dialogue models, eva @xcite , pangu-bot @xcite and plato @xcite @xcite are also excellent options. in recent research, however, it has been demonstrated that decoding strategies play an important role in performance even beyond model architecture @xcite , whereas standard strategies remain relatively unchanged @xcite . stochastic decoding algorithms are widely used for dialogue generation task. users expect varying responses from a chatbot when they input similar queries, or they tend to become bored and lose interest if it only responds with fixed reply. for such a chit-chat scenario, deterministic decoding algorithms, such as greedy search or beam search, are not suitable. additionally, even when using large pre-trained language models, decoding strategies that aim for high probability output, suffer from in-credible degeneration issue @xcite . consequently, dialogue generation models are inclined to employ stochastic sampling methods such as top-k sampling @xcite or nucleus sampling @xcite , where the probability distribution will be shaped by the temperature t . aside from chit-chat, however, there is another scenario for chatbots, namely factual question answering (qa). unfortunately, since the size of the decoding space required for two different dialog scenarios is different, stochastic sampling methods are not able to handle both simultaneously due to the unified and constant randomness of their decoding processes. as shown in to resolve the drawbacks of existing stochastic decoding algorithms, we propose a dynamic decoding strategy (dds) for dialogue generation, which can be combined with mainstream stochastic sampling. the key intuition of dynamic sampling is that the decoding space varies according to the context, therefore the shape of probability distribution should be adjusted adaptively. to achieve this goal, we incorporate an additional diversity predicting head into the dialogue generation model, which is capable of producing the score based on decoding diversity to guide the sampling process adaptively. it only introduces a few parameters and performs decoding at a similar speed to standard dialogue models. the labeled data for training the head is derived from the pre-trained model automatically. three types of mapping functions are designed, projecting the diversity score to the temperature for shaping the sampling distribution. in order to control the token generation in a more fine-grained manner, the regression head can be applied to each output token or the whole context, allowing us to control the randomness of decoding at both levels. apart from inference, adaptive temperature can also be introduced to dialogue training stage to balance the model prediction confidence. we perform extensive experiments on two union of datasets with two chinese pre-trained dialogue models. the results show that the dds can largely improve the performance of four sampling-based decoding algorithms. human evaluation is also conducted to ensure relevance and fluency of responses while improving diversity. in summary, our contributions are as follows: ## methodology we propose the dynamic decoding strategy to dynamically compute temperature t ′ w.r.t. different contexts, which replaces t in equation 4 for all four sampling methods outlined above. the value of this parameter t ′ will vary adaptively according to the size of the decoding space. in this section, we first describe how to build the labeled data about dialogue decoding diversity automatically. after that, we elaborate the regression head trained by it for predicting diversity scores on two levels, which will then be projected to temperature t ′ in accordance with three different mapping strategies. besides, the dynamic t ′ can also be applied to training stage. the overview of the proposed framework is illustrated in figure 1 .