Improving Multi-party Dialogue Generation via Topic and Rhetorical Coherence (2024)

Abstract: Previous studies on multi-party dialogue generation predominantly concentrated on modeling the reply-to structure of dialogue histories, always overlooking the coherence between generated responses and target utterances. To address this issue, we propose a Reinforcement Learning approach emphasizing both Topic and Rhetorical Coherence (RL-TRC). In particular, the topic- and rhetorical-coherence tasks are designed to enhance the model’s perception of coherence with the target utterance. Subsequently, an agent is employed to learn a coherence policy, which guides the generation of responses that are topically and rhetorically aligned with the target utterance. Furthermore, three discourse-aware rewards are developed to assess the coherence between the generated response and the target utterance, with the objective of optimizing the policy. The experimental results and in-depth analyses on two popular datasets demonstrate that our RL-TRC significantly outperforms the state-of-the-art baselines, particularly in generating responses that are more coherent with the target utterances.

## introduction in a two-party dialogue, a response is always generated for the last utterance in the dialogue history. this contrasts with multi-party dialogue, which always involves multiple participants and each utterance can be uttered by any participant and reply to any participant else @xcite . in a multiparty dialogue, the position of the target utterance (replied utterance) is not fixed. figure 1 illustrates a multi-party dialogue involving four participants. in this example, the bot of multi-party dialogue generation (mdg) is required to assume the role of participant p 1 and generate an appropriate response to target utterance u 2 . previous studies @xcite @xcite @xcite dataset. emmdg @xcite and madnet @xcite are two sota baselines. marily focused on leveraging the reply-to structure within dialogue history to enhance generation. however, these methods do not adequately guarantee the coherence between the generated response and the target utterance. firstly, the modeling of reply-to structures does not guarantee that the model will comprehend the topics of the dialogue history, which is of the utmost importance for generating responses that align with the topic of the target utterance. multi-party dialogue always has multiple entangled parallel dialogue flows, covering different topics. as shown in figure 1 , the example primarily concerns two topics, among which u 1 , u 2 , and u 7 (human response) discuss the browsers (firefox and opera) selection and resource usage, while u 3 , u 4 , u 5 , and u 6 primarily discuss the usage of emacs. since the reply-to structure does not unravel the entangled topics in a multi-party dialogue, it is challenging for the model to consistently generate a response that aligns with the target utterance's topic. as illustrated by the response generated by emmdg @xcite in figure 1 , the response's topic does not relate to the target utterance u 2 , despite emmdg modeling the reply-to structure. secondly, modeling reply-to structures does not guarantee logical and meaningful interactions between the generated response and the target utterance. as illustrated in figure 1 , while the response generated by madnet @xcite remains relevant to the topic (opera) of the target utterance, its internal contradictions undermine the logical coherence between it and the target utterance. to address these challenges, we propose a reinforcement learning approach emphasizing both topic and rhetorical coherence (rl-trc), guiding the generation of responses that are both topically and logically consistent with the target utterance. we initially design tasks for topic and rhetorical coherence, utilizing the target utterance to predict the topic of the generated response and its rhetorical relation with the target utterance. subsequently, our reinforcement learning agent learns a coherence policy to guide the model in generating responses that are topically and rhetorically consistent with the target utterance. to optimize this policy, we meticulously design two local rewards and one global reward. the local rewards guide policy learning by scoring the topic coherence and rhetorical coherence between the generated response and the target utterance, respectively. the global reward guides policy learning by measuring whether the target utterance in the dialogue history can be correctly recognized based on the generated response. we conduct extensive experiments on two popular ubuntu irc channel datasets, and results from both automated and manual evaluations indicate that our rl-trc significantly outperforms previous sota baselines. in-depth analysis reveals that rl-trc can generate responses for target utterances that are more coherent in both topic and rhetoric. furthermore, some studies have explored enhancing mdg from a discourse perspective. chernyavskiy and ilvovsky (2023) suggested using dialogue action planning to improve the consistency and quality of dialogue generation. meanwhile, @xcite proposed leveraging various fine-grained linguistic inputs, including abstract meaning representation, discourse relations, sentiment, and grounding information, to facilitate multi-party dialogue generation. recently, with the advent of large language models (llms), @xcite evaluated the performance of chatgpt and gpt-4 on mdg. despite significant progress in mdg, incoherence between the generated response and the target utterance remains a challenge that needs to be addressed. dialogue coherence modeling dialogue coherence modeling is primarily applied in personabased dialogue systems, aiming to help models generate responses that are consistent with the given persona @xcite . most previous work has focused on minimizing contradictions between responses and personas using natural language inference models @xcite @xcite , and over-sampling followed by post-evaluation @xcite . unlike previous work on the coherence between the generated response and the persona, we seek to enhance the coherence between the generated response and the target utterance in multi-party dialogues. uniquely, we model coherence from the perspectives of topic and rhetoric, addressing the multiple topic flows and logical structures in multi-party dialogues. ## method we propose an approach rl-trc for mdg, illustrated in figure 2 , that begins with using the dialogue history to initialize the sequence state of the reinforcement learning (rl) agent, which is dialogue history 在每个utterance增加一个[sep]代表utt 的表示 在histtory的开头增加一个cls代表history 的表示 s resfers to speaker u resfers to utterance rt is short for respond to history encoder topic reward r tc rhetorical reward r rc reply-to reward r rt topic coherence task rhetorical coherence task policy network decoder state s i action space a action a response state update state trigger sequence ts k decoder state s i+1 = [s i ; ] = + + state trigger sequence ts k i  figure 2: framework of our method. the parameters of both decoders are shared. ## analysis in this section, we focus on an in-depth analysis on the hu dataset, and the results on the ou5 dataset are presented in appendix e, g, and h. ## conclusion in this paper, we propose a reinforcement learning method based on discourse coherence for multiparty dialogue generation. by designing tasks centered on topic coherence and rhetorical coherence, we enable the model to perceive coherence with the target utterance. furthermore, a reinforcement agent is employed to guide the model to generate responses that are topically and rhetorically aligned with the target utterances. to optimize the agent, three types of discourse-aware rewards are designed to guide the model to maintain coherence with the target utterance. experimental results validate the effectiveness of our method. our future work will focus on how to optimize rhetorical coherence.