Enhancing Goal-oriented Proactive Dialogue Systems via Consistency Reflection and Correction
Authors: Didi Zhang, Yaxin Fan, Peifeng Li, Qiaoming Zhu
Accepted at: acl25

Enhancing Goal-oriented Proactive Dialogue Systems via Consistency Reflection and Correction

Didi Zhang*, Yaxin Fan*, Peifeng Li†, and Qiaoming Zhu
School of Computer Science and Technology, Soochow University, Suzhou, China
{ddzhang2023, yxfansuda}@stu.suda.edu.cn, {pfli, qmzhu}@suda.edu.cn

Abstract
Goal-oriented proactive dialogue systems are designed to guide user conversations seamlessly towards specific objectives by planning a goal-oriented path. However, previous research has focused predominantly on optimizing these paths while neglecting the inconsistencies that may arise between generated responses and dialogue contexts, including user profiles, dialogue history, domain knowledge, and sub-goals. To address this issue, we introduce a model-agnostic two-stage Consistency Reflection and Correction (CRC) framework. Specifically, in the consistency reflection stage, the model is prompted to reflect on the discrepancies between generated responses and dialogue contexts, identifying inconsistencies and suggesting possible corrections. In the consistency correction stage, the model generates responses that are more consistent with the dialogue context based on these reflection results. We conducted experiments on various model architectures with different parameter sizes, including encoder-decoder models (BART, T5) and decoder-only models (GPT-2, DialoGPT, Phi3, Mistral and LLaMA3), and the experimental results on three datasets demonstrate that our CRC framework significantly improves the consistency between generated responses and dialogue contexts. Our code is publicly available at: https://github.com/zhyidi/CRC.

1 Introduction
The Goal-oriented Proactive Dialogue System (GPDS) focuses on achieving specific objectives by actively guiding and anticipating user needs (Liu et al., 2020; Wang et al., 2024a, 2023b). Unlike traditional dialogue systems that passively respond to user requests (Touvron et al., 2023; Achiam et al., 2024), GPDS strategically steers the conversation along a goal-oriented path, ensuring that a goal is naturally achieved while maintaining a positive user experience. GPDS has a wide range of applications in various domains, such as recommendation systems (Fu et al., 2020; Liu et al., 2020, 2021) and medical consultations (Xu et al., 2024b).

Figure 1 presents an example of GPDS, which generates responses (e.g., \( S_i \)) based on the dialogue context, such as the user profile, dialogue history, domain knowledge, and subgoals within a goal-oriented path. GPDS can be divided into two primary sub-tasks: goal-oriented path planning and response generation. Initially, the system plans a goal-oriented path where each step is represented by an <action, topic> pair (e.g., “Q&A | Jimmy Lin’s constellation”→“Chat about the Star | Jimmy Lin”→“Movie recommendation | Grandpa’s Love” in Figure 1). Following this, the system generates responses aligned with the planned path, thereby guiding the conversation proactively and naturally toward achieving the final target (recommending the movie Grandpa’s Love).

Most previous studies on GPDS have primarily focused on planning goal-oriented paths using techniques such as CNN-based classifiers (Liu et al., 2020), Seq2seq paradigms (Deng et al., 2023; Wang et al., 2023c, 2024b.a), and graph interaction methods (Zhang et al., 2024). However, these approaches often overlook the inconsistencies that arise between generated responses and dialogue contexts. These inconsistencies manifest in several ways. First, there is an inconsistency with the dialogue history. As shown in Figure 1, the system asserts “of course he knows Lin” in \( S_2 \), even though the dialogue history does not inquire about his acquaintance with Lin. Second, there is an inconsistency with the subgoal. Although \( S_4 \)’s action is to recommend a movie, it fails to address the topic of Grandpa’s Love, resulting in an invalid recommendation. Third, there is an inconsistency with the domain knowledge. \( S_5 \) states that Yanping Zhu is the star of Grandpa’s Love, whereas the do-
Figure 1: An example of GPDS from the DuRecDial (Liu et al., 2020) dataset, where the system responses are generated by TPNet (Wang et al., 2024a).

main knowledge indicates that Zhu is the director of the movie. Lastly, there is an inconsistency with the user profile. The system might generate a response that does not align with the user’s profile, as illustrated in Appendix A. For instance, the user’s profile shows a preference for news about Nicholas Tse. However, the system recommends unrelated social news. Hence, these inconsistencies can lead to a poor user experience in real-world scenarios, causing conversations to abruptly break down and failing to achieve the intended targets.

To address these inconsistencies, we propose a Consistency Reflection and Correction (CRC) framework, drawing inspiration from the reflective practice theory (Checkoway and Schön, 1985) of human cognition that emphasizes the systematic reflection on experiences to identify and improve areas of weakness. Specifically, in the consistency reflection stage, we guide the model to reflect its experience, i.e., whether its generated responses are consistent with the elements of dialogue contexts. If any inconsistencies are identified, the model is prompted to categorize the types of discrepancies and suggest potential corrections. In the consistency correction stage, the model is instructed to regenerate the responses more consistent with dialogue contexts based on the insights gained from the reflection stage.

To validate the effectiveness of our framework, we conducted extensive experiments on three widely-used datasets: DuRecDial (Liu et al., 2020), DuRecDial 2.0 (Liu et al., 2021) and TopDial (Wang et al., 2023b). Since our framework is model-agnostic, we tested it on different model architectures and various parameter sizes, including encoder-decoder models (BART and T5) and decoder-only models (GPT-2, DialoGPT, Phi3-3.8B, Mistral-7B and LLaMA3-8B). The experimental results demonstrate that our CRC framework significantly improves the consistency between generated responses and dialogue context.

2 Related Work

2.1 Goal-oriented Proactive Dialogue System

Previous studies on GPDS typically began by planning a sequence of subgoals, followed by generating responses based on the subgoals to guide the conversation toward specific objectives. Most of them concentrated on goal-oriented path planning, employing the techniques such as CNN-base classifier (Liu et al., 2020), target-driven method (Wang et al., 2022, 2024a), Brownian bridge (Wang et al., 2023c), prompt-based method (Deng et al., 2023), graph-grounded planning (Liu et al., 2023), graph-interaction planning (Zhang et al., 2024), and bidirectional planning (Wang et al., 2024b).

However, these efforts often emphasize the importance of planning a goal-oriented path, overlooking the consistency between generated responses and dialogue contexts. This paper primarily fo-
cuses on response generation, aiming to improve GPDS by enhancing the consistency between generated responses and dialogue contexts.

2.2 Discourse Consistency in Dialogue

Discourse consistency in dialogue refers to the logical coherence and uniformity of information and themes, which is essential for fostering understanding and effective communication among participants. Previous research (Song et al., 2021; Wang et al., 2023a; Chen et al., 2023; Zhou et al., 2023) has frequently employed Natural Language Inference (NLI) models to assess and enhance discourse consistency between generated responses and dialogue contexts. However, these NLI models often depend on additional training data, which can hinder their generalizability. In an innovative approach, we propose leveraging the model’s inherent reflective capabilities to enhance the consistency between generated responses and dialogue contexts, thereby improving its generalizability.

3 Task Definition

Given a dataset \( D = \{U^i, K^i, H^i, G^i\}_{i=1}^N \), where \( N \) is the size of the dataset. \( U^i = \{u_j^i\}_{j=1}^{N_i} \) is the \( i \)-th user profile, where each item \( u_j^i \) is a key-value pair, representing the user’s personal information (e.g., name and gender). \( K^i = \{k_j^i\}_{j=1}^{N_k} \) is the domain knowledge related to the \( i \)-th conversation of \( D \), where each item \( k_j^i \) is a triple <head, relation, tail>. \( H^i = \{h_m^i\}_{m=1}^M \) is the content of the \( i \)-th conversation, consisting of \( M \) turns. \( G^i = \{g_m^i\}_{m=1}^M \) is the goal-oriented path for the dialogue \( H^i \), where each \( g_m^i \) consists of a dialogue action \( a_m^i \) and a dialogue topic \( t_m^i \). The final goal of the dialogue is represented by \( g_M^i \). GPDS can be divided into two primary sub-tasks: goal-oriented path planning and response generation.

Goal-oriented Path Planning Goal-oriented path planning aims to plan a sequence of subgoals to proactively guide the conversation to achieve the final target \( g_M^i \). Each subgoal \( g_m^i (1 \leq m \leq M) \) which is formulated as follows.

\[
g_m^i = GPP(U^i, K^i, H_{\leq m}^i, G_{<m}^i)
\]

where \( GPP \) is a path prediction model, which has attracted the attention of most previous work. In this paper, we adopted the same path prediction model as Wang et al. (2024a) and mainly focus on generating responses that are consistent with dialogue contexts.

Response Generation A generative model is employed to generate a response that aligns with the action and topic in \( g_m^i \), thereby actively steering the conversation towards the final target. This process is represented as follows.

\[
r_m^i = RG(U^i, K^i, H_{\leq m}^i, g_m^i)
\]

where \( r_m^i \) is the generated response, and \( RG \) denotes an autoregressive model. Specifically, \( RG \) autoregressively generates \( r_m^i \) conditioned on the concatenation of the dialogue context, and it is optimized by minimizing the negative log-likelihood as follows.

\[
\mathcal{L}(\theta) = - \sum_{i=1}^N \sum_{t=1}^T \log P(r_{m,t}^i \mid r_{m,<t}^i, U^i, K^i, H_{\leq m}^i, g_m^i; \theta)
\]

where \( \theta \) represents the trainable parameters, \( r_{m,t}^i \) and \( r_{m,<t}^i \) are the \( t \)-th token and the previous \( t-1 \) tokens of the response \( r_m^i \), respectively, and \( T \) is the length of \( r_m^i \). In this paper, we focus on improving \( RG \) to enhance the consistency between generated responses and dialogue contexts.

4 CRC Framework

Motivation Most of the Response Generation (RG) models in GPDS cannot produce responses that align with dialogue context. This is likely because their learning experience primarily involves imitation learning, which lacks deep reflection and correction mechanisms. According to the theory of reflective practice (Checkoway and Schön, 1985), the process of learning and growth comes from a cycle of experience, i.e., reflection and correction. Without this reflective practice, most models struggle to learn from their own experiences and summarize effectively, making it challenging for them to consistently generate responses that align with the dialogue context. Therefore, establishing a framework that can involve the reflection and correction mechanism to improve the consistency between responses and dialogue contexts is crucial.

In this paper, we proposed a model-agnostic, two stage CRC framework including consistency reflection and consistency correction, as shown in Figure 2. In the consistency reflection stage, we first guide a \( RG \) model to reflect on its experience, i.e., reflecting on the types of inconsistency between generated responses and dialogue contexts, and then suggest possible corrections. In the correction
stage, we further guide the RG model to regenerate responses that are consistent with dialogue contexts on the reflection results.

Consistency Reflection As introduced in Section 1, the responses generated by the RG model may exhibit inconsistencies with dialogue context. These inconsistencies primarily pertain to the user profile \( U \), the domain knowledge \( K \), the dialogue history \( H \), and the subgoal \( g \). To address this issue, we prompt the model to reflect on and consider ways to correct these inconsistencies. Specifically, we not only ask the RG model to generate responses, but also encourage it to analyze the types of inconsistencies between dialogue responses and dialogue contexts, providing suggestions for improvement. This can be formalized as follows.

\[
r^i_m, e^i_m, s^i_m = RG\_R(U^i, K^i, H^i_{\leq m}, g^i_m)
\]

where \( RG\_R \) represents an autoregressive model with reflective ability, \( e^i_m \) denotes the inconsistent type of the reflection results on the response \( r^i_m \), and \( s^i_m \) is a correction suggestion on the inconsistent type \( e^i_m \). Therefore, the key lies in how to obtain the high-quality inconsistent type \( e \) and the correction suggestion \( s \) to stimulate the model’s reflective capabilities.

While manually annotating inconsistency types and correction suggestions is an ideal approach, the time-consuming and costly nature of manual annotation hinders its practical application. Thanks to the powerful understanding capabilities of Large Language Models (LLMs) like ChatGPT, which have already achieved success in data annotation across various fields (Wang et al., 2023d; Xu et al., 2024a), we utilize ChatGPT \( ^1 \) to act as an annotator for the annotations of inconsistency types and correction suggestions. The prompt used is illustrated in Appendix B.

We feed the dialogue context and the generated response to ChatGPT and let it evaluate the consistencies. ChatGPT first requires to identify the inconsistency types and then provide correction suggestions. An example is shown in Figure 2, where the inconsistency type is related to the domain knowledge, and the correction suggestion is that “Yanping Zhu is the director, and the stars are Zhiying Lin, Steven Hao, and Long Huang.”. After obtaining the annotations \( e \) and \( s \), we continue to fine-tune the RG model to obtain a new RG_R model with reflective capabilities. Let the concatenation of \( r^i_m, e^i_m, \) and \( s^i_m \) be denoted as \( c^i_m \), the optimization of RG_R is as follows.

\[
\mathcal{L}_{cr}(\theta) = - \sum_{i=1}^{N} \sum_{t=1}^{T} \log P(c^i_{m,t} \mid c^i_{m,<t}, U^i, K^i, H^i_{\leq m}, g^i_m; \theta)
\]

where \( \theta \) represents the trainable parameters, \( N \) is the data size and \( T \) is the token length of \( c^i_m \). During the learning process, the RG_R model needs to generate not only the response \( r \) but also both the reflective results \( e \) and \( s \) regarding \( r \).

Consistency Correction During the consistency correction phase, we continue to train RG to generate the response \( r^i_m \) that is more consistent with the dialogue context on the reflective results from RG_R. This can be formalized as follows.

\[
r^{i'}_m = RG\_C(U^i, K^i, H^i_{\leq m}, g^i_m, e^i_m)
\]

\( ^1 \)The version used is GPT-4o-2024-05-13.

![Overview of our CRC framework.](page_186_172_1077_393.png)

Figure 2: Overview of our CRC framework.
where \( RG_C \) represents an autoregressive model with correction ability. Similar with Equ 5, \( RG_C \) is trained by minimizing the negative log-likelihood as follows.

\[
\mathcal{L}_{cc}(\theta) = - \sum_{i=1}^{N} \sum_{t=1}^{T} \log P(r_{m,t}^{i'} | r_{m,<t}^{i'}, U^i, \\
K^i, H_{m,m}^i, g_m^i, c_m^i; \theta)
\]

Training The training process is primarily divided into three stages. First, we train an initial model \( RG \) by optimizing \( \mathcal{L} \). Next, we enhance \( RG \) by optimizing \( \mathcal{L}_{cr} \) to obtain \( RG_R \), which possesses reflective capabilities. Finally, we further optimize \( RG \) by optimizing \( \mathcal{L}_{cc} \) to achieve \( RG_C \), which incorporates corrective capabilities.

Inference During the inference phase, we first feed the dialogue context into \( RG_R \) to obtain \( c \), which includes the response \( r \), the inconsistency type \( e \), and the correction suggestion \( s \). Next, we feed both the dialogue context and \( c \) into \( RG_C \) to generate a response \( r' \) that is more consistent with the dialogue context.

5 Experimentation

5.1 Experimental Settings

Datasets We conducted experiments on three widely recognized datasets: DuRecDial (Liu et al., 2020), DuRecDial 2.0 (Liu et al., 2021) and Top-Dial (Wang et al., 2023b). We followed the data processing procedures and splits outlined in previous work (Wang et al., 2024a; Zhang et al., 2024) and the statistics are presented in Appendix C.

Baselines We compared our CRC framework with several state-of-the-art baselines as follows. MGCG (Liu et al., 2020) utilizes CNN for goal prediction and employs modified generation-based models for response generation. UniMIND (Deng et al., 2023) unifies goal planning and response generation using prompt-based learning. TCP (Wang et al., 2022) uses a Transformer-based planner to generate a sequence of actions and topic paths to guide response generation. MGNN (Liu et al., 2023) employs graph neural networks to model complex interactions between dialogue elements. GIGF (Zhang et al., 2024) utilizes a directed heterogeneous graph to capture goal sequence information across different levels. TPNet (Wang et al., 2024a) is an enhanced version of TCP that leverages several pre-trained models, including BART (denoted as TP-BART), GPT-2 (denoted as TP-GPT2), and DialoGPT (denoted as TP-Dial). In this paper, we adopted the same goal-oriented path as TPNet, primarily focusing on response generation. In addition to the aforementioned language models, we also applied our CRC framework to T5 (denoted as TP-T5), Phi3 (denoted as TP-Phi3), Mistral (denoted as TP-Mistral) and LLaMA3 (denoted as TP-LLaMA3). Furthermore, we employed the golden goal-oriented path on LLaMA3 (denoted as Golden-LLaMA3) to demonstrate the general applicability of our CRC framework, independent of the performance of the goal-oriented path planning task.

Evaluation Metrics We follow previous work (Wang et al., 2024a) and use the following metrics: Word-level F1 (W F1), BLEU, Distinct (Dist), Knowledge F1 (K F1), and Goal Success Rate (Succ). Word-level F1 measures the exact word overlap between generated and reference responses. BLEU measures the n-gram overlap with reference responses. Distinct evaluates the diversity of the generated responses. Knowledge F1 measures the correctness of generated knowledge against domain knowledge triples. Goal Success Rate evaluates whether the dialogue successfully achieves both the target action and topic.

Implementation Details Please refer to Appendix D for details.

5.2 Experimental Results

The results of response generation on the three datasets are presented in Tables 1 and 2, as well as in Table 12 in Appendix E. It can be observed that our CRC substantially enhances the performance of various model architectures across multiple metrics, showing notable improvements in Word-level F1, BLEU-2, Knowledge F1, and Goal Success Rate, while having minimal impact on Distinct. This suggests that by improving the consistency between generated responses and dialogue contexts, GPDS can more effectively guide conversations toward final targets without compromising the diversity of the responses. These findings demonstrate the effectiveness and generality of our CRC framework.

The observed enhancements in the Word-level F1 and BLEU scores suggest that our framework enables the model to generate responses that more closely match the reference responses. The notable improvement in Knowledge F1 can be attributed to our CRC framework, which prompts the model
<table>
  <tr>
    <th>Method</th>
    <th>W F<sub>1</sub></th>
    <th>BLEU-2</th>
    <th>Dist-2</th>
    <th>K F<sub>1</sub></th>
    <th>Succ</th>
  </tr>
  <tr>
    <td>MGGC</td>
    <td>33.48</td>
    <td>0.203</td>
    <td>0.043</td>
    <td>35.12</td>
    <td>46.80</td>
  </tr>
  <tr>
    <td>UniMIND</td>
    <td>40.58</td>
    <td>0.231</td>
    <td>0.078</td>
    <td>44.51</td>
    <td>-</td>
  </tr>
  <tr>
    <td>TCP</td>
    <td>41.40</td>
    <td>0.299</td>
    <td>0.072</td>
    <td>48.63</td>
    <td>68.57</td>
  </tr>
  <tr>
    <td>MGNN</td>
    <td>43.50</td>
    <td>0.274</td>
    <td>0.064</td>
    <td>45.00</td>
    <td>-</td>
  </tr>
  <tr>
    <td>GIGF</td>
    <td>47.52</td>
    <td>0.348</td>
    <td>0.078</td>
    <td>56.02</td>
    <td>-</td>
  </tr>
  <tr>
    <td>TP-BART<sub>(140M)</sub></td>
    <td>37.22</td>
    <td>0.255</td>
    <td>0.083</td>
    <td>44.52</td>
    <td>71.50</td>
  </tr>
  <tr>
    <td>TP-BART w/ CRC</td>
    <td>42.44<sup>±1.22</sup></td>
    <td>0.280<sup>±0.025</sup></td>
    <td>0.073</td>
    <td>51.33<sup>±7.01</sup></td>
    <td>75.78<sup>±4.28</sup></td>
  </tr>
  <tr>
    <td>TP-T5<sub>(390M)</sub></td>
    <td>36.86</td>
    <td>0.250</td>
    <td>0.080</td>
    <td>50.11</td>
    <td>55.68</td>
  </tr>
  <tr>
    <td>TP-T5 w/ CRC</td>
    <td>41.19<sup>±4.33</sup></td>
    <td>0.278<sup>±0.028</sup></td>
    <td>0.077</td>
    <td>54.47<sup>±4.36</sup></td>
    <td>72.25<sup>±16.57</sup></td>
  </tr>
  <tr>
    <td>TP-GPT2<sub>(102M)</sub></td>
    <td>41.53</td>
    <td>0.301</td>
    <td>0.075</td>
    <td>48.81</td>
    <td>74.70</td>
  </tr>
  <tr>
    <td>TP-GPT2 w/ CRC</td>
    <td>46.75<sup>±1.22</sup></td>
    <td>0.344<sup>±0.043</sup></td>
    <td>0.074</td>
    <td>54.77<sup>±5.96</sup></td>
    <td>75.67<sup>±0.97</sup></td>
  </tr>
  <tr>
    <td>TP-Dial<sub>(99M)</sub></td>
    <td>31.98</td>
    <td>0.262</td>
    <td>0.041</td>
    <td>35.68</td>
    <td>41.66</td>
  </tr>
  <tr>
    <td>TP-Dial w/ CRC</td>
    <td>43.76<sup>±11.78</sup></td>
    <td>0.323<sup>±0.061</sup></td>
    <td>0.062</td>
    <td>52.66<sup>±16.98</sup></td>
    <td>72.65<sup>±30.99</sup></td>
  </tr>
  <tr>
    <td>TP-Phi3<sub>(3.8B)</sub></td>
    <td>39.95</td>
    <td>0.253</td>
    <td>0.082</td>
    <td>43.08</td>
    <td>62.78</td>
  </tr>
  <tr>
    <td>TP-Phi3 w/ CRC</td>
    <td>44.67<sup>±4.72</sup></td>
    <td>0.275<sup>±0.022</sup></td>
    <td>0.082</td>
    <td>49.18<sup>±6.10</sup></td>
    <td>69.54<sup>±6.76</sup></td>
  </tr>
  <tr>
    <td>TP-Mistral<sub>(TB)</sub></td>
    <td>33.12</td>
    <td>0.275</td>
    <td>0.060</td>
    <td>42.38</td>
    <td>61.45</td>
  </tr>
  <tr>
    <td>TP-Mistral w/ CRC</td>
    <td>36.78<sup>±3.66</sup></td>
    <td>0.311<sup>±0.036</sup></td>
    <td>0.061</td>
    <td>48.63<sup>±6.25</sup></td>
    <td>71.92<sup>±10.47</sup></td>
  </tr>
  <tr>
    <td>TP-LLaMA3<sub>(8B)</sub></td>
    <td>40.24</td>
    <td>0.276</td>
    <td>0.095</td>
    <td>51.11</td>
    <td>60.72</td>
  </tr>
  <tr>
    <td>TP-LLaMA3 w/ CRC</td>
    <td>45.96<sup>±5.72</sup></td>
    <td>0.318<sup>±0.042</sup></td>
    <td>0.095</td>
    <td>56.86<sup>±5.75</sup></td>
    <td>75.03<sup>±14.31</sup></td>
  </tr>
  <tr>
    <td>Golden-LLaMA3</td>
    <td>43.57</td>
    <td>0.301</td>
    <td>0.099</td>
    <td>53.43</td>
    <td>65.64</td>
  </tr>
  <tr>
    <td>Golden-LLaMA3 w/ CRC</td>
    <td>47.03<sup>±3.46</sup></td>
    <td>0.317<sup>±0.016</sup></td>
    <td>0.089</td>
    <td>57.76<sup>±4.33</sup></td>
    <td>80.76<sup>±15.12</sup></td>
  </tr>
</table>

Table 1: Experimental results on the Chinese DuRecDial dataset. The parameter sizes of the models are annotated as subscripts adjacent to the model names.

<table>
  <tr>
    <th>Method</th>
    <th>W F<sub>1</sub></th>
    <th>BLEU-2</th>
    <th>Dist-2</th>
    <th>K F<sub>1</sub></th>
    <th>Succ</th>
  </tr>
  <tr>
    <td>MGGC</td>
    <td>32.26</td>
    <td>0.182</td>
    <td>0.051</td>
    <td>29.35</td>
    <td>32.20</td>
  </tr>
  <tr>
    <td>UniMIND</td>
    <td>33.66</td>
    <td>0.184</td>
    <td>0.135</td>
    <td>23.75</td>
    <td>-</td>
  </tr>
  <tr>
    <td>TCP</td>
    <td>33.12</td>
    <td>0.201</td>
    <td>0.070</td>
    <td>34.86</td>
    <td>-</td>
  </tr>
  <tr>
    <td>MGNN</td>
    <td>36.75</td>
    <td>0.194</td>
    <td>0.073</td>
    <td>31.32</td>
    <td>-</td>
  </tr>
  <tr>
    <td>GIGF</td>
    <td>38.35</td>
    <td>0.241</td>
    <td>0.089</td>
    <td>66.91</td>
    <td>-</td>
  </tr>
  <tr>
    <td>TP-BART</td>
    <td>36.28</td>
    <td>0.204</td>
    <td>0.093</td>
    <td>40.22</td>
    <td>63.60</td>
  </tr>
  <tr>
    <td>TP-BART w/ CRC</td>
    <td>38.87<sup>±2.59</sup></td>
    <td>0.210<sup>±0.006</sup></td>
    <td>0.094</td>
    <td>60.62<sup>±20.40</sup></td>
    <td>65.37<sup>±11.77</sup></td>
  </tr>
  <tr>
    <td>TP-T5</td>
    <td>36.26</td>
    <td>0.233</td>
    <td>0.076</td>
    <td>42.98</td>
    <td>53.48</td>
  </tr>
  <tr>
    <td>TP-T5 w/ CRC</td>
    <td>38.44<sup>±2.18</sup></td>
    <td>0.229</td>
    <td>0.076</td>
    <td>63.58<sup>±20.60</sup></td>
    <td>56.92<sup>±3.44</sup></td>
  </tr>
  <tr>
    <td>TP-GPT2</td>
    <td>34.62</td>
    <td>0.217</td>
    <td>0.082</td>
    <td>38.80</td>
    <td>60.70</td>
  </tr>
  <tr>
    <td>TP-GPT2 w/ CRC</td>
    <td>36.02<sup>±1.40</sup></td>
    <td>0.219<sup>±0.002</sup></td>
    <td>0.083</td>
    <td>59.73<sup>±20.93</sup></td>
    <td>63.78<sup>±3.08</sup></td>
  </tr>
  <tr>
    <td>TP-Dial</td>
    <td>34.45</td>
    <td>0.214</td>
    <td>0.074</td>
    <td>36.15</td>
    <td>52.87</td>
  </tr>
  <tr>
    <td>Decoder-Only</td>
    <td>TP-Dial w/ CRC</td>
    <td>36.34<sup>±1.89</sup></td>
    <td>0.215<sup>±0.001</sup></td>
    <td>0.077</td>
    <td>58.17<sup>±22.02</sup></td>
    <td>55.05<sup>±2.18</sup></td>
  </tr>
  <tr>
    <td>TP-Phi3</td>
    <td>35.72</td>
    <td>0.225</td>
    <td>0.093</td>
    <td>43.39</td>
    <td>50.59</td>
  </tr>
  <tr>
    <td>TP-Phi3 w/ CRC</td>
    <td>38.56<sup>±2.84</sup></td>
    <td>0.231<sup>±0.006</sup></td>
    <td>0.092</td>
    <td>57.45<sup>±14.06</sup></td>
    <td>53.19<sup>±2.60</sup></td>
  </tr>
  <tr>
    <td>TP-Mistral</td>
    <td>34.56</td>
    <td>0.215</td>
    <td>0.082</td>
    <td>41.50</td>
    <td>51.45</td>
  </tr>
  <tr>
    <td>TP-Mistral w/ CRC</td>
    <td>36.72<sup>±2.16</sup></td>
    <td>0.228<sup>±0.013</sup></td>
    <td>0.082</td>
    <td>54.93<sup>±13.43</sup></td>
    <td>53.12<sup>±1.67</sup></td>
  </tr>
  <tr>
    <td>TP-LLaMA3</td>
    <td>36.00</td>
    <td>0.230</td>
    <td>0.095</td>
    <td>43.72</td>
    <td>51.85</td>
  </tr>
  <tr>
    <td>TP-LLaMA3 w/ CRC</td>
    <td>38.54<sup>±2.54</sup></td>
    <td>0.233<sup>±0.003</sup></td>
    <td>0.096</td>
    <td>65.91<sup>±22.19</sup></td>
    <td>53.99<sup>±2.14</sup></td>
  </tr>
  <tr>
    <td>Golden-LLaMA3</td>
    <td>37.02</td>
    <td>0.231</td>
    <td>0.096</td>
    <td>46.19</td>
    <td>52.92</td>
  </tr>
  <tr>
    <td>Golden-LLaMA3 w/ CRC</td>
    <td>38.92<sup>±1.90</sup></td>
    <td>0.233<sup>±0.002</sup></td>
    <td>0.096</td>
    <td>66.20<sup>±20.01</sup></td>
    <td>56.37<sup>±3.45</sup></td>
  </tr>
</table>

Table 2: Experimental results on English DuRecDial 2.0.

to address inconsistencies between responses and domain knowledge, thereby enhancing the model’s ability to accurately utilize the domain knowledge. Likewise, the increase in Goal Success Rate is due to the fact that CRC can guide the model to identify and rectify discrepancies between responses and subgoals. By ensuring greater consistency between responses and each subgoal in the goal-oriented path, the model is better equipped to achieve the final objective.

Comparing TP-LLaMA3 with other pre-trained models, TP-LLaMA3 has a natural advantage in terms of the Distinct and Knowledge \( F_1 \) metrics. This may be attributed to its incorporation of more diverse dialogue scenarios and knowledge-intensive corpora during the pre-training stage,
<table>
  <tr>
    <th>Method</th>
    <th>W F<sub>1</sub></th>
    <th>BLEU-2</th>
    <th>Dist-2</th>
    <th>K F<sub>1</sub></th>
    <th>Succ</th>
  </tr>
  <tr>
    <td>CRC</td>
    <td>45.96</td>
    <td>0.318</td>
    <td>0.095</td>
    <td>56.86</td>
    <td>75.03</td>
  </tr>
  <tr>
    <td>w/o UP</td>
    <td>43.48</td>
    <td>0.306</td>
    <td>0.095</td>
    <td>56.30</td>
    <td>73.00</td>
  </tr>
  <tr>
    <td>w/o DH</td>
    <td>43.62</td>
    <td>0.311</td>
    <td>0.093</td>
    <td>55.73</td>
    <td>73.49</td>
  </tr>
  <tr>
    <td>w/o DK</td>
    <td>42.54</td>
    <td>0.299</td>
    <td>0.098</td>
    <td>52.25</td>
    <td>71.53</td>
  </tr>
  <tr>
    <td>w/o SG</td>
    <td>42.24</td>
    <td>0.297</td>
    <td>0.096</td>
    <td>52.65</td>
    <td>65.66</td>
  </tr>
</table>

Table 3: Ablation results using TP-LLaMA3 on DuRecDial where UP, DH, DK and SG refer to user profile, dialogue history, domain knowledge and subgoal, respectively.

which enhances its ability to generate diverse responses and accurately utilize domain knowledge. Notably, our CRC can still significantly enhance the performance of TP-LLaMA3 across various metrics, indicating that our framework remains effective even for those LLMs with a larger number of parameters and can be effective supplement to LLMs.

Additionally, in comparison with TP-LLaMA3, Golden-LLaMA3 using annotated goal-oriented path improves all metrics on two datasets and these results indicate that the performance improvements in the task of goal-oriented path planning can enhance GPDS. Our CRC can further boost the performance of Golden-LLaMA3, demonstrating its generality, regardless of the performance of the goal-oriented path planning. Besides, the performance gaps between Golden-LLaMA3 and TP-LLaMA3 are relatively small, suggesting that further improvements in goal-oriented path planning may offer limited benefits for GPDS.

6 Analysis

6.1 Ablation Study

Our CRC aims to enhance response generation by improving consistency with the dialogue context, including the user profile, dialogue history, domain knowledge, and subgoals. Ablation experiments on the DuRecdial dataset using TP-LLaMA3 (see Table 3) reveal that removing any element leads to a performance decline in all metrics, highlighting the effectiveness of our CRC framework in maintaining and enhancing consistency with each element.

The removal of reflection and correction related to domain knowledge (w/o DK) significantly reduces Knowledge F1, highlighting the importance of consistency with domain knowledge for effective information utilization. Similarly, without subgoals (w/o SG), the Goal Success Rate drops markedly, demonstrating the importance of aligning responses with subgoals. The absence of user profiles (w/o UP) and dialogue history (w/o DH) negatively impacts all metrics except Distinct, showing the benefits of maintaining consistency with user profiles and dialogue history in enhancing GPDS.

Additionally, regardless of which element is removed, the performance on the Distinct metric remains almost unchanged. This demonstrates that our CRC framework not only improves the model’s ability to effectively guide conversations towards final targets but also maintains the model’s capacity to generate diverse responses.

6.2 Consistency Analysis

We conducted a pairwise human evaluation to compare the models with (w/) and without (w/o) CRC, assessing the consistency of the responses generated with each element of the dialogue context. We randomly selected 500 pairs of system responses from the DuRecDial dataset. The pairwise human evaluation results for TP-LLaMA3 are shown in Figure 3. The labels “win”, “tie”, and “lose” are used to indicate that TP-LLaMA3 w/ CRC is more consistent, equally consistent, or less consistent than TP-LLaMA3 w/o CRC, respectively. Appendix F provides the details of human evaluation.

The tie rates for TP-LLaMA3 w/ CRC and w/o CRC decrease across user profile (UP), dialogue history (DH), domain knowledge (DK), and the subgoal (SG). This suggests that the challenge of generating responses consistent with these elements becomes more pronounced.

Notably, TP-LLaMA3 w/ CRC exhibits a higher win rate compared to TP-LLaMA3 w/o CRC across all four elements. This underscores that our CRC framework effectively improves the consistency of the TP-LLaMA3’s responses and the elements of the dialogue context. Appendix F provides the

![Pairwise evaluation results for TP-LLaMA3 w/ CRC vs. TP-LLaMA3 w/o CRC.](page_1012_120_410_246.png)

Figure 3: Pairwise evaluation results for TP-LLaMA3 w/ CRC vs. TP-LLaMA3 w/o CRC.
results of human evaluation using TP-BART, TP-T5, TP-GPT2 and TP-DialoGPT, which illustrate the same trend and further ensure the effectiveness of our CRC.

6.3 SubGoals Failure Analysis
It is essential for a goal-oriented proactive dialogue system to seamlessly steer the conversation towards the ultimate objective by generating the responses that align with each subgoal along the goal-oriented path. Consequently, we assessed whether each subgoal was successfully accomplished. Figure 4 presents the subgoal failure rates (the rate at which the model are unable to achieve the subgoals) on DuRecDial for the models both without and with CRC. It is evident that the models without CRC exhibit a higher rate of current turn goal failures, with a percentage exceeding 20%. Such failures in subgoals have the potential to diminish the naturalness of the conversation, which may in turn lead to a poor user experience and complicate the achievement of the final objective. In contrast, our CRC framework has been demonstrated to significantly reduce the failure rate of subgoals, thereby facilitating a more natural conversational process. A similar trend is evident on the DuRecDial 2.0, as illustrated in Appendix G.

![Bar chart showing subgoal failure rates on DuRecDial for different models](page_184_120_579_246.png)

Figure 4: Subgoal failure rates on DuRecDial.

The results show that ChatGPT correctly identified 94% (245/261) of inconsistencies and generated accurate corrective suggestions for 97% (237/245) of inconsistent cases (i.e., generating reasonable reflective content), demonstrating the high reliability of using ChatGPT for annotating reflection data. In contrast, our reflection model correctly identified 94% (227/242) of inconsistencies and provided accurate suggestions for 90% (205/227) of the inconsistent cases. These results indicate that the reflection model successfully learned to identify and analyze inconsistencies from the annotations provided by ChatGPT, further validating the effectiveness of our CRC framework.

6.5 Analysis of Model Combination
We examine the performance when the reflection model (\( RG_R \)) and the correction model (\( RG_C \)) differ in architecture, as detailed in table 4 and table 5.

The results in Table 4 show that incorporating any models as a correction model significantly enhances performance compared to a setup without CRC. However, the impact varies across different metrics. For instance, utilizing T5 as a correction model enhances K F\(_1\) relative to DialoGPT; however, it does not demonstrate substantial benefits in other metrics. Conversely, BART demonstrates a marked enhancement in Dist-2 and Succ metrics. It is noteworthy that utilizing a more large model, such as LLaMA3, as the correction model, results in substantial enhancements across all metrics compared to DialoGPT, with the exception of BLEU-2. The BLEU score is calculated as the overlap between generated and reference responses, so increased diversity in the generated responses (as indicated by Dist-2) may have a negative effect on the BLEU-2 score. These findings imply that employing a correction model with a greater number of parameters can yield substantial performance enhancements.

Similarly, Table 5 demonstrates that using any framework model as a reflection model results in a substantial enhancement of performance in comparison to a model devoid of CRC. However, it is important to note that as the parameter size of the reflection model increases, its impact on the same correction model remains consistent.

In summary, the employment of a framework or parameter size for reflection or correction models has been shown to significantly enhance the initial model’s performance. It is notable that a correc-
<table>
  <tr>
    <th>Reflection Model (\( RG_R \))</th>
    <th>Correction Model (\( RG_C \))</th>
    <th>W F<sub>1</sub></th>
    <th>BLEU-2</th>
    <th>Dist-2</th>
    <th>K F<sub>1</sub></th>
    <th>Succ</th>
  </tr>
  <tr>
    <td>-</td>
    <td>-</td>
    <td>31.98</td>
    <td>0.262</td>
    <td>0.041</td>
    <td>35.68</td>
    <td>41.66</td>
  </tr>
  <tr>
    <td>DialoGPT</td>
    <td>DialoGPT</td>
    <td>43.76</td>
    <td>0.323</td>
    <td>0.062</td>
    <td>52.66</td>
    <td>72.65</td>
  </tr>
  <tr>
    <td>DialoGPT</td>
    <td>T5</td>
    <td>41.02</td>
    <td>0.265</td>
    <td>0.062</td>
    <td>54.20</td>
    <td>72.15</td>
  </tr>
  <tr>
    <td>DialoGPT</td>
    <td>BART</td>
    <td>41.72</td>
    <td>0.266</td>
    <td>0.074</td>
    <td>50.87</td>
    <td>75.55</td>
  </tr>
  <tr>
    <td>DialoGPT</td>
    <td>LLaMA3</td>
    <td>45.22</td>
    <td>0.300</td>
    <td>0.089</td>
    <td>56.28</td>
    <td>74.81</td>
  </tr>
</table>

Table 4: Performance comparison on DuRecDial when the reflection model are DialoGPT and the correction model uses different model architectures.

<table>
  <tr>
    <th>Reflection Model (\( RG_R \))</th>
    <th>Correction Model (\( RG_C \))</th>
    <th>W F<sub>1</sub></th>
    <th>BLEU-2</th>
    <th>Dist-2</th>
    <th>K F<sub>1</sub></th>
    <th>Succ</th>
  </tr>
  <tr>
    <td>-</td>
    <td>-</td>
    <td>31.98</td>
    <td>0.262</td>
    <td>0.041</td>
    <td>35.68</td>
    <td>41.66</td>
  </tr>
  <tr>
    <td>DialoGPT</td>
    <td>DialoGPT</td>
    <td>43.76</td>
    <td>0.323</td>
    <td>0.062</td>
    <td>52.66</td>
    <td>72.65</td>
  </tr>
  <tr>
    <td>T5</td>
    <td>DialoGPT</td>
    <td>43.52</td>
    <td>0.298</td>
    <td>0.072</td>
    <td>52.96</td>
    <td>72.39</td>
  </tr>
  <tr>
    <td>BART</td>
    <td>DialoGPT</td>
    <td>43.60</td>
    <td>0.324</td>
    <td>0.063</td>
    <td>52.54</td>
    <td>73.85</td>
  </tr>
  <tr>
    <td>LLaMA3</td>
    <td>DialoGPT</td>
    <td>44.23</td>
    <td>0.328</td>
    <td>0.065</td>
    <td>53.48</td>
    <td>73.62</td>
  </tr>
</table>

Table 5: Performance comparison on DuRecDial when the correction model are DialoGPT and the reflection model uses different model architectures.

<table>
  <tr>
    <th>Type</th>
    <th>System Response</th>
  </tr>
  <tr>
    <td colspan="2"><i>Consistency with the dialogue history</i></td>
  </tr>
  <tr>
    <td>w/o CRC</td>
    <td>You're welcome. Jimmy Lin is a character in Gu Long's classic TV series. <u>Of course I know it.</u></td>
  </tr>
  <tr>
    <td>w/ CRC</td>
    <td>You're welcome. Jimmy Lin is a character in Gu Long's classic TV series.</td>
  </tr>
  <tr>
    <td colspan="2"><i>Consistency with the domain knowledge</i></td>
  </tr>
  <tr>
    <td>w/o CRC</td>
    <td>The leading actor of this movie is <u>Yanping Zhu.</u></td>
  </tr>
  <tr>
    <td>w/ CRC</td>
    <td>The leading actors in this movie, besides Jimmy Lin, are <u>Steven Hao and Long Huang.</u></td>
  </tr>
  <tr>
    <td colspan="2"><i>Consistency with the subgoal</i></td>
  </tr>
  <tr>
    <td>w/o CRC</td>
    <td>That's great. I have a movie starring him here, you can take a look, it'll touch your tears!</td>
  </tr>
  <tr>
    <td>w/ CRC</td>
    <td>Have you seen the movie <u>Grandpa's Love</u> starring him? I don't remember much about it, but I do remember the emotional bond between the grandfather and grandson.</td>
  </tr>
</table>

Table 6: A case generated by TP-GPT2.

tion model with more parameters improves performance across most metrics. However, regardless of the architecture or parameter size of the reflection model, their contributions to model performance remain quite similar.

6.6 Case Study

We conducted case studies to demonstrate the effectiveness of our CRC framework in enhancing the consistency between generated responses and each element of the dialogue context , as shown in Table 6 where the complete dialogue is presented in Figure 1. Regarding the dialogue history, TP-GPT2 emphasizes “of course he knows Lin” which reduces the consistency with the history. In contrast, our CRC improves the consistency by omitting this statement. In terms of the domain knowledge, TP-GPT2 incorrectly identifies the director of the movie as the leading actor. Conversely, our CRC correctly utilizes the domain knowledge (refer to the domain knowledge item in Figure 1). For the subgoal, although TP-GPT2 mentions recommending a movie, it fails to address the topic of Grandpa’s Love, resulting in an ineffective recommendation. In contrast, our CRC successfully recommends the movie Grandpa’s Love. Additional case studies of TP-GPT2 concerning the user profile, as well as LLaMA3, are provided in Appendix H.

7 Conclusion

This paper introduces a model-agnostic Consistency Reflection and Correction framework CRC aimed at enhancing the consistency between the generated responses and the dialogue contexts in GPDS. The CRC framework adeptly guides models to detect and correct inconsistencies, thereby significantly boosting the performance. Comprehensive experiments and detailed analysis on the DuRecDial, DuRecDial 2.0 and TopDial datasets, across various model architectures and parameter scales, validate the effectiveness of our CRC framework. Future research will focus on maintaining consistency between generated responses and dialogue contexts while simultaneously enhancing the diversity of the generated responses.
Limitation

Our Consistency Reflection and Correction (CRC) framework has two limitations. First, it relies on the closed-source GPT-4 as an annotator, raising concerns about the transparency and interpretability of the inconsistency detection process, despite our efforts to validate annotation quality. This dependence on a proprietary model may also limit the framework’s applicability and reproducibility in settings with restricted access, even if the specific version of GPT-4 is disclosed. Second, while the CRC framework effectively improves consistency between generated responses and dialogue contexts, achieving a balance between this consistency and output diversity presents a challenge. Aiming for greater diversity may inadvertently introduce inconsistencies, complicating the framework’s objectives and potentially reducing response quality. Addressing these limitations in future research will be crucial for optimizing the CRC framework and expanding its applicability across various dialogue systems.

Acknowledgments

The authors would like to thank the three anonymous reviewers for their comments on this paper. This research was supported by the National Natural Science Foundation of China (Nos. 62376181 and 62276177), and Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions.

References

Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. arXiv preprint arXiv:2404.14219.

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2024. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774.

Barry Checkoway and Donald A. Schön. 1985. The Reflective Practitioner: How Professionals Think in Action. Journal of Policy Analysis and Management, 4:476.

Ruijun Chen, Jin Wang, Liang-Chih Yu, and Xuejie Zhang. 2023. Learning to Memorize Entailment and Discourse Relations for Persona-consistent Dialogues. In Proceedings of the AAAI conference on artificial intelligence, pages 12653–12661.

Yang Deng, Wenxuan Zhang, Weiwen Xu, Wenqiang Lei, Tat-Seng Chua, and Wai Lam. 2023. A Unified Multi-task Learning Framework for Multi-goal Conversational Recommender Systems. ACM Transactions on Information Systems, 41(3):1 – 25.

Zuohui Fu, Yikun Xian, Yongfeng Zhang, and Yi Zhang. 2020. Tutorial on Conversational Recommendation Systems. In Proceedings of the 14th ACM Conference on Recommender Systems, page 751–753.

Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In Proceedings of the Tenth International Conference on Learning Representations, pages 1–13.

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825.

Zeming Liu, Haifeng Wang, Zheng-Yu Niu, Hua Wu, and Wanxiang Che. 2021. DuRecDial 2.0: A Bilingual Parallel Corpus for Conversational Recommendation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4335–4347.

Zeming Liu, Haifeng Wang, Zheng-Yu Niu, Hua Wu, Wanxiang Che, and Ting Liu. 2020. Towards Conversational Recommendation over Multi-Type Dialogs. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1036–1049.

Zeming Liu, Ding Zhou, Hao Liu, Haifeng Wang, Zheng-Yu Niu, Hua Wu, Wanxiang Che, Ting Liu, and Hui Xiong. 2023. Graph-Grounded Goal Planning for Conversational Recommendation. IEEE Transactions on Knowledge and Data Engineering, 35(5):4923–4939.

Haoyu Song, Yan Wang, Kaiyan Zhang, Wei-Nan Zhang, and Ting Liu. 2021. Bob: BERT over BERT for training persona-based dialogue models from limited personalized data. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 167–177.

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. LLaMA: Open and Efficient Foundation Language Models. arXiv preprint arXiv:2302.13971.
Fucheng Wang, Yunfei Yin, Faliang Huang, and Kaigui Wu. 2023a. Please Don’t Answer Out of Context: Personalized Dialogue Generation Fusing Persona and Context. In Proceedings of the 2023 International Joint Conference on Neural Networks, pages 1–8.

Jian Wang, Yi Cheng, Dongding Lin, Chak Leong, and Wenjie Li. 2023b. Target-oriented Proactive Dialogue Systems with Personalization: Problem Formulation and Dataset Curation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1132–1143.

Jian Wang, Dongding Lin, and Wenjie Li. 2022. Follow Me: Conversation Planning for Target-driven Recommendation Dialogue Systems. arXiv preprint arXiv:2208.03516.

Jian Wang, Dongding Lin, and Wenjie Li. 2023c. Dialogue Planning via Brownian Bridge Stochastic Process for Goal-directed Proactive Dialogue. In Findings of the Association for Computational Linguistics: ACL 2023, pages 370–387.

Jian Wang, Dongding Lin, and Wenjie Li. 2024a. A Target-Driven Planning Approach for Goal-Directed Dialog Systems. IEEE Transactions on Neural Networks and Learning Systems, 35(8):10475–10487.

Jian Wang, Dongding Lin, and Wenjie Li. 2024b. Target-constrained Bidirectional Planning for Generation of Target-oriented Proactive Dialogue. ACM Transactions on Information Systems, 42(5):1–27.

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023d. Self-Instruct: Aligning Language Models with Self-Generated Instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 13484–13508.

Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024a. WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions. In Proceedings of the Twelfth International Conference on Learning Representations, pages 1–22.

Kaishuai Xu, Yi Cheng, Wenjun Hou, Qiaoyu Tan, and Wenjie Li. 2024b. Reasoning Like a Doctor: Improving Medical Dialogue Systems via Diagnostic Reasoning Process Alignment. In Findings of the Association for Computational Linguistics ACL 2024, pages 6796–6814.

Xiaotong Zhang, Xuefang Jia, Han Liu, Xinyue Liu, and Xianchao Zhang. 2024. A Goal Interaction Graph Planning Framework for Conversational Recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 19578–19587.

Junkai Zhou, Liang Pang, Huawei Shen, and Xueqi Cheng. 2023. SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 9945–9959.
A Example of Inconsistency with User Profile

Table 7 shows an example of inconsistency with the user profile. The user profile indicates a preference for news about Nicholas Tse and Leehom Wang, yet TP-GPT2 recommends the social news from Hong Kong, which does not align with the user’s stated interests.

B Prompt for ChatGPT-based Annotation

we utilize ChatGPT \( ^2 \) to act as an annotator for the annotations of inconsistency types and correction suggestions. We feed the dialogue context and the generated response to ChatGPT and let it evaluate the consistency of them. The prompt is as follows:

Currently, the prediction task is performed: respond to user utterances based on information such as user profile, domain knowledge, dialogue history, and domain and current dialogue subgoal. However, there may be situations where the response is inconsistent with the four predefined elements above. For a dialogue, you need to analyze the AI Assistant’s response from the perspective of whether the response is consistent with the four predefined elements above, and identifies inconsistency types and correction suggestions. The consistency requirements of the response with the four predefined elements are:
(1) Is user profile information applied?
(2) Is the consistency with the dialogue history maintained?
(3) Is domain knowledge information applied?
(4) Is the current dialogue subgoal achieved?

[Start of Predefined Elements]
$ {User Profile}
$ {Dialogue History}
$ {Domain Knowledge}
$ {Subgoal}
[End of Predefined Elements]

[Start of the Assistant’s Response]
$ {Response}
[End of the Assistant’s Response]

\( ^2 \)The version used is GPT-4o-2024-05-13.

C Statistics of Datasets

THe DuRecDial and DuRecDial 2.0 datasets are the most widely used corpora, while the recently released TopDial dataset comprises a greater number of dialogues and a more diverse range of content, generated by LLM agents. The data statistics of the three datasets are shown in Table 8.

D Implementation Details

Our implementation is mainly based on TPNet\(^3\). We adopted the same goal-oriented path as TP-Net, with a primary focus on response generation. The hyperparameters, detailed in Table 9, are kept consistent across the experience, reflection, and correction stages for all models.

During training, we randomly selected 75% of the training set for the experience stage, while the remaining 25% was used for the reflection and correction stage. We employed LoRA (Hu et al., 2022) to fine-tune LLaMA3 (Touvron et al., 2023), Phi3 (Abdin et al., 2024) and Mistral (Jiang et al., 2023) using the LLaMA-Factory framework\(^4\). The rank \( r \) and scaling parameter \( \alpha \) are set to 8 and 16, respectively. The best model is selected based on its performance on the validation set. All our experiments were conducted on a single NVIDIA V100 GPU.

We follow previous work (Wang et al., 2024a) by directly concatenating the elements of the dialogue context, using the default delimiters for different models to distinguish between these elements, as detailed in Table 11. For LLaMA3, Phi3 and Mistral, we employ spaces to separate various types of elements. The input and output formats for the experience, reflection, and correction stages are presented in Table 10. For LLaMA3, Phi3 and Mistral, we provide appropriate prompts for each stage.

E Experimental Results on TopDial

DuRecDial and DuRecDial 2.0 are the most widely used corpora in the GPDS field. To ensure a fair comparison with previous baselines, we mainly conducted experiments and analyses on these two corpora. Furthermore, the experiments are augmented by the incorporation of the recently released GPDS dataset, TopDial, which comprises a greater number of dialogues and a more diverse

\(^3\)https://github.com/iwangjian/Plan4RecDial
\(^4\)https://github.com/hiyouga/LLaMA-Factory
<table>
  <tr>
    <th>User Profile</th>
    <th>Name:Xiaoling Liu; Agreed News: “Nicholas Tse News; Wang Leehom News”, etc.</th>
  </tr>
  <tr>
    <td>Domain Knowledge</td>
    <td>&lt;“Nicholas Tse”, News, “Recently, Nicholas Tse opened a Weibo account, named ‘Fengwei’, and certified as ‘Nicholas Tse, founder of Fengwei Holdings’, which seems to be a big push to promote his own brand. On the evening of July 8, Nicholas Tse sent out his first Weibo: ‘Only here can you find the complete Nicholas Tse. Let’s go together.’”&gt;, etc.</td>
  </tr>
  <tr>
    <td>Dialogue History</td>
    <td>[USER] “When is Nicholas Tse’s birthday?”<br>[System] “Nicholas Tse’s birthday is August 29th.”<br>[USER] “You are amazing, you know so much.”<br>[System] “Thank you for your compliment, Nicholas Tse is also a male star with outstanding temperament in the entertainment industry, do you like him?”<br>[USER] “I like Nicholas Tse very much.”<br>[System] “I remember he also won the World Music Award for the best-selling singer in Asia, he is really excellent.”<br>[USER] “Yes, yes, I hope he can get better and better.”<br>[System] “Yes, by the way, I have some news about him here, do you want to see it?”<br>[USER] “I am very interested, what is the news?”</td>
  </tr>
  <tr>
    <td>Subgoal</td>
    <td>Action: News Recommendation; Topic: Nicholas Tse</td>
  </tr>
  <tr>
    <td>TP-GPT2 w/o CRC</td>
    <td>“Recent news reports said that in Hong Kong, the police caught pickpockets, selling fake tickets, and impersonating staff to defraud money and tickets, and arrested more than a dozen suspects.”</td>
  </tr>
  <tr>
    <td>TP-GPT2 w/ CRC</td>
    <td>“Yes, there is. Recently, there was news that <b>Nicholas Tse and his wife had a second child, but the second child is a daughter. Both children have grown up, but the second child is still very young and hasn’t been born yet.”</b></td>
  </tr>
</table>

Table 7: Example of the inconsistency with the user profile.

<table>
  <tr>
    <th>Dataset</th>
    <th>Language</th>
    <th>Annotators</th>
    <th>Domains</th>
    <th>#Dialogue</th>
    <th>#Utterance</th>
  </tr>
  <tr>
    <td>DuRecDial</td>
    <td>Chinese</td>
    <td>Crowd workers</td>
    <td>Movies, music, food, point-of-interest, restaurants</td>
    <td>8,004</td>
    <td>126,186</td>
  </tr>
  <tr>
    <td>DuRecDial 2.0</td>
    <td>English</td>
    <td>Human experts</td>
    <td>Movies, music, food, point-of-interest, restaurants</td>
    <td>6,080</td>
    <td>98,719</td>
  </tr>
  <tr>
    <td>TopDial</td>
    <td>English</td>
    <td>LLM agents</td>
    <td>Movies, music, food, point-of-interest, restaurants</td>
    <td>18,009</td>
    <td>141,928</td>
  </tr>
</table>

Table 8: Statistics of DuRecDial, DuRecDial 2.0 and TopDial.

<table>
  <tr>
    <th>Parameter</th>
    <th>GPT2</th>
    <th>BART</th>
    <th>DialoGPT</th>
    <th>T5</th>
    <th>LLaMA3</th>
    <th>Phi3</th>
    <th>Mistral</th>
  </tr>
  <tr>
    <td>version</td>
    <td>base</td>
    <td>base</td>
    <td>base</td>
    <td>base</td>
    <td>8B</td>
    <td>3.8B</td>
    <td>7B</td>
  </tr>
  <tr>
    <td>learning rate</td>
    <td>5e-5</td>
    <td>2e-5</td>
    <td>5e-5</td>
    <td>1e-4</td>
    <td>1e-4</td>
    <td>1e-4</td>
    <td>1e-4</td>
  </tr>
  <tr>
    <td>batch size</td>
    <td>8</td>
    <td>8</td>
    <td>8</td>
    <td>4</td>
    <td>1</td>
    <td>1</td>
    <td>1</td>
  </tr>
  <tr>
    <td>msl</td>
    <td>512</td>
    <td>512</td>
    <td>432</td>
    <td>512</td>
    <td>1024</td>
    <td>1024</td>
    <td>1024</td>
  </tr>
  <tr>
    <td>epoch</td>
    <td>10</td>
    <td>10</td>
    <td>10</td>
    <td>10</td>
    <td>10</td>
    <td>10</td>
    <td>10</td>
  </tr>
  <tr>
    <td>mdl</td>
    <td>80</td>
    <td>80</td>
    <td>80</td>
    <td>80</td>
    <td>80</td>
    <td>80</td>
    <td>80</td>
  </tr>
  <tr>
    <td>optimizer</td>
    <td>Adam</td>
    <td>Adam</td>
    <td>Adam</td>
    <td>Adam</td>
    <td>AdamW</td>
    <td>AdamW</td>
    <td>AdamW</td>
  </tr>
  <tr>
    <td>decoding strategy</td>
    <td colspan="7">greedy search</td>
  </tr>
</table>

Table 9: Hyperparameter settings where “msl” denotes max sequence length and “mdl” denotes max decoding length.

range of content, generated by LLM agents. The results, shown in Table 12, indicate significant improvements across all evaluation metrics after integrating CRC, highlighting the effectiveness and generalizability of our CRC framework.

F Human Evaluation

Assuming the system-generated responses before and after applying our method are defined as \( R_{\text{before}} \) and \( R_{\text{after}} \), respectively, we manually evaluated which of \( R_{\text{before}} \) and \( R_{\text{after}} \) is more consistent with the dialogue context, including user profiles, dialogue history, domain knowledge, and subgoals.
<table>
  <tr>
    <th>Phase</th>
    <th>BART, T5, GPT2, DialoGPT</th>
    <th>LLaMA3, Phi3, Mistral</th>
  </tr>
  <tr>
    <td>Experience</td>
    <td><b>Input:</b> [DK]delim[SG]delim[UP]delim[DH]<br><b>Output:</b> [Response]</td>
    <td><b>Input:</b> Respond to user utterances based on domain knowledge, user profile, dialogue history, and the current dialogue goal.<br>[DK]delim[SG]delim[UP]delim[DH]<br><b>Output:</b> [Response]</td>
  </tr>
  <tr>
    <td>Reflection</td>
    <td><b>Input:</b> [DK]delim[SG]delim[UP]delim[DH]<br>delim ###stage2_R<br><b>Output:</b> [Response]###[e]:###[s]</td>
    <td><b>Input:</b> Respond to user utterances based on domain knowledge, user profile, dialogue history, and the current dialogue goal, and annotate the response with the types of inconsistencies compared to predefined information, along with suggestions for generating better responses.<br>[DK]delim[SG]delim[UP]delim[DH]<br><b>Output:</b> [Response]###[e]:###[s]</td>
  </tr>
  <tr>
    <td>Correction</td>
    <td><b>Input:</b> [DK]delim[SG]delim[UP]delim[DH]<br>delim [Response]###[e];###[s] ###stage3_C<br><b>Output:</b> [Corrected Response]</td>
    <td><b>Input:</b> Correct the pre-response and respond to user utterances based on domain knowledge, user profile, dialogue history, current dialogue goal, pre-response, types of inconsistencies between the pre-response and predefined information, and suggestions for generating a better response.<br>[DK]delim[SG]delim[UP]delim[DH]<br>delim[Response]###[e];###[s]<br><b>Output:</b> [Corrected Response]</td>
  </tr>
</table>

Table 10: The input and output formats for the experience, reflection, and correction stages. Delim is short for delimiter. The delimiters used by various models are shown in Table 11. The abbreviations used are: DK (domain knowledge), SG (subgoal), UP (user profile), DH (dialogue history), e (inconsistent type) and s (suggestion).

<table>
  <tr>
    <th>Model</th>
    <th>DuRecDial</th>
    <th>DuRecDial2.0, TopDial</th>
  </tr>
  <tr>
    <td>GPT2<sup>5</sup></td>
    <td>[SEP]</td>
    <td>&lt;|endoftext|&gt;</td>
  </tr>
  <tr>
    <td>BART<sup>6</sup></td>
    <td>[SEP]</td>
    <td>&lt;/s&gt;</td>
  </tr>
  <tr>
    <td>DialoGPT<sup>7</sup></td>
    <td>[SEP]</td>
    <td>&lt;|endoftext|&gt;</td>
  </tr>
  <tr>
    <td>T5<sup>8</sup></td>
    <td>&lt;/s&gt;</td>
    <td>&lt;/s&gt;</td>
  </tr>
  <tr>
    <td>LLaMa3-8B<sup>9</sup></td>
    <td>Space</td>
    <td>Space</td>
  </tr>
  <tr>
    <td>Phi3-3.8B<sup>10</sup></td>
    <td>Space</td>
    <td>Space</td>
  </tr>
  <tr>
    <td>Mistral-7B<sup>11</sup></td>
    <td>Space</td>
    <td>Space</td>
  </tr>
</table>

Table 11: Delimiters used by different models on three datasets.

During the manual evaluation, the evaluators are asked to assign one of three possible labels (i.e., “win”, “tie”, or “lose”) to two anonymous responses, designated as A and B. One of these responses is \( R_{\text{before}} \), while the other is \( R_{\text{after}} \). The labels “win”, “tie”, and “lose” are used to indicate that A is more consistent, equally consistent, or less consistent than B, respectively. Specific guidelines of the manual evaluation are as follows:

<b>Consistency with User Profile:</b> The evaluators first assess whether A and B contradict the user profile, such as gender, hobbies, age, etc., and then determine which response is more consistent. If both A and B are consistent or inconsistent with the user profile, the evaluators choose “tie”.

<b>Consistency with Dialogue History:</b> The evaluators first assess whether A and B accurately understand the user’s intent and are logically coherent with the dialogue history, avoiding abrupt or irrelevant content. Then, the evaluators determine which response is more consistent with the dialogue history. If both A and B are consistent or inconsistent, the evaluators choose “tie”.

<b>Consistency with Domain Knowledge:</b> The evaluators assess whether A and B correctly utilize domain knowledge and choose the more consistent response. If both A and B correctly or incorrectly use domain knowledge, the evaluators choose “tie”.

<sup>5</sup>https://huggingface.co/uer/gpt2-chinese-cluecorpusmall/tree/main
<sup>6</sup>https://huggingface.co/fnlp/bart-base-chinese/tree/main
<sup>7</sup>https://huggingface.co/thu-coai/CDial-GPT-LCCC-base/tree/main
<sup>8</sup>https://huggingface.co/google/mt5-base
<sup>9</sup>https://huggingface.co/shenzi-wang/Llama3-88-Chinese-Chat/tree/main
<sup>10</sup>https://huggingface.co/microsoft/Phi-3-mini-4k-instruct
<sup>11</sup>https://huggingface.co/shenzi-wang/Mistral-7B-v0.3-Chinese-Chat
<table>
  <tr>
    <th>Method</th>
    <th>W F<sub>1</sub></th>
    <th>BLEU-2</th>
    <th>Dist-2</th>
    <th>K F<sub>1</sub></th>
    <th>Succ</th>
  </tr>
  <tr>
    <td>TP-Dial<sub>(99M)</sub></td>
    <td>33.87</td>
    <td>0.162</td>
    <td>0.068</td>
    <td>36.62</td>
    <td>43.88</td>
  </tr>
  <tr>
    <td>TP-Dial w/ CRC</td>
    <td>36.12<sup>±2.25</sup></td>
    <td>0.196<sup>±0.034</sup></td>
    <td>0.069</td>
    <td>42.51<sup>±5.89</sup></td>
    <td>47.98<sup>±4.10</sup></td>
  </tr>
  <tr>
    <td>TP-Phi3<sub>(3.8B)</sub></td>
    <td>34.56</td>
    <td>0.186</td>
    <td>0.080</td>
    <td>40.04</td>
    <td>41.02</td>
  </tr>
  <tr>
    <td>TP-Phi3 w/ CRC</td>
    <td>37.12<sup>±2.56</sup></td>
    <td>0.208<sup>±0.022</sup></td>
    <td>0.079</td>
    <td>44.23<sup>±4.19</sup></td>
    <td>46.75<sup>±5.73</sup></td>
  </tr>
  <tr>
    <td>TP-Mistral<sub>(7B)</sub></td>
    <td>34.12</td>
    <td>0.171</td>
    <td>0.077</td>
    <td>39.05</td>
    <td>42.56</td>
  </tr>
  <tr>
    <td>TP-Mistral w/ CRC</td>
    <td>36.76<sup>±2.64</sup></td>
    <td>0.192<sup>±0.021</sup></td>
    <td>0.078</td>
    <td>43.24<sup>±4.19</sup></td>
    <td>46.67<sup>±4.11</sup></td>
  </tr>
  <tr>
    <td>TP-LLaMA3<sub>(8B)</sub></td>
    <td>35.94</td>
    <td>0.182</td>
    <td>0.082</td>
    <td>40.77</td>
    <td>43.32</td>
  </tr>
  <tr>
    <td>TP-LLaMA3 w/ CRC</td>
    <td>37.45<sup>±1.51</sup></td>
    <td>0.205<sup>±0.023</sup></td>
    <td>0.082</td>
    <td>47.89<sup>±7.12</sup></td>
    <td>47.11<sup>±3.79</sup></td>
  </tr>
</table>

Table 12: Experimental results on TopDial. The parameter sizes of the models are annotated as subscripts adjacent to the model names.

![Bar charts comparing pairwise evaluation results for TP-BART, TP-T5, TP-GPT2, and TP-Dial. Four bar charts labeled (a) TP-BART w/ CRC vs. TP-BART, (b) TP-T5 w/ CRC vs. TP-T5, (c) TP-GPT2 w/ CRC vs. TP-GPT2, (d) TP-Dial w/ CRC vs. TP-Dial. Each chart shows three categories: Win, Tie, Lose, with percentage values.](page_320_670_1108_370.png)

Figure 5: Pairwise evaluation results for TP-BART, TP-T5, TP-GPT2, and TP-Dial. Abbreviations: UP (user profile), DH (dialogue history), DK (domain knowledge), SG (subgoal).

Consistency with Subgoals: The evaluators assess whether A and B correctly achieve the subgoals and choose the more consistent response. If both A and B achieve or fail to achieve the subgoals, the evaluators choose “tie”.

We randomly selected 500 pairs of system responses and invited three NLP graduate students to independently evaluate them, taking the average of the results. The pairwise evaluation results for BART, T5, GPT2 and DialoGPT (Dial) are shown in Figure 5. It can be seen that all four models using CRC have a much higher win rate in all four aspects compared to these without CRC. The manual evaluation results also demonstrate the effectiveness of our CRC framework in improving consistency.

G Results of Subgoals Failure

As illustrated in Figure 6, our CRC significantly reduces the goal failure rate for the current turn on the DuRecDial 2.0 dataset. This reduction not only further highlights the effectiveness of our framework but also demonstrates its robustness and adaptabil-
<table>
  <tr>
    <th>User Profile</th>
    <th>Name:</th>
    <th>Liqian Niu; Favorite Stars: “Leslie Cheung, Jiong He”, etc.</th>
  </tr>
  <tr>
    <td>Domain Knowledge</td>
    <td></td>
    <td>&lt;“Jiong He”, Singing, “Gardenia Blossoms”&gt;, etc.</td>
  </tr>
  <tr>
    <td colspan="3">[System] “Good evening, Liqian Niu.”<br>[USER] “Good evening!”<br>[System] “How was work today?”<br>[USER] “We got our bonuses today, so I’m very happy.”</td>
  </tr>
  <tr>
    <td>Dialogue History</td>
    <td colspan="2"></td>
  </tr>
  <tr>
    <td>Subgoal</td>
    <td colspan="2">Action: <b>Music Recommendation;</b> Topic: <b>Gardenia blooms</b></td>
  </tr>
  <tr>
    <td>LLaMA3 w/o CRC</td>
    <td colspan="2">“Congratulations, you can feel the fragrance of gardenia floating lightly in the air.”</td>
  </tr>
  <tr>
    <td>LLaMA3 w/ CRC</td>
    <td colspan="2">“Congratulations, I recommend you a song ‘<b>Gardenia Blossoms</b>’. The gentle music describes the mood of parting in a subtle way. The simple music rhythm makes people feel the fragrance of gardenias floating lightly in the air when listening.”</td>
  </tr>
</table>

Table 13: Case study of the inconsistency with the subgoal.

<table>
  <tr>
    <th>User Profile</th>
    <th>Name:</th>
    <th>Shuaigou Xu; Favorite Stars:Bo Huang, Leslie Cheung, Age Range:18-25, etc.</th>
  </tr>
  <tr>
    <td>Domain Knowledge</td>
    <td></td>
    <td>&lt;“He’s a Woman, She’s a Man.”, Director, “<b>Peter Chan</b>”&gt;, etc.</td>
  </tr>
  <tr>
    <td colspan="3">[USER] ……<br>[System] “Then let’s chat about the most gossipy celebrity topics. How about we talk about the lead singer of this song, Leslie Cheung?”<br>[USER] “Leslie Cheung, huh? Not bad, not bad. He’s an outstanding man, and his songs are really good.”<br>[System] “Yeah, his songs are indeed quite good. He even won the Best Original Song award at the Golden Horse Awards in Taiwan.”<br>[USER] “He’s also the record holder for the best-selling Chinese album in South Korea. Impressive.”<br>[System] “Indeed. By the way, he also starred in a very interesting movie, which has a bit of an autobiographical and metaphorical touch—it’s called ‘Yang ± Yin: Gender in Chinese Cinema.’ You should check it out.”<br>[USER] “I actually really like that movie; I’ve already watched it.”<br>[System] “You’re quite a fan of his, then! In that case, let me recommend another one: ‘He’s a Woman, She’s a Man.’ It explores love beyond gender. You should give it a try.”<br>[USER] “I haven’t seen that one yet, but who’s the director of this film?”</td>
  </tr>
  <tr>
    <td>Dialogue History</td>
    <td colspan="2"></td>
  </tr>
  <tr>
    <td>Subgoal</td>
    <td colspan="2">Action: Movie Recommendation; Topic: He’s a Woman, She’s a Man.</td>
  </tr>
  <tr>
    <td>LLaMA3 w/o CRC</td>
    <td colspan="2">“The director of this movie is <b>Leslie Cheung</b>.”</td>
  </tr>
  <tr>
    <td>LLaMA3 w/ CRC</td>
    <td colspan="2">“The director is <b>Peter Chan</b>.”</td>
  </tr>
</table>

Table 14: Case study of the inconsistency with the domain knowledge.

ity across various dialogue scenarios. By decreasing the subgoal failure rates, our framework shows its potential to enhance user satisfaction through more reliable achievement of intended outcomes. Consequently, this improvement elevates the overall quality and utility of dialogue systems in practical applications.

H Case Study

The case study of TP-GPT2 does not align with the user profile presented in Table 7. TP-GPT2 recommends social news from Hong Kong, which does not match the user’s stated interests. However, our TP-GPT2 with CRC can generate responses related to Nicholas Tse news, ensuring better consistency with the user profile.

Similarly, the case study of LLaMA3 does not align with the subgoal and domain knowledge illustrated in Tables 13 and 14, respectively. Table 13 shows that LLaMA3 without CRC failed to recommend the song “Gardenia Blossoms” whereas LLaMA3 with CRC achieved this goal. Table 14 demonstrates that LLaMA3 without CRC failed to utilize domain knowledge, incorrectly identifying
![Comparison of subgoal failure rates on DuRec-Dial2.0.](page_186_153_495_312.png)

Figure 6: Comparison of subgoal failure rates on DuRec-Dial2.0.

Leslie Cheung as the director of the movie *He’s a Woman, She’s a Man*. In contrast, LLaMA3 with CRC correctly utilized the domain knowledge. Furthermore, LLaMA3 rarely exhibits inconsistencies with the user profile and dialogue history, thanks to its robust capabilities.

Reviewer paper summaries: This paper addresses the issue of normal goal-oriented proactive dialogue systems often producing responses that are inconsistent with dialogue contexts. The authors propose Consistency Reflection and Correction (CRC), a two-step framework to help smaller language models reflect and correct existing inconsistencies in responses. The authors first use ChatGPT to synthesize reflections on the inconsistencies present in the dialogue context and then fine-tune a model to generate these reflections when provided with the dialogue history. The next step involves training a model to produce more consistent outputs according to the reflection in the prior step. The authors show that their framework successfully improves the consistency between generated responses and the dialogue context after experimenting with a variety of model architectures using automated metrics. Human evaluation demonstrates the advantage of using CRC on TP-LLaMA3.

**Note:** This summary is partially similar to the summary I provided in the October 2024 cycle, which is still an accurate summary in my opinion.

The paper introduces a Consistency Reflection and Correction (CRC) framework to enhance goal-oriented proactive dialogue systems by addressing inconsistencies between generated responses and dialogue contexts. The CRC framework operates in two stages: the consistency reflection stage, where the model identifies and reflects on discrepancies, and the consistency correction stage, where it generates more contextually consistent responses. Experimental results across various models and datasets demonstrate that the CRC framework significantly improves the consistency of dialogue systems, enhancing user experience and achieving intended conversational goals.

The paper presents a framework for correcting in-consistencies in model response, given a conversation context and other information, e.g., user profile and background knowledge. It shows that improving consistency between conversation history and generated response, through training on annotated reflection and corrected responses improves model performance. In addition, paper presents detailed empirical analysis and ablation to demonstrate the effective of their method.
Summary of strengths: ## Strengths from the 2024 October Cycle
1. The authors conduct experiments on a variety of models, including smaller models like GPT-2, which indicates that their framework can improve the performance of even smaller models. I appreciate that the authors are conducting research that can successfully improve performance on smaller models!
2. There is an extensive survey into the prior baselines for this task.
3. The framework successfully improves the dialogue consistency of model generations, resulting in improvement along multiple metrics, especially the Knowledge F1 score.
4. The paper focuses on an under-studied area in the Goal-oriented Proactive Dialogue Systems field, and presents a relatively novel methodology.

## Additional Strengths
1. The human evaluation demonstrated the effectiveness of the CRC approach on TP-LLaMA3.

The motivation is clear and targeting consistency and alignment with dialog context
User outlined several important aspects of consistency 
It can be easily applied in various applications
The CRC framework is designed to be model-agnostic, allowing it to be applied across various model architectures and parameter sizes, enhancing its versatility and applicability in different contexts.
The paper provides extensive experimental validation across multiple datasets and models, demonstrating the effectiveness of the CRC framework in improving the consistency of generated responses, which supports the robustness and generalizability of the proposed method.

1- Language models are prone to hallucinations or inconsistencies which is why this is an important problem to solve.  
2- The authors empirically show that improving consistency in model responses through their framework improves model performance.  They also test their approach on a wide variety of open-source models which reflects the robustness of the proposed framework.
3- The authors conduct an ablation study to highlight the impact of their suggested framework and indicate that the proposed method improves performance across multiple dimensions.
Summary of weaknesses: The authors addressed all weaknesses I mentioned in the previous cycle, including the lack of human evaluations and headroom for clarity improvement of the paper, and they have answered all of my questions thoroughly in the addressing revision pdf document.

My main issue is that the improvement of TP-LLaMA3 w/ CRC is not significant enough (the tie rate is still very high).

I found that there is limited new research insights coming from this work as most of the techniques are pretty established. 
I think some comparison are missing where 
OSS models are used as the annotator instead of GPT-4. 
Different combinations of models are used at different stages (reflection and correction)

1- Deploying conversation models (specially speech-based conversation systems) is a latency sensitive problem. The proposed approach improves model consistency with conversation context and other background information but that would come at the cost of latency. It would be interesting to see a discussion on the ramifications of this direction.   
2- The current biggest impediment to the paper is organization with a lot of content in the appendix. Maybe tables 1 and 2 could be combined to make some extra space and the size of figure 1 could be reduced by highlighting only the important components.
Comments, suggestions and typos: N/A, I appreciate the authors' revisions of the paper based on my previous comments and questions!

How much will the learnings from this work be helpful as the model capabilities improve?
How can we leverage the insights from your results to combine different models for each of the reflection and correction steps? 
I understand the decision to use GPT-4 as the annotator, but do we have results for the performance using other models as the annotator ? and what’s the delta there ?

1- The authors state in Section 4, subsection Consistency Reflection:    
"As introduced in Section 1, the responses generated by the RG model always exhibit inconsistencies with dialogue context." -- Isn't "always" a strong word to use here? Depending on the conversation task and setting, rate of such inconsistencies would vary or is this a characteristic of the datasets used?   
2- The authors state: "The experimental results are shown in Table 10 of Appendix E" -- why not add these to the tables 1 and 2 as well?   
3- Clarifcation: "TP-LLaMA3 w/ CRC" when evaluation your framework with large model like Llama3 are you fine-tuning them as well as explained in the section 4 of your approach or is it a purely prompt based modification?   
4- It would be great if you could mention this earlier: "We also conducted an in-depth analysis of the ability of ChatGPT and our correction model  (DialoGPT) to detect inconsistencies and generate meaningful explanations, as detailed in Appendix H".

Comment: Authors have address most of my comments from my previous review and I will be increasing my scores of the work accordingly.
Overall assessment scores: 4

3

4