SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation
Year: 2023

Abstract:
Language models trained on large-scale corpora can generate remarkably fluent results in open-domain dialogue. However, for the persona-based dialogue generation task, consistency and coherence are also key factors, which are great challenges for language models. Existing works mainly focus on valuable data filtering, model structure modifying, or objective function designing, while their improvements are limited and hard to generalize to all types of pre-trained language models. However, we find that language models can produce consistent and coherent responses if we consider enough generations. Thus, the problems lay in largescale response generation and target response selection. In this work, a simple but effective two-stage SimOAP strategy is proposed, i.e., over-sampling and post-evaluation. The over-sampling stage takes large-scale responses from existing trained models efficiently via offthe-shelf distilling and compressing methods, and the post-evaluation stage selects a good response based on multiple well-designed evaluation metrics from large-scale candidates. Experimental results show that the proposed plug-in SimOAP strategy improves the backbone models and outperforms the baseline strategies in both automatic and human evaluations.

heading: 1 Introduction
text: Open-domain dialogue systems need to give appropriate responses based on history utterances. An ideal open-domain dialogue system should generate consistent, coherent, and diverse responses. Part of the existing open-domain dialogue generation work focuses on improving the diversity of responses (Wang et al., 2021), while avoiding generating generic responses and achieving good results. How to improve the consistency of dialogue generation is also an urgent problem to be solved (Kim et al., 2020). In addition, there is still the
∗Corresponding authors
problem of poor coherence in dialogue generation (Ye et al., 2021).
To improve the consistency and coherence of dialogue generation, the existing works mainly improve from three aspects: valuable data construction and filtering (Zhang et al., 2018; Song et al., 2020b), model structure modifying (Song et al., 2021; Zou et al., 2021; Liu et al., 2023) and objective function designing (Li et al., 2020; Hao et al., 2020). However, the problem of poor consistency and coherence is still a tough challenge, especially in persona-based dialogue generation (Song et al., 2020a). Because multiple constraints need to be satisfied simultaneously, part of which cannot be directly optimized, and part of the constraints conflict with each other, such as the conflict between the fluency of responses and the consistency of persona information. In addition, the above methods need to retrain the model and can only adapt to the
9945
part of the existing dialogue models. For example, Boyd et al. (2020) carefully design the objective function and scale model sizes from 117M to 8.3B parameters, which brings a lot of training costs. Fortunately, we find that the existing dialogue models actually have strong capabilities that can generate consistent and coherent responses, and we just need to find ways to release their capabilities.
First, we take a deep look at the characteristics of dialogue models, which believe that the response with the highest probability is the best. However, we wonder whether the high-probability responses generated by dialogue models are necessarily better than the low-probability responses. Based on the statistics in Figure 1, when the generation probability of responses decreases, the ratio of good responses increases first and then decreases. It shows that the ratio of good responses among lowprobability responses is higher than that of highprobability responses, which is counter-intuitive. This is most likely because dialogue models use PPL as an optimization goal, but it is inconsistent with the requirements of coherence and consistency. To verify whether the good response with high TFIDF similarity and high probability of entailment1 is indeed superior to the response directly generated by the model, we use the human evaluation for experimental validation. As shown in Table 1, such responses are better than those directly generated by the model. Therefore, it only needs to sample large-scale diverse responses from existing dialogue models and then select good responses.
Inspired by the aforementioned motivations, We propose a simple two-stage method: over-sampling and post-evaluation (SimOAP) to improve the coherence and consistency in persona-based dialogue. There are two challenges in our work. The largescale sampling will bring additional time cost, how to accelerate the model is a challenge. Large-scale sampling can produce good responses, how to pick good responses from them is another challenge. We address the above two challenges using oversampling and post-evaluation, respectively. In the over-sampling stage, SimOAP uses existing dialogue models for large-scale sampling, and the distilled or compressed models are used to reduce the
1The high TF-IDF similarity means the TF-IDF similarity between the response and ground truth is above 0.25, and the high probability of entailment means the entailment probability of the natural language inference model between the response and persona information is above 0.5. When the above two constraints are relaxed to 0.15 and 0.35, respectively, the trend of the curve in Figure 1 is still the same.
additional time cost. In the post-evaluation stage, the TF-IDF algorithm (Salton and Buckley, 1988) and natural language inference (NLI) are used for coherence and consistency evaluation, respectively.
To verify the effectiveness of our method, we conduct experiments on a persona-based dialogue dataset Personachat (Zhang et al., 2018). Automatic evaluations and human evaluations show that our method effectively boosts the performance of dialogue models and outperforms two baselines (Li et al., 2016; Adiwardana et al., 2020) on most metrics. In addition, applying our method to small models can also achieve a better performance than using large models directly.
Our contributions in this paper are three folds:
• We verify that the high-probability responses generated by dialogue models are not necessarily better than the low-probability responses. That is, dialogue models can generate good responses, but they are not selected.
• We propose a simple two-stage method: oversampling and post-evaluation to improve the coherence and consistency in persona-based dialogue generation and it is model-agnostic.
• We conduct comprehensive experiments on a persona-based dialogue dataset. Automatic evaluations and human evaluations show that our method improves the backbone models and outperforms the baselines.

heading: 2 Related Work
text: Dialogue generation has made remarkable progress in recent years. Many pre-trained dialogue models have been proposed (Zhang et al., 2019; Bao et al., 2019; Adiwardana et al., 2020; Roller et al., 2021). To improve the consistency of dialogue generation and make dialogue models appli-
Over-sampling Stage Post-evaluation Stage
cable to various scenarios, Zhang et al. (2018) propose a persona-based dialogue dataset PersonaChat. Persona-based dialogue generation is limited by the scale of data and expensive annotation costs. Song et al. (2019) generate persona-based dialogue by using additional natural language inference data. Cao et al. (2022) use data augmentation to extend data and use data distillation to make it easier to fit. However, labeling data for persona-based dialogue takes a high cost, and data from other domains is difficult to apply to persona-based dialogue fully.
Part of the work modifies the model structure for persona-based dialogue. Zheng et al. (2019) propose a pre-trained model, which uses personabased sparse data for pre-training. Song et al. (2020a) design a three-stage framework of generating, deleting, and rewriting. Song et al. (2021) learn the persona features by designing a response generation decoder and a consistency understanding decoder. However, there are multiple constraints that need to be satisfied simultaneously, some of which cannot be directly optimized. The above works also bring a huge training cost.
Part of the work designs the related objective function. Li et al. (2020) modify the unlikelihood loss to improve the consistency of dialogue. Boyd et al. (2020) use the previous dialogue content of users to control the dialogue of specific personas. However, it is difficult to design the objective function. We found a simple strategy without filtering valuable data, modifying the model structure, or designing objective functions, but only needs to use existing models for large-scale sampling and post-evaluation to improve the performance.
Nye et al. (2021) use dual systems to improve the coherence and consistency of neural sequence models. This work uses a neural system to generate the candidate and a logic system to evaluate it. The candidate is generated and evaluated one by one until it meets the criteria. However, the small number of candidates limits the effectiveness of dialogue generation. In addition, the logic system evaluates the candidate by tracking common sense information. It is difficult to apply to dialogue generation. In dialogue generation, maximum mutual information (MMI) (Li et al., 2016) uses the mutual information between history utterances and responses to evaluate responses. MMI can reduce the generation of generic responses but brings the large-scale time cost. To eliminate the influence of response length on likelihood, Adiwardana et al. (2020) use length-normalized loglikelihood score (LLS) to evaluate candidate responses. However, it is verified that using large-scale sampling for LLS performs worse than fewer candidate responses. It shows that LLS cannot release the ability of models by over-sampling. Furthermore, simple evaluation methods for the above two methods are difficult to work well in complex persona-based dialogue.

heading: 3 Our Approach
text: Persona-based dialogue consists of persona information sentences P = {p1, p2, ..., p|P |}, history utterances H = {h1, h2, ..., h|H|}, and a gold response g. Dialogue models need to generate a response r, which is coherent with history utterances H and consistent with persona sentences P .
The framework of SimOAP is shown in Figure 2.
SimOAP consists of two stages: over-sampling and post-evaluation. In the over-sampling stage, SimOAP uses existing dialogue models for largescale sampling, and accelerates the model to reduce the extra time cost. In the post-evaluation stage, the TF-IDF algorithm (Salton and Buckley, 1988) and natural language inference are used for coherence and consistency evaluation, respectively.

heading: 3.1 Over-sampling Stage
text: To do efficient and diverse over-sampling, we face two challenges to be solved. The first challenge is that generating large-scale responses is time-consuming, which will bring additional time cost. We have to speed it up. Another challenge is how to achieve diversity among different responses. The generated responses need to be diverse, not just those with high generation probability. Because we need to select a good response from the sampled responses, there should be differences between them rather than a large number of similar responses. To address the above challenges, we use distilled or compressed models to accelerate. Then the top-k sampling (Fan et al., 2018) with large k value and large sample number are used to introduce diversity. The capabilities of well-trained dialogue models can be released by introducing diversity and large-scale sampling.
Generation of Candidate Responses The existing dialogue models actually have strong capabilities that can generate consistent and coherent responses, but they are just not being released. We choose existing dialogue models for dialogue generation without re-training. To introduce diversity, we use top-k sampling to take large-scale samples from existing dialogue models and generate candidate responses. At each step of generating the response, the dialogue model generates the probability of each word in the vocabulary being the likely next word, forming a probability distribution. Then we randomly sample from the k most likely vocabs from this probability distribution. All tokens in each response are generated with top-k sampling. To ensure the diversity of candidate responses and the effectiveness of over-sampling, we use the large k in top-k sampling. For each history dialogue, s candidate responses will be generated, denoting them as R = {r1, r2, ..., rs}, and s is also set to be large to introduce diversity.
Model Acceleration Due to the extra time cost incurred in large-scale sampling, we use distilled or
compressed models to replace the backbone models to speed up. For example, when the backbone model is Multi-GPT2 (Cao et al., 2020), we use DistilGPT2 (Sanh et al., 2019) replace GPT2 (Radford et al., 2019) to build Multi-GPT2. When the backbone model is BERT-over-BERT (Song et al., 2021), we use the compressed model BERTmedium (Devlin et al., 2018) replace BERT-base (Devlin et al., 2018) to build it.

heading: 3.2 Post-evaluation Stage
text: The over-sampling stage produces diverse responses, but how to select good responses from them is a challenge. Although there are many metrics to automatically evaluate the effectiveness of dialogue (Gao et al., 2021; Chan et al., 2021; Ji et al., 2022; Ghazarian et al., 2022a), most of them evaluate the responses only from a single aspect. For example, perplexity can only be used to evaluate the fluency of responses and cannot reflect the quality of responses in other aspects. When multiple methods are used in combination to evaluate responses, it may bring additional time cost, especially for learnable methods. The oversampling stage already brings the additional time cost, so we want to reduce the time cost in the postevaluation stage. How to reduce it is another challenge. To address the above challenges, we first use the TF-IDF algorithm to evaluate the coherence of candidate responses and filter out those with poor coherence2. Then the consistency evaluation with the NLI model is used to select the final response. Since both coherence and consistency need to be satisfied, the fast coherence evaluation based on TF-IDF is first used to evaluate and reduce candidate responses, which can reduce the time cost, then the learnable NLI is used.
Coherence Evaluation Coherence requires the response to be context-related to history utterances (Ye et al., 2021). Some learnable coherence evaluation methods (Ghazarian et al., 2022b; Ye et al., 2021) have been proposed, but they will bring the additional time cost. To reduce the time cost of the post-evaluation stage and improve the coherence of responses, we use the TF-IDF algorithm (Salton and Buckley, 1988) to calculate the semantic similarity between the candidate responses R and history utterances H . We take history utter-
2Evaluating coherence using the TF-IDF algorithm is sufficient for our method to perform well and it is fast, which is verified in Section 4.
ances H as the first document and each candidate response as a document, which together with H constitute the corpus. The TF-IDF value of the i-th word ti in the corpus of the j-th document dj is:
tfidfi,j = tfi,j ∗ idfi, (1)
where tfi,j is the term frequency of the i-th word in the j-th document, idfi is the inverse document frequency of the i-th document:
tfi,j = ni,j∑ k nk,j , idfi = lg |D| 1 + {j : ti ∈ dj} ,
(2) where ni,j is the number of the i-th word that appears in the j-th document, ∑ k nk,j is the sum of the number of all words in the j-th document. |D| is the number of documents in the corpus, {j : ti ∈ dj} is the number of documents which containing the i-th word ti. Suppose there are I unique words in the corpus, so each document vector can be represented as:
Vj = (tfidf1,j , ..., tfidfi,j ..., tfidfI,j) . (3)
Finally, we calculate the cosine similarity between the representation of H and the representation of each candidate response ri separately, and c responses with the highest similarity are selected as candidate R̂, which is a subset of R.
Consistent Evaluation In persona-based dialogue, consistency requires the response to be consistent with persona information (Song et al., 2020a). Welleck et al. (2019) propose a dialogue inference dataset DialogueNLI. Many persona-based dialogue works using NLI models fine-tuned on it to evaluate the consistency between persona information and responses have proven successful (Kim et al., 2020; Song et al., 2019, 2020a; Cao et al., 2022). Following them, we use the NLI model to calculate the possibility of entailment between the candidate responses R̂ and persona sentences P to improve the consistency. The RoBERTa (Liu et al., 2019) is fine-tuned on DialogueNLI, where the inference labels are entailment, contradiction, or neutral. Then the fine-tuned RoBERTa is used to compute the possibility of entailment between candidate responses and persona sentences. Finally, the response with the highest possibility is selected as the final response r.

heading: 4 Experiments

heading: 4.1 Dataset
text: To verify the effectiveness of our proposed method, experiments have been carried out on a public dialogue dataset PersonaChat (Zhang et al., 2018). PersonaChat is a persona-based dialogue dataset that includes rich persona features. During the dialogue process, the dialogue agent needs to give an appropriate response according to persona features. PersonaChat contains 10,907 dialogues (162,064 utterances), 8,939/1,000/968 dialogues of which are set for training/validation/testing.

heading: 4.2 Experiment Settings
text: Models and Baselines Two persona-based dialogue models and two baseline strategies are used for experimental verification.
BERT-over-BERT (BoB) (Song et al., 2021) is a persona-based dialogue model which learns the persona features by designing an encoder, a response generation decoder, and a consistency understanding decoder.
Multi-GPT2 (Cao et al., 2020) is a personabased dialogue model with encoder-decoder architecture adapted from GPT2.
Maximum Mutual Information (MMI) (Li et al., 2016) use the backward model to predict history utterances from candidate responses. Then the prediction probability is used to rerank the responses and reduce generic responses.
Length-normalized Loglikelihood Score (LLS) (Adiwardana et al., 2020) is used to eliminate the influence of response length on likelihood. It is calculated as log PT , where P is the likelihood of the response and T is the token number of the response.
Implementation Details In the over-sampling stage, k in top-k sampling and the number of oversampling s are set to 100 and 2000. After the coherence evaluation, 100 candidate responses with the highest similarity are retained. BoB has two decoders, the first decoder is used to generate a preliminary response and the second decoder is used to modify the preliminary response and generate the final response. We only use top-k sampling in the first decoder. The second decoder is a response modifier, so we use greedy search. For Multi-GPT2, we directly use top-k sampling for sampling. We keep the same as BoB3 and Multi-
3https://github.com/songhaoyu/BoB
GPT24 for the parameter settings of the model. For MMI, following as Zhang et al. (2019), we use a pre-trained backward model DialoGPT-reverse to predict source utterances from candidate responses. Source utterances are composed of the concatenation of persona sentences and history utterances. The candidate responses are the same as our method. For LLS, we use the best parameters in Adiwardana et al. (2020): top-k sampling is used to generate responses, k is set to 40, and the number of responses generated is set to 20. The RoBERTa used in the consistency evaluation is RoBERTa-large. The experiments were completed via PyTorch on 4 32GB NVIDIA V100 GPUs.

heading: 4.3 Evaluation Metrics
text: Automatic Metrics In automatic evaluation, we choose the metrics in different aspects to evaluate the quality of responses. For diversity assessment, we use distinct-1/2 (Dis-1/2) (Li et al., 2016). Furthermore, we propose a sentence-level repetition rate (Rep) for evaluating diversity. It is calculated as Rep = nrepN , where nrep is the number of the responses which are the same as at least one other response and that response differs from the ground truth, N is the total number of responses.
For fluency assessment, we use perplexity (PPL) to evaluate the fluency of responses. GPT2 and BERT are chosen as language models to calculate the PPL of responses (Dathathri et al., 2019; Qian et al., 2022), and calculation details are given in Appendix A. For consistency assessment, we use consistency score (C) (Madotto et al., 2019). The BERT-large (Devlin et al., 2018) fine-tuned on DialogueNLI dataset (Welleck et al., 2019) as NLI model is used to evaluate the consistency be-
4https://github.com/caoyu-noob/ Multi-GPT2
tween persona sentences and responses. When the relation between them is entailment, neutral, and contradiction, C is 1, 0, and -1, respectively. To evaluate the overall performance of responses, we calculate the average of the min-max normalized score of each indicator except Rep, recorded as the average score (Avg). PPL and Rep are the lower the better, so use their negative numbers when calculating. The average score which includes Rep is recorded as Avg-R.
Human Evaluations We randomly select 50 examples each from the baselines and our method for human evaluation. Three graduate students with good English skills are asked to rate the quality of responses from fluency (Flue), coherence (Cohe), informativeness (Info), and consistency (Cons). Fluency, coherence, and informativeness are scored on a scale of 1 to 5, where 5 is good, 3 is moderate, and 1 is poor. The score for consistency is 0 or 1, where 0 indicates that the response is inconsistent or irrelevant to persona sentences, and 1 indicates that the response is relevant and consistent with persona sentences.

heading: 4.4 Results
text: Results on Full-size Models As shown in Table 2, our method surpasses two backbone models on all automatic metrics except Dis-1/2, indicating that our method can effectively improve the performance of persona-based dialogue models.
Our method outperforms MMI on all automatic metrics except Dis-1/2, indicating that our postevaluation stage can select the better response from candidate responses. Furthermore, the generation speed of our method is faster than MMI5.
5The details of experiments with generation speed are given in Appendix B.
For LLS, the responses generated by our method outperform it in almost all metrics. Only responses generated by Multi-GPT2 using LLS are lower than those generated by our method on PPL. However, the responses generated by Multi-GPT2 using the LLS have many repetitive responses, of which Rep is 34.80%. The Rep of our method is only 0.93%, indicating that the over-sampling stage of our method can effectively generate diverse responses. Although LLS is faster than our method for generation speeds, it is on average 0.33 lower than our method on two average scores. It is also significantly lower than MMI. In addition, the overall performance of our method outperforms all backbone models and baselines on Avg and Avg-R.
Finally, we use human evaluation to further evaluate responses. As shown in Table 4, our method outperforms backbone models and baselines on all metrics. It shows that the responses generated by our method are more fluent and informative. Meanwhile, they are more coherent to history utterances and more consistent with persona sentences.
Further Analysis of SimOAP First, we analyze the reasons for the choice of method in the postevaluation stage. As shown in Table 8 of Appendix C, the time cost of the learnable coherence evaluation method approaches or even exceeds the generation time of Multi-GPT2, which is unaccept-
able. The TF-IDF algorithm is fast and shows a good evaluation effect, so we choose it.
Furthermore, we compare the effectiveness of Multi-GPT2 using all history dialogue and only the last two sentences of it in the coherence evaluation. The average score of the latter (Multi-GPT2 + SimOAP-Q in Table 2) is slightly higher. We think the reason is that too much history dialogue will cause interference. BoB only uses the last utterance of the history dialogue to generate responses, so we do not need to verify it.
Results on Accelerated Models To speed up our method and verify whether small models using SimOAP can surpass large models, we use BERTmedium and BERT-mini to replace the BERT-base used in BoB. As shown in Table 3, the BERTmedium-based BoB using our method outperforms BoB on PPL, and its size is only 36.6% of BoB. It is worth noting that the BERT-medium-based BoB using SimOAP to generate responses significantly improves diversity and consistency. The BERT-mini-based BoB performs worse than BoB on PPL, which indicates that the original ability of models is also important. For Multi-GPT2, we use DistilGPT2 to replace the GPT2 used in it. After using our method, DistilGPT2-based Multi-GPT2 also surpasses Multi-GPT2 on PPL and consistency score, and its size is only 61.05% of Multi-GPT2.
Number of Candidate Responses Generated To verify the impact of generating different numbers of candidate responses on the performance of SimOAP, we use 100 pieces of data in PersonaChat for experimental verification. BoB is used to generate different numbers of candidate responses, and post-evaluation is used to select the final responses. We use PPL to evaluate the response, and PPL is computed by GPT2. As shown in Figure 3, the PPL of generating 2000 candidate responses is lower than generating 200 or 1000 candidate responses.The PPL increases when the number of
candidate responses is further scaled up to 5000, 7500, or 10000. We believe that the plethora of candidate responses disrupts the post-evaluation stage. Thus, we set the number of generated candidate responses as 2000. In addition, we use the PPL of the backbone model to rerank the candidate responses. The rank of the selected responses is calculated, and the results are shown in Figure 4. The average rank of the selected responses is 1135, and 86 responses are located in the rank with the PPL from 500th to 2000th. This shows that sometimes the dialogue model can generate good responses, but they are just not selected.

heading: 4.5 Ablation Study
text: To verify the effectiveness of coherence evaluation and consistency evaluation, we conduct ablation experiments. As shown in Table 6, when only the coherence evaluation is used, the PPL of the responses increases, indicating that the fluency of the sentences has become worse. The consistency between the responses and the persona sentences also reduce. When only the consistency evaluation is used, although the consistency score is further improved, the PPL of the responses increases, which means the fluency of responses is reduced. Therefore, consistency evaluation and consistency evaluation in the SimOAP method are essential. Finally, we present an example generated using our method and baselines, as shown in Table 5.

heading: 5 Conclusion
text: In this work, we propose a simple but effective two-stage strategy to improve the coherence and consistency in persona-based dialogue generation.
In the over-sampling stage, we use dialogue models for large-scale sampling, and compressed or distilled models are used to accelerate. In the postevaluation stage, multiple well-designed evaluation metrics select the final response from large-scale candidates. Experimental results show that our method improves the backbone models and outperforms the baseline strategies. For reproducibility, we publish the source code6. In future work, we will consider further acceleration of our method.
Limitations
In this work, we generate diverse responses through large-scale sampling in the oversampling stage. Although we use the compression and distillation models to speed up, the problem of generation speed still exists. Thus, one of the limitations of this work is the additional time cost when generating large-scale candidate responses. In addition, we use existing dialogue models for dialogue generation, mainly used in short text generation and unsuitable for long text generation, which is another limitation of this work.
Ethics Statement
Persona-based dialogue generation aims to improve the consistency of open-domain dialogue generation while enabling dialogue generation to be extended to more application scenarios. In persona-based dialogue, the dialogue model uses persona information in the process of dialogue generation. The purpose of using persona information is to improve the consistency of the dialogue system rather than guessing user identities or associating persona information with real-world users. In this work, we use public datasets and do not involve aggression or privacy concerns. Furthermore, we use existing dialogue models for research, so we have the same concerns as other dialogue generation research. For example, there is a risk of generating toxic or biased language.

heading: Acknowledgements
text: This work was supported by the National Key R&D Program of China (2022YFB3103700, 2022YFB3103704), the National Natural Science Foundation of China (NSFC) under Grants No. 62276248, and the Youth Innovation Promotion Association CAS under Grants No. 2023111.
6https://github.com/934865517zjk/ SimOAP

heading: A Additional Indicator Descriptions
text: We use PPL in our automatic evaluation metric for experimental verification. Since our method does not change models, the PPL of models does not change. Thus we choose GPT2 and BERT as language models and input the response into them to calculate PPL. Since the vocabulary of BERT is small, rare words generated by dialogue models may not be in the vocabulary of BERT, neither the baselines nor our method. This will cause the PPL to be huge. So when we use BERT to calculate PPL, the response with PPL greater than 10,000 are removed, both the response generated baselines and our method. Due to the vocabulary of GPT2 being large, the above operations are not required.

heading: B The Experimental Results of Speed
text: We test the generation speed of our method and baselines, and the results are shown in Table 7. The speed in Table 7 is the time required to generate one response. All the generation speed is tested via PyTorch on 4 32GB NVIDIA V100 GPUs. As shown in Table 7, the generation speed of our method is faster than MMI, but slower than LLS. Although the LLS method is fast, the generation effect of LLS is significantly worse than our method as shown in Table 2. Furthermore, the performance of LLS is also significantly lower than that of MMI. Our method mainly brings additional time cost in the over-sampling stage, and the time cost in the postevaluation stage is small. However, MMI takes a lot of time in both the generation and evaluation stages. It also proves that it is reasonable for us to use the TF-IDF algorithm instead of some learnable coherence evaluation method in the post-evaluation stage.
In addition, one of the reasons why BoB is generated significantly slower than Multi-GPT2 is that BoB has two decoders. The first decoder generates a preliminary response, and the second decoder modifies the preliminary response and generates the final response. Thus BoB generates two responses each time. Furthermore, the compression and distillation models effectively speed up our method.

heading: C Further Analysis of Post-evaluation
text: To further analyze the method selection in the over-sampling stage of our method, we choose a learnable coherence evaluation method Quantifiable Dialogue Coherence Evaluation (QuantiDCE)
(Ye et al., 2021) to compare with TF-IDF. QuantiDCE trains a quantifiable coherence metric to reflect the actual human rating standards. QuantiDCE consists of multi-Level ranking pre-training and knowledge distillation fine-tuning. QuantiDCE uses BERT as a feature extraction module to encode the input context-response pair and then inputs the encoded features into a multi-layer perceptron (MLP) to obtain the final coherence evaluation score.
We use 500 pieces of data from the Personachat dataset for experimental validation. We first use the backbone models to generate 2,000 candidate responses each for the 500 pieces of data. Then QuantiDCE or TF-IDF is used to evaluate the coherence of the responses and select the 100 most coherent responses for each piece of data. Finally, the same natural language inference model is used to select the final response.
As shown in Table 8, coherence evaluation in the over-sampling stage using QuantiDCE outperforms TF-IDF on diversity. However, it is worse than TF-IDF in all other indicators. At the same time, the speed of QuantiDCE is much slower than TF-IDF. It is worth noting that for Multi-GPT2, the evaluation time cost of QuantiDCE is close to or even exceeds the time cost required by Multi-GPT2 in the oversampling phase in Table 7. For BoB, the evaluation time cost of QuantiDCE is more than 31% of the over-sampling stage of BoB based on BERT-medium. Such evaluation time cost is unacceptable and avoidable. Combining the above two reasons, we chose fast and effective TF-IDF rather than other learnable methods in the coherence evaluation of the post-evaluation stage.
After the coherence assessment in the postevaluation, 100 highly coherent responses among 2000 candidates responses are selected. In the subsequent consistency evaluation, we use the natural
language inference model to evaluate the consistency of 100 candidate responses. Although the evaluation speed of the natural language inference model is also slow, there are only 100 candidate responses to be evaluated for each dialogue at this time, and the time cost of this process is small, as shown in Table 7. At the same time, the natural language inference dataset DialogueNLI we use is specially built for persona-based dialogue. Many previous works on persona-based dialogue generation have also verified that it works well (Kim et al., 2020; Song et al., 2019, 2020a; Cao et al., 2022). So we chose the natural language inference model fine-tuned on DialogueNLI in the consistency evaluation of the post-evaluation stage.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section Limitations

heading: 3 A2. Did you discuss any potential risks of your work?
text: Section Ethics Statement

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract and Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 4

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: Section 4

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 4

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Section Ethics Statement

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 4
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4
C 3 Did you run computational experiments? Section 4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 4
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 4

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 4

heading: 3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? Section 4

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Section 4

heading: 3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to
text: crowdworkers explain how the data would be used? Section 4
7 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? We only used human evaluation, no data was collected.
7 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? We only used human evaluation, no data was collected.

heading: Future Work
text: In future work, we will consider further acceleration of our method. Limitations
In this work, we generate diverse responses through large-scale sampling in the oversampling stage. Although we use the compression and distillation models to speed up, the problem of generation speed still exists. Thus, one of the limitations of this work is the additional time cost when generating large-scale candidate responses. In addition, we use existing dialogue models for dialogue generation, mainly used in short text generation and unsuitable for long text generation, which is another limitation of this work