A Large Collection of Model-generated Contradictory Responses for Consistency-aware Dialogue Systems (2024)

Abstract: Mitigating the generation of contradictory responses poses a substantial challenge in dialogue response generation. The quality and quantity of available contradictory response data play a vital role in suppressing these contradictions, offering two significant benefits. First, having access to large contradiction data enables a comprehensive examination of their characteristics. Second, data-driven methods to mitigate contradictions may be enhanced with large-scale contradiction data for training. Nevertheless, no attempt has been made to build an extensive collection of model-generated contradictory responses. In this paper, we build a large dataset of response generation models’ contradictions for the first time. Then, we acquire valuable insights into the characteristics of model-generated contradictions through an extensive analysis of the collected responses. Lastly, we also demonstrate how this dataset substantially enhances the performance of datadriven contradiction suppression methods.

heading: 1 Introduction
text: Recent large-scale neural response generation models (RGMs) have made significant progress (Adiwardana et al., 2020; Bao et al., 2021, 2022). However, they still struggle to generate semantically appropriate responses (Roller et al., 2021; Shuster et al., 2022). Among various issues, contradictory responses1 pose a particularly grave concern. For example, in a conversation between speakers A and B, imagine that in response to speaker A’s initial statement, I like tennis, speaker B asks, How often do you play tennis? If speaker A then replies, I hardly ever play. I don’t like tennis, this response would be inconsistent with speaker A’s initial statement. Since these contradictions disrupt the dialogue flow and create a detrimental perception of
1Note that we focus on the contradictions against what is stated in the local context rather than those against the facts in the world. More details are described in Appendix A.
the RGM’s lack of comprehension of the dialogue content (Nie et al., 2020b; Li et al., 2022), addressing them is crucial in developing RGMs to establish a trustworthy and symbiotic relationship with users, even if they are relatively infrequent.
The persistence of contradictory responses in advanced models like ChatGPT2 (Appendix F) suggests that the problem is not merely a matter of scaling but requires targeted efforts. Given this background, several works have proposed approaches for mitigating contradictions (Section 2), but the problem remains unresolved and demands further improvements.
A significant obstacle hindering further progress in suppressing contradictions is the lack of a largescale collection of RGM-generated contradictory responses. This deficiency poses two problems.
First, studies to understand the nature of RGMgenerated contradictions may be impeded. For instance, investigating the correlation between the
2https://openai.com/chatgpt.
16047
presence of a certain feature (e.g., a specific dialogue act label) in a dialogue context and the occurrence of an RGM contradiction can aid in developing more effective strategies for mitigating contradictions. Regrettably, the available resources are limited to Nie et al. (2020b)’s small collection (a few hundred) of RGM-generated contradictory instances intended as test data, making it insufficient for such investigation.
Second, the efficacy of data-driven contradiction suppression may be limited by a scarcity of training data. As evidenced in various NLP tasks (Leite et al., 2020; Mosbach et al., 2020), the performance of data-driven systems is dependent on the volume of available data. Therefore, the efficacy of data-driven contradiction suppression could be improved with access to large-scale RGM contradiction data. Although data-driven approaches have been discussed in previous studies (Section 2.1), they were based on alternative resources such as automatically synthesized contradictions or humanwritten contradictions (Nie et al., 2020b; Li et al., 2022). However, contradictions generated by automatic synthesis or manually are different in characteristics from those actually generated by RGMs (Section 4.1). If one tries to handle RGMs’ contradictions with models trained on alternative resources, the potential of data-driven methods may not be fully realized because of the discrepancy between the training data and the practical inference targets, as demonstrated in Section 5.
In this paper, we demonstrate the effectiveness of having a vast repository of RGM-generated contradictory responses in tackling RGM contradictions. To begin with, we build a large-scale dataset comprising 10K contradictory and 17K noncontradictory responses generated by various highperformance RGMs. The consistency of each response is judged by three human annotators, as illustrated in Table 1. To our knowledge, this is the first work to construct a dataset containing more than 1K contradictory RGM responses with human annotations. We then analyze our collection from various angles, yielding valuable insights into RGM contradictions. We also demonstrate that a contradiction detector trained on human-written contradiction data exhibits limited accuracy in identifying RGM contradictions, and training on our dataset improves this situation. Our dataset will be made publicly available (https://github.com/ shiki-sato/rgm-contradiction).

heading: 2 Related studies

heading: 2.1 Major methods to handle contradictions
text: The mainstream approaches of prior studies to mitigate contradictions have been data-driven. Welleck et al. (2019) developed a dialogue-domain natural language inference dataset by applying a rule-based method to transform an existing dialogue corpus. They employed this dataset to train a contradiction detector that automatically identifies contradictions within pairs of dialogue domain sentences. Nie et al. (2020b) gathered and used 15,605 contradictory and 15,605 noncontradictory human-written dialogue utterances to train a contradiction detector. They also collected 382 RGM-generated contradictory responses as test data to evaluate detectors. Meanwhile, Li et al. (2020) and Li et al. (2022) updated RGMs using a loss function that reduces the likelihood of generating inconsistent responses. This study would be the first attempt to collect RGM-generated contradictions to provide valuable resources for these data-driven methods.

heading: 2.2 Effective inputs to collect contradictions
text: Previous studies have demonstrated that RGMs tend to generate contradictions when they repeat previously stated facts or opinions (Nie et al., 2020b; Li et al., 2021). Nevertheless, posing questions that prompt dialogue partners to repeat previously stated information can be uncommon in natural dialogues. On the other hand, we aim to collect RGM contradictions by identifying contradictions in RGM responses to follow-up questions. Followup questions seek additional information related to the information previously stated by the dialogue partner. These types of questions commonly arise during dialogues. Follow-up questions are similar to the abovementioned queries (i.e., questions requesting repetitions of previously mentioned facts or opinions) as they both seek information related to the previously stated content. With this similarity, we hypothesized that follow-up questions would also tend to induce RGM contradictions and employed them as inputs for data collection.

heading: 3 Dataset construction
text: This paper showcases the importance of employing extensive datasets containing RGM-generated contradictory and noncontradictory responses to mitigate RGM contradictions effectively. As stated earlier, large-scale data are currently lacking. To address this issue, we first perform an extensive
2. Collect responses
1
Figure 1: Overview of our data collection process.
collection of RGM-generated instances. This section outlines the methodology used to build our dataset, followed by the detailed settings and the data collection results for this study.
3.1 Method Figure 1 illustrates our data collection process. We rst prepare dialogue contexts as input and then collect their RGM responses. The collected responses are classi ed into contradictory or noncontradictory groups according to their contexts. This process is based on that used by Nie et al. (2020b), with the differences being the approach to the context preparation and the focus on RGM responses instead of human-written ones.
3.1.1 Dialogue context preparation Contradictory responses areinconsistentwith contexts; hence, their occurrences depend on their contexts. For instance, it is improbable that a contradiction will occur in a context where only greetings are exchanged. Based on previous insights (Section 2.2) and our preliminary analysis (Appendix B), we gatherfollow-up questions (FQs) as the prime contexts for eliciting contradictions.
Note that we acknowledge that addressing all contradiction types solely by examining the contradictions to FQs is impractical since RGMs do not generate contradictions exclusively to FQs. Nevertheless, we believe that re ning contradiction suppression techniques using contradictory responses to these representative inputs can establish the groundwork for attempts to mitigate contradictions in a broader input range. In fact, the experimental results in Section 5.2 show that a contradiction detector trained on our collection effectively identi ed the contradictions in responses to non-FQ contexts.
Figure 2: Example ofCw=of u q ;C mid g andCw=of u q ;r g.
Notations. In Figure 2,C refers to a dialogue context concluding with an utterance3 uq that contains a questionq. We useur 2 C to represent the utterance that precedesuq by dur utterances. 4 Whenq is a question that refers to a speci c segment r in ur ,5 as illustrated on the left side of Figure 2, we regardq as an FQ for . Throughout this paper, the segmentr to which q refers will be termed the referent ofq. In addition, as Figure 2 shows,Cw=of uq ;Cmid g refers toC, excluding uq and the intervening utterancesCmid betweenur anduq. Similarly, Cw=of uq ;r g representsC with bothuq andr removed. Idea for collecting FQ. To determine whetherr is the referent ofq, i.e., whetherq is an FQ forr , we must check whetherr andq are relevant. Fortunately, some of the currently available RGMs are reliable enough to generate responses with a level of relevance comparable to human responses (Zhang et al., 2020; Adiwardana et al., 2020) while the issue of inconsistency remains unresolved; hence, these RGMs are expected to effectively capture the relevance between utterances. In other words, if a large-scale neural RGM deduces the strong relevance betweenr andq, we can reasonably consider q as an FQ for . Therefore, in the following paragraph, we introduce a new automatic metric that uses a neural RGM to assess the relevance between q andr .6 Method for collecting FQ. If q is an FQ forr , it is improbable forP (uqjCw=of uq ;Cmid g) to exhibit
3 Utterance in this study refers to all sentences in a turn. 4Note that we only consider scenarios wherein theur and uq speakers are distinct individuals. 5Let q represent all interrogative sentences within an utt rance, whiler encompasses all other sentences in the same utterance. Any sentences terminated with a question mark are identi ed as interrogative. The segmentation of an utterance into individual sentences is accomplished utilizing spaCy (en_core_web_sm) (Honnibal and Montani, 2017).
6We see no critical problem in employing an RGM for this end since we employ human evaluation for contradictoriness labeling to ensure the quality of the nal product of data collection as described in Section 3.1.3.
a decrease compared to uq’s original conditional probability. Moreover, P (uq|Cw/o{uq ,r}) is likely lower than uq’s original conditional probability. Consequently, the following value, FQness score, is deemed high when q is an FQ for r:
P (uq|Cw/o{uq ,Cmid})/P (uq|Cw/o{uq ,r}).
Here, we compute the probabilities using an RGM.7 We collect FQs by selecting samples with the highest FQness scores from a pool of C. Appendix C illustrates the preliminary experiment to assess the effectiveness of picking contexts based on FQness for efficiently gathering RGM contradictions.

heading: 3.1.2 RGM response collection
text: For every gathered C with a high FQness, multiple RGMs generate responses to gather diverse contradictions from various RGMs efficiently.

heading: 3.1.3 RGM response annotation
text: We assign three human workers to assess each generated response and categorize it into two groups, contradictory and noncontradictory, according to their preceding referent r in ur. If at least two workers determine the presence of contradictions in a response, this response is labeled contradictory. If all workers agree that a response is consistent, this response is labeled as noncontradictory.

heading: 3.2 Construction settings

heading: 3.2.1 Settings of dialogue context preparation
text: A pool of C is formed by extracting dur or more consecutive utterances from dialogue corpora, ensuring that the final utterance includes questions. From this pool, we gather those with the highest FQness scores. For this study, we gathered C from the Multi-session Chat (MSC) dataset (Xu et al., 2022). This dataset exhibits distinctive features that make it an ideal source for collecting C: (1) low noise (e.g., few misspellings), and (2) realistic dialogues between acquaintances, wherein speakers engage in in-depth discussions on various topics.
Since the annotation cost extremely increases as the value of dur increases,
8 the highest value assigned to dur was 5 for this study, and we intensively gathered FQs with dur = 1, 3 to prepare for
7We employed Blenderbot (Roller et al., 2021) implemented in ParlAI (Miller et al., 2017), a well-known highperformance RGM.
8As written in Section 3.1.1, dur refers to the distance between ur and uq . When the value of dur is large, q denotes the FQ associated with the earlier utterance in the context.
collecting contradictory responses. From the set of approximately 59K of C in the MSC dataset, we extracted 3,250, 1,000, and 100 samples for dur = 1, 3, and 5, respectively, based on the FQness scores.

heading: 3.2.2 RGM response collection
text: We employed eight high-performance RGMs: Plato-2 (P2), Plato-XL (PX) (Bao et al., 2021, 2022), Blenderbot1-3B (B1), Blenderbot2-3B (B2) (Xu et al., 2022; Komeili et al., 2022), Blenderbot3-3B (B3) (Shuster et al., 2022), Blenderbot3-30B (BL), Opt-66B (O6) (Zhang et al., 2022), and ChatGPT (CG). Each RGM generated a response to an input, resulting in eight responses for each C. Appendix D presents detailed settings.

heading: 3.2.3 RGM response annotation
text: We used Amazon Mechanical Turk9 to recruit workers. We ensured the creation of a high-quality, cost-effective data set by carefully selecting highly skilled workers. The selection procedure is shown in Appendix E. During the data collection phase, we published tasks to workers that required classifying 40 responses for five C.

heading: 3.3 Construction results
text: Table 2 reports our dataset statistics. Throughout the annotation process, the groups of three workers achieved average Fleiss’ kappa values of 0.405, 0.465, and 0.408 for dur = 1, 3, and 5, respectively. Given the intricacies involved in identifying contradictions, the substantial level of consensus signified the successful creation of a high-quality dataset. Table 1 provides examples of our dataset. Each sample comprises a dialogue context C containing ur and uq, and an RGM response contradictory or noncontradictory to r in ur with the labels assigned by the three workers.
9www.mturk.com.

heading: 4 Dataset analysis
text: Analyzing the characteristics of RGM contradictions is crucial in devising innovative approaches to mitigate contradictions. The examinations performed by prior studies have been limited by the lack of extensive data, a gap that our dataset effectively fills. Sections 4.1 and 4.2 delineate the results of analyzing these characteristics using our data set. Section 4.1 delves into the intrinsic features of the generated responses, while Section 4.2 presents an examination centered around the dialogue contexts that trigger RGM contradictions.

heading: 4.1 Analysis of RGM-generated responses
text: Our analysis identified two types of characteristic contradictions in the RGM responses: contradictions arising from intra-utterance inconsistencies and those related to ambiguous expressions.

heading: 4.1.1 Intra-utterance inconsistencies
text: A qualitative analysis of the RGM contradictions in our dataset revealed that one distinctive way contradictions occur is through intra-utterance inconsistency. Table 3 provides an example of this type of contradiction. In this conversation, Opt66B generated inconsistent information within a single utterance, saying, I’ve been to this place while stating, It’s in a country I’ve never been to. When conflicting information coexists within a statement, it becomes highly probable that at least one of them contradicts the context. Instances of contradictions stemming from intra-utterance inconsistencies were occasionally observed across multiple RGMs. To delve deeper into this phenomenon, we counted the inconsistencies among the 50 randomly selected contradictory responses in our dataset for each of the eight RGMs. Our findings indicated that seven RGMs generated at least 4 (8%) contradictory responses featuring an intra-utterance inconsistency. Conversely, none of the 200 human-written contradictory responses
randomly sampled from the DECODE dataset exhibited an intra-utterance inconsistency. See Appendix G for detailed results. These results suggest contradictory responses featuring intra-utterance inconsistencies are particularly frequent in RGM responses.

heading: 4.1.2 Ambiguous expression
text: We observed a notable distinction in the human annotation tendency on the existence of contradictions between the set of human-written responses in DECODE and our compilation of RGM-generated responses. Both our study and Nie et al. (2020b) employed a similar approach in selecting the human workers who identified contradictions during the data creation process (Section 3.2.3). However, within the subset of instances where at least one worker detected the contradictions, a significant gap was observed in the proportions where the other two workers also concurred on the existence of contradictions. This proportion was 78.4% for the human-written responses and 30.4% for the RGM-generated ones. This dissimilarity could have stemmed from the RGM’s propensity to generate ambiguous expressions concerning consistency, as demonstrated in Table 4. Such responses appeared to result in differing judgments regarding the presence of contradictions, depending on how individual workers interpreted them. Suppressing these contradictions is crucial, even if some workers may miss the inconsistencies, because they, once perceived as contradictory by actual users, can significantly detriment the quality of dialogues.

heading: 4.2 Analysis of dialogue contexts
text: If specific dialogue contexts induce contradictory responses from various RGMs, identifying their contributing characteristics may become crucial in developing more effective contradiction mitigation
techniques. Our dataset is suitable for this investigation because it contains a lot of C for which diverse RGMs generate responses. We conducted a further examination to identify features of C that induce contradictions from a lot of RGMs, employing statistical tests on our dataset.10 It is noteworthy that this type of statistical analysis becomes feasible due to the creation of a large collection of RGM-generated contradictions, such as our dataset. Our analysis focused on dialogue act labels and lexical attributes, which are highly interpretable and seem particularly well-suited as focal points of the first analysis.
Analysis results of dialogue acts. When we assigned SWBD-DAMSL dialogue act labels (Jurafsky et al., 1997) to uq in our dataset,11 we observed a notable trend, that is, uq categorized as ‘Declarative Yes-No-Questions’ or ‘Statement-nonopinion’ were more prone to triggering contradictions. Among the 193 assigned instances for the former label, the average count of the contradictory responses from the eight RGMs per C was 2.77 (i.e., 2.77 contradictory responses / 8 generated responses = 35%). The average for the 4084 unassigned instances was lower at 2.41 (30%). This phenomenon could have arisen from a deficiency in the RGM’s ability to generate appropriate responses while being cognizant that a repetition of previous information is being solicited. Focusing on 2,118 assigned instances for the latter label indicated a higher average of 2.49 (31%) contradictory
10Initially, we categorized each of C into two sets based on the presence or absence of a certain feature, such as whether the utterance uq contains the word “how.” Subsequently, for each of these two sets, we computed the average number of contradictory responses elicited by one C from the eight RGMs. We regarded a feature of C as the one inducing many RGM contradictions if a statistically significant difference in the average number between the two sets was identified with a one-tailed t-test at a 1% significance level.
11See Appendix H for detailed labeling settings.
responses compared to an average of 2.36 (30%) for the 2,159 unassigned ones. This disparity could have arisen from the RGMs’ inability to differentiate between the dialogue partners’ statements and their own utterances in dialogue contexts. Hence, RGMs might have generated responses incorporating the partners’ information as if it were their own, even if it is inconsistent with their past statements, as exemplified in Table 5.
Analysis results of lexical features. The uq containing the interrogative term “how” can provoke contradictions. More precisely, the mean count of the contradictory responses within the 764 applicable contexts stood at 2.60 (33%), while that in the 3,513 inapplicable contexts was 2.39 (30%). Answering “How questions” while upholding consistency with the context poses a challenge for the current RGMs.

heading: 5 Experiments
text: This section presents compelling evidence to support the hypothesis that employing the RGMgenerated contradiction collection as a training resource yields notable enhancements in the effectiveness of data-driven contradiction suppression methods. As a case in point, we focus on developing a contradiction detector that automatically classifies whether or not a given utterance pair is contradictory by training it on our dataset. Contradiction detectors are commonly employed in post-processing tasks that filter out RGMs’ contradictory response candidates (Welleck et al., 2019; Nie et al., 2020b) and automatic evaluations of RGMs’ contradiction frequencies (Li et al., 2021), effectively playing a crucial role in mitigating contradictions.
Existing detectors have been developed by employing automatically synthesized or humanwritten contradictions as substituting training resources for RGM contradictions. We hypothesize that their performance can be enhanced by utilizing RGM contradiction data for their training. Our experiments validate the potency of our dataset by assessing the contradiction detection performance of a detector trained on our dataset against that of a detector trained with human-written contradictions.

heading: 5.1 Settings
text: Inputs and outputs. Like the utterance-based detectors of Nie et al. (2020b), given a dialogue response and the corresponding preceding utterance
ur, a detector yielded a binary classification result indicating whether the response contradicted ur.
Evaluation Metrics. We evaluated the performance of detectors based on the accuracy of the binary classification results.
Contradiction detectors. We conducted a performance analysis of a detector that underwent training on our dataset, juxtaposed with a detector fashioned similarly to the state-of-the-art detector devised by Nie et al. (2020b). Their detector was developed by fine-tuning RoBERTa (Liu et al., 2019) on the DECODE dataset specifically for the binary classification tasks requiring the prediction of consistency within a pair of given utterances. Following their settings, we developed a Contradiction Detector by fine-tuning RoBERTa on our dataset, denoted as CDOUR. Similarly, we constructed a rival detector, CDDEC, using an equivalent number of instances from the DECODE dataset as CDOUR.
Training data for CDOUR. Our dataset contains both contradictory and noncontradictory responses from eight RGMs. Our experiment performed a cross-validation test by selecting one RGM (i.e., target RGM) and using its responses as the test data. The samples excluding the target RGM’s responses were used for training. We realized a comprehensive assessment of the detectors’ performance by conducting the evaluation process eight times, varying the target RGMs each time. When we selected B2 as the target model, the number of training data samples was minimized to 8,023 contradictory and 8,023 noncontradictory responses; we reduced the number of training data samples to align with this number when we specified one of the other RGMs as a target RGM. Appendix J presents the training details.
Training data for CDDEC. We randomly selected 8023 contradictory and 8,023 noncontradictory human-written responses from the DECODE dataset. Other settings are the same as CDOUR.
In-domain test sets. As test samples, we randomly selected 100 contradictory and 100 noncontradictory responses of the target RGM responses from our dataset. Note that a training set might also contain responses of non-target RGMs that share the same contexts as these 200 test samples. We excluded these samples from the training set to ensure a fair evaluation of the detectors’ ability to
identify contradictions from unknown RGMs for unfamiliar contexts.
Out-of-domain test sets. The above RGMgenerated test sets are derived from the corpus used to develop the training set for CDOUR. Furthermore, these sets exclusively comprise responses to FQs. To assess the detector’s effectiveness in identifying contradictions in RGM responses to non-FQ contexts from unfamiliar dialogue corpora, we prepared two out-of-domain test sets. One set originated from the Topical-Chat dataset (Gopalakrishnan et al., 2019), and the other from the DailyDialog dataset (Li et al., 2017). Each of these sets comprises seven subsets, each containing 50 contradictory and 50 noncontradictory responses from P2, PX, B1, B2, B3, BL, or O6.12 The contexts of these sets were randomly selected from all contexts concluding with utterances containing questions not limited to FQs, in the corpora.13 Appendix I details the construction process. The subsets from these two test sets were used in a manner resembling a cross-validation test, akin to how the subsets of the in-domain test set were employed. However, unlike the in-domain subsets, even the subsets of non-target RGMs’ responses were excluded from the training set to prevent detectors from being trained on the same domain data. In addition, we employed Nie et al. (2020b)’s Human-Bot dataset, which possesses 382 contradictory and 382 noncontradictory RGM responses in human-bot dialogues.
Human-written test set. We utilized Nie et al. (2020b)’s By-Human test set comprising 2,108 contradictory and 2,108 noncontradictory humanwritten responses. This allows us to verify that CDDEC is reasonably well-trained in our settings, although detecting human-written contradictions falls beyond the scope of our study.

heading: 5.2 Results
text: Table 6 (a), (b), and (c) display the accuracy of the contradiction detectors for the human-written, indomain, and out-of-domain test sets, respectively.
(a) Human-written test set. CDDEC obtained a high accuracy of 0.952 on the By-Human test set,
12CG was omitted from the test set construction due to cost considerations, as CG’s contradiction frequency was low (Appendix F).
13Considering that non-question contexts may allow contextually irrelevant replies, such as prompting changes in the topic, we anticipate a lower occurrence of contradictions. Our focus on responses only to questions is in accordance with cost considerations.
confirming that CDDEC was properly trained.14
(b) In-domain test sets. CDDEC, in contrast to the results for the Human-written test set, achieved low accuracy for the subsets of our RGM-generated dataset. Particularly, it had an accuracy of only 0.540 when B2 was the target RGM, which is problematic in practical applications. In contrast, CDOUR gained higher accuracy on our RGMgenerated test sets. The training process for CDOUR excluded any contradiction data from the target RGMs and samples that shared the same dialogue contexts as the test data, thereby effectively detecting contradictions from unknown RGMs when confronted with unfamiliar contexts.
(c) Out-of-domain test sets. For all three test sets, the performance of CDOUR significantly outperformed CDDEC. These results emphasize that detectors trained on our dataset can effectively detect contradictions in RGM responses for contexts that are out-of-domain and non-FQ.
Note that it has been confirmed that CDOUR exhibited superior performance even when the entirety of DECODE’s samples was employed for training CDDEC, although the above experiments employed only approximately half of DECODE’s samples during CDDEC’s training. Furthermore, we have verified that CDOUR outperformed all seven detectors constructed similarly to the seven baseline detectors employed in Nie et al. (2020b)’s experiments. See Appendix K for detailed results.
14See Appendix L for more detailed results on this test set.

heading: 5.3 Analysis: Performance improvement in detecting RGM-specific contradictions
text: The above results exhibited that training detectors with the RGM contradictions led to a noticeable enhancement in the detectors’ capability to identify RGM contradictions. We hypothesized that this outcome could be attributed, at least in part, to the training on the RGM-generated instances, which facilitated the acquisition of identifying features typical of RGM contradictions, encompassing those expounded upon in Section 4.1. We investigated this hypothesis’ validity by taking the contradiction type mentioned in Section 4.1.2, contradiction with ambiguous expression, as an example.
Idea and method. Our experiments revealed that 1,377 RGM contradictory responses from our in-domain test sets and the validation sets used when training CDOUR (see Appendix J for details) were missed by CDDEC but successfully flagged by CDOUR. Plausibly, some of these instances may exhibit certain features inherent to the RGM contradictions, which the training with RGM-generated data facilitated CDOUR to recognize. Therefore, we investigated if the 1,377 contradictory responses encompass the distinguishing characteristic, i.e., ambiguous expression. For simplicity, in this analysis, we considered a contradictory response to be a sample with ambiguous expressions if only two of the three workers judged it contradictory.
Results. Within those above 1,375 contradictory responses, the proportion of the samples classified as contradictory by only two workers amounted to 51.3%. Conversely, among the 4,378 contradictory responses from our validation and test sets that both
CDDEC and CDOUR successfully identified, only 43.3% of the samples were determined contradictory by two workers. This proportion gap exhibited statistical significance at the 1% significance level in the chi-square test, underscoring that training on RGM-generated data enhanced the detector’s capacity to recognize the contradictions characterized by ambiguity.

heading: 6 Conclusion
text: No attempt has been made to build an extensive collection of RGM-generated contradictory responses, which is problematic in two aspects: the scarcity of data for analysis and training.
In this paper, we built a large collection of contradictions generated by various RGMs for the first time. We comprehensively analyzed our collection, producing valuable insights into the RGM contradictions, which we believe are crucial for effective contradiction suppression. We also demonstrated that a contradiction detector trained on our dataset could identify RGM contradictions effectively.
Future challenges include applying the collected dataset to other data-driven methods and collecting data with a broader context variety than FQs.
Ethical concerns
This study uses existing datasets, the MSC, TopicalChat, and DailyDialog datasets, which we consider not to bring any ethical concern. We added to these datasets and released a set of RGM-generated responses along with binary labels denoting the presence or absence of contradictions. Regarding the responses, it is conceivable that the aggressive expressions generated by the RGMs may be present in the collected contradictory and noncontradictory responses. As for the labels, we have meticulously removed any personal information belonging to the workers to share our dataset ethically.
Limitations
Dialogue context types. Our compiled dataset exclusively comprises contradictions in RGM responses to dialogue contexts that conclude with FQs extracted from the MSC dataset. The experiments detailed in Section 5 demonstrated that utilizing our dataset addressed RGM contradictions effectively, even for the responses to non-FQ dialogue contexts sourced from different corpora. However, broadening the collection of contradictory responses to encompass a diverse array of dia-
logue contexts could facilitate a more comprehensive analysis of RGM contradictions and potentially lead to enhanced contradiction suppression through a data-driven approach.
Target RGMs. It remains unclear whether employing our gathered dataset can effectively mitigate contradictory responses by any RGM. Our data compilation involved the responses from various recent and representative high-performance RGMs. More importantly, Section 5 outlined that the contradiction detector, trained using our dataset, effectively identified contradictory responses from unknown RGMs. Nonetheless, we must acknowledge the possibility that it might struggle to suppress contradictions in responses from newer RGMs. Despite this uncertainty, we believe collecting contradictory responses from recent high-performance RGMs is crucial for developing dialogue systems that can generate consistent responses.
Indirect contradictions. This study focuses exclusively on pairs of utterances where the contained information directly contradicts each other. However, even when the information within each utterance is consistent, some utterance pairs might still be considered contradictory. For instance, when the information stated in the previous RGM utterance is repeated in the subsequent RGM response, the unnaturalness of the situation might be perceived as contradictory. Since such contradictions do not involve directly contradictory information, they cannot be identified as contradictions using this study’s dataset and approach.
Modeing contradictoriness. How the definition of contradiction can be refined beyond binary labeling is an open question. In this study, we stayed with the standard binary view of contradictions since the binary definition of contradiction has long stood in the literature of natural language inference (Dagan et al., 2013). Nevertheless, we also observed ambiguous or vague cases in our dataset, as discussed in Section 4.1.2, which could motivate introducing more fine-grained multi-class classification or scaling.

heading: Acknowledgments
text: This work was partly supported by JSPS KAKENHI Grant Numbers JP22K17943, JP21J22383, and JST Moonshot R&D Grant Number JPMJMS2011-35 (fundamental research).

heading: A Contradiction types
text: Two major contradiction types are identified in the context of dialogue response generation: (i) contradictions against the facts in the world outside of the ongoing dialogue (e.g., personas) and (ii) those against what is stated in the local preceding context (e.g., opinions) (Li et al., 2020; Nie et al., 2020b). This study focuses on suppressing the second type. While several studies have addressed the issue of avoiding the first contradiction type (Li et al., 2016; Zhang et al., 2018; Qian et al., 2018; Kottur et al., 2017; Kim et al., 2020), given that the multi-turn human-bot interaction is attracting increasing interest, we believe that tackling the issue of the second type is becoming increasingly important.

heading: B FQ analysis in existing dataset
text: We randomly examined 50 responses from a pool of 382 RGM-generated contradictory responses in the human-bot dialogues collected by Nie et al. (2020b). Remarkably, 25 (50%) of these 50 contradictory responses were elicited by FQs, strongly indicating that FQ plays a prominent role in provoking RGM contradictions.

heading: C Preliminary experiment
text: C.1 Experimental procedures We first extract samples from a pool of C for three cases (i.e., dur = 1, 3, and 5) by random sampling (RANDOM) and picking samples with the highest FQness (TOP). Subsequently, we employ multiple RGMs to generate responses to the C collected by RANDOM and TOP. We then compare the number of the RGM responses contradicting the C obtained through the two abovementioned methods.
C.2 Experimental settings Settings of dialogue context preparation. We utilized the pool of C described in Section 3.2.1 as the source for extracting instances using RANDOM and TOP. Each TOP and RANDOM extracted 100, 100, and 50 samples from the pool for dur = 1, 3, and 5, respectively.
Settings for RGM response collection. Seven RGMs were employed to generate the responses for each C: Plato-2, Plato-XL, Blenderbot1-3B, Blenderbot2-3B, Blenderbot3-3B, Blenderbot330B, and Opt-66B.15 We specifically had each
15Note that the ChatGPT API service was not yet available when the preliminary experiment was conducted.
RGM generate 100 response candidates for each input through top-p sampling (Holtzman et al., 2020), with a value of p set to 0.5. We chose the response with the highest generation probability among the 100 candidates. Among all the employed RGMs, only OPT-66B generated responses using a 5-shot approach implemented in ParlAI. We employed Knover16 for Plato-2 and Plato-XL, and ParlAI for the others.
Settings for RGM response annotation. Each RGM response was manually assessed to determine its consistency with the context. Amazon Mechanical Turk’s 10 workers assigned to each response performed a binary classification task to distinguish between the contradictory and noncontradictory responses. We solely focused on evaluating the consistency with ur due to cost considerations.
C.3 Experimental results
Table 7 displays the comparison results, which confirmed that more contradiction labels were assigned to the responses for C with a high FQness. This observation underscored the tendency of C with a higher FQness to provoke more RGM contradictions.
16www.github.com/PaddlePaddle/Knover.

heading: D Settings for dataset construction
text: Each of the eight RGMs generated one response to an input, resulting in eight responses for each C. We enhanced the efficiency of gathering the contradictions by choosing the final response of an RGM to an input from the top 100 candidates with the highest contradiction probability predicted by the state-of-the-art contradiction detector (Nie et al., 2020b). We utilized top-p sampling to collect the 100 candidates. We set a value of p to 0.5, which was lower than the default value of 0.9 used in major platforms, such as ParlAI (Miller et al., 2017), to avoid sampling responses with low generation probabilities. This allows us to gather candidates with a high generation probability by the RGM and a high likelihood of being contradictory. As in Appendix C, only OPT-66B generated responses using a 5-shot approach. We employed OpenAI’s API17 for ChatGPT. For the other RGMs, we used the same platforms as described in Appendix C.

heading: E Worker selection for dataset construction
text: We first presented a task with obviously correct answers. It contained 21 dialogue responses requiring classification into contradictory or noncontradictory according to their preceding referent r. We exclusively handpicked workers who scored fewer than two incorrect answers in this task.

heading: F Details of collected dataset
text: Table 8 illustrates the number of contradictory and noncontradictory responses obtained from each of the eight RGMs outlined in Section 3.
17https://platform.openai.com.

heading: G Frequency of intra-utterance inconsistencies
text: Table 9 illustrates the frequency of intra-utterance inconsistencies in 50 randomly sampled contradictory responses of each of the eight RGMs in our dataset and 200 randomly extracted human-written contradictory responses from the DECODE dataset. The counting of intra-utterance inconsistencies was carried out by the author.
Our findings indicated that seven RGMs generated at least 4 (8%) contradictory responses featuring an intra-utterance inconsistency. The sole exception was ChatGPT (CG), whose subset of contradictory responses did not encompass intrautterance inconsistency. This suggests that a small number of large-scale RGMs, such as ChatGPT, are progressing toward eradicating inconsistencies within individual utterances. Nevertheless, even a sophisticated model like Opt-66B generates contradictions with intra-utterance inconsistency.
Conversely, none of the 200 responses randomly sampled from the DECODE dataset exhibited an intra-utterance inconsistency (“Human” in Table 9). These results suggest contradictory responses featuring intra-utterance inconsistencies are particularly frequent in RGM responses.

heading: H Settings of dialogue act labeling
text: For the analysis in Section 4.2, we developed a labeler to assign dialogue act labels to the utterances uq in our dataset.
H.1 Development of dialogue act labeler
Inputs and outputs. Suppose that the t-th utterance ut in a dialogue consists of the n segments (ut,1, ut,2, ..., ut,n) and that each segment is assigned one dialogue act label. In addition, let the utterance immediately preceding ut be ut−1.
We design a labeler to predict the dialogue act label of the i-th segment ut,i within ut, using ut−1 and the segments of ut up to the i-th segment (ut,1, ut,2, ..., ut,i) as inputs.
Dataset for training. We developed a labeler by fine-tuning RoBERTa.18 We employed the Switchboard Dialogue Act Corpus with the 42 clustered SWBD-DAMSL dialogue act labels (Jurafsky et al., 1997) for fine-tuning. The preprocessed dataset we used for this study19 comprises training data with 192,390 segments, validation data with 3,272 segments, and evaluation data with 4,078 segments.
Hyperparameters. We fine-tuned RoBERTa by employing the implementation of Hugging Face (Wolf et al., 2020) with its default settings, excluding a few parameters.20
Performance of developed labeler. The test set accuracy on the aforementioned preprocessed dataset, used for fine-tuning RoBERTa, was 80.4%.
18https://huggingface.co/FacebookAI/ roberta-large.
19https://github.com/shreyangshu12/ Dialogue-act-classification.
20early_stopping_patience: 2, learning_rate: 1e-5, train_batch_size: 256, and weight_decay: 0.01
H.2 Dialogue act labeling for our dataset The developed labeler was utilized to assign dialogue act labels to the utterance uq in our dataset. To simplify the process, we treated each sentence in uq as a segment,21 assigning a dialogue act label to each sentence. We used the utterance immediately preceding uq and all sentences up to the i-th in uq as input in order to predict the dialogue act label for the i-th sentence in uq. This process was applied to all sentences within uq. In our analysis in Section 4.2, for the sake of simplicity, we regarded that a certain label had been assigned to uq if one or more sentences within uq were assigned that specific label.

heading: I Settings for test set construction
text: We first constructed two pools of C by extracting dur = 1, 3 or more consecutive utterances from the Topical-Chat dataset and the DailyDialog dataset, respectively, ensuring that the final utterance contains questions. From each of the two pools, we randomly sampled those consecutive utterances. Specifically, for the Topical-Chat dataset, we sampled 300 and 100 samples from the pool for
21We split an utterance into sentences using NLTK sentence tokenizer (Bird and Loper, 2004).
dur = 1 and 3, respectively. Similarly, for the DailyDialog dataset, we sampled 200 and 100 samples from the pool for dur = 1 and 3, respectively. The other settings were the same as in our large-scale dataset construction described in Section 3, except for the method of collecting C described above and that responses of ChatGPT were not collected.

heading: J Settings of detector training
text: Samples for training. A negative pair comprised a contradictory response in our dataset and the preceding utterance ur. In contrast, a positive pair comprised a noncontradictory response from our dataset and one randomly selected from its preceding utterances by the same speaker. This was because the responses annotated as noncontradictory with ur are also likely to be noncontradictory with the other preceding statements. By introducing randomness into the selection of preceding utterances for pairing with a noncontradictory RGM response, we aimed to create positive pairs comprising unrelated utterances. These pairs could be valuable for training detectors to recognize that unrelated pairs should be categorized as noncontradictory.
Hyperparameters. We fine-tuned RoBERTa22 by employing the implementation of Hugging Face (Wolf et al., 2020) with its default settings, excluding a few parameters.23 We updated the model parameters until we reached a point where early stopping was triggered. Early stopping was determined by assessing the accuracy of validation data, a distinct subset comprising 10% of the training data and withheld from the training process. We saved the model parameters with the highest accuracy on the validation data at each learning rate and ultimately selected that with the highest validation accuracy among all the saved parameters.

heading: K Comparison to diverse rival detectors
text: Table 11 presents the outcomes of evaluating the contradiction detection capabilities of CDOUR in comparison to diverse rival detectors.
Comparison to baselines employed in Nie et al. (2020b). In addition to CDDEC, we developed seven detectors named CDSNLI+MNLI, CDALL,
22https://huggingface.co/FacebookAI/ roberta-large.
23train_batch_size: 128, weight_decay: 0.01, eval_steps: 200, early_stopping_patience: 1, and learning_rate: {1e-6, 5e-6, 1e-5, 5e-5}.

heading: 1 .762 (772 / 1013) .956 (930 / 973)

heading: 3 .762 (425 / 558) .968 (430 / 444)

heading: 5 .770 (191 / 248) .987 (376 / 381)

heading: 7 .833 (160 / 192) .973 (182 / 187)

heading: 9 .887 ( 63 / 71) .976 ( 80 / 82)

heading: 11 .714 ( 10 / 14) 1.00 ( 33 / 33)

heading: 13 1.00 ( 8 / 8) 1.00 ( 5 / 5)
text: CDALL−DNLI, CDALL−ANLI, CDALL−DEC, CDDNLI, and CDANLI, corresponding to the seven RoBERTa-based baseline detectors utilized in Nie et al. (2020b)’s experiments. The training settings for all detectors were consistent with those of CDDEC, with the only variation being the source of the training data (Table 10). We subsequently juxtaposed these detectors with CDOUR. Table 11 illustrates that CDOUR, formed using our dataset, exhibited a higher accuracy in identifying RGM contradictory responses compared to those rival detectors.
Comparison to a detector trained with more human-written data. Additionally, we developed the detector CDDECfull through training on all instances within the DECODE dataset, encompassing 15,605 contradictory and 15,605 noncontradictory responses. This training process followed the methodology employed for CDDEC. Noteworthy is the observation that, despite CDOUR’s training dataset being approximately half the size of CDDECfull, it demonstrated superior performance across all RGM-generated test sets, with the exception of the Human-Bot set. The HumanBot test data comprises the utterances in firstmeeting dialogues. Given that CDDECfull’s training dataset also encompasses human-written contradictory responses following first-meeting dialogues, it is conceivable that the overlap in domains enabled CDDECfull to recognize contradictions in the Human-Bot test data.

heading: L Distance-wise accuracy of our detector
text: This study collected only contradictory responses with dur ≤ 5 because the annotation cost prohibitively increases as the value of dur increases. To examine whether our dataset also contributes to suppressing contradictions with dur > 5, we
computed the distance-wise accuracy of some of the results reported in Section 5.
Settings. In our experiments in Section 5, we trained a contradiction detector (CDOUR) with our dataset and showed its performance on several test sets. Among those test sets, only the By-Human test set contained a large number of contradictory instances with dur > 5. Therefore, we calculated CDOUR’s distance-wise accuracy on the ByHuman test set.
Results. Table 12 shows the distance-wise decomposition of the performance of CDOUR on ByHuman, where we computed the percentage of the test instances correctly classified by CDOUR for each dur . The results demonstrate that the detector’s performance did not degrade for instances with a longer distance (dur > 5).

heading: Future Work
text: Future challenges include applying the collected dataset to other data-driven methods and collecting data with a broader context variety than FQs. Ethical concerns
This study uses existing datasets, the MSC, TopicalChat, and DailyDialog datasets, which we consider not to bring any ethical concern. We added to these datasets and released a set of RGM-generated responses along with binary labels denoting the presence or absence of contradictions. Regarding the responses, it is conceivable that the aggressive expressions generated by the RGMs may be present in the collected contradictory and noncontradictory responses. As for the labels, we have meticulously removed any personal information belonging to the workers to share our dataset ethically. Limitations
Dialogue context types. Our compiled dataset exclusively comprises contradictions in RGM responses to dialogue contexts that conclude with FQs extracted from the MSC dataset. The experiments detailed in Section 5 demonstrated that utilizing our dataset addressed RGM contradictions effectively, even for the responses to non-FQ dialogue contexts sourced from different corpora. However, broadening the collection of contradictory responses to encompass a diverse array of dia-
logue contexts could facilitate a more comprehensive analysis of RGM contradictions and potentially lead to enhanced contradiction suppression through a data-driven approach. Target RGMs. It remains unclear whether employing our gathered dataset can effectively mitigate contradictory responses by any RGM. Our data compilation involved the responses from various recent and representative high-performance RGMs. More importantly, Section 5 outlined that the contradiction detector, trained using our dataset, effectively identified contradictory responses from unknown RGMs. Nonetheless, we must acknowledge the possibility that it might struggle to suppress contradictions in responses from newer RGMs. Despite this uncertainty, we believe collecting contradictory responses from recent high-performance RGMs is crucial for developing dialogue systems that can generate consistent responses. Indirect contradictions. This study focuses exclusively on pairs of utterances where the contained information directly contradicts each other. However, even when the information within each utterance is consistent, some utterance pairs might still be considered contradictory. For instance, when the information stated in the previous RGM utterance is repeated in the subsequent RGM response, the unnaturalness of the situation might be perceived as contradictory. Since such contradictions do not involve directly contradictory information, they cannot be identified as contradictions using this study’s dataset and approach. Modeing contradictoriness. How the definition of contradiction can be refined beyond binary labeling is an open question. In this study, we stayed with the standard binary view of contradictions since the binary definition of contradiction has long stood in the literature of natural language inference (Dagan et al. , 2013). Nevertheless, we also observed ambiguous or vague cases in our dataset, as discussed in Section 4. 1. 2, which could motivate introducing more fine-grained multi-class classification or scaling