Contrastive Learning for Inference in Dialogue
Authors: Etsuko Ishii
Yan Xu
Bryan Wilie
Ziwei Ji
Holy Lovenia
Willy Chung
Pascale Fung
Venue: EMNLP 2023 Main (2023)
Keywords: inference in dialogue
commonsense reasoning in dialogue
contrastive learning
semantic gap
dialogue comprehension
information gap
inductive reasoning
TL;DR: Assess the difficulty of inferences in dialogue based on their information gap and propose a contrastive learning approach to bridge the gap.

Abstract: Inference, especially those derived from inductive processes, is a crucial component in our conversation to complement the information implicitly or explicitly conveyed by a speaker. 
While recent large language models show remarkable advances in inference tasks, their performance in inductive reasoning, where not all information is present in the context, is far behind deductive reasoning. 
In this paper, we analyze the behavior of the models based on the task difficulty defined by the semantic information gap -- which distinguishes inductive and deductive reasoning. 
Our analysis reveals that the information gap between dialogue contexts and desired inferences renders the inductive inference process more challenging. 
To mitigate this information gap, we investigate a contrastive learning approach by feeding negative samples.
Our experiments suggest negative samples help models understand what is wrong and improve their inference generations.

Contrastive Learning for Inference in Dialogue
Etsuko Ishii, Yan Xu, Bryan Wilie, Ziwei Ji, Holy Lovenia, Willy Chung, Pascale Fung The Hong Kong University of Science and Technology (eishii, yxucb, bwilie, zjiad)@connect. ust.hk, hk, pascale@ust. hk
Abstract
User A: I'm hungry, let's order up something
User B: Ok, maybe we can order a soup and salad from the restaurant down the street.
Inference, especially those derived from induc- tive processes, is crucial component in our conversation complement the information implicitly explicitly conveyed by speaker. While recent large language models show re- markable advances in inference tasks, their per- formance in inductive reasoning, where not all information present in the context, is far be- hind deductive reasoning. In this paper, alyze the behavior of the models based on the task difficulty defined by semantic informa- tion gap which distinguishes inductive and de- ductive reasoning (Johnson-Laiir, 1988, 1993). Our analysis reveals that the disparity in infor- mation hetween dialogue contexts and desired inferences poses a significant challenge to the inductive inference process. To mitigate this information gap, we investigate contrastive learning approach by feeding negative samples. Our experiments suggest negative samples help models understand what is wrong and improve their inference generations.
User A: was thinking of getting a hamhurger, fries, and chocolate sundae.
User B: You stuff clogs your arteries and is very high in chol- esterol.
much junk food. That of
Dial.
User A: Well, never seem to gain weight, so don't mind.
User B: it's not only about getting fat it's ab- out being heulthy. could really have some heulth problems later on. [Torget]
User A: How about pizza or maybe some fried chic-
ken? Better yet, let's order some bot dogs!
User B:
You are lost cause.
Ques.
What or could he the prerequisite target?
The speaker fitness freak keeps track
Gold
daily diet.
The speaker cats too much junk food as it clogs up arteries very high cholesterol,
TS-hase
Ours
The speaker is health-consscous person.
One example in "Conceivable" difficulty level. comparing the generated inferences from our method, T5- base, and the gold inference. Dial. and Ques. are short for Dialogue and Question. The snippets of in- ferences highlighted in pink are not explicitly stated in the dialogue and require the model to conduct infer- ence inductively. We refer this phenomenon the "information gap" to accomplish this task.
Table 1:
1 Introduction
In conversations, inference is essential to uncover what the speaker intended deliver, which of- ten goes beyond the information explicitly pressed (Rieger, 1974; Thorndyke, 1976). Infer- ences can be made by an explicit or implicit log- ical reasoning based on utterances and common ground among speakers (Clark, 1975). By reading between the lines, these inferences enable appropri- ate responses in dialogues. This inference process has been intensely discussed in the early age of re- search dialogues (e.g., Thorndyke, 1976). How- ever, research in dialogue systems nowadays often overlook such an aspect and instead rely solely on the capabilities of large language models (LLMs) to understand and comprehend dialogues.
Current LLMs. such as ChatGPT (OpenAI, 2022), lack the so-called "inductive reasoning" abil- ity, while tending to accomplish the reasoning tasks deductively (Bang et al., 2023). It might be due to the fundamental difference berween inductive and deductive processes. According to (Johnson- Laird, 1988, 1993), inductive reasoning involves increase in semantic information from input to output while remains the same in deductive reasoning, In the context of dialogue inference pro- cesses, especially when reading implicit messages, there are information gaps that need to be filled. For instance, somebody's ' invitation for a "quick
'The code and anmotated data is available at https://gituub..cm//HCtUUUUS com/HLLCKKUST//onrratiiii inference_dialogue

---

lunch as always" might be enough to specify the location and time without further interaction.
2 Related Work
2.1 Inference in Conversation
In this paper, we inspect the semantic infor- mation gap between dialogue contexts and in- tended inferences using a recently introduced dataset designed for generating inferences in di- alogue (Ghosal et al., 2022). We hypothesize that the difficulty of the task can be associated with the amount of information gap required to bridge. We manually annotate the randomly sampled subset of the dataset regarding their information gap, and assess the performance of the models. The analy- sis shows a decline model performance as the information gap increases.
As of the most fundamental forms of the use of natural language (Jurafsky and Marin, 2023), advance in inference in conversation has been in- separable from the flourish of the field of natu- ral language processing (NLP) (e.g., Mann, 1979; Phillips, 1975). Initially, the research focus of infer- ence in conversation was to uncover the underlying rules of human conversations (e.g., Grosz, 1978;
Carbonell Jr, 1978: Morgan, 1978). While it re- mains
research question, recent works tend to be formed in question answering (QA) style that we can test models in a handier way. Thanks to the powerful deep learning models, we can per- form inference tasks sufficiently well yet leave un- derlying rules unclear. Recently, a number of QA datasets in conversational formats have been in- troduced (Choi et al., 2018; Reddy et al., 2019; Ma et al., 2018), and their main focus tends to be comprehension of non-conversational texts. To evaluate the comprehension of dialogues, various tasks have been proposed in different task formu- lations such as span extraction (Li et al., 2020; Yang and Choi, 2019; Wu et al., 2022), multiple choice (Sun et al., 2019), next utterance predic- tion (Cui et al., 2020), or natural language infer- ence (NLI) (Welleck et al., 2019). Some tasks fo- cus specific aspect of conversational inference, such as speaker guessing (Sang et al., 2022), and temporal reasoning (Qin et al., 2021). In natural language generation format, Ghosal et al. (2021, 2022) presents datasets for generating inferences based on dialogue, while Ghosal et al. (2021) only contains overt inferences and Ghosal et al. (2022) contains implicit guesses as well.
Furthermore, we propose to apply contrastive learning approach to improve inference perfor- mance. One limitation of the current sequence-to- sequence training, especially for reasoning tasks, is that models are never exposed to negative sam- ples (Lee al., 2021). In deductive reasoning. all the information required to generate an output provided in the input, and there is no information gap. However, inductive reasoning requires includ- ing something that may not be explicitly stated in the input, and that is not simply learnable by only exposing gold samples. Thus, we need to teach the model with more guidance on the reason- ing path. In our preliminary experiment using the same dataset and multiple-choice framework with Roberta-large (Liu et al., 2019), we observed sig- nificant improvement from an Fl score 83.91 to 96.6 simply by feeding negative samples together with the other candidate, which indicates that feed- ing negative samples will help the model learn how to fill the information gap. Building this initial experiment, our experimental results in the gener- ative settings show that contrastive learning helps improve both overall and breakdown performance in each task difficulty level, especially for fully deductive and inductive cases. Additionally, we explore various sampling methods for generating negative samples for contrastive learning.
2.2 Task Difficulty and Information Gap
Controlling the difficulty of tasks requires delicate tuning as it crucial for further advance in NLP; too challenging or too easy tasks cannot facilitate the growth of the technology. A task becomes more challenging if we impose additional condi- tions, such as limiting the amount of data and com- putational power or adding modality or other lan- guages. Recently, some work has investigated the specific task with controlled annotated data. For example, Williams et al. (2022) annotates on infer- ence types such as numerical or reference to see which type is the most challenging in NLI. Cui et al. (2023) limit the data to assess the models'
Our contributions are three-fold: (1) we provide data annotation based on the information gap and the assessment; (2) we suggest that the informa- tion gap accounts for the difficulty of the inference generation in dialogue; and (3) our experimental results show that the contrastive learning approach helps to fill the information gap.

---

capability to properly understand what the word "respectively" refers to in NLI.
3 Information Gap in Inference
While existing work focuses on improving the model performance on inference tasks with var- ious methods, there is still a lack of in-depth in- vestigation on the task itself and how the model behavior is changed with the improved results. To fill this gap, we first propose to connect task diffi- culty with the "information gap" between contexts and target inferences and classify the inference task difficulty into three levels. Then, we focus on the generative inference in dialogues with the CICERO dataset (Ghosal et al.. 2022). We collect additional annotations to assess the task difficulty of a subset of samples for further analysis.
Discussing the task difficulty independent of the models' performance is non-trivial. Current as- sessment of the task difficulty tends to be insep- arable from the performance comparison of the models (e.g., Bang et al., 2023). In this way, can observe the models' strengths and weaknesses across different tasks, but there is still a lack of absolute difficulty rankings of the tasks. One possi- ble way to discuss the difficulty in a model- or task-agnostic way might be based on the infor- mation gap, which is the core challenge in induc- tive reasoning (Johnson-Laird, 1988, 1993). It has been discussed as "given and new information" in (Clark and Haviland, 1974; Clark, 1975) the foundation in conversations, but this concept can be extended to any tasks (McKeown, 1979). In this line of work, Rudinger et al. (2020) proposes an NLJ task in which an inference can be shifted when there is new information offered. These days, not many works explicitly mention "information gap" (Hayashi, 2022). However, still have the concept underlain. For example, QA datasets com- monly contain some portion of unanswerable ques- tions (e-g.. Rajpurkar et al., 2018; Bajaj et al., 2016) with the context provided.
3.1 Preliminaries the CICERO Dataset
We denote dialogue dataset as {D*) J.W Jn=1 and di- alogue as D) = {U.Y= where U₁ is an utterance at turn i. Given an input X = =(1, Q. U, where Q is a question and U, E DD is a target utterance, we aim to learn model fo to generate plausible inference À = fX()
CICERO dataset comes with five types of ques-
tions:
1. Cause: What is could be the cause of the target utterance?
2. Prerequisite: What or could be the prerequi-
site of target?
2.3 Contrastive learning in NLG
3. Subsequent Event (SE): What subsequent event
happens could happen following the target? 4. Motivation: What is or could be the motivation
Contrastive learning teaches a model to embed sim- ilar data sample pairs are closer and disparate sam- plc pairs stay apart (Chopra et al., 2005; Smith and Eisner, 2005). Not only in obtaining better representations of words (Mikolov et al., 2013) or sentences (Fang et al., 2020; Gao et al., 2021; Liu et al., 2021a), contrastive learning is reported to improve wide range of NLP tasks (e.g., Li et al., 2022b; Klein and Nabi, 2020) including text gener- ation tasks (e.g.. Cai et al., 2020; Li et al., 2021: Liu et al., 2021b; Paranjape et al., 2021; Li et al., 2022a; Shu et al., 2021). The main motivation for applying contrastive learning for sequence-to-sequeme text generation tasks is that it allows the model to be exposed to negative samples during training (T.ee et al., 2021). Indeed, negative samples are gener- ated by some rule-based perturbations (Shu al., 2021) or machine-generatee texts (Cao and Wang, 2021) such as entity-swap (Tang et al., 2022) are re- ported to be effective for faithful, less hallucinatory text generation.
of target?
5. Reaction: What is the possible emotional reac-
tion of the listener in response to target? For subsequent event category, it also offers a more challenging setting called Subsequent Event Clipped (SE 'SE_Clippedl where the dialogue is clipped until the target utterance: D, = (U,)
3.2 Task Difficulty of the CICERO dataset
The CICERO dataset provides commonsense in- ferences made by human annotators. According to the annotation instructions, generated answers must be grammatically correct and consistent with the dialogue, yet they can be overt speculative depending on contextual scenarios (Ghosal et al., 2022). While treated equally, some question types seem significantly more challenging than others ac- cording to the results breakdown reported in Ghosal et al. (2022). For example, Motivation scores the

---

highest even though only accounts for 14% of the training set.
User A
Jim, could you do favor?
User B
Sure, what can I for you?
User A
My cur has problem starting. Could you pleuse
Although the surface format of the task is uni- fied and thus cannot distinguish at glance, we can sense that they challenge different things. For ex- ample. SE can be executed simply by summarizing the utterances after the turn while SF_Clipped required to predict future sequences from the dia- logue. The difficulty differs even among questions in the same question type. Some inferences can be derived simply by paraphrasing the utterances, while others require logical guessing to read be- tween the lines. These differences boil down to the information gap between the answer A and the dialogue D,. Here, we take an initial step to inves- tigate the task difficulties systematically and define three levels of difficulty based on the amount of information in the answer covered by the dialogue: Sufficient, Likely, and Conceivable.
take a look at it for me?
User B
Sure thing.
Question
What subsequent event happens following the
target utterance?
Answer
User B tries to non on the car engine.
Level 3: Conceivable The answer contains some pieces of information that are stated in the dia- logue, and there is no clear guidance for a "hop". The answer plausible but hardly verifiable. Ques- tions at this level are not easy even with certain knowledge sources provided and can be compared to check hallucinations in open- open-domain text genera- tions (Ji et al., 2023). For example, in the dialogue below, Bob may be a brother of User B, and his occupation could be radio journalist, which is plausible reason to call Bob to ask about the fire the factory. However, we cannot verify the answer as the dialogue lacks the evidence to guess the re- lationship between the speakers and Bob, nor his occupation.
Level I: Sufficient All the information in the an- swer is available in the given dialogue. Since there is no information gap between inputs and outputs, questions this level are the easiest to answer. For example, from the given dialogue context below, is overt that User A will be available on Saturday morning for delivery.
User A
There's 's heen a fire at the factory.
User B
Are you sure? There is nothing in the news-
paper about it.
User
just saw the o'clock news.
User
will phone Bob.
User A
Yeah, he always knows what's s going on.
User
Can you deliver it, please?
Question
What the prerequisite of the target utterance?
User B
Yes, costs two pounds fifry.
Answer
User B's brother Bob is a radio journalist.
User A
All right, can you deliver here on Saturday?
User B
Sure. Does morning work for you?
User
Sounds good.
3.3
Human Assessment of the Difficulty
Question
What is the prerequisite of the target utterance?
Answer
User A will be availuble on Saturday morning.
To the best of our knowledge, there is no absolute automatic metric to compare two pieces of text in terms of the amount of semantic information they contain. Here, we assess the difficulty of the task defined in Section 3.2 by human annotation. We randomly select 75 samples per question type (in total 450 samples) from the CICERO test set. In our annotation scheme, we assign two well-trained annotators per sample to give a difficulty-level la- bel and the other one expert to double-check and finalize the label. In a few cases where the three an- notators disagreed on the label, an additional expert is assigned for confirmation.
Level 2: Likely Some pieces of information in the answer are not available directly stated, but it is possible to guess by combining the clues in the dialogue. Questions at this level can be com- pared multi-hop question answering tasks (Yang et al., 2018; Welbl al., 2018; Inoue et al., 2020), There are arguably different degree of hops to de- rive an answer depending on the context (Kumar et al., 2019; Cheng et al., 2021), however, here we classify all the questions that requires some sort of "hop" over c.g., knowledge graph (Speer al., 2017; Sap al., 2019; Hwang et al., 2021) regard- less of the degree. For example in the dialogue below, we can guess that User will check the car as per User A's request. To check the car, User B will likely try to turn on the engine.
In Table 2, we summarize the annotated results and the T5-base (Raffel al., 2020) performance of the same subset that is fine-tuned on the C1- CERO training set. The CICERO dataset has a hal- anced mixture of the three levels (sufficient: 34.2%, likely: 33.6%, conceivable: 32.2%), and the per-

---

Difficulty
BLBU-2
METEOR
ROUGE I.
CIDEr
to the dissimilarity of the sentences. We denote the contrastive loss for in-batch negative samples 46.07
Sufficient (34,2%)
18.78
16.80
29.37
Likely (33,6%)
16,38
15.89
26.76
32.27
as AbCCLE. Besides, we aim to feed more infor- mative negative samples per gold inference, which we denote as Cg Then, the training objective can be formed as L = CNLI + A, + CC Since the CICERO dataset also serves as an MCQ task, each inference has four high-quality plausible- looking yet not appropriate candidates. These coun- terfactual candidates are machine-ger -eeeenatee and then filtered by human annotators. In our experi- ments, we explore the following ways for generat- ing negative samples in fully-automatic:
Concervablee (32.2%)
11.92
12.72
21.87
22.23
Table 2:
The performance of the fine-tuned T5-hase gets worse along with the decrease the amount of information available in the dialogue.
formance of T5-base uniformly degraded with the decrease of the amount of available information. As reported in Table 3, different question types have different proportions of difficulty levels as anticipated. Although the proportion of likely and conceivable questions can explain the difference in T5-base performance to certain extent, it does not have simple correlation. It may be due to the difference in which kind of information is re- quired bridge the gap between the dialogue and the answer. For example, speakers' emotional reac- tions might be easily guessed by the sentiment of the utterances, while identifying the cause of the utterance may involve more complicated under- standing of background knowledge.
Non-Optimal Generation Since the simple fine- tuning with CNLL does not yield the optimal fo as reported in Table 3, we directly use generated inferences by the fine-tuned model. We use top-k sampling with k = 10 for diversed generation.
Replacement of Tokens Inspired by (Park al., 2021), we manipulate tokens of the gold inference using the prediction of masked language model. More specifically, we compute the probability of each token in the gold inference A when whole context X and A are given and when only A is given. In this way, we can estimate which tokens in A are more affected by the context X. We di- rectly compare the log-likelihood score of each token and select tokens that differ more than threshold. The selected tokens will be replaced by the randomly selected tokens top-k prediction by a masked language model. We apply the pre- trained Roberta-large model (Replacezs) and the Roberta-large trained on the CICERO dataset for MCQ (ReplacemcQ), set k = 10, and the threash- old 0.75.
4 Methodology
We primarily train our model fe by minimizing the negative log-likelihood:
CLLL
Σ
Σ
log logplazlazj. X"),
IEnENIsjsk
where: a generated inference is denoted as Á" {a) j=00- The contrastive learning objective is de- fined by:
LeL - log explsim(hx Je )/7))
WNEA explsim Ax J )/7)
where sim is cosine similarity function, A is set of negative samples of inferences, hx, h jn- A are the hidden representations of X, A",",AA A', and T is a temperature, respectively. Following
Experiments
5.1 Baselines
(Cao and Wang. 2021; Lee et al., 2021), the final training objective L = LNLI + XCCI. where À is coefficient.
We evaluate our proposed method across multiple Transformer-based models: T5- small/base/large (Raffel et al., 2020), and GPT2-base (Radford et al., 2019). To have fair comparison, these baselines are finetuned on the CICERO training set only with LXLL. In addition, compare our results with the performance of GPT-J (Wang and Komatsuzaki. 2021) and LLaMA-7B (Touvron al., 2023) in a 3-shot setting. We report an average of three trials randomly sample manually crafted prompts and
4.1 Selection of Negative Samples
Automatically generating set of negative sam- ples A for contrastive learning is non-trivial task. The easiest method to sample negative samples is randomly sampling other inferences in the dataset (usually within the same batch), while the supervi- sion of these negative samples might be weak due

---

Difficulty
Automatic Metrics
Sufficient
Likely
Conceivable
BLEU-2
METEOR
ROUGE_I
CIDEr
Cause
46.77 7%
33.3%
20.0%
11.93
13.78
21.88
34.65
SE
41.3%
20.0%
38.7%
14.83
15.19
26.39
29.70
SE_ cl_ppip
4.0%
38.7%
57.3%
13.76
15.58
26.00
35,28
Prerequisite
32.0%
28.0%
40.0%
6.77
10.19
15.82
12.31
Motivation
58.7%
24.0%
17.3%
21.33
17.02
32.40
42.32
Reaction
22.7%
57.3%
20.0%
23.30
18.72
33.96
35.75
Total
34.2%
33.6%
32.2%
15.62
15.08
26.08
32.51
Table 3: The difficulty the inferences varies on the type questions, and so does the performance of the finetuned
T5-base. The corresponding performance is calculated the same subset of the CICERO test set.
BLEU1
BLEU2
BLEU3
BLEU4
METEOR
ROUGE-L
CIDEr
GPT-J-3shot
24.70
13.04
6.11
3.04
13.74
25.03
19.87
+ tf-idf
22.83
11.64
5.52
2.83
12.19
22.45
17.99
LLaMA-3shot
28.34
15.18
7.25
3.73
15.26
27.43
26.47
+ tf-idf
25.36
13.33
6.56
3.49
13.72
24.91
24.06
T5-small
29.20
15.66
8.19
4.67
15.88
27.34
33.58
CL
29.46
15.83
8.29
4.71
15.88
27.63
33.44
T5-hase
29.77
16.38
8.87
5.26
16.40
28.32
38.91
CL.
30.67
17.09
9.45
5.65
16.62
28.50
40.53
TS-large
29.57
16.79
9,45
5.81
16.60
29.06
43.38
+ CL.
30.07
17.02
16.67
28.90
43.80
GPT22-base
25.09
13.65
6.92
3.89
14.45
26.48
25.73
+ CL
27.55
14.91
7.56
4.22
15.13
27.94
26.59
Table 4: Automatic results CICERO test set. CL short for contrastive learning. We bold the better results between our method and the corresponding baseline model. We also highlight the best results across different models with underline.
Sufficient
Likely
Conceivable
Plausibility
Win
Tie
Lose
Win
Tie
Lose
Tie
Ours vs TS-base
38.7%
35.8%
25.5%
0.73
45.7%
34.6%
19.7%
34.7%
40.4%
24.9%
35.4%
32.4%
32.2%
Ours vs Gold
24.7%
52.4%
22.9%
0.21
22.3%
53.0%
24.7%
23.4%
53.0%
23.6%
28.5%
51.3%
20.2%
Table 5: Human evaluation results on Plausibility, together with breakdown performance on each difficulty level. "Our model achieves a significant advantage over T5-base Gold with pair-wise individual (-test (p < 0.05).
a strategic prompt using tf-idf to retrieve 3-most similar in-context examples.
whether the answers are rational or not. We eval- uate the same data samples those for task diffi- culty analysis. More specifically, comparing with both generated inferences from the T5-base model and the gold inferences. A/B testing utilized to compare our proposed method and the corre- sponding baseline on the CICERO test set. Each comparison requires three judgments. The human evaluation is conducted based on a crowd-soureing platform offered by Appen 2. More details about human evaluation, such as annotator instructions and how the results are calculated, are included in Appendix A.2.
5.2 Evaluation Metrics
Automatic Metrics In line with the CICERO paper, assess the answers generated using gram overlap- evaluation metrics: BLEU (Pa- pineni et al., 2002), METEOR (Banerjee and Lavie, 2005), ROUGE-L (Lin, 2004), and CIDEr (Vedan- tam al., 2015). Notably, CIDEr is calculated based on stem forms.
Human Evaluation For comprehensive eval- uation, we also conduct human evaluation on Plausibility aspect which focuses on evaluating
hhttps://client.appee

---

Model
BLEU-2
METEOR
ROUGE
CIDEr
are computed using Fleiss' Kappa (x) assess the reliability of the evaluation. As shown in
Ours
17.09
16.62
28,50
40.53
Table 5, contrastive learning significantly improves Table 6: Ablation study with the base model as TS-base.
LCL.
16.97
16.53
28.34
40.71
LCLs
16.95
16.53
28.49
40.18
the plausibility of the generated inferences over T5- base with a substantial agreement. The generated inferences from T5 15-base with contrastive learning show comparable plausibility with gold ones in the CICERO test set with fair inter-annotator agrec- ment. The human evaluation further proves the effectiveness of our proposed method in improv- ing inference ability. We further investigate the improvement breakdown each difficulty levels to further analyze the effect of contrastive learning in Section 6.5.
CCL
16.38
16,40
28.32
38.91
5.3 Training Details
The models are trained using batch size of 64 after gradient accumulation, with learning rate set at 1e-4 for T5 models and le-5 5 for GPT-2 models. We limit the training to a maximum of 10 epochs, employing linear learning rate scheduler. The checkpoint exhibiting the lowest perplexity on the validation set is chosen as the optimal model for each trial. In the case of contrastive learning, the temperature т for LCI, and LCL, learning is set to 0.1 and 2.5, respectively, each contributing equally to the total loss with coefficient Å X 0.5. All the experiments are executed single RTX 3090 Ti GPU.
Discussion
6.1 Case Study
Table illlustrates one example in "Conceivable". comparing the generated inferences from our method, T5-base, and the gold inference. While T5-base tends to copy from the dialogue (high- lighted in blue ), contrastive learning promotes the model to infer more rational information which is not stated in the context (highlighted in pink ). We include more examples in Appendix B.2.
5.4 Results
We report the automatic results of both our method and the baselines in Table 4. Automatic metrics based on n-gram overlap are mostly improved thanks to contrastive learning. Moreover, our pro- posed method is model architecture-agnosie given that it shows consistent improvement in differ- ent encoder-decoder T5 models and encoder-only GPT2. For GPT-J and LLaMA, could not any improvement introduced by ff-idf. We suspect that even though lexically similar, these examples may mislead the model to make wrong predictions
6.2 Ablation Study
We perform an ablation study on our proposed method using T5-base as the foundational model. The effectiveness of our model is compared against those trained without the application of either LCL,, LCL,, both LCL X,LCL, + XLCCL.- In Table 6, our proposed method, employing both contrastive losses, amplifies the performance. A model devoid of LCL, surpasses our own in terms of CIDEr, yet our method achieves superior results across all other metrics. Furthermore, the impact of the different contrastive losses varies across the range of automated methods. While LeL, exhibits minimal impact on ROUGE-L, it proves more effec- tive for CIDEr. The most significant contribution the ROUGE-L improvement is derived from LCL,-
Overlap-basee metrics reflect the general quality of the generated inferences with respect to the gold answers. However, it does not reflect the inference ability of the generations, not to mention the inductive inference ability. In this work, we also explore the feasibility of NLJ metrics for inference ability evaluation. More discussion is included in Section 6.6.
6.3 Comparison of Sampling Methods
In addition to the negative samples provided by the CICERO dataset, three different fully-automated methods of generating negative samples are ex- plored as stated in Section 4.1. We train the model
Human Evaluation For more comprehensive evaluation of inference ability, we conduct hu- man evaluation of the plausibility of the generated inferences and report in Table We leverage pair- wise individual f-tests to validate the significance the improvements. Inter-annotator agreements
htt //www. statsmodels. www.eelsssso///oeeeeeeoeeeeeeees httpaeesssrgggstabeeeeeerrae statsmodels. stats. inter_ www.r.tter..fa... www.ssskappaa...oooooea html

---

BLEU-2
METROR
ROUGF_L I. CTDEr
Difficalty
BLEU-2
METEOR
ROUGBL
CIDEr
TS-base
16.38
16.40
28.32
38,91
Sufficient
25.24
(+.773)
36.58 (+7.21)
82.07 (+36.00)
Likely
19.32 (+2.94)
17.98 (42.09)
30 (44.04)
(-.1871
Contradiction
17.09
16.62
28,50
40.53
Conceiv.
17.09 (+5.17)
15.39 (+2.67)
28.62 (+6.75)
38.99 (+16.76)
Non-optimal
16.40
16.40
28.19
39.17
Replacezs 16.39
16.36
28.44
40.09
Table 9: The performance is improved thanks to the con- trastive learning across all the difficulty levels. Conceiv is short for Conceivable. The performance is calculated on the same subset the CICERO test set in Table
ReplaсeииQ
16.48
16.42
28.45
39.42
Table 7: Comparison of different sampling methods for generating negative samples. All the models are implemented based on T5-base.
the effective amount of negative samples for con- trastive learning is under discussion (e.g., Awasthi et al., 2022; Nozawa and Sato, 2021), we conduct control experiment by feeding randomly sampled counterfactual candidates (m 1, observe the effect of the number of negatives. We report the results in Table 8; note that we report the av- crage of three trials with different random seeds for m = 1,2,3. The performance generally im- proves along with the increase in the number of negative samples, implying that the high-quality negative samples contribute to teach the model to inference. Encouraged by our results, it would be interesting to quantify how much guidance is nec- essary for each level. For example, the "Sufficient" level may need fewer negative samples than the "Conceivable" level to achieve similar performance. It would be also beneficial to investigate the pos- sibility of dynamically controlling the number of negative samples to feed.
BLEU-2
METEOR
ROUGE_L I.
CIDEr
TS-base
16.38
16.40
28.32
38.91
16.67
16.43
28.31
40.43
2
16.81
16.53
28.54
40.69
+ m 3
16.82
16.52
28.45
40.94
4
17.09
16.62
28.50
40.53
Table 8: The effect of the amount of negative samples. We report the average of three trials for m = 1,2.3.
leveraging the negative samples obtained from dif- ferent methods, respectively and present the per- formance of the models Table for comparison. While different generation methods yield differ- ent improvements in the automatic metrics, in gen- eral, feeding negative samples does not hurt taining the models to perform dialogue inference. The "contradiction" negative samples from the dataset provide the largest improvement the model per- formance, which suggests that higher quality of negative samples can guide the models better with a smaller amount. Another method that shows ef- fectiveness is to replace the words that affect the predictions largely for the RoBERTa-Large model trained to differentiate positive samples from nega- tive samples. ReplacemcQ. while replacement mea- sured by the RoBERTa model in zero-shot way (Replacezs) less helpful. This indicates that fine- tuned RoBERTa assigns the probability of the to- kens more informatively for inference. Our explo- ration of using a non-optimal T5-base model to generate negative samples is expected to improve the model performance by iterative self- aof-contrasting. However, self- seff-improvement may not be effective without further human filtering since we might in- clude rational answers as negative samples, which introduce noise during training.
6.5 Analysis of Improvements based on Task Difficulty
We further investigate how contrastive learning im- proves the model performance in different task dif- ficulties. Table reports the automatic score break- down based on the difficulty annotated. Compared to the performance of the T5-base model reported in Table 2, our method yields improvement for all the levels, especially on "Sufficient" and "Conceiv- able". Similarly, we list the breakdown of human evaluation results to each task difficulty level in Table T5-base with contrastive learning out- performs TS-base on plausibility in all difficulty levels, especially for "Sufficient" and "Conceiv- able". which is consistent with the trend of auto- matic metrics. In the "Sufficient" level, the advan- tage of our model is significant over the basb This proves that contrastive learning can effectively improve the model's inference ability. Moreover, our method even significantly wins over gold in the "Conceivable" level in human evaluation with < 0.05. Conceivable level gold answers tend to
6.4 Effect of the Amount of Negative Samples
In our main experiments, we feed all four of the the counterfactual candidates provided by the CICERO dataset as negative samples to compute LCL. As

---

UNLI
UNL
ASoold
AScon
hardly interpretable. showing the consistent trend of contrastive learning degrading except for the GPT2-base. Even gold answers are labeled as "neutral" and undeterminable, and it is difficult to associate numbers with the quality of generated inferences. Although NLI metrics are an effective method to quantify factuality (Zha et al., 2023), this result suggests that NLI metrics are not suitable for inference in dialogue. Future work is needed investigate possible evaluation metrics of the infor- mation gap since it can also benefit wide range of NLP tasks.
GOLD
1,0000
0.5220
0.9995
0.3670
15-small
0.2283
0.6584
0.0577
0.6224
+ CL
0.2271
0.6422
0.0565
0.5940
T5-base
0.2607
0.6894
0.0765
0.6355
+ CL
0.2572
0.6586
0.0760)
0.6009
TS-large
0.2947
0.7015
0.0973
0.6282
+ CL
0.2940
0.6993
0.0931
0.6106
GPTe-base
0.2783
0.6460
0.0783
0.5723
+ CL
0.3066
0.6610
0.0964
0.5561
Table 10: NLI-based metric results on the CICERO test set. AS is short for AlignScore.
7 Conclusion
include something that not stated/verifiable in the dialogue contexts provided, while ours tends to be more supported by the dialogue context (see Table 1). We believe this resulted in ours being more favored by annotators.
In this paper, we conduct an analysis of inferences in dialogue, focusing on the availability of seman- tic information between inputs and outputs. As expected, the models perform worse on the sam- ples with larger information gaps. We investigate contrastive learning approach to teach what is wrong in inference. Our experimental results sug- gest the effectiveness of our approach, showing the promising direction to bridge the information gap. especially for smaller models with <IB parameters.
6.6 Challenges in Evaluation of Inductive Reasoning
As we have discussed in the previous sections, it is extremely challenging to evaluate inductive pro- cesses because, by nature, outputs contain new in- formation that is not stated in the inputs (Johnson- Laird, 1988, 1993). While the field has been aware of the fundamental difference between inductive and deductive for more than 60 years (Watanabe, 1960), there is no way to directly compare two pieces of text in terms of the amount of "semantic information" until now. Recently, with the aris- ing demand in faithful and factual text generation, several metrics have been applied mainly by com- puting overlap in named entities extracted key- words (Mao et al., 2021). Although the overlap- based metries could be decent starting point for not appro-
Limitations
The main drawback of the proposed method is that it requires more computational resources and longer training time we increase the amount of training data to yield improvement with contrastive learning over the baselines. Although our method model, dataset, and language agnostic, our explo- ration limited to the popular transformer-base architectures and the single dataset in English.
The other significant aspect we have not covered in the paper (and in most of the literature to the best of our knowledge) is the stopping rule of the infer- ence process in dialogue. As suggested in Clark (1975), there is a clear boundary between what por- tion of the untold information should be guessed and what can be left unknown in a speaker's inten- tion. However, even in dataset construction phases, this aspect has been neglected (e.g., Bhagavatula et al., 2020; Ghosal et al., 2022). The stopping rule is essential since it can be one factor separating "Likely" questions and "Conceivable" questions. An important question for future studies is how to deal with the stopping rule, it can be also associated with boundary of hallucination and acceptable freedom in open-domain dialogue sys- tems.
many tasks such as summarization, priate for inference in dialogue as non-overlap is something desired rather than being avoided. Another common choice to measure the plau- sibility today would be adopting NL.-based met- ries (Honovich al., 2022). In Table 10, we report model-based NLI metrics of UNLI (Chen et al., 2020) and AlignScore (Zha et al., 2023). We mea- sure entailment between generated inferences and or between
the gold references (UNLIgola/AS generated inferences and the corresponding dia- logue context (UNLlco/AAScon)
scale of [0, 1]. The training specifics of the NLI models, well as their performance, can be found in Appendix A.
Despite being promising, the NLI scores are

---

Ethics Statement
Ji. Tiezheng Yu, Willy Chung, et al. 2023. A multi- task, multilingual, multimodal evaluation of chatgpt reasoning. hallucination, and interactivity. arXiv preprint arXiv:2302.044 230..0402...
In this work, we collect additional human annota- tions on the task difficulty in the CICERO dataset. The CICERO dataset is publicly available, and we will also release our task difficulty annotations upon acceptance. We consider the target inference provided by the dataset as the gold inference and analyze the task difficulty fully based on the re- lationship between the target inferences and the dialogues. We conduct human evaluation relying on a public crowd-sourcing platform, Appen. Each judgment takes four seconds on average, and assign 15 cents as the payment to each judgment for annotators. No personal information except the country-level location is used to ensure the En- glish proficiency of the annotators to guarantee the annotation quality, while all annotations were anonymized before the analysis.
Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Han- nah Rashkin, Doug Downey, Wen tau Yih, and Yejin Choi. 2020. Abductive commonsense reasoning. In International Conference on Learning Reprexenta- tions.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empiri- cal Methods Natural Language Processing, pages 632-642, Lisbon, Portugal. Association for Compu- tational Linguistics.
Hengyi Cai, Hongshen Chen, Yonghao Song, Zhuoye Ding, Yongjun Bao, Weipeng Yan, and Xiaofang Zhao. 2020. Group-wise contrastive learning for neural dialogue generation. In Findings of the As- sociation for Computational Linguistics: EMNLP 2020, pages 793-802, Online. Association for Com- putational Linguistics.
Acknowledgement
The authors thank all the anonymous reviewers for their valuable comments and constructive feedback. This work has been partially supported by China NSFC Project (No. NSFC21EG14), the Hong Kong Jockey Club (RG (DG19/HHJCCT21EG00 and the Hong Kong PhD Fellowship Scheme, Research Grant Council, Hong Kong (PF18-25016).
Shuyang Cao and Lu Wang. 2021. CLIFF: Contrastive learning for improving faithfulness and factuality in abstractive summarization. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6633-6649, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Jaime G. Carbonell Jr. 1978. Intentlonality and human conversations. American Journal of Computational Linguistics, pages 48-55. Microfiche 79.
References
Pranjal Awasthi, Nishanth Dikkala, and Pritish Kamath. 2022. Do more negative samples necessarily hurt contrastive learning? In Proceedings of the 39th International Conference on Machine Learning, vol- ume 162 of Proceedings of Machine Learning Re- search, pages 1101-1116 16. PMLR.
Tongfei Chen, Zhengping Jiang, Adam Poliak, Keisuke Sakaguchi, and Benjamin Van Durme. 2020. Un- certain natural language inference. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8772-8717 8779, On- line. Association for Computational Linguistics.
Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. 2016. Ms marco: A human generated ma- chine reading comprehension dataset. arXiv preprint 1rXiv:1611.09266
Yi Cheng, Siyao Li, Bang Liu, Ruihui Zhao, Sujian Li, Chenghua Lin, and Yefeng Zheng. 2021. Guiding the growth: Difficulty controllalce question genera- tion through step-by-step rewriting. of the 59th Annual Meeting of the Association for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing ( (Volume I: Long Papers), pages 5968-5978, Online. Association for Computational Linguisties.
Proceedings
Satanjeev Banerjee and Alon Lavie. 2005. METEOR:
An automatic metric for MT evaluation with im- proved correlation with human judgments. In Pro- ceedings of the ACL Workshop on Intrinsic and Ex- trinsic Evaluation Measures for Machine Transla- tion and/or Summarization, pages 65-72, Ann Arbor, Michigan, Association for Computational Linguis- tics.
Eunsol Choi, He Hc, Mohit lyyer, Mark Yatskar, Wen- tau Yih, Yejin Choi, Percy Liang, and Luke Zettle- moyer. 2018. QuAC: Question answering in context. In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 2174-2184, Brussels, Belgium. Association for Com- putational Linguistics.
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wen- liang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei

---

S. Chopra, R. Hadsell, and Y. LeCun. 2005. Learning a similarity metric discriminatively, with application to face verification. In 2005 IEFE Computer Society Conference on Computer Vision and Pattern Recog- nition (CVPR'0S). volume I, pages 539-546 vol. 1.
3984-3993, Marseille, France. European Language Resources Association.
Or Honovich, Roce Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022. TRUE: Re-evaluating factual consistency evaluation. In Proceedings of the Second DialDoc Workshop on Document- groounddd Dialogue and Conversational Question Answering, pages 161- 175, Dublin, Ireland. Association for Computational Linguistics.
Herbert H. Clark. 1975, Bridging. In Theoretical Issues in Natural Language Processing.
Herbert H Clark and Susan E Haviland. 1974. Psycho- logical processes as linguistic explanation. Explain- ing linguistic phenomena, pages 91-124.
Leyang Cui, Yu Wu, Shujie Liu, Yue Zhang. and Ming Zhou. 2020. MuTuull: A dataset for multi-turn dia- logue reasoning. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages 1406-1416, Online. Association for Computational Linguistics.
Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi. 2021. Comet-atomil 2020: On sym- bolic and neural commonsense knowledge graphs. In AAAI.
Ruixiang Cui, Seolhwa Lee. Daniel Hershcovich, and Anders Søgaard. 2023. What does the failure to rea- son with "respectively" in zero/few-shot settings tell
Naoya Inoue, Pontus Stenetorp, and Kentaro Inui. 2020. R4C: A benchmark for evaluating RC systems to get the right answer for the right reason. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6740-6750, Online. Association for Computational Linguistics.
us about language models? In Proceedings of the 61st Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 8786-8800, Toronto, Canada. Association for Com- putational Linguisties.
Ziwei Ji. Nayeon Lec, Rita Frieske. Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12):1-38.
Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, and Pengtao Xie. 2020. Cert: Contrastive self-supervised learning for language understanding. arXiv preprint arXiv:2005.1211 12766.
Tianyu Gao, Xingeheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence em- beddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pages 1894-6910 Online and Punta Cana, Do- minican Republic. Association for Computational Linguistics.
Philip N. Johnson-Laird. 1988. The Computer and the Mind. Cambridge, Mass. : Harvard University Press.
Philip N. Johnson-Laird 1993, Human and Machine Thinking. Hillsdale, N.J. L. Erlbaum Associates.
Daniel Jurafsky and James H. Marin. 2023. Speech and Language Processing, chapter 15. Draft of Jan- uary, 2023, Website: https:/ http://wwbb..... http://we.b........ooee// stanford. edu/ =jurafsky/ssl3/15. httplp/............
Deepanway Ghosal, Pengfei Hong, Siqi Shen, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. 2021. CIDER: Commonsense inference for dialogue expla- nation and reasoning. In Proceedings of the 22nd Annual Meeting of the Special Interest Group on Dis- course and Dialogue, pages 301-313, Singapore and Online. Association for Computational Linguistics.
Tassilo Klein and Moin Nabi. 2020. Contrastive self- supervised learning for commonsense reasoning. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 7517- 7523, Online. Association for Computational Lin- guistics.
Deepanway Ghosal, Siqi Shen, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. 2022. CICERO: A dataset for contextualized commonsense inference dialogues. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers). pages 5010-5028, Dublin, Ireland. Association for Computational Linguistics.
Vishwajeet Kumar, Yuncheng Hua, Ganesh Ramakr- ishnan, Guilin Qi. Lianli Gao, and Yuan-Fang Li. 2019. Difficult--cnttrllabbe eration from knowledge graphs. The Semantic - ISWC 2019 18th International Semantic Web Conference, Auckland, New Zealand, October 26-30, 2019, Proceedings, Part I, page 382-398, Berlin, Hei- delberg. Springer- Verlag.
controllable multi- question gen-
Barbara J. Grosz. 1978. Focusing in dialog. American Journal of Computational Linguistics, pages 3-10. Microfiche 79.
Yoshihiko Hayashi. 2022. Towards the detection of a semantic gap in the chain of commonsense knowl-
Seanie Lee, Dong Bok Lee, and Sung Ju Hwang. 2021. Contrastive learning with adversarial perturbations for conditional text generation. In International Con- ference on Learning Representations.
edge triples. In Proceedings of the Thirteemth Lan- guage Resources and Evaluation Conference, pages

---

Haonan Li. Yeyun Gong, Jian Jiao, Ruofei Zhang, Timo- thy Baldwin, and Nan Duan. 2021. KFCNet: Knowl- edge filtering and contrastive learning for generative commonsense reasoning. In Findings of the Associ- ation,fo for Computational Linguistics: EMNLP 2021, pages 2918-2928 Punta Cana, Dominican Republic. Association for Computational Linguisties.
In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume I (Long Papers), pages 2039-2048, New Or- leans, Louisiana. Association for Computational Lin- guistics.
William C. Mann. 1979. Design for dialogue compre- hension. In 17th Annual Meeting of the Association for Computational Linguistics, pages 83-84, La Jolla, California, USA. Association for Computational Lin- guistics.
Jiaqi Li, Ming Liu, Min- Yen Kan, Zihao Zheng. Zekun Wang, Wenqiang Lei, Ting Liu, and Bing Qin. 2020. Molweni: A challenge multiparty dialogues-bar -based machine reading comprehension dataset with dis- course structure. In Proceedings of the 28th Inter- national Conference on Computational Linguistics, pages 2642-2652, Barcelona, Spain (Online). Inter- national Committee on Computational Linguistics.
Yuning Mao, Wenchang Ma. Deren Lei, Jiawei Han, and Xiang Ren. 2021. Extract, denoise and enforce: Evaluating and improving concept preservation for text-to-text generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing, pages 5063-5074 Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Mingzhe Li, XieXiong Lin, Xiuying Chen, Jinxiong Chang, Qishen Zhang, Feng Wang. Taifeng Wang. Zhongyi Liu, Wei Chu, Dongyan Zhao, and Rui Yan. 2022a. Keywords and instances: A hierarchical con- trastive learning framework unifying hybrid granu- larities for text generation. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume I: Long Papers), pages 4432 4441, Dublin, Ireland. Association for Compu- tational Linguistics.
Kathleen R. McKeown. 1979. Paraphrasing using given and new information in a question answer system. In 17th Annual Meeting of the Association for Computa- tional Linguistics, pages 67-72, La Jolla, California, USA. Association for Computational Linguistics.
Weizhao Li. Junsheng Kong, Ben Liao, and Yi Cai. 2022b. Mitigating contradictions in dialogue based on contrastive learning. In Findings of the Associa- tion for Compulational Linguistics; ACL 2022, pages 2781-2788, Dublin, Ireland. Association for Compu- tational Linguistics.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions words and phrases and their compositionality. In Advances in Neural Information Processing Sys- lems, volume 26. Curran Associates, Inc.
Chin-Yew Lin. 2004. ROUGE: A package for auto- matic evaluation of summaries. In Text Summariza- tion Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
Jerry L. Morgan. 1978 Toward rational model of discourse comprehension. In Theoretical Ixsues in Natural Language Processing-2.
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial NLI: A new benchmark for natural language under- standing. In Proceedings of the 58th Anmual Meet- ing of the Association for Computational Länguistics. pages 4885-4901, Online, Association for Computa- tional Linguistics.
Che Liu, Rui Wang. Jinghua Liu, Jian Sun, Fei Huang, and Luo Si. 2021a. DialogueCSE: Dialogue-bused contrastive learning of sentence embeddings. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pages 2396- 2406. Online and Punta Cana. Dominican Republic. Association for Computational Linguistics.
Kento Nozawa and Issei Sato. 2021. Understand- ing negative samples in instance discriminative self- supervised representation learning. In Advances in Neural Information Processing Systems, volume 34, pages 5784 5797. Curran Associates, Inc.
Junpeng Liu, Yanyan Zou, Hainan Zhang, Hongshen Chen, Zhuoye Ding, Caixia Yuan, and Xiaojie Wang. 2021b. Topic-aware contrastive learning for abstrac- tive dialogue summarization. In Findings of the Asso- ciation for Computational Linguistics: EMNLP 2021, pages 1229-1243, Punta Cana, Dominican Republic. Association for Computational Linguistics.
OpenAL. 2022. Openai: Introducing chatgpt. https: //openai. coh///oo//chatptt.. [Online; accessed 12-Oct-2023).
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 1907
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 31 1-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Kaixin Ma. Tomasz Jurezyk, and Jinho D. Choi. 2018. Challenging reading comprehension on daily con- versation: Passage completion on multiparty dialog.

---

Bhargavi Paranjape. Julian Michael, Marjan Ghazvininejad. Hannaneh Hajishirzi, and Luke 7er. 2021. Prompting contrastive explana- tions for commonsense reasoning tasks. In Findings of the Association for Computational Linguistics: ACI-IICNLP 2021, pages 4192, Online. Association for Computational Linguistics.
Yisi Sang, Xiangyang Mou, Mo Yu, Shunyu Yao, Jing Li, and Jeffrey Stanton. 2022. TVShowGess: Char- acter comprehension in stories speaker guessing. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 4267-4287, Seattle, United States. Association for Computational Linguistics.
ChaeHun Park, Eugene Jang. Wonsuk Yang, and Jong Park. 2021. Generating negative samples by manip- ulating golden responses for unsupervised learning of response evaluation model. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 1525-1533 Online, Association for Computational Linguistics.
Maarten Sap. Ronan Le Bras, Emily Allaway. Chan- dra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A. Smith, and Yejin Choi. 2019. Atomic: An atlas of machine commonsense for if- then reasoning. Proceedings of the AAAI Conference Arrificial Intelligence, 33(01):3027-3033
Chang Shu, Yusen Zhang, Xiangyu Dong, Peng Shi Tao Yu, and Rui Zhang. 2021. Logie-consistency text generation from semantic parses. In Findings of the Association for Computational Linguistics: ACL- IJCNLP 2021, pages 4414-4426,, Online. Association for Computational Linguistics.
Brian Phillips. 1975. Judging the coherency of dis- course. American Journal of Computational Linguis- rics, pages 36-49. Microfiche 35.
Lianhui Qin, Aditya Gupta, Shyam Upadhyay, Luheng He, Yejin Choi, and Manaal Faruqui. 2021. TIME-
DIAL: Temporal commonsense reasoning in dialog. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I Ith International Joint Conference on Natural Lan- guage Processing (Volume I: Long Papers), pages 7066-7076, Online. Association for Computational Linguistics.
Noah A. Smith and Jason Eisner. 2005. Contrastive esti- mation: Training oogllnnar models on unlabeled data, In Proceedings of the 43rd Annual Meeting of the As- sociation for Computational Linguistics (ACI,005) '05), pages 354-362, Ann Arbor, Michigan. Association for Computational Linguistics.
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5:
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
An open multilingual graph of gen- eral knowledge. In Proceedings of the Thirty-Fire AAAI Conference on Artificial Intelligence, AAAI 17, page 4444-4451, AAAI Press.
Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of Machine Learning Research,
Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Yejin Choi, and Claire Cardie. 2019. DREAM: A challenge data set and models for dialogue- based reading compre- hension. Transactions of the Association for Compu- tational Linguistics, 7:217-231.
21(140):1-67.
Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don't 't know: Unanswerable ques- tions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume 2: Short Papers), pages 784-789, Melbourne, Australia. Association for Computational Linguisties.
Xiangru Tang, Arjun Nair, Borui Wang. Bingyao Wang. Jai Desai, Aaron Wade, Haoran Li, Asli Celikyil- maz, Yashar Mehdad, and Dragomir Radev. 2022. CONFIT: Toward faithful dialogue summarization with sitiistiicall-ifommeee contrastive fine-tuning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 5657-5668, Seattle, United States. Association for Computational Linguistics.
Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019. CoQA: A conversational question answering challenge. Transactions of the Association for Com- putational Linguistics. 7:249-266.
Perry W. Thorndyke. 1976. The role of inferences in dis- course comprehension. Journal of Verbal Learning and Verbal Behavior, 15(4):37-446.
Chuck Rieger. 1974. Understanding by conceptual in- ference. American Journal of Computational Lin- guistics. Microfiche 13.
James Thorne, Andreas Vlachos, Christos Christodouloppulos, and Arpit Mittal. 2018 FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Papers), pages 809-819, New Orleans, Louisiana. Association for Computational Linguistics.
Rachel Rudinger, Vered Shwartz, Jena D. Hwang, Chan- dra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A. Smith, and Yejin Choi. 2020. Thinking like a skeptic: Defeasible inference in natural language. In Findings of the Association for Computational Lin- guistics: EMNLP 2020, pages 4661-4675, Online. Association for Computational Linguistics.

---

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux. Timothée Lacroix, Baptiste Rozière, Naman Goyal, Fric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13971.
Zhilin Yang, Peng Qi. Saizheng Zhang, Yoshua Bengio. William Cohen, Ruslan Salakhutdinov, and Christo- pher D. Manning. 2018. HotpoiQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 2369-2380, Brussels, Belgium. Association for Com- putational Linguistics.
Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015, Cider: Consensus- image de- scription evaluation. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4566-4575.
Yuheng Zha, Yichi Yang. Ruichen Li, and Zhiting Hu. 2023. AlignScore: Evaluating factual consistency with a unified aligmment function. In Proceedings of the 61st Annual Meeting of the Association for Computational I Linguistics (Volume I: Long Papers), pages 11 11328-11348 Toronto, Canada, Association for Computational Linguistics.
Ben Wang Aran Komatsuzaki. 2021. GPT-J- 6B: A 6 Billion Parameter Autoregressive Lan- guage Model. https: //gi com/kingoflolz/ Sesshttransffrmer--a
Satosi Watanabe. 1960. Information-theoretiin aspects of inductive and deductive inference. IBM journal of research and development, 4(2):208-231
Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2018. Constructing datasets for multi-hop reading comprehension across documents. Transactions of the Association Computational Linguistics, 6:287- 302.
Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2019. Dialogue natural language inference. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 3731-3741, Florence, Italy Association for Computational Linguistics.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume I (Long Pupers). pages 1112-1122, New Orleans, Louisiana. Association for Computational Linguis- tics.
Adina Williams, Tristan Thrush, and Douwe Kiela. 2022. ANLIzing the adversarial natural language inference dataset. In Proceedings of the Society for Computation in Linguistics 2022, pages 23-54, on- line. Association for Computational Linguistics.
Chien-Sheng Wu, Andrea Madotto, Wenhao Liu, Pas- cale Fung, and Caiming Xiong. 2022. QAConv: Question answering on informative conversations. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume
1: Long Papers). pages 5389-541 Dublin, Ireland. Association for Computational Linguistics.
Zhengzhe Yang and Jinho D. Choi. 2019. FriendsQA: Open-domain question answering on TV show tran- seripts. In Proceedings of the 20th Annual SIGdial Meeting Discourse and Dialogue, pages 188-197, Stockholm, Sweden. Association for Computational Linguistics.

---

A Additional Details of Experiments
respect T5-base and Gold, respectively. The corresponding results are shown in Table 5. More- over, the inter-annotator agreement also calcu- lated based on Fleiss Kappa. We implement Fleiss Kappa (x) based on the "statsmodels" package A.3 To calculate the significance level of the advantage of our method over the baseline and gold, we also calculate the winning rate of each model under eval- uation in Table A.2. More specifically, the model will gain one score if the annotator chooses the cor- responding option or "both", or the model will gain zero score instead. We take an average over all the scores and consider that to the human evalu- ation result of the model. For better representation. we show these results as percentages. We calcu- late the significance level with pair-wise individual t-tests given the scores from all the samples.
A.1 Details of NLI Models
UNLI Model Following Chen et al. (2020), we apply RoBERTa-large model (Liu et al., 2019) on the w-SNLI datasets. We first train the model on ANLI dataset which is to classify into three cate- gories (entail, neutral, contradict ) on SNLI (Bow- man et al., 2015). MNLI (Williams al., 2018), FEVER-NAL (Thorne al., 2018), ANLI (RI, R2, R3) (Nie et al., 2020), and then switch the classifier to be regression way and train on u-SNLI datasets. A.I In our observation, this warmup helps to im- prove the performance on u-SNLI. Our training batch size is 16, and we train for 3 epochs with the learning of le - 5. In Table A.1, we report the scores on u-SNLI development set along with the numbers reported in Chen al. (2020): the Person's correlation coefficient r, the Spearman rank correlation p. and the mean square error (MSE). our main experiments, we use the best model as the model-based evaluation metric.
and test
A.3 Choice of X and A
The coefficient bb and A is set be X 0.5 based on the preliminary experiments reported in Table A.3.
AlignScore Model Following Zha et al. (2023), we apply the RoBERTa-large model, and the check- point distributed by the A22
B Additional Experimental Results
B.1 Score breakdown of question types In Table B.I, we report the breakdown of the au- tomatic evaluation results of each question type of the CICERO test set.
A.2 Details of Human Evaluation
As it is stated in Section 5.2, human evaluation on Plausibility is conducted 450 data samples in total across six subtasks from the CICERO test set. We compare the generated inferences from our model with those from T5-base and gold in- ferences. A/B testing utilized and we ensure that each data sample are evaluated by three dif- ferent annotators. For quality control, we limit the annotators' locations to the United States, United Kingdom, Canada, Australia ensure English proficiency. All the annotators are required swer 20 test questions with more than 80% accu- racy before starting the annotation. During the human evaluation, we present the same context and two options from different models to annotators in comparison. The annotators are required to decide which inference is more plausible by choosing from "Option 1", "Option 2", "both", and "neither". The annotator instruction is presented in Figure A.I.
B.2 Case Study
In Table 1, we show one example in the "Conceiv- able" level. We also present the examples in the "Sufficient" and "Likely" levels in Table B.3 and Table B.2, respectively.
After collecting all the annotations, we first cal- culate the ratio of Win/Tie/Loss of our model with
^ www.e-snniggnnniiieiieeiie _aan__aan______ R1_____11____________
A / htt.//gtttttttttteeee/eete...e hatphuub.cmm/uuuhaaaeeee cww/yuh- htwryyyuhhhh/AinnSeeeee
html

---

Dev
Test
Roberta-large
P
MSE
r
p
MSE
Chen et al. (2020)
0.6383
0.6408
0.0751
0.6271
0.6346
0.0777
Ours
0.7334
0.7499
0.0635
0.7369
0.7525
0.0643
Ours w/o warmup
0.7115
0.7209
0.0666
0.7226
0.7370
0.0655
Table A.I: The performance the u-SNLI development and test set.
Model
Plausibility
Sufficient
Likely
Conceivable
Ours
47.5 %
53.7 % 27.7%
45.0%
43.4%
Our vs T5-base
T5-base
34.3%
35.3%
40.2%
Ours
35.1%
35.3%
30.9%
39.3%
Ours Gold
Gold
33.3%
37.7%
31.1%
31.0%
Table A.2: Human evaluation results calculated based on the winning rate of each model, The results in bold are significantly better than that for the other model with pair-wise individual t-tests (p 0.05).
Ab
BLEU-1
BLEU-2
BLEU-3
BLEU-4
METEOR
ROUGE_L
CIDEr
0.5
0.5
30.67
17.09
9.45
5.65
16.62
28.50
40.53
1.0
1.0
30.29
16.69
9.09
5.42
16.55
28.31
38.77
2.0
2.0
29.81
16.37
8.88
5.28
16.32
28.05
38.18
Table A.3: Automatic results on CICERO test set with different A and As-

---

Which Answer Is More Plausible?
Instrucioos
Overview
job, presented conversation determine more plausible.
Steps
Read the conversation and the question.
Choese plousible
Examples
Dialogue
Plausible: Option
User A: you deliver it on Saturday?
Reason: Option 1 answers the qurrtion using information stated the provideed dislngar while Optinn 2 sevens contradi ling to the conversation
User R:Scm Dow morning work for you?
User A; Sounds sood,
Question: What prerequisite the target etterancc?
be available
Option 2:User not
Dialogue
Plausible: Both
User A: Jim, my car has problmm Could you please take look at mac?
Reason: Bulh Option 1 and Ootion seam plausible based on the dialogue
User D: Sune thing.
Question: What subsveg aent event happens fellowing the target ufterance?
the together parking
Option
Dialogue 3
Plausible: Neither
Uier A: There's been fire at factory.
Reason: While both Option 1 and Oytion 2 might be possible, there no clear clue choese nelther of the option.
User BtArc sure? There is nothing in the newspaper about it. wwlll Bebs
Yeah, prerequisite tarnet Question:
what's goinng
Option brocher works restaurant nearty
Option User wes annoyed by Bob being posslay.
Read dialogue below assess the iuusibiliiyy of options on question regarding the target utterancer
Dialogue::
feeling
know!
User A: was talking with Bob yesterday and learnt your right leg had been injured How did haopen?
User B: Their right back Tam knocked me down when inssed to their goal with the ball.
User A: Wow! He must have you hard.
User Of
He
from the back and memtime rolling and the time had lol of pain Anywayy Lhey brought haree
Nothling serious.
doctor said
any Injuries,
stay couple dlys.
Well,
easy.
User B: Thorkkyou for your coming, thanks for the flowers.
Target: right
when
with
Question: What is or could the cause torget?
over.
Option The listener inquired about the speaker's Iijury.
Which
more plausible? (required)
Option 1
Option
Both
Neither
Figure A.I: Annotator instruction of the human evaluation on Plausibility.

---

BLEU-1
BLEU-2
BLEU-3
BLEU-4
METEOR
ROUGE_
CIDEr
LLaMA
24.72
12.55
5.87
3.02
13.78
23.94
26.11
T5-small
27.21
13.54
7.06
4.24
15.56
24.88
36.89
+CL
27.60
13.70
6.94
4.06
13.61
25.09
35.36
T5-hase
27.65
14.52
7.92
4.91
16.13
25.80
43.37
Cause
CL
29.15
15.35
8.61
5,36
16.65
26.32
45.67
T5-large
27.79
14.87
8.27
5.19
16.26
26.29
46.81
CL
27.64
15.03
8.44
5.28
16.32
26.11
48.40
GPT2-basc
21.51
11.05
5.33
3.04
13.52
23.32
24.55
CL
18.67
9.95
4.83
2.67
12.85
22.81
23.08
LLaMA
30.99
13,43
6.76
3,43
13.76
28.12
26.61
T5-small
30.27
14.88
6.52
3.43
13.85
26.87
28.14 14
CL
30.75
15.12
6.80
3.64
15.90
27.32
28.38
b5-base
30.92
15.70
7.44
4.18
16.31
27.91
33.54
SE
CL
31.19
16.17
8.01
4.58
16.51
27.90
34.95
T5 15-large
30.65
16.18
8.20
4.82
16.45
28.62
36.56
CL. 31.26
16.37
8.26
4.79
16.56
28.65
37.30
Gase-base
26.85
13.67
6.02
3.23
14.65
26.63
22.64
CL
26.10
13.25
5.95
3.18
14.61
27.49
23.98
LLaMA
30.69
15.23
6.61
3.30
15.84
28.25
26.60
T5-small
29.91
14.64
6.36
3.35
15.70
26.84
27.95
CL. 30.18
14.84
6.58
3.47
15.73
27.20
27.79
T5-base
30.39
15.24
7.05
3,85
16.08
27.67
31.14
SE_ _l_ppipeed
CL
31,06
15.92
7.65
4.25
16.21
27.68
32.45
T5-
30.09
15.73
7.80
4,47
16.17
28.49
35.24
CL
30.80
15.96
7.83
4.45
16.33
28.37
35.45
Gase-base
26.74
13.45
5.70
2.98
14.67
26.48
21.68
+ CL.
26.55
13.34
5.81
3.04
14.67
27.44
22.33
LLaMA
21.38
5.46
2.68
13.11
23.04
22.05
T5-
18.36
9.61
4.76
2.52
12.60
20.53
26.02
+ +LL
18.49
9.81
4.78
2.41
12.48
20.79
26.37
TS-base
19.32
10.21
5.03
2.64
13.15
21.46
29.89
Prerequisite
CL
20.04
10.94
5.69
3.20
13.26
21.92
31.66
T5-large
19.18
10.63
5.60
3.17
13.46
22.28
33.78
CI.
19.75
10.87
5.80
3.27
13.53
21.92
34.13
GPT2-hase
15.73
8.75
4.39
2.37
11.35
20.32
20.53
+ CL
13.09
7.56
3.69
1.97
10.91
19.64
20.28
LLaMA
31.22
20.65
11.90
6.43
17.36
32.88
36.77
T5-small
34.26
24.25
16.56
10.51
18.73
36.10
56.23
+ +L.
34.11
24.35
10.54
18.66
36.24
56.39
T5-hase
35.23
25.07
17.20
11.31
19.81
37.60
66.52
Motivation
+ +LL
35.74
25.53
17.59
11.67
19.97
38.00
68.46
Ta-ge
34.94
25.38
17.83
12.29
20.52
38.83
77.02
CL
35.59
25.73
17.92
12.14
20.30
38.41
75.54
GPT2-base
30.12
21.14
14.09
8.88
17.14
35.05
42,98
+ CL
29.05
20.73
14.14
17.34
35.85
43.77
LLaMA
20.77
15.03
9.21
5.31
15.24
30.54
18.45
T5-small
34.15
23.55
15.20
9.44
18.50
36.17
39.39
+CL
33.51
23.16
15.01
9.46
18.26
36.10
39.96
TS-base
33.98
23.73
15.62
10.10
19.05
37.18
45,98
Reaction
CL
33.95
23.81
15.68
10.04
19.08
37.04
46.78
Ta-ge
33.26
23.67
15.92
10.53
19.34
37.52
55.21
CL
34.51
24.44
16.37
10.80
19.35
37.23
54.82
GPT2-hase
26.15
16.89
10.73
6.54
16.17
32.20
29.90
CL.
25.34
18.24
12.04
7.40
16.56
34.52
25.75
Table B.1: Automatic results on the CICERO test set for each question type.

---

User A: Hello, what can for you?
User B: Hello, come to pay my water and electricity fees.
User A: Give me your water and electricity bills, please.
User B: Here they are.
User A: You should pay 160 yuan for the electricity fee and 80 yuan for the water fee.
Dialogue
User B: Do you mean that should pay 240 yuan in total? (Target]
User A: Yes. Will you pay by cash or credit card?
User B: Cash, please. Here
money. User A: I get 250 yuan from you, and this the change, 10 yuan.
User B: OK. Thank you. Bye-bye.
User A: Bye
Question
What is could be the motivation of the target?
Gold
The speaker shocked to hear the total amount be paid for his utility bill.
TS-base
The speaker curious to know should pay 240 yuan for electricity 80 yuan water fees.
Ours
The speaker is curious to know how much he has to for electricity and water.
Table B.2: One example in "Likely" difficulty level, comparing the generated inferences from our method, T5-base, and the gold inference. We highlight the false inference green

---

Example
User A: Airports sad places.
User Sometimes, guess. But write each other. You'l 'll come down at Christmas.
User A: If we can find the money.
User Don't worry, Marta. Everything will be taken of. They say that fares going reduced in next six months.
And when graduate, well
User That's two years from now. Two years long time.
Dialogue
User B: The time will pass quickly. You'll Il see. I might even be able back to New York next summer.
User Oh, John, you'll 'Il forget about me. Your mother will find you nice girl, you'll get married, and live
happily ever after.
User B: No, won't. I swear won't. Believe please.
User A: Whatever you say, all know is that you are going to be taken away from me.
User B: That's ridiculous I'll write every day, whether you answer me not.
User Don't silly. You'll have other things to (She begins cry.)
User B: Don't cry, Marta, please. [Target]
Question
What the possible emotional reaction of the listener in response to the target?
Gold
The listener was sad that she wouldn't be able to see john for long time.
TS-base
The listener is happy help marta.
Ours
The listener foeling sad marta.
Example
User A: Honey I think you should quit smoking.
User B: Why? You said was hot when smoking.
User A: But want you he fit.
User B: Smoking is killing. know.
User A: Check out this article. It says smoking lead to lung cancer. [Target]
Dialogue
User B:
believe it.
User A: But you know that smoking does harm to health, right?
User Of course know it, you know it's hard quit smoking...
User A: Stop beating around the bush. Will you quit or not?
User B: Yes, ma' am. Whatever you say.
Question
What could be the prerequisite of the target?
Gold
The article in the magazine was about lung cancer caused by smoking.
TS-base
The speaker is a smoker.
Ours
The speaker has read article about smoking.
Table B.3: Two examples in "Sufficient" difficulty level, comparing the generated inferences from our method. TS-base, and the gold inference. We highlight the false inference
green


Reviews:
[{"review_id": "v7fKUtd2pL", "reviewer": "Reviewer_eCnU", "summary": "", "questions": "One comment I would like to make is regarding the choice of automated evaluation metrics, themselves. The authors mainly show improvements using text similarity metrics which seem like a poor choice to judge inductive reasoning capabilities of a model.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments.\n\n>Weakness 1-1: The average performance improvements on the CICERO test set are minor. \n\n\n**Response**: As shown in Table 5, although the improvements on the CICERO test set (Table 4) are minor on automatic metrics, our method is significantly better based on human evaluation. \n\n\n>Weakness 1-2: Similarly, an analysis of why NLI metrics don't improve would have been welcome.\n\n**Response**: We have included a discussion about NLI metrics in Section 6.5. The reason why we put the results on the UNLI metric is not to evaluate the performance of our proposed models, but to analyze whether it is a feasible metric for commonsense reasoning or not. To mitigate the potential confusion, we will put this information separately in the final version of the paper in section 6.5.\n\n\n>Question 1: One comment I would like to make is regarding the choice of automated evaluation metrics, themselves. The authors mainly show improvements using text similarity metrics which seem like a poor choice to judge inductive reasoning capabilities of a model.\n\n**Response**: We agree that showing improvements using only text similarity metrics is a poor choice to judge the inductive reasoning capabilities of a model. \nTo further evaluate the performance, we provided a comprehensive human evaluation in Section 5.2 and our method achieved significant improvement over the baseline.\n\n>Typos etc.\n\n**Response**: Thank you for your correction. We will modify them accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1- The authors present a way to improve the inductive reasoning capability language models through contrastive learning.\n2- Authors formalize the notion of semantic information gap and empirically show that their approach improves the model's performance in both human and automatic evaluations.", "reasons_to_reject": "1- The average performance improvements on the CICERO test-set are minor. Similarly, an analysis of why NLI metrics don't improve would have been welcome.", "questions_to_authors": "One comment I would like to make is regarding the choice of automated evaluation metrics, themselves. The authors mainly show improvements using text similarity metrics which seem like a poor choice to judge inductive reasoning capabilities of a model.", "paper_topic_and_main_contributions": "The paper tries to analyze the problems in the inductive reasoning ability of the language models as a function of semantic information gap present in the language task model is trying to solve. They also present a way to improve the model performance and empirically show that their method is effective in many cases.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "small spelling error:\nline 193: lake -> lack"}, "extra_text": {}}, {"review_id": "hB3fvanXmK", "reviewer": "Reviewer_X1eN", "summary": "", "questions": "(A) Based on human evaluation, the proposed method's performance is better than gold. Do you have more thorough analysis on it to learn where the gain is from?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments.\n\n> Weakness 1-1: (Table 4) The proposed method (+CL) shows improvements in n-gram overlap-based evaluation metrics across all test models, but NLI based metrics produce mixed results: it seems there's only improvements in GPT2-base model, while no improvement (some even with performance hurt) in T5 models.\n\n**Response**: We have already included a discussion about NLI metrics in Section 6.5, and our results are consistent. The reason why we put the results on the UNLI metric is not to evaluate the performance of our proposed models, but to analyze whether it is a feasible metric for commonsense reasoning or not. We conclude that NLI metrics cannot evaluate the reasoning ability of the generated texts, since most of the time gold answers are labeled as “neutral” and undeterminable, and it is extremely difficult to associate the numbers with the quality of generated inferences. To mitigate the potential confusion, we put this information separately in the final version of the paper.\n\n> Weakness 1-2: Based on overlap-based evaluation metrics (Table 8), the authors reach to the conclusion that the method \"promote the model to conduct inference either fully deductively or inductively\", while based on human evaluation (Table A.2), the improvement is only significant in \"Sufficient\" level.\n\n**Response**: In Table A.2, ours is significantly better than t5-base at the sufficient level, and than gold at the conceivable level.\nTable 5 clearly shows that even though not significant at the Conceivable level, a majority of annotations assess our model's wins over the T5-base in both Sufficient and Conceivable levels, which aligns with the overlap-based results. In Table 8, we show that our method improves the performance more at sufficient and conceivable levels.\nThus, we concluded that our method promotes the model to conduct inference either fully deductively or inductively.\n\n> Question-A: Based on human evaluation, the proposed method's performance is better than gold. Do you have more thorough analysis on it to learn where the gain is from?\n\n**Response**: Based on the insights gained from the human evaluation (see Table 5), our proposed method is NOT better than gold, but it is comparable to gold, except at the Conceivable level. Conceivable level gold answers tend to include something that is not stated/verifiable in the dialogue contexts provided, while Ours tends to be more supported by the dialogue context (see the examples in Table 1 for a comparison of ours vs. gold). We believe this resulted in Ours being more favored by annotators.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The idea behind the proposed method is intuitive and simple (which is good) and the method is model architecture-agnostic.\n\n- The problem the paper tries to address, Improving LLM's capability of inductive reasoning, is an important research topic for LLM, especially now LLMs, like ChatGPT, are becoming increasingly popular in the public eye.\n\n- The paper clearly states the proposed method with sufficient details and has done thorough evaluation and ablation study to help readers to gain a thorough understanding. ", "reasons_to_reject": "- The results of different metrics are not telling a consistent story, as the authors claimed in the paper.  \n  - (Table 4) The proposed method (+CL) shows improvements in n-gram overlap-based evaluation metrics across all test models, but NLI based metrics produce mixed results: it seems there's only improvements in GPT2-base model, while no improvement (some even with performance hurt) in T5 models.\n  - Based on overlap-based evaluation metrics (Table 8), the authors reach to the conclusion that the method \"promote the model to conduct inference either fully deductively or inductively\", while based on human evaluation (Table A.2), the improvement is only significant in \"Sufficient\" level.\n\n- As the authors also mentioned, the paper only has experiment results on one dataset. It'd be helpful to test the method on at least another one.\n\n", "questions_to_authors": "(A) Based on human evaluation, the proposed method's performance is better than gold. Do you have more thorough analysis on it to learn where the gain is from?", "paper_topic_and_main_contributions": "The paper proposes a contrastive learning approach to improve LLM's capability of inductive reasoning. \n\nThe authors first demonstrate that a finetuned baseline T5 model performs worse when the information gap for the inference task is bigger. Then they propose a contrastive learning approach with different variants in negative samples selection. The authors evaluate the method in dataset CICERO using automatic metrics and human evaluations. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fSwOK3yHga", "reviewer": "Reviewer_Qjsz", "summary": "", "questions": "A: Line 442, how is the coefficient chosen? Do you have results from a preliminary experiment? \n\nB: Is table 8 on T5-base, why are BLUE-2 and ROUGE-L for all difficulties higher than the aggregate result? \n\nC: Have you checked for information leakage from the training set to the test set? I'm a bit surprised that T5-small is almost good as T5-large. And what is the prompt you used for LLAMA? you may want to pay more attention to the prompt, as LLAMA is not instruction fine-tuned.\n", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments.\n\n> Weakness 1-1: My main complaint is that the two topics of this work, information gap and contrastive learning, do not glue together very well. Although results for different difficulties were shown in Table 5 and Table 8, the CL method itself has nothing to do with the discussion of inductive vs. deductive, and is not using the information gap labels. That is to say, the CL part is using the standard CL method, plus two heuristics of getting negative samples, in a task-agnostic way on this specific dataset, which offers limited new insight to the community.\n\n**Response**: As described in paragraph lines 66-83, contrastive learning is intended to expose negative samples during training to give more guidance. In deductive reasoning, all the information required to generate an output is provided in the input, and there is no information gap. However, inductive reasoning requires including something that may not be explicitly stated in the input, and that is not simply learnable by only exposing gold samples. Thus, we need to teach the model with more guidance on the reasoning path. In our preliminary experiment using the same dataset and a multi-choice framework, we see a significant improvement by feeding negative samples together with the other candidates, which indicates that feeding negative samples will help the model learn how to fill the information gap.\n\n> Weakness 1-2: I would encourage the author to look a bit more into how will the CL algorithm be different if we know what is the information gap for each sample. For example, maybe samples with a larger information gap are more difficult and need more help from the negative samples, while the \"sufficient\" ones are easy enough so some computation can be pruned during the training.\n\n**Response**: This is a great point and a really interesting idea, however, this requires an expensive annotation over all the samples in the train set. As discussed in the paper, there is no automatic approximation method to estimate information gaps, and thus applying such techniques is not realistic. In the final version of the paper, we will add an analysis of the impact of the number of negative samples during training, observing the performances on the different levels. \n\n> Weakness 2-1: There are several claims made without sufficient analysis. For line 322 and the examples followed, if the emotion is easily guessed by the sentiment, then that should reflect in the annotations as well. The argument in the paper explains the distribution of difficulty, NOT how it relates to the automatic metrics.\n\n**Response**: In Table 2 and Table 3, we reported the breakdown of the automatic evaluation metrics based on the levels so that we support our claim that “information gap can explain the difficulty in inference in dialogue”. For line 322 and the examples followed, we wanted to point out that the poorly correlated pair between difficulty levels and automatic metrics is caused by another aspect, which is the type of information gap. \n\n> Weakness 2-2: Line 522, for \"non-optimal\" and \"Replace ZS\", I don't think they follows the claim of \"have a positive impact\", and for the \"Replace MCQ\", we would expect a statistic significance before making the claim\n\n**Response**: Thank you for your clarification. What we meant here is that negative samples in relatively good quality (Replace_MCQ and contradiction) have a positive impact on the performance, but more noisy negative samples are less helpful.\n\n> Weakness 2-3: Line 562, \"promote the model to conduct inference either deductively or inductively\", there is no proof and no example for this claim, and feels like it comes out from nowhere.\n\n**Response**: “Deductively” and “inductively” correspond to “sufficient” level and “conceivable” level, respectively.  We made this claim based on the human judgment of plausibility presented in Table 5. Please refer to Figure A.1 for annotator instructions to see how they correlate to these levels. For examples to support this claim, please refer to Table 1, B.2, and B.3, where we present dialogues, questions, and gold/T5-base/Ours for comparisons on inference ability. Given your comments, we would like to add more explanations when we introduce the difficulty levels.\n\n> Weakness 2-4: Line 459, \"the UNLI scores are hardly interpretable given SNLI\". NO, the number listed suggests it works just as intended on SNLI. Consider replacing the claim with something from section 6.5 instead, which is a section that is worth more emphasis.\n\n**Response**: Thank you for your suggestion, we will replace the claim and reorganize the corresponding part.\n\n> Weakness 3: Some results seem to be an error either from the write-up or experiment. Compare the BLEU-2 and ROUGE-L columns in Table 8 with Table 4, It is not possible that each difficulty has higher metrics than the aggregate results through average.\n\n> Question B: Is table 8 on T5-base, why are BLUE-2 and ROUGE-L for all difficulties higher than the aggregate result?\n\n**Response**: This is because Table 8 reports T5-base with contrastive learning scores on the subset of the test set that has task difficulty annotation, resulting in different numbers reported in Table 4. The scores of T5-base without contrastive learning in the same subset are reported in Table 2, and the comparisons are made against Table 2.\n\n> Weakness 4: A few mistakes in the formulation in section 4, line 333, all the j should be replaced with \"i\". Line 337, according to the description, the negative set is different for each utterance, either remove the SUM on \"n\" and state it is the loss for an utterance, or add superscript \"n\" to A as done with A_tilt.\n\n**Response**: Thank you so much for the correction! We will update correspondingly in the final version.\n\n> Question A: Line 442, how is the coefficient chosen? Do you have results from a preliminary experiment?\n\n**Response**: We test with several coefficients and report the one with the best results in the paper. The results from the preliminary experiment with T5-base are shown below:\n\n| lambda_b | lambda_s | BLEU-1 | BLEU-2 | BLEU-3 | BLEU-4 | METEOR | ROUGE_L | CIDEr |\n|----------|----------|--------|--------|--------|--------|--------|---------|-------|\n| 0.5      | 0.5      | 30.67  | 17.09  | 9.45   | 5.65   | 16.62  | 28.50   | 40.53 |\n| 1.0      | 1.0      | 30.29  | 16.69  | 9.09   | 5.42   | 16.55  | 28.31   | 38.77 |\n| 2.0      | 2.0      | 29.81  | 16.37  | 8.88   | 5.28   | 16.32  | 28.05   | 38.18 |\n\n> Question C-1: Have you checked for information leakage from the training set to the test set? I'm a bit surprised that T5-small is almost good as T5-large. And what is the prompt you used for LLAMA? you may want to pay more attention to the prompt, as LLAMA is not instruction fine-tuned.\n\n**Response**: Regarding the information leakage from the training set to the test set following [Elangovan et al., 2021]. For each sample in the test set, the similarity between all the samples is evaluated and the highest similarity sample is chosen. Then, we take the average of the similarity over all the test instances as an indicator to measure the extent of train/test overlap. We compute their overlap by representing instances with a count vector of uni/bi/trigrams and computing the cosine similarity. We compare the similarity between dialogues and answers in the train set and test set. The results are as follows:\n\n|          | unigram (%) | bigram (%) | trigram (%) |\n|----------|-------------|------------|-------------|\n| dialogue | 68.54       | 20.15      | 9.67        |\n| answer   | 59.91       | 35.62      | 13.31       |\n\nConsidering that 60% is the threshold of being “highly overlapped” [Elangovan et al., 2021], the dialogue and answers in the CICERO train and test set have some level of overlap between them in unigrams. However, the overlap does not necessarily mean leakage. For example, a generic answer \"the speaker is satisfied with the listener's proposal\" can appear in both train/test sets.\n\n[Elangovan et al., 2021] Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluation. EACL 2021.\n\n>Question C-2:  And what is the prompt you used for LLAMA? you may want to pay more attention to the prompt, as LLAMA is not instruction fine-tuned.\n\n**Response**: You can find prompts used in the supplemental material. In our preliminary experiments, we tried different prompts and retrieved 3-most similar samples to input, but it did not help. In the paper, we put the best results of LLAMA. Please refer to the answer to Reviewer FcSR Weakness 1.\n\n> Suggestions: For table3, maybe look a bit more into the performance of samples with the same difficult level but with different relationships, before making the claim in line 322\n\n**Response**: Thank you for your suggestion. Due to the page limit, we have put the breakdown scores for each relation in Table B.1 in the appendix. We will clearly point it out in the camera-ready.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The framing of the question is very interesting. Based on the observation that LMs do worse on inductive reasoning, the author attempts to connect that to differences in task difficulty based on information gap. This framing can provide some insights into issues faced by LLMs, for example, if we find the response from LLM has a large information gap, we may want to be more cautious and introduce extra guardrails, etc.  So overall I would like to see more analysis from the community on this. \n\n2. A dataset is presented with annotations of task difficulty for commonsense inference. It has 450 samples which is a decent number, and can be useful. \n\n3. The author tested CL on the task, and show that it improves LMs' performance both in automatic metrics and human evaluation.", "reasons_to_reject": "1. My main complaint is that the two topics of this work, information gap and contrastive learning, do not glue together very well. Although results for different difficulties were shown in Table 5 and Table 8, the CL method itself has nothing to do with the discussion of inductive vs. deductive, and is not using the information gap labels. That is to say, the CL part is using the standard CL method, plus two heuristics of getting negative samples, in a task-agnostic way on this specific dataset, which offers limited new insight to the community.  \nI would encourage the author to look a bit more into how will the CL algorithm be different if we know what is the information gap for each sample. For example, maybe samples with a larger information gap are more difficult and need more help from the negative samples, while the \"sufficient\" ones are easy enough so some computation can be pruned during the training. \n\n2. There are several claims made without sufficient analysis. For line 322 and the examples followed, if the emotion is easily guessed by the sentiment, then that should reflect in the annotations as well. The argument in the paper explains the distribution of difficulty, NOT how it relates to the automatic metrics. \n\nLine 522, for \"non-optimal\" and \"Replace ZS\", I don't think they follows the claim of \"have a positive impact\", and for the \"Replace MCQ\", we would expect a statistic significance before making the claim\n\nLine 562, \"promote the model to conduct inference either deductively or inductively\", there is no proof and no example for this claim, and feels like it comes out from nowhere. \n\nLine 459, \"the UNLI scores are hardly interpretable given SNLI\". NO, the number listed suggests it works just as intended on SNLI. Consider replacing the claim with something from section 6.5 instead, which is a section that is worth more emphasis. \n\n3. Some results seem to be an error either from the write-up or experiment. Compare the BLEU-2 and ROUGE-L columns in  Table 8 with Table 4, It is not possible that each difficulty has higher metrics than the aggregate results through average.\n\n4. A few mistakes in the formulation in section 4, line 333, all the j should be replaced with \"i\". Line 337, according to the description, the negative set is different for each utterance, either remove the SUM on \"n\" and state it is the loss for an utterance, or add superscript \"n\" to A as done with A_tilt.", "questions_to_authors": "A: Line 442, how is the coefficient chosen? Do you have results from a preliminary experiment? \n\nB: Is table 8 on T5-base, why are BLUE-2 and ROUGE-L for all difficulties higher than the aggregate result? \n\nC: Have you checked for information leakage from the training set to the test set? I'm a bit surprised that T5-small is almost good as T5-large. And what is the prompt you used for LLAMA? you may want to pay more attention to the prompt, as LLAMA is not instruction fine-tuned.\n", "paper_topic_and_main_contributions": "This paper studies how language models behave on dialogue inference tasks with different difficulties in terms of information gap, and then uses contrastive learning to improve LMs' performance on the inference task. \n\nThe author takes  450 samples from CICERO, a dialogue dataset for commonsense inference, and recruits annotators to classify each sample into three levels of difficulty. The difficulty is defined based on the information gap between the context and target inference, e.g. \"sufficient\" means all the information for making the inference is presented in the context already. Based on the labels of difficulty, the author shows that LM performs worse for samples with a larger information gap. \n\nThe author then proposes to use contrastive learning to improve models' performance on the task. For the CL loss, the objective uses regular in-batch negatives, with the addition of hard negatives from the wrong answers in CICERO datasets, as it is a multiple QA dataset. The author then proposes two other approaches to get hard negatives based on some heuristics, namely non-optional generation based on ideas similar to bootstrapping, and token replacement those tokens affected the most by context are replaced. \n\nThe author measured LM's performance with commonly used generation metrics, and added UNLI, a model-based metric for NLI. A human evaluation is carried out and proves the CL method improves more favorable answers over the baseline. \n\n\nMain Contribution: \n\n1. This work provides a human-annotated dialogue dataset for commonsense inference with task difficulty annotated.\n\n2. The author frames the task difficulty as the information gap and shows that model performance are correlate with an information gap\nThe author tests contrastive learning on the task with a few ways to get hard negatives and shows that CL increases LMs' performance. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "line 26, \"go beyond\" should be \"goes beyond\".\n\nGiven sec6.5, maybe consider removing UNLI if you think it doesn't really make sense. \n\nFor table3, maybe look a bit more into the performance of samples with the same difficult level but with different relationships, before making the claim in line 322"}}, {"review_id": "14hxt3BROa", "reviewer": "Reviewer_FcSR", "summary": "", "questions": "Question A: Is the drop in model performances in Table 2 due to the increases of the information gap or the inaccurate evaluation metrics? It seems that the answer can be more diversified for Conceivable than Sufficient. So, the unrelated but appropriate generated answers compared to the references may show poor performances with those evaluation metrics.\n\nQuestion B: In Table 3, the subset SE_Clipped is supposed to be more difficult with more Conceivable samples, while its performance is significantly better than SE on CIDEr and Prerequisite on all of the metrics.\n\nQuestion C: In Table 5, it seems that Ours is more competitive on the Conceivable samples when compared with Gold than the Sufficient samples. Does it contradict the hypothesis? How about the human evaluation result of T5-base vs Gold?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments.\n\n> Weakness 1: The baselines are too weak. The backbone language models (T5 and GPT2) are also a bit out-of-date. More recent models are recommended, such as T5-Flan, GPT-J, etc. More effective few-shot in-context learning strategies are also expected for the LLAMA baseline, such as selecting the 3 most similar samples instead of random sampling.\n\n**Response**: First, we have to clarify that we have put LLAMA, the most recent and SOTA LM, as one of our baselines.  In our preliminary experiments with GPT-J, we found its performance is even lower than LLAMA. We also have tried a more strategic prompt using tf-idf to retrieve 3-most similar in-context examples, but it seems that does not help. Note 3-shot are average of three trials with different random seeds. The results are as follows:\n\n| Model |                | BLEU-1 | BLEU-2 | BLEU-3 | BLEU-4 | METEOR | ROUGE_L | CIDEr |\n|-------|----------------|--------|--------|--------|--------|--------|---------|-------|\n| GPT-J | 3-shot         | 24.70  | 13.04  | 6.11   | 3.04   | 13.74  | 25.03   | 19.87 |\n| GPT-J | 3-shot +tf-idf | 22.83  | 11.64  | 5.52   | 2.83   | 12.19  | 22.45   | 17.99 |\n| LLaMA | 3-shot         | 28.43  | 15.18  | 7.25   | 3.73   | 15.26  | 27.43   | 26.47 |\n| LLaMA | 3-shot +tf-idf | 25.36  | 13.33  | 6.56   | 3.49   | 13.72  | 24.91   | 24.06 |\n\nWe suspect that even though lexically similar, these examples may mislead the model to make wrong predictions. The reported LLAMA results in this paper are an average of three trials with our well-selected representative examples.\n\n> Weakness 2\n\n> Question A: Is the drop in model performances in Table 2 due to the increases of the information gap or the inaccurate evaluation metrics? It seems that the answer can be more diversified for Conceivable than Sufficient. So, the unrelated but appropriate generated answers compared to the references may show poor performances with those evaluation metrics.\n\n**Response**: Thank you for your question. We believe this diversity in Sufficient samples explains that Ours wins Gold on human evaluation in Table 5 (see the answer to Question C as well). Although automatic metrics are not sufficient, they capture a certain degree of quality of generated answers as they align with the human evaluation results (see Table 5 and Section 6.4).\n\n> Question B:  In Table 3, the subset SE_Clipped is supposed to be more difficult with more Conceivable samples, while its performance is significantly better than SE on CIDEr and Prerequisite on all of the metrics.\n\n**Response**: As described in line 319-329, the proportion of likely and conceivable questions can explain the difference in T5-base performance to a certain extent, it does not have a simple correlation. It may be due to the difference in which kind of information is required to bridge the gap between the dialogue and the answer.  The prerequisite is the most difficult task because the difficulty proportion is more distributed and the model cannot learn the reasoning skill and tendency whether to rely on the dialogue history or extend out of the dialogue history.\n\n> Question C: In Table 5, it seems that Ours is more competitive on the Conceivable samples when compared with Gold than the Sufficient samples. Does it contradict the hypothesis? How about the human evaluation result of T5-base vs Gold?\n\n**Response**: Conceivable level gold answers tend to include something that is not stated/verifiable in the dialogue contexts provided, while Ours tends to be more supported by the dialogue context (see the examples in Table 1 for a comparison of ours vs. gold). We believe this resulted in Ours being more favored by annotators. The sufficient level golds are clearly aligned with dialogue contexts, thus it is more challenging to outperform gold for Ours. We have not conducted the human evaluation of T5-base vs Gold, but will include in the final version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The topic of this paper is well-motivated and should be of great interest to the dialogue understanding community.\n\n2. Comprehensive experiments including human evaluation, case study, ablation study, improvements on different question types and task difficulty, etc., are carried out with detailed discussions.\n\n3. The approach is simple and easy to implement.", "reasons_to_reject": "1. The baselines are too weak. The backbone language models (T5 and GPT2) are also a bit out-of-date. More recent models are recommended, such as T5-Flan, GPT-J, etc. More effective few-shot in-context learning strategies are also expected for the LLAMA baseline, such as selecting the 3 most similar samples instead of random sampling.\n\n2. Some conclusions are inconsistent and confusing. More in Questions.", "questions_to_authors": "Question A: Is the drop in model performances in Table 2 due to the increases of the information gap or the inaccurate evaluation metrics? It seems that the answer can be more diversified for Conceivable than Sufficient. So, the unrelated but appropriate generated answers compared to the references may show poor performances with those evaluation metrics.\n\nQuestion B: In Table 3, the subset SE_Clipped is supposed to be more difficult with more Conceivable samples, while its performance is significantly better than SE on CIDEr and Prerequisite on all of the metrics.\n\nQuestion C: In Table 5, it seems that Ours is more competitive on the Conceivable samples when compared with Gold than the Sufficient samples. Does it contradict the hypothesis? How about the human evaluation result of T5-base vs Gold?", "paper_topic_and_main_contributions": "This paper analyzes the models' performance on dialogue understanding with the question-answering task from the aspect of the semantic information gap. They hypothesize that the difficulty of the task is proportional to the amount of this gap, and classify 450 test samples from the CICERO dataset into three levels with human annotators: sufficient, likely, and conceivable, where level 3 \"conceivable\" should require a higher level of the inductive reasoning ability of models to generate the answer to the corresponding question. The results of a preliminary study with the fine-tuned T5-base models verify the authors' hypothesis.\nThen, a contrastive learning approach with different ways for constructing negative samples is proposed to fine-tune the smaller models, such as T5 and GPT2, and achieves consistent improvements with different n-gram-based automatic evaluation metrics and human evaluation on the newly-labeled CICERO test set.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ufl15iAmaF", "reviewer": "Reviewer_LNAb", "summary": "", "questions": "- What is the motivation behind applying a contrastive learning approach to mitigate the issue of the information gap?\n- Do current LLMs truly exhibit a deficiency in their abilities for inductive reasoning? ", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments.\n\n> Weakness 1-1: The motivation behind applying a contrastive learning approach to mitigate the issue of the information gap is unclear. The proposed selection of negative samples might not be a straightforward approach to fill the information gap. \n\n> Question 1: What is the motivation behind applying a contrastive learning approach to mitigate the issue of the information gap?\n\n**Response**: As described in paragraph lines 66-83, contrastive learning is intended to expose negative samples during training to give more guidance. In deductive reasoning, all the information required to generate an output is provided in the input, and there is no information gap. Thus, we may not have to feed negative samples to teach models to reason. However, inductive reasoning requires including something that may not be explicitly stated in the input and that is not simply learnable by only exposing gold samples. Thus, we need to teach the model with more guidance on the reasoning path. In our preliminary experiment using the same dataset and a multi-choice framework, we see a significant improvement by feeding negative samples together with the other candidates, which indicates that feeding negative samples will help the model learn how to fill the information gap.\n\n>Weakness 1-2: I think the Non-Optional Generation does not always produce a wrong (negative) output, causing the model to struggle with distinguishing between positive/negative samples.\n\n**Response**: We use the Non-Optional Generation as one of the models for the ablation study to prove that it is not a proper source for producing negative samples. It is quite noisy and not always wrong/negative, as you have mentioned. \n\n>Weakness 2: While the authors claim the “Current LLMs, such as ChatGPT, lack the called “inductive reasoning” ability” in line 39, this paper evaluated only the LLaMA model as the current LLM in Table 4. Therefore, I didn’t understand how current LLMs can perform inductive reasoning and suggest that other LLMs, such as ChatGPT, might be compared with proposed methods.\n\n>Question 2: Do current LLMs truly exhibit a deficiency in their abilities for inductive reasoning?\n\n**Response**: Inductive reasoning is one of the major weaknesses of LLMs reported and less discussed compared to deductive reasoning ability [Huang and Chang, 2023]. For example, [Laskar et al., 2023] report that even ChatGPT performs much worse in inductive reasoning tasks compared to deductive reasoning tasks. \n\n[Huang and Chang., 2023] Towards Reasoning in Large Language Models: A Survey. ACL2023-Findings.\n\n[Laskar et al., 2023] A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets. ACL 2023.\n\n>Weakness 3: As discussed in Sec. 6.5, I understand evaluating inductive reasoning is challenging. But, I failed to understand that overlap-based metrics (e.g., BLEU) do not reflect its abilities. CICERO Dataset contains many samples requiring inductive reasoning abilities, and therefore I think there is some extent to which overlap-based metrics can be employed to evaluate inference ability.\n\nWe agree that automatic metrics can capture a certain degree of quality of generated answers as they align with the human evaluation results (see Table 5 and Section 6.4). \nHowever, as you can see from the human evaluation results of Ours vs. Gold on the Conceivable level, Ours wins against Gold. \nIt is because the conceivable level gold answers tend to include something that is not stated/verifiable in the dialogue contexts provided, while Ours tends to be more supported by the dialogue context (see the examples in Table 1 for a comparison of ours vs. gold and the Conceivable example between line 294-295). In such cases, we believe that solely relying on overlap-based metrics is insufficient.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This paper provides task difficulty annotation based on the information gap. The annotations will facilitate the research of understanding how the model can perform dialogues with inductive reasoning.\n- This paper investigates that existing models struggle with accomplishing inductive reasoning, requiring the ability to understand the information gap in the context of dialogue inference processes. \n- Proposed contrastive learning significantly improves the plausibility of the generated inferences.\n- Well-written and easy to follow.", "reasons_to_reject": "- The motivation behind applying a contrastive learning approach to mitigate the issue of the information gap is unclear. The proposed selection of negative samples might not be a straightforward approach to fill the information gap. I think the Non-Optional Generation does not always produce a wrong (negative) output, causing the model to struggle with distinguishing between positive/negative samples.\n- While the authors claim the “Current LLMs, such as ChatGPT, lack the called “inductive reasoning” ability” in line 39,  this paper evaluated only LLaMA model as the current LLM in Table 4. Therefore, I didn’t understand how current LLMs can perform inductive reasoning and suggest that other LLMs, such as ChatGPT, might be compared with proposed methods.\n- As discussed in Sec. 6.5, I understand evaluating inductive reasoning is challenging. But, I failed to understand that overlap-based metrics (e.g., BLEU) do not reflect its abilities. CICERO Dataset contains many samples requiring inductive reasoning abilities, and therefore I think there is some extent to which overlap-based metrics can be employed to evaluate inference ability.", "questions_to_authors": "- What is the motivation behind applying a contrastive learning approach to mitigate the issue of the information gap?\n- Do current LLMs truly exhibit a deficiency in their abilities for inductive reasoning? ", "paper_topic_and_main_contributions": "This paper conducts an analysis of inferences in dialogue, focusing on the availability of semantic information between dialogue context and outputs.\nThe authors manually annotate three levels of task difficulty based on the amount of information in the answer covered by the dialogue in the CICERO dataset. To verify the annotation quality, this paper investigates that the existing dialogue model (T5-base) uniformly degrades with the decrease in the amount of available information. Furthermore, the authors propose to apply contrastive learning and this approach helps to fill the information gap.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- LLAMA -> LLaMA in Table4 and Table B.1\n- 27.34 ROUGE-L score had better the normal font than the bold one in T5-small of Table4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper examines the inductive reasoning ability of language models in dialogue-based tasks by analyzing the semantic information gap between the context and the inference required. The authors propose a contrastive learning approach to improve model performance. They conduct thorough experiments and evaluations, including human evaluation, to support their claims.

The reviewers generally find the topic interesting and well-motivated. They appreciate the clarity and thoroughness of the experiments and analysis. The proposed method is considered simple and easy to implement.

However, there are some concerns raised by the reviewers. One reviewer points out that the average performance improvements on the test set are minor and suggests analyzing why NLI metrics don't improve. Another reviewer questions the consistency of the results across different metrics and asks for further analysis. There are also remarks about inconsistent conclusions, weak baselines, and the need for more recent models and few-shot learning strategies.

In terms of the contributions, the human-annotated dialogue dataset for commonsense inference task difficulty classification is considered valuable. The framing of the research question is also seen as interesting. The contrastive learning approach shows consistent improvements.

Overall, while there are some weaknesses and areas for improvement, the paper provides sufficient support for its major claims. The reviewers have mixed levels of excitement, with some suggesting further revisions. But none of them explicitly object to accepting the paper if co-reviewers support it.

In summary, the paper addresses an important research topic, presents a valuable dataset, and proposes a promising method to improve language model performance in dialogue-based tasks. Some revisions and clarifications are recommended to address the concerns raised by the reviewers.
Decision: Accept-Main