Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verification (2024)

Abstract: Making inferences in text comprehension to understand the meaning is essential in language processing. This work studies the entailment verification (EV) problem of complex, multi-sentence premises requiring a system to make multiple inferences implicitly. Modern applications of EV in detecting inconsistent model-generated rationales require complex multi-hop reasoning. However, current textual inference datasets mostly contain short-sentence premises that partially focus on this. To address this, we compile an EV benchmark that includes datasets from three NLP domains (NLI, contextual QA, and rationales) containing multi-sentence premises. On benchmarking humans and LLMs, we find that LLMs are better than humans in multi-hop reasoning across extended contexts, while humans perform better in simple deductive reasoning tasks. We also finetune a Flan-T5 model for EV using two training objectives to obtain a strong open-source model that outperforms GPT-3.5 and rivals GPT-4. Finally, we use our finetuned model to filter out inconsistent model-generated rationales in self-consistency decoding, resulting in a 6% accuracy improvement on average across three MCQ datasets.

## introduction a prevailing notion in cognitive psychology exists that humans make numerous inferences to understand discourse and text @xcite . these inferences play a crucial role in linking information from disparate sections of a text to establish its literal meaning and are closely associated with reasoning. with the recent applications of large language models (llms) @xcite @xcite figure 1 : distinctions between human and llm inferences. examples of each reasoning type, along with the human and gpt-4 prediction, are shown. a green box means the prediction matches the true label, and red otherwise. humans are more consistent in simple deductive reasoning, whereas llms excel in complex, multi-step inferences over long contexts. both humans and llms are comparable in instances with missing knowledge. please refer to section 2 for more details. in nlp tasks that require inference skills @xcite , it is thus essential to understand and improve upon the limitations of llms concerning different aspects of language inferences. in this work, we focus on the task of entailment verification (ev) that classifies whether a given context supports a hypothesis. to assert the validity of a hypothesis, a system has to make multiple inferences from the given context and its internal knowledge, which requires complex multi-hop reasoning. 2 verifying the entailment of such complex premises has important applications in rationalegenerating llms, such as chain-of-thoughts (cot) @xcite , that typically generate multisentence rationales with incomplete information and suffer from inconsistent reasoning @xcite . while ev is similar to natural language inference (nli) @xcite , some interesting challenges distinguish it from nli. many existing textual inference datasets, such as snli @xcite , mnli @xcite , etc., mostly contain short sentence premises that only partially encapsulate the challenges of multisentence premises requiring complex reasoning. predicting the entailment of such complex premisehypothesis pairs often requires multi-hop reasoning, inferring missing information, etc., which is lacking from standard nli datasets @xcite . thus, studying ev in the context of modern llms is essential and is currently missing. to this end, we first compile an evaluation benchmark to study the ev problem by selecting multiple datasets across three categories: nli, contextual qa, and rationales. as shown in next, we evaluate llms and humans on this benchmark and make some interesting observations. cognitive studies @xcite have shown that an average human brain has a limited capacity to retain only four chunks in short-term memory, indicating a limitation of human inference abilities over long contexts. our analysis shows that llms are indeed stronger than humans at tasks that involve multi-hop reasoning across long contexts, reinforcing this human limitation. in contrast, humans are better in cases that require simple deductive reasoning using substitutions, negations, etc., indicating that current llms lack consistency along these reasoning aspects. further, humans and llms perform comparably in instances requiring inferring missing knowledge. these findings are depicted in figure 1 with motivating examples. additionally, on comparison between llms, we find that models finetuned on a specific dataset category are usually strong within the category but don't generalize well on unseen categories. in contrast, instruction-finetuned models are better on average, with the best performing model, gpt-4, achieving 0.79 macro-f1 and outperforming the best-open-sourced model flan-t5-xxl by 0.08 macro-f1 on average. in order to bridge this gap, we finetune a flan-t5 @xcite model on a training subset containing datasets from each category of the above data collection. to this end, we explore two different training approaches: a classification-based finetuning that learns to directly predict the label and a ranking-based finetuning that learns to rank the most supported hypothesis from a given pair of hypotheses for a given premise. ranking-based finetuning is better than classification, specifically in contextual qa datasets, as it can learn a softer decision boundary. overall, our fine-tuned flan-t5 outperforms gpt-3.5, baseline flan-t5, and performs comparably to gpt-4 on the benchmark, thus providing a strong open-sourced model for entailment verification. finally, we demonstrate the utility of our finetuned models on a downstream application of filtering unfaithful model-generated explanations. selfconsistency (sc) @xcite decoding samples multiple model-generated reasoning paths from the llm decoder and aggregates them to predict the most consistent answer. we use our finetuned flan-t5 to filter out non-entailed reason-ing chains before aggregating the final prediction, which leads to 6% performance improvement on average across three mcq datasets. overall, our contributions can be summarized as follows: 1. using existing resources, we develop a benchmark for entailment verification (ev) of multisentence premises for both training and evaluation purposes. in this section, we formally define the task of entailment verification (ev), the datasets used to create the evaluation benchmark, and the evaluation procedure for evaluating different llm baselines. for a given premise p and a hypothesis (or claim) h, the task of entailment verification is to determine whether the context has information that directly confirms the hypothesis or not, i.e., whether the hypothesis follows from the information present in the context. this is a binary classification task defined as f (p, h) = {support, not support}, where f is a classifier (human/llm). ## evaluation of humans and llms first, we randomly sample 100 instances from each dataset (i.e., 1000 instances in total) and conduct a human evaluation on this subset to estimate average human performance. please refer to appendix e.1 for more details on the annotation procedure. additionally, we evaluate the above llms on this sampled subset and report those numbers for fair comparisons with humans. ## training llms for entailment verification in section 3.2, we observed that open-sourced llms lack performance compared to humans and close-sourced models such as gpt-4. from section 3.1, we know that task-finetuned models perform well on a category when finetuned on data from the same category. using this insight, we finetune the flan-t5-xxl model on the train splits of datasets from each category, resulting in using anli @xcite , race @xcite , and ecqa @xcite for finetuning. please refer to appendix f.1 for more details on our training dataset selection criteria. next, we describe the finetuning approaches and our key findings. ## conclusion we studied the ev problem in the context of llms. specifically, we sourced datasets across three different categories (nli, contextual qa, and rationales) and analyzed human and llm performance on the benchmark. we found that llms are better than humans in complex reasoning, while humans are more consistent on simpler reasoning tasks. we also explored different training objectives to finetune open-sourced llms for ev. our finetuned model outperforms gpt-3.5, baseline flan-t5-xxl, and is comparable to gpt-4. finally, we applied the ev model to filter out inconsistent model-generated cots in self-consistency decoding, achieving improvements over the baseline selfconsistency approach.