C ontra D oc: Understanding Self-Contradictions in Documents with Large Language Models (2024)

Abstract: In recent times, large language models (LLMs) have shown impressive performance on various document-level tasks such as document classification, summarization, and question-answering. However, research on understanding their capabilities on the task of self-contradictions in long documents has been very limited. In this work, we introduce ContraDoc, the first human-annotated dataset to study self-contradictions in long documents across multiple domains, varying document lengths, self-contradiction types, and appearance scope. We then analyze the current capabilities of four state-of-the-art open-source and commercially available LLMs: GPT3.5, GPT4, PaLM2, and LLaMAv2 on this dataset. While GPT4 performs the best and can outperform humans on this task, we find that it is still unreliable and struggles with self-contradictions that require more nuance and context. We release the dataset and all the code associated with the experiments.

## introduction detecting contradictions in texts has long been pivotal in natural language understanding(nlu), with most of the works falling under the umbrella of natural language inference(nli) @xcite @xcite . detecting contradictions is often regarded as determining the relation between a hypothesis and a piece of premise. however, understanding contradictions when they occur within the confines of a single text (self-contradictions), and furthermore, doing so holistically at the document-level, is still under-explored @xcite . a text is considered self-contradictory when it contains multiple ideas or statements that inherently conflict. this could manifest in various ways, such as the existence of logical paradoxes, antithetical assertions, or inconsistent descriptions. figure 1 shows an example of self-contradiction in a document. the highlighted two sentences provide contradictory information about the number of patients, thus resulting in a self-contradictory document. psychological research @xcite indicates that humans struggle to identify contradictions in unfamiliar, informative texts, particularly when contradictions are widely separated in long documents, underscoring the need for automated text analysis tools to tackle this challenge. previous research on document-level contradictions either focused on sentence-document pair nli @xcite or has been restricted to a single type of document @xcite . @xcite defined self-contradiction detection as a binary classification task, proving inadequate for accurately evaluating since they do not require locating selfcontradictions within texts. to further explore the study in this domain, we propose a new document-level self-contradictory dataset contradoc with the following characteristics: to create contradoc, we utilize a humanmachine collaborative framework. we first use llms and nlp pipelines to automatically create and introduce self-contradiction into a consistent document. then, human annotators verify and label attributes for the self-contradictory documents, ensuring the quality and utility of our dataset. the advent of large language models (llms) pre-trained on extensive context lengths @xcite has shown promising results over various document-level tasks spanning document classification @xcite , document summarization @xcite , document-level question answering @xcite , and document-level machine translation @xcite . yet, we argue that llms' abilities to handle tasks with long context are inconsistent, given their significant dependence on the specific characteristics of the task. to investigate how well can large language models detect self-contradiction in documents, we evaluate state-of-the-art, open-source and commercially available llms: gpt3.5(openai, 2022), gpt4(openai, 2023), palm2 @xcite , and llamav2 @xcite on con-tradoc. we design three evaluation tasks and corresponding metrics to assess llms' zero-shot performance. in our experiments, we find that even sota models cannot achieve applicable performance. we did a thorough study on the effects of different aspects of documents and selfcontradictions and found that llms can detect object self-contradictions(e.g., facts) much better than subject self-contradictions(e.g., emotion or perspective). in summary, this paper makes the following contributions: ## contradoc dataset contradoc contains 449 self-contradictory (referred to as contradoc-pos) and 442 non-contradictory documents (referred to as contradoc-neg). non-contradictory documents are defined as documents that do not contain any self-contradictions and are considered negative examples for the task. we include them in our dataset to evaluate if the models can identify the documents that do not contain any selfcontradictions sampled from the same source of contradictory documents. furthermore, the documents in contradoc cover three domains, vary in length and scope of dependencies, and contain different types of contradictions. this allows us to see how these variations affect the performance of the llms. in the development of our dataset, we leverage a human-machine collaborative framework, where human experts evaluate and verify machine-generated self-contradictions, ensuring the created data is both rich and reliable. we only use documents written in english in this work. ## additional sensitivity analysis effect of prompts since we enforce model outputs to a fixed format, this might negatively affect the model performance. this is more true for gpt3.5 than gpt4, which has better instructionfollowing capability. thus, for 75 documents with self-contradictions, we ask gpt3.5 to generate predictions without putting constraints on the output format (prompt in appendix d) and ask humans to evaluate the responses. for 26.4% cases, it answers "no"; for 45.8% of the cases, it provides incorrect evidence; only for 27.8% of the cases is it able to find the correct evidence (alongside other incorrect evidence). this suggests that the model performance is still far from satisfactory. figure 4 in the appendix shows the gpt-3.5 outputs for the two cases. detecting self-contradictory sentence since we observe that models find it hard to find contradictions in a document, we evaluate the model's capability on an easier task to find a statement that directly contradicts a given sentence. since our dataset contains documents that contain a pair of contradictory sentences, we provide the evidence sentence to the model and ask it to find the contradictory sentence in the document. gpt3.5 can detect 51.6% of the cases, while gpt4 can detect 77.2% of them. such results suggest that llms do reasonably well in document-level contradiction detection if the exact sentence with contradiction is pointed out but not so otherwise, but perform much worse in finding self-contradiction if the exact sentence isn't pointed out for its reference. ## conclusion in this work, we present one of the first steps in investigating the task of document-level selfcontradictions. we create contradoc, a wellannotated dataset for this task, which contains 449 self-contradictory documents spanning over three domains and containing multiple types of self-contradictions. the dataset is annotated by humans and contains information about the scope and type of self-contradiction as well as the evidence to detect self-contradictions. we then investigate the capabilities of four state-of-the-art llms, namely, gpt3.5, gpt4, palm2, and llamav2, on this dataset. we find that overall, gpt4 performs the best and even outperforms humans on the task. however, we also find that there is still a long way to go before gpt4 can reliably detect self-contradictions. we release this dataset and all the associated code for the community to use and develop better document-level reasoning capabilities in llms. as part of future work, we plan to investigate the capabilities of llms to fix the self-contradictions in the documents.