Red Teaming Language Models for Processing Contradictory Dialogues (2024)

Abstract: Most language models currently available are prone to self-contradiction during dialogues. To mitigate this issue, this study explores a novel contradictory dialogue processing task that aims to detect and modify contradictory statements in a conversation. This task is inspired by research on context faithfulness and dialogue comprehension, which have demonstrated that the detection and understanding of contradictions often necessitate detailed explanations. We develop a dataset comprising contradictory dialogues, in which one side of the conversation contradicts itself. Each dialogue is accompanied by an explanatory label that highlights the location and details of the contradiction. With this dataset, we present a Red Teaming framework for contradictory dialogue processing. The framework detects and attempts to explain the dialogue, then modifies the existing contradictory content using the explanation. Our experiments demonstrate that the framework improves the ability to detect contradictory dialogues and provides valid explanations. Additionally, it showcases distinct capabilities for modifying such dialogues. Our study highlights the importance of the logical inconsistency problem in conversational AI.

## introduction dialogue systems have made significant advancements in recent years @xcite , propelled by the rapid development of language modeling and learning technologies. the focus on understanding and analyzing conversations between humans and machines has become paramount in this field @xcite , serving as a vital component in the development of intelligent interaction systems. the emergence of large language models @xcite @xcite , such as chatgpt, has played a substantial role in shaping dialoguerelated research. these models have showcased impressive abilities in comprehending sophisticated context and generating fluent dialogue responses, and exhibit an exceptional level of control by performing summarization @xcite , explanation @xcite , enquiries @xcite , and role-playing @xcite within dialogues. however, semantic conflicts in the dialogue, such as contradictions @xcite and factual errors @xcite , pose challenges for language models (lms) in recognizing and resolving them, resulting in significantly declined experience of human-machine interaction. according logical studies @xcite , a contradiction refers to a situation where two or more statements cannot be simultaneously true. these discrepancies are primarily instigated by machines rather than humans @xcite , and even the llm sometimes fails to provide satisfactory responses foot_1 @xcite @xcite . this is demonstrated by the model's fluctuating attitudes and inconsistent faithfulness to the same issue throughout the dialogue. for example, in fig. 1 , the dialogue agent initially stated an inability to consume spicy food, but later claimed a habit of eating spicy food every day. this inconsistency between utterances loses the anthropomorphic essence of human-computer conversation. similarly, if a conversational participant conveys inconsistent intents or thoughts, they can also cause confusion in dialogue understanding. this type of controversy, observed for a single protagonist and reflected as factual conflicts or changes in attitude, is known as the self-contradiction. contradiction is a key factor in enhancing comprehension of text and has been extensively studied in q: are there contradictions in the above dialogue? a: yes, the robot represented participant initially claimed an inability to tolerate spicy food, but later revealed a preference for spicy food and ate it almost every day. really? i find spicy food to be a perfect balance of flavor and heat. they all seem to be very spicy. are you a fan of spicy food? ## task and dataset we hereby define the task of dialogue contradiction resolution, and introduce the contributed dataset. ## experiments to evaluate the proposed red teaming framework for processing contradictory dialogues, we conduct several comparative experiments on the provided dataset. specifically, in addition to assessing contradictory dialogue detection and explanation, we also evaluate the impact of lm-generated explanations on the final task ( §4.2- §4.4). moreover, we provide case studies to demonstrate the generative outcome of the lms. ## related work red teaming of language models. due to the increasing demand on responsible llms, red teaming has attracted much attention recently. red teaming is aimed to complement manual reviews and help reduce lapses by automating the process of detecting where lms are inappropriate @xcite . recent red teaming studies @xcite @xcite focus on exposing the limitations of the model and inducing unwanted content from the lm by crafting prompts. this approach can work as a human-in-the-loop or an lm that is testing the output of another lm. for example, @xcite instructed llms to role-play as malicious characters, and @xcite use llm prompting to attack different generated text detectors. while previous work has primarily used red teaming for harmful text triggering or detection, we choose to apply it to the processing of contradictory dialogues due to its efficiency in llm-interaction. however, our work focuses on contradictions from the defense standpoint and strives to minimize their occurrence in lms. ## conclusion we propose a new task for contradictory dialogue processing, seeking to detect and modify contradictions in dialogues. to facilitate research in this area, we develop a dataset of over contradictory 12,000 dialogues, including 6,000 dialogues that feature one-sided self-contradiction and their corresponding explanation labels. additionally, we propose a red teaming framework, in which we fine-tune the llama series, vicuna, and mistral to enhance dialogue inconsistency detection. explanation and modification selection modes are integrated to enhance performance. experimental results demonstrate that the framework performs well in detecting and explaining contradictory dialogues, and can effectively modify them.