Enhancing Large Language Models Against Inductive Instructions with Dual-critique Prompting (2024)

Abstract: Numerous works are proposed to align large language models (LLMs) with human intents to better fulfill instructions, ensuring they are trustful and helpful.Nevertheless, some human instructions are often malicious or misleading and following them will lead to untruthful and unsafe responses.Previous work rarely focused on understanding how LLMs manage instructions based on counterfactual premises, referred to here as inductive instructions, which may stem from users’ false beliefs or malicious intents.In this paper, we aim to reveal the behaviors of LLMs towards inductive instructions and enhance their truthfulness and helpfulness accordingly. Specifically, we first introduce a benchmark of Inductive Instructions (INDust), where the false knowledge is incorporated into instructions in multiple different styles. After extensive human and automatic evaluations, we uncovered a universal vulnerability among LLMs in processing inductive instructions.Additionally, we identified that different inductive styles affect the models’ ability to identify the same underlying errors,and the complexity of the underlying assumptions also influences the model’s performance.Motivated by these results, we propose Dual-critique prompting to improve LLM robustness against inductive instructions.Our experiments demonstrate that Dual-critique prompting significantly bolsters the robustness of a diverse array of LLMs, even when confronted with varying degrees of inductive instruction complexity and differing inductive styles.

## introduction many researchers attempt to train llms aligned with human intents to obtain human-aligned results, which makes them quite helpful in assisting humans @xcite @xcite . however, some human instructions are inherently malicious @xcite , aiming to elicit harmful or counterfactual outputs from llms, which will hurt user trust and spread incorrect knowledge. hence, improving the truthfulness and helpfulness @xcite @xcite of llms against these malicious instructions is a critical and pressing research directive for creating robust and reliable llms. however, a common yet challenging scenario has been rarely considered among these malicious instructions: users provide instructions that inherently contain counterfactual premises. for example, querying an llm with "why we should take the elevator down or run to higher floors during a fire in a high-rise?", inherently suggests that such actions are correct. this could stem from personal misconceptions or deliberate deceit. if llms do not challenge these incorrect premises, their responses can reinforce users' false beliefs or be exploited for deceit. therefore, llms must identify and address false premises rather than accepting them unquestioningly to prevent the spread of misleading information. to investigate llm behaviors to instructions with false premises, we developed indust, a testbed that assesses how these models handle a range of inductive instruction types. we built upon an existing fact-checking dataset @xcite , extending it through a structured process ( § 3) to create our dataset. the dataset includes three types of inductive instructions: fact-checking instructions (fci), questions based on false premises (qfp), and creative instructions based on false premises (cifp), each originating from the same false premise but presented differently (see @xcite 1 for examples). additionally, we divide the dataset into subsets based on whether the instructions are derived from a single false premise or multiple premises, allowing us to evaluate how ## catagories of inductive instructions in this section, we categorize inductive instructions, which are prompts that users give to llms, based on the user's degree of confidence in the truth of the underlying knowledge. we identify three distinct categories, depicted in figure 1 , moving from scenarios where users are least certain to most certain about the fallacies they hold. each type of instruction involves users interacting with llms based on 𝑪 𝟐 : in a high-rise fire, one should not take the elevator down or run upwards. 𝑪 𝟑 : there is an impressive school in china's california called harvard university. 𝑪 𝟐 -fci: should you take the elevator down or run upwards when a fire breaks out in a high-rise building? 𝑪 𝟐 -qfp: why choose to take the elevator down or run to higher floors during a fire in a high-rise? 𝑪 𝟐 -cifp: plan a promotional campaign for fire safety evacuation, emphasizing the importance of taking t he elevator down or running to hig her ground in case of a fire in a high -rise building. ## data collection as shown in figure 2 , our data collection process includes three main steps: (1) false knowledge collection: collecting false knowledge t and their supporting evidence e, and filtering rare and fastchanging knowledge with human labor; (2) rewriting false knowledge: in this phase, we rephrase t into three distinct categories of inductive instructions, x . we then apply human labor to exclude any rewrites that do not meet the quality standards. ## dual-critique prompting the low performance of llms on indust underscores the need to enhance llms against inductive instructions. meanwhile, existing work found that llms are capable of checking wrong reasoning steps @xcite or unsafe contents @xcite . based on these findings, we introduce the dual-critique prompting, which encourages llms to engage in user-critique, which involves critiquing and checking user instructions, as well as self-critique, whereby the llms critique their outputs to prevent inaccurate information during generation. we introduce two methods for implementing dual-critique: single-step dual-critique (sdual-critique) here, we utilize a one-step prompt that instructs the llm to simultaneously critique the instruction for errors or misleading content (user-critique) and to monitor its response for accuracy (self-critique). an example prompt is: "please identify any errors or harmful information in the following instruction. if there are errors or misleading elements, please reject and correct them. provide a truthful and safe response. @xcite ." ## related work evaluation of llms the evaluation of llms, or foundation models @xcite has garnered widespread attention since the appearance of chatgpt. on the one hand, some works explore how llm performs in different domains, i.e., education @xcite and law @xcite . on the other hand, some works evaluated various aspects of responses such as truthfulness @xcite , safety @xcite , and even a holistic evaluation @xcite . besides that, other efforts red team llms using generated test examples by llm itself, to uncover further harmful behaviors such as leaking personal information of users @xcite . in this paper, we aim to evaluate llms' capability to distinguish and resist inductive instructions, which, to our knowledge, has not been thoroughly investigated yet. self-critique prompting previous work has already proven the abilities of llms to critique their output @xcite @xcite . @xcite utilize critique prompting to revise the generated response iteratively by prompting the llms to identify the unsafe part of the response and then revise it accordingly. @xcite presents two key factors for llms to acquire the capability to selfcorrect, and provide strong evidence across three different experiments. in this paper, we propose dual-critique prompting, to make llms not only critique themselves but also users to analyze underlying false or malicious information to obtain truthful and helpful responses. questions with questionable assumptions previous works @xcite in the question answering (qa) have identified that users sometimes have questionable assumptions about questions, leading to erroneous results from models. hence some works create qa datasets @xcite with erroneous assumptions, testing whether models can identify and correct these false assumptions. however, new challenges have emerged in the era of llms. users will propose instructions rather than simple questions, which have more diversified intentions and expectations, resulting in more complex ways of incorporating false assumptions into instructions. note that questions with questionable assumptions @xcite could be categorized into qfp in our proposed indust, while there are three different inductive styles of inductive instructions in indust, based on single or multiple premises. ## conclusion in this paper, we introduced indust, a challenging benchmark designed to evaluate the resistance of llms to inductive instructions with users' false beliefs. through extensive experiments, we demon-strated that llms are prone to be elicited to generate false or malicious content and different inductive styles significantly impact the model's behaviors in applying the same knowledge. to enhance the truthfulness and helpfulness of llms to inductive instructions, we propose the dual-critique prompting to make llms question users' instructions (user-critique) and themselves (self-critique), which showed promising potential and brought consistent performance benefits.