Boosting LLM Agents with Recursive Contemplation for Effective Deception Handling (2024)

Abstract: Recent advances in large language models (LLMs) have led to significant success in using LLMs as agents. Nevertheless, a common assumption that LLMs always process honest information neglects the widespread deceptive or misleading content in human and AI-generated material. This oversight might expose LLMs to malicious manipulations. To enhance LLMs’ ability to identify and counteract deceptive information, in this paper, inspired by humans’ recursive thinking and perspective-taking, we introduce a novel cognitive framework, Recursive Contemplation (ReCon). ReCon combines formulation and refinement contemplation processes; formulation contemplation produces initial thoughts and speech, while refinement contemplation further polishes them. Additionally, we incorporate first-order and second-order perspective transitions into these processes respectively. Specifically, the first-order allows an LLM agent to infer others’ mental states, and the second-order involves understanding how others perceive the agent’s mental state. After integrating ReCon with various LLMs, extensive experiment results from the Avalon game and BigTom benchmark indicate ReCon’s efficacy in aiding LLMs to discern and maneuver around deceptive information without extra fine-tuning and data. Finally, we demonstrate ReCon’s scaling trend with model parameters, and explore the current limitations of LLMs in terms of safety and reasoning, potentially furnishing insights for subsequent research. Our project page can be found at https://shenzhi-wang.github.io/avalon_recon.

## introduction recent advancements in large language models (llms) have propelled their success in the area of llm-as-agent @xcite @xcite @xcite , among which a series of works focus on multi-agent communications @xcite @xcite @xcite , demonstrating intriguing observations and emergent cooperative behaviors. however, a typical underlying assumption in these studies is that the information processed by llms is consistently honest, devoid of deception or misinformation. this results in llms being cognitively straightforward but unprepared for deceptive contexts. in reality, human society and ai-generated content are full of deceptive or misleading content @xcite @xcite . llm agents, if unprepared to discern and manage deceptions, risk aligning with immoral or even malevolent values, making them vulnerable to malicious manipulations @xcite . for instance, assuming the use of llm agents as e-commerce assistants, facing customers' bargaining and competition from peers, misaligning the intentions of customers or competitors could lead to significant economic losses. therefore, it is imperative to equip llms with the capacity to identify and counteract deceptive inputs. in this paper, we aim to explore the potential of llms in more realistic linguistic tasks with misinformation and understand the challenges of implementing llms in deceptive contexts. we notice that, in deceptive environments, humans typically engage in thought patterns like recursive thinking @xcite and perspective-taking @xcite . inspired by these cognitive strategies, we introduce a new framework, recursive contemplation (re-con), designed to enable llms to detect and handle deceptive information. as shown in figure 1 , recon integrates two cognitive processes, namely, evil roles try to deceive good roles good roles counteract evil roles ## background in this section, we first justify testing recon on avalon and bigtom by introducing these benchmarks ( §3.1) and analyzing deception sources in their contexts (deferred to §d). furthermore, we justify our aim to enhance llms' ability to handle deceptions by discussing the related challenges for llms ( §3.2). related work can be found in §a. ## recursive contemplation to deal with the challenges in §3.2, in this section, we introduce the design of recursive contemplation (recon). for simplicity, here we explain re-con in the context of avalon. as shown in figure 1 , recon contains two key mechanisms, specifically the formulation contemplation in §4.1 and the refinement contemplation in §4.2. the pseudocode of recon can be found in §b. ## experimental evaluations we assess recon's efficacy using avalon and big-tom @xcite . besides experimental results in this section, we defer qualitative analyses to §f, more experiment results to §g, and statistical tests on our experiment results to §h. ## conclusion this work underscores the susceptibility of llms to deceptive information and introduces a novel framework, recursive contemplation (recon). inspired by humans' recursive thinking and perspective-taking, recon employs formulation and refinement contemplation processes, integrated with first-and second-order perspective transitions. after integrating recon with various llms, extensive experimental results, both quantitative and qualitative, from avalon and bigtom demonstrate recon's efficacy in enhancing llm agents' ability to discern and tackle deceptions, without the need for additional fine-tuning. moreover, we show re-con's scaling trend that the performance of recon improves as the capabilities of the employed llm increase, and further analyze some interesting observations in §e that might inspire future study.