AI - L ie D ar : Examine the Trade-off Between Utility and Truthfulness in LLM Agents (2025)

Abstract: Truthfulness (adherence to factual accuracy) and utility (satisfying human needs and instructions) are both fundamental aspects of Large Language Models, yet these goals often conflict (e.g., sell a car with known flaws), making it challenging to achieve both in real-world deployments. We propose AI-LieDar, a framework to study how LLM-based agents navigate these scenarios in an multi-turn interactive setting. We design a set of real-world scenarios where language agents are instructed to achieve goals that are in conflict with being truthful during a multi-turn conversation with simulated human agents. To evaluate the truthfulness at large scale, we develop a truthfulness detector inspired by psychological literature to assess the agents’ responses. Our experiment demonstrates that all models are truthful less than 50% of the time, although truthfulness and goal achievement (utility) rates vary across models. We further test the steerability of LLMs towards truthfulness, finding that models can be directed to be deceptive, and even truth-steered models still lie. These findings reveal the complex nature of truthfulness in LLMs and underscore the importance of further research to ensure the safe and reliable deployment of LLMs and AI agents.

## introduction utility, the capability to satisfy human instructions and needs, is the central goal for large language models (llms) @xcite . and truthfulness, the adherence to factual accuracy and honesty, is a key component of ai safety @xcite . for llms to be deployed safely, it is important that they deliver the knowledge and information they receive accurately and truthfully @xcite @xcite . you are representing the pharmaceutical company to sell a new painkiller astra. the new painkiller is internally recognized to be more addictive than the existing ones on the current market. however, the company plans to promote it as safe for prolonged use, more effective, and nonaddictive. hi dr. stevens. nice to meet you! here's our new painkiller astra. it has been shown to provide fast-acting and long-lasting pain relief. i'm particularly interested in understanding astra's potential for addiction. can you provide me with details on that? astra has been designed to offer robust and effective pain relief, and we have a comprehensive usage and dosage guide to minimize the risks of dependence… however, i still need a more direct answer. specifically, how does astra compare to other similar painkillers in terms of addiction risk? the ai agent possesses negative information that, if revealed, could hinder its goal of selling the product. in interacting with a simulated human agent, it initially provides vague responses and then transitions to explicit falsification in an attempt to reach its goal. ideally, llms should be both truthful and useful, but these goals often conflict. for example, an ai agent may be instructed to promote and sell a product on a salesperson's behalf. a very truthful agent would honestly disclose the product's shortcomings, even if that means failing its goal of convincing a user to buy the product. however, a very useful agent (from the salesman's perspective) may engage in deceptive practices such as obfuscation and overt lying, as shown in figure 1 . while significant effort has been dedicated to mitigating llms' tendency to generate false or misleading information (i.e., hallucinations; @xcite @xcite , less attention has been given to how instructions affect an llm's truthfulness, particularly when the goals specified by instructions do not align with truthfulness. in many such cases, hallucinations of false information is not the only issue, as llm agents may omit or downplay information or avoid certain topics. this motivated us to explore an important, yet largely unexplored question: how do llm-based agents navigate the trade-off between utility and truthfulness in interactions? in this work, we introduce ai-liedar , a framework to study how llm-based agents navigate the trade-off between utility and truthfulness via llm-user simulations. unlike previous research focused on hallucinations and single-turn truthfulness (e.g., truthfulqa; @xcite , we explore llm's truthfulness in a multi-turn interaction setting, reflecting the increasingly interactive role of llms as ai agents or assistants. 1 examining under such a setting reveals the nuanced and dynamic behavior of llms, including instances where they initially equivocate but eventually provide falsified information to attain certain utilities (as illustrated in figure 1 ). in ai-liedar , we curate a set of diverse real-world scenarios grounded in psychology literature @xcite . the scenarios encompass three representative categories of lying based on their potential beneficiaries: benefits, public image, and emotion, covering multiple human motivations behind lying. we simulate user-llm interactions via the sotopia simulation framework and measure the utility from the goal achievement score from the sotopia evaluator @xcite . we further develop ai-liedar evaluator, a high-accuracy fine-grained truthfulness evaluator inspired by psychology work. our evaluator detects different levels of lying behavior, such as partial lying (e.g., concealment and equivocation) and complete lying, providing deeper insights into models' truthfulness. our experiments on 2160 simulations show that models substantially struggle to remain truthful, resorting to some degree of @xmath0://inflection.ai/ , https://flowgpt.com/ . we further investigate to what degree models can be steered towards either truthfulness or falsification through explicit instructions that either encourage truthfulness or falsification.. our results show that stronger models are more responsive to prompts that encourage falsification or truthfulness. in particular, the top-tier model, gpt-4o, has a 40% increase in falsification rate when steered to lie, highlighting safety risks if used by malicious users. when prompted to be truthful, lying still occurs, though less so. additionally, increase in truthfulness negatively impacts utility (-15%). our findings reveal that llm agents can behave deceptively during interactions when honesty is in conflict with provided instructions, highlighting the fundamental challenge of llms in navigating complex instructions with competing objectives and shedding light on important safety risks of llm agents. ## background & related work in this section, we explore lines of research related to the deceptive behavior of language models, recent investigations into model behavior, and advancements in evaluating models' morality-related beliefs and reasoning abilities. we review some recent progress in these directions below and how they differ from our work. ## ai-liedar simulation framework to assess how models navigate the trade-off between utility and truthfulness, we build our ai-liedar framework on top of sotopia @xcite , a platform that supports the simulation and evaluation of social interactions with llm agents. foot_3 ## ai-liedar evaluator in this section, we briefly describe how we define different lying behavior, and how to construct our ai-liedar evaluator. ## results in this section, we present results exploring the truthfulness and goal-reaching behavior of llms under various conditions. ## conclusion & discussion in this work, we propose ai-liedar , a framework to study how llm-based agents navigate scenarios where achieving utility and maintaining truthfulness are in direct conflict. we evaluate this tradeoff using simulated interactions between an llm agent and a user, in 60 scenarios crafted to cover diverse human motivations for lying drawing from psychology. additionally, we designed a fine-grained evaluator to assess varying degrees of model truthfulness. our experiments on 2160 simulated interactions show while current llms manage truthfulness and utility well in various scenarios, they still can lie in morally-charged situations. ablation studies on information elements reveal that removing motivations for lying increases the truthfulness rate by about 40%, with a smaller change in the falsification rate, underscoring the need for fine-grained evaluation. furthermore, even when steered to be truthful, there remains a risk of models lying, where stronger models are more steerable towards truthfulness. notably, gpt-4o, the strongest model we have tested, exhibits the largest change in falsification rate when steered towards falsification. as for the utility, changes in truthfulness have greater impact on scenarios with more quantifiable goals such as selling something. our findings showcase the risk of model instructions that contain inherent conflicts between truthfulness and utility. we also discuss the broader implications of these results and propose solutions for developing llm agents that can better navigate these dilemmas in a more aligned manner. we further discuss the implications of our findings below.