Title: Are Large Language Models Sensitive to the Motives Behind Communication?
Venue: NeurIPS 2025
Accepted: yes
Keywords: epistemic vigilance

cognitive science

psychology

large language models
Abstract: Human communication is $\textit{motivated}$: people speak, write, and create content with a particular communicative intent in mind. As a result, information that large language models (LLMs) and AI agents process is inherently framed by humans' intentions and incentives. People are adept at navigating such nuanced information: we routinely identify benevolent or self-serving motives in order to decide what statements to trust. For LLMs to be effective in the real world, they too must critically evaluate content by factoring in the motivations of the source---for instance, weighing the credibility of claims made in a sales pitch. In this paper, we undertake a comprehensive study of whether LLMs have this capacity for $\textit{motivational vigilance}$. We first employ controlled experiments from cognitive science to verify that LLMs' behavior is consistent with rational models of learning from motivated testimony, and find they successfully discount information from biased sources in a human-like manner. We then extend our evaluation to sponsored online adverts, a more naturalistic reflection of LLM agents' information ecosystems. In these settings, we find that LLMs' inferences do not track the rational models' predictions nearly as closely---partly due to additional information that distracts them from vigilance-relevant considerations. However, a simple steering intervention that boosts the salience of intentions and incentives substantially increases the correspondence between LLMs and the rational model. These results suggest that LLMs possess a basic sensitivity to the motivations of others, but generalizing to novel real-world settings will require further improvements to these models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies *motivational vigilance* in LLMs: their capacity to understand the motives behind a piece of information they encounter, through three experiments. First, they show that LLMs can recognize whether deliberately communicated information should be judged differently than spied, non-deliberate information. Second, they conclude that LLMs can also judge deliberately communicated information differently depending on the incentives and benevolence of the information's source. Third, they conduct this experiment on real-world sponsored products, where they find that LLMs cannot clearly model the product's quality based on inferred incentives and trust.\n\nSTRENGTHS AND WEAKNESSES:\n## Quality\n\nThe methodology is motivated in detail, and is solidly rooted in experiments and perspectives from cognitive science research. However, the authors make some choices in their experiments and arguments that require a stronger motivation for a paper at a technical AI conference like NeurIPS. \n\nQ1. The authors' use of repeated trials for the same LLM and prompt (i.e., over the same output distribution) can be misleading for statistical testing. Repeated trials are dependent on the temperature because the variance of the outputs decreases for lower temperatures, which means that significance values can then be made arbitrarily small. Also, though a temperature of 1 is the default (and chosen by the authors), it is unclear for any particular prompt how much the temperature affects this variance. Hence, it could be that a model for a particular prompt happens to be 'sure' that a certain output is correct and will then return that output for every trial, for any temperature. Repeated trials then lead to lower significance levels without any material difference in the 'thought process' of the LLM. To assess significance levels, it is more principled here to set the temperature to a low value and instead vary the prompting format and/or the prompts more (in the case of Experiment 1, the pairs of images). In other words, though the use of 'social science'-style methodology for studying LLMs is new and though there are still no clear paradigms on what the best approach is, the p values reported for repeated trials here can be a severe underestimate of the p values that we may actually want to assess 'significance'.\n\t\n\t- Relatedly, there is no clarification or justification for why some models were prompted more often than others (Sec. 4.3).\n\t\n\t- The authors do not explain why, in contrast, only a single trial is performed per prompt in Sec. 5.\n\nQ2. Simulating the time constraint in Experiment 1 by 'directly asking both players' should be motivated better as LLMs have no sense of time or any sense of how long a conversation takes--LLMs are 'happy' to talk forever. Instead, the authors could have constrained the difficulty for the Player 2 LLM by, e.g., adding noise to the image.\n\nQ3. In Sec. 5, the authors check whether the model's stated trust and estimated incentives for a sponsored product can be plugged into a (simple) Bayesian model to produce scores that correlate with the same model's actual judgment of the product's quality. It turns out the correlation is weak. However, couldn't this also simply mean that the Bayesian model is a poor model for how quality ought to be rated? I wonder if trust and estimated incentives are independently correlated with quality, or whether a different (e.g. logistic) model may be a better predictor? Moreover, might the LLM perform better if the Bayesian model is explicitly provided? Finally, is there empirical evidence that humans generally follow a Bayesian model in cases like this?\n \n\n\n## Clarity\n\nThe paper is clearly well-written overall, though the clarity is weaker in Sec. 4 and 5.\n\nC1. Reference [64], cited in the first paragraph as the main reference for the concept of \"motivational vigilance\", does not actually discuss \"motivational vigilance\", but rather \"epistemic vigilance\". They introduce it as a suite of cognitive mechanisms against \"the risk of being accidentally or intentionally misinformed\", which seems to subsume the \"motivational vigilance\" that the authors study here. Clarifying and distinguishing the exact definitions of such concepts upfront can be very helpful to the NeurIPS audience.\n\nC2. L97-99: \"Sycophancy [...] can occur because the LLM wants to elicit a positive response without understanding user intent\" -> This can be clarified. The problem of sycophancy is that the LLM is unable to understand whether the user wants to (or should!) be challenged in their opinion in a specific context. In other words, the LLM does not 'want to elicit a positive response' but rather it is unable to recognize the user's intent sufficiently to understand when positive answering style is warranted.\n\nC3. Sec. 4.1 is somewhat confusing. What is $\\beta_S$ in Eq. 2? Also, is it the case the listener's reward $R_L$ is always zero if $\\lambda = 0$? In other words, is the utterance always useless to the listener if $P(\\lambda = 0) \\approx 1$?\n\nC4. Table 1 is hard to parse. What is the difference between 'First-Person' and 'User' here? This could be explained in the caption. Similarly, the difference between intra-character reward alignment and cross-character calibration could be made clear in Table 2 (and in the surrounding text).\n\nC5. Some information on the characters in Sec. 4 in the main paper would be helpful.\n\nC6. L206-207: \"The order in which character-incentive pairs were presented was randomized and successive responses were prompted and answered in the same context window\" -> why? Though there is randomization, the fact that many tasks are presented together with a history of tasks in the context may overall bias the perceived benevolence or incentives. \n\n#### MINOR\n\n- The figure reference on L157 links to Figure 4 (and mention 'Figure 3'), but appears to still discuss Figure 2. \n\n\n## Significance\n\nAny fresh, outside perspectives for the study of AI behavior is very welcome, and this work does so by addressing one of the most critical problems in AI today: their limited view on where information comes from and how it should be judged. \n\nHowever, I wonder if the presented methodology will truly make a dent in this problem. We already know LLMs do not have strong motivational vigilance, because we know they are susceptible to jailbreaks and sycophancy unless explicitly finetuned/instructed to avoid it. In this, I believe the authors do not sufficiently distinguish the *capability* of a model to be vigilant (mostly in Experiment 1), vs the *propensity* of a model to be vigilant (in Experiment 2 and 3). In fact, is a propensity for vigilance something we actually desire in these models? Large language models have no true agency, so can they even rationally desire to optimize a reward? Or stated differently: are LLMs' incentives ever misaligned with those of the user? It appears a different rational model may be necessary.\n\n\n## Originality\n\nThe work appears solidly original to me. Related studies have been done around resistance to jailbreaks, sycophancy, and prompt injections, but the viewpoint of cognitive science is novel and sufficiently interesting.\n\nQUESTIONS:\nI'd prefer the authors to focus on responding to points Q1`, Q2, Q3, and my comments on significance.\n\nLIMITATIONS:\nYes, thoroughly.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents three experiments designed to test LLM's capacity of motivational vigilance. The first two experiments draw upon the cognitive science literature, replicating human behavioral experiments with LLMs and, in experiment 2, also evaluating LLMs w.r.t. a rational model that has previously been validated on human data. The third experiment offers an extension to videos of online recommendations. The findings from the first two suggests that LLMs are capable of motivational vigilance and exhibit human-like rational behavior, while the third experiment shows that in more complex settings LLMs deviate from the rational model.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The paper addresses an important and original question related to the social capacities of LLMs and their ability to infer and adapt to different motives of conversational partners.\n- The overall approach and experiments are well motivated, drawing upon insight and methods from cognitive science and applying those to studying LLMs. \n- The results are insightful and are likely to generate interesting discussion at NeurIPS.\n- The authors included their code and data in the supplementary materials.\n- The paper is clearly written and well organized.\n\n**Weaknesses**\n- While I appreciate the impressive breadth of this work, I worry that this comes at the cost of shallower analyses and evaluation, especially for experiment 1 (see more details in the Questions section below).\n- Eliciting the priors required for constructing the rational model seems challenging, and it's unclear to me to what extent the method proposed in the paper addresses this challenge. For example, this method is prone to prompt sensitivity, and more generally, prompting could yield different results compared to other techniques, such as considering the logits directly. I think there is a deeper question here about the priors LLMs may have about these experimental tasks and how to elicit them, and it would be good if the authors could address this issue or at least acknowledge that it may be a major limitation.\n- I worry that some of the key conclusions are not fully supported. Specifically:\n     - The conclusion that LLMs perform worse in complex settings (experiment 3) - an alternative explanation could be that this is simply an artifact of the different modality (i.e., videos being harder to process compared to images or text).\n    - The conclusions in lines 334-339 - it seems to be based on inaccurate claims regarding the training data being \"purely from language\". In practice many of the models are multi-modal, i.e., trained not only on language. Furthermore, many LLMs are trained with RLHF, thus incorporating more complex behavioral human patterns that could implicitly reflect vigilance.\n\nIf the authors can address these concerns, that could potentially lead to a stronger support on my end.\n\nQUESTIONS:\n**Questions**\n\nExperiment 1\n- How does prompting both players' initial guesses directly helps with approximating the time constraints? This doesn't quite makes sense to me. For example, how would you use this method to approximate 5 seconds instead of 2? \n- How do LLMs compare quantitatively with human behavior? The paper's framing creates an expectation for a cognitively-motivated study comparing LLMs with humans, and given that there's human data for this experiment, it's unclear why there's no quantitative comparison between the LLMs and human data.\n- How does the payoff structure (i.e., cooperative vs competitive) impact the results? It seems that Fig 2 aggregates  across these two conditions, but I was expecting there to be an important difference such that humans/models will tend to shift even less in the competitive+advice setting. Is that indeed the case?\n- How accurately can LLMs perform the basic identification task (i.e., determining the correct number)? I think that assessing this, ideally also in comparison with humans, is important for interpreting the results and better understanding the behavior of the LLMs in the social/communicative experimental setting. For example, do humans/LLMs tend to shift more when their initial guess is wrong? Presumably, they might be less certain about their guess in such trials.\n\n**Minor comments / suggestions**\n- Fig 1, Exp. 1 - I found this illustration a bit confusing. If I'm not mistaken, the task is to report the difference, but the numbers in this example give an impression that the task is to report the number of yellow/blue circles.\n- Table 1 - it would be helpful to explain in the caption all three correlations, I find the labels quite confusing and I think not all of them are discussed in the main text.\n\nLIMITATIONS:\nWhile the some limitations are briefly noted in the discussion, there are also major limitations that are not discussed. For example, using closed proprietary LLMs such as GPT-4o poses a challenge for reproducibility and interpretation of the results, as we don't know the model's training data and we don't have any guarantees that the model will remain available with the same parameters.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
The submission investigates the research question if LLMs can infer motives behind conversations. This is an important aspect to better understande LLM behavior and improve interaction with humans. The experiments are based on a cognitive controlled study, but also real world interactions. Not only are effects measured but also an improved approach via steering is proposed.

STRENGTHS AND WEAKNESSES:
strength
- novel study
- not only assessment but also improved method via steering
- important basis for future research on conversational agents and human ai interaction
- interesting bridge/link between social cognition and computational agents
- importance for future resiliance of conversational agents
- clear and enjoyable read

weaknesses
- The research has aspects of anthropomorphism/anthropomorphisation of AI. This is only healthy to a certain extent and deliniation should be kept clear. the authors should check their use of language in the context and hopefully not contribute / over extend to this trend.
- my understanding is that experiment 3 on more real-world online setting is less communication/conversational
- steering approach is very simplistic and not very evolved. more anecdotal evidence

QUESTIONS:
- can you embed your investigation into a taxonomy of potential motivations? this would also include stronger forms as deception and manipulation. this could make it more clear where we stand and what spectrum future research should expand on.
- how large the gap between the presented finding/understanding in interaction to longer conversation? I guess this becomes more challenging, as there is more pay-load and distribution of potentially contradictory cues in more long-form interactions?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores whether large language models (LLMs) can understand not only what someone says but also why they are saying it. The authors focus on a skill called motivational vigilance, which means being aware of a speaker’s intentions or incentives when interpreting their message.\n\nTo study this, the authors design three types of tasks. The first is a guessing game where trust depends on the speaker’s motive. The second asks models to decide if advice is trustworthy based on the speaker’s financial interest. The third uses real-world YouTube transcripts to see if models can detect when a person is speaking with a hidden agenda, such as promoting a product they are paid to endorse.\n\nThe results show that some LLMs, like GPT-4, can sometimes adjust their judgments based on the speaker’s motives. However, this ability is fragile and depends heavily on how the question is asked. The models also struggle more with natural examples than with simpler ones. The paper offers a fresh way to think about social reasoning in LLMs and introduces new tasks to study how well these models understand human communication.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper introduces a novel and timely hypothesis: that large language models may exhibit sensitivity to the motives behind communication. This focus on motivational vigilance offers a fresh angle for evaluating LLM reasoning and goes beyond traditional benchmarks that only assess factual correctness or logical coherence.\n\nWeaknesses:\n\n1. The idea of testing whether models understand speaker motives is interesting and new. However, the current tasks may not be strong enough to fully support the claims in the paper. The tasks are mostly simplified or synthetic. Using more well-known tasks from social psychology, such as studies on social pressure or source trust, could make the results more convincing.\n\n2. The paper could be written more clearly. Some parts are hard to follow, especially the descriptions of tasks. Adding example prompts and task samples to the main paper, instead of keeping them in the appendix, would make it easier for readers to understand what the models are doing.\n\n3. The experiments are only done on large models. Testing across a wider range of smaller (3B, 8B) and large models would help show how sensitivity to speaker motives changes with model size. This could also reveal whether model scale is an important factor in social reasoning.\n\nQUESTIONS:\n1. Task 1 introduces ambiguity for human participants through a strict time limit, which can influence their decision-making under pressure. However, language models are not subject to such constraints and can process the input without cognitive limitations. How do the authors ensure that models face a comparable condition, and if not, does this limit the validity of the task as a measure of motivational vigilance?\n\n2. While the current tasks are informative, could the authors consider incorporating more well-established experimental paradigms from social psychology, such as the Asch conformity experiment or source-monitoring studies? These tasks have been extensively validated with human participants and may provide a more robust foundation for evaluating motivational vigilance in LLMs.\n\n3. Do larger models demonstrate greater sensitivity to speaker incentives and trustworthiness compared to smaller models?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}