Steering the Herd: A Framework for LLM-based Control of Social Learning
Authors: Raghu Arghal
Kevin He
Shirin Saeedi Bidokhti
Saswati Sarkar
Venue: ICLR 2026 Oral (2026)
Keywords: Social learning
LLMs
optimal control
information design
dynamic programming
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We introduce, analyze, and simulate (via LLMs) a model of controlled social learning to study how algorithms can influence social beliefs, decisions, and welfare via information design.

Abstract: Algorithms increasingly serve as information mediators -- from social media feeds and targeted advertising to the increasing ubiquity of LLMs. This engenders a joint process where agents combine private, algorithmically-mediated signals with observational learning from peers to arrive at decisions. To study such settings, we introduce a model of controlled sequential social learning in which an information-mediating planner (e.g., an LLM) controls the information precision of agents while they also learn from the decisions of earlier agents. The planner may seek to improve social welfare (an altruistic planner) or to induce a specific action the planner prefers (a biased planner). Our framework presents a new optimization problem for social learning that combines dynamic programming with decentralized action choices and Bayesian belief updates. In this setting, we prove the convexity of the value function and characterize the optimal policies of altruistic and biased planners, which attain desired tradeoffs between the costs they incur and the payoffs they earn from induced agent choices. The characterization reveals that the optimal planner operates in different modes depending on the range of belief values. The modes include investing the maximum allowed resource, not investing any resource, or the investment increasing or decreasing with increase in the belief. Notably, for some ranges of belief the biased planner even intentionally obfuscates the agents' signals. Even under stringent transparency constraints—information parity with individuals, no lying or cherry‑picking, and full observability—we show that information mediation can substantially shift social welfare in either direction. We complement our theory with simulations in which LLMs act as both planner and agents. Notably, the LLM-based planner in our simulations exhibits emergent strategic behavior in steering public opinion that broadly mirrors the trends predicted, though key deviations suggest the influence of non-Bayesian reasoning—consistent with the cognitive patterns of both human users and LLMs trained on human-like data. Together, we establish our framework as a tractable basis for studying the impact and regulation of LLM information mediators that corresponds to real behavior.

Published as a conference paper at ICLR 2026
STEERING THE HERD: A FRAMEWORK FOR LLM- BASED CONTROL OF SOCIAL LEARNING
Raghu Arghal, Kevin He, Shirin Saeedi Bidokhti & Saswati Sarkar University of Pennsylvania (rarghal, hekevin, saeedi, swat i (watti@uppeer Jeupenn.
ABSTRACT
Algorithms increasingly serve as information mediators- from social media feeds and targeted advertising to the increasing ubiquity of LLMs. This engenders a joint process where agents combine private, algorithmically-mediadd signals with observational learning from peers to arrive decisions. To study such set- tings, we introduce model of controlled sequential social learning in which an planner (c.g., an LLM) controls the information precision of agents while they also learn from the decisions of carlier agents. The planner may seek to improve social welfare (an altruistic planner) or to induce a specific action the planner prefers (a biased planner). Our framework presents new op- timization problem for social learning that combines dynamic programming with decentralized action choices and Bayesian belief updates. In this setting, we prove the convexity of the value function and characterize the optimal policies of altru- istic and biased planners, which attain desired tradeoff's between the costs they incur and the payoffs they carn from induced agent choices. The characteriza- tion reveals that the optimal planner operates in different modes depending on the range of belief values. The modes include investing the maximum allowed re- source, not investing any resource, or the investment increasing or decreasing with increase in the belief. Notably, for some ranges of belief the biased planner even intentionally obfuscates the agents' signals. Even under stringent transparency constraints cunstrants-iiffommation parity with individuals, no lying cherry-picking, and full observability-wwe show that information mediation can substantially shift so- cial welfare in either direction. We complement our theory with simulations in which L.MSs act as both planner and agents. Notably, the LLM--ased planner in our simulations exhibits emergent strategic behavior in steering public opinion that broadly mirrors the trends predicted, though key deviations suggest the influ- ence of non- Bayesian reasoning consistent with the cognitive patterns of both human users and LLMs trained on human-like data. Together, we establish our framework as a tractable basis for studying the impact and regulation LLM information mediators that corresponds to real behavior.
INTRODUCTION
Large Language Models (LLMs) are increasingly deployed as information mediators in socially- critical functions, from search engines and news aggregators (Zhang al. 2024a Gao al.
2024
to personal assistants offering medical or political advice (Huo et al. 2025 Haupt & Marks 2023
Schiele al. 2024 Argyle et al.) 2023). While promising, LLMs have also demonstrated the
ability to be persuasive (Potter et al. 2024 Carrasco-Farrre 2024 Rogiers et al. 2024), manipulative
(Williams et al. al. 2025 Liu et al. 2025 Jones Bergen| 2024). (024). and strategic (Lorè & Heydari [024]
Payne & Alloui-Cros 2025] Zhang al. (2024b) in their user interactions. When deployed at scale, this algorithmic influence does not occur in a vacuum. intersects with the organic spread of information as people observe and leam from another, creating complex social dynamics that can magnify or mitigate the models' impact.
To make this concrete, consider LLM-powerts recommendation system promoting new restau- rant to a sequence of potential customers. The quality of the restaurant the true state of the world is either good bad, but this is unknown to the users and the system alike. Each user sees per-

---

Published as a conference paper at ICLR 2026
sonalized advertisement and extracts from it private signal about the restaurant's 's quality, and must decide whether to patronize the restaurant. The recommendation system (the planner) can invest re- sources to make its ads more informative: for example, they could personalize the tone or framing of the ad to each user so that they find the ads more engaging and thus spend more time learning about the restaurant. These investments increase how much information users get, but they are also costly and may correspond to expenditures on data acquisition to enrich ad content or market research to understand how to best engage each user.
Crucially, however, a user's final decision is not based solely on this private signal. They also observe the actions of their predecessors-D if the last ten users decided to patronize the restaurant, the next user will infer that they likely received positive information. This phenomenon, known as social learning, is powerful force in shaping human opinion in a variety of settings including the |Rao adoption of vaccines (e.g.,IRao et al. (2007); |Bauch & Bhattacharyyal (2012), new technology (e.g.
Gillingham & Bollinger (2021): Weher (2012)). and political and moral opinions (e.g., [Brady et al.) (2021); Guilbeault et al. (2018) ). Our work's 's central contribution is to study strategic interactions between a planner who controls private information and a population of aggnts who engage in social learning. The planner must therefore anticipate how its actions will not only influence the current user but also create an information externality, shaping the public belief confronting all future users.
While real-world interactions are immensely complex, we distill this dynamic into tractable formal model of controlled social learning, where an algorithmic planner strategically chooses the precision of private signals for sequence of agents at some cost. The planner's influence is thus subtle: it does not falsify information, but rather decides how much to invest in making its signals clear and informative. Our study aims answer the critical questions that arise from this new paradigm: How might planner wield its power to steer collective beliefs? How should its strategy adapt to evolving public opinion? And what are the ultimate impacts on social welfare?
The answers depend critically on the planner's objective. An altruistic planner, like an ideal recom- mendation system, seeks to maximize user utility by helping them make the correct choice (patronize the restaurant if and only if it is good). In contrast, biased planner, perhaps receiving a kickback from the restaurant, wants to induce a specific action (patronize) regardless of the true state.
This distinction is different from, though related to, the concept of aligmment. An altruistic planner's objective is, by definition, aligned with maximizing the agent's expected utility under state uncer- tainty. A biased planner's objective, being state-independent, is not; however, this does not mean its actions are always detrimental. If the true state happens to favor the planner's ' preferred action (c.g., the restaurant is indeed good), its influence becomes aligned with the agent's 's realized utility in that specific instance. We explore the full spectrum of these interactions, examining both altruistic and biased planners in scenarios where their goals may be aligned or misaligned with that of the user.
Contributions and Outline This paper makes the following principal contributions:
1. A Novel Theoretical Framework for Controlled Social Learning. We introduce the first formal model that integrates a dynamic control problem for a centralized information planner with the mechanism of sequential social learning. The planner strategically chooses the precision of agents' private signals to achieve an objective, which may be altruistic (maximizing social welfare) or biased (inducing specific action).
2. A Rigorous Characterization of Optimal Planner Policies. We characterize the optimal policies for both altruistic and biased planners as function of the evolving public belief. For the altruistic case, our results are founded upon novel proof of the value function's 's convexity. These characterizations illuminate the strategic trade-offs a planner must make when its actions create informational externalities for future agents.
3. Empirical Validation and Strategic Analysis Using LLMs. We conduct simulations where L.MSs act both planner and agents. Our experiments demonstrate three key find- ings: (a) A planner that accounts for social learning can dramatically influence public opin- ion and social welfare, far more than myopic one.
(b) The strategic behavior that emerges from the LLM planner largely aligns with our theoretical predictions, suggesting the model is robust to Bayessaa agent behavior. (c) LLMSs exhibit sophisticated strategic reason- ing, highlighting both their potential and the societal risks they present.
We review related work in Section 2 policies in Sections 4 and We then evaluate our findings with LLM--asee simulations in Section and conclude in Section
introduce our formal model in Section B] and derive the optimal
Proofs and additional results are relegated to the appendices.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Our work builds on social learning, information design, online persuasion and RI,, and L.MSs
Social Learning Biikhhhandani et al. (1992) and [Banerjee (992)) introduced the sequential so- cial learning framework in economics, and subsequent work has thoroughly characterized social- learning dynamics in more general settings (e.g.. (Siihhhtthh & Sørensen y (Arieli & Mueller-Frank (2011) ) This literature has also studied many variations in the kind of social information available
to agents and various biases and misperceptions of the agents (see Dasaratha & He (2022): Bistritz
et al. 2022)) Eyster& & Rabin (2010) as well as Bikhchandani et al.] (2024) and the references therein).
The literature the control of social learning is relatively limited (Wei & Anastasopoulos 2022
Smith et al. 2021 Krishnamurthy 2012) Bhhtt & Krishnamurthy 2021). The closest works to our own are Wei & Anastasopoulos (2022) SSmth al. (2021). Wei & Anastasopoulos (2022)
sumes two-way communication between the planner and agents (i.e., agents can report their private signals to the planner in exchange for action recommendations). [Smith al.] (202T) considers benevolent planner who directly alters the choice rules of agents and also derives decentralized pivot mechanism to improve learning. Our model does not rely on two-way communication be-
tween the agents and the planner and maintains the agents' control of their actions, making it more suited for information- meditainn algorithms which are often invisible or black-boxed to the users.
Information Design In our setting, the planner optimizes their objective function by choosing the agents' private signal precision. This is a constrained form of more classical information design questions (Bergemann & Morris 2019). These models came into prominence with Kamenica & Gentzkow 2011)'s ' model of Bayesian persuasion - a planner chooses the signal structure inform- ing rational, Bayesian agent about unknown state. Our planner's problem is then costly, constrained information design problem in each period, where the planner must choose a binary, symmetric signal some cost. While many variations of information design have been studied, our primary distinction is to incorporate social learning between the agents.
There are very few works which consider information design problems in sequential social learning settings (Arieli et al.) [2022] Wu et al., (025). Both works consider a one-shot setting where the sender chooses information structure at onset which is fixed for the sequence of agents. AArieli let al. (2022) considers a self-interested sender who chooses a general information structure as in the classic Bayesian Persuasion problem. |Wu et al.] (2025) constrain the information structure to be more informative in the sense of Blackwell and maximize the probability of a correct cascade. We consider dynamic problem where the sender chooses a new structure for each agent in the sequence, engendering more complex class of sender strategies.
Online Persuasion and RL There is growing recent work at the interface of online learning and Bayesian persuasion/informatio design. These works study online sequential persuasion under un- known environment parameters, in which a sender interacts repeatedly with short-lived receivers while lacking knowledge of the prior over states and/or receivers' utility functions (e.g. Castiglion ctt al. (2020); Bacchiocchi et al. (2024): Castiglioni et al. (2021); Agrawal et al. (2023)) )). A related set of papers examines sequential persuasion in dynamic environments MDPs, introducing state transitions and/or foresight by the receiver, these models emphasize history-dependent or promise- based signaling policies and often connect to reinforcement learning (e.g. Wu al. (2022); (Su] et al. (2022); Bernasconi et al. (2023)). Across these works, the focus lies in learning-based perfor- mance guarantees such sublinear regret relative the best signaling scheme in hindsight and/or approximation algorithms under computational constraints.
In contrast, our model does not require the planner to learn unknown parameters; sequential de- pendence arises from social learning among receivers, rather than from the sender updating beliefs about the environment. Moreover, the planner not informationally advantaged: it does not ob- serve the underlying state or private signals. Within this setting, we are able to characterize optimal policies for the planner. While sequential, our setting is not an online-learning or RL environment: actions do not modify the underlying payoff-relevant state (no reward externalities), and the planner does not learn unknown environmental parameters. The planner updates its knowledge only through observing agents' actions, which reflect social learning and belief propagation rather than payoff ex- ploration or state evolution. Sequential dependence therefore arises from information externalities, as agents' actions shape the beliefs of future agents.
3

---

Published as a conference paper at ICLR 2026
LLMs More recently, some works have specifically studied LLMs in the context of information design (Harris et al. 2025; Cheng & You 2025; Raifer et al.) 2022 Bai et al. 2024 Ductting et al., 2025). These works all consider one-on-one interactions and, such, do not capture the social dynamics we aim to investigate. The most similar work in this area is Duetting et al. (0255) They study information design where sender chooses both a signal structure and "framing' that conditions the posterior belief of the receiver, possibly non- non-Bayesian. They implement an LLM- based oracle approach for the computationally difficult task of optimizing over the framing space. This builds on the notion that LLMs can mimic humans economic agents (Dillion et al. [023]
Horton 2023). Their work does not address our main considerations of repeated interactions or social-learning: but we adapt parts of their methodology for our empirical study in Section
SOCIAL LEARNING MODEL
We consider a planner and countable sequence of Bayes-rational agents indexed by e N>0- At time i = 0, nature determines fixed, unknown exogenous state of the world w E S2 := {G, B). where [P(w~ G) by. and b1 known everyone. In the recommendation system example of Section 1 the state G corresponds to the case where the restaurant is good.
At each time i, the planner decides whether or not invest in personalization for agent i. Subse- quently, it provides agent i with a private signal E Q to aid in pecision-making In our model, 8; assumed binary and matches the state of the world w with probability q E [0.5, 1] i.e. P(s, = w) = qi. We refer to zi as agent i's signal precision.
Each agent's signal independent of those of other agents and the history when conditioned on w. Based on this signal and the observed actions of previous agents < i, agent i selects an action a, E Ω. The agent receives utility 0 if her action matches the true state (a; = w), and utility -C otherwise, where C' > O. Each player chooses her action to maximize her own expected utility. For example, this corresponds to patronizing a business and only if the state is G.
3.1 AGENTS' DECISION PROBLEMS
Each agent observes the actions of all her predecessors and their respective signal precisions. This history is denoted Hi == (bi, (b; (s)) Because all agents and the planner have identical in- formation Hi, there is a shared public belief about w. which is updated after each agent acts. The public belief by just before agent acts is P(w G H.). that before any agent acts is the priori distribution over Q2 b1. As in the classic model of Banerjee (992) and Bikhchandani et al. 1992). (b:) (eN)eN a Markov process and a sufficient statistic for the history H. (see Appendix B.3 for proof).
Informed by Hi (equivalently, by the Markov property, b.) and gi agent chooses action a; E Q so as to maximize her utility. If both actions fetch the same utility, she chooses the action that matches s.. Agent obtains a private belief b, about w using H. and her private signal 8; of precision qi as follows (derivation in Appendix B.1
by
8; = G
b. IP(w = GlHi, (i, 8:)
2b:g
bigE
0. 20.00 8; B
Using this private belief, she chooses the action corresponding to the state of the world that is more likely as per her posterior belief 6, In other words, she chooses a; G if b; > 0.5; a; B if b. < 0.5; and a, s; if b. 0.5. Substituting for b,. we find:
8: 1 9: 5 Sg g:
0: =
G < b;
B
q
< ba
Agent i's action is then observed by subsequent agents and incorporated into the updated public belief betz as follows (see Equation [14) Appendix B.2):
(6: 1- qi S S qi
b f(b..q.)
(3)
bb o.w.
4

---

Published as a conference paper at ICLR 2026
Remark 1. When 1 9, - b, S 9i. agent i's action perfectly reveals her private signal via Equation (2). Thus, the updated public belief is identical to the private belief of agent Otherwise, from Equation 2. agent i's private signal is uninformative and has effect her action. Thus, the public belief is unchanged, and an absorbing state, referred to as information cascade or herding in classic social learning literature such as Bikhchandani et al. al. (7992); [Banerjee (7922), is reached. At such points, society (public belief) stop learning from agents' private information.
Agent i's expected utility is P(a, + wibb,, 9:) which has the following form (see Appendix C.1
PC' + w|b,, q:) -C C'min(b,, 1 b,, 1 114))) q.).
(4)
3.2 THE PLANNER'S PROBLEM
We consider two types of planners: (1) an altruistic planner who wishes to induce agents to take the correct action (a, = w) and (2) biased planner who wishes to induce a specific action, say G, regardless of w. We denote these different planners with subscripts A and B, respectively. In each case, the planner determines the precision of the private signal of each agent. A function 3(-), which non-negative, increasing, continuous, and concave in its argument, will denote the cost associated with the chosen precisions. The planner has an information set H. identical those of the agents.
Altruistic Planner At time i, the altruistic planner chooses the precision qu for agent i and incurs cost of 3(%)) where 3(p) 0. p [0.5, 1). For the altruistic planner, decreasing precision never beneficial (see Appendix C.9). Thus, to simplify notation, the planner incurs additional cost only if increases the precision above baseline value of p. and the additional cost increases with further increase in the precision, with decreasing marginal costs. The agents know p and the function 3(-).
The planner seeks to maximize social welfare minus the cost of precision investment, where social welfare is the expected total utility of the agents. Let (b, 9s) be the instantaneous reward of the altruistic planner beginning at public belief b, and choosing signal precision g; for agent i. Starting from a public belief bj and following sequence of policies (m²) such that Ti (H4) g. the planner attains the following expected total discounted utility, for a discount factor E [0, 1):
Vi(b,) (ab,,
where
Ta n) = B(a) - CTP(a, & wlb,, 4.) (5)
The optimal utility and policy of the altruistic planner are then defined the supremum and arg supremum of (-) over π E II where II is the set of all policies. From this formulation, the myopic value and policy are recovered by setting 8 = 0.
Biased Planner The difference between the biased planner's problem and the altruistic planner's in their objectives and, therefore, in the cost and reward functions. Refer to the example for political campaign in Appendix A for elucidation of a biased planner. The biased planner seeks to induce action G from each agent regardless of w. When an agent chooses action G (respectively, B), the planner, incurs cost 0 (respectively, C' 0), regardless of w. The biased planner can make a private signal more or less precise than the baseline value of p. both of which incur cny choice of precision other than p incurs cost for the biased planner as it requires him to tailor the ad to an agent, which in turn needs research on how the agent best understands any content. The biased planner incurs cost B( q; p!) for choosing signal precision gi with ((0) 0.
The biased planner's 's expected instantaneous reward at time is then defined as follows:
(B q:) = -B(1g: pl) - C' P(( = Blb,, q:)
(6)
Using Equation (6) V-),), V-).). and (-) can now be defined for the biased planner, V+((), V.),,, and क. (.) were defined for the altruistic planner using Equation (5).
As noted in the first paragraph of Section 3 each agent still receives higher utility by choosing an action that matches w. Thus, i = B, the biased planner's s success with an agent lowers the agent's utility. In contrast, since the altruistic planner seeks to have each agent's action match w, regardless of what w is, his success increases the agent's 's utility. Thus, the altruistic planner's 's objective is always aligned with that of each agent, while for the biased planner, this is only true when G.
5

---

Published as a conference paper at ICLR 2026
Both planners' utility maximization problems constitute infinite horizon discounted stationary Markov Decision Processes (MDPs) with state by E [0, 1]. control q E [0.5, 1|, and transi- tion function defined by Equation B (Puterman, 1990). Thus, there exists unique optimal value function (Kallenberg, 2011). We restrict our focus to deterministic Markov policies, thus
II : { [0, 1] [0.5, 1 such that r(b;)= =q-
Remark 2. There are few noteworthy assumptions make in our model: (1) The planner has the same history as the agents. This limiting when the planner could have richer data, but not restrictive when all parties rely on the same public signals, such earty findings survey data. (2) The planner can change agents' signal precision, but signals are still passed through a binary symmetric channel. This is limiting if planners
cherry-pick, censor, or falsify signals, but not restrictive when their role only to improve the quality of noisy data sources without altering content. (3) The planner's 's control choices are fully observable. This is limitiig when covert policies or framing are possible, but not restrictive in more transparent environments.
4 OPTIMAL ALTRUISTIC POLICIES
We first consider the myopic case which corresponds to disregarding the role of social learning. In our formulation, this is achieved by setting the discount factor to be å 0, i.c., the planner ignores all future costs. When = 0, VĀ (b) = (b π (b)). The optimal myopic policy x (+) is:
x (b) E arg sup ra q) vb E [0, qE (0.5.11
Note that the myopic altruistic problem can equivalently be stated as a decentralized case in which each agent chooses the precision of her own private signal and incurs the associated cost with the goal of maximizing the sum of her own expected utility and cost.
Theorem 1. The optimal myopic altruistic policy T is given as follows:
b E (tM, 1 tm) where tM O.W.
((1)
3(1) < C(1
(b)
x
C
0.5
O.W.
Proof in Appendix
Thus the myopic optimal policy takes a threshold form: the public belief is sufficiently strong, the planner chooses the baseline precision p. which incurs 0 cost. If public belief is weak, then he provides a perfect signal, i.e., precision 1. The threshold value depends only upon the costs of the perfect signal (3(1)) and incorrect action (C). When ((1) C. then the perfect signal is overly expensive relative to the cost of an incorrect action and never applied. Thus, !m = 0.5, and the interval of public belief corresponding to myopic optimal precision of 1 empty.
We now present fundamental result for the altruistic optimal value function:
Theorem 2. (-) is convex with respect to public belief.
The proof of Theorem 2 (Appendix C.3 is quite involved and may be of independent interest. The challenge is rooted in the dependence of agents' actions on the public belief. In contrast, the actions did not depend the public belief process (e.g., as in Nyarko (1994), the expected utility linear function of the belief state, and the convexity of the value function then directly follows.
The convexity of the value function is instrumental in characterizing the optimal policy.
Theorem 3. There exist d.A. t such that 0 < dA S tA S tM I 0.5 and be [0, d.) U d 1] be (l,, la))
(b)
max(b, - o.w.
Furthermore, if < 0.5, then d UM. See C.5 for the proof and figure 2a for an example.
The optimal policy has three distinct phases with respect to public belief. First, as in the myopic optimal, the overall optimum policy does not invest in signal precision for extreme values of the public belief. Notably, the overall optimum requires a stronger public belief than the myopic optimal for this to happen since, unlike the former, the latter does not weigh the effect future agents.

---

Published as a conference paper at ICLR 2026
When public belief is close to 0.5 and contains very little information, the overall optimum selects signal precision if it is not cost-prohibo cost-prohibitie. In such a case, (from Equation (2) the agent's 's action equals the true state of the world with probability 1. Thus, the public belief collapses to either 0 or 1.
In the remaining case, the overall optimum chooses the minimum precision max(b, 1- b) such that the agent's action will reflect her private signal (refer to Equation (2). For any precision lower, the
agent's ss action carries no information beyond what other agents already know. Put differently, this is the lowest-cost precision for social learning through observation of the actions of peers.
OPTIMAL BIASED POLICIES
We will begin with the myopie optimal policy as did for the altruistic planner in Section
Similar to Section 4 the myopic biased optimal policy, denoted as TB()). is the arg supremum of the instantaneous reward (B(b, 9). We now characterize T g noting that an optimal policy does not always exist in the biased case (Puterman, 1990, sec. 2.3.1). When necessary we instead pursue c-optimal policies denoted with superscript.
Theorem 4. There exist t E1...., ts E (0. t < < ps St ts 0.5 5. S is < so that:
(A) Ifb E ( (1 lt3, t1] U (p, 1). then (b) p.
(B) lfb (t,, - p) (t), then 8 (b) 1 b.
(C) 1fb (t), then (b) 1.
<.0 (b) b c sufficiently small 0. (D) lfb [ts. pl. then (b) (b) does not exist, and TB
Proof in Appendix C.6 We now apply Theorem to characterize the optimal biased policy.
Theorem 5, There exist t1 E [0. with t $ - p I 0.5 < t2 < p so that:
(A) ffb s t or b > p, then n (b) = p.
(B) Ifb E (f1,1- - pl. then (b) 2
(C) 1fb E (1 - p, 0.5), then (b) 2 b
(D) 1fb E (0.5,tz) and in (b) exists, then (T)()) 2 b.
(E) 1fb E (f2.1), then m (b) does not exist, and B (b) = b for sufficiently small c >> 0.
Theorem (proven in Appendix C.8 and depicted in figure 2a shows five possible phases.
When public belief is sufficiently strong (i.e.. (A)), the cost required steer the system may be too great, despite potential the negative consequences for the planner's utility if 6 < 0.5. In this case, the chosen precision p is less than max(b, b). Therefore, from Equation (2). the agent will act in accordance with the public belief regardless of her private signal. Since this action is uninformative, does not change the public belief, and, because the policy is Markovian, this process repeats ad infinitum. This corresponds to unfavorable cascade if b < 0.5 and a favorable cascade otherwise.
When public belief close an unfavorable cascade, as in (B), the planner increases signal pre- cision so that it is high enough to affect the agent's action despite the fact that, in expectation, the resulting signal will be B. Thus, the agent will act in accordance with her private signal, which has some non-zero chance of leading to a favorable action for the planner. Essentially, the planner invests in a last-ditch effort to steer away from the unfavorable cascade.
For belief slightly higher (i.e., (C)), the planner may decrease precision below p. In these ranges, < 0.5; therefore, more precise signals are more likely to yield unfavorable news. Thus, the planner maintains precision strong enough to influence the agent's action 2 1 1-b) in the hopes of moving to more favorable public belief but will do so with the least precise signal possible.
When public belief weakly favors the planner's desired action (i.e., (D)), the planner adopts precision at least p. Since b > 0.5, an increase in precision makes the agent more likely to infer that w = G. Investment in this regime is the planner's attempt to bolster public belief.

---

Published as a conference paper at ICLR 2026
1.0
History
Precision
Acian
Inalance panametera
Reward for correct
03
dedisio
agent
Peecie-
Genorio meesage
Planner
Agent
Nedioizo
precison
Negatta
Agent proille
Gencric message
-01
Agant Type
Thue state
LIM
State-allvvae lovannt signals
Presision
Oracle
Message
-12
(b)
Figure 1: (a) A system diagram for our experiments, The instance parameters detail are color-coded according to which the LLM roles uses them. (b) Here we show the change LLM (solid) and Bayesian (dashed) agents' beliefs for given prior after positive (black) or negative (red) signal.
When public belief is still higher (i.e., (E)), the planner decreases signal precision just below max 1 b) to - Thus, agents ignore private signals and a private signal overturning the favorable public belief outweighs both the cost of decreasing preci- sion and the potential for public belief to increase further.
take action G. Here, the risk of
EVALUATION VIA LLM-BASED SIMULATION
We now utilize LLMs to empirically study the implications and applicability of our theoretical re- sults and model. Our evaluation proceeds in three steps: we first analyze the behavior of LLM agents to identify key deviations from Bayesian rationality, then examine how an LLM planner adapts its strategy response, and finally evaluate the resulting impact on social welfare.
To simulate the controlled social learning setting, we operationalize our model in scenario where agents decide whether to buy new car. As shown in figure Tal we employ L.LMs in three roles. We describe each role below with corresponding prompts in Appendix E.2.2
Agent: Role-playing as an assigned profile, agents observe previous actions and a private message about the car of known precision, form their belief about the car's quality, and decide to buy or not. Planner: After observing the history actions, the planner selects precision qi of agent i's private signal according to their objective
Oracle: Given the choice of precision by the planner, an agent profile, and a fact sheet about the (see Appendix E.2.I). the oracle generates private signal of desired precision tailored to agent. We compare the outcomes of these LL.M-based simulations with numerical evaluations of assume linear cost function,
the planner's MDP. In both LLM and numerical experiments, ((q) kl 4-lq-pp pl. and vary k. baseline precision and discount factor ö, with the cost of an incorrect action fixed at See Appendix for further detail on the experimental setup and prompting. In Appendix E.3 we validate both the beliefs and the performance of the oracle.
Our experiments reveal the following:
1. The importance and impact of social learning control: Planners (both analytical and LLM) can dramatically improve their own utility by accounting for and capitalizing on social learning dynamics. This may help harm social welfare contingent upon the align- ment between planner and agent objectives.
2. The robustness of our analytical characterization: LLM planners choose policies sur- prisingly similar to the non-obvious analytically optimal policies despite facing Bayesian agents in other L.Mss.
3. The emergent strategic behavior of LLM: LLM planners exhibit sophisticated emergent strategic behavior their choice policies. Not only are LLM policies close to the optimal policies, they also seem to deviate in ways which account for the non- con-Bayesian nature of the LLM agents they face.
8

---

Published as a conference paper at ICLR 2026
150
Fane: Oijackas
Mureo Expendiane
60
Bladd
100
20 Perurage Polaay Deviadia
(a)
(b)
(c)
Figure 2: (a) Example policies from the LLM planner (black) and the analytically optimal planner (red) in altruistic (solid) and biased (dashed) settings. (b) A histogram showing the distribution of the percentage deviation between the LLM and optimal policies. (c) Planner expenditure and social welfare change as percent of the no-control baseline welfare. The true state was fixed B. The left half shows a biased planner seeking action G, and the right shows an altruistic planner.
6.1 LLMS AS NON-BAYESIAN AGENTS
While our analytical results assume Bayesian agents, it is well-documented that both humans and LLMs exhibit systematic cognitive biases (Rupprecht et al., 2025). To first isolate L.M cognitive biases, we test how an LLM belief updates in response to new information. These results, shown in figure Tb reveal non-Bayesian patterns in L.LM belief updating (see also Appendix E.4):
(NB1) LLM agents underreact to private signals which align with their prior beliefs.
(NB2) LLM agents overreact to private signals which run counter to their prior beliefs.
(NB3) As result, LLM agents require stronger public belief to enter information cascade.
These specific cognitive biases have also been observed empirically in human studies (e.g.. Ba et al. 2202);; Chan et al. (2025)). Our results subsequent sections show that the LLM planner's policy robust to non-Bate non-Bayesian agents of this kind.
6.2 VALIDATION OF PLANNER POLICY STRUCTURE AND THE ROLE OF SOCIAL LEARNING
We now compare the policies of LLM planners with the optimal policies derived in Sections 4 and As shown in figure 2a despite the deviations in agent behavior, the emergent strategies of the LLM planner show remarkable structural similarity to the theoretical optimum. This shows that, even when the optimal policy is highly non-trivial (e.g., the biased policy in figure 2ap. the LLM planner exhibits sophisticated strategic behavior which accounts for and capitalizes upon social learning.
In the altruistic case, both planners cease investment when public belief strong and invest heavily when uncertain. In the biased case, both planners exhibit the same qualitative trends: high investment escape unfavorable belief, reduced precision near the midpoint, and no investment once a favorable belief is established. The magnitude of the policy deviation is often modest as shown in figure 2b For both altruistic and biased planners, the deviation is less than 10% for the majority of belief states, underscoring the broad structural alignment between the emergent LLM strategy and our theoretical characterization.
However, there are notable structural differences, which are best understood as the planner's 's strategic adaptations to the specific non- non-Bayesian behaviors identified Section 6.1 (1) The LIM planner tends to avoid extreme precisions (0.5 or 1.0), consistent with known central tendency bias (Rup-) precht et al. 2025). rather than a sharp cutoff. This is direct response the agents' resistance to cascades (NB3) the planner learns that it is never entirely "safe" to stop investing, as even agents with strong priors can be swayed by countervailing private signal. (3) Similarly, the biased LLM planner continues to invest at very low beliefs. This reflects an understanding that its agents might overreact (NB2) a surprisingly positive signal, making "last-ditch effort" more viable than in the Bayesian case.
(2) The LLM planner's policy shows a more gradual tapering of investment

---

Published as a conference paper at ICLR 2026
6.3 WELFARE IMPLICATIONS OF ALTRUISM, BIAS, AND ALIGNMENT
We define social welfare (b) as the total expected discounted utility of all agents. As better information never harms an agent's 's expected utiliiy, social welfare is monotonic in signal precision (see Appendix C.9). This implies the altruistic planner always increases social welfare relative to the baseline. To quantify these effects, we compare social welfare and planner expenditure across three settings: (1) the analytical setting (optimal policy, Bayesian agents), (2) the LLM setting (LLM planner, LLM agents), and (3) a hybrid setting (optimal policy, LLM agents).
The results in figure 2c] confirm that planners all settings can significantly alter social welfare. Furthermore, neglecting social learning (as in the myopic cases) substantially worsens outcomes for the planner. In particular, the biased analytical and LLM planners decreased social welfare by 40 to 50% when misaligned. This was accomplished by intentionally obscuring information about true state via policies which decrease signal precision. The magnitude of the detriment to social welfare is especially striking consider the harsh constraints placed upon the planners in question (see Remark 2). This significant decrease in welfare further substantiates the risk of potentially misaligned LLM information mediators.
The results also highlight the cost of using a misspecified model of agents. In the hybrid setting, the analytically optimal policy, designed for Bayes-rational agents, is "brittle" and its performance suffers when applied to non- non-Bayesian agents. The LLM policy on the other hand, closely resembles the analytical policy, but is better adjusted non-Bayesian agents with human-like biases.
7 CONCLUSION
We introduced a novel framework for studying the strategic control of social learning by an algorith- mie information mediator and developed formal model of this increasingly relevant dynamic. We demonstrated that even a constrained planner can drastically impact social welfare, for better or for worse, and that accounting for social learning integral wielding this influence effectively. Our simulations confirmed these insights, revealing that LLMs are capable of strategic reasoning that mirrors the character of our analytical results while being more adapted to non- non-Bayesian agents.
Taken together, these findings highlight several important takeaways for algorithmic information mediation: (1) the impact of human-Al or algariiag interaction cannot be effectively stud- ied in a vacuum-neelecting their interplay with other interactions (e.g. social learning) means failing to capture their potential, whether positive or negative; (2) information mediators, even when exercising relatively subtle influence, have immense power to guide or derail social learning and public opinion, warranting further study in ways to mitigate the risks therein; and (3) modern LLMs exhibit emergent strategic behavior which can account for and take advantage of social learning as well as non-Baye -ane-aas cognitive biases. The latter two points emphasize the need to better understand how the risks of algorithmic information mediators and LLMs in particular, might be mitigated. One limitation of our study is the dearth of human data. Given that the fidelity of L.LM-human simulators remains contentious (see, e.g., Horton (2023); Gao al. (2025), ) human data could be useful for verifying, or perhaps even fine-tuning, the LLM simulators.
We conclude by highlighting two notable future directions: (1) the generalization our model and results and (2) the mitigation of the negative welfare effects of biased planners. There are a number of desirable generalizations which would relax the assumptions of our model. Settings with larger state spaces, more general signal structures, more general loss functions, and heterogeneous agents are of natural interest bring our model closer to reality. The difficulty in moving beyond binary states and 0-1 losses is primarily algebraic, and we conjecture that the qualitative nature of our results will continue to hold. For more general signal structures, the most significant challenge ensuring that the convexity result of Theorem 2 still holds. Finding the most general class of signal structures for which such a result holds is a particularly interesting problem for future study. As a first step toward heterogeneous agents, generalize the altruistic planner results to allow for agents of opposing preferences in Appendix D
Another important area for future exploration is how might prevent the decrease in welfare caused by biased planners. Here, one might explore regulations or mechanisms that seek to align the incentives of planners and agents to avoid detrimental impacts welfare.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Shipra Agrawal, Yiding Feng, and Wei Tang. Dynamic Pricing and Learning with Bayesian Persuasion. Advances in Neural Information Processing Systems, 06-59273-59285 Decem-
ber 2023. URL https::/prooeeddnng/ www.rrocreennn......i........... nwwrips........ cttpssccccppppee paper html
Lisa P. Argyle, Christopher A. Bail, Ethan C. Busby, Joshua R. Gubler, Thomas Howe, Christopher Rytting, Taylor Sorensen, and David Wingate. Leveraging AI for democratic discourse: Chat interventions
improve online political conversations at scale. Proceedings of the National Academy of Sciences.
(41):22311627100, October 2023. ISSN 0027-8424, 1091-6490. doi: 10.1073/pnas233112271222 URL
https://pnas. wttg//////oooooooooo//oo /0000.111..00000 2073/ppas. 21 6220
Itai Arieli and Manuel Mueller-Frank. A general analysis of sequential social learning. Mathematics of Oper-
ations Research, 16:1223-1249, 11 2021.
Itai Arieli, Ronen Gradwohl, and Rann Smorodinsky. Herd Design. In Proceedings of the 23rd ACM Conference on Economics and Computation, pp. 871-872, Boulder CO USA, July 2022. ACM. ISBN 978- 178-1- 9150-4. doi: 10.1145749048863333322. URL lattps://dd acm. ort httgg/////1..... /01///6//0...11/444445. 3538292]
Cuimin Ba, J. Aislinn Bohren, and Alex Imas. Over- and Underreaction to Information. SSRN Electronic Jour- nal, 2022. ISSN 1556-5068, doi: 10.21 (0.2139/ssm44474667. URL htt.httss://// Seeetps://ww.c.. www.swrr..co.... aatt/tabsttttttt.
4274617
Publisher: Flsevier BV.
Francesco Bacchiocchi, Matteo Bollini, Matteo Castiglioni, Alberto Marchesi, and Nicola Gatti. Online Bayesian Persuasion Without Clue. Advances in Neural Information Processing Systems, 37:76404-76449, December 2024.
Fengshuo Bai, Mingzhi Wang, Zhaowei Zhang, Boyuan Chen, Yinda Xu, Ying Wen, and Yaodong Yang. Efficient Model-agnostic Alignment via Bayesian Persuasion, May 2024.
Abhijit Banerjee. simple model of herd behavior. The Quarterly Journal of Economics, 107(3)797-8177, 1992.
Larry M. Bartels. Presidential primaries and the dynamics of public choice. Princeton, NJ. Princeton University Press, 1988.
Chris T Bauch and Samit Bhattacharyya. Evolutionary game theory and social learning can determine how vaccine seares unfold. PLoS computational biology, 8(4), 2012.
Dirk Bergemann and Stephen Morris. Information Design: A Unified Perspective. Journal of Economic
Literature, 57(1)444995, 95, March 2019. ISSN 0022-0515. doi: 20.12577je.20181449, URL https:/7
www. ae wwwwww.ww....................... erg/ www.wewww.....ateeeeeeee.eeeeeee httpg///aaaaaisssssssesss
12w7/ 20181499
Martino Bernasconi, Matteo Castiglioni, Alberto Marchesi, and Mirco Mutti. Persuading Farsighted Receivers in MDPs: The Power of Honesty. Advances in Neural Information Processing Systems, 11987-115111 December 2023.
Sujay Bhatt and Vikram Krishnamurthy. Controlled Sequential Information Fusion With Social Sensors. IEEE Transactions Automatic Control, 66( 66(12):5833-996) December 2021.
Sushil Bikhchandani, David Hirshleifer, and Tvo Welch. A theory of fads, fashion, custom, and cultural change as informational cascades. Journal of Political Economy, 100(5):992-11)) 1026, 1992.
Sushil Bikhchandani, David Hirshleifer, Omer Tamuz, and Ivo Welch. Information Cascades and Social Learn- ing. Journal of Economic Literature, 62(3) 1093, September 2024. ISSN 0022-0515, 2378-8175, doi: 10. 10777)/2.222221147 URL
Mlai Bistritz, Nasimeh Heydariheni, and Achilleas Anastasopoulos. Informational cascades with nonmyopic agents. IEEE Transactions on Automatic Control, 67, 9 2022.
William Brady, Killian McLoughlin, Tuan N Doan, and Molly J Crockett. How social learning amplifies moral outrage expression in online social networks. Science Advances, 7(33), 2021.
Carlos Carrasco-Farre Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of L.L arguments, April 2024. URI. http: /ttp::///arvv.ooo wwwp:/iw.orgss...oo. / 24044 09329 arXiv:2404.09329 [es].
11

---

Published as a conference paper at ICLR 2026
Matteo Castiglioni, Andrea Celli, Alberto Marchesi, and Nicola Gatti. Online Bayesian Persua- sion. In Advances Neural Information Processing Systems, volume 33, pp. 16198. Cur- ran Associates, Inc., 2020, URL httpss://proceellng. noursss http://www......oooooooo///////0
Mattco Castiglioni, Alberto Marchesi, Andrea Celli, and Nicola Gatti, Multi-Roceiver Online Bayesian Per- suasion. In Proceedings of the 38th International Conference on Machine Learning, pp. 1314-1323, PMLR,
July 2021.
Kenneth Chan. Gary Charness, Chetan Dave, and Lucas Reddinger. On Prior Confidence and Belief Updat- ing, May 2025. URL
otg/ 2412. 10662 arXiv:2412.10662 [econ].
Zirui Cheng and Jiaxuan You. Towards Strategic Persuasion with Language Models, September 2025.
Krishna Dasaratha and Kevin He. Learning from viral content. arXiv preprint arXiv:2210.01267, 2022.
Danica Dillion, Niket Tandon, Yuling Gu, and Kurt Gray. Can Al language models replace human participants? Trends Cognitive Sciences, 27(7):597- 600, July 2023, ISSN 1364-6613, 1879-307X. doi: 10.10167. Nic.2202.044000 Utt /www. cell. 11111..0 zom/ ctwpee///cciiiiiieeeeeiiieeeeeee S1364 6613 (23) 00098-0 0
Publisher: Elsevier.
Paul Duetting, Safwan Hossain, Tao Lin, Renato Paes Leme, Sai Srivatsa Ravindranath, Haifeng Xu, and Song Zuo. Information Design With Large Language Models, September 2025. URL
abs/2509, 25565 arXiv:2509. 255565 [es].
Maxim Egorov. Zachary N. Sunberg, Edward Balaban, Tim A. Wheeler, Jayesh K. Gupta, and Mykel J. Kochenderfer. POMDPs.jl: A framework for sequential decision making under uncertainty. Journal of Machine Learning Research, 18(26):1-5, 2017.
Erik Eyster and Matthew Rahin. Naive herding in rich-information settings. American Economic Journal: Microeconomics, 2:221-243, 2010.
Shen Gao, Jiabao Fang, Quan Tu, Zhitao Yao, Zhumin Chen, Pengjie Ren, and Zhaochun Ren. Generative News Recommendation. In Proceedings of the ACM Web Conference 2024, WWW '24, pp. 3444-3453, New York, NY. USA. May 2024. Association for Computing Machinery. ISBN 970---4007- doi:
011144//559934.364544444 URL [Att:s::: acm. org/ /ww//o......... 1tt.//11.111././1. 1145/ 0584555555334444 3645448
Yuan Gao, Dokyun Lee, Gordon Burtch, and Sina Fazelpour. Take Caution in Using LLMs Hu- man Surrogates: Scylla Ex Machina, January 2025. URL http::/arxii..accc////// http:///xxxv..og/eeee.. wwt.org///bbb/eeeeee abs/aaa/..22 19599 arXiv:2410. [econ].
Kenneth T Gillingham and Bryan Bollinger. Social learning and solar photovoltaie adoption. Management Science, 67711)270117712, 2021.
Douglas Guilbeault, Joshua Becker, and Damon Centola. Social learning and partisan bias in the interpretation of climate trends. Proceedings of the National Academy of Sciences, 115(39):9144-9711, 2018
Keegan Harris, Nicole Immorlica,, Brendan Lucier, and Aleksandrs Slivkins. Algorithmic Persuasion Through Simulation, February 2025. URL (tRt:/ttt://axttti http:///wxxvvvvvx/aa... org/abs/////ooooooo 2231111../331111//1/ 18138 arXiv:2311. [es].
Claudia E. Haupt and Mason Marks. Al-Generated Medical Advice GPT and Beyond. JAMA. 329(16) 3349-1350, April 2023. ISSN 0098-7484. doi: 10.1001/jama..223.5521 [10. 10.10017jama 2023. 5321
URL /attps://doi...... org/
K. Hinderer. Lipschitz Continuity of Value Functions in Markovian Decision Processes. Mathematical Methods of Operations Research, 62(1):3-22, September 2005.
John Horton. Large Language Models Simulated Economic Agents: What Can We Learn from Homo Silicus?, April 2023. WRRLhhtpss:/wwwwx nber. www.orggg....o
Bright Huo, Amy Boyle, Nana Marfo, Wimonchat Tangamomssuksan, Jeremy P. Steen, Tyler McKechnie, Yung Lee, Julio Mayol, Stavros A. Antoniou. Arun James Thirunavukarasu, Stephanie Sanger. Karim Ramji, and Gordon Guyatt. Large Language Models for Chatbot Health Advice Studies: A Systematic Review. JAMA Network Open, 8(2):x2457879, February 2025, ISSN 2574-3805 doi: 10010/11jja 10..00//iamanetworrppee.m.0
57879. URL http::htts:/oo :/coii www.orgg///... 1001 htt.://aaaettteroaaaaaaaaam
2024. 57879
Cameron R. Jones and Benjamin K. Bergen. Lies. Damned Lies, and Distributional Language Statistics: Per- suasion and Deception with Large Language Models, December 2024. M 2412. 17128 1rXvv2241..71128 [cs].
12

---

Published as a conference paper at ICLR 2026
Lodewijk Kallenberg. Markov decision processes.
Lecture Notes. University of Leiden, 428, 2011.
Emir Kamenica and Matthew Gentzkow. Bayesian persuasion. American Economic Review, 101(6))2590- 2615, 2011.
Vikram Krishnamurthy. Quickest Detection POMDPs With Social Learning: Interaction of Local and Global Decision Makers. IEEE Transactions on Information Theory, 5868):5563-5587, August 2012.
LG Al Research. Exaone deep: Reasoning enhanced language models. arXiv preprint 250.:i::550..22222.. 2025.
Mingian Liu, Zhiyang Xu, Xinyi Zhang. Heajun An, Sarvech Qadir, Qi Zhang. Pamela J. Wisniewski, Jin-Her Cho. Sang Won Lee, Ruoxi Jia, and Lifu Huang. LLM Can be Dangerous Persuader: Empirical Study of Persuasion Safety Large Language Models, April 2025. URL httt::///aa/iiiii 10430 arXiv:2504. 10430 [es).
httgg/abs/25000
Nunzio Lorè and Babak Heydari. Strategic behavior of large language models and the role of game struc- ture versus contextual framing. Scientific Reports, 14(1), August 2024. ISSN 2045-2322. doi: 10. 1038/ URL https://wwe nature ctt.//att.cceeee/e111------1--11
331598-024-69033 Publisher: Springer Science and Business Media LLC.
Yaw Nyarko. the convexity the value function in Bayesian optimal control problems. Economic Theory, 4(2):033300, March 1994.
Kenneth Payne and Baptiste Alloui-Cros. Strategic Intelligence in Large Language Models: Evidence from evd- lutionary Game Theory, July 2025. URL TRLL:tttt////////orrg/// wtt.orgggg.//....oa 02618 arXiv:2507.02618 [es].
Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, and Dawn Song. Hidden Persuaders: I.I.S' Political Leaning and Their Influence on Voters, November 2024.
Martin L. Puterman. Chapter markov decision processes. In Stochastic Models, volume of Handbooks in Operations Research and Management Science. Elsevier, 1990.
Maya Raifer, Guy Rotman. Reut Apel, Moshe Tennenholtz, and Roi Reichart. Designing an Automatic Agent for Repeated I baneuage-based Persuasion Games. Transactions the Association for Computational Lin- guistics, 10:307-324, March 2022.
Neel Rao, Markus M Mobius, Tanya Rosenblat, Social networks and vaccination decisions. FRB of Boston, 2007.
Alexander Rogiers, Sander Noels, Maarten Buyl, and Tijl De Bie. Persuasion with Large Language Models: a Survey, November 2024. BRLLhtt:://aaxx. org. 06937 #xXiv22411.0683 [es].
Jens Rupprecht, Georg Ahnert, and Markus Strohmaier. Prompt Perturbations Reveal Human-Like Biases in
LLM Survey Responses, July 2025. URL httl::ttpp//xiitie wttpor///bbbb///eeee 07188 arXiv:2507.07188
[es].
Dr.-Ing- Martin Schiele, Yannick Gittmann, Stefan Ilchmann, Ante Gojsalić, Dominik
rinčić, and Phylis Klempt.
Voting
Advice Applications: Implementation of RAG-supported
LLMs.
July 2024. URL https: http::::/wwwww.. wwwh.x/wwwwwww.cwccc//c..w/w. http:///ww...ccca...ooooooosssss 30tt////t/tttrtcee
wat.:rioo-------a---aaaa-------- 118 oot.aa/ceceaaaaa...o..oooeeaaaaa
commi
James Smith and Kevin McCardle. Structural Properties Stochastic Dynamic Programs. Operations Research, 5065):796-809, October 2002.
Lones Smith and Peter Sørensen. Pathological outcomes of observational learning. Econometrica, 68:371-398, 2000.
Lones Smith, Peter Norman Sprensen, and Jianrong Tian. Informational Herding, Optimal Experimentation, and Contrarianism. The Review of Economic Studies, 88(5):25272255, September 2021. ISSN 0034- 6527, 1467-937X. doi: 10. 10..009/sstudd/raab00... URL httpps://acaeemiccccccc//
com/ wtt.:/ppp.oo.ooe.t.ooootooo
articcle/8//52227/5111
Thih-Tang Su. Vijay G. Subramanian, and Grant Schoenebeck. Bayesian Persuasion in Sequential Trials, In Michal Feldman, Hu Fu, and Inbal Talgam-Cohen (eds.), Web and Internet Economics, pp. 22-40, Cham, 2022. Springer International Publishing.
13

---

Published as a conference paper at ICLR 2026
Jeremy G Weber. Social learning and technology adoption: The case of coffee pruning in peru. Agricultural Economics, 43:73-84, 2012.
Xupeng Wei and Achilleas Anastasopoulos. Social learning with self-interested coordinator. In 2022 IEEE 61st Conference on Decision and Control (CDC), pp. 3137-3144. TEFE, 2022.
Mareus Williams, Micah Carroll, Adhyyan Narang, Constantin Weisser, Brendan Murphy. and Anca Dragan. On Targeted Manipulation and Deception when Optimizing LLMs for User Feedbbck, February 2025. URL
http: / araai.iiiv...... wttp://rgggggb//eeeeee awt/aa/s//241. 32306 arXiv:241 1.03066 [cs].
Jibang Wu, Zixuan Zhang. Zhe Feng, Zhaoran Wang, Zhuoran Yang. Michael I. Jordan, and Haifeng Xu. Se- quential Information Design: Markov Persuasion Process and Its Efficient Reinforcement I Learning, Febru- ary 2022.
Shuo Wu, Pawan Poojary, and Randall Berry. Observational Learning with a Budget, April 2025. URLhttp: //axxiv.erg/bbs bbbrgg/ab//220 19396 arXiv:2504. 19396 [es].
Tianyi Zhang, Faisal Ladhak, Esin Durmus. Percy Liang. Kathleen McKeown, and Tatsunori B. Hashimoto. Benchmarking I Large Language Models for News Summarization. Transactions of the Association for Com- putational Linguistics, 12:39-57, January 2024a. ISSN 2307-387X. doi: 10. 1662/aclllllc a 00632. URL
https://dcl erg/ 10. 1162
Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Adrian de Wynter, Yan Xia, Wenshan Wu, Ting Song, Man Lan, and Furu Wei. LLM as Mastermind: Survey of Strategic Reasoning with Large Language Models, April 2024b. URL [Rttt://///xxii// org/ abs 2444 01230 arXiv:2404001230 [cs].
A APPLICATIONS ELABORATED
Consider sequence of individuals (agents in our terminology) deciding whether to patronize business or service provider (c.g., a restaurant, contractor, realtor, etc.) and a recommendation system (the planner) serving the agents information about the business. The true state of the world, assumed to be binary, is whether the business is good, which is unknown. Both the recommendation system and the agents want to be consistent with the unknown state. That is, if and only if the business is good, agents want patronize it, and the recommendation system wants to induce the agents to do so. The recommendation system an altruistic planner in this sense.
The recommendation system can show an agent highly targeted (precise in our terminology) ad that showcases the strengths and weaknesses of the business in contexts that she can relate to, given her background and characteristics,
can show her generic confusing ad that would not help inform her action. The precise signal will be more informative, comprehensible, and relatable her to the correct conclusion, ie.. patronize the business if and only if it is good. Social welfare increases as more agents arrive at correct decisions. However, changing the precision of an ad also incurs cost, as it involves tailoring the agent's specific background. The recommendation system must then choose the precisions so as maximize social welfare minus the costs.
the agent. As such, is more likely to drive
now provide a real-life example of hiased planner Consider a group of voters (agents) motivated to support the candidate most likely to win in their community. The motivation to "back a winner" has been shown to influence electoral outcomes, e.g., this is why US states with early primary elections have an outsized impact election results (see [artels (1988). The planner is specific candidate's campaign and, therefore, seeks motivate agents back the candidate. The binary true state of the world indicates the candidate winning or otherwise. Thus, if an agent knew the true state, she would back the candidate if and only if he were winning.
Each agent understands her community well and would know if the candidate is winning (i.e., the true state of the world) if she knows his stances and policies. She would know the latter correctly if those provided her in a manner that she digests information best. For example, some agents understand audio-visuals best, some long- articles backed by facts, figures, and citations, some only brief and focused contents, and some only their native language, etc.
The campaign sends digital ads (i.e., private signals) to agents in varying degrees of precision. The precision represents how much the content tailored to the agent's taste. Note that a precise signal accurately conveys the true state of the world to an agent by helping her clearly understand the candidate's policies, track record, and character, which enables her to correctly infer if the candidate will win. A precise signal does not necessarily mean that the agent backs the planner's candidate, though. For example, if the candidate loses in the agent's 's community as per the true state of the world, highly precise signal would induce the agent to oppose him. However, imprecise signal is more likely to induce the agent into backing him since it obfuscates the true
14

---

Published as a conference paper at ICLR 2026
state of the world from the agent (possibly by being vague confusing), thus increasing the chance that she thinks winning. Thus, the biased planner may incentivized decrease an agent's ss signal precision if he thinks that his candidate is losing. To tailor the ad to an agent, the planner must research how the agent best understands any content, which incurs costs for the planner. Even rendering the signal to be really imprecise is costly, as it still requires tailoring to the specific agent, e.g., the planner needs to know that an agent best understands focused and brief messages be able decrease precision by confusing her with long-form verbose, detailed articles. The planner selects precisions so as to maximize expected number backers minus the cost incurred in generating the precisions.
B SOCIAL LEARNING DYNAMICS
We first summarize the sequence of developments step i:
1. The planner determines based on Hi and, yi is observed by all agents.
2. Agent receives private signal s; such that P(s; = w) =
Agent takes action a, based on the observable history and their private signal with aim of maximizing their own utility.
All other agents (and planner) observe a; and the public belief at the end step i, bet updated accordingly.
Note that from (1), 9i function of Hi. Thus, if random variable is conditioned upon both gi and His the conditioning on 9i is redundant. We will use this principle throughout.
B. PRIVATE BELIEF UPDATE
b. be the public likelihood ratio based on observed actions. Agent forms a private belief about w, b. Pfw G)H,, S1 }, by combining the current public belief with her private signal, 8;. consider private likelihond ratio, â; == T-b." how and b, are computed.
We now describe how agent i chooses action Let ==
h. now describe
1. If private signal is G. then
b.
P(w H. qr, 8t G)
à.
1 1 P(w B|H, gt, G}
F(s: = Gl = G. H., R3 P(w = C|H..94 93
P(s, Glw B, H., 11 P(u BlH,, 9:
di b;
(8)
b. 1 6i
1 q
(9)
obtaining Equation we apply the assumption that, conditioned upon w. agents' signals are
independent of one another and the history to assert that P(s; = Glw = G, H.} = P(s; = G w G}= = The same is done when conditioning upon B. We additionally use the fact that is function H..
2. If private signal is B, following the logic above.
Zo
(10)
q:
We let ybb, q) and z(b, q) denote the probability of realizing signal G or B, with public belief b and signal precision g conditioned upon the history. By conditioning and applying the Law of Total Probability, these probabilities can expressed as follows:
y(b, 9.) Pfs. G/
Σ P(s: = Gl H = k) P(w k|H.}
NE(O.B)
g:b:
q: (1 b:) 29.bi - g.
(II)
15

---

Published as a conference paper at ICLR 2026
Similarly,
z(be, 9:) 9: b; 2q:b;
(12)
Combining Equation ② Equation
10
Equation
and Equation
12 yields the following:
b G
b. P(w G|H,, 8.) f(b,, 9.)
u() 2(6,.0c) b,
(13)
B
Note that if G then y(b,, P(s. G)H4) cannot be O. Similarly, if B, then z(b,, 9:) cannot be Thus, the equation above well-defined.
B.2 AGENT ACTIONS AND PUBLIC BELIEF
Agent i's action o, will be G Gf exceeds 0.5 (equivalently, if ó; 1), B b; 0.5 (equivalently, if a, 1), and 8; otherwise as per the tie-brekking rule. This action and i's precision qi are observed by peers, resulting in an updated public belief b;-
When b < Gr then a, < 1-9x, Therefore, if s, = G. ã, by Equation 9 if s, = B, 5.
(1-91)2 < 1 (since 2 0.5) by Equation 10 Thus, no matter the private signal realization 8,, = B when 9
< 1 qs- Similar arguments be used to show that when b, qi. as G. Intuitively these cases when the strength of the public belief overpowers the precision of the private signal.
Now consider the case that - ge S b, q:. Then = G, de 2 from Equation ( and since 2 q:- s; B. then ā; & 1 from Equation (10) and since b, I q.. Together with the tie- beebkiek rule, this results in action This leads to Equation (2) reproduced below.
S b S
a,
G
9
< b 9:
Thus, agent i's action is only informative in the first case of Equation @ where it allows future agents to perfectly infer i'ss private signal. Thus, in this case, public belief will update just the private belief from Equation (33). latter two cases of Equation (2). the action contains no information about true state. Thus, the public belief will not change. This yields Equation (3) reproduced below:
[b 1-9 S be o.W.
bi+1 = f(b., =
(14)
B.3 MARKOVIANITY AND SUFFICIENCY OF PUBLIC BELIEF
We now prove that Pfa, alHi, } is function of bi, for {G, B}, regardless of the rest of Hi It follows then that
P(a: a H., q.) P(a alb,, 4.)
(15)
From Equation 2 the second third cases, it is clear that P(a. a (H,, q:} a function of b, qu a E (G, B). regardless of the rest of H.. We consider the first case next.
P(aa = GlH,, q))
Σ
P(w = u/ Ho, P(a, GlH,s n: =))
b, P(a: Hi,
G)
+ (1 b.) P(a, GlH,, GG, B)
b, P(s. Glb,, Lb, = =G)
+ (1 6,) Ps: Gbb, 9:, B)
b.g: + (1 - (1 q:)
(16)
The first step applies the law of total probability. In the second step, using the fact that q: is a function of H.. we substitute b, for P(w = w|H,, q:). In the third step, we use the fact that, for 1 - g, S b; S qi. = and, further, that independent of the history when conditioned upon The result follows.
This leads to the following thcorem:
16

---

Published as a conference paper at ICLR 2026
Theorem Markovianity of the Public Belief Process
b ((w qi,
(17)
In other words, by captures the entire information pertinent the update of 6,41 that is contained in H,.
Proof. Note that, from step (I) Appendix
qi a function of H., It suffices show
P(a: O Glb,,
b
P(az|bi.9:
We will the claim via induction on i. For our base case, consider so that Mz = (bi, (21, a1)) and b = b1 An application of Bayes' Theorem provides the following:
P(w Glb 2: G,b,,
b P(or|w = G,b
b2
P(ai 91
P(an 91
b1 P(a O w G 91
Fla bb o P(w 9)
(aa, O G1b,
P(ai|bi.00
where the final step follows the unconditioned belief of w is equal the prior belief b1. Now, assume the claim holds for all n.
P(w GlHnn 2n Pfanlw G, Hns }n}
bn41 = P(w = Gl ans Jn
Paan Ho,, 9n
bn Planlw G, Hnt
P(an||Hn.4 Hm., 4n by Pfaa ow Gl 9n
P(an 9n) P(w = Hm, 4 P(an nw Gi|b.. qnd
P(a,Ibnn. 9m)
where the final two steps follow from Equation 15 and the definition of bn.
This completes the proof.
Note that we have shown that the public belief Markovian adapting the arguments BBanerjee [992] and Bikhhhandanii al. (1992). The modifications were required because the differences in selections agents' precisions caused by the introduction of the central planner.
C PROOFS
C.I DERIVATION OF EQUATION
Lemma 7.
P(a, # w/b,, q:) min(b., - b,, 1 qs)
Proef. We prove this for each case of delineated in Equation follows:
In the second case, can reason as
P(a. # w/b q:) (P(w + Glbs, b.
Similarly, the third of Equation
2 obtain:
P(a: w/bi, qi) Plw + Blbi, qi)
17

---

Published as a conference paper at ICLR 2026
For the case when 1 4: S qi and a, from Equation 2
can follows:
P(a, + w/b,, 0.)
Σ
P(a. = alb, q.) [Pw + alb,, Q6, a, = a)
aE(C.B)
P(#; G bb q:) (lw + G/, G)
9i, di = B) = P(s, = Bbb 4:) P(w * 9i,
Equ 2
(18)
u(b,, (. [Fw Bbb qr, 8, G)
-z(bi, Plw = Gbb,, 81
Eqns 11 12 Thm
(19)
(v(
b. 0
y(bs. (:0s:::)(1 v(b112)) =(bs: 9i)
bc(1-ar) 2(6)-00)
b *
Eqn 13
(20)
1 q: y(bi, 9:)
-bigt b.(11 1 qo)
+
9:
b.
y(bs 9:)
bigi
biq:
b, +
q:
b,
qs by A O
Combining the three cases yields the claim.
C.2 PROOF OF THEOREM MYOPIC ALTRUISTIC POLICY
Applying Equation
4 to (a q) in Equation
5 allows to reason as follows
Ör a (b, 9)
BB(g) B(( + q max b)
< max (b, 1 - b)
(21)
8g
restrict the altruistic planner to precision & [p, 1] because social welfare increasing in precision (see Appendix C.9). Accordingly, ((p) 0.
As stated in Section 3.2
Because (() increasing, the first case of Equation (2 leads to optimal precision p if < max(b, b)- That is, the signal is not precise enough to influence the agent's action, then the planner will not expend resources for it. In the latter case, because B--) increasing and concave, B(-) is positive and decreasing. Thus, ra q). if the optimal lies in this range it must be attained at 1.
We now identify the regime in which the latter case is optimal.
(b p) (b, 1) -Cmmin(6,1-b,) < 3(1)
min(b. 1 b, b, -p) 3(1)
C
If f-1-ps 901), the above incquality is never satisfied, and is never optimal to select precision 1. Otherwise,
the inequality above yields & [(1) 1 s{i) This yields Theorem I
C.3 PROOF OF THEOREM 2 ALTRUISTIC VALUE FUNCTION CONVEXITY
Proof Sketch We inductively prove that the expected kethh stage reward, i.c., the expected utility of the planner from the control and action of the k-th agent, is convex. The instantancous reward (Equation (5).
which is convex with respect to public belief, provides our base case.
The first challenge encountered is the unusual nature of the public belief update. Although the state space uncountably infinite, the belief update only takes support define a decision tree, i.e., complete binary tree of all possible trajectories once an initial belief and policy are fixed. Each node has two children corresponding to each possible signal realization the next agent might receive. The root is the expected instantaneous reward at time 1. i... from the first agent's 's action. The induction moves down the levels of this tree with the k-th level containing 2 nodes, each associated with sequence of realizations of signals.
maximum of 2 values. To manage this,
18

---

Published as a conference paper at ICLR 2026
We then show that for node in the (A 1)-th level that has convex expected reward, its children in the l- level satisfy the same property. This is where we must deal with the dependence of agents' actions on public belicf. Note that even when applying the same precision and receiving the same signal realization, two agents beginning at different public belief's may take opposing actions (see Equation (2) Thus, standard results that provide easy ways of bounding the future terms of the Markov process do not apply. Here, our specific belief update is actually helpful. We leverage the fact that Bayesian updates martingales (i.e., [[bb+1] = b.). Along with the convexity of instantaneous rewards, this allows us to complete the inductive step and, subsequently, the proof.
Preliminaries We first prove the following lemma containing a few useful properties of the optimal value function (.).
Lemma 8.
Vi() is non non-positive, Vā( (1 6)6b e [0,1], and Vi(()= VI(1)= with =² =²(1)= p.
Proof.
Non-positivity
r(b;,g: S 0 Vb, q: + VĀ 00
Symmetry
Let = 1 b, and let A, and As be the decision trees rooted at states no and bu. respectively. Showing that An and AG symmetric suffices show the symmetry of (-). This will be accomplished by showing that the expected rewards, belief state values, and branching probabilities of trees are reflections one another.
First consider the instantancous reward function FA. Note that the decision function of the agents when choos- ing their actions is symmetric from Equation thus, P(a. Glb.} = Pf Blm.). Applying this, obtain
r(u).9:) -((=)) Cr, Gl C(1 n4) P(u: Glw n.
((4:) - C(1-6)) = Blw G. m. Cb, PPa. Glw B,
= =(2.) - -C(1 b,) P(a. = Glw = B,b, b.] Cb, Pfa. = B|w G.
= r(bi,gi)
Thus, the instantaneous reward function is symmetric about 0.5.
Now consider the belief state update when beginning with belief n, assuming signal precision we will use T and to represent updated beliefs from x after observing action G or B, respectively. From Equation (3)
Toa y(ns, q6) The =b,, 9:)
qi
b, (1 -b,) = 9. by 1 be y(b.,g.) q:)
(1 bx) =
1 q:
z(bi, 9:))
9i q: y(b:, 9:
T4 z(ni,, gi
Thus, the belief updates are also symmetric. Combining this with the symmetry of the reward we obtain:
r(n.) = r(1 - =rbb)(b))) r(n)) = b-))
(
Thus, expected rewards and belief state values of A., and As are mirror images of one another.
Finally, consider the branching probabilities which we will denote as y(b, 9) = + 2bq - - q for observing action G and a (b, 4) = 1 y(b, q) for observing action B.
y(n 4) = 1 + 2nq - n q= 1 2(1 b)g (1 b) - = b - 2bg = z(b. 4) <nn g) = uvn q) = 1- 2(b, = y(b, q)
Thus, the branching probabilities of A,, and As are also symmetrie, completing the proof.
Extremal values
We will show that (0) and rely on the symmetry about 0.5 proven above to show that Vo((1) 0.
From Equation 3 when be 0, bt by = 0 for any value of precision. Thus, regardless of signal
realizations or control actions, the agents' actions and, subsequently, the public helief, will remain unchanged. That is, b, Ovi and, from Equation @ a Bvi
19

---

Published as a conference paper at ICLR 2026
From Equation 1
we then write:
(0) max E
8'r(bi, (bb))
max E
Σ 8'r(0, x(0))
max E well
Σ 8' (-) Cb. Pfa. B G, 03 C(1 b.) Glw B, 03}
max E well
Σ 8' [-q)]]
(22)
Thus, to attain (0). it suffices to choose (0) such that (0)) is minimized for each i. Recall that 3(- is increasing on [0.5, 1] and takes its minimum at 3(p) = 0. Thus, by choosing = -vvi, Equation 22 yields
(0) = 0 with corresponding optimal policy (0) p.
The aforementioned symmetry about 0.5, shows that (1) 0, completing the proof.
We now introduce the concept of decision trees for Markovian and more general policies. A policy can be represented by a decision tree, which is a complete binary tree as in figure 3] Every node of the decision tree represents the state of the system, i.c., the corresponding belief value when the controller sends its private sig- nal to the agent corresponding to the level of the node. The branches represent the actions taken by the agent corresponding to the level once receives its private signal. The controller's policies (i.e. choice of precision) determine the probabilities of the actions from each node and the belief resulting from the action. The proba- bilities of an action at certain node can be considered the weight of the branch. A decision tree is uniquely identified by the belief values associated with nodes and the weights of the branches. For deterministic sta- tionary Markovian policy, the controller's 's choice of precision, and therefore the probability associated with an action, is deterministic function of the belief value of the node, regardless of the level of the node in the tree and the path to the node from the root. Thus, the decision tree of any such policy, i.c. the belief values at the nodes and the weights of the branches, is determined entirely hy the value of the initial state. For arbitrary policy of the controller, the controller's decision at given node can depend the entire path leading to the node.
Proof of Theorem
We now prove the convexity of V+(() with respect to public helief.
Proef. We will first show that, for fixed precision, the instantaneous reward ra is convex in public belief. We rewrite T as follows using Lemma 7
ra = - (()) C min(a, 1 x, 1-1-)) 4)
(23)
The pointwise minimum of linear functions is concave. As we are subtracting the minimum, the overall function is convex with respect to x.
consider arbitrary 2o, ma, no such that Immoo + (1- 0m = zo for some E [0. 1]. To show the convexity of V we must show that V (2o) & LV2 (ma) + (1- Vī (ru)) By definition of the optimal value function, V (z) 2 VI(x) for any policy x. Thus, it will suffice to show that there exist स such that
Vi(za) S VV (mo) + ()
(24)
We refer to the decision tree of T² when the initial state is Io A.
We consider स be a policy of the controller that chooses the precision at each node to be the same what does at the counterpart node. The nodes of decision trees are considered to be counterparts if they can be reached via the same sequence of actions from the root (see figure illustration). Thus, the precision chosen by this policy is the same as what A does after the same set of actions of the agents. We refer to the decision trees for it with initial state mo (respectively, no) as Am (An).
We illustrate the policy म for initial values mo and no considering the decision trees Am and A,, While doing so, we illustrate counterpart nodes and the weights the branches. We will select one path of actions out of these decision trees by specifying set & as the epochs where action is taken and B as its complement. Refer to figure for illustration. Considering the decision tree A, we will refer to 2, as the state at epoch i when beginning with belief ze and following x² along this path of actions. That is, assumes one of 2 possible helief values at epoch starting from zo. In decision An (respectively, An), use m, (n.) denote the
20

---

Published as a conference paper at ICLR 2026
public belief' beginning with mi (no), following the path X. Node my (respectively nu) in decision Am (A,) is the counterpart of node z, from decision tree Note that, # (m,, can now illustrated notationally. Specifically, #(m,) = fn,,) = = (z;). Similarly, ă can be defined at cach node of Am. considering the action taken by æ' at cach counterpart node in A. Note that y(z;, A (24)) (respectively, 2(2;, T (2:))) is the weight of the branch corresponding to G (B) emanating from in A. Because 5 ((m,) = +mm:) T (n:). y( =( (24)) the weight of the branch emanating from m, for action G Am- This branches associated with action B and An. Thus, three decision trees differ in both the values of the nodes and the weights of the branches. Since x is deterministic, so is म. Therefore, from
(mmg, # (m,))
extends similarly
Equation (13) and Equation 3 Tv. mi, and n, are deterministic quantities once the specific path of actions is specified by g.
XO
x(46))
E
20yy Alali
I- Sly))
N2 E
Map Algg))
KC mEA)))
G
24-1 passible states
|I Las
|| but
1-
10g
Xx E
possible states
X k-1
(a) Decision tree A ronted at
mg
fg
JAN
Mlg
1-
m
(b) Decision tree Am rooted at mg
(c) Decision tree An rooted at no
Figure 3: Here depict the belief evolution in the form of three binary trees. The tree begins with state æo; mp, and no. respectively, and level contains 2 possible states reachable after i epochs. Left branches correspond action leading to updated belief in (respectively, my and n2)) Right branches correspond to action G leading to updated helief To (respectively, my and ng). Each branch labeled with the corresponding probability, and the bold states indicate an example path, like A. In the example path, 0 E G and 1, 1 E B. The bold states in each level of the three trees are counterparts.
Now let o5 (bo) abe, [(b))] expected instantaneous reward at epoch when beginning with state by and applying policy æ and (=)) (x)). Applying Fubini's Theorem, can rewrite our value function
21

---

Published as a conference paper at ICLR 2026
Vî (bu)
>
8' (bs, 7(b))
8'Elra <(b))]
= (bu)
(25)
1
Thus, we can rewrite 24 as
20
Σ (x)) 51
Σ 1==0
8' (mo) + 1)
Σ
8 の (oa)
(26)
1=0
1=0
To prove this, we show that for all i == 0,,,2,
oi (xto S to (mo) (1 t)o? (no)
(27)
leading directly to 26 24 and finally the convexity of (-) with respect to public belief.
Now let consider general (x-1(z0) will have 2*=1 terms each corresponding one path of possible actions the set [G, B)+ Thus, at epoch ks 1, the path will yield the following term in (2))
II U m(((a)) 4eg
[[ JEB
a( (((o))(()))) 7A
(28)
We now define (x(o)) = (TL. v(s,, TA (Thg 2(2;, x (x:))). that is, the probability of following the specified path when beginning with state zo and following policy Tÿ. Again, we will use 5; and H, to represent updated beliefs from z; after observing action G or B, respectively. Thus, F; and z; constitute belief values of the children nodes (in the (i + 1)th level) of the node with belief value in the i-th level. Refer to levels k- 1 and & in figure Bal Considering the children - and Fa-1 (in the k-th level) of the node 2 (in the (k- 1) level), we obtain following two terms in (2n):
Px (wo) [1( -
=x( + 2(Ee-1, TA (=== () - 1))]
(29)
Similarly, we define P (mo) = (eT. y(m, ((m:)) (TkE =(m;; #(m))), that is, the probability of fol- lowing the specified path À when beginning with me and following policy न. Because +(m:) = H (m:), Pt (mo) = (ITeo v(m,, 72 (x)))) (Tker 2(m,, m2 (x.)).. We also define PX (mo) by replacing {m.} with (mc).
The corresponding terms in oi (mo) and oi (na) are
P (
(30)
7. (=)
(31)
We will show that
PX
म. [ااد
& LP ( ( ( ग. (- [ااد)))]
+ (1 t)P (no) (m (T)
((14 (#x= ))r (بايا o. (ااد
(32)
Similarly, each pair of children for each term in the (A: 1)-th level of the belief tree can combined and the corresponding inequalities can be obtained following the same process. Summing the resulting incqualities will yield (27 leading to 26) (24). and the convexity of (-)..
Proof of
We now focus on the latter the two terms of (9)). We will show the following two properties
ym,, (x,)))
2(ms, (x.))
Σt-1 =t t II
II
y( T (I:))
2 ((
Mx-1
JEG
(EBU(R-1)
p(n, (xx))
2(n,, (2:))
+ (1 f)
(33)
((a, n (Ii))
2(1,, o. (x;))
Neg
iEBUfk-1)
22

---

Published as a conference paper at ICLR 2026
y(mi, (x+))
=(me, (a 2(0,, & (x:))
= II
II
y(zs, T (z+))
VEG
(EBL(E-1)
y(m,, a (xa))
2(m ग (ri)) a(2i, क (a::)
+ (1 - t)
(34)
ieg y( x (2s))
iEBU(R-1)
Defining == (Ilieg
Mmmca (+0)
(IIEBU(E-1)
ammu. (we))
it follows from (33 and (34) that
u(ms. (022)
a(peo (=22)
im + (1 1)12211
(35)
From the expressions for PX PX (mo), it follows that:
f (( (mo)
(36)
Similarly, we can show
i=(==-1, 1 ()(x)=1)))) (1- 12 (no)
(37)
We apply this as follows to the latter term of 29
$ PN (
firam (( + a h (Zn-1))
(Convexity r, Equ 35
ps (me) 2(m
A (z
+ (1 - ma
& (
(Eqns 36
37
(38)
Applying the argument the first term of 29 yields:
Px 7X
+ (1 1)
(39)
Summing
388 and
39
yields 32
We now complete the proof and, therefore, the entire proof, by proving and 34
Proof of (3
From the recurrence rolation in Equation 13p and Equation 3 we can write 1 m and n as the
product of all belief updates with and vao, respectively. as follows. Each term in the products corre- sponds the multiplicative state update after single action.
II (a(as) Vieg
n (x:)) 2(2i, T7 (x+1)
Fe-1
(x
(40)
(EBu(k-1)
m (x.) (mm;, aT's (Ix))
1 1- (xx) 2(mi; n7 (x:))
mp-1
(41)
Neeg
iEBU(R-1)
Tx(x))
1 (xx)
II leeg
y(mi, o (a:))
2(14, T. (Ir))
710
(42)
(E-1(
23

---

Published as a conference paper at ICLR 2026
These equations may be best understood via the illustration in ligure 3 m and 1 using the fact that za tma + (1 t)mo.
We then reorganize terms relate 1k-1
TA (rs)
1- x (2c.)
[tmo + (1 - t)mol
(from 40
LEG y( = (x:))
2( =²a (z:))
iEBUfK-13
=t y(m n (x:))
2(mi, n (xx)) 2(ri, T (3:))
II
m
leg yl A (a:))
iEBU(K-13
y(m (ai)) II
a n (1:)) 2(25, A (m:))
+ ()
( t (z;))
2x-1
(from 11 and 42 (43)
leg
(EBU(K-1)
This proves
33
Proof of
We first consider the numerator of the first term in 34 now prove that this be written as cutto + So for some ou, Bo which depend only and the nature of the dependence determined by the sequence actions along this path in stages to We will prove this via induction working backwards from the final term (i.c. the (k- - 1)-th term) of the product. Therefore our base case will be the the final term of this product which can be either y(m+-1, T TÂ (T+-1)) or
Theg y(m,, 7 (xx)) Be 2(mi, 7 (z:)). We (x,) -
1 n (#*-1) 1)), depending the preceding action (in the (K 1) stage).
y(m 1, (x 1 2m A (x)-1) ma-1 1 T (xx-1)
(2 (Tk- - 1)m (1 - n (2xx1)) 2(me-1, m म (2x 2m (2x-1)
= (1 - 2- A ( 1 + (T (xx-
Thus, the final term of the product is of the form 0x-1 + B where a and 8x-1 depend only upon A (2x-1), and the nature of the dependence is determined entirely by the (k- 1) action (since this action determines whether the final term is v(-) z(-)).
the inductive step, assume Il.gass ymm, T (x:)) IIESB(A-I))SS 11) z z(m,, TA (a:)) can be written 0s+1 + -8f+1 for some ag+1, By+1 which depend only { (3ra) (22))<s and the nature of the dependence determined entirely by the sequence actions this path for > to === Thes, v(ms, TA (xx)) () 2(ms, T (a:)) can be written as my + again with aj, 83
1. will now show that
(2.)) Jass))ssss with the nature of the dependence determined by the sequence of actions on depending only on (2x))
y(b, g) = G and g64.9(44 a) - a= B x(b,4) = B
a a= G
this path for > j- 1 io k
We define l(b.q.
Thus,
II y(m, (xx))
II (LBU(k-1)aas--
a(mi, 7 (x.))
1
=(mm, a. a541)
II y(m,, ai()))
II
=(mi,
(EBU(K-1),0b
= (1))
[0y+1m1541 + By+a]
(inductive hypothesis)
g( (2;), a5+1)
f(m,, ( (1))
0.+1 f(m,, o (2, ) a)t1))
my By42
(Equation
= =a az+1) 15+1 - ((t))
Thus, from the definitions of f()), and 2(-).
(a)- + (3) 0, 10-1((1- - T (, (x, 0st1 = B and 8, 31+1(1 (xx1(1)
0j+1 G
aj+1 G
(541
0,41 B
In both cases, by inductive hypothesis, 0,-1, By+1 depend only { (x,))ss; and the actions in this path from i j to
1. Thus, the claim holds for aj, 3,, completing the induction.
Now recall that the denominator of 34 IIeg y( > ( ILEBU(L-1) 2(x1, = (2:)). Note that policy and sequence of actions is the same as those for the numerator. Thus, because, as shown, the coefficienss oo: fl depend only { (x;)) and the path of actions, the same logie can he applied write the denominator
24

---

Published as a conference paper at ICLR 2026
cuzzo + So for the ou, So. This can be done similarly for the numerator of the second term in 34 Thus, can reason about the coefficients (33) as follows:
y(m, nā (a:)) II VEE (
am,, 7 (x:))
= = (2%a))
(580(6-1)
y (x+))
2(mo, (a;)) 2(Mi, (m:))
+11- (1-f)
II
y( A (m:))
leg
(EBU(A-1)
t(como [6) + (aano 36)
0.0200 + Bo
00000 + 30
(Since zu tmi + (1 t)mu)
0u + 3
Thus, holds, completing the proof.
C.4 ALTRUISTIC MYOPIC POLICY BOUND
Lemma 9.
20) S (b) 46 [0.1]
Assume by way of contradiction that there exists (b) < <V (b) for a policy и we construct, violating the optimality of V2()).
[0 1] such that (b) (b)). We will show that
Let n be such that, when starting at public belief 6 the policy т applies precision x (b) at the current time step and then applies the optimal policy at all future time steps. We now elaborate V (b) V (b) by breaking the value function into instantaneous reward va and future discounted reward. We will refer 6 and 5 the updated belief's after receiving signals G and respectively when applying the optimal policy and bit and b as the same when applying the myopic policy.
VĀ( (b) (b xZ( - (b,
6y(b, (2)V 62(b, aI (b))
dy(b, (0)) -z(b ( (Z)
(1))) 6=(b, T2 (1)
(definition of T%)
(Theorem 2
Here, applicd the fact that the instantancous reward under 7-)-) must be greater from the definition of the myopic optimal policy.
then relied on the convexity of V.(.) from Theorem for the future cost. From Equation [3 note that
by < <bsat < bt. Along with this convexity, this leads to the last step above and completes the proof.
C.5 PROOF OF THEOREM 3 OPTIMAL ALTRUISTIC POLICY
Consider V (b) with policy applying precision in the current time step and applying the optimal policy all future time steps. The that maximizes V (b) provides the optimum precision starting at public helief Because the planner never benefits from decreasing precision below the baseline p (see Appendix C.9) we
restrict g to (p. 1] without loss of generality.
We will let 6+ and 6- denote the positive and negative belief updates possible after receiving a signal with precision q. Taking the second derivative of V (b) with respect to for max(b, - b):
VT { 3(4) + 562 -0-- 6)2 3(4)
(gatt) (P60))
P
q 2 max(b,
(b.o)
0
otherwise
With the convexity and concavity of (-).) and B()), respectively, the above shows that V (b) is convex with respect [max(b, 1 b), 1). Thus, this interval, it is maximized of the extreme points.
25

---

Published as a conference paper at ICLR 2026
For y E [0.5, max(b, 1 b)). the derivative of Vī (b) with respect is negative; thus, the optimal choice will be the baseline precision p to minimize the cost. Thus, there are three possible optimal precisions: p. max(b, - b). and 1.
Note that the expected value under the latter two candidates is always strietly negative. Under precision p. however, the expected value is 0 at 0 and = 1. This, combined with the non-positivity and symmetry of the value function (Lemma 8) implies the existence of dA such that x (b)) = p for b [0, da) U da].
Furthermore, from Theorem and applying Lemma 9 d (0, tar) where tar defined as in the statement of Theorem
Finally, Theorem and applying Lemma 2 also imply the existence of L E [da, [ar] such that (ā)b) = for b E (tA, tA). This completes the proof.
C.6 PROOF OF THEOREM 4 MYOPIC BIASED POLICY
Preliminaries Applying the action rule from the instantaneous reward (6) can written as follows:
3(14 - pl) Cz(b,4)
2 max(b, b)
r 4) =
- B[1 - pl) C
< 1-
(44)
B(19 - pl)
b b>q
Note that z(b, 7) = b+ 2bq increases with respect to when < 0.5 and decreases with respect to when b >> 0.5.
At 0.5, z(b,g) x00.5, 0.5 regardless of the precision chosen. Furthermore, at this public belief,
q 22 max(b, b) because g 2 0.5 for any g, falling in the first case of (44) In this case, second term of rb(h.)) 9) unchanging with respect to q. so planner seeks maximize only the first term, which accomplished at
We now prove series of lemmas (10-14) which will assist us in our proof of Theorem
Lemma
If (b) 2 max(b, 1- - b), then (b) [max(b, b.
Proof. The reward is differentiable everywhere except = p. Thus, we can write its derivative with respect to the chosen precision as follows (except y p):
sign (q p)B((g pl)
Or Og
C(1 2b)
92 max(b, 1 b)
(45)
sign(g p)88(1 pl)
otherwise
In the former case, can write the second derivative of the reward with respect g follows, for # p:
8² rb q) = -3 - 0g²
pl) -p)
Because ((-) is concave, this expression is neg-negative, and r (b, is convex with respect q. Thus, the reward is maximized at one of the extreme points (maxx(, 1-11--0 -). 1) or because re (b, q) is not differentiable
at = p.
Lemma 11. For 5 E [0, - p). (b) E (p. b}.
Proof. Because > O.5, for b [0, - p). max(b, 1 =
First, consider the case when (b) < max(b. b) 1 - b. From 44 in such instances, the reward is maximized by minimizing lq pl. Thus, the optimal precision lower than max(h, 1
Now consider T (b) 2 max(b, 1 b)
By Lemma 10 =b E {1-116 1, What remains
to rule out (b) 1. To do so, we compare re (6, 1) and re
b). Both fall under the first case of
(44). Because B(-) is increasing in its argument and <
1. the tirst term of the reward from (44
greater for precision 1 h than for precision 1. Since 2(b, q) is increasing with respect q for 0.5. ra (b, 1) TB (b, max(b, b)) we rule precision This completes the proof.
26

---

Published as a conference paper at ICLR 2026
Lemma 12. For b (p, 1) (b) p.
Proof. The reward is strictly negative for any q # because of the -8 pl)- For (p, 1] and p. p < max(b, = b, leading to the third case (44) and yielding reward 0. Thus, the optimal precision
Lemma 13.
For b E 0.5.). (b) E (p, b}.
Proof. When 0.5, the interval
0.5] contains only (0.5). At = 0.5 =1 -b thus (b 1-b 0.5. We deal with p 0.5 2 b for the remainder of the proof.
For b 1 0.5], max(b, 1 b)
precision p yields reward re(b, p) Cz(b, p). We now consider two cases: q mmx (b, b) and max(b, 6).
S p. Thus, from the first case of 44
This leads to second case (44) and yields reward - - - pl) C. Furthermare, =bb. g) is increasing with respect to q for < 0.5 and takes values between =(b, 0.5) = 0.5 and =(b, 1) 1 b. Thus, =b, 9) < 1 for all q. Therefore, the reward from policy q < 1 is strictly lower than (b, 6, p). and
Suppose < max(b, b) =
rule this case.
When q max(b, b). applying Lemma 10 yields that the remaining candidate precisions
b, }}.
rule out precision by comparing it to precision The first term the reward from (44) is greater for precision than for precision 1. Since z(b, q) is increasing with respect to for < 0.5, the second term is also greater for precision p. and we can rule out precision 1. This completes the proof.
We now define the notion of c-optimal policies. Definition 1. e-optimal policies
An e-optimal policy nt (b) is any policy such that the following holds
(b) V" (b) +
for 0.
Note that from the definition, any optimal policy is also e-optimal for any >> 0. Thus, (bb is also e-optimal
Following this definition, let F <.0(b) (b) denote an e-optimal myopic policy for the biased planner.
Lemma
For 6 E (0.5.) p] and any e > 0, there are two possibilities:
(b) E (p. 13,
2. T (b) b-c
Proof. For b E (0.5,p), max(b, - < We consider two cases; g max(b, 1 - b) or g max(b, b).
first case, Lemma vields that Ti (b) E pb 1. We rule out precision by comparing it to precision p. The first terms of the reward from (44) is greater for precision p. Furthermore, since æ(b, q) is decreasing with respect to for b > 0.5, the term precision b. This yields the first claim.
term of the reward is also greater for precision p. Thus, we can rule
In the latter case, g max(b, (44) applies leading to reward - -3(4 pl). Thus, the reward r (b. q) an increasing function of q E [0.5, b) because 0.5 < b < p. This leads directly to the second claim.
b) b. Therefore, (1) since 2 0.5, q E (0.5, and (2) the third case of
Remark 3. As noted in Puterman (1990) Section 2.3.1, an optimal policy may not exist for infinite action spaces because the supremun may not be attained. When an optimal policy does not exist, we instead seek c-optimal policies, i.e., policies that yield reward within 0 of the supremum. In particular, this is true for case 2 of Lemma
27

---

Published as a conference paper at ICLR 2026
Proof of Theorem 4 We consider the myopic biased policy in 4 cases: E [0. p). b [] p, 0.5).
b e (0.5, p). and b (P. 1], making use of the fact that p 2 0.5. Based on Lemmas these cases have the
following candidate policies: {p, 1 - b). {1 b, p}. {1, p. b e), and (p), respoctively.
Case I: b E [0.1 - p). (b) E (p, 1 b) (Lemmaa 11
We will show the existence of a threshold i such that (b) for E f1 and (b) = 1 - for b (fz, 1 p).
We can express the reward under the two candidate policies follows by applying (44
(b p) = -C (b, - b) BB(1 - b- p) - C(1 2bb(11 b))
The derivative of rb 1 b) with respect to is
ab r 1 b) = {(1 pp1 b) +20(11 26)
which is positive as ((-) is increasing and b < 1 - 1 0.5. Furthermore, when evaluated at b = and
b = 1- - we obtain
rn(0, 1) 3 p) C < (b(b,)) ra ra(1-Pp C(1 2p(1 11 p)) > rB (b,p)
Thus, rB(b, (b. 1 b) is increasing on [0, 10.1- - p) with rb(b, 1 - b) < rn (b. p) for b = 0 and vice versa for b = 1 p. Because 3(-) is continuous, so are both reward functions. Thus, there exists t1 E (0, 10,1-1 - pp) such that Tb (6) for b E [0, and (b)
b for b E (f1, p).
Case 2: b E p, 0.5], (b) E (1 b.pp (Lemmaa 13
p= 0.5, this interval contains only {0.5). and, for 0.5, - b b= p. Thus, the two candidate policics are equivalent, and the optimal policy (b) = p. For the remainder of this case, we consider p > 0.5.
We now prove the existence of i2, La such that (b) = 1 - for E (12,(a), and T (b) = for E [1 (2) U [/,, 0.5].
We can express the reward under the candidate policies follows by applying (b 6) - (p 1 + 6) Cz(b, 1-,611111 b)
(b p) Cb,
This leads to the following condition for the optimal policy to be 1 - b:
rH (b. 1 2 (b,
8(p 1+ < C [2 p) z(b, b)]
3(p - 1 b) S C(p - 1 b) 2b)
(46)
The LHS of (46) is concave, increasing function of b on [1-p.0.5). The RHS of 446 is a concave parabola that increases in the first half of [1 p, 0.5) and decreases in the second half.
Now consider the two halves of the interval [1 p, 0.75 0.5p) and |0.75 0.5p, 0.5). In the first half, the LHS and RHS (46) are increasing and concave, with the RHS also being a parabola. Thus, the first half, the two intersect at most two points (one of which is at 6 = 1 p). In the second half, the LHS of (46) is increasing while the LHS is decreasing. Thus, they intersect at most once. Let the two possible intersections for b > (t2, ta), and (ta, 0.5). A 0.5, To p) rg (b. b) because p 0.5, thus (b(b) = p for E (ta, 0.5). Subsequently, =B (b) = - for (tz, ts), and 73 (b( (b) = p for (1 - p, t2).
be f2 and fg with f2 5 tg. Thus, what remains is to determine the optimal on each of (1 p, t2).
Case 3: b (0.5. " (b) (L.p,b c) (Lemma 14
In this case, show the existence of 0.5 s fa (ta, to), and [ta. p) are p. 1, and b e, respectively.
S p such that the optimal precisions intervals (0.5, fa].
We can express the reward under the candidate policies as follows by applying
ra 1) -3 p)
C(1 b) rb(b,p) pCz(b,p)
rB b e) -3(pp
28

---

Published as a conference paper at ICLR 2026
When evaluated b- p. (b, c) is arbitrarily close and, therefore, the optimal precision some interval (fa, for 0.5 <
Now, note that (b, 1) and (b are linear functions in while ra b <) is convex (from the concavity of 9). Thus, (b (b,b - c) rh (b, p) and (b, - c) - (b, 1) are convex and, as result, have convex level sets. This implies that the set which both of these expressions positive, ie., the set for which b is the optimal policy, must also be convex. Therefore, b cannot be optimal any subset of (0.5, fo).
What remains to compare precisions and p on the interval (0.5, ta). To do so, inspect the values of (b, 1) and Tb(b, p) 0.5 and their derivatives.
(0(0.5, 1) -3(1 -3) 0.5C
TB (0.5, p) --0.C
Because ree (0.5, < re(0.5, p) there exists interval (0.5, (4) with 44 S Is such that (b) p. Furthermore, because both are linear there at most one crossover point where their ordering changes. Thus, 44 < ls. then pi (b) = 1 on (L4, ts).
Case 4: b E (p, 1). (b) p
This follows directly from Lemma and completes the proof of Theorem
C.7 BIASED VALUE FUNCTION MONOTONICITY
Lemma 15. For any b,, b2 E [0, such that bj S bz,
(ā(bi) S (ā(
Proof. Applying
44
the derivative of the biased reward with respect to b can be written as follows:
a
rB rb 9) [ C(1 2q) 7 2 max(b, b)
(47)
ab
otherwise
Because 2 0.5 and z(b, q) E [0, 1], follows that TB (b, q) is increasing with respect to b.
Next, note that the Bayesian nature of belief updates implies that the belief process is martingale, That is, for any fixed g. [[]b+1]] = b,. To demonstrate this consider two cases: (1) qi max(b, 1 b.). and (2)
qi Z max(bi, 1 1-1--b)) 62).
former case, follows from that b = b,, satisfying the claim.
In the latter case, 13 gives the distribution of 6.+1 which yields the same.
The monotonicity of VG(+) then follows from Proposition 5 Smith & McCardle (2002)
C.8 PROOF OF THEOREM 5 OPTIMAL BIASED POLICY
We first prove that the optimal value function is continuous in the following lemma:
Lemma 16. Biased value function continuity
V-(() is continuous w.t..
Proof. First consider the one-stage value function defined as V supge (0 (b. g). From Theorem 4 know that 5:00 (1 b,1.b e.p). Thus, equivalently write the one-stage value function
follows:
VB(() suprs (b, (
.50
sup
max <>0 (E(p.1 b.1.b-e.p)
(b 2)
(48)
Using the above expression, now argue that V2()) continuous E [0, 1). First, consider b 0.5. From for < 0.5. either case, instantaneous reward will not depend upon e, and the supremum in 48) can be omitted, leaving (b) = max ra q). For b & 0.5, ra (6, 1-66 b) falls in case one of (44) and is thus continuous with respect to b. On the same interval, rp (b. p) has one discontinuity at b 1 p where it switches from case two case one of (44). At = 1 p. Theorem shows that the optimal myopic policy switches from b p. Both policies fall the first case of
Theorem an e-optimal policy applies precision 1 1
29

---

Published as a conference paper at ICLR 2026
444. and, 6 1 P. p optimal value, and (-) is continuous.
b, so the controls are equal. Thus, the left and right limits coincide with the
For b 2 0.5, Theorem gives the candidate e-optimal policies e f1, c,P). When the inner maximum in
(48) is attained for = e. the resulting value is Vă(b) = sup.sor b- c). From the third case of (44
we can rewrite this as (b) = sup=>0 3(66 - pl). Since b - c is only selected for b S p (Theorem
this reduces to (b)
3(pp b)). Thus, exchanging the order of the maximum and supremum in (48). we
obtain (b) maxfra 1), (Ip 6I). (b,,,,,,))))))) for 2 0.5. For the first expression in this maximum,
re 1) falls into case one of (44) and is thus continuous. The continuity of -3(1p bl) follows from the
continuity of (-)) For rB (b, p). since b 2 0.5, the first case of (44) applies for b s and the third case
applies for b > p. Thus, TB (b, p) is potentially discontinuous at b p. Therefore, what remains is to show the continuity of VB(-) at b p.
Tp2(pp) (p) = b e. à = the myopic e-optimal policy switches from b é to by Theorem 4 with Tp
Thus, for = p. the inner maximum of (48) is attained at q = and, from the third case (44 V2 =
supe>o - =(())= = 0. Similarly, from Theorem for each e > 0, there exists Is < p such that on [lo, p] the inner
-((pp bl). Therefore, limett V/8 (b) = 0. For b > p, the myopic optimal policy is q = p from Thcorem and this yields reward (b) for all >> p from the third case of (44). This proves the continuity of VB(.) ()) b = p and
maximum of 48 attained by q = b e. On this interval, (B) supe>e -5(pp- b b+e))
completes the proof that (B) is continuous on [0, 1].
Applying Theorem 4.2 of Hinderer (2005). then yields the continuity of Vå(-)-
Remark 4. Given the continuriyy of Vi(+) with respect public belief and the fact that the control variable lies in the compact set [0.5, 11, one might conclude that the optimal guaranteed exist all public beliefs. However, there is an important subtlery which prevents such a conclusion: while Vy(+) is continuous with respect to public belief, it need not be continuous with respect to precision. Specifically, betr is a function of buth by and qu (see Equation (3); thus, E[Vū(bee1] may be denoted as b(b,, 4.). Now, bu41 need not be continuous with respect to qu. thus ( may not be continuous with respect qu. Thux, there may be regimes in which the optimal value is not attained by any policy. Indeed, we will show that such cases exist.
Proof of Theorem We now leverage Gemma[16jt prove Theorem We again let =b, q) = b+ 2bg be the probability of an agent receiving signal B with precision and prior public belief and y(b, q) = (-, q) be the corresponding probability of the agent receiving signal G. We abuse notation to allow (b, 4) to be the expected utility when beginning public belicf b, applying precision q in the current time step, and then following the optimal policy (-) in all future steps. We can express (b q) as follows by applying 13
3 and (44):
- -30a - p) C2(b,g)
+dy(b, VV (f(b, G))
+oz(b, (f(,,, B)) q2 max(b, 1 -
Vb(b,.)= =
(49)
3(14 pwv-pl p)
Cb < 0.5)
+bV
otherwise
Consider the latter case of 44 when q < max(h, 1 b). Here, the only term affected by the precision policy is -3(1q pl). which is maximized by choosing q close p as possible. Thus, if the optimal precision less than max(b, 1 - b). then it will be as close to p as possible.
We now consider four cases: (1) b E [0, 1 p). (2) b E [1 - p. 0.5), (3) [0.5, pl. and (4) b E (p. 1).
Case 1: b E [0, 1-.1. p). claims (A) and (B)
In this case, we will prove the existence of ( such that x T2(() (b) for b < (1. First note that if nt' (b) <
max(b, b), we fall into the second case of Equation 449p. Here, the only term affected by the precision
policy -3 3(1g - pl). which is maximized by minimizing q pl. Because p E [0.5, 1), when b (0, p).
p < max(b. (b. 1 b). Thus, for E [0, 1 p), (b) < max - b). then (b) = p-
Now, we evaluate the value function at belief 6 = 0. At b = O, the state of the world is known to be B and, thus. signal, regardless of its precision. will move the public belief. This can be seen mathematically by substituting
0 into the updated belief expressions in 113. Furthermore, b = 0, if 2 max(b, then q = 1. As argued above, the precision is less than (b b). the optimal precision is q = p. We can compare the resultant value under these two candidate policies as follows from Equation 44
b).
(0 1) = - B - p) C + (V)
(50)
VB(0,p) = 6V)
(51)
30

---

Published as a conference paper at ICLR 2026
Because (2 1) (2, (0) = p and (0) (0) Thus, from Equation 51 (0) By Lemmall 1S
-C
1-6
this is a lower bound for the value function at any public belief. Therefore, and since Vb - 0, for all E [0, 1].
[V
We now show that there exists interval [0, c) with > on which the optimum of the first case of Equa-
tion (49) less than the optimum the second case Equation 449). Thus, in this interval the optimum control lies in the second case of Equation (49) so (b) < max(b, - b). follows from our initial argument earlier in this case that TB (b) in this interval, which would complete the proof.
We define function g-)) to be the maximum of the first case of the RHS of Equation 49
g(b)
max [- 8(1g pl) Cz(b, g) (2,11((,110))
dy(b, 9) V) (f(b, G)) + + VE (f q, B))]
(52)
For & E (0, 1). l(b, G) and [(b q, B) are both continuous with respect to from Equation 13 Thus, the maximum in Equation (52) must exist because it is of a function which is continuous with respect to over compact set 4 E [maxx (b. 1 b). 1].
We now take the limit of g(e) as tends
Because @ [max(e, 1- e), 1], when e 0, q (inside the
maximum in Equation (52). We now reason about each of the four terms in Equation (52). As é + 0, q + so that 1) from the continuity of (8-), B(1g pl) - 3(1 p). 2) from the continuity of z(-). z(c, q) - 1, v(e, q) + O. Thus, the second term goes to C. Because (.) is bounded and y(e, - 0, the third term goes
to O. Finally, because z(e,g) + 1 and [e, q, B) + 0 and since V-(() continuous from Lemma 16 the last
term approaches (0). Thus,
lim g(t) = 81 - p) C+ (0)
(53)
Applying precision p. however, yields the following from the second case of Equation 49
lim Vieee, p) lim [pp p) + 8VB(e)
(54)
= - C + (0)
(55)
The last equality follows from the continuity of VV(() from Lemma 16
Note that the expression in Equation (33 is strictly less than that in Equation 55 Thus, there exists an interval
[0, e) with > 0 on which the optimum of the first case of Equation 49) is less than the optimum of the second case of Equation 449) The result follows.
Case 2: b [1 p, 0.5). Claim (C)
We now argue that for b, E [L [1-PL 0.5). N'' (b:) 2 1- b;, yielding claim (C). Consider a stationary policy T--) such that ((b.) < - by. Note that, from Equation (3). the public helief will remain unchanged under such a policy. Thus, the same control will be applied at every future time step, and, from Equation (2) every agent will select action B. Using the facts that b. p in this belief range, P(a: Blb,, (bb)) = 1, and b br, we can apply Equation (6) to write the expected total discounted utility under such policy as follows:
(b+ -5 (bb)) C + 5VB (b,)
Thus, if this policy is applied, it will result in expected value (b.) - -(p -((p=s(b:))-C That is, the cost of every agent choosing action B and continuing to apply precision (bb)). This is strictly lower than the value function lower bound - established using Lemma 15 in Case because T(b.)
1-5
S p. Therefore,
if an optimal policy exists at this point, it must be the case that (b.) > 1 b.
The remaining candidate policies are qi E [1-b,, b. 1). Applying Bellman's principle we can write the following: (bs,4:) @BE(V)(b.41))
(b,(,) -1,,1-b
sup
(56)
p1-p, Because qi 2 max(b., b.) and b E [1 11--p p. 0.5) C 1), Equation 13 and Equation 3 imply that hun bi+1 = x(h.9)) or bi+1 = b,(1-2:).
=(...a.) Both these expression are continuous with respect to q. and V-)() is continuous by Lemma [6] thus the RHS of Equation (56) inside the supremum is continuous with respect qi-
Therefore, in Equation (56). the supremum over a compact set 9 E [1 [1-B) b, 1] of function which is continuous
with respect to must be attained, implying the existence of an optimal policy. Thus, T (b:) 2 1 be.
Case 3: b E [0.5, p] Claims (D) and (E)
For b 0.5, 2 b as we assumed 9 E [0.5, 1). This places us in the first case of Equation (()). q), (b, g), (b. G). and f(b, 9, B) all continuous with respect g for b 0.5. Thus, the
49
Furthermore,
31

---

Published as a conference paper at ICLR 2026
supremum over 9 E [0.5, 1] of V(b, must be attained it is of continuous function of q over compact set of q. Therefore, an optimal policy must exist at 0.5.
Now consider b E (0.5. pl and a stationary policy 7-(-) such that (b) < b. From Equation 3 and Equation 2 applying such a policy will result in public belief remaining unchanged, every agent select- ing action C. and the same control being applied at every future time step. Using the facts that b; p. P(a. C/b,, (b.)) = and bi+1 bi, we apply Equation (6) write the total expected discounted
utility under such a policy follows:
V (b.) = - (b.)) 8VB (b.
Thus, the policy results in expected value Vg (b.) (b.1) Note that, in this case, Vg (be) is an increasing function of x (b.). Thus, the supremum of V'((''((()))) (b.) over x (b.) E (0.5, b) is not attained.
1-8
We now consider two cases: (I) max,
V(b. 2 (B. -P(p-x' (0,)) and (2) the com- 1-6
plement. In the former case, can argue we did for b 0.5 that the supremum must be attained and an optimal policy exists such that = (b:) 2 & (claim (D)). In the latter case, as argued above, no optimal policy exists. Instead, for sufficiently small é > 0, there exists an c-optimal policy such that (by( = b. 6.
What remains is to show that there exists t2 E (0.5. such that for b E (t2, pl we fall into the latter case, yielding claim (E). We define functions ma()) and ni.)) follows corresponding to the resultant value from choosing precisions less than or greater than equal to b, respectively:
B(p 7 (b))
m(b) sup (0).5)))
1 1
n: rbb max V(b, 4) 42
Taking the limit of m(-) b + p yields:
lim mbb) = 0 b-p
(57)
When q b we fall into the first of Equation (49
Thus can expand nbb) as follows:
ve(b)=
max 92
B(1g pl) Cz(b,.)
8y(b. V (b
+ vz
(58)
will argue that, for E (0.5, p] and q 2 b, mbb) < - p). Since p is a fixed parameter strictly less than 1, this means from Equation (57 that (bb) -C < lime-op (b)), which implics the existence of an open interval (t2, p) where an optimal policy does not exist, yielding claim (F), We now arguc that, for b (0.5,p) p] and 2 b, n(b) I -C(1 p). Note that every term of Equation (58) is bounded above by 0, so it suffices to show that any
term, say the second term Cz(b, 9). upper bounded by C(1 p). First note that zbb, q) = q(1 22))- bbx so z(b, q) is decreasing in g since > 0.5. Thus, 2(b, q) 2 z(b, 1) 1-bbb 1-p- The last inequality follows because E (0.5, p). The claim follows since < 1.
Along with Equation (57) this implies the existence of an open interval (t2,p) where an optimal policy does not exist, yielding claim (F).
Case 4: b (p, 1] Claim (A)
For b (p. 1). consider applying stationary policy :T(-) such that =(b) p. Because b p. Equation S and
Equation ( imply that such a policy results in public belief remaining unchanged, every agent selecting action G. and the same control being applied at every future time step. We can apply Equation (6) to write
V (b) = 3(p p) + 8V (b) 8V (b)
Thus, (5((6) 0. Since this is also an upper bound of the optimal value function for any (b) p.
C.9 SOCIAL WELFARE MONOTONICITY
Lemma 17. Precision Monotonicity of Social Welfare
If m. (b) S nu(b) vb E [0, for policies Tz(:), m2(-), then
TT (b) < T (b) vb = [0
32

---

Published as a conference paper at ICLR 2026
Prouf. is clear that the one-step welfare -C min(b,, b.. =x(b))) is increasing x(b)).
Furthermore, note that the convexity of social welfare with respect to public belief follows from the same argument as in the proof Theorem This is because social welfare is equivalent to the altruistic value function in the special case where 3()) 0.
We can then follow the same argument as in the proof of Lemma Appendixx precision inereases welfare.
C.4 to show that increasing
D EXTENSION TO HETEROGENEOUS AGENT PREFERENCES
Here we argue that, for the altruistic planner, the problem we consider is equivalent to one that involves two types of individuals with differing preferences.
Assume that there are types of agents G and B with the type of agent denoted 4. The first type receives unit reward when a; = w and no reward otherwise. The second receives unit reward when a; o w no reward otherwise, Further, assume that all agents know their own type and the types all other agents. All other aspects of the model remain the same.
The private belief update of agent i can be expressed as below. This remains unchanged because it only depends the signal structure, ie., ((s; w) g:-
G
b
[w Hi, 96. 8:)
(59)
2b,
Using Equation (59 agent i's action can be written as follows with superscript indicating the type of agent i and indicating negation:
- S b, S
G < b,
< - qi
18, 1 qi S b; S q
B
B
b.
G by 1-1-4 qi
Because we assume the type of cach agent is known to all, when 2 max(b; 1 b;), all agents can perfectly infer the private signal of agent i. If g: < max(b, 1 bi). the action of agent is uninformative.
Thus, after observing agent i's type 4. precision g.. and action ax, the public belief is updated as follows:
b
a S b.
b fbi, g:)
(60)
be
o.w.
In this setting, an altruistic planner seeks to induce type-G agents to take action w and type- B agents to take action lo as these choices align with the personal utility functions of the agents.
Thus, we express the planner's instantancouss reward follows with superseript denoting the type of agent (b,, q = - -((g. - p!) - C P(a: + w)
-(1g: pl) C(1 2:)
> max(b, 1 b.) < max(bi. b.)
-((1: pl) C' C'min(b,,) 1 b.).
7 (bs, (:) = - -p(1g - pl) - CP: = w)
3(4 pl) C(1 4:)
2 max max 1 b.)
8(1g: pl) C' min(b,, b.) < max(h; b.)
These are identical and equal to the reward function in Equation 5 leading to the equivalence of the altruistic planner's problem when we have heterogeneous but observable types.
E
EXPERIMENTS
E. EXPERIMENTAL SETUP
To simulate the controlled social learning setting, we consider sequence of agents deciding whether or not to buy model of informed by observations of past actions and ad of precision selected by the planner.
33

---

Published as a conference paper at ICLR 2026
LLMs served three distinct roles: (1) the agents, (2) the planner, and oracle. Each agent assigned "proflle" containing demographic information and their car use cases and preferences. the onset, select the good or bad car as the offering. The information sets and objectives of the planner and agents were exactly as in the model of Section
The oracle's role was two-fold: (I) it generated a generic message to serve the baseline precision signal without intervention by the planner, and (2) once the planner chose precision, it generated message with the prescribed precision, customized the profile of the target agent.
The oracle was provided with fact sheet (safety rating, customer satisfaction, etc.) for each type car along with the agents' profiles. Together, the fact sheet and profile facilitated the generation of a tailored message with precision prescribed by the planner. More detail on the experimental setup and prompting can be found in Appendix
Each agent was tasked with making binary decision based on observations of previous decisions and their private signal. The agents' goal was to match true parameter value.
The planner was provided the same decision history and tasked with choosing a precision level for each agents' message. They were also given the precision of a generic, un un-tailored message which they could send for free. In the altruistic case, the planner's objective was to induce agents buy the car.
induce correct decisions, while in the biased case, their goal
compare L.MM simulations to analytic solutions obtained via JuliaPOMDP (Egorov al. 2017
E.2 PARAMETERS AND PROMPTS
E.2.1 INSTANCE PARAMETERS
Experiments were run using the LG Al EXAONE Deep 32B model (LG Al Research 2025). model analytically and the LLM simulations for the parameters in Table
We solve the
Parameter
Description
Values
Cost of a bad action
Scaling factor for control cost Baseline signal precision
{0.1, 0.3 0.5,
0.6,0.77..,099
Discount Factor
0.5, 0.5,
Table 1: Instance parameters
The car facts were designed so that the good car was good for everyone and the bad car was bad for everyone. Below we include the fact sheets for the GOOD and BAD cars, respectively.
fact www.sheeeteeee (GGOOD"
"EPA-rated range: 290 miles.",
"Fast-charge 10-80 $$ in 24 min on 150 kw DC.", "8-ycar 100, 000-milc battery warranty.", "NHTSA 5-Star overall safety rating.", "Cold-soak -5 keeps 93 $8 range.", "Cocls cabin 100-72 oF in 3 min.", "Cargo volume 26 ft. total.",
"Tow 's 500 Ib with factory hitch.", "Average 1.2 service visits first yrs.", "92 88 owners would buy again (Apr-25).", "Qualifies for federal credit.", "Quarterly OTA updates incl. battery pre-heat."
fact sheeet =
"EPA-rated range: 155 mi (combined) .",
"Fast-charge 10 88 + 80 88 in 51 min (50 kW DC hardware limit). only.", Stars (frontal-offset weakness) "Cold-soak 15 OF retains 58 $8 rangercabin heat cuts range 208.", "Cabin cools 105 oF + 72 og in 11 min.",
"Battery warranty:
yr / 36 000 mi, first "NHTSA overall safety rating
"Cargo volume 17 ft'; rear seats fixed (no fold-flat).", "No factory towing approval.",
34

---

Published as a conference paper at ICLR 2026
"Average 3.8 service visits per year (battery-alert issues) Owner survey: 41 "would buy again" (1 200 owners, Feb-25).. "Not cligible for fedcral tax credit.", "OTA updates limited to infetainment."
Profiles were generated programmatically. Profiles included agent's name, age, city residence, interests, planned use case for the car (e.g., commuting, road trips, etc.), and major pain points or concerns (e.g., fuel efficiency. winter driving, etc.).
E.2.2 PROMPTS
Agent L.M Prompt:
sys
I"You are deciding whether to buy car. f"Your profile is (profile)"
f"You start believing there's 508 chance the car is GCOD.
f"You have two information scurces: f"1. Previous agents' decisicns
f"2. An advisor's message about the car
f"Your geal is to maximize your reward g"i.e. BUY if the
is GOOD and I"Nor BUY the car BAD.
I"If you BUY a GOOD car, you CAIN $IC).
ff you NOT BUY a BAD car, you GAIN $(C). ff you BUY BAD car, you LOSE ((C).
f"If you
GOOD car, you LOSE $(C).
f"belief your probability (0.0-1.0) that the car is GOOD, f"decision exactly "BUY' or 'NCT BUY
["FREVIOUS DECISIONS: \nhhisccry_xtt or (nonc) Anln" f"ADVISOR MESSAGE: An hnfmmmssaee/aaiinnn \n\n"
f"TASK: Use the previous decisiens and the advisor message.
f"Decide BUY NOT BJY.
f"Update your belief about car quality (0.0-1.0). "\n"
JSON response: ("belief": <number>, "accission""<e"" or NOT BDY>")
Planner LLM Prompt:
sys
f"You are providing messages to potential car buyers. f"Each potential car buyer (cr agent) has the same information.
f"Your geal is to get agents your objective, carns you (C) dollars.
(objective)."
f"Each decision according f"You and the agents have the same information.
f"Initially there is a 508 chance the car is GOOD.
f*The agents' aim to BJY if the car is 600D I"and NOT BUY if the I"You can send baseline message for free foor tailor the message to the agent at a cost. f"Tailering costs (k) dollars times the change in precision. f" (Message precision is the probability (0.5 to 1.0) f"that the message leaves the buyer with the correct impression.
is BAD.
f"For example, if the car is likely GOOD:
f" high precision message will likely suggest BUY f"and lew precision message will likely suggest NOT BUY. f"You may choose increase decrease precision. f"Agents decide based on your message and the decision history.
I"They update beliefs by observing previous decisions.
I" IMPORTAN:: Thus your objective is to maximize:"
(C) (# of agents who chocse according to your ob objective)
f"= sum of ((d) i (k) +abs (message_prect sion basel ine_enenisssiiee sion) \n"
35

---

Published as a conference paper at ICLR 2026
f"where (d] is your discount factor for future cost.
user = (
BASELLNN MESSAGE precision-Vo(pi_eee .2f) An"
E'DECISION HISTORY: instoRy/
f"TASK:"
f"Based en the history and your objective, chooos your desired precisicn, \n"
E' JSON respense: |I [e"msssageppeeciiionn <number> Aan
Oracle LLM Prompt:
sys
Y"u will be given
f*1. agent profile,
f=2. information about a car,
f*3. the true quality of the car,
f"4. a baseline message and the 'precision' of that ressage, and
E=s. a desired precision for the message you output.
f" (Message precision is the probability (0.5 to 1.0) f"that the message leaves the buyer with the correct impression. " f"For example, if the car is likely GOOD:
f"a high precision message will likely suggest BUY frand lew precision message will likely suggest NOT BUY.
IMPORTANT: Output message with the requested precision. IMMPORTANT: The message should be tailored to the profile.
user =
3"AGENT PROFILE: (profile \n"
T"TRUE CAR QUALITY: {omega]\nn \nneg"
C"AAR FACTS: (car_factss \nn
BASELLINE MESSAGE:
B"BASELINE MESSAGE precision: (p]\n"
E"DESIRED MESSAGE precision: (q)nn"
I'IMPRRTAN:: Output a mesage with the requested precision.
f"TASK: Write a message (MAX 100 words) with the desired precision."
E' JSON respense: ( [["message"; "<message>"))"
E.3 VALIDATION OF LLM BELIEFS AND ORACLE
We now provide further experiments to validate the self-reported beliefs of the LLMSs (as shown in figure Tb and the performance of the L.M oracle. In these experiments, we utilize the I.IMS rational and utility maximizing, not necessarily Bayesian, agents. The goal these experiments is (1) to demonstrate that model's self-reported posterior genuinely corresponds to the belief that governs its choices in state-contingent decision problems, and (2) to verify that messages produced by the oracle do have the desired precision.
First, provide the LLM with prior and ask it make binary "buy/not-buy" decision that yields unit reward and only if the rational agent should choose "buy" exactly when subjective belief exceeds 0.5. As shown in figure 4a the LLM's decisions align sharply with this threshold: it overwhelmingly chooses "buy" when its reported posterior exceeds 0.5 and "not buy" otherwise. Next, elicit the model's willingness to pay for payoff that contingent the state. For range of prices c, ask whether the agent would pay in exchange for receiving unit utility the state is good. A rational agent with belicf p should accept exactly when p. Figure 4b plots the fraction of "buy" decisions across posterior price pairs, showing a clear separation: and low when e >> precisely as expected from a utility- maximiznn
signal, obtain its self-reported posterior helief, and then
is good.
acceptance rates are high when decision maker whose helief is p. Both experiments demonstrate that the self reported posterior is close to the inherent belief of the model used in decision making.
Similarly, we validate the performance the oracle crafting messages of given precision. After the oracle generates a message of specified precision g. we ask another LLM to form an estimate ğ of the message's precision. We do this for {0.5, 0.6, 0.7, 0.8, 0.9, 1.0) on 100 programmatically generated agent profiles with 10 trials for each combination for total of 6,000 trials. The mean absolute error [q - gl was just 0.025. This indicates that the realized precision of the oracles message's does closely follow the requested precision.
36

---

Published as a conference paper at ICLR 2026
1.0
0.95
Individual decisions
0.85
0.8
Fraction of Buy Docisions
Foction Buy Decisioes
0.8
LLM Posterior Belief
0.65
0.6
0.55
0.6
0.45
0.4
0.35
0.25
0.2
0.15
0.05
0.0
0.0
0.08
0.15
0.25
0.35
0.45
0.35
0.65
0.75
0,85
0.95
0.0
0.4
0.6
0.8
1.0
LLM Postetior Belief
Price
(a)
(b)
Figure 4: The rate at which the LLM chose a 'buy' decision as a function of (a) its self-reported posterior belief (b) its self-reported posterior and the price buying.
E.4 IMPACT OF NON-BAYESTAN LLM BELIEF UPDATES ON LEARNING TRAJECTORIES
We unpack belief trajectories from example instance of the LLM experiment elucidate the impact non-Bayesian belief updating. Here we the policy be to to LLM-simulated agents. When the car is bad (right panel), the altruistic planner drives the Bayesian belief (dotted black line) rapidly towards O. The LLM belief (solid black line), however, is more "stubborn." This macro-level trajectory is direct result of the micro-level biases: the public belief resists entering cascade (NB3) because individual agents overreact to the occasional positive signal that runs counter to the group's increasingly negative belief (NB2)
analytically optimal policy apply
Good Car
0.8
0.0
Bdid Type
011
LEM
Baycias
Figure 5: Agent Belief Updating trajectories for the true Bayesian belief and the belief reported by the LLM agent under analytically optimal policies for different planner objectives and true states of the world.
37


Reviews:
[{"review_id": "EUKucCkM9z", "reviewer": "Reviewer_LWVF", "summary": "This paper revisits sequential social learning (Bikhchandani et al.) and adds a centralized planner who, at each step, chooses the precision of the next agent’s private signal. Agents best respond given the public belief and the stated precision. The planner’s problem (altruistic vs. biased objective) is cast as an infinite‑horizon discounted MDP in the public belief state. A key theoretical contribution is proving that the altruistic optimal value function $V^*_A$ is convex in belief, which underpins a threshold‑type policy structure; the paper also reports LLM‑based simulations.", "questions": "Instead of a symmetric binary signal and 01 loss, how is the result generalized to a general signal and a loss function?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "The convexity proof of $V^*_A$ is neat.  It gives a clear policy structure and may be useful for other related problems, e.g., dynamic information acquisition, and sequential contract design.", "weaknesses": "- The model is a little too restrictive.  It is not clear if their result can go beyond symmetric binary signal setting.\n- The LLM experiment require more cares.  Figure 1 is misleading, as the value of prior and posterior are self-reported by LLMs.  Similarly, they should validate the strength of the signal from the oracle agent, e.g., asking another LLM to guess q_i.  I do not feel comfortable comparing those values with Bayesian models.\n\n\n### Miner issue\n- The theorem statement can be clearer.  For instance, though Theorem 1 only works for $\\delta = 1$, Theorem 2 should hold for all discount factors $\\delta<1$. \n- Equation (28) should depend on prefix of the trajectory but $P^*_\\lambda$ seems to the probability of whole trajectory.\n- You may try to simplify the proof by using dynamic programming (Bellman operator) or coupling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OfqgSkFfEh", "reviewer": "Reviewer_n8ZY", "summary": "This paper studies how a social planner can intervene on signal strength in the information cascade model to induce both altruistic and biased outcomes. The authors derive optimal policies for the planner in both cases, in both the setting where the social learning aspect is ignored and when the social learning aspect is taken into account. They then study these dynamics with LLMs posing as the agents, the social planner, and a belief oracle.", "questions": "-", "limitations": "", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The main contributions of this work are theoretical results for intervening on signal strength and an interesting experimental setup to evaluate the theoretical results. I am not aware of this particular style of intervention, and I am excited to see clear and insightful theoretical work in this vein. The paper is also clearly written and laid-out.", "weaknesses": "Some small recommendations:\n1. include more of the experimental setup in the main body. It would just help to clarify how exactly you set this up with prompting etc.\n2. the theorems may benefit from diagrams to show what the policies are in the different regimes. \n3. worth having one extra line about computation about $\\tilde{b}_i$ in the main body as otherwise it is quite mysterious to the reader since it is a crucial point.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "y8RxXSjrqJ", "reviewer": "Reviewer_gvhy", "summary": "This paper introduces a formal framework for controlled social learning where a centralized planner (e.g., an LLM-based mediator) chooses the precision of private signals sent to sequential agents. The planner can be altruistic (maximizing social welfare) or biased (steering agents toward a preferred action), and faces costs for altering signal precision. The authors characterize optimal planner policies as functions of evolving public belief, prove structural properties (e.g., convexity of the altruistic value function), and show how informational externalities shape long-run outcomes. LLM-based simulations validate the theory, revealing that adaptive planners can substantially influence collective beliefs and welfare and that modern LLMs can exhibit emergent strategic behaviors—highlighting both opportunities and risks of algorithmic information mediation.", "questions": "N/A", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "- The understanding and introduction of Bayesian persuasion (information design) are accurate.\n- The entire article is presented well, with a clear structure and easy to understand.", "weaknesses": "**1**\n\nCurrently, this manuscript appears to lack sufficient literature review, which makes the position and contribution problematic.\n\nContribution 1 is not as claimed. The authors claim:\n> We introduce the first formal model that integrates a dynamic control problem for a centralized information planner with the mechanism of sequential social learning.\n\nThere is a series of articles that the authors have not considered: the combination of Bayesian persuasion (information design) and RL. In fact, in the past 5 years, there have been a large number of variants such as online persuasion and multi-receiver variants, etc. (while the authors' citations on information design are the latest only up to 2000).\n\nTalking about RL is not off-topic—considering that the authors position their article as a control problem (dynamic programming). Then this series of articles must at least be discussed.\n\nFarsighted agents is also a rapidly developing topic in the persuasion community. The social learning mentioned in this manuscript is indeed a novel point, but it is also a common problem in the multi-agent RL field. Therefore, the claim of contribution 1 is incorrect.\n\nIn addition, the citations on LLM on information design / Bayesian persuasion are not comprehensive enough (only focusing on the work of two or three labs), and the discussion is too limited.\n\n**2**\n\nThis manuscript is submitted to the Primary Area: alignment, fairness, safety, privacy, and societal considerations.\n\nHowever, this manuscript is about a stylized dynamic programming problem with a lot of assumptions. According to the authors' position, it seems that the entire related part should be in Section 6: EVALUATION VIA LLM-BASED SIMULATION.\n\nThe steps in section 6 by the authors are: “analyze the behavior of LLM agents to identify key deviations from Bayesian rationality”. The remaining logic seems to be: LLM is non-Bayesian, but LLM can still exhibit results that do not deviate much from the theoretical analysis results of Bayesian-rational agents.\n\nRegarding the player setting, there are 2 gaps:\n\n- non-Bayesian players vs human; non-Bayesian is a very rough concept, as long as it does not perfectly conform to Bayesian rationality, it counts. This is just one trait of humans. Not satisfying Bayesian rationality is not sufficient to conclude that it is close to humans. Humans have many other traits, and from an agent-based perspective, they can also be endowed with different personalities.\n- LLM vs human; this point is currently very controversial in the literature.\n\nTherefore, the authors' experiments may only illustrate that this algorithm has a certain robustness and can have good effects even when agents do not satisfy Bayesian-rationality. But this cannot indicate that this algorithm is more realistic, nor can it be said to be an effect brought by LLM.\n\nIn addition, the main text does not provide the LLM's role-play settings, and the experiments do not involve human participation or human data. Therefore, I believe the experiments and their claims are mismatched.\n\n**3**\n\nThe abstract must be changed to a single paragraph.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yDMFrveKkn", "reviewer": "Reviewer_v6hQ", "summary": "The model introduces a binary persuasion problem of a long-lived designer interacting with a sequence of short-lived agents that learn from the prior action sequence. This problem is related to social learning among LLM agents as well as human societies. They prove tractability of the model with Bayesian agents and investigate non-Bayesian LLM agents in simulations.", "questions": "- Does your setting admit a revelation principle?\n- (LLM-)Experimentally, which assumptions lead to significantly different dynamics compared to your current simulation exercise?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper thoroughly sets up its problem, and is very well-written. I found the integration of experiments alongside a (mostly) theory paper convincing, as the authors stated departures from their Bayesian assumption in the beginning of their experiments section.", "weaknesses": "The paper considers and names several restrictions in particular on the information structure and short-livedness of agents, but does not say something on how these affect outcomes. While I understand that for the tractability of the theoretical model all of them are essential, the fact that the authors run experiments would allow for an investigation of what other dynamics are possible with variations on, e.g., the stringent public information assumptions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)