# Decentralized Learning of Belief-Dependent Generalized Nash Equilibria under Partial Observability

Fredy Pokou, Hélène Le Cadre

► To cite this version:

Fredy Pokou, Hélène Le Cadre. Decentralized Learning of Belief-Dependent Generalized Nash Equilibria under Partial Observability. 2026. hal-05523464

HAL Id: hal-05523464

https://inria.hal.science/hal-05523464v1

Preprint submitted on 23 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Decentralized Learning of Belief-Dependent Generalized Nash Equilibria under Partial Observability

Frédy Pokou

Inria, University of Lille, CNRS, Centrale Lille

Villeneuve-d'Ascq, France

fredy-vale-manuel.pokou@inria.fr

Hélène Le Cadre

Inria, University of Lille, CNRS, Centrale Lille

Villeneuve-d'Ascq, France

helene.le-cadre@inria.fr

## ABSTRACT

We study distributed decision making under partial observability in constrained multi-agent systems. Agents operate with local sensing, limited communication, and noisy exogenous information while jointly satisfying coupled proximity constraints and learning an unknown latent parameter governing payoffs. We model the interaction as a belief-dependent generalized Nash equilibrium (GNE), defined as the equilibrium of an augmented belief-state game induced by agents' posterior beliefs. We develop a fully decentralized discrete-time learning scheme combining quadratic-penalty constraint enforcement with distributed Thompson sampling, enabling agents to simultaneously learn the payoff parameter and coordinate on constraint-feasible equilibria. Under mild regularity, constraint qualification, and posterior concentration assumptions, we prove that the induced stochastic dynamics converges almost surely to a belief-dependent GNE of the true constrained game. Finite-time performance is characterized through information-theoretic Bayesian regret bounds. Numerical experiments validate the theory and demonstrate that information quality, rather than information quantity, is the dominant driver of coordination.

## KEYWORDS

Generalized Nash Equilibrium, Incomplete Information Games, Decentralized Multi-Agent Learning

### ACM Reference Format:

Frédy Pokou and Hélène Le Cadre. 2026. Decentralized Learning of Belief-Dependent Generalized Nash Equilibria under Partial Observability. In *Proc. of the 25th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2026)*, Paphos, Cyprus, May 25–29, 2026, IFAAMAS, 10 pages.

## 1 INTRODUCTION

Classical Lanchester-type combat models [11] and their asymmetric extensions provide early analytical foundations for adversarial multi-agent interaction. Subsequent formulations incorporated mobility, dispersion, and heterogeneous forces, leading to guerrilla warfare models [2] and fire-allocation games [6]. While these models yield tractable equilibrium characterizations, they typically assume centralized or perfectly aggregated information and rely on static decision rules, limiting their ability to capture coordination under decentralized information.

Operations research has emphasized the role of intelligence and situational awareness in adversarial decision-making [8, 9]. However, information is usually treated as an exogenous parameter rather than an object learned and coordinated upon by agents operating under uncertainty. From a multi-agent perspective, this abstraction overlooks how decentralized information acquisition and sharing shape equilibrium behavior and collective performance.

More generally, agents in networked systems must simultaneously learn unknown parameters, anticipate others' actions, and satisfy shared feasibility constraints using only local information. Understanding how such information structures affect equilibrium selection, coordination efficiency, and social welfare remains a central challenge at the intersection of multi-agent learning and game theory, motivating frameworks that jointly couple belief formation, strategic decision making, and constrained coordination.

In parallel, the learning and control literature has developed posterior sampling and regret-minimization tools for partially observable environments. Thompson sampling achieves optimal regret guarantees in single-agent and competitive reinforcement learning [1, 12–14, 17, 18]. Extensions to multi-agent settings include communication-efficient Thompson sampling [7], posterior-sampling exploration [20], and no-regret learning in general-sum Markov games [4]. Complementary work studies decentralized equilibrium-seeking dynamics [3, 5] and MARL environments [16, 19, 20]. Nevertheless, these approaches largely abstract away spatially localized sensing, heterogeneous exogenous information sources, and coordination inefficiencies arising purely from decentralized information structures.

Despite recent progress, a unified framework capturing decentralized information acquisition, equilibrium coordination under coupled constraints, and Bayesian learning of unknown payoff parameters remains largely unexplored. Existing game-theoretic learning models often assume known payoffs or centralized information, while decentralized learning approaches focus on unconstrained environments or ignore equilibrium consistency. Consequently, the interaction between learning, information decentralization, and constraint-coupled strategic behavior remains insufficiently understood.

This paper makes three main contributions.

(i) **Framework.** We introduce a game-theoretic model for decentralized equilibrium learning under endogenous information acquisition, formulating agent interaction as a belief-dependent Generalized Nash Equilibrium (GNE) arising from an augmented belief-state game with coupled feasibility constraints. This framework captures the joint effects of partial observability, heterogeneous information sources, and decentralized coordination.

(ii) **Algorithm.** We propose a fully decentralized discrete-time

<footnote>*Proc. of the 25th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2026)*, C. Amato, L. Dennis, V. Mascardi, J. Thangarajah (eds.), May 25–29, 2026, Paphos, Cyprus. © 2026 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). This work is licenced under the Creative Commons Attribution 4.0 International (CC-BY 4.0) licence.</footnote>
learning algorithm that integrates quadratic-penalty equilibrium-seeking dynamics with distributed Thompson sampling, enabling agents to simultaneously learn unknown payoff parameters and coordinate on constraint-feasible equilibria using only local sensing and limited communication.

(iii) **Guarantees and metrics.** We establish almost sure convergence of the induced stochastic dynamics to a Generalized Nash Equilibrium of the true constrained game and derive sublinear Bayesian regret guarantees. We further introduce the Price of Miscoordination (PoM), a welfare-based metric that isolates coordination losses induced purely by decentralized information and complements regret-based performance analysis.

## 2 GAME ENVIRONMENT, INFORMATION, AND PAYOFFS

Let $\mathcal{N} := \{1, \dots, N\}$ denote the set of agents. Each agent $n \in \mathcal{N}$ selects a continuous action $x_n \in \mathcal{X}_n \subset \mathbb{R}^d$, where $\mathcal{X}_n$ is nonempty, compact and convex. The joint action profile is $\mathbf{x} := (x_1, \dots, x_N) \in \mathcal{X} := \prod_{n \in \mathcal{N}} \mathcal{X}_n$.

Agents interact locally over an undirected connected graph

$$G := (\mathcal{N}, \mathcal{E}), \quad \mathcal{V}_n := \{n' \in \mathcal{N} : \{n, n'\} \in \mathcal{E}\}.$$

Each agent observes only neighbors' actions $x_{n'}$ for $n' \in \mathcal{V}_n$, which induces local strategic coupling and decentralized information flow.

The environment depends on an unknown parameter $\theta \in \Theta \subset \mathbb{R}^K$ drawn once and fixed over time. Agents do not observe $\theta$ directly and instead maintain subjective beliefs over $\Theta$.

Let $\pi_n \in \Delta(\Theta)$ denote agent $n$'s belief distribution over $\theta$, where $\Delta(\Theta)$ denotes the space of probability measures over $\Theta$.

Agent $n$ evaluates the joint action profile $\mathbf{x}$ through the payoff

$$f_n(x_n, \mathbf{x}_{-n}; \theta, \mu) = U_n(x_n, \mathbf{x}_{-n}; \theta, \mu) - C_n(x_n), \quad (1)$$

where $U_n$ denotes a continuously differentiable interaction utility capturing strategic coupling with neighboring agents and dependence on the latent environment parameter $\theta$, while $C_n$ is a continuously differentiable individual cost function. We assume that $C_n$ is convex and that $U_n$ is jointly continuous in all arguments and continuously differentiable in $(x_n, \mathbf{x}_{-n})$.

The payoff admits an additive decomposition into an interaction component $U_n(\cdot; \theta, \mu)$ and a separable individual cost $C_n(\cdot)$. Under symmetric interaction neighborhoods and standard regularity conditions, the induced game admits an exact potential function given by the aggregate interaction utility minus the sum of individual costs. This form of payoffs is specific to coordination games, e.g. in [10]. This structure is consistent with gradient-based decentralized equilibrium-seeking dynamics and facilitates the analysis of convergence under learning and stochastic perturbations.

Agents may receive signals from non-strategic exogenous information sources. The parameter $\bar{\rho} \in [0, \infty)$ specifies the sensing or communication radius determining which sources are locally observable by agent $n$, while the trust parameter $\mu \in [0, 1]$ captures the intrinsic informativeness (or reliability) of these signals and determines how they enter the interaction utility $U_n$. Hence, $\bar{\rho}$ controls information availability and $\mu$ controls information quality, independently of the interaction radius $\rho \in [0, \infty)$ governing feasibility constraints.

Throughout the paper, the parameter $\mu$ is assumed to be fixed and common knowledge, and therefore remains constant over the time horizon of the game. For notational simplicity, and since no learning or strategic adaptation occurs with respect to $\mu$, we omit its explicit dependence in the payoff functions and write $f_n(x_n, \mathbf{x}_{-n}; \theta)$ and $U_n(x_n, \mathbf{x}_{-n}; \theta)$ whenever no ambiguity arises.

**ASSUMPTION 1 (PAYOFF REGULARITY).** For every $\theta \in \Theta$ and $n \in \mathcal{N}$:

(1) $f_n(\cdot, \mathbf{x}_{-n}; \theta)$ is concave in $x_n$,

(2) $\nabla_{x_n} f_n(\cdot, \mathbf{x}_{-n}; \theta)$ is Lipschitz continuous,

(3) $f_n$ is continuously differentiable in $(\mathbf{x}, \theta)$.

These conditions ensure well-posed best-responses and enable variational characterizations of equilibria.

Let $0 < \rho < \infty$ be a parameter defining the communication range of the agents. Actions must satisfy pairwise proximity constraints

$$g_{n,n'}(x_n, x_{n'}) := \|x_n - x_{n'}\| - \rho \le 0, \quad \forall n \in \mathcal{N}, \quad \forall n' \in \mathcal{V}_n.$$

The joint feasible set is

$$\mathcal{X}_c := \{\mathbf{x} \in \mathcal{X} : g_{n,n'}(x_n, x_{n'}) \le 0, \forall \{n, n'\} \in \mathcal{E}\}.$$

Since $\mathcal{X}_c$ is defined by pairwise edge constraints, the section $\mathcal{X}_n(\mathbf{x}_{-n}) = \{x_n \in \mathcal{X}_n : (x_n, \mathbf{x}_{-n}) \in \mathcal{X}_c\}$ reduces to

$$\mathcal{X}_n(\mathbf{x}_{-n}) = \{x_n \in \mathcal{X}_n : g_{n,n'}(x_n, x_{n'}) \le 0, \forall n' \in \mathcal{V}_n\},$$

since constraints associated with edges not incident to $n$ do not depend on $x_n$.

**ASSUMPTION 2 (CONSTRAINT QUALIFICATION).** At any feasible point where $g_{n,n'}(x_n, x_{n'}) = 0$ and $x_n \neq x_{n'}$, the gradients $\nabla g_{n,n'}$ are nonzero.

### 2.1 Belief-Dependent Generalized Nash Game

The environment is characterized by a fixed but unknown parameter $\theta \in \Theta$. The true interaction therefore corresponds to the generalized Nash game (GNG)

$$\mathcal{G}(\theta) := (\mathcal{N}, (\mathcal{X}_n)_{n \in \mathcal{N}}, \mathcal{X}_c, (f_n(\cdot; \theta))_{n \in \mathcal{N}}). \quad (2)$$

Since agents do not know $\theta$, they instead play a sequence of belief-dependent GNGs

$$\mathcal{G}(\boldsymbol{\pi}^{(t)}) := (\mathcal{N}, (\mathcal{X}_n)_{n \in \mathcal{N}}, \mathcal{X}_c, (\tilde{f}_n(\cdot; \pi_n^{(t)}))_{n \in \mathcal{N}}),$$

where $\tilde{f}_n(x_n, \mathbf{x}_{-n}; \pi_n^{(t)})$ is the belief-dependent payoff

$$\tilde{f}_n(x_n, \mathbf{x}_{-n}; \pi_n^{(t)}) = \mathbb{E}_{\theta \sim \pi_n^{(t)}} [f_n(x_n, \mathbf{x}_{-n}; \theta)]. \quad (3)$$

#### 2.1.1 Time-Varying Game Formulation.
Time is discrete $t \in \mathbb{N}$. At each time step, agents play a stage generalized Nash game induced by the current information state (beliefs and observations) and the active set $\mathcal{N}_n^{(t)}$. We denote the stage payoff by $f_n^{(t)}(x_n, \mathbf{x}_{-n}; \theta)$ when emphasizing time dependence; when no ambiguity arises the superscript $(t)$ is omitted for readability. The learning algorithm defined later therefore induces a stochastic process over the sequence of stage games and the joint state $(\mathbf{x}^{(t)}, \pi_n^{(t)}, \lambda^{(t)})$. Because the feasible set of each agent depends on the actions of its neighbors through (3), the interaction is a generalized Nash game rather than a standard Nash game and $\lambda^{(t)}$ is the dual variable of the coupling constraints.
### 2.1.2 Solution Concepts.

**DEFINITION 1 (GENERALIZED NASH EQUILIBRIUM).** A joint action $\mathbf{x}^* \in \mathcal{X}_c$ is a generalized Nash equilibrium (GNE) if no agent can improve its utility by a unilateral feasible deviation, i.e.,

$$f_n(\mathbf{x}_n^*, \mathbf{x}_{-n}^*; \theta) \geq f_n(\mathbf{x}_n, \mathbf{x}_{-n}^*; \theta) \quad \forall \mathbf{x}_n \in \mathcal{X}_n(\mathbf{x}_n^*), \forall n \in \mathcal{N}.$$

Among GNEs, we focus on equilibria compatible with a shared set of constraint multipliers leading to the same dual variables $\lambda$.

**DEFINITION 2 (VARIATIONAL GNE).** A GNE $\mathbf{x}^*$ is called a variational GNE (v-GNE) if it satisfies the first-order optimality conditions of a common variational inequality over the collective feasible set $\mathcal{X}_c$. Equivalently, all agents share consistent dual variables associated with the coupling constraints.

We extend these definitions to the belief-dependent GNG setting.

**DEFINITION 3 (BELIEF-DEPENDENT GENERALIZED NASH EQUILIBRIUM).** Fix a belief profile $\boldsymbol{\pi} = (\pi_n)_{n \in \mathcal{N}}$. A joint action $\mathbf{x}^* \in \mathcal{X}_c$ is a generalized Nash equilibrium (GNE) of the belief-dependent game $\mathcal{G}(\boldsymbol{\pi})$ if for every agent $n \in \mathcal{N}$,

$$\mathbf{x}_n^* \in \arg \max_{\mathbf{x}_n \in \mathcal{X}_n(\mathbf{x}_n^*)} \bar{f}_n(\mathbf{x}_n, \mathbf{x}_{-n}^*; \pi_n), \quad (4)$$

**DEFINITION 4 (BELIEF-DEPENDENT VARIATIONAL GENERALIZED NASH EQUILIBRIUM).** A point $\mathbf{x}^* \in \mathcal{X}_c$ is a variational generalized Nash equilibrium (v-GNE) of $\mathcal{G}(\boldsymbol{\pi})$ if it solves the variational inequality

$$\langle F(\mathbf{x}^*; \boldsymbol{\pi}), \mathbf{x} - \mathbf{x}^* \rangle \le 0, \quad \forall \mathbf{x} \in \mathcal{X}_c, \quad (5)$$

where the pseudo-gradient mapping is

$$F(\mathbf{x}; \boldsymbol{\pi}) := (\nabla_{\mathbf{x}_n} \bar{f}_n(\mathbf{x}; \pi_n))_{n \in \mathcal{N}}.$$

In practice, agents implement a posterior-sampling (Thompson-style [1, 15]) decision rule. At each time step $t$, agent $n$ samples a parameter realization $\theta_n^{(t)} \sim \pi_n^{(t)}$ and computes a best-response under the sampled model

$$\mathbf{x}_n^{(t+1)} \in \arg \max_{\mathbf{x}_n \in \mathcal{X}_n(\mathbf{x}_n^{(t)})} f_n(\mathbf{x}_n, \mathbf{x}_{-n}^{(t)}; \theta_n^{(t)}). \quad (6)$$

Hence, while equilibrium analysis is naturally formulated in the belief game $\mathcal{G}(\boldsymbol{\pi}^{(t)})$, the implemented dynamics correspond to stochastic best-responses generated by sampled games $\mathcal{G}(\theta_n^{(t)})$, $\forall n$.

## 2.2 Potential Game

We next investigate whether the game admits a potential structure, which is useful for equilibrium characterization and learning convergence analysis.

**DEFINITION 5 (POTENTIAL GAME).** The complete-information game $\mathcal{G}(\theta)$ is a potential game if there exists a continuously differentiable function $\Phi(\mathbf{x}; \theta)$ such that, for all $n \in \mathcal{N}$,

$$\nabla_{\mathbf{x}_n} f_n(\mathbf{x}_n, \mathbf{x}_{-n}; \theta) = \nabla_{\mathbf{x}_n} \Phi(\mathbf{x}; \theta), \quad \forall \mathbf{x} \in \mathcal{X}_c.$$

In this case, v-GNEs of $\mathcal{G}(\theta)$ coincide with optima of $\Phi(\cdot; \theta)$ over $\mathcal{X}_c$.

In the incomplete-information setting, define the belief-averaged potential

$$\bar{\Phi}(\mathbf{x}; \boldsymbol{\pi}) := \mathbb{E}_{\theta \sim \pi} [\Phi(\mathbf{x}; \theta)].$$

**PROPOSITION 1 (POTENTIAL STRUCTURE).** Fix $\theta \in \Theta$ and consider $\mathcal{G}(\theta)$ with utilities $f_n(\mathbf{x}_n, \mathbf{x}_{-n}; \theta)$ and feasible set $\mathcal{X}_c$. Assume that

$$f_n(\mathbf{x}_n, \mathbf{x}_{-n}; \theta) = u_n(\mathbf{x}_n; \theta) + \sum_{n' \in \mathcal{V}_n} \psi_{n,n'}(\mathbf{x}_n, \mathbf{x}_{n'}; \theta) - C_n(\mathbf{x}_n), \quad (7)$$

where for all $\{n, n'\} \in E$, $\psi_{n,n'}(\mathbf{x}_n, \mathbf{x}_{n'}; \theta) = \psi_{n',n}(\mathbf{x}_{n'}, \mathbf{x}_n; \theta)$. Then $\mathcal{G}(\theta)$ is an exact potential generalized Nash game with potential

$$\Phi(\mathbf{x}; \theta) = \sum_{n \in \mathcal{N}} f_n(\mathbf{x}_n, \mathbf{x}_{-n}; \theta). \quad (8)$$

**PROOF.** Let $\mathbf{x} \in \mathcal{X}_c$ and fix $\theta \in \Theta$. Using the separable structure,

$$f_n(\mathbf{x}_n, \mathbf{x}_{-n}; \theta) = u_n(\mathbf{x}_n; \theta) + \sum_{n' \in \mathcal{V}_n} \psi_{n,n'}(\mathbf{x}_n, \mathbf{x}_{n'}; \theta) - C_n(\mathbf{x}_n).$$

Define $\Phi(\mathbf{x}; \theta) = \sum_{n \in \mathcal{N}} f_n(\mathbf{x}_n, \mathbf{x}_{-n}; \theta)$. Consider any unilateral deviation $\mathbf{x}_n \to \mathbf{x}_n'$, keeping $\mathbf{x}_{-n}$ fixed. Only the terms involving agent $n$ change in the potential. By symmetry of pairwise interactions, $\psi_{n,n'}(\mathbf{x}_n, \mathbf{x}_{n'}; \theta) = \psi_{n',n}(\mathbf{x}_{n'}, \mathbf{x}_n; \theta)$, so each pairwise term appears exactly once in $\Phi$. Hence, $\Phi(\mathbf{x}_n, \mathbf{x}_{-n}; \theta) - \Phi(\mathbf{x}_n, \mathbf{x}_{-n}; \theta) = f_n(\mathbf{x}_n, \mathbf{x}_{-n}; \theta) - f_n(\mathbf{x}_n, \mathbf{x}_{-n}; \theta)$. Therefore, $\Phi$ is an exact potential for $\mathcal{G}(\theta)$ on $\mathcal{X}_c$, which proves the result. $\square$

**PROPOSITION 2 (BELIEF-DEPENDENT POTENTIAL GAME).** Assume that for every $\theta \in \Theta$, the game $\mathcal{G}(\theta)$ is an exact potential game with potential (8). Then, the belief-dependent game $\mathcal{G}(\boldsymbol{\pi})$ is an exact potential game with potential

$$\bar{\Phi}(\mathbf{x}; \boldsymbol{\pi}) = \sum_{n \in \mathcal{N}} \mathbb{E}_{\theta \sim \pi_n} [f_n(\mathbf{x}_n, \mathbf{x}_{-n}; \theta)].$$

**PROOF.** Since $\mathcal{G}(\theta)$ is an exact potential game, unilateral payoff variations equal potential variations. Taking expectations w.r.t. beliefs and using linearity of expectation preserves this equality. $\square$

Proposition 2 implies that computing a GNE of the belief-dependent game $\mathcal{G}(\boldsymbol{\pi})$ is equivalent to solving the constrained optimization problem

$$\max_{\mathbf{x} \in \mathcal{X}_c} \bar{\Phi}(\mathbf{x}; \boldsymbol{\pi}) = \max_{\mathbf{x} \in \mathcal{X}_c} \sum_{n \in \mathcal{N}} \mathbb{E}_{\theta \sim \pi_n} [f_n(\mathbf{x}_n, \mathbf{x}_{-n}; \theta)]. \quad (9)$$

## 3 BELIEF AND LEARNING MODEL

The game $\mathcal{G}(\theta)$ depends on an unknown parameter $\theta$. The induced interaction is therefore a GNG with incomplete information, where agents must act based on beliefs inferred from observations. To address this uncertainty, we introduce a probabilistic observation model that induces a stochastic sequence of belief-dependent GNGs.

**ASSUMPTION 3 (COMMUNICATION GRAPH).** The communication graph $G = (\mathcal{N}, E)$ is fixed, undirected, and connected. Agents may become inactive over time; the active agent set is denoted by $\mathcal{N}_n^{(t)} \subseteq \mathcal{N}$. The active neighborhood of agent $n$ at time $t$ is

$$\mathcal{V}_n^{(t)} = \{n' \in \mathcal{N}_n^{(t)} : \{n, n'\} \in E\}.$$

Inactive agents keep their last action and do not transmit observations or messages.

**REMARK 1.** A fixed graph models persistent communication links, while time variation arises through the active set $\mathcal{N}_n^{(t)}$ rather than through topology changes.
We denote by $\mathcal{I}^{(t)}$ the global information available in the system at time $t$, and by $\mathcal{I}_n^{(t)}$ the information available to agent $n$ at $t$.

**ASSUMPTION 4 (CONDITIONAL INDEPENDENCE OF OBSERVATIONS).**
Conditioned on $(\mathbf{x}^{(t)}, \theta)$, observations and messages are independent across agents and time. In particular,

$$P(\mathcal{I}^{(t)} | \mathbf{x}^{(t)}, \theta) = \prod_{n \in \mathcal{N}_n^{(t)}} P_n(\mathcal{I}_n^{(t)} | \mathbf{x}^{(t)}, \theta).$$

This assumption is realistic when observations originate from local sensors or communication channels affected by independent noise, which is common in distributed systems. It is also consistent when, conditional on the global system state, each agent collects data without direct stochastic interaction with others. Finally, it is a standard approximation that enables tractable aggregation laws while remaining close to real decentralized architectures.

## 3.1 Observation and Information Structure

Let $\theta \in \Theta$ denote the fixed but unknown environment parameter. At each time $t$, the environment generates signals according to

$$y^{(t)} \sim P(\cdot | \mathbf{x}^{(t)}, \theta),$$

where $P$ is a known likelihood model.

Each agent $n$ observes a private signal

$$o_n^{(t)} \sim P_n(\cdot | \mathbf{x}^{(t)}, \theta),$$

and receives messages from active neighbors

$$m_{n' \to n}^{(t)} \sim Q_{n',n}(\cdot | \mathbf{x}^{(t)}, \theta).$$

The information available to agent $n$ is

$$\mathcal{I}_n^{(t)} = (o_n^{(t)}, \{m_{n' \to n}^{(t)} : b' \in \mathcal{V}_n^{(t)}\}).$$

## 3.2 Belief State and Induced Belief-Dependent Game

Each agent maintains a posterior belief

$$\pi_n^{(t)}(\theta) = \mathbb{P}(\theta | \mathcal{H}_n^{(t)}), \quad \mathcal{H}_n^{(t)} = (\mathcal{I}_n^{(0)}, \dots, \mathcal{I}_n^{(t)}).$$

The belief acts as an information state summarizing past observations and induces a sequence of belief-dependent GNGs with expected utilities

$$\bar{f}_n(x_n, \mathbf{x}_{-n}; \pi_n^{(t)}) = \mathbb{E}_{\theta_{-\pi_n^{(t)}}} [f_n(x_n, \mathbf{x}_{-n}; \theta, \mu)].$$

**REMARK 2 (POLICY REGULARITY).** We assume that each agent's action selection rule is measurable and depends only on the current action profile and local belief, i.e., agent $n$'s policy satisfies

$$x_n^{(t+1)} \sim \kappa_n(\cdot | \mathbf{x}^{(t)}, \pi_n^{(t)}),$$

for some measurable stochastic kernel $\kappa_n$. In particular, the posterior-sampling best-response policy defined below satisfies this property.

**PROPOSITION 3 (MARKOV BELIEF STATE).** The augmented local state

$$s_n^{(t)} = (\mathbf{x}^{(t)}, \pi_n^{(t)})$$

is Markov for agent $n$.

**PROOF.** By Bayes' rule, the posterior $\pi_n^{(t)}$ is a sufficient statistic of the local information history $\mathcal{H}_n^{(t)}$ for inference about $\theta$.

Under Assumption 4, the observation model satisfies

$$P(\mathcal{I}_n^{(t+1)} | \theta, \mathbf{x}^{(0:t+1)}, \mathcal{H}_n^{(t)}) = P(\mathcal{I}_n^{(t+1)} | \theta, \mathbf{x}^{(t+1)}),$$

i.e., future observations are conditionally independent of past observations given $(\mathbf{x}^{(t+1)}, \theta)$. The action update rule is measurable with respect to $(\mathbf{x}^{(t)}, \pi_n^{(t)})$. Therefore, the conditional distribution of $(\mathbf{x}^{(t+1)}, \pi_n^{(t+1)})$ depends only on $(\mathbf{x}^{(t)}, \pi_n^{(t)})$. Hence, $s_n^{(t)}$ is a Markov state. □

Beliefs evolve recursively according to Bayes' rule

$$\pi_n^{(t+1)}(\theta) \propto P_n(o_n^{(t+1)} | \mathbf{x}^{(t+1)}, \theta) \prod_{n' \in \mathcal{V}_n^{(t+1)}} Q_{n',n}(m_{n' \to n}^{(t+1)} | \mathbf{x}^{(t+1)}, \theta) \pi_n^{(t)}(\theta).$$

## 3.3 Belief-Based Best-Response Dynamics

At time $t$, agents compute best-responses in the belief-dependent game. In practice, we consider a posterior-sampling implementation where agent $n$ samples

$$\tilde{\theta}_n^{(t)} \sim \pi_n^{(t)},$$

and computes $x_n^{(t+1)} \in \arg \max_{x_n \in \mathcal{X}_n(\mathbf{x}_{-n}^{(t)})} f_n(x_n, \mathbf{x}_{-n}^{(t)}; \tilde{\theta}_n^{(t)})$.

This defines a decentralized posterior-sampling policy. The resulting stochastic process $(\mathbf{x}^{(t)}, \boldsymbol{\pi}^{(t)})_{t \ge 0}$ can therefore be interpreted as decentralized stochastic best-response dynamics evolving across a sequence of belief-dependent generalized Nash games.

# 4 PENALIZED LEARNING DYNAMICS

We now define a decentralized learning dynamics combining posterior sampling and constraint penalization. The objective is to solve the belief-dependent constrained game while ensuring asymptotic feasibility.

## 4.1 Penalized Payoff

To handle coupled constraints, each agent optimizes a penalized objective

$$J_n(x_n, \mathbf{x}_{-n}; \tilde{\theta}_n^{(t)}, \boldsymbol{\lambda}^{(t)}) = f_n(x_n, \mathbf{x}_{-n}; \tilde{\theta}_n^{(t)}) - \sum_{n' \in \mathcal{V}_n} \lambda_{n,n'}^{(t)} g_{n,n'}(x_n, x_{n'}^t)^2, \quad (P2)$$

where $\lambda_{n,n'}^{(t)} \ge 0$ are time-varying penalty coefficients and $\tilde{\theta}_n^{(t)}$ is a posterior sample drawn from $\pi_n^{(t)}$.

## 4.2 Action Update

Given the current belief sample, agents perform a local projected gradient ascent step

$$x_n^{(t+1)} = \text{Proj}_{\mathcal{X}_n}[x_n^{(t)} + \alpha^{(t)} \nabla_{x_n} J_n(x_n^{(t)}, \mathbf{x}_{-n}^{(t)}; \tilde{\theta}_n^{(t)}, \boldsymbol{\lambda}^{(t)})],$$

where $\text{Proj}_{\mathcal{X}_n}$ denotes projection onto $\mathcal{X}_n$.

**ASSUMPTION 5 (STEP SIZES).** The sequence $(\alpha^{(t)})_{t \ge 0}$ satisfies

$$\sum_{t=0}^{\infty} \alpha^{(t)} = \infty, \quad \sum_{t=0}^{\infty} (\alpha^{(t)})^2 < \infty.$$
## 4.3 Penalty Update

Penalty coefficients increase proportionally to constraint violations

$$ \lambda_{n,n'}^{(t+1)} = \left[ \lambda_{n,n'}^{(t)} + \beta^{(t)} g_{n,n'}(x_n^{(t)}, x_{n'}^{(t)}) \right]_+ \quad (10) $$

where $[\cdot]_+$ = $\max(\cdot, 0)$ and $(\beta^{(t)})_{t \ge 0}$ is a nonnegative sequence.

**ASSUMPTION 6 (PENALTY GROWTH).** The sequence $(\beta^{(t)})$ satisfies

$$ \beta^{(t)} \to 0, \quad \sum_{t=0}^{\infty} \beta^{(t)} = \infty. $$

**REMARK 3 (TIME-SCALE RELATION).** No specific time-scale separation between $(\alpha^{(t)})$ and $(\beta^{(t)})$ is required unless stated otherwise.

## 4.4 Belief Sampling

At each time step,

$$ \tilde{\theta}_n^{(t)} \sim \pi_n^{(t)}(\theta), $$

and beliefs are updated according to the Bayesian recursion defined in the previous section.

This implements decentralized posterior sampling (Thompson sampling).

## 4.5 Distributed Learning Process

The overall algorithm is summarized below.

<table><caption>**Algorithm 1 Penalized Distributed Thompson Learning**</caption><thead><tr><th></th><th></th></tr></thead><tbody><tr><td>1:</td><td>for t = 0, 1, 2, ... do</td></tr><tr><td>2:</td><td>for each active agent n ∈ N<sub>n</sub><sup>(t)</sup> do</td></tr><tr><td>3:</td><td>Sample θ̂<sub>n</sub><sup>(t)</sup> ~ π<sub>n</sub><sup>(t)</sup></td></tr><tr><td>4:</td><td>Update action using projected gradient ascent</td></tr><tr><td>5:</td><td>Update local belief via Bayes rule</td></tr><tr><td>6:</td><td>for each edge {n, n'} ∈ E do</td></tr><tr><td>7:</td><td>Update penalty λ<sub>n,n'</sub><sup>(t)</sup></td></tr></tbody></table>

This induces a stochastic approximation process in the joint space

$$ (\mathbf{x}^{(t)}, \pi^{(t)}, \lambda^{(t)}). $$

# 5 THEORETICAL GUARANTEES AND COMPLEXITY

We establish feasibility, equilibrium convergence, and finite-time performance guarantees for the penalized distributed learning dynamics. All results rely on the standing assumptions introduced in previous sections: graph connectivity (Assumption 3), conditional independence of observations (Assumption 4), payoff regularity (Assumption 1), constraint qualification (Assumption 2), and stochastic approximation step sizes together with a diverging penalty schedule (Assumptions 5–6).

When stated explicitly, we additionally exploit the potential-game structure introduced in Section 2.

Throughout this section, we denote by $K := \dim(\theta)$ the dimension of the unknown latent environment parameter.

## 5.1 Asymptotic Feasibility

The penalty update enforces satisfaction of the coupled constraints.

**ASSUMPTION 7 (BOUNDED ITERATES).** The sequence $(\mathbf{x}^{(t)})_{t \ge 0}$ remains in a compact subset of $\mathcal{X}$ almost surely.

This assumption is standard in projected stochastic approximation and is automatically satisfied when $\mathcal{X}_n$ is compact.

**THEOREM 1 (ASYMPTOTIC FEASIBILITY).** Under Assumptions 5, 6, and 7, every limit point of $(\mathbf{x}^{(t)})_{t \ge 0}$ generated by the algorithm belongs to $\mathcal{X}_c$, i.e.,

$$ g_{n,n'}(x_n, x_{n'}) \le 0, \quad \forall \{n, n'\} \in E. $$

**PROOF.** The penalty recursion performs a stochastic dual ascent on the quadratic augmented Lagrangian associated with the coupled constraints.

Because $\sum_t \beta^{(t)} = \infty$ while $\beta^{(t)} \to 0$, persistent constraint violations would imply divergence of penalty multipliers. Under bounded iterates and Lipschitz gradients (Assumption 1), this contradicts stability of the primal recursion.

Therefore constraint violations vanish asymptotically. □

## 5.2 Convergence to Generalized Nash Equilibrium

We now incorporate learning and equilibrium structure.

**ASSUMPTION 8 (POSTERIOR CONSISTENCY).** For each agent n,

$$ \pi_n^{(t)} \Rightarrow \delta_{\theta} \text{ almost surely.} $$

**ASSUMPTION 9 (POTENTIAL STRUCTURE).** For every $\theta \in \Theta$, the complete-information game $\mathcal{G}(\theta)$ is an exact potential game with potential $\Phi(\cdot; \theta)$ as defined in Section 2.

This assumption allows equilibrium convergence to be characterized through potential optimization.

**THEOREM 2 (ALMOST SURE CONVERGENCE TO GNE).** Under Assumptions 1–2, 5–6, 7, 8, and 9, the iterates $(\mathbf{x}^{(t)})$ converge almost surely to the set of variational generalized Nash equilibria of the true game $\mathcal{G}(\theta)$.

**PROOF.** Under posterior consistency, sampled parameters satisfy

$$ \tilde{\theta}_n^{(t)} \to \theta \text{ almost surely.} $$

Hence the stochastic gradient admits the decomposition

$$ \nabla J_n(\mathbf{x}^{(t)}; \tilde{\theta}_n^{(t)}) = \nabla J_n(\mathbf{x}^{(t)}; \theta) + \varepsilon_n^{(t)}, $$

where $\varepsilon_n^{(t)} \to 0$ almost surely.

The dynamics is therefore a stochastic approximation process with vanishing noise. Standard Robbins–Monro arguments imply convergence to stationary points of the true penalized potential.

Using asymptotic feasibility and the potential structure, stationary points coincide with variational GNE of $\mathcal{G}(\theta)$. □
## 5.3 Finite-Time Bayesian Regret

We quantify the cost of learning through Bayesian regret.

**DEFINITION 6 (BAYESIAN REGRET).** The regret of agent n over horizon T is

$$R_n(T) = \sum_{t=0}^{T-1} \left( V_n^{\pi^*}(s^{(t)}; \theta) - \mathbb{E}[V_n^{\pi^{(t)}}(s^{(t)}; \theta)] \right).$$

**ASSUMPTION 10 (LINEAR VALUE REPRESENTATION).** There exists a bounded feature map $\phi_n$ such that

$$V_n^{\pi}(s; \theta) = \langle \phi_n^{\pi}(s), \theta \rangle.$$

**THEOREM 3 (BAYESIAN REGRET BOUND).** Under Assumption 10,

$$\mathbb{E}[R_n(T)] \leq \frac{M}{1-\gamma} \sqrt{2T I(\theta; H_T)}.$$

**PROOF.** The result follows from the information-theoretic analysis of posterior sampling. The belief-dependent game induces a bandit over policies whose reward is Lipschitz in $\theta$. Bounding the mutual information yields sublinear regret. □

## 5.4 Computational Complexity and Scalability

We analyze the per-iteration complexity of the algorithm.

**PROPOSITION 4 (PER-ITERATION COMPLEXITY).** Each agent update requires

$$O(|V_n| + K)$$

operations, where $|V_n|$ is the neighborhood size and $K = \dim(\theta)$ is the latent parameter dimension.

**PROOF.** Gradient evaluation depends only on neighbors' actions and local belief statistics. Posterior sampling and belief updates scale linearly in $K$, while penalty updates scale linearly in $|V_n|$. □

**PROPOSITION 5 (COMMUNICATION COMPLEXITY).** The communication load per iteration is $O(|E|)$ messages and does not scale with the number of observations.

Compared to centralized equilibrium computation requiring global optimization over $X_c$, the proposed dynamics scales linearly with network size and remains fully decentralized.

These results establish convergence of decentralized learning towards belief-consistent GNEs with provable finite-time efficiency and scalable implementation.

# 6 EFFICIENCY LOSS UNDER DECENTRALIZED INFORMATION

The previous section establishes that the learning dynamics converges towards a GNE of the true game $G(\theta)$. We now quantify the efficiency loss induced solely by decentralized information.

Throughout this section, expectations are taken with respect to the prior on $\theta$ and all sources of randomness in the learning dynamics.

## 6.1 Centralized and Decentralized Benchmarks

Let the social welfare be defined as

$$W(\mathbf{x}; \theta) := \sum_{n \in N} f_n(\mathbf{x}_n, \mathbf{x}_{-n}; \theta, \mu). \quad (11)$$

We compare two information structures.

**Centralized information.** Agents observe the true parameter $\theta$ and can condition their actions on it.

**Decentralized information.** Each agent n only observes its local information filtration and forms beliefs $\pi_n^{(t)}$ according to the learning dynamics described previously.

**ASSUMPTION 11 (WELL-POSED SOCIAL WELFARE MAXIMIZATION).** For every $\theta$:

(1) Action sets $X_n$ are compact and convex.

(2) $f_n(\cdot, \mathbf{x}_{-n}; \theta)$ is continuous in $\mathbf{x}$.

(3) $W(\mathbf{x}; \theta)$ is bounded.

This guarantees existence of maximizers and finiteness of expectations.

**Policy classes.** Let $\Pi^{\text{full}}$ denote the set of admissible policies measurable with respect to full information ($\theta$, history).

Let $\Pi^{\text{dec}}$ denote the set of admissible policies measurable with respect to local information and local beliefs ($\pi_n^{(t)}, I_n^{(t)}$).

Define

$$W^{\text{full}} := \sup_{\pi \in \Pi^{\text{full}}} \mathbb{E}[W(\mathbf{x}_{\pi}; \theta)], \quad (12)$$

$$W^{\text{dec}} := \sup_{\pi \in \Pi^{\text{dec}}} \mathbb{E}[W(\mathbf{x}_{\pi}; \theta)]. \quad (13)$$

By construction,

$$W^{\text{dec}} \leq W^{\text{full}}. \quad (14)$$

## 6.2 Price of Miscoordination

**DEFINITION 7 (PRICE OF MISCOORDINATION).**

$$\text{PoM} := \frac{W^{\text{full}}}{W^{\text{dec}}} \geq 1. \quad (15)$$

This quantity isolates the efficiency loss generated purely by informational decentralization, holding incentives and feasibility constraints fixed.

## 6.3 Connection with Equilibrium Structure

We now relate decentralized social welfare to equilibrium outcomes of the decentralized game.

**ASSUMPTION 12 (POTENTIAL STRUCTURE).** There exists a potential function $\Phi(\mathbf{x}; \theta)$ such that

$$\nabla_{\mathbf{x}_n} f_n(\mathbf{x}; \theta) = \nabla_{\mathbf{x}_n} \Phi(\mathbf{x}; \theta), \quad \forall n. \quad (16)$$

**ASSUMPTION 13 (EQUILIBRIUM EFFICIENCY WITHIN INFORMATION CLASS).** Among all decentralized policies, the social welfare-maximizing ones induce equilibrium play ($v$-GNE) of the decentralized game.

This assumption is standard in potential games where equilibria solve a constrained potential maximization problem.

## 6.4 Relation to Learning Dynamics

Let $\mathbf{x}^{(t)}$ denote the trajectory generated by the decentralized learning algorithm.

**THEOREM 4 (ASYMPTOTIC DECENTRALIZED EFFICIENCY).** Suppose:

* the convergence theorem holds,
• Assumptions 11–13 hold.

Then

$$ \lim_{t \to \infty} \mathbb{E}[W(\mathbf{x}^{(t)}; \theta)] = W^{\text{dec}}. \quad (17) $$

PROOF. By the convergence theorem, $\mathbf{x}^{(t)}$ converges (in probability or almost surely depending on the result used) to the set of GNEs of the decentralized game, including v-GNEs.

Under the potential structure, v-GNEq correspond to stationary points (i.e., optima) of the decentralized potential maximization problem.

Under Assumption 13, social welfare-maximizing decentralized policies induce equilibrium play (v-GNEs). Therefore, the limiting social welfare equals the optimal social welfare achievable under decentralized information. The theorem statement follows. □

## 6.5 Finite-Time Welfare Loss

Let $R_n(T)$ denote the cumulative regret of agent $n$.

THEOREM 5 (FINITE-TIME INEFFICIENCY BOUND). Under bounded rewards and sublinear regret for each agent,

$$ W^{\text{dec}} - \frac{1}{T} \sum_{t=1}^{T} \mathbb{E}[W(\mathbf{x}^{(t)}; \theta)] \le \sum_{n \in N} \frac{R_n(T)}{T}. \quad (18) $$

PROOF. Define the decentralized optimal trajectory $\mathbf{x}^{*,\text{dec}}$.

By additivity of social welfare,

$$ W(\mathbf{x}^{*,\text{dec}}; \theta) - W(\mathbf{x}^{(t)}; \theta) = \sum_{n} \left( f_n(\mathbf{x}_n^{*,\text{dec}}, \mathbf{x}_n^{*,\text{dec}}; \theta) - f_n(\mathbf{x}_n^{(t)}, \mathbf{x}_n^{(t)}; \theta) \right). $$

Summing over $t$ and taking expectations yields

$$ T W^{\text{dec}} - \sum_{t=1}^{T} \mathbb{E}[W(\mathbf{x}^{(t)}; \theta)] \le \sum_{n} R_n(T). $$

Divide by $T$ to conclude. □

## 6.6 Structural vs Learning Loss

The Price of Miscoordination decomposes total inefficiency into

### (1) Structural loss

$$ W^{\text{full}} - W^{\text{dec}}, $$

caused by informational decentralization.

### (2) Learning loss

$$ W^{\text{dec}} - \frac{1}{T} \sum_{t=1}^{T} \mathbb{E}[W(\mathbf{x}^{(t)}; \theta)], $$

which vanishes as regret becomes sublinear.

If $R_n(T) = O(\sqrt{T})$, the learning loss decays at rate $O(T^{-1/2})$.

## 6.7 Experimental Setup

Agents evolve in a 25 × 25 continuous domain under a fixed connected communication graph. The latent parameter is

$$ \theta = (12, 15, 21, 13, 9), \quad K := \dim(\theta) = 5, $$

and observations follow the generative model of Section 2.

Unless specified otherwise, the environment contains 95 blue agents, 70 red agents, and {115, 145, 185} civilians. The time horizon

is $T = 60$, sensing radii are $\rho = 1.3$ and $\bar{\rho} = 1.5$, the discount factor is $\gamma = 0.95$, and trust levels satisfy $\mu \in \{0, 0.65, 1\}$.

Red agents are sampled from Gaussian distributions centered at fixed strongholds ($Z_1, \dots, Z_5$), while civilians follow Brownian motion. Utilities and learning rules for blue agents are identical across regimes; only the information structure varies, isolating a pure value-of-information effect. Each reported curve averages 50 independent runs.

## 6.8 Convergence and Feasibility

We evaluate feasibility and equilibrium convergence predicted by Theorems 1–2. Constraint satisfaction is quantified through

$$ \text{Viol}(t) = \frac{1}{|E|} \sum_{\{n,n'\} \in E} \max\{0, g_{n,n'}(x_n^{(t)}, x_{n'}^{(t)})\}. $$

Constraint violations decrease monotonically and reach numerical zero after a short transient phase, confirming asymptotic feasibility.

Equilibrium convergence is assessed through the pseudo-gradient norm

$$ G(t) = \| \nabla J(\mathbf{x}^{(t)}; \theta) \|. $$

The pseudo-gradient norm vanishes while the joint value stabilizes, indicating convergence towards a stationary generalized Nash equilibrium. Empirically, stabilization occurs after approximately 38 iterations.

## 6.9 Finite-Time Regret

We evaluate the finite-time guarantee using empirical social welfare regret

$$ \hat{R}(T) = \sum_{t=0}^{T-1} (\hat{V}^* - W(\mathbf{x}^{(t)}; \theta)), $$

where $\hat{V}^*$ is estimated from a long-run equilibrium simulation. Empirical regret grows sublinearly and satisfies

$$ \hat{R}(T) = O(\sqrt{T}), $$

which is consistent with the information-theoretic bound. Figure 1 compares empirical value trajectories with the theoretical regret upper bound. After an initial exploration phase ($t \approx 15$), the value stabilizes while the bound continues to grow sublinearly, confirming that the theoretical bound is conservative and that learning operates in a near-efficient regime.

<figure><img src="image_2.png" /><figcaption>Figure 1: Convergence analysis.</figcaption></figure>
Posterior sampling is further compared with UCB and $\epsilon$-greedy exploration. UCB induces persistent over-exploration, while $\epsilon$-greedy explores inefficiently. Both methods produce larger cumulative regret across horizons, whereas Thompson sampling remains uniformly below their regret envelopes.

## 6.10 Efficiency Loss and Value of Information

We evaluate social welfare using

$$ \widehat{W}(T) = \frac{1}{T} \sum_{t=1}^{T} W(x^{(t)}; \theta), $$

and define the empirical Price of Miscoordination as

$$ \widehat{PoM} = \frac{\widehat{W}^{\text{full}}}{\widehat{W}(T)}. $$

Figure 2 reports average social welfare for different trust levels $\mu$. Performance increases sharply when $\mu$ increases from 0 to 0.65 and exhibits diminishing returns beyond this value, revealing an information reliability threshold.

<figure><img src="image_3.png" /><figcaption>Figure 2: Comparison of the impact $\mu$.</figcaption></figure>

Varying the number of civilians has limited impact on long-run performance, indicating that coordination depends primarily on information reliability rather than observation volume. Communication-enabled regimes strictly dominate no-communication baselines. After learning transients, $PoM > 1$ stabilizes, indicating a persistent efficiency gap induced purely by decentralized information.

Figure 3 reports $PoM$ as a function of sensing radius $\bar{\rho}$. Small radii generate fragmented beliefs and negligible coordination gains, whereas larger radii yield stable social welfare improvements until saturation. Beyond a critical sensing range, additional sensing no longer improves coordination, indicating global belief alignment.

Spatial agent configurations further illustrate decentralized coordination dynamics (see Fig. 5a–5d in Appendix). Initially, agents are spatially dispersed and beliefs are misaligned due to weak local observations. As learning progresses, agents concentrate around high-probability regions inferred from shared information, forming coordinated clusters through decentralized updates. At later stages, positions stabilize near latent centers, reflecting reduced uncertainty and sustained coverage of inferred regions. These spatial patterns are consistent with equilibrium convergence and uncertainty reduction predicted by the theory.

<figure><img src="image_4.png" /><figcaption>Figure 3: Sensitivity analysis of Empirical Price of Miscoordination</figcaption></figure>

## 6.11 Summary of Empirical Finding

The experiments confirm the theoretical results. Constraint violations vanish and equilibria emerge. Regret is sublinear and remains below classical exploration baselines. Decentralized information induces a bounded efficiency loss. Coordination performance depends primarily on information quality rather than observation quantity. These results validate the learning dynamics and support the interpretation of the Price of Miscoordination as a measurable coordination efficiency gap.

## 7 CONCLUSION

This framework enables the simulation of interactions between clusters of strategic agents under uncertainty and decentralized information, a setting common in defense and security contexts. By combining Bayesian learning and game-theoretic equilibrium analysis, it captures both adversarial adaptation and cooperative coordination. The approach supports realistic modeling of multi-force interactions and information-driven strategic behavior.

## REFERENCES

[1] Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. 2002. Finite-time analysis of the multiarmed bandit problem. *Machine learning* 47 (2002), 235–256.

[2] Seymour J Deitchman. 1962. A Lanchester model of guerrilla warfare. *Operations Research* 10, 6 (1962), 818–827.

[3] Zhenhua Deng and Tao Chen. 2024. Distributed Nash equilibrium seeking for constrained multicluster games of second-order nonlinear multiagent systems. *IEEE Trans. Automat. Control* 69, 11 (2024), 7855–7862.

[4] Liad Erez, Tal Lancewicki, Uri Sherman, Tomer Koren, and Yishay Mansour. 2023. Regret minimization and convergence to equilibria in general-sum markov games. In *International Conference on Machine Learning*. PMLR, 9343–9373.

[5] Michael I Jordan, Tianyi Lin, and Manolis Zampetakis. 2023. First-order algorithms for nonlinear generalized Nash equilibrium problems. *Journal of Machine Learning Research* 24, 38 (2023), 1–46.

[6] Edward H. Kaplan, Moshe Kress, and Roberto Szechtman. 2010. Confronting Entrenched Insurgents. *Operations Research* 58, 2 (2010), 329–341.

[7] Amin Karbasi, Nikki Lijing Kuang, Yian Ma, and Siddharth Mitra. 2023. Langevin thompson sampling with logarithmic communication: bandits and reinforcement learning. In *International Conference on Machine Learning*. PMLR, 15828–15860.

[8] Moshe Kress and Niall J MacKay. 2014. Bits or shots in combat? The generalized Deitchman model of guerrilla warfare. *Operations Research Letters* 42, 1 (2014), 102–108.

[9] Moshe Kress and Roberto Szechtman. 2009. Why defeating insurgencies is hard: The effect of intelligence in counterinsurgency operations—A best-case scenario. *Operations Research* 57, 3 (2009), 578–585.

[10] Ashok K. S. Krishnan, Hélène Le Cadre, and Ana Busic. 2025. How Irrationality Shapes Nash Equilibria: A Prospect-Theoretic Perspective. In *Proceedings of the 64th International Conference on Decision and Control*. IEEE, Rio de Janeiro, Brazil.

[11] Frederick William Lanchester. 1916. *Aircraft in warfare: The dawn of the fourth arm*. Constable limited.
<figure><img src="image_5.png" /><figcaption>Figure 4: Constraint satisfaction and pseudo-gradient convergenc</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure 5: Agents progressively concentrate around high-probability regions inferred from observations.</figcaption></figure>

<footnote>[12] Ian Osband, Charles Blundell, Alexander Pritzel, and Benjamin Van Roy. 2016. Deep exploration via bootstrapped DQN. In *Advances in Neural Information Processing Systems (NeurIPS)*.</footnote>

<footnote>[13] Ian Osband, Daniel Russo, and Benjamin Van Roy. 2013. More efficient reinforcement learning via posterior sampling. In *Advances in Neural Information Processing Systems (NeurIPS)*.</footnote>

<footnote>[14] Shuang Qiu, Ziyu Dai, Han Zhong, Zhaoran Wang, Zhuoran Yang, and Tong Zhang. 2023. Posterior sampling for competitive rl: Function approximation and partial observation. *Advances in neural information processing systems* 36 (2023), 26585–26637.</footnote>

<footnote>[15] Daniel Russo and Benjamin Van Roy. 2016. An information-theoretic analysis of thompson sampling. *Journal of Machine Learning Research* 17, 68 (2016), 1–30.</footnote>
[16] David Silver, Aja Huang, Christopher J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016), 484–489. https://doi.org/10.1038/nature16961

[17] William R. Thompson. 1933. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. *Biometrika* 25, 3/4 (1933), 285–294.

[18] Daniil Tiapkin, Denis Belomestny, Daniele Calandriello, Eric Moulines, Remi Munos, Alexey Naumov, Pierre Perrault, Michal Valko, and Pierre Ménard. 2023. Model-free posterior sampling via learning rate randomization. *Advances in Neural Information Processing Systems* 36 (2023), 73719–73774.

[19] Kaiqing Zhang, Zhuoran Yang, and Tamer Basar. 2019. Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms. arXiv preprint arXiv:1911.10635 (2019).

[20] Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. 2021. Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms. *Handbook of Reinforcement Learning and Control* (2021), 321–384.