# On the Value of Information in Counterinsurgency

Fredy Pokou, Hélène Le Cadre

► To cite this version:

Fredy Pokou, Hélène Le Cadre. On the Value of Information in Counterinsurgency. 2025. hal-05245586

HAL Id: hal-05245586

https://hal.science/hal-05245586v1

Preprint submitted on 8 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# On the Value of Information in Counterinsurgency

Fredy POKOU¹ and Hélène Le Cadre²

¹, ²Inria, CNRS, Univ. of Lille, Centrale Lille, UMR 9189 - CRISTAL, F-59000 Lille, France

September 8, 2025

## Abstract

Lanchester's laws, originally designed for symmetric conflicts, have recently been extended to address the complexities of asymmetric warfare. However, related works often overlook information asymmetry and the role of civilians in shaping combat dynamics. To overcome these limitations, we formulate an asymmetric information game in which government forces strategically plan their attacks on insurgent strongholds, while taking into account observations and information retrieved from noisy civilian reports. Relying on Bayesian regret, we evaluate the government forces' strategies proving convergence to a generalized Nash equilibrium under mild assumptions. Key contributions include (i) extension of fire distribution models; (ii) models of civilian collaboration willingness; (iii) Bayesian regret-based strategy evaluation; and (iv) sensitivity analysis on information quality, highlighting that moderate civilian collaboration is sufficient for optimal coordination. Simulations highlight the impact of information quality gathered from civilians' reports on the policy average value and confirm theoretical regret bounds.

**Keywords:** Asymmetric information, Bayesian regret, Counterinsurgency operations, Game theory

## 1 Introduction

The Lanchester combat models (Lanchester, 1916), developed in the early 20th century, were pioneers in modeling warfare dynamics, initially focusing on symmetric conflicts between two forces with comparable strength, tactics, and equipment. However, as conflicts evolved, particularly asymmetric ones, these models showed limitations, requiring adaptations to account for factors such as mobility, stealth, and imperfect information. Asymmetric conflicts, such as those between government forces and insurgents, involve complex dynamics in which victory is determined by tactical flexibility, limited resources, and information asymmetry. The literature has then evolved significantly, with key contributions like Deitchman (1962)'s guerrilla warfare model, which adapted Lanchester's principles to irregular warfare, capturing mobility, harassment tactics, and uneven force distribution. An extension to fire distribution in asymmetric conflicts was provided in Kaplan, Kress, and Szechtman (2010), computing equilibrium of a sequential force allocation game between insurgents and the government. However, traditional models often overlook the information asymmetry. Kress and MacKay (2014) partially addressed this dimension by introducing situational awareness parameters to affect combat dynamics. Our work extends these models by incorporating all previously unaddressed dimensions, such as information asymmetry, spatial coordination, and civilian-soldier interactions. We propose an asymmetric information game framework in which government forces optimize their actions in real time based on locally updated beliefs through Bayesian processes. Our approach relies on distributed soldier communications influenced by noisy civilian reports, allowing soldiers to dynamically update their beliefs about insurgent distribution. Further, we integrate Bayesian regret to evaluate the performance of the computed strategy with respect to the best possible strategy in hindsight, measuring how much is lost, on average, due to uncertainty and asymmetric information.
We now describe the main contributions of our work:

* We generalize Kaplan, Kress, and Szechtman (2010) fire distribution model, incorporating Bayesian belief updates, allowing strategy adaptations based on the information of civilian and neighboring soldiers;

* We introduce a novel model where civilians, as non-strategic agents, influence soldiers' perceptions, addressing information asymmetry in conflict dynamics, an aspect often overlooked in classical models;

* We formulate a Bayesian regret-based optimization model to assess the strategies of the soldiers, proving convergence to a generalized Nash equilibrium (GNE);

* We analyze how information quality impacts soldier performance, revealing that moderate civilian collaboration is sufficient for optimal coordination, underscoring the value of information quality over quantity.

# 2 Agents, Communication and Perception

During counterinsurgency operations, government forces with superior firepower confront weaker low-signature insurgents hidden in strongholds Kaplan, Kress, and Szechtman (2010). We consider a noncooperative game model involving government forces and insurgents operating within a well-delimited nonempty, open, and bounded area $C \subset \mathbb{R}^2$. One specificity of our model is to involve civilians. The agents operate according to predefined rules and local information. The set of agents is defined as $N \stackrel{\text{def}}{=} N_F \cup N_C$, where $N_F$ denotes the set of soldiers (strategic agents), $N_C$ is the set of civilians (non-strategic agents). In addition, the forces and civilian sets are disjoint, that is, $N_F \cap N_C = \emptyset$.

## 2.1 Forces

Strategic agents $N_F$ are divided into two disjoint adversarial groups $N_F = N_B \cup N_R$, $N_B \cap N_R = \emptyset$, where $N_B$ refers to the set of blue agents of the government force and $N_R$ refers to the set of red agents acting as insurgents. In a game setting, each soldier is modeled as a strategic agent updating its trajectory in $C$ at discrete time periods. In addition, the agents' trajectories and decision-making processes are constrained by spatial and tactical constraints, which induce coupling between the agents' feasible sets. Let $N_B \stackrel{\text{def}}{=} \{1, \dots, N_B\}$ denote the set of $N_B$ blue soldiers. We define $X_b \subseteq C$ as the local convex and compact feasible set of agent $b \in N_B$. At each time period $t \in \mathbb{N}$, every agent $b \in N_B$ selects a strategy $x_b^{(t)} \in X_b$, corresponding to a spatial position in the continuous two-dimensional operational environment. We define the strategy profile of all agents in $N_B$ at $t$ as $x^{(t)} \stackrel{\text{def}}{=} \text{col}(x_1^{(t)}, x_2^{(t)}, \dots, x_{N_B}^{(t)}) \in X \subseteq C^{N_B}$, where $X$ is the joint feasible set of the blue force. The structure of $X$ depends on both static constraints (e.g., presence of obstacles) and dynamic constraints encoding inter-agent dependencies. Thus, for each agent $b \in N_B$, we define its coupling feasible set at time period $t$ as a function of the positions of the other agents

$$ \mathcal{X}_b(x_{-b}^{(t)}) \stackrel{\text{def}}{=} \{x_b \in \mathcal{X}_b \mid \forall b' \in \mathcal{V}_b^{(t)}, \|x_b - x_{b'}^{(t)}\| \le \rho\}, $$

where $x_{-b}^{(t)} \stackrel{\text{def}}{=} \text{col}(x_j^{(t)})_{b' \in N_B \setminus \{b\}}$ is the stack of the strategies of all agents in $N_B$ except $b$, $\mathcal{V}_b^{(t)}$ contains the set of neighbors which communicate with $b$, $0 < \rho < \infty$ determines the communication range between neighbors.

The set $\mathcal{V}_b^{(t)}$ is defined according to fixed spatial coordination, leading to $\mathcal{V}_b^{(t)} = \mathcal{V}_b$, and is assumed to be predetermined by the mission such as in, e.g., squad assignments. Coordination is enforced among the set of blue soldiers by defining the joint feasible set, containing the concatenation of all coupling constraints

$$ \mathcal{X} \stackrel{\text{def}}{=} \{x \in C^{N_B} \mid \forall b \in N_B, x_b \in \mathcal{X}_b(x_{-b})\}. $$
In this model of conflict, both soldiers and insurgents are subject to elimination through engagements. As a result, the sets of active agents evolve over time. A soldier eliminated at time period $t$ is removed from $\mathcal{N}_B^{(t+1)}$; an insurgent neutralized at time period $t$ is removed from $\mathcal{N}_R^{(t+1)}$. These dynamics imply that the sizes $N_B^{(t)} := |\mathcal{N}_B^{(t)}|$ and $N_R^{(t)} := |\mathcal{N}_R^{(t)}|$ decrease over time. Since $\theta^{(t)}$ encodes the allocation of insurgents across $K$ strongholds, with $K \in \mathbb{N}^*$, it must also evolve accordingly

$$ \theta^{(t)} \in \Delta^{K-1}(N_R^{(t)}), \text{ with } \Delta^{K-1}(\cdot) := \{\theta \in \mathbb{N}^K \mid \sum_{j=1}^K \theta_j = \cdot\}, $$

where $\theta_j$ is the number of insurgents in the stronghold $Z_j$.

Thus, the support of the latent variable $\theta^{(t)}$ shrinks as insurgents are eliminated, tightening the uncertainty over time and naturally guiding the posterior toward more concentrated beliefs.

The real spatial spread of insurgents in $K$ strongholds is not observed and constitutes the core of the strategic inference task of the soldiers. Although the exact allocation vector $\theta^{(t)}$ remains latent, soldiers may obtain noisy observations of insurgent positions $x$. We define the stronghold assignment function $\psi: \mathcal{X} \to \{1, \dots, K\}$ as

$$ \psi(x) := \arg \min_{j \in \{1, \dots, K\}} \|x - Z_j\|^2, \quad (1) $$

which maps any location $x$ to the nearest stronghold, which provides a principled basis for aggregating field observations into belief updates on the insurgent distribution.

## Civilians

Each civilian $c \in \mathcal{N}_C$ is modeled as a non-strategic agent whose position at time period $t$ is given by

$$ x_c^{(t)} \in \mathcal{C}, \quad x_c^{(t)} \sim \mathcal{N}(Z_{\psi(x_c^{(t)})}, \sigma^2 \mathbf{I}), $$

that is, civilians are spatially distributed around strongholds according to a Gaussian distribution function centered on a stronghold location, with standard deviation $\sigma > 0$, but without strategic intent. They may report information about the presence of insurgents to nearby soldiers. However, the quality of shared information is modulated by a trust factor $\mu$, reflecting the willingness of the civilian to collaborate.

## 2.2 Communication

In asymmetric conflict environments, effective inference critically depends not only on individual sensing capabilities, but also on information exchange among agents. In our setting, the communication is distributed, time-varying, and dual-sourced: it integrates both inter-soldier communication and passive civilian contributions, modulated by trust. This section details the information acquisition and exchange mechanisms between agents.

### 2.2.1 Soldiers observation model.

Each soldier $b \in \mathcal{N}_B$ is equipped with a local sensing device enabling the detection of nearby insurgents within a fixed sensor radius $0 < \rho < \infty$. At time period $t$, the set of perceived insurgent positions (possibly over multiple past timesteps) is defined as

$$ \hat{\eta}_b^{(t)} := \{x_r \in \mathcal{N}_R^{(t)} \mid \exists \tau < t \text{ such that } \|x_b^{(\tau)} - x_r\| \le \rho\}. $$

These detections are assumed to be noisy due to occlusion or limited sensor resolution, but provide probabilistic evidence of the presence of insurgents in the vicinity. Soldiers are also capable of communicating with nearby teammates.
### 2.2.2 Civilians observation model.

Each civilian $c \in \mathcal{N}_C$ moves according to an exogenous process and is capable of observing insurgent activity within their own visual field $0 < \bar{\rho} < \infty$ and $\bar{\rho} \ll \rho$. The set of insurgent positions observed by civilian $c$ up to time period $t$ is denoted

$$ \hat{\eta}_c^{(t)} := \{x_r \in \mathcal{N}_R^{(t)} \mid \exists \tau < t, \|x_c^{(\tau)} - x_r\| \le \bar{\rho}\}. $$

These observations are inherently noisy and partial, reflecting imprecise civilian reports of nearby activity. However, they represent a valuable source of local knowledge, especially in regions less covered by soldiers.

### 2.2.3 Civilian-to-soldier communication.

In each time period $t$, a civilian $c \in \mathcal{N}_C$ can choose to transmit their observations to nearby soldiers $b \in \mathcal{N}_B$ located within a civilian communication radius $\bar{\rho} > 0$. The set of civilians that can transmit information to the soldier $b$ is given by

$$ \mathcal{V}_{b,c}^{(t)} := \{c \in \mathcal{N}_C \mid \|x_c^{(t)} - x_b^{(t)}\| \le \bar{\rho} \text{ and } q_{cb} = 1\}, $$

where the variable $q_{cb} \in \{0, 1\}$, determines whether $c$ shares information with $b$, follows a Bernoulli density function $q_{cb} \sim \text{Bernoulli}(\mu)$, with $\mu \in [0, 1]$ encoding the average local trust of civilians in blue soldiers. This mechanism captures the stochastic and non-systematic nature of civilian collaboration, driven by both spatial proximity and socio-political dynamics.

### 2.2.4 Aggregated local perception.

At time period $t$, each soldier $b \in \mathcal{N}_B$ aggregates all the information received from their own past observations; their neighboring soldiers; trusting civilians in their vicinity. We define the extended communication neighborhood of the soldier $b$ as $\tilde{\mathcal{V}}_b^{(t)} := \mathcal{V}_b^{(t)} \cup \{b\}$. Then, the effective aggregated observation set used by the soldier $b$ in time period $t$ is

$$ \bar{\eta}_b^{(t)} := \left( \bigcup_{b' \in \bar{\mathcal{V}}_b^{(t)}} \hat{\eta}_{b'}^{(t)} \right) \cup \left( \bigcup_{c \in \mathcal{V}_{b,c}^{(t)}} \hat{\eta}_c^{(t)} \right). $$

This set contains all perceived (though potentially noisy) insurgent locations available to the soldier $b$, forming the basis for subsequent Bayesian inferences over the latent configuration $\theta$.

## 3 Noncooperative Game Formulation

In asymmetric conflicts where insurgents simultaneously threaten civilians and soldiers, government forces have to arbitrage between maximizing the neutralization of insurgents and minimizing the individual risks incurred. This tactical compromise is based on partial local observations, reinforced by distributed communications between soldiers and noisy reporting from civilians. The utility function of each government soldier is thus made up of a collective utility term, relying on the information gathered by the soldiers in the team, and an individual cost reflecting the perceived risk.

### 3.1 General formulation.

For a soldier $b \in \mathcal{N}_B^{(t)}$ at time $t$, the expected utility is

$$ \mathbb{E}_{\xi_b^{(t)}} [U_b(x_b^{(t)}, \boldsymbol{x}_{-b}^{(t)}, \hat{\theta}_b^{(t)}; \mu) - S_b(x_b^{(t)}, \xi_b^{(t)})], $$

where:
* $U_b(\cdot)$ represents the collective utility for the blue force resulting from the neutralization of the insurgents,

* $S_b(\cdot)$ is soldier b's subjective risk of being killed,

* $\hat{\theta}_b^{(t)} \in \Delta^{K-1}$ is the soldier's current belief about the distribution of insurgents between strongholds,

* $\xi_b^{(t)}$ is a random term modeling contextual uncertainties (e.g., delayed reaction, visibility, fatigue).

### 3.1.1 Intelligence model.

At each time period $t$, the agent $b$ maintains a local estimate $\hat{\theta}_b^{(t)} \in \mathbb{N}^K$ of the number of insurgents in each stronghold $(Z_j)_{j=1,...,K}$, based on distributed Bayesian inference. Following the linear intelligence model of Kress and Szechtman (2009), we define the belief weighted-threat level for each $b \in \mathcal{N}_B$

$$p(\mu, \hat{\theta}_{b,\psi(x_r)}^{(t)}) := \mu + (1 - \mu) \cdot \frac{\hat{\theta}_{b,\psi(x_r)}^{(t)}}{n_r}, \quad \forall r \in \mathcal{N}_R^{(t)},$$

where $\psi(x_r)$ is defined in Eq (1),

* $\mu \in [0, 1]$ encodes the local trust of civilians in the blue force,

* $n_r > 0$ is a normalization constant.

This formulation balances baseline caution and evidence-based engagement: when civilian trust $\mu$ is high, threat levels remain conservative even with limited local evidence; when $\mu$ is low, soldiers rely more directly on their own estimates $\hat{\theta}_{b,\psi(x_r)}^{(t)}$. The parameter $\mu$ thus modulates how civilian confidence influences perception of military threat and tactical decisions.

### 3.1.2 Probabilistic engagement model.

Let $x_r \in \bar{\eta}_b^{(t)}$ be an observed insurgent's location. We define the engagement score of the soldier $b$ towards the insurgent $r$ as $z_{b,r}^{(t)} := \beta_0 - \beta_1 \cdot \|x_b^{(t)} - x_r\|^2 + \beta_2 \cdot p(\mu, \hat{\theta}_{b,\psi(x_r)}^{(t)})$, where:

* $\beta_0 \in \mathbb{R}$ is a base propensity for engagement,

* $\beta_1 > 0$ penalizes squared distance to discourage low-probability long-range attempts,

* $\beta_2 > 0$ captures the influence of the perceived threat level on aggressiveness.

This score is then passed through the logistic function to model the stochastic probability of a successful neutralization Score($x_b^{(t)}, x_r$) := $\sigma(z_{b,r}^{(t)}) = \frac{1}{1+\exp(-z_{b,r}^{(t)})}$, leading to the probabilistic model

$$T(x_b^{(t)}, x_r, \hat{\theta}_{b,\psi(x_r)}^{(t)}; \mu) \sim \text{Bernoulli}(\sigma(z_{b,r}^{(t)})),$$

where $T(x_b^{(t)}, x_r, \hat{\theta}_{b,\psi(x_r)}^{(t)}; \mu) = 1$ if the insurgent in $x_r$ is successfully neutralized by the soldier $b$ in the time period $t$, conditioned on the belief vector $\hat{\theta}_b^{(t)}$; 0 otherwise.
### 3.1.3 Probabilistic coordination and collective utility

Under the assumption of independent engagement attempts between agents, the probability that the insurgent $x_r \in \bar{\eta}_b^{(t)}$ is not neutralized by any member of $\tilde{V}_b^{(t)}$ is given by

$$ \mathbb{P}[r \text{ is eliminated}] = 1 - \prod_{b' \in \tilde{V}_b^{(t)}} (1 - T(x_{b'}^{(t)}, x_r, \hat{\theta}_{b', \psi(x_r)}^{(t)}; \mu)) $$

where $\sigma(z_{b',r}^{(t)})$ is the probability of engagement of agent $b'$ against insurgent $r$, as defined by the logistic transformation of the engagement score. This expression is deterministic and differentiable with respect to positions $x_b^{(t)}$ and belief vector $\hat{\theta}_b^{(t)}$, enabling efficient distributed learning and optimization. We thus define the collective deterministic utility of agent $b$ at time $t$ as

$$ U_b(x_b^{(t)}, \boldsymbol{x}_{-b}^{(t)}, \hat{\theta}_b^{(t)}; \mu) := \sum_{x_r \in \bar{\eta}_b^{(t)}} \left( 1 - \prod_{b' \in \tilde{V}_b^{(t)}} (1 - \sigma(z_{b',r}^{(t)})) \right), $$

This definition captures the expected contribution of the agent $b$ to the collective neutralization effort, based on its coordination with neighboring agents and the perception of local threats. Importantly, it aligns with distributed optimization frameworks and allows agents to compute gradients of their utility functions with respect to local states, thus facilitating decentralized convergence towards equilibria under uncertainty.

### 3.1.4 Subjective cost of vulnerability $S_b$.

Each soldier is exposed to personal risk, depending on proximity to insurgents and local stochastic conditions

$$ S_b(x_b^{(t)}, \xi_b^{(t)}) := \alpha \cdot \sum_{x_r \in \bar{\eta}_b^{(t)}} \|x_b^{(t)} - x_r\|^2 + \xi_b^{(t)}, $$

where the sum weighs the squared distances to known insurgent locations, reflecting a continuous spatial risk.

**Assumption 1.** Let each soldier $b \in \mathcal{N}_B^{(t)}$ and each $x_r \in \bar{\eta}_b^{(t)}$ be such that the engagement score is $z_{b,r}^{(t)}$. We assume that there exists a constant $D > 0$ such that $\|x_b^{(t)} - x_r\|^2 \le D^2$, $\forall b \in \mathcal{N}_b$, $\forall x_r \in \bar{\eta}_b^{(t)}$. This implies that $z_{b',r}^{(t)} > 0$ whenever $\beta_0 + \beta_2 \cdot \mu > \beta_1 \cdot D^2$.

This condition ensures that each term $\sigma(z_{b',r}^{(t)})$ lies in the concave region of the sigmoid, that is, where $z > 0$.

**Proposition 1 (Sufficient Condition for $U_b$ Concavity).** Suppose Assumption 1 holds. Then, the function

$$ x_b^{(t)} \mapsto U_b(x_b^{(t)}, \boldsymbol{x}_{-b}^{(t)}, \hat{\theta}_b^{(t)}; \mu) $$

is concave with respect to its own position $x_b^{(t)} \in \mathcal{X}_b$, for any given $\boldsymbol{x}_{-b}^{(t)} \in \prod_{b' \in \tilde{V}_b^{(t)}} \mathcal{X}_{b'}$, $\hat{\theta}_b^{(t)} \in \Delta^{K-1}$, and $\bar{\eta}_b^{(t)}$.

*Proof.* We proceed in the following steps:

1. Convexity of the squared distance term:

The mapping $x \mapsto \|x - x_r\|^2$ is convex, so $x \mapsto -\beta_1 \|x - x_r\|^2$ is concave.

2. Linearity of the belief term:

The function $p(\mu, \hat{\theta})$ does not depend on $x_b^{(t)}$. Thus, $x \mapsto z_{b,r}^{(t)}$ is concave in $x$ since it is a sum of a concave and a constant term.

3. Concavity of $\sigma(z)$ when $z > 0$: The sigmoid function $\sigma(z) = \frac{1}{1+e^{-z}}$ is concave in $z > 0$. Hence, if $z_{b',r}^{(t)} > 0$, the composition $x \mapsto \sigma(z_{b',r}^{(t)}(x))$ is concave.
4. Convexity of $1 - \sigma(z)$: Since $\sigma(z)$ is concave and increases on $z > 0$, the function $1 - \sigma(z)$ is convex in this domain.

5. Log-convexity of the product:

The product $\prod_{b'} (1 - \sigma(z_{b',r}^{(t)}))$ is convex if each term is convex and strictly positive. This is satisfied when all $z_{b',r}^{(t)} > 0$.

6. Concavity of $1 - \prod_{b'} (1 - \sigma(z))$: Since $f := \prod_{b'} (1 - \sigma(z_{b',r}^{(t)}))$ is convex and takes values in $(0, 1)$, the function $x \mapsto 1 - f(x)$ is concave.

7. Sum of concave functions:

Finally, $U_b$ is the sum over $x_r \in \bar{\eta}_b^{(t)}$ of functions $x_r^{(t)} \mapsto 1 - \prod_{b'} (1 - \sigma(z_{b',r}^{(t)}))$, each concave under the above conditions. Hence, $U_b$ is concave.

□

**Proposition 2.** $\nabla_{x_b^{(t)}} f_b(x_b^{(t)}, x_{-b}^{(t)}) \in \mathbb{R}^2$ is Lipschitzian.

*Proof.* We consider the sigmoid-based engagement model with utility

$$z_{b,r}^{(t)} := \beta_0 - \beta_1 \|x_b^{(t)} - x_r\|^2 + \beta_2 p(\mu, \hat{\theta}_{b,\psi}(x_r)),$$

and collective utility

$$U_b(x_b^{(t)}, x_{-b}^{(t)}, \hat{\theta}_b^{(t)}; \mu) = \sum_{x_r \in \bar{\eta}_b^{(t)}} \left( 1 - \prod_{b' \in \tilde{\mathcal{V}}_b} (1 - \sigma(z_{b',r}^{(t)})) \right).$$

**Gradient of $U_b$**. Since only $z_{b,r}^{(t)}$ depends on $x_b^{(t)}$, we obtain

$$\nabla_{x_b^{(t)}} U_b = \sum_{x_r \in \bar{\eta}_b^{(t)}} \left[ \left( \prod_{b' \neq b} (1 - \sigma(z_{b',r}^{(t)})) \right) \sigma(z_{b,r}^{(t)}) (1 - \sigma(z_{b,r}^{(t)})) (2\beta_1(x_r - x_b^{(t)})) \right].$$

**Hessian of $U_b$**. Defining

$$u_r := \sigma(z_{b,r}^{(t)})(1 - \sigma(z_{b,r}^{(t)})) \prod_{b' \neq b} (1 - \sigma(z_{b',r}^{(t)})),$$

we rewrite

$$\nabla_{x_b^{(t)}} U_b = \sum_{x_r \in \bar{\eta}_b^{(t)}} 2\beta_1 u_r (x_r - x_b^{(t)}).$$

Then the Hessian is

$$\nabla_{x_b^{(t)}}^2 U_b = - \sum_{x_r \in \bar{\eta}_b^{(t)}} 2\beta_1 u_r I_2 + \sum_{x_r \in \bar{\eta}_b^{(t)}} 2\beta_1 (\nabla_{x_b^{(t)}} u_r) (x_r - x_b^{(t)})^\top,$$

where $(v)(w)^\top$ denotes the outer product between $v, w \in \mathbb{R}^2$.

Let's put:

* $N^{(t)} := |\bar{\eta}_b^{(t)}|$, number of insurgents around $b$,

* $R^{(t)} := \max_{x_r \in \bar{\eta}_b} \|x_r - x_b^{(t)}\|$,

* $M^{(t)} := \max_r |\nabla_{x_b^{(t)}} u_r|$.
Bounding the Lipschitz constant

• First term: Since $\sigma(z)(1 - \sigma(z)) \le \frac{1}{4}$,

$$ \left\| \sum_{x_r \in \bar{\eta}_b^{(t)}} 2\beta_1 u_r I_2 \right\| \le \frac{\beta_1 N^{(t)}}{2}. $$

• Second term: From $|\nabla_{x_b^{(t)}} z_{b,r}^{(t)}| \le 2\beta_1 R^{(t)}$ and $\sigma(z)(1 - \sigma(z))|1 - 2\sigma(z)| \le \frac{1}{4}$, we obtain

$$ \| \nabla_{x_b^{(t)}} u_r \| \le \frac{\beta_1 R^{(t)}}{2}. $$

Thus,

$$ \left\| \sum_{x_r \in \bar{\eta}_b^{(t)}} 2\beta_1 \nabla_{x_b^{(t)}} u_r (x_r - x_b^{(t)})^{\top} \right\| \le \beta_1^2 N^{(t)} R^{(t)2}. $$

**Final bound.** Combining both terms gives the Lipschitz constant

$$ L^{(t)} \le \frac{\beta_1 N^{(t)}}{2} + \beta_1^2 N^{(t)} R^{(t)2}. $$

Finally, since $S_b$ is a quadratic form in $|x_b^{(t)} - x_r|^2$, its gradient is also Lipschitz. By the triangle inequality,

$$ \nabla_{x_b^{(t)}} f_b(x_b^{(t)}, x_{-b}^{(t)}) = \nabla_{x_b^{(t)}} U_b - \nabla_{x_b^{(t)}} S_b $$

is Lipschitz with constant at most $L^{(t)} + L_S$, where $L_S$ is the Lipschitz constant of $\nabla S_b$.

<figure><img src="image_2.png" /></figure>

In the context of our asymmetric warfare model, each soldier $b \in \mathcal{N}_B$ interacts with a local neighborhood of peers denoted by $\mathcal{V}_b^{(t)} \subset \mathcal{N}_B^{(t)}$. As a result, the soldier must solve a constrained optimization problem that balances tactical objectives with coordination requirements:

$$ \begin{array}{ll} \max_{x_b^{(t)} \in \mathbb{R}^2} & \mathcal{J}_b^{(P1)}(x_b^{(t)}, x_{-b}^{(t)}) := f_b(x_b^{(t)}, x_{-b}^{(t)}) \\ \text{s.t.} & g_{b',b''}(x_{b'}^{(t)}, x_{b''}^{(t)}) \le 0, \quad \forall b'' \in \mathcal{V}_{b'}^{(t)}, \forall b' \in \mathcal{N}_b^{(t)} \end{array} \qquad (P1) $$

where $g_{b',b''}(x_{b'}^{(t)}, x_{b''}^{(t)}) := \|x_{b'}^{(t)} - x_{b''}^{(t)}\| - \rho$ and $f_b(x_b^{(t)}, x_{-b}^{(t)}) := \mathbb{E}_{\xi_b^{(t)}} [U_b(x_b^{(t)}, x_{-b}^{(t)}, \hat{\theta}_b^{(t)}; \mu) - S_b(x_b^{(t)}, \xi_b^{(t)})]$. Here, the constraint enforces a cohesion radius $\rho > 0$ between neighboring agents (soldiers), ensuring that local formations remain spatially coordinated. However, the presence of symmetric coupling constraints across overlapping neighborhoods introduces significant computational and coordination challenges in a decentralized setting. To address this issue, we introduce an unconstrained penalized relaxation of the problem (P1), where the hard constraints are replaced by soft penalties in the objective function.

**Proposition 3 (NSCQ for Pairwise Proximity Constraint).** At any time period $t \in \mathbb{N}$, for any given pair $b, b' \in \mathcal{N}_B^{(t)}$ such that $x_b^{(t)} \ne x_{b'}^{(t)}$, the constraints $[g_{b,b'}(x_b^{(t)}, x_{b'}^{(t)})]_{b,b'}$ satisfy the Nonsingular Shared Constraints Qualification (NSCQ) at any feasible configuration.

The NSCQ is a sufficient condition ensuring the well-posedness of shared Lagrange multipliers in games with coupled constraints. It guarantees that KKT optimality conditions are valid locally for each soldier and that distributed algorithms (e.g., penalized gradient or primal-dual methods) converge to a GNE.

We formulate the penalized form of (P1) as follows

$$ \max_{x_b \in \mathbb{R}^2} \mathcal{J}_b^{(P2)}(x_b, x_{-b}) := f_b(x) - \sum_{b' \in \mathcal{V}_b} \lambda_{b,b'}^{(t)} \cdot [g_{b,b'}(x)]^2, \qquad (P2) $$

where the penalty weights are defined by $\lambda_{b,b'}^{(t)} := \zeta^{(t)} \cdot \max\{0, g_{b,b'}(x_b, x_{b'})\}$.
**Assumption 2.** The step size $\alpha^{(t)}$ is chosen such that $\forall t \in \mathbb{N}$, $\sum_t \alpha^{(t)} = \infty$, $\sum_t (\alpha^{(t)})^2 < \infty$.

**Proposition 4.** Jordan, Lin, and Zampetakis (2023) suppose Assumption 2 holds, then the sequence $\{x_b^{(t)}\}_{t \in \mathbb{N}}$ generated by the distributed gradient-based update

$$x_b^{(t+1)} = x_b^{(t)} - \alpha^{(t)} (\nabla_{x_b^{(t)}} f_b(x^{(t)}) - \sum_{b' \in \mathcal{V}_b} \lambda_{b,b'}^{(t)} \nabla_{x_b} g_{b,b'}(x^{(t)})),$$

converges to a GNE of the original constrained problem. Further, the implicit penalty multipliers $\lambda_{b,b'}^{(t)}$ converge to the shared Lagrange multipliers associated with the KKT system of the constrained problem (P1).

Thus, for a soldier $b \in \mathcal{N}_B^{(t)}$ at time $t$, the expected utility becomes

$$\mathbb{E}_{\xi_b^{(t)}} [U_b(s_b^{(t)}, a_b^{(t)}; \mu) - S_b(s_b^{(t)}, \xi_b^{(t)})] - \Lambda^{(t)\top} \Phi(x_b^{(t)}, x_{-b}^{(t)})$$

where $\Lambda^{(t)} = [\lambda_{b,1}^{(t)}, \dots, \lambda_{b,|\mathcal{V}_b^{(t)}|}^{(t)}]^{\top} \in \mathbb{R}_{+}^{|\mathcal{V}_b^{(t)}|}$, $\Phi_b(x_b^{(t)}, x_{-b}^{(t)}) \stackrel{\text{def}}{=} [ [\phi_{b,b'}(x_b^{(t)}, x_{b'}^{(t)})] ]_{b' \in \mathcal{V}_b^{(t)}} \in \mathbb{R}^{|\mathcal{V}_b^{(t)}|}$ is the vector of constraint violations.

*Proof.* We consider a networked game with agents $\mathcal{N}_B^{(t)}$, where each agent $b$ maximizes a local objective $f_b(x_b^{(t)}, x_{-b}^{(t)})$ subject to pairwise shared constraints of the form $g_{b,b'}(x_b, x_{b'}) \le 0$, with symmetry $g_{b,b'} = g_{b',b}$. We analyze the following penalized gradient-based update scheme:

$$x_b^{(t+1)} = x_b^{(t)} + \alpha_t \left( \nabla_{x_b} f_b(x_b^{(t)}, x_{-b}^{(t)}) - \sum_{b' \in \mathcal{V}_b^{(t)}} \lambda_{b,b'}^{(t)} \nabla_{x_b} g_{b,b'}(x_b^{(t)}, x_{b'}^{(t)}) \right),$$

where the penalty multipliers are defined as

$$\lambda_{b,b'}^{(t)} := c_t \cdot \max\{0, g_{b,b'}(x_b^{(t)}, x_{b'}^{(t)})\}, \quad c_t \to +\infty,$$

and the learning rate sequence satisfies $\sum_t \alpha_t = \infty$, $\sum_t \alpha_t^2 < \infty$.

Under standard regularity assumptions, namely, concavity and Lipschitz continuity of each $f_b$ in $x_b^{(t)}$, monotone convexity of constraint functions $g_{b,b'}$, and the non-singular shared constraint qualification (NSCQ), this update scheme converges to a Generalized Nash Equilibrium (GNE) of the constrained problem. Furthermore, the penalization terms $\lambda_{b,b'}^{(t)}$ asymptotically approximate the shared KKT multipliers, yielding distributed consistency of the dual variables. $\square$

# 4 Belief Inference Model

We model the unknown strategic distribution of insurgents across a fixed set of strongholds as a latent discrete allocation variable. Let $\theta^{(t)} = (\theta_1^{(t)}, \dots, \theta_K^{(t)}) \in \mathbb{N}^K$ with $\sum_{j=1}^K \theta_j^{(t)} = N_R^{(t)}$, where $\theta_j^{(t)}$ denotes the number of insurgents located within the stronghold $Z_j$, and $N_R^{(t)}$ is the total number of insurgents in the system. This variable $\theta^{(t)}$ is latent and constitutes the principal object of Bayesian inference for each soldier $b \in \mathcal{N}_B$.

## Hierarchical prior structure.

To encode the prior uncertainty about the allocation of the insurgents $\theta^{(t)}$, we adopt a hierarchical Dirichlet-Multinomial model. The inference proceeds in two steps

* Latent probability vector $q^{(t)} \sim \text{Dir}(\alpha^{(t)})$, $\alpha^{(t)} \in \mathbb{R}_+^K$, where $\alpha^{(t)}$ represents dynamically updated pseudo-counts incorporating aggregated field observations up to time period $t$.

* Allocation model $\hat{\theta}^{(t)} \sim \text{Multinomial}(N_R, q^{(t)})$, that describes the effective (but unobserved) distribution of insurgents in the strongholds over time period $t$.
**Assumption 3.** We assume the following conditions:

(A1) Each insurgent $r \in \mathcal{N}_R^{(t)}$ is assigned to a stronghold $Z_j$ with probability proportional to $\hat{\theta}_j$;

(A2) Conditional on being assigned to stronghold $Z_j$, the insurgent's location $x_r \in \mathbb{R}^2$ is sampled from an isotropic Gaussian distribution centered at $Z_j$ with covariance matrix $\sigma_r^2 I$, where $\sigma_r > 0$ denotes the insurgent's dispersion radius around the stronghold.

**Proposition 5** (Generative Model for Insurgent Locations). Suppose Assumptions (A1)-(A2) hold. Then, the marginal distribution of the spatial location $x_r$ given $\theta$ is a hierarchical Gaussian mixture model

$$P(x_r | \theta^{(t)}) = \sum_{j=1}^{K} \frac{\hat{\theta}_j^{(t)}}{N_R^{(t)}} \cdot \mathcal{N}(x_r | Z_j, \sigma_r^2 I),$$

Belief representation via pseudo-counts

Each soldier $b$ maintains a private belief in $\theta$ via a pseudo-count vector $\alpha_b^{(t)} = (\alpha_{b,1}^{(t)}, \dots, \alpha_{b,m}^{(t)})$, initialized as $\alpha_b^{(\cdot)} \leftarrow \alpha^{(\cdot)}$, and recursively updated based on their own and neighboring observations $\bar{\eta}_b^{(t)}$ (within a sensing radius) $\alpha_{b,j}^{(t)} := \sum_{x \in \bar{\eta}_b^{(t)}} \mathbf{1}_{\{\psi(x)=j\}}$. This vector reflects the perceived concentration of insurgent activity per stronghold, enabling the soldier $b$ to build a Dirichlet prior to frequency vectors.

Bayesian update of beliefs.

Given the aggregated observation set $\bar{\eta}_b^{(t)}$ at time $t$, the posterior over $\theta$ is obtained using the Bayes rule

$$\mathbb{P}_b^{(t)}(\theta^{(t)} | \bar{\eta}_b^{(t)}) \propto P(\bar{\eta}_b^{(t)} | \theta^{(t)}) \cdot \mathrm{Dir}(\theta^{(t)} | \alpha_b^{(t)}),$$

where the likelihood is derived from the hierarchical mixture model

$$P(\bar{\eta}_b^{(t)} | \theta^{(t)}) = \prod_{x \in \bar{\eta}_b^{(t)}} \left( \sum_{j=1}^{K} \frac{\theta_j^{(t)}}{N_R^{(t)}} \cdot \mathcal{N}(x | Z_j, \sigma^2 I) \right).$$

The resulting posterior is non-conjugate due to the Gaussian mixture structure and hence is not analytically tractable. However, it admits tractable evaluation for numerical inference methods such as Thompson sampling, MCMC, or variational approximations.

**Assumption 4.** We assume the following hierarchical generative model for each observation $x_i \in \bar{\eta}_b^{(t)}$

* (B1) A latent stronghold index $z_i \in 1, \dots, m$ is drawn according to the multinomial distribution $\mathbb{P}(z_i = j | q) = q_j$.

* (B2) Conditional on $z_i = j$, the spatial observation is drawn from a Gaussian centered at $Z_j$ with isotropic covariance $x_i | z_i = j \sim \mathcal{N}(Z_j, \sigma^2 I)$.

* (B3) The prior distribution over $q$ is Dirichlet $q \sim \mathrm{Dir}(\alpha_b^{(t)})$.

**Proposition 6** (Posterior over Frequency Vectors). Suppose Assumptions (B1)-(B3) hold. Then, the posterior distribution of the latent frequency vector $p$ given the set of observations $\bar{\eta}_b^{(t)}$ satisfies

$$\mathbb{P}_b^{(t)}(q | \bar{\eta}_b^{(t)}) = \frac{1}{B(\alpha_b^{(t)})} \cdot \left( \prod_{x \in \bar{\eta}_b^{(t)}} \sum_{j=1}^{K} q_j \cdot \mathcal{N}(x_i | Z_j, \sigma^2 I) \right) \cdot \prod_{j=1}^{K} q_j^{\alpha_{b,j}^{(t)} - 1},$$

where $B(\alpha_b^{(t)})$ is the posterior normalization constant, and the Dirichlet prior contributes via the multivariate Beta function $B(\alpha_b^{(t)}) \stackrel{\text{def}}{=} \frac{\prod_{j=1}^{K} \Gamma(\alpha_{b,j}^{(t)})}{\Gamma(\sum_{j=1}^{K} \alpha_{b,j}^{(t)})}$.
Proof. Let $q \sim \text{Dir}(\alpha_b^{(t)})$ be the Dirichlet prior over the latent insurgent frequency vector, as in (B3). Each observation $x_i \in \bar{\eta}_b^{(t)}$ is assumed to be generated by a hierarchical latent variable model:

* A latent index $z_i \sim \text{Multinomial}(q)$ selects one of $K$ strongholds (B1),

* Given $z_i = j$, the observed spatial position follows $x_i \sim \mathcal{N}(Z_j, \sigma^2 I)$ (B2).

Marginalizing out the latent variable $z_i$, the likelihood of a single observation given $p$ becomes:

$$ \mathbb{P}(x_i | q) = \sum_{j=1}^{K} q_j \cdot \mathcal{N}(x_i | Z_j, \sigma^2 I). $$

Assuming conditional independence of observations given $p$, the joint likelihood is:

$$ \mathbb{P}(\bar{\eta}_b^{(t)} | q) = \prod_{i=1}^{N} \sum_{j=1}^{K} q_j \cdot \mathcal{N}(x_i | Z_j, \sigma^2 I). $$

The prior over $q$ is:

$$ \mathbb{P}(q) = \frac{1}{B(\alpha_b^{(t)})} \prod_{j=1}^{K} q_j^{\alpha_{b,j}^{(t)} - 1}, $$

where $B(\alpha) = \frac{\prod_j \Gamma(\alpha_j)}{\Gamma(\sum_j \alpha_j)}$ is the multivariate Beta function. Applying Bayes' theorem, the posterior is proportional to:

$$ \mathbb{P}_{b}^{(t)}(q | \bar{\eta}_{b}^{(t)}) \propto \mathbb{P}(\bar{\eta}_{b}^{(t)} | q) \cdot \mathbb{P}(q). $$

Substituting the expressions for likelihood and prior, we obtain:

$$ \mathbb{P}_{b}^{(t)}(q | \bar{\eta}_{b}^{(t)}) \propto \left( \prod_{x \in \bar{\eta}_{b}^{(t)}} \sum_{j=1}^{K} q_j \cdot \mathcal{N}(x | Z_j, \sigma^2 I) \right) \cdot \prod_{j=1}^{K} q_j^{\alpha_{b,j}^{(t)} - 1}. $$

This posterior is not conjugate, but its explicit form allows efficient approximate inference via Monte Carlo or variational methods. $\square$

This posterior corresponds to a Dirichlet–multinomial compound likelihood model with soft assignments, where the latent assignment to strongholds is integrated, resulting in a mixture of Gaussians weighted by the frequency vector $q$.

# 5 Thompson Sampling Algorithm

We formulate the decision-making problem faced by soldiers operating in a counterinsurgency setting as a Generalized Nash Equilibrium Problem (GNEP) in a dynamic, partially observable environment. Each soldier operates under local information constraints and interacts strategically with nearby peers. The GNEP framework captures shared constraints while allowing for agent-specific information and decision processes. Let $s_b^{(t)} := (x_b^{(t)}, x_{-b}^{(t)}, \bar{\eta}_b^{(t)})$ denote the local state of agent $b$ at time $t$, where $x_b^{(t)} \in \mathbb{R}^2$ is the position of the agent, $x_{-b}^{(t)}$ the positions of its neighboring agents and $\bar{\eta}_b^{(t)}$ the aggregated observation history.
At each time step, the agent selects a discrete action $a_b^{(t)} \in \mathcal{A}$, where $\mathcal{A}$ consists of movement directions in an 8 connected grid with resolution $d > 0$:

$$ \delta(a_b) = \begin{cases} (0,0) & \text{stay} \\ (0,+d) & \text{up} \\ (0,-d) & \text{down} \\ (-d,0) & \text{left} \\ (+d,0) & \text{right} \\ (-d,+d) & \text{up-left} \\ (+d,+d) & \text{up-right} \\ (-d,-d) & \text{down-left} \\ (+d,-d) & \text{down-right} \end{cases} $$

The deterministic displacement function $\delta: \mathcal{A} \to \mathbb{R}^2$ maps each action to a spatial update such that

$$ x_b^{(t+1)} = x_b^{(t)} + \delta(a_b^{(t)}). $$

Agents evaluate actions using an instantaneous utility function $u_b$, which balances offensive coordination (neutralizing insurgents) with subjective costs and a coordination penalty that captures local deviations from desired formations. Formally:

$$ u_b(s_b^{(t)}, a_b^{(t)}, \xi_b^{(t)}; \mu) := \mathbb{E}_{\xi_b^{(t)}} [U_b(s_b^{(t)}, a_b^{(t)}; \mu) - S_b(s_b^{(t)}, \xi_b^{(t)})] - \Lambda^{(t)\top} \Phi(x_b^{(t)}, x_{-b}^{(t)}) $$

where $\mu \in [0, 1]$ captures the trust level in civilian reports, and $\Phi$ encodes coordination constraints. To act under uncertainty, each soldier follows a stochastic policy $\pi_b$, mapping state belief pairs to distributions over actions:

$$ \pi_b : (s_b^{(t)}, \xi_b^{(t)}) \mapsto \Delta(\mathcal{A}), $$

where $\Delta(\mathcal{A})$ is the action simplex.

This probabilistic formulation enables adaptive planning in dynamic and partially observable contexts, aligning with the Thompson Sampling paradigm for structured exploration.

Algorithm 1 implements a decentralized Thompson sampling mechanism integrated with strategic planning. At each round, agents sample a belief over insurgent distributions from a Dirichlet posterior, optimize their action sequence through a finite-horizon value function, and update their belief via pseudo-count aggregation from local and civilian observations. This decentralized update allows each agent to maintain a personalized and dynamically evolving estimate of the latent threat configuration, even with limited communication. Crucially, the algorithm supports local consistency while enabling global coordination through iterative best-responses. By embedding the Bayesian update within a strategic planning loop, the method enhances both learning and coordination, accelerating convergence toward equilibrium behaviors in an uncertain setting.
<caption>**Algorithm 1** Distributed inference of insurgent distribution</caption>

**Require:** Prior $\alpha^{(0)}$, budget $N_R^{(0)}$, stronghold positions $(Z_1, \dots, Z_K)$, horizon T, trust $\mu$
1: $\eta_b^{(0)} \leftarrow \emptyset, P_b^{(0)}(\theta) \sim \text{DirMult}(N_R^{(0)}, \alpha^{(0)})$ for all $b \in \mathcal{B}$
2: **for** $t=0$ to $T-1$ **do**
3: &nbsp;&nbsp;&nbsp;&nbsp;**for** each soldier $b \in \mathcal{N}_B$ **do**
4: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**Step 1: Belief Sampling**
5: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; $\hat{\theta}_b^{(t)} \sim P_b^{(t)}(\theta)$
6: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**Step 2: Policy Optimization**
7: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; $\pi_b^{*(t)} \leftarrow \arg\max_{\pi_b} \mathbb{E}_{\pi_b} \left[ \sum_{\tau=t}^{T-1} \gamma^{\tau-t} u_b(s_b^{(\tau)}, a_b^{(\tau)}, \hat{\theta}_b^{(t)}) \right]$
8: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**Step 3: Strategy Execution**
9: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; $a_b^{(t)} \sim \pi_b^{*(t)}(\cdot | s_b^{(t)})$
10: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; $x_b^{(t)} \leftarrow x_b^{(t)} + \delta(a_b^{(t)})$
11: &nbsp;&nbsp;&nbsp;&nbsp;**end for**
12: **Step 4: Civilian Observation and Transmission**
13: &nbsp;&nbsp;&nbsp;&nbsp;**for** each civilian $c \in \mathcal{N}_C$ **do**
14: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Compute:
15: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; $\eta_c^{(t)} \leftarrow \{x_r : \tau < t, \|x_r - x_c^{(\tau)}\| \le \bar{\rho}\}$
16: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; **for** each soldier $b$ such that $\|x_c^{(t)} - x_b^{(t)}\| \le \bar{\rho}$ **do**
17: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sample $q_{cb} \sim \text{Bernoulli}(\mu)$
18: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; **if** $q_{cb} = 1$ **then**
19: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Transmit $\eta_c^{(t)}$ to soldier $b$
20: &nbsp;&nbsp;&nbsp;&nbsp;**end if**
21: &nbsp;&nbsp;&nbsp;&nbsp;**end for**
22: &nbsp;&nbsp;&nbsp;&nbsp;**end for**
23: **Step 5: Aggregation and Bayesian Update**
24: &nbsp;&nbsp;&nbsp;&nbsp;**for** each soldier $b \in \mathcal{N}_B$ **do**
25: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Form the local observation set:
$$ \bar{\eta}_b^{(t)} \leftarrow \left( \bigcup_{b' \in \mathcal{V}_b^{(t)}} \eta_{b'}^{(t)} \right) \cup \left( \bigcup_{c \in \mathcal{C}} \eta_c^{(t)} \right) $$
26: Compute pseudo-counts:
27: &nbsp;&nbsp;&nbsp;&nbsp; $\alpha_{b,j}^{(t)} \leftarrow \sum_{x \in \bar{\eta}_b^{(t)}} \mathbf{1}_{\{\phi(x)=j\}}$
28: Update belief:
$$ P_b^{(t)}(\theta^{(t)} | \bar{\eta}_b^{(t)}) \propto P(\bar{\eta}_b^{(t)} | \theta^{(t)}) \cdot P_b^{(t)}(\theta^{(t)}) $$
29: **end for**
30: **end for**

# 6 Performance

**Definition 1** (Value Function under Strategic Belief). Let $\pi_b$ be a stochastic policy, $s \in \mathcal{S}$ the local state of the soldier $b$, and $\theta$ a latent parameter representing the strategic allocation of insurgents across $K$ strongholds. The finite-horizon value function under policy $\pi_b$, given latent parameter $\theta$, is defined as

$$ V_b^{\pi_b}(s, \theta) := \mathbb{E}_{\pi_b} \left[ \sum_{\tau=0}^{T-1} \gamma^{\tau} \cdot u_b(s_b^{(\tau)}, a_b^{(\tau)}, \xi_b^{(\tau)}; \mu) \mid s_b^{(\tau)} = s \right] $$

where $\gamma \in (0, 1)$ is the discount factor.

**Definition 2** (Bayesian Regret). Let $\theta^*$ denote the true latent insurgent distribution. Let $\pi_b^* \in \arg\max_{\pi_b} V_b^{\pi_b}(s; \theta^*)$. Let $\pi_b^{(t)} := \pi_b(\cdot | \theta_b^{(t)})$ be the policy selected at time $t$ by sampling $\theta_b^{(t)} \sim P_b^{(t)}(\theta | \mathcal{H}_t)$, where $\mathcal{H}_t$ is the agent's observation history. Then, the Bayesian regret incurred up to horizon T is defined as $R_b(T; \theta^*) := \sum_{t=0}^{T-1} [V_b^{\pi_b^*}(s_b^{(t)}; \theta^*) - \mathbb{E}_{\pi_b^{(t)}}[V_b^{\pi_b^{(t)}}(s_b^{(t)}; \theta^*)]]$.

**Assumption 5.** We assume the following conditions:

(C1) $\theta^* \in \Delta^{K-1}$ is fixed and unknown.

(C2) Agent $b$'s Bayesian belief $P_b^{(t)}(\theta | \mathcal{H}_t) = \text{Dir}(\alpha^{(t)})$ is updated through a Dirichlet multinomial model.

(C3) $u_b(s, a, \xi; \mu)$ is bounded, i.e., $\exists M > 0$ such that $|u_b| \le M$ almost surely.
(C4) $V_b^{\pi_b}(s; \theta) = \langle \phi_b^{\pi_b}(s), \theta \rangle$, where $\phi_b^{\pi_b}(s) \in \mathbb{R}^K$ is a known bounded feature vector induced by the policy.

**Lemma 7.** Suppose that agent b operates under Assumptions (C1)-(C4), using a stochastic policy $\pi_b^{(t)}$ sampled from its posterior over $\theta^* \in \Delta^{K-1}$. Then

$$ \mathbb{E}[\mathcal{R}_b(T; \theta^*)] \leq \frac{M}{\gamma} \cdot \sqrt{2T \cdot \mathbb{I}(\theta^*; \mathcal{H}_T)}, $$

where $\mathbb{I}(\theta^*; \mathcal{H}_T)$ is the mutual information between the latent parameter and the agent's observation history.

*Proof.* We start from the definition of Bayesian regret:

$$ \mathcal{R}_b(T; \theta^*) := \sum_{t=0}^{T} \left[ V_b^{\pi_b^*}(s_b^{(t)}; \theta^*) - \mathbb{E}_{\pi_b^{(t)}} \left[ V_b^{\pi_b^{(t)}}(s_b^{(t)}; \theta^*) \right] \right], $$

and note that the optimal policy $\pi_b^*$ is chosen with full knowledge of $\theta^*$, while $\pi_b^{(t)} \sim \mathbb{P}_b^{(t)}(\theta | \mathcal{H}_t)$. Using the linearity assumption (C4), we write:

$$ V_b^{\pi}(s; \theta) = \langle \phi_b^{\pi}(s), \theta \rangle, \quad \text{for any fixed } \pi. $$

Thus, the regret at time $t$ becomes:

$$ V_b^{\pi_b^*}(s_b^{(t)}; \theta^*) - \mathbb{E}_{\pi_b^{(t)}} \left[ \langle \phi_b^{\pi_b^{(t)}}(s_b^{(t)}), \theta^* \rangle \right]. $$

We define the random variable $A_t := \pi_b^{(t)}$, and the regret becomes:

$$ \mathcal{R}_b(T) = \sum_{t=0}^{T} \left( \langle \phi_b^{\pi_b^*}(s_b^{(t)}), \theta^* \rangle - \mathbb{E}_{A_t} \left[ \langle \phi_b^{A_t}(s_b^{(t)}), \theta^* \rangle \right] \right). $$

Let us denote by $D = \sup_{\pi,s} \|\phi_b^{\pi}(s)\| \cdot \|\theta^*\| \le M/\gamma$, due to the boundedness of the value function under discounting and bounded reward. Then, we apply the following regret-information inequality (Russo and Van Roy, 2016): For bounded utility and linear value function with Thompson Sampling,

$$ \mathbb{E}[\mathcal{R}_b(T)] \leq D \cdot \sqrt{2T \cdot \mathbb{I}(\theta^*; \mathcal{H}_T)}. $$

Substituting $D = \frac{M}{\gamma}$ completes the proof. $\square$

**Theorem 8.** Let agent b maintain a Bayesian belief $\mathbb{P}_b^{(t)}(\theta | \mathcal{H}_t) = \text{Dir}(\alpha^{(t)})$, with updates from multinomial observations (e.g., via local sensors and civilian reports). Then, the mutual information between the true parameter $\theta^* \in \Delta^{K-1}$ and the history $\mathcal{H}_T$ satisfies

$$ \mathbb{I}(\theta^*; \mathcal{H}_T) \leq \frac{K}{2} \log \left( 1 + \frac{n}{K} \right), $$

where $n = \sum_{k=1}^K \alpha_k^{(t)}$ is the total number of pseudo-counts. If $n = \mathcal{O}(T \log T)$, then the expected regret is bounded as

$$ \mathcal{R}^{\text{Bayes}}(T) = \mathbb{E}[\mathcal{R}_b(T; \theta^*)] = \mathcal{O}\left(\frac{M}{\gamma} \cdot \sqrt{KT \log T}\right). $$

*Proof.* Under a Dirichlet-multinomial model:

* The posterior is $\text{Dir}(\alpha^{(t)})$,

* The prior is $\text{Dir}(\alpha^{(0)})$,

* The sufficient statistic is the observation vector $\mathcal{H}_T \equiv \mathbf{n} = (n_1, \dots, n_K)$,
• Let $n = \sum_k n_k = \sum_k \alpha_k^{(T)}$.

Using the result from information theory (see Russo and Van Roy, 2016), the mutual information for Dirichlet-multinomial satisfies:

$$ \mathbb{I}(\theta^*; \mathcal{H}_T) \leq \frac{K}{2} \log \left(1 + \frac{n}{K}\right). $$

Combining with Lemma 7, we get:

$$ \mathbb{E}[\mathcal{R}_b(T)] \leq \frac{M}{\gamma} \cdot \sqrt{2T \cdot \frac{K}{2} \log \left(1 + \frac{n}{K}\right)}. $$

If $n = \mathcal{O}(T \log T)$, then:

$$ \mathbb{E}[\mathcal{R}_b(T)] = \mathcal{O}\left(\frac{M}{\gamma} \cdot \sqrt{KT \log T}\right). $$

□

## Comparison with UCB and $\epsilon$-Greedy Regrets

We define the expected utility (or value) of policy $\pi_k$ under the true parameter $\theta^*$ as $\nu_k := V_b^{\pi_k}(s; \theta^*) := u_b(s_b^{(t)}, \theta^*)$, where expectation is taken over the randomness induced by the environment and the evolution of belief under policy $\pi_k$.

Let $\pi_b^* \in \arg\max_{\pi_k} \{\nu_k\}$ be an optimal policy that achieves the expected utility $\nu^* := \max_k \nu_k$. We define the suboptimality gap of policy $\pi_k$ as $\Delta_k := \nu^* - \nu_k \geq 0$.

The Upper Confidence Bound (UCB) algorithm selects, at each time step $t$, the policy maximizing an upper confidence estimate of its expected utility. The classical regret bound for UCB in the stochastic multiarmed bandit setting (Auer et al., 2002) applies here under the assumption that the observed utilities are conditionally sub-Gaussian and bounded in $[0, 1]$, taking the form

$$ \mathcal{R}^{\text{UCB}}(T) = \mathcal{O}\left(\sum_{k:\Delta_k>0} \frac{\log T}{\Delta_k}\right). $$

The $\epsilon$-Greedy strategy with a decaying exploration rate $\epsilon_t = c/t$ or $\epsilon_t = c/\sqrt{t}$ achieves a regret bound of

$$ \mathcal{R}^{\epsilon\text{-Greedy}}(T) = \mathcal{O}(K\sqrt{T} \log T), $$

as shown by Even-Dar et al. (2006).

<figure><img src="image_3.png" /><figcaption>Figure 1: Numerical comparison of the regret bounds.</figcaption></figure>

Figure 1 illustrates the cumulative regret trends. The Bayesian method exhibits slower regret growth, reflecting stronger adaptivity over time in accordance with its theoretical bound. In contrast, $\epsilon$-Greedy and UCB incur greater regret, due to less efficient or overly conservative exploration.
# 7 Simulations

We consider a counter-insurgency operation in an urban environment involving 32 government soldiers and 20 insurgents, randomly spread over $K = 3$ strongholds, each comprising three hidden cells embedded within a $10 \times 10$ urban grid. The latent allocation of insurgents is governed by the asymmetric prior $\theta^* = (6, 10, 4)$. The position of each insurgent $x_r \in \mathbb{R}^2$ is sampled from an isotropic Gaussian centered on their assigned bastion $Z_j$, with covariance $\sigma_r^2 I$, where $\sigma_r = 1.0$. Around each stronghold, 40, 70, and 100 civilians introduce environmental complexity, respectively. The initial position of each civilian is drawn from an isotropic Gaussian uniformly centered in the feasible domain $[0, 10]^2$, with covariance $\sigma_c^2 I$, where $\sigma_c = 4.5$. At every timestep, civilians move stochastically via Brownian dynamics. We adopt a finite-horizon planning scheme with a temporal horizon $T = 50$ and a discount factor $\gamma = 0.95$.

<figure><img src="image_4.png" /><figcaption>Figure 2: Comparison of the impact $\mu$.</figcaption></figure>

Figure 2 illustrates the impact of information quality quantified by the parameter $\mu$ on the average value achieved under optimal policies.

<figure><img src="image_5.png" /><figcaption>Figure 3: Convergence analysis.</figcaption></figure>

Figure 3 compares the average value functions of the soldiers (blue curve) with the upper bound of the Bayesian regret $R^{\text{Bayes}}(T)$ (red dashed curve) over time. In the early stages, the average performance of the agents increases rapidly, indicating an effective exploration phase. Around $t = 20$, the blue curve reaches a plateau, suggesting convergence toward a near-optimal strategy.
Meanwhile, the theoretical bound continues to grow sublinearly, reflecting the relatively diminishing regret. This behavior confirms the efficiency of the distributed Thompson sampling approach, even at a moderate level of trust ($\mu = 0.65$).

# 8 Conclusion

We propose a decentralized Bayesian framework for multi-agent planning under uncertainty, formulating counterinsurgency coordination as a dynamic Generalized Nash Equilibrium. By integrating Thompson sampling into a finite-horizon utility-maximizing strategy, agents iteratively refine beliefs and coordinate actions using partial, trust-weighted civilian observations. The results demonstrate that information quality rather than quantity drives strategic effectiveness.

# References

P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite-time analysis of the multiarmed bandit problem. *Machine learning*, 47:235-256, 2002.

S. J. Deitchman. A lanchester model of guerrilla warfare. *Operations Research*, 10(6):818-827, 1962.

E. Even-Dar, S. Mannor, Y. Mansour, and S. Mahadevan. Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems. *Journal of machine learning research*, 7(6), 2006.

M. I. Jordan, T. Lin, and M. Zampetakis. First-order algorithms for nonlinear generalized nash equilibrium problems. *Journal of Machine Learning Research*, 24(38):1-46, 2023.

E. H. Kaplan, M. Kress, and R. Szechtman. Confronting entrenched insurgents. *Operations Research*, 58(2):329-341, 2010.

M. Kress and N. J. MacKay. Bits or shots in combat? the generalized deitchman model of guerrilla warfare. *Operations Research Letters*, 42(1):102-108, 2014.

M. Kress and R. Szechtman. Why defeating insurgencies is hard: The effect of intelligence in counterinsurgency operations—a best-case scenario. *Operations Research*, 57(3):578-585, 2009.

F. W. Lanchester. *Aircraft in warfare: The dawn of the fourth arm*. Constable limited, 1916.

D. Russo and B. Van Roy. An information-theoretic analysis of thompson sampling. *Journal of Machine Learning Research*, 17(68):1-30, 2016.
## A Parameters

<table><caption>Table 1: Parameters</caption><thead><tr><th>Variable</th><th>Description</th><th>Value</th></tr></thead><tbody><tr><td>N<sup>(0)</sup><sub>B</sub></td><td>No. of soldiers</td><td>32</td></tr><tr><td>N<sup>(0)</sup><sub>C</sub></td><td>No. of civilians</td><td>40, 70, 100</td></tr><tr><td>N<sup>(0)</sup><sub>R</sub></td><td>No. of insurgents</td><td>20</td></tr><tr><td>K</td><td>No. of strongholds</td><td>3</td></tr><tr><td>Z<sub>1</sub></td><td>Coordinates of stronghold 1</td><td>(2, 6.5)</td></tr><tr><td>Z<sub>2</sub></td><td>Coordinates of stronghold 2</td><td>(5.5, 8)</td></tr><tr><td>Z<sub>3</sub></td><td>Coordinates of stronghold 3</td><td>(9, 7)</td></tr><tr><td>T</td><td>Time horizon</td><td>50</td></tr><tr><td>ρ</td><td>Sensor radius</td><td>1.5</td></tr><tr><td>ρ̄</td><td>Sensor radius</td><td>1.5</td></tr><tr><td>σ<sub>r</sub></td><td>Insurgent's dispersion radius</td><td>1.0</td></tr><tr><td>σ<sub>c</sub></td><td>Civilians movement parameter</td><td>2</td></tr><tr><td>β<sub>0</sub></td><td>Engagement score parameter</td><td>3</td></tr><tr><td>β<sub>1</sub></td><td>Engagement score parameter</td><td>0.2</td></tr><tr><td>β<sub>2</sub></td><td>Engagement score parameter</td><td>4.0</td></tr><tr><td>μ</td><td>Trust parameter</td><td>0.0, 0.65, 1.0</td></tr><tr><td>α</td><td>Objective function parameter</td><td>1.5</td></tr><tr><td>θ<sup>*</sup></td><td>latent allocation of insurgents</td><td>6|10|4</td></tr></tbody></table>

## B Sensitivity analysis on civilian numbers

<figure><img src="image_6.png" /><figcaption>Figure 4: $N_C = 40$</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 5: $N_C = 70$</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>Figure 6: $\mathcal{N}_C = 100$</figcaption></figure>