Title: Safe Exploitative Play with Untrusted Type Beliefs
Venue: NeurIPS 2024
Accepted: yes
Keywords: Bayesian games

type beliefs

opportunity and risk tradeoff
Abstract: The combination of the Bayesian game and learning has a rich history, with the idea of controlling a single agent in a system composed of multiple agents with unknown behaviors given a set of types, each specifying a possible behavior for the other agents. The idea is to plan an agent's own actions with respect to those types which it believes are most likely to maximize the payoff. However, the type beliefs are often learned from past actions and likely to be incorrect. With this perspective in mind, we consider an agent in a game with type predictions of other components, and investigate the impact of incorrect beliefs to the agent’s payoff. In particular, we formally define a tradeoff between risk and opportunity by comparing the payoff obtained against the optimal payoff, which is represented by a gap caused by trusting or distrusting the learned beliefs.Our main results characterize the tradeoff by establishing upper and lower bounds on the Pareto front for both normal-form and stochastic Bayesian games, with numerical results provided.

Safe Exploitative Play with Untrusted Type Beliefs

Abstract

The combination of the Bayesian game and learning has a rich history, with the idea of controlling a single agent in a system composed of multiple agents with unknown behaviors given a set of types, each specifying a possible behavior for the other agents. The idea is to plan an agent's own actions with respect to those types which it believes are most likely to maximize the payoff. However, the type beliefs are often learned from past actions and likely to be incorrect. With this perspective in mind, we consider an agent in a game with type predictions of other components, and investigate the impact of incorrect beliefs to the agent's payoff. In particular, we formally define a tradeoff between risk and opportunity by comparing the payoff obtained against the optimal payoff, which is represented by a gap caused by trusting or distrusting the learned beliefs. Our main results characterize the tradeoff by establishing upper and lower bounds on the Pareto front for both normal-form and stochastic Bayesian games, with numerical results provided.

1 Introduction

' The Chinese symbol for crisis is composed of two elements: one signifies danger and the other opportunity. ' - Lewis Mumford, 1944

The famous interpretation of the Chinese word for 'crisis' (known as 危 機 ), although based on mistaken etymology, captures the dual nature of risk and opportunity. It provides an interesting metaphor for the inherent complexities within real-world multi-agent systems. These systems, where risk and opportunity are often inextricably linked, play a pivotal role across diverse domains, ranging from human-AI collaboration [1] and cyber-physical systems [2], to highly competitive environments like real-time strategy games [3] and poker [4].

In conventional applied and theoretical frameworks, it is typically assumed that all agents either cooperate or adhere to pre-defined policies [5, 6, 7, 8]. However, real-world scenarios often defy these simplifications, presenting agents that display a spectrum of behaviors ranging from cooperative, to heterogeneous, irrational, or even adversarial [9]. This deviation from expected behavior patterns

complicates the dynamics of multi-agent systems, as agents cannot reliably predict the actions of their counterparts. The uncertainty regarding whether to trust or distrust predictions naturally leads to a critical tradeoff between the coexisted risk and opportunity, reflecting the dual aspects highlighted in Mumford's remark.

As critical examples, Bayesian games [10, 11] provide an approach for modeling differing types of agents in strategic environments. In these games, players form beliefs about others' types and update beliefs in response to observed actions and choose their actions accordingly. For example, in competitive settings like poker or even in simple games like matching pennies, deviating from the game theoretic optimal (GTO) strategy [12, 13] to exploit weaker opponents can be beneficial, but this approach also relies on potentially flawed type beliefs [6], making it risky to take advantage of such side-information. Similarly, in real-world problems like security games, having a prior distribution of attackers' behavioral types in a Bayesian game setting leads to advantages. However, exploiting incorrect types can be risky compared to just using minimax strategies. Despite the ubiquity of incorrect type beliefs in practical scenarios, limited attention has been paid to explore such a tradeoff, with exceptions in designing heuristically safe and exploitative strategies in specific contexts, such as with Byzantine adversaries [9] and sequential games [14].

Inaccuracies in beliefs about others' types may arise from factors such as using out-of-distribution data to generate priors, changes in opponents' behaviors, or mismatches between hypothesized types and the optimal type space, etc. As noted in [15], it is evident that prior beliefs significantly affect the long-term performance of type-based learning algorithms like the Harsanyi-Bellman Ad Hoc Coordination (HBA). Although algorithms like HBA demonstrate asymptotic convergence to correct predictions or type distributions through various methods of estimating posterior beliefs, and methods exist for detecting inaccuracies in type beliefs using empirical behavioral hypothesis testing [15], the theoretical capabilities of general algorithms remain uncertain.

In general, relying on learned beliefs presents a fundamental tradeoff between the potential payoffs from exploitative play and the risk of incorrect type beliefs. Given the potential inaccuracies in these type beliefs, using them to exploit opponents could lead to high-risk strategies. Conversely, not exploiting these beliefs might result in overly cautious play. Therefore, it is natural to investigate the impact of incorrect beliefs on the agent's payoff, in terms of a tradeoff between trusting or distrusting the beliefs of types provided by type-based learning algorithms (e.g., Bayesian learning [16], best response dynamics [17], and policy iteration with neural networks [18], etc.) in multi-agent systems. To summarize the focus of this paper, we aim to address the following critical question:

What is the fundamental tradeoff between trusting/distrusting type beliefs in games?

Contributions. Motivated by the above question, we analyze the following payoff gap that arises from erroneous type beliefs:

where θ and θ ⋆ are predicted and true type beliefs respectively; d ( · , · ) measures the distance between θ and θ ⋆ bounded from above by ε and will be formally specified with concrete model contexts in Section 3 and Section 4, together with the payoff, denoted by Payoff ( · , · ) as a function of the true type θ ⋆ and the deployed strategy π ∈ Π . We denote the given strategy space by Π . The agent's strategy π ( θ ) may depend on the type belief θ . Overall, the payoff difference (1) above quantifies the worst-case gap between the optimal payoff (obtained by an optimal strategy in Π that maximizes Payoff ( · , θ ⋆ ) ), and the payoff corresponding to π .

In particular, when the agent's strategy π trusts the belief θ , it takes the opportunity to close the gap in (1) when the belief error ε is small. However, when ε increases, such a strategy incurs a high risk since it trusts the incorrect θ . Evaluating the payoff gap in (1) naturally yields a tradeoff between opportunity and risk, as illustrated on the right of Figure 1. To be more precise, we measure the tradeoff between two important quantities:

(Missed) Opportunity : ∆(0; π ) , corresponding to the case when the type beliefs are correct;

Risk : max ε> 0 ∆( ε ; π ) measuring the payoff difference incurred by worst-case incorrect beliefs.

In summary, the (missed) opportunity measures the discrepancy of a strategy π from the optimal strategy, which is aligned with the ground truth belief θ ⋆ of other players in terms of the obtained payoff. Additionally, the risk quantifies how inaccurate beliefs impact the difference in terms of

Env

Action

Actions

State

Risk

Trusting

Type belief θ E

Type 1

Type2

Distrusting

Agent

Trusting vs Distrusting?

Opponents

θ

MissedOpportunity

payoffs in the worst case. The goal of this paper is to investigate safe and exploitative strategies in Bayesian games that achieve near-optimal opportunity and risk.

Our main results are two-fold. Firstly, in normal-form Bayesian games, we characterize a tradeoff between the opportunity and risk. We consider a strategy as a convex combination of a safe strategy and the best response given type beliefs. Upper bounds on opportunity and risk are provided in Theorem 3.1. Conversely, lower bounds that hold for any mixed strategy are shown in Theorem 3.2. Notably, when the game is fair, and the hypothesis set Θ is sufficiently large, these bounds tightly converge. Secondly, we explore a dynamic setting in stochastic Bayesian games, where an agent, provided with type beliefs about other players, engages in interactions over time, as illustrated in Figure 1. Unlike the normal-form approach, we utilize a value-based strategy that establishes upper bounds on opportunity and risk, as outlined in Theorem 4.1. Additionally, Theorem 4.2 provides lower bounds on opportunity and risk that differ from the upper bounds by multiplicative constants, yielding a characterization of the opportunity-risk tradeoff. Finally, a case study of a security game, simulating a defender protecting an elephant population from illegal poachers, is provided in Section 5.

2 Related Work

Learning Types in Games. The concept of type-based methods dates back to the development of Bayesian games, as first established by Harsanyi in the late 1960s [10, 11]. The concepts of learning and updating beliefs appear in pioneering works like the adaptive learning [19]. While much game theory research, including work by Kalai et al. [20], focuses on equilibrium analysis through Bayesian belief-based learning, other studies, such as those by Nachbar et al. [21, 22] and [23, 24], reveal the challenges players face in making correct predictions while playing optimally under certain game conditions and assumptions. From an application perspective, Southey et al. [25] applied type-based methods to poker, where players' hands are partially hidden. They demonstrated how to maintain and use beliefs to determine the best strategies in this setting. Besides classic results exemplified above, closely related to our results, the recent work by Milec et al. [14] addresses the limitations of Nash equilibrium strategies in two-player extensive-form games, particularly their inability to exploit the weaknesses of sub-optimal opponents. They defined the exploitability of a strategy as the expected payoff that a fully rational opponent can gain beyond the game's base value and introduced a method that ensures safety, defined as an upper limit on exploitability compared to the payoff obtained using a Nash equilibrium strategy. However, the proposed tradeoff between the exploitation of the opponent given the correct model and safety against an opponent who can deviate arbitrarily from the predicted model is applicable specifically to an algorithm that employs a continual depth-limited restricted Nash response.

Online Decision-Making with Predictions. The tradeoff between opportunity and risk analyzed in this work is motivated by the recent progress in algorithms with predictions, also known as learning-augmented algorithms [26, 27] for online decision-making problems such as caching [28, 29, 30], bipartite matching [31], online optimization [32, 33], control [34, 35, 36, 37], valued-based reinforcement learning [38, 39, 40], and real-world applications [41, 42, 43, 44]. This line of work investigates the impact of untrusted predictions on two key metrics known as consistency and robustness, which are defined based on the competitive ratio of the considered contexts. It is also worth highlighting that previous works in decision-making often provide best-of-both-worlds guarantees for both stochastic and adversarial environments [45, 46], while the results in this work shed light on studying the intermediate regimes that do not fully align with either stochastic or

adversarial settings and delves into interactive environments formed by players whose behaviors may deviate from the type beliefs.

Stochastic Bayesian Games. Our results presented in Section 4 draw on foundational concepts from stochastic Bayesian games as outlined by Albrecht et al. [47, 15], which merge concepts of the Bayesian games [10, 11] and the stochastic games [48], with applications in cooperative multi-agent reinforcement learning [9]. In [15], three methods-product, sum, and correlated-for integrating observed evidence into posterior beliefs have been explored. Specifically, it has been demonstrated that the Harsanyi-Bellman Ad Hoc Coordination (HBA) eventually make right future predictions under specific conditions using the product posterior beliefs. However, while HBA may eventually align with the correct type distribution, it is not guaranteed to learn it with the product posterior beliefs. Under certain conditions, HBA with the sum and correlated posterior converges to the correct type distribution. Nonetheless, even though methods like empirical behavioral hypothesis testing are available to detect inaccuracies in type beliefs, a comprehensive theoretical analysis is still lacking.

3 Normal-Form Bayesian Games with Untrusted Type Beliefs

Let ∥ · ∥ 1 denote the ℓ 1 -norm and ∥ · ∥ max denote the (element-wise) max norm.

3.1 Problem Setting, Opportunity, and Risk

Consider a normal-form game (NFG) with two players: Player 1 possesses a payoff matrix A ∈ R a × b with ∥ A ∥ max ≤ α , where the first player has a choices of the rows and the second player has b choices of the columns of A . Player 1 forms a belief about Player 2's mixed strategy, denoted by a distribution ρ over a set of hypothesized strategies Θ that contains a ground truth strategy y ⋆ . 2

As a concrete example of the payoff gap proposed in (1), the following benchmark characterizes the gap between payoffs obtained by a strategy with the machine-learned belief ρ and an optimal strategy knowing y ⋆ beforehand:

given a fixed policy π : P Θ → P a that outputs a mixed strategy of the first player knowing ρ , where d ( ρ, y ⋆ ) := ∥ E ρ [ y ] -y ⋆ ∥ 1 , P Θ and P a are the sets of probability distributions on Θ and a different choices of rows respectively.

In particular, we focus on measuring a tradeoff between two important quantities, the (missed) opportunity ∆ NFG (0; π ) and the risk max ε> 0 ∆ NFG ( ε ; π ) . The former measures how far the considered strategy π is away from the optimal strategy knowing the ground truth type y ⋆ of Player 2 in terms of the payoff obtained; the latter quantifies the worst-case impact of inaccurate belief on the payoff difference.

3.2 Motivating Example: Matching Pennies

Before presenting general results, we illustrate the underlying concepts through a simple normal-form game. Consider the classic matching pennies game as an example, where the mixed strategies for the players are defined as follows.

Suppose a = b = 2 and each of the two players has the option to choose either Heads ( H ) or Tails ( T ). Let y ⋆ ∈ [0 , 1] be the true probability that Player 2 plays H and 1 -y ⋆ the probability of playing T . Suppose the hypothesis set Θ contains all possible mixed strategies, each corresponding to a type of Player 2. Then, Player 1 receives a belief of y ⋆ , denoted by y , and chooses a strategy π that depends on y . Similarly, let x = π ( y ) be the probability that Player 1 plays H and 1 -x the probability of playing T . If two players' actions match, Player 1 will receive 1 and -1 otherwise. The problem setting is summarized in Figure 2. Given y ⋆ and x , the expected payoff is therefore (2 y ⋆ -1)(2 x -1) . The best response of Player 1, depending on y , is characterized by:

H

T

H

1.

一

1

-1,1

y

★

Risk

2

Trusting

T

-1,1

1,

一

-y

Risk+Opportunity=2

RowPlayer

ColumnPlayer

Distrusting

Player 1

π

1 -π(y)

Typebelief y

Player 2

2

Trusting vs Distrusting?

△mp(O; π) Missed Opportunity

Given a strategy π : [0 , 1] → [0 , 1] , the payoff gap in (2) for this example is instantiated as

Impossibility. We first consider an arbitrary strategy π : [0 , 1] → [0 , 1] that misses at most (1 -λ ) opportunity. Suppose π chooses H with a probability π ( y ) and T with a probability 1 -π ( y ) . Therefore, setting ε = 0 , the payoff gap must satisfy

which implies π (0) ≤ (1 -λ ) / 2 and π (1) ≥ (1 + λ ) / 2 by setting y = 0 and y = 1 , respectively. Thus, plugging in y ⋆ = 1 and y = 0 , we conclude that

since BR (1) = 1 by (3). This implies that π has at least (1 + λ ) risk. Note that this argument holds for any arbitrarily chosen π . In a word, any strategy misses at most 1 -λ opportunity must incur at least 1 + λ risk, in terms of the payoff.

Mixed Strategy Existence. Now, let us construct a concrete strategy to derive upper bounds on the (missed) opportunity and risk. The idea is to combine the strategy BR ( y ) , which exploits the belief y of types, and the Nash equilibrium strategy of this game, known as x = 1 / 2 , which is a solution of the minimax problem min y ∈ [0 , 1] max x ∈ [0 , 1] (2 y -1)(2 x -1) that enhances safety.

Fix π ( y ) := λ BR ( y ) + (1 -λ ) x a mixed strategy as a convex combination of the two strategies.

The payoff gap in (4) satisfies

Opportunity: Therefore, there exists a mixed strategy π such that when ε = 0 (i.e., BR ( y ) = BR ( y ⋆ ) ), its (missed) opportunity is bounded by

Risk: Moreover, maximizing over ε such that y = 0 , y ⋆ = 1 , BR ( y ⋆ ) -BR ( y ) = 1 , the risk for π always satisfies max ε ∆ MP ( ε ; π ) ≤ 1 + λ .

In a word, we find a strategy that misses 1 -λ opportunity and meanwhile has 1 + λ risk.

Pareto Optimality In conclusion, above construction shows that there is a mixed strategy π for the matching pennies that misses (1 -λ ) opportunity and incurs (1 + λ ) risk. Conversely, any strategy that misses at most (1 -λ ) opportunity must have at least (1 + λ ) risk. The segment on the right of Figure 2 represents a Pareto front, and the strategy constructed as a convex combination is confirmed to be Pareto optimal following the arguments above. This motivating example indicates a tight tradeoff between opportunity and risk for matching pennies. In the sequel, we further generalize this result to normal-form games.

3.3 Opportunity-Risk Tradeoff for Normal-Form Games

In general, the opportunity-risk tradeoff depends on the hypothesis set Θ that contains a subset of candidate strategies and the payoff matrix of the game. We state useful definitions to characterize properties of the hypothesis set Θ and the considered normal-form game.

Definition 1. Given a hypothesis set Θ , we let the diameter of Θ with respect to the ℓ 1 -norm be η (Θ) := max y,z ∈ Θ ∥ y -z ∥ 1 . Define the following type intensity

Furthermore, with fixed Θ and A , we define the maximum and value of the game by

The type intensity, as defined in (5), quantifies the divergence between two distributions within the set Θ , specifically those that maximize the given objective. As the density of Θ increases, the value of κ (Θ) approaches 1 .

Theorem 3.1 (NFG EXISTENCE) . Fix any Θ and consider a general-sum normal-form game where Player 1 has a payoff matrix A ∈ R a × b with µ Θ ( A ) ≤ µ and ν Θ ( A ) ≥ ν . For any 0 ≤ λ ≤ 1 , there exists a mixed strategy π : P Θ → P a for Player 1 that misses (1 -λ ) ( µ -ν ) opportunity and has (1 -λ ) ( µ -ν ) + λµη (Θ) risk.

To show Theorem 3.1, we construct a mixed strategy as follows. Denote by x as the following safe/minimax strategy of Player 1, which is the Nash equilibrium when the game is zero-sum:

Motivated by the matching pennies example in Section 3.2, Player 1 implements a mixed strategy π ( ρ ) := λ ˜ x +(1 -λ ) x given the predicted belief ρ as a distribution over types in Θ , where ˜ x ∈ P a is a best response strategy given ρ such that x ⊤ A E ρ [ y ] is maximized. The detailed proof of Theorem 3.1 is provided in Appendix A.

Moreover, we further show the following impossibility result, indicating that the tradeoff in Theorem 3.1 is tight. We relegate the proof of Theorem 3.2 to Appendix B.

Theorem 3.2 (NFG IMPOSSIBILITY) . For any Θ satisfying κ (Θ) ≥ 0 , there is a payoff matrix A ∈ R a × b with µ Θ ( A ) ≤ µ and ν Θ ( A ) ≥ ν such that for any 0 ≤ λ ≤ 1 , if any mixed strategy π : P Θ → P a for Player 1 misses at most (1 -λ ) ( µ -ν ) opportunity, then it incurs at least ( κ (Θ) µ -ν ) (1 + λ ) risk.

In particular, Theorem 3.1 and 3.2 together imply the following special case when the hypothesis set Θ contains all possible mixed strategies in P b . This corollary highlights that for a fair game, the tradeoff is tight and the mixed strategy π ( ρ ) := λ ˜ x +(1 -λ ) x with λ ∈ [0 , 1] is Pareto optimal when the hypothesis set contains all possible mixed strategies.

Corollary 3.1 (NFG PARETO OPTIMALITY) . Suppose Θ = P b and the game is fair, where Player 1 has a payoff matrix A ∈ R a × b with ∥ A ∥ max ≤ α . For any 0 ≤ λ ≤ 1 , there exists a mixed strategy π for Player 1 that misses (1 -λ ) α opportunity and has (1 + λ ) α risk. Furthermore, there is a payoff matrix A ∈ R a × b with ∥ A ∥ max ≤ α such that for any 0 ≤ λ ≤ 1 if any mixed strategy π for Player 1 misses at most (1 -λ ) α opportunity, then it incurs at least (1 + λ ) α risk.

It is worth noting that the requirement can be relaxed straightforwardly so that as long as the hypothesis set contains two mixed strategies such that κ (Θ) = 1 in (5), the statement holds. The proof of Corollary 3.1 can be found in Appendix C.

4 Stochastic Bayesian Games with Untrusted Type Beliefs

In many real-world applications, agents are coupled through a shared state in stochastic Bayesian games (SBG) [47, 15], which combines standard Bayesian games [10, 11] with the stochastic games [48]. In this section, we consider an infinite-horizon time-varying discounted n -player stochastic Bayesian game, represented by a tuple M := ⟨S , A , Θ , σ, p, r, γ ⟩ , and analyze the impact of beliefs on the opportunity-risk tradeoff.

4.1 Preliminaries: MDP for Stochastic Bayesian Games

̸

Let S be a finite set of states. Write N := { 1 , . . . , n } . Let A ( i ) be the set of finitely many actions that player i ∈ N can take at any state s ∈ S . Furthermore, A := A (1) ×··· × A ( n ) denotes the set of action profiles a = ( a ( i ) : i ∈ N ) with a ( i ) ∈ A ( i ) . The types of other players are unknown to the i -th player. The set Θ = ∏ j = i Θ( j ) is a product type space where each opponent j chooses types from Θ j . We focus on the decision-making of the i -th player, considering stationary (Markov) strategies. 3 At each time t ≥ 0 each player j = i uses a mixed strategy π j : S × Θ( j ) → P A ( j ) parameterized by a type that depends on the current state and the true type θ ⋆ j ∈ Θ( j ) . We use r : S × A → R to denote the reward function for the i -th player, which is assumed to be bounded, as formally defined below.

̸

Assumption 1. The reward r : S × A → R satisfies that | r ( s, a ) | ≤ r max for all s ∈ S , a ∈ A .

For any pair of states ( s, s ′ ) and action profile a ∈ A , we define p ( s ′ | s, a ) as a transition probability from s to s ′ given an action profile a . Finally, let γ ∈ (0 , 1) be a discount factor. We define the expected utility of the i -th player using a strategy π i as the expected discounted total payoff

where θ ⋆ -i := ( θ ⋆ j ∈ Θ( j ) : j = i ) , with respect to stochastic processes ( s t ∼ p ( · | s t -1 , a t -1 )) t> 0 and ( a t ( i ) ∼ π i ( s t )) t ≥ 0 , which generate the state and the action trajectories. The expectation in (8) is taken with respect to all randomness induced by the initial state distribution s 0 ∼ p 0 ∈ P S , the state transition kernel p , and strategy profile π (as well as opponents' types in Θ ).

̸

4.2 Opportunity, Risk, and Type Beliefs

Suppose the i -th player has beliefs of the other players' types, provided by a machine-learned forecaster, denoted by θ -i . For notational simplicity, we write π i , the strategy for the i -th player as π , and the strategies π -i for the opponents as σ (parameterized by θ -i ). Furthermore, we omit the subscripts and denote the type beliefs and true types as θ and θ ⋆ , and write the payoff in (8) as J ( π ) if there is no ambiguity. The i -th player uses a strategy π : S × Θ → P A ( i ) , which is a function of the type beliefs of the other players. Given a strategy π used by the agent and strategies by other players, denoted by σ , we define the following useful value functions of the game:

which satisfies the Bellman equation V π,σ ( s ) = E a ∼ π ( s ) ,a ′ ∼ σ ( s ) [( r + γ P V π,σ ) ( s, ( a, a ′ ))] , where we define an operator P V π ( s, ( a, a ′ )) := E s ′ ∼ p ( ·| s, ( a,a ′ )) [ V π,σ ( s ′ )] .

The following worst-case payoff gap is defined similarly as the one in (2) for normal-form games, which in our context can be considered as the dynamic regret for a strategy π i against an optimal strategy. Note that an optimal strategy is also a best response strategy π ⋆ knowing the true types of all players in hindsight maximizing (9). The optimal value function given σ and π ⋆ is denoted by V ⋆,σ ( s ) , whose Bellman optimality equation is V ⋆,σ ( s ) = max a ∈A ( r + γ P σ V ⋆,σ ) ( s, a ) .

Definition 2 (SBG PAYOFF GAP) . Given a stochastic Bayesian game M = ⟨S , A , Θ , σ, p, r, γ ⟩ , for a fixed strategy π , the payoff gap is defined as

where s 0 denotes an initial state and d ( θ, θ ⋆ ) := max s ∈S ∥ σ ( s ; θ ) -σ ( s ; θ ⋆ ) ∥ 1 ≤ ε .

Similar to normal-form Bayesian games, we define the (missed) opportunity and risk below.

Definition 3. Given a payoff gap ∆ SBG ( ε ; π ) in (10) , the (missed) opportunity of π is β ( π ) := ∆ SBG (0; π ) and the risk is δ ( π ) := max ε> 0 ∆ SBG ( ε ; π ) .

102

Upper bound, y= 0.0

Lower bound, y= 0.0

Upper bound, y= 0.3

Riskmax△(E,

E

Lower bound, y= 0.3

Upper bound,y=0.5

0<

101

Lower bound, y = 0.5

Upper bound,y=0.8

Lower bound, y= 0.8

100

10-2

10-1

100

101

102

Missed Opportunity △(0,π)

Our goal is to characterize a tradeoff between the opportunity and risk for stochastic Bayesian games. Finally, we define the value of the game.

Definition 4. Given a hypothesis set Θ and a stochastic Bayesian game M = ⟨S , A , Θ , σ, p, r, γ ⟩ , we denote the value of M by

4.3 Opportunity-Risk Tradeoff for Stochastic Bayesian Games

Based on the definitions above, our first result states an upper bound on the opportunity and risk for stochastic Bayesian games. Proving the bounds in Theorem 4.1 is nontrivial because the players are coupled by a shared state. Consequently, the analysis used in Section 3 for a simple convex combination of the best response to type beliefs θ and a safe strategy in normal-form games cannot be directly applied to the stochastic setting.

Theorem 4.1 (SBG EXISTENCE) . Consider a stochastic Bayesian game M = ⟨S , A , Θ , σ, p, r, γ ⟩ with ν Θ ( M ) ≥ ν . For any 0 ≤ λ ≤ 1 , there exists a strategy π whose (missed) opportunity and risk satisfy

where r max is defined in Assumption 1.

The proof of Theorem 4.1 is given in Appendix D, which constructs the following strategy.

Given type beliefs θ , we denote a parameterized strategy ˜ σ := σ ( θ ) , and let σ = σ ( θ ) be the safe strategies of the opponents, with θ being an optimal solution of the minimax optimization in (11). Denote ˜ V := V ⋆, ˜ σ and V := V ⋆,σ the optimal value functions with the opponents' strategies being ˜ σ and σ respectively. Unlike the convex combination used to show Theorem 3.1 for normal-form Bayesian games, the strategy constructed for proving Theorem 4.1 is a value-based strategy below

̸

where λ is a parameter that indicates the level of trusts on type beliefs, and for a fixed state s ∈ S and a given value function V : S → R , R V is an A ( i ) × ∏ j = i A ( j ) matrix defined as

As indicated by Theorem 4.1, this design ensures safe and exploitative play, with its efficacy validated through the case studies presented in Section 5. Finally, the following lower bounds can be derived, implying that the upper bounds on β ( π ) and δ ( π ) are tight and the value-based strategy formed in (12) are Pareto optimal up to multiplicative constants. Together, the bounds are illustrated in Figure 3.

4

2x2 game

入= 0

入 = 0.25

T1

2.5 2.5 2.5 2.5 2.5 2.5

T1

2.6 2.42.52.42.5 2.5

Action 1

(3,1)

(1,4)

Realized Type

2.5

52.5 2.4 2.5 2.5 2.5

Realized Type

2.1

2.92.5

52.82.5 2.5

2.5

2.5 2.5 2.5 2.5 2.5

T3-

2.5 2.5 2.5 2.5 2.5 2.5

2.5

2.52.5

2.5 2.5 2.5

T4

2.4

2.72.5

2.62.5 2.5

Action 2

(2,2)

(4,3)

T5

2.5

2.52.5

2.52.5

2.5

2.3

2.62.5

2.62.5

2.5

T6

2.5

2.52.5 2.5 2.52

2.5

T6

2.3

2.62.5

2.7

2.5

2.5

Action 1

Action 2

T1

T2

T3

T4

T5

T6

T1

T2

T3

T4

T5

T6

Predicted Type

Predicted Type

入 = 0.5

入 = 0.75

入 = 1

T1

2.82.2

2.5

2.2

2.52.5

T1

2.92.12.5

2.1

2.52.5

T1

3.0

2.0

2.5

2.0

2.52.5

Realized Type

T2

1.8

3.2

2.6

3.3

2.52.5

Realized Type

1.3

3.7

2.6

3.6

2.4 2.5

Realized Type

12

1.0

4.0

2.5

4.0

2.5

2.5

2.5

2.5

2.52.5

2.52.5

T3

2.5

2.62.5

2.5

2.5

2.5

T3

2.5

2.5

2.5

2.5

2.5

2.5

2.2

2.7

2.52.7

2.42.5

2.1

2.9

2.5

2.8

2.52.5

T4 -

2.0

3.0

2.5

3.0

2.5

2.4

T5

1.9

2.5 2.5 2.6 2.5 2.5

T5

1.4

2.52.5

2.52.5

2.5

T5

1.0

2.52.5

2.5

2.5

2.5

T6

2.1

2.8

2.5

2.82.6 2.4

T6

2.03.0

2.5

2.9

2.52.6

T6

1.7

3.0

2.5

3.0

2.5

2.5

T1

T2T3

T4

T5

T6

T1

T2

T3T4

T5

T6

T1

T2

2T3

T4

T5

T6

Predicted Type

Predicted Type

Predicted Type

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Average Utility

Theorem 4.2 (SBG IMPOSSIBILITY) . There is a stochastic Bayesian game M = ⟨S , A , Θ , σ, p, r, γ ⟩ whose value satisfies ν Θ ( M ) ≥ ν such that for any 0 ≤ λ ≤ 1 , if any strategy π misses at most 1 1 -λγ ( r max -ν ) (1 -λ ) opportunity, then it incurs at least 1 1 -λγ ( r max -ν ) (1 + λ ) risk.

In Theorem 4.2, the bounds on (missed) opportunity and risk differ from those in Theorem 4.1 by multiplicative constants. Theorem 4.2 can be derived by applying Theorem 3.2 for normal form games to a stateless MDP. We relegate the proof of Theorem 4.2 to Appendix E.

5 Case Studies

2 × 2 Game. We provide a set of numerical evaluations to showcase the opportunity-risk tradeoff across a range of game theoretic scenarios. We begin with a focus on 2 × 2 games and to that end we sample games spanning the topology of 2 × 2 matrix games as shown in [49]. This topology includes the comprehensive set of 78 strictly ordinal 2 × 2 games introduced by [50]. We formulate our evaluation as a single state Stochastic Bayesian Game with the state being a specific 2 × 2 game. This evaluation is helpful in providing a concrete illustration of the tradeoff dynamics on a wide range of games within a particular topology given the comprehensive bench-marking done on the set of 2 × 2 games.

We make use of 3 broad type classes (Markovian, Leader-Follower-Trigger-Agents, and Co-evoloved Neural Networks) inspired from work by [51]. It is from these classes that we construct our types which are then used in simulations with a player leveraging a strategy that is a convex combination of a safe strategy and the best response give type beliefs or predictions. Figs 4 and 5 shows the tradeoff in one particular game as an agent moves from being fully robust as indicated by λ = 0 to fully trusting of the advice when λ = 1 . In Appendix F, we provide further information on the particular types as well as illustrations on a couple of other games sampled from the topology of 2 × 2 games to showcase the range of tradeoffs which exist in this landscape.

Security Game. Extending our empirical study to real world settings, we also provide an evaluation of the opportunity-risk tradeoff using data from wildlife tracking studies. Security games are a clear domain wherein the tradeoff between opportunity and risk is very critical to the application. Building on a large body of work that has sought to understand the implications of game theoretic analysis in the environmental conservation domain, we formulate and explore the opportunity-risk tradeoff using data from real world wildlife movements. In particular, we formulate a green security game motivated by works such as [52] wherein we simulate a defender who is trying to protect an elephant population from attackers who are illegal poachers. The defender and attacker engage a multi-state Stochastic Bayesian Game where the states are constructed from historic elephant movement data sourced from [53]. Our evaluation shows the practicality and wide ranging impact of this investigation. Figure 6

3.0

=1

6BehavioralTypes

2.8

2.6

Always play action 0

2.4

Row Player

Player1

Type Belief

Column Player

Player2

Always play action 1

Minimax strategy

Risk

2.2

78

Random strategy

2.0

→2×2Games

Leader-Follower-Trigger Agents

1.8

Co-evolved Neural Networks

1.6

=0

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Loss of Opportunity

Example state: Mean elephant location data (Apr-Sept 2013)

入= 0

入 = 0.25

T1

6.66.5 6.7 6.76.56.7

T1

6.5

2.7

6.66.5 6.76.6

0

0

0

Realized Type

-2.4-2.4 -2.4 -2.5-2.4 -2.3

Realized Type

T2

-2.4-1.1-2.4 -2.3 -2.4 -2.4

T3

0.3-0.3-0.1-0.1-0.1-0.1

T3

-0.2-0.3-0.2-0.1-0.1 0.0

0

1

2

T4

0.3

'000Z'00

0.4

0.10.20.30.40.2

T5 --0.6 -0.5 -0.6 -0.5 -0.6 -0.6

T5 -

-0.60.8-0.6-0.7-0.5-0.6

0

2

8

T6

0.30.20.4 0.30.3 0.3

T6 -

0.4

0.20.50.3 0.40.4

T1

T2

T3

T4

T5

T6

T1

T2

T3

T4

T5

T6

Predicted Type

Predicted Type

入 = 0.5

入 = 0.75

入 = 1

T1

6.6

-0.1

6.76.56.66.6

T1

6.8

-3.2

6.76.46.65.7

T1

6.6

-6.4

6.66.36.75.9

Realized Type

T2

-2.40.0-2.3-2.3-2.3-2.4

Realized Type

T2

-2.31.1

-2.3-2.4 -2.4-2.1

Realized Type

T2

-2.42.4-2.4-2.4-2.4-2.1

T3

-0.4 -0.3 -0.2 -0.2 -0.3 -0.1

13

0.1-0.9-0.1-0.2-0.3-0.2

T3

-0.1 -1.0 0.2 -0.0 0.0 -0.1

T4

0.2 -0.2 0.3 0.2 0.2 0.4

T4

0.4-0.60.2 0.2 0.2 0.2

0.2 -0.7 0.4 0.0 0.0 0.1

T5

-0.5-0.1-0.6-0.3-0.6-0.5

T5

-0.5

-3.5

-0.4-0.2-0.6-0.0

T5

-0.5

6.6

-0.5-0.4-0.6-0.0

T6

0.4-0.2 0.30.40.30.3

T6

0.4-0.50.3 0.2 0.3 0.5

F91

0.3

-0.80.50.40.40.4

T1

T2

T3

T4

T5

T6

T1

T2

T3

T4

T5

T6

T1

T2

T3

T4

T5

T6

Predicted Type

Predicted Type

Predicted Type

2

0

2

Average Utility

shows the tradeoff for this setting evaluated for each particular type. We include implementation details on this game in Appendix F.

6 Concluding Remarks

In this study, we explored the fundamental limits of safe and exploitative strategies within both normal-form and stochastic Bayesian games, where pre-established type beliefs about opponents are considered. Given that these type beliefs may be inaccurate, relying on them to exploit opponents can result in high-risk strategies. Conversely, not leveraging these beliefs yields overly cautious play, leading to missed opportunities. We have quantified these dynamics by providing upper and lower bounds on the payoff gaps corresponding to different type belief inaccuracies, thereby characterizing the tradeoffs between opportunity and risk. These bounds are consistent up to multiplicative constants.

Limitations and Future Directions. In our current problem setting, the dynamics of a stochastic Bayesian game is assumed to be stationary, aligning with the canonical models in the related literature (for example [51]). To address this limitation, we plan to extend our framework to include time-varying type beliefs, addressing the absence of analysis for MDPs with dynamic transition probabilities and reward structures. Additionally, refining the opportunity and risk bounds to make them tighter would provide more precise strategic insights.

References

Xue Yan, Jiaxian Guo, Xingzhou Lou, Jun Wang, Haifeng Zhang, and Yali Du. An efficient endto-end training approach for zero-shot human-ai coordination. Advances in Neural Information Processing Systems , 36, 2024.

Shiyong Wang, Jiafu Wan, Daqiang Zhang, Di Li, and Chunhua Zhang. Towards smart factory for industry 4.0: a self-organized multi-agent system with big data based feedback and coordination. Computer networks , 101:158-168, 2016.

Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature , 575(7782):350-354, 2019.

Noam Brown and Tuomas Sandholm. Superhuman ai for multiplayer poker. Science , 365(6456):885-890, 2019.

Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems , 30, 2017.

Stefano V Albrecht and Peter Stone. Autonomous agents modelling other agents: A comprehensive survey and open problems. Artificial Intelligence , 258:66-95, 2018.

Tabish Rashid, Mikayel Samvelyan, Christian Schroeder De Witt, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research , 21(178):1-51, 2020.

Ioannis Anagnostides, Ioannis Panageas, Gabriele Farina, and Tuomas Sandholm. On the convergence of no-regret learning dynamics in time-varying games. Advances in Neural Information Processing Systems , 36, 2024.

Simin Li, Jun Guo, Jingqiao Xiu, Ruixiao Xu, Xin Yu, Jiakai Wang, Aishan Liu, Yaodong Yang, and Xianglong Liu. Byzantine robust cooperative multi-agent reinforcement learning as a bayesian game. In The Twelfth International Conference on Learning Representations , 2023.

John C Harsanyi. Games with incomplete information played by 'bayesian' players, i-iii part i. the basic model. Management science , 14(3):159-182, 1967.

John C Harsanyi. Games with incomplete information played by 'bayesian' players part ii. bayesian equilibrium points. Management science , 14(5):320-334, 1968.

Lawrence Friedman. Optimal bluffing strategies in poker. Management Science , 17(12):B-764, 1971.

Norman Zadeh. Computation of optimal poker strategies. Operations Research , 25(4):541-562, 1977.

David Milec, Ondˇ rej Kubíˇ cek, and Viliam Lis` y. Continual depth-limited responses for computing counter-strategies in sequential games. arXiv preprint arXiv:2112.12594 , 2021.

Stefano V Albrecht, Jacob W Crandall, and Subramanian Ramamoorthy. Belief and truth in hypothesised behaviours. Artificial Intelligence , 235:63-94, 2016.

James S Jordan. Bayesian learning in normal form games. Games and Economic Behavior , 3(1):60-81, 1991.

Martin Bichler, Max Fichtl, and Matthias Oberlechner. Computing bayes-nash equilibrium strategies in auction games via simultaneous online dual averaging. Operations Research , 2023.

Martin Bichler, Maximilian Fichtl, Stefan Heidekrüger, Nils Kohring, and Paul Sutterer. Learning equilibria in symmetric auction games using artificial neural networks. Nature machine intelligence , 3(8):687-695, 2021.

Paul Milgrom and John Roberts. Adaptive and sophisticated learning in normal form games. Games and economic Behavior , 3(1):82-100, 1991.

Ehud Kalai and Ehud Lehrer. Rational learning leads to nash equilibrium. Econometrica: Journal of the Econometric Society , pages 1019-1045, 1993.

John H Nachbar. Prediction, optimization, and learning in repeated games. Econometrica: Journal of the Econometric Society , pages 275-309, 1997.

John H Nachbar. Beliefs in repeated games. Econometrica , 73(2):459-480, 2005.

Dean P Foster and H Peyton Young. On the impossibility of predicting the behavior of rational agents. Proceedings of the National Academy of Sciences , 98(22):12848-12853, 2001.

Eddie Dekel, Drew Fudenberg, and David K Levine. Learning to play bayesian games. Games and economic behavior , 46(2):282-303, 2004.

Finnegan Southey, Michael P Bowling, Bryce Larson, Carmelo Piccione, Neil Burch, Darse Billings, and Chris Rayner. Bayes' bluff: Opponent modelling in poker. arXiv preprint arXiv:1207.1411 , 2012.

Mohammad Mahdian, Hamid Nazerzadeh, and Amin Saberi. Online optimization with uncertain information. ACM Transactions on Algorithms (TALG) , 8(1):1-29, 2012.

Manish Purohit, Zoya Svitkina, and Ravi Kumar. Improving online algorithms via ml predictions. In Advances in Neural Information Processing Systems , pages 9661-9670, 2018.

Dhruv Rohatgi. Near-optimal bounds for online caching with machine learned advice. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms , pages 1834-1845. SIAM, 2020.

Thodoris Lykouris and Sergei Vassilvitskii. Competitive caching with machine learned advice. Journal of the ACM (JACM) , 68(4):1-25, 2021.

Sungjin Im, Ravi Kumar, Aditya Petety, and Manish Purohit. Parsimonious learning-augmented caching. In International Conference on Machine Learning , pages 9588-9601. PMLR, 2022.

Antonios Antoniadis, Christian Coester, Marek Elias, Adam Polak, and Bertrand Simon. Online metric algorithms with untrusted predictions. In International Conference on Machine Learning , pages 345-355. PMLR, 2020.

Nicolas Christianson, Tinashe Handina, and Adam Wierman. Chasing convex bodies and functions with black-box advice. In Conference on Learning Theory , pages 867-908. PMLR, 2022.

Pengfei Li, Jianyi Yang, Adam Wierman, and Shaolei Ren. Robust learning for smoothed online convex optimization with feedback delay. Advances in Neural Information Processing Systems , 36, 2024.

Tongxin Li, Yue Chen, Bo Sun, Adam Wierman, and Steven Low. Information aggregation for constrained online control. ACM SIGMETRICS Performance Evaluation Review , 49(1):7-8, 2021.

Tongxin Li, Ruixiao Yang, Guannan Qu, Guanya Shi, Chenkai Yu, Adam Wierman, and Steven Low. Robustness and consistency in linear quadratic control with untrusted predictions. ACM SIGMETRICS Performance Evaluation Review , 50(1):107-108, 2022.

Yiheng Lin, Yang Hu, Guannan Qu, Tongxin Li, and Adam Wierman. Bounded-regret mpc via perturbation analysis: Prediction error, constraints, and nonlinearity. Advances in Neural Information Processing Systems , 35:36174-36187, 2022.

Tongxin Li, Ruixiao Yang, Guannan Qu, Yiheng Lin, Adam Wierman, and Steven H Low. Certifying black-box policies with stability for nonlinear control. IEEE Open Journal of Control Systems , 2:49-62, 2023.

Noah Golowich and Ankur Moitra. Can q-learning be improved with advice? In Conference on Learning Theory , pages 4548-4619. PMLR, 2022.

Tongxin Li, Yiheng Lin, Shaolei Ren, and Adam Wierman. Beyond black-box advice: Learningaugmented algorithms for mdps with q-value predictions. Advances in Neural Information Processing Systems , 36, 2024.

Jianyi Yang, Pengfei Li, Tongxin Li, Adam Wierman, and Shaolei Ren. Anytime-competitive reinforcement learning with policy prior. Advances in Neural Information Processing Systems , 36, 2024.

Tongxin Li, Bo Sun, Yue Chen, Zixin Ye, Steven H Low, and Adam Wierman. Learningbased predictive control via real-time aggregate flexibility. IEEE Transactions on Smart Grid , 12(6):4897-4913, 2021.

Nicolas Christianson, Christopher Yeh, Tongxin Li, Mahdi Torabi Rad, Azarang Golmohammadi, and Adam Wierman. Robustifying machine-learned algorithms for efficient grid operation. In NeurIPS 2022 Workshop on Tackling Climate Change with Machine Learning , 2022.

Tongxin Li. Learning-Augmented Control and Decision-Making: Theory and Applications in Smart Grids . PhD thesis, California Institute of Technology, 2023.

Tongxin Li and Chenxi Sun. Out-of-distribution-aware electric vehicle charging. IEEE Transactions on Transportation Electrification , 2024.

Tiancheng Jin, Longbo Huang, and Haipeng Luo. The best of both worlds: stochastic and adversarial episodic mdps with unknown transition. Advances in Neural Information Processing Systems , 34:20491-20502, 2021.

Idan Amir, Guy Azov, Tomer Koren, and Roi Livni. Better best of both worlds bounds for bandits with switching costs. Advances in neural information processing systems , 35:15800-15810, 2022.

Stefano V Albrecht and Subramanian Ramamoorthy. A game-theoretic model and bestresponse learning method for ad hoc coordination in multiagent systems. arXiv preprint arXiv:1506.01170 , 2015.

Lloyd S Shapley. Stochastic games. Proceedings of the national academy of sciences , 39(10):1095-1100, 1953.

Bryan Bruns. Names for games: Locating 2 × 2 games. Games , 6(4):495-520, October 2015.

Anatol Rapoport and Melvin Guyer. A taxonomy of 2 × 2 games. General Systems: Yearbook of the Society for General Systems Research , 11:203-214, 1966.

Stefano V. Albrecht, Jacob W. Crandall, and Subramanian Ramamoorthyc. Belief and truth in hypothesised behaviours. arXiv preprint arXiv:1507.07688 , 2015.

Fei Fang, Peter Stone, and Milind Tambe. When security games go green: Designing defender strategies to prevent poaching and illegal fishing. IJCAI , 2015.

Chamaille-Jammes. African elephant (migration) chamaillé-jammes hwange np, 2009-2017.

Satinder P Singh and Richard C Yee. An upper bound on the loss from approximate optimalvalue functions. Machine Learning , 16:227-233, 1994.

Jacob W. Crandall. Towards minimizing disappointment in repeated games. Journal of Artificial Intelligence Research (JAIR) , 49:111-142, 2014.

John R. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection . The MIT Press, 1992.

Broader Impacts. The implications of our research extend beyond theoretical interests and have practical significance in fields such as economics, cybersecurity, and strategic planning, where decision-making under uncertainty is crucial. By improving the understanding of how predictive information can be used safely and effectively in competitive environments, our work supports the development of more robust strategies in these areas. This can lead to better risk management practices and enhance the ability of systems to make informed decisions even when faced with unreliable or incomplete information. However, there are potential negative impacts, such as the risk of misuse of predictive models that may lead to biased or unfair decisions if the underlying data or assumptions are flawed. Future research could focus on reducing these risks, ensuring AI systems remain reliable and safe.

A Proof of Theorem 3.1

We first consider bounding the opportunity, focusing on the case when the predicted belief contains no error, i.e., y ⋆ = E ρ [ y ] . By the definition of π ( ρ ) , for any payoff matrix A with µ Θ ( A ) ≤ µ and ν Θ ( A ) ≥ ν , we get

where we have maximized the two terms max x ∈ P a x ⊤ Ay ⋆ and x ⊤ Ay ⋆ separately over y ⋆ ∈ Θ to obtain (13). Therefore, noting the definition of the safe strategy x in (7),

Now, we consider bounding the risk. By the definition of the following mixed strategy π ( ρ ) used by Player 1, for any payoff matrix A with µ Θ ( A ) ≤ µ , ν Θ ( A ) ≥ ν , and ∥ A ∥ max ≤ α ,

In (15), we replace max x ∈ P a x ⊤ Ay ⋆ by max x ∈ P a x ⊤ A E ρ [ y ] since y ⋆ ∈ Θ and there exists a feasible ρ with E ρ [ y ] = y ⋆ that always satisfies the constraint d ( ρ, y ⋆ ) ≤ ε .

Since ˜ x is a best response strategy given ρ , it holds that max x ∈ P a x ⊤ A E ρ [ y ] = ˜ x ⊤ A E ρ [ y ] . Decomposing max x ∈ P a x ⊤ Ay ⋆ = (1 -λ ) max x ∈ P a x ⊤ Ay ⋆ + λ max x ∈ P a x ⊤ Ay ⋆ and maximizing the two terms (1 -λ ) max x ∈ P a x ⊤ Ay ⋆ and λ max x ∈ P a x ⊤ Ay ⋆ over d ( ρ, y ⋆ ) ≤ ε respectively, ∆( ε ; α, π ) can be further bounded from above by

Maximizing over ε ≤ η (Θ) for (17), we conclude the theorem.

B Proof of Theorem 3.2

We first suppose a mixed strategy π : P Θ → P a for Player 1 misses at most (1 -λ ) ( µ -ν ) opportunity, given a belief of types ρ ∈ P Θ .

Let y ′ and y ′′ be two mixed strategies in Θ that achieve κ (Θ) ≥ 0 (select arbitrary strategies if there are multiple to break the tie), i.e.,

Let I + := { i ∈ [ b ] : y ′ i > y ′′ i } and I -:= { i ∈ [ b ] : y ′ i ≤ y ′′ i } be two indices of actions with non-negative and positive coordinates of z where [ b ] := { 1 , . . . , b } . Consider a payoff matrix A µ with the following form. Suppose without loss of generality that a is even; otherwise, we append an all-zero row to A µ . Let β := µ -ν . For the i -th column of A µ , we set it as ( β, -β, . . . , -β ) ⊤ if i ∈ I + and ( -β, β, . . . , β ) ⊤ if i ∈ I -. Formally, we define:

where ( A ν ) ij = ν for all i ∈ [ a ] and j ∈ [ b ] . Clearly, with this A , µ Θ ( A ) ≤ µ and ν Θ ( A ) ≥ ν . Then, by definition, setting ε = 0 with E ρ [ y ] = y ⋆ (the type belief contains no error), the payoff gap satisfies

Now, plugging in y ′ into (18) above. This implies

For notational convenience, denote 𭟋 (Θ) := ∑ i : y ′ i >y ′′ i y ′ i -∑ i : y ′ i ≤ y ′′ i y ′ i . Combining (18) and (19), π ( ρ ) ⊤ Ay ′ = π ( ρ ) ⊤ ( A µ + A ν ) y ′ ≥ (1 -λ ) ν + λµ. Thus, π ( ρ ) ⊤ A µ y ′ ≥ λ ( µ -ν ) . Equivalently, 𭟋 (Θ)( µ -ν ) π ( ρ ) ⊤ f ≥ λ ( µ -ν ) , where f := (1 , -1 , 1 , . . . , 1 , -1) ⊤ . By the construction of A , we get

Therefore, since 0 < 𭟋 (Θ) ≤ 1 ,

Hence, we get

C

Proof of Corollary 3.1

If Θ = P b , then κ (Θ) = 1 . Furthermore, by definition, we get

where the equality on the RHS holds since the game is fair. Applying Theorem 3.1 and 3.2, we obtain the corollary.

D Proof of Theorem 4.1

Given type beliefs, we denote strategies ˜ σ := σ ( θ ) , σ := σ ( θ ⋆ ) , and let σ = σ ( θ ) be the safe strategies of the opponents, with θ being an optimal solution of the minimax optimization in (11). Denoting ˜ V := V ⋆, ˜ σ , V := V ⋆,σ , V ⋆ := V ⋆,σ the value functions with the opponents' strategies being ˜ σ , σ , and σ , respectively. Given the strategy π defined in (12), we first state two useful bounds.

Lemma 1. Given the strategy defined in (12) , the following bounds on the reward function hold:

where η := max s ∈S ∣ ∣ ∣ V ⋆ ( s ) -˜ V ( s ) ∣ ∣ ∣ , and

Proof of Lemma 1. First, we prove BOUND 1 in (20).

Based on the definition of the policy π in (12), it follows that for any fixed state s ∈ S ,

Furthermore, considering the difference between the expected rewards corresponding to implementing π ⋆ and the strategy π defined in (12), since by Assumption 1, | r | ≤ r max for all t ∈ [ T ] and any strategy π ∈ Π ,

Therefore, using (23),

Combining (22) and (24), we obtain

By definition,

and similarly, ∣ ∣ ∣ E p,π ⋆ , ˜ σ [ V ⋆ ( s ) ] -E p,π ⋆ , ˜ σ [ ˜ V ( s ) ] ∣ ∣ ∣ ≤ η , thus, (25) yields

Next, we show BOUND 2 in (21).

By the definition of the policy π in (12), it follows that for any fixed state s ∈ S , similarly we obtain

Rearranging the terms in (26),

Using Lemma 1, the following lemma provides a bound on ∆ SBG ( ε ; π ) , from which bounds on both risk and (missed) opportunity can be directly derived.

Lemma 2. The payoff gap for the policy π defined in (12) satisfies

Proof. By definition of the payoff gap (see Definition 2) and the value function in (9), we get

Our first goal is to derive an upper bound on ∆ SBG ( ε ; π ) , aligning with the classic loss bound of approximate value functions [54]. However, the policy considered in our context is not directly maximizing the approximate value function, but a mixed strategy defined by an optimization as in (12). For notational convenience, we write for s ∈ S ,

For notational simplicity, we denote V π ( s ) := V π,σ ( s ) . For any state s ∈ S and strategies ( σ, ˜ σ ) satisfying d ( θ, θ ⋆ ) = max s ∈S ∥ σ ( s ; θ ) -σ ( s ; θ ⋆ ) ∥ 1 ≤ ε , applying BOUND 1 in Lemma 1, the Bellman equations corresponding to π ⋆ and π imply

Applying the Bellman equations, for any s ∈ S ,

Similarly, it also holds that

Plugging (28), and (29) into (27),

Let V π ( s ) := V π,σ ( s ) for any s ∈ S . For the term in (30), we always have V π ( s ) ≥ V π ( s ) (c.f. [48]), thus,

Furthermore, the Bellman equation implies

Applying BOUND 2 in Lemma 1,

Since above holds for any state s ∈ S , let s ′ be a state such that the value gap V ⋆ ( s ) -V π ( s ) is maximized. Therefore, (34) implies

Then, for the term in (31), by definition V ⋆ ( s ) ≤ r max / (1 -γ ) , and

Combing the bounds in (35) and (36) with (32), we conclude that

Now, noting that by definition we have

and similarly,

Therefore, rearranging the terms and simplifying above, (37) becomes

Since above holds for any state s ∈ S , let s ∗ be a state such that the value gap V ⋆ ( s ) -V π ( s ) is maximized. Thus,

Continuing from (38),

Therefore, rearranging the terms we obtain

Now, the terms in (30) can be bounded alternatively as

Using this and following the same steps, we obtain

Part I: Proof of Risk Bound

Since d ( θ, θ ⋆ ) := max s ∈S ∥ σ ( s ; θ ) -σ ( s ; θ ⋆ ) ∥ 1 ≤ 2 , setting the worst ε = 2 , and noticing that η ≤ 2 r max 1 -γ ,

Rearranging above leads to the result.

Part II: Proof of (Missed) Opportunity Bound

Finally, setting ε = η = 0 , we obtain

E Proof of Theorem 4.2

We prove the theorem by considering a stateless MDP, which reduces a stochastic Bayesian game M = ⟨S , A , Θ , σ, p, r, γ ⟩ to a normal-form game in Theorem 3.2.

̸

Let L := { 1 , . . . , ∏ j = i |A ( j ) } . We fix a strategy kernel σ such that

̸

Therefore, the construction of the payoff matrix A ∈ R |A ( i ) |× ∏ j = i |A ( j ) | in the proof of Theorem 3.2 with µ Θ ( A ) ≤ r max 1 -γ and ν Θ ( A ) ≥ ν yields that for any 0 ≤ λ ≤ 1 , if any mixed strategy π misses 1 1 -λγ ( r max -ν ) (1 -λ ) opportunity, then since

the strategy π has at least ( r max 1 -γ -ν ) (1 + λ ) risk. Furthermore, since

π has at least 1 1 -λγ ( r max -ν ) (1 + λ ) risk.

F Details on Experiments

F.1 Details on 2 × 2 Game simulations

F.1.1 Game definition

We define a Stochastic Bayesian game with one state: a particular 2 × 2 game sampled from the topology of 2 × 2 games.

The payoffs for each player are stipulated by the 2 × 2 game. We set the time horizon for the game be 1,000 giving us an empirical evaluation of the expected payoff for each strategy

F.1.2 Type definitions

As mentioned we have 3 classes of types and below we provide a description of the general classes as well as the specific types we used in our evaluation.

Markovian Types: These are types whose strategy only depends on the current state. We made use of 4 types of Markovian strategies

Type 1: Always play action 0

Type 2: Always play action 1

Type 3: The minimax strategy for the player

Type 4: Returns a random strategy

Leader- Follower-Trigger Agents: Inspired by [55], these agents have a preferred sequence of play they seek to enforce. Importantly they have access to history of play and when the other player does not play according to their preferred sequence the alter their strategy by engaging in 'punishing' behavior such as playing the minimax strategy or by resetting to a previous action. In our case, we evaluated against simple versions of such agents, wherein our agent had a preferred mixed strategy and when the empirical observed strategy over a preset number of previous plays from the opponent did not match their preferred strategy, they 'punished' the opponent by playing a minimax strategy. In particular, we had an agent look at the previous 4 actions of the opponent and if they selected action 1 more than twice, they chose to play a minimax strategy for the next round.

Co-evolved Neural Networks: Inspired by work in [51] we use ideas of genetic programming [56], to generate agents from neural networks. We randomly initialized 10 neural networks with a single hidden layer for both the row and column player. All networks would take as input the previous 4 actions of both players and the corresponding state information. We sample randomly from populations of the row and column players and simulate the game. After simulation we calculate a fitness score based on average payoffs for each agent and a similarity score so as to ensure diversity in the models. We 'evolve' the populations by selecting random portions of both populations to mutate whilst also having cross-over between members of the populations selected by fitness (this is done for both populations hence 'co-evolve'). We then proceed to create new populations using the most fit agents from a previous generation (i.e., we take the top 50% of a population pre-evolution and 50% post evolution and then create a new population if the average fitness of this new constructed population is greater than the average fitness of the previous population.)

F.1.3 Experimental illustration of tightness of bounds

Wehave included additional adversarial examples to demonstrate the tightness of the bounds presented in Theorems 3.1 and 3.2. In particular, we consider the zero-sum Matching Pennies (MP) and an Adjusted Matching Pennies (AMP) as our two examples. We assume the hypothesis set Θ for Player 2 contains 6 behavioral types:

· Markovian Types :

-Type 1 : Always play action 0

-Type 2 : Always play action 1

-Type 3 : Minimax strategy

-Type 4 : Random strategy

Type 5 : Leader-Follower-Trigger Agents

Type 6 : Co-evolved Neural Networks

The payoff matrices for the MP and AMP, respectively are

It's worth mentioning that the AMP payoff matrix constructed above is an adversarial example that follows the same construction of the adversarial payoff matrix A used in the proof of Theorem 3.2. Given the considered Θ , we know that κ (Θ) = 1 and η (Θ) = 2 . The upper and lower bounds in Theorem 3.1 and 3.2 therefore read:

where for MP, µ = 1 , ν = 0 ; for AMP, µ = 1 . 2 , ν = 0 . 2 .

Besides the bounds above, we also plot simulated risk and opportunity values in the figures using an algorithm that has varying trust of type beliefs in 100 and 1,000 runs. The algorithm is presented with a type prediction and takes a convex combination of the best response to the type prediction and the minimax strategy with the trust parameter λ determining how much to weigh the best response. It is the same as π , the mixed strategy used to prove Theorem 3.1. Fully trusting ( λ = 1 ) and distrusting ( λ = 0 ) type beliefs yield a best response strategy and a minimax strategy correspondingly. The agent then samples from their resulting mixed strategy an action to play while the opponent also samples from whatever mixed strategy they are using an action to play.

We sample this interaction for the number of runs and gather empirical payoff information, which we use to plot the opportunity risk tradeoff. The variance of the risk and opportunity values decreases as the number of runs increases, as illustrated by the error bars in the attached figures. Moreover, we include as plots the bounds on the opportunity risk tradeoff and show the tightness or looseness of these bounds in two games we pick. Please note that some of the simulated risk and opportunity values fall outside the bounds. This is because the bounds are applicable only to expected payoff gaps, and individual simulations may deviate from these expected values.

In the Fig 7, the top two figures display the results from 100 runs, while the bottom two figures present the results from 1,000 runs. The left two figures correspond to the MP example, while the right two figures are for the AMP example, demonstrating the gap between the lower and upper bounds. In particular, in an AMP game, we see that the upper bound is loose, and the empirical opportunity-risk tradeoff matches the lower bound we derive. In future work, we will investigate if this gap can be closed. This 'adversarial' example illustrates the looseness in the upper bound we derive. In the canonical Matching Pennies game, we find that the lower and upper bounds are tight with the empirical tradeoff matching these theoretical bounds, coinciding with Corollary 3.1.

2.0

Upper bound

2.4

Upper bound

Lower bound

Lower bound

2.2 -

1.8

2.0 -

1.6

1.8

1.4

1.6

1.4

1.2

1.2

1.0

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

Opportunity

Opportunity

2.0

Upper bound

2.4

Upperbound

Lower bound

Lower bound

2.2

1.8

2.0

1.6

1.8

1.4

1.6

1.4

1.2

1.2

1.0

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

Opportunity

Opportunity

F.2 Details on Green Security Game simulations

F.2.1 Data description

The data is from a study tracking geo-location information of 32 African elephants in the Hwange National Park in Zimbabwe, Africa. The data was collected from 2009 to 2017 and includes time stamp information as well as longitudinal and latitudinal information of the elephants

F.2.2 Data processing

We divided each year into two based on the seasons in Zimbabwe which would affect the location of the elephants as they migrate given changes in the environment. A year was divided into a 'Rainy' season which runs from October to March and a 'Dry' season which runs from April to September.

We divided the area covered in the dataset into 9 locations and thus had a 3 × 3 grid which served as a surveillance area. For each elephant in the season, we calculated its mean location and so for each of the seasons we had a mean location for the elephants. It is important to note, given migratory nature of elephants across national borders, each season does not necessarily have all 32 elephants as they move as a result of a wide range of factors (e.g., availability of water). Some seasons, also notably, do not have any recorded elephant presence in the area under surveillance. We do not see this as a limitation as the dynamic environment presents changes in the game which the defender has to take into account.

F.2.3 Green Security Game

Each of the seasonal elephant information presents us with a state for our Stochastic Bayesian Game. We define the game in the following manner:

State: We have 16 states from the seasonal information we get from the data

Actions: Each player (attacker or defender) has 9 available actions each corresponding to a selection of an area in the 3 × 3 grid defined above.

Payoffs: Let n be the number of elephants recorded in a particular grid square:

-If the attacker and defender select the same grid square, the defender gets n while the attacker gets -2

-If the attacker and defender select different grid squares, let n att be the number of elephants in the grid square selected by the attacker. The defender gets payoff -n att while the attacker gets payoff n att

These payoffs were designed to model the asymmetric nature of the defending task. An offender often will get a fixed penalty as stipulated by law, whilst the defender always depends on the number of elephants the attacker has access to.

Transitions: To take into account the effect of changes in weather whilst also bringing in some stochasticity, we made it such that there is some transition probability between any 'Rainy' state to any 'Dry' state and vice-versa. We do however, slightly, increase the probability of transition between adjacent historical states, to reflect historical data. We have the probability of transitioning between any two 'Rainy' or 'Dry' states to be zero. Our transition probabilities do not depend on the actions taken by the agents.

F.2.4 Type definitions

We make use of the same types as in the 2 × 2 game simulations. We make adjustments to the Markovian types in that Types 1 and 2 now select the grid with the highest population of elephants and second highest population of elephants, respectively. The Leader-Follower-Trigger Agents now looks to see if the other player is selecting the grid with the highest number of elephants for more than 50% of the historic play in which case they turn to play their minimax strategy

F.3 Additional 2 × 2 game evaluations

We include as an illustration as well as for completeness a couple of other 2 × 2 games we also evaluated against. This is helpful as it shows the variety of tradeoffs that exist within the topology of 2 × 2 games. In particular, we see some games exhibiting gradation as the agent moves from fully robust to fully trusting whilst in others there does not exist such a tradeoff because of the existence of a dominant strategy for the row player regardless of the type of the column player (e.g., the last game we show in this section). We note that the games provided in this file do not exhaust the entire topology of 2 × 2 games. They are however added to show the range of tradeoffs that could exist in games.

2x2game

入 = 0

入 = 0.25

T1

2.00 2.00 2.002.00 2.00 2.00

T1

2.002.00 2.00 2.00 2.00 2.00

Action 1

(2,2)

(3,4)

Realized Type

OO000000000E

0000000000E

3.00

T3

2.00 2.00 2.002.00 2.00 2.00

2.00 2.00 2.00 2.00 2.00 2.00

T4

2.50 2.49 2.50 2.50 2.48 2.49

T4

2.51 2.51 2.50 2.51 2.51 2.49

Action 2

(1,1)

(4,3)

T5 -

OO0000000000E

OO000000000

T6-228 228 228 226 228 227

T6 -

2.282.26 2.27 2.29 2.28 2.27

Action 1

Action 2

T1

T2

T3T4

T5

T6

T1

T2T3T4T5

T6

Predicted Type

Predicted Type

入 = 0.5

入 = 0.75

入 = 1

T1

2.002.00

2.00 2.00 2.00 2.00

T1

2.00

1.24

2.00 2.00 2.00 2.00

T1

2.00

2.001.75

2.002.00

Realized Type

OOE0000000O0E

Realized Type

T2

3.00

3.75

OOEOOOOEOOE

Realized Type

3.00

4.00

3.00

3.25

3.003.00

2.00 2.00 2.00 2.00 2.00 2.00

T3

2.00

2.00 2.00 2.00 2.00

T3

2.00

2.00

1.76

2.002.00

T4 -2.51 2.49 2.50 2.50 2.50 2.48

T4

2.50 2.47 2.50 2.52 2.52 2.49

T5

O00000000000E

T5

3.00

ZET

T5

3.00

3.002.883.00

3.00

T6 -2.28 2.27 2.29 2.26 2.29 2.30

T6

2.27

1.96

2.27 2.27 2.29 2.26

T6

2.29

1.872.242.142.272.28

T1

T2T3

T4T5

T6

T1

T2

T3T4

T5

T6

T1

T2T3

T4T5

T6

Predicted Type

Predicted Type

Predicted Type

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Average Utility

2x2game

入= 0

入 = 0.25

T1

2.45 2.50 2.50 2.50 2.50 2.51

T1

2.61

2.10

2.45 2.41 2.48 2.48

Action 1

(3,2)

(2,4)

Realized Type

T2 -250 2.52 2.52 2.54 2.48 249

Realized Type

T2

2.38

2.88

2.532.492.46 2.48

T3 - 2.44

42.46 2.49 2.47 2.49 2.49

2.51

2.47

2.532.55

2.51

T4 -254 2.50 2.50 2.50 2.50 2.51

2.50 2.51 2.48

2.50

Action 2

(1,3)

(4,1)

T5

2.53

32.46 2.52 2.50 2.49 2.48

T5

2.422.64

2.432.55

2.47

2.49

T6

2.52 2.50 2.48 2.52 2.53 2.53

T6

2.562.302.48 2.44 2.50 2.51

Action 1

Action 2

T1

T2

T3T4

T5

T6

T1

T2

T3

T4T5

T6

Predicted Type

Predicted Type

入 = 0.5

入 = 0.75

入 = 1

T1

2.74

1.76

2.482.372.542.49

T1

2.87

37

2.452.302.46 2.51

T1

3.00

2.442.24

42.52 2.48

Realized Type

T2 -

2.27

3.24

2.55

2.692.50 2.53

Realized Type

T2

2.12

3.63

2.51

12.652.48 2.47

Realized Type

T2

2.00

2.53

2.73

2.51

2.45

T3

2.52 2.51 2.50

2.54

2.52

2.51

2.51

2.502.50

2.48

2.52

2.52

T3

2.492.51

2.49 2.45

2.50

2.52

2.52

2.50

2.54

2.52

2.50

2.49

T4 - 2.50

2.462.45

2.48

2.51

2.51

T4 -

2.48 2.48

2.462.51

2.46

2.49

T5

2.28

2.632.53

2.60 2.44 2.52

T5

2.13

2.51 2.47 2.60 2.42

2.51

T5

2.00

2.50 2.482.62

2.51

2.55

T6 -

2.62

2.12

2.47 2.46 2.42 2.53

T6

2.68

1.98

2.52 2.42 2.48 2.47

T6

2.74

1.80

2.512.37

2.502.52

T1

T2

T3

T4T5

T6

T1

1T2T3T4

T5

T6

T1

T2

T3

T4T5

T6

Predicted Type

Predicted Type

Predicted Type

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Average Utility

2x2game

入=0

入 = 0.25

T1

2.00 2.00 2.00 2.00 2.00 2.00

T1

2.00 2.00 2.00 2.00 2.002.00

Action 1

(1,4)

(4,2)

Realized Type

T2

O000000000

Realized Type

T2

00000000000

T3

2.00 2.00 2.00 2.00 2.00 2.00

T3 -

2.00 2.00 2.00 2.00 2.00

2.00

T4

2.50 2.53 2.48 2.48 2.49 2.53

T4 -

2.52 2.48 2.47 2.49 2.49 2.51

Action 2

(2,3)

(3.1)

T5

2.00 2.00 2.00 2.00 2.00 2.00

T5

2.00 2.00 2.00 2.00 2.002.00

T6 -2.50 2.50 2.51 2.51 2.50 2.50

- 9.1

2.52 2.48 2.50 2.48 2.51 2.52

Action 1

Action 2

T1

T2

T3T4

T5T6

T1

T2

T3

T4T5

T6

Predicted Type

Predicted Type

入 = 0.5

入 = 0.75

入 =1

T1

2.00 2.00 2.00 2.00 2.00 2.00

T1 - 2.00

2.002.00  2.002.00

T1 - 2.00

2.001.75

2.002.00

Realized Type

T2

000000000000

Realized Type

T2

3.00

3.76

OOEOOOOEOOE

Realized Type

T2 -

3.00

4.00

3.00

3.25

3.003.00

2.00  2.00 2.00 2.00 2.00 2.00

T3-

2.00

2.00 2.00 2.00 2.00

2.00

2.00

1.75

2.002.00

T4

2.51 2.52 2.50 2.49 2.51 2.53

T4 -2.48 2.52 2.50 2.48 2.52 2.47

T4

2.50 2.55 2.51 2.51 2.53 2.51

T5

2.00 2.00 2.00 2.00 2.00 2.00

T5 -

2.003.15

2.00 2.00 2.00 2.00

T5

2.00

2.001.83

2.002.00

T6

2.50 2.51 2.49 2.49 2.50 2.51

T6 -

2.51 2.54 2.51 2.50 2.52 2.51

T6

2.49 2.46 2.50 2.52 2.48 2.49

T1

T2T3T4T5

T6

T1

T2

T3T4

T5

T6

T1

T2T3

T4T5

T6

Predicted Type

Predicted Type

Predicted Type

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Average Utility

2x2game

入 = 0

入 = 0.25

T1

2.492.43 2.44 2.50 2.55 2.52

T1

2.62

2.05

2.502.41 2.46 2.51

Action 1

(1,4)

(4,2)

Realized Type

T2

2.53

2.49 2.51 2.48 2.50 2.51

2.36

2.87

2.52

2.60

2.52

2.50

2.50 2.47 2.46 2.49 2.49

2.48 2.52 2.49 2.53 2.49 2.49

T4-2.48 250 250 2.49 248 2.44

T4 -

2.49 2.48 2.48 2.49 2.46 2.52

Action 2 -

(3,3)

(2,1)

T5 -2.44 2.51 2.48 2.51 2.54 2.52

T5 -2.51 2.48 2.49 2.54 2.51 2.53

T6 -2.46

2.47 2.46 2.46 2.49 2.53

T6 -

2.53

2.322.492.46 2.55

2.51

Action 1

Action 2

T1

T2

T3T4

T5

T6

T1

T2T3T4T5

T6

Predicted Type

Predicted Type

入 = 0.5

入 = 0.75

入 = 1

T1

2.74

1.76

2.492.39

2.492.53

T1

2.84

1.41

2.492.362.53 2.49

T1

3.00

2.522.23

2.502.51

Realized Type

3.26

2.48

2.63

2.50

2.50

Realized Type

T2

2.13

3.62

2.51

2.712.47 2.48

2.00

2.51

2.71

2.48

2.47

T3

2.492.56

2.50

2.50 2.492.50

T3 2.50

2.52

2.51

2.532.46 2.49

T3

2.532.51

2.44

2.50

2.56

2.50

T4-2.52 251 2.49 2.53 2.50 2.51

T4 -248 2.45 2.54 2.51 2.48 2.51

T4 -2.48 2.41 2.48 2.44 2.50 251

T5 -

2.52

2.86

2.532.48

2.50 2.50

T5

2.52

3.53

2.482.51

2.52

2.53

T5

2.48

4.00

2.422.53 2.49

2.51

T6 -2.57

2.142.45 2.43 2.46 2.56

T6

2.67

1.99

2.532.422.48 2.49

T6

2.73

1.85

2.56 2.42 2.52 2.50

T1

T2T3

T4T5

T6

T1

T2

T3T4

T5

T6

T1

T2T3

T4T5

T6

Predicted Type

Predicted Type

Predicted Type

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Average Utility

2x2game

入= 0

入 = 0.25

T1

000000000

T1

0000000000

Action 1

(2,4)

(1,1)

Realized Type

T2

4.00 4.00 4.00 4.00 4.00 4.00

Realized Type

T2

4.00 4.00 4.00 4.00 4.00 4.00

T3

O0000000000

T3

000000000000

T4 -

3.50 3.51 3.50 3.50 3.51 3.51

T4

3.50 3.51 3.49 3.483.52 3.51

Action 2

(3,3)

(4,2)

T5 -

T5

000000000000

T6 -

3.313.33 3.31 3.29 3.31 3.32

T6

Action 1

Action 2

T1

1T2T3T4T5T6

T1T2T3T4T5

T6

Predicted Type

Predicted Type

入 = 0.5

入 = 0.75

入 = 1

T1

00000000000

T1

OO000000000

T1

00000000000

Realized Type

4.004.004.004.004.004.00

Relied Type

T2

4.004.004.004.004.004.00

Realied Type

T2

4.00 4.004.004.004.004.00

000000000000

T3

O0000000000

T3 -

0000000000

T4

3.493.50 3.47 3.51 3.483.53

T4

3.52 3.51 3.48 3.49 3.50 3.49

T4

3.473.49 3.48 3.50 3.49 3.50

T5 -

0000000000

T5 -

OO000000000

T5

000000000000

T6 -

3.313.313.303.313.283.32

T6 -

T6

3.313.323.32 3.323.31 3.28

T1

T2T3T4T5

T6

T1

T2T3T4T5

T6

T1

T2T3T4T5

T6

Predicted Type

Predicted Type

Predicted Type

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Average Utility

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The claims made in the abstract and introduction align with our theoretical and experimental results.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations of this work are discussed in our last concluding remark section.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We have clearly stated our model assumptions for the derived results to hold. Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have detailed the settings and parameters used in the experiments, and we will further release our code after the anonymous review stage.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: Our contributions focus on the theoretical side. The experimental results shown in the submitted manuscript do not depend on private datasets and can be reproduced using the provided settings and parameters.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have clearly stated our experimental settings and details to reproduce the results.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: When there is randomness in our experiments, we characterize the variability.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Our contributions focus on the theory side, and the experimental setup is sufficiently basic that it does not require intense computing resources such as GPUs. Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted conform with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We have discussed both positive and negative societal impacts in our concluding remark section.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper does not use existing assets.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
The paper considers a setting of stochastic bayesian games where the beliefs of the opponents’ strategies may not be fully trusted. The authors first analyze the problem in the stationary setting and then move to an MDP setting. The main outcome is that there is a trade-off between risk and opportunity depending on the trust in the belief.

STRENGTHS:
1. The paper is well-written and can be understood by a non-expert such as myself. 
1. There are few examples that explain well the trade-offs and the design choices 
1. The case studies underline the practical implications of this theoretical work.

WEAKNESSES:
While it is interesting to consider and make the decision based on the trust in the belief, I think a discussion on alternatives is needed. For example, many machine learning algorithms would provide uncertainty bounds or estimates. One can similarly derive strategies based on these bounds

QUESTIONS:
1. It’s hard to understand how tight the bounds and the estimates in the theorems 3.1 and 3.2 are. Some additional examples with explicit bound computations would be beneficial
1. Section 4.3 could be reworked to provide first the strategy so that Theorem 4.1 can be read in this context.
1. Line 173. Seems like a word is missing there
1. Line 196. It is not clear what requirement can be relaxed
1. Lines 593-594. A typo in reward assignment

LIMITATIONS:
limitations are discussed
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies a setting in which an agent is trying to exploit opponents in a game based on uncertain beliefs about the opponents' strategies. In this setting, exploiting opponents can be risky if the beliefs might be wrong. The paper studies the trade-off between exploiting as much as possible, while minimizing the risk in case the opponent beliefs are wrong. The paper explicitly characterizes this trade-off for normal form and stochastic games, and they provide strategies that achieve theoretically optimal trade-offs. Finally, the authors provide an experimental study in 2x2 games and in a security game.\n\nSTRENGTHS:\n- Characterizing the trade-off between exploitation of beliefs and being robust to wrong beliefs seems like an important and fundamental contribution to me. I am somewhat surprised that this apparently hasn't been done before.\n- The overall approach of the paper seems substantial and comprehensive to me. I like both the thorough theoretical analysis and empirical study.\n- Overall I found the paper clear and easy to follow.\n- I think it's great that the paper discusses matching pennies as a motivating example\n- Formal definitions are properly introduced and mathematical statements are rigorous\n\nWEAKNESSES:\n- I think the abstract and introduction could be written more clearly. E.g., lines 4-5 \"The idea is to plan an agent’s own actions with respect to those types which it believes are most likely to maximize the payoff\" this confuses me rather than explaining the core idea (which is best responding given the belief about opponent types?). Or lines 20-21 \"players may not be fully trustworthy and thus do not know the behaviors of the others\" this sentence doesn't really compute/make sense to me.\n- One could improve some of the explanations of technical content. E.g., in matching pennies, explicitly discuss the motivation in terms of the strategies in the game (rather than just computing the different quantities). In the different definitions and theorem statements, one could give a bit more intuition for the results (e.g., can one state Theorem 4.1 in terms of rough order of magnitudes/Big O notation, and give some intuitive explanation why it has to be that way?) and why definitions are chosen one way and not another way (e.g., why is $d(\\rho,y^*)$ defined the way it is).\n- Minor comment: the minimax strategy can be very bad for a player in general sum games where players can harm each other without benefit to themselves. In that case, being better than minimax might not be that useful (e.g., in a great power conflict, the minimax strategy corresponds to all out nuclear war). Hence, I think the results here make most sense for zero-sum games or other games where the minimax strategy is a reasonable strategy.\n\nQUESTIONS:\n- Are there more baselines one could compare the set of strategies in figures 5 and 6 to? I guess $\\lambda =1$ and $\\lambda = 0$ can be viewed as the two baselines?\n- Figure 1: I think the arrow in the diagram on the right goes in the wrong direction. It should be higher = less risk?\n- Maybe this is obvious but it might be helpful to define a fair game, especially since you include general-sum games as far as I can tell. It seems less obvious to me what a fair general-sum game is (I think you just define it as a game where the minimax values for both players are zero?).\n- Line 133: Can you give an intuition for why you use the distance of the expected strategy $E_\\rho[y]$ to the true strategy here? Is this because the best response to belief $\\rho$ depends on this expectation?\n- It would be helpful to say in words what is going on with the strategies in Matching Pennies in Section 3.2. As far as I understand, the idea is that when choosing $\\lambda=1$, in the worst case, you get -1 payoff but could have achieved payoff 1 (if your opponent is deterministic and you have exactly the opposite belief), leading to 2 risk in this case. In the case $\\lambda =0$, one gets the equilibrium payoff 0, but maximal exploitation could get a payoff of 1, so in this case the opportunity would be 1.\n- Can you provide more intuition for Definition 1? Why is $\\kappa$ chosen the way it is?\n\n- In lines 70-71: repetition \"in normal-form Bayesian games\", we characterize ... \"for the normal-form game\".\n\nLIMITATIONS:\nThe authors adequately address limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper explores how players in Bayesian games can manage the inherent uncertainty about other players in a strategic way, aiming to maximize their outcomes (exploitative) while minimizing potential risks (safe).\n\nThe main theme of the paper is the tradeoff between the opportunity and risk of exploiting unreliable beliefs. The paper discusses the balance between the safe yet conservative strategy and the risky yet exploitative strategy, with parameters in the mixed strategies indicating the level of trust on type belief.\n\nFor both normal-form games (simpler, static games) and stochastic Bayesian games (more complex, dynamic games), the paper provides upper and lower bounds on the payoff gaps between the optimal possible strategy and the actual strategy of trusting/distrusting the beliefs.\n\nFor normal-form games, in Theorem 3.1, the authors establish upper bounds on the payoff gap, indicating limits on potential gains or losses based on the accuracy of type beliefs. In Theorem 3.2, the authors provide lower bounds, showing fundamental risk levels that cannot be mitigated through strategic adjustments alone.\n\nFor stochastic Bayesian games, in Theorem 4.1, the authors offer upper bounds on opportunity and risk over an infinite horizon, accounting for the evolving nature of the game states and strategies. In Theorem 4.2, they affirm the lower bounds in dynamic settings, reinforcing the existence of inherent trade-offs between exploiting accurate type beliefs for higher returns and safeguarding against errors to minimize losses.\n\nSTRENGTHS:\n-The characterization for the trade-offs between opportunity and risk is neat and relatively complete. \n-The research direction of this paper is interesting to me.\n\nWEAKNESSES:\nBoth the technical novelty and the importance of the results are unclear to me. For the technical novelty, it seems to me that the analysis in this paper, although non-trivial, is standard in machine learning theory. The authors did not mention about the new technical tools used in this paper. I would recommend the authors briefly summarize about the technical novelty of this paper, compared with existing work.\n\nI am also uncertain about the impact of the results. While I agree that the research direction of this paper is interesting, I am not sure the specific choice of studying the trade-off between “opportunity” and “risk” is the most natural one.\n\nIn addition, this paper exceeds the page limit!\n\nQUESTIONS:\nI do not have more questions. Please feel free to comment on anything you do not agree with my review.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-11"}

comments: SUMMARY:
The paper explores the dynamics of Bayesian games where players form beliefs about other players' types and update these beliefs based on observed actions. It specifically examines the risks associated with relying on potentially inaccurate type beliefs and how these inaccuracies impact the payoffs of strategies. The main focus is on quantifying the tradeoff between opportunity (the potential payoff when beliefs are accurate) and risk (the potential loss when beliefs are incorrect). The paper provides theoretical bounds on this tradeoff for both normal-form and stochastic Bayesian games.

STRENGTHS:
The paper tackles the under-explored area of safe and exploitative strategies in Bayesian games with incorrect type beliefs, contributing new theoretical insights. The authors provide upper and lower bounds on the payoff gap, which characterises formally the interesting tradeoff between opportunity and risk.

WEAKNESSES:
Although the paper is fair well-structured, the paper's notation is quite dense and could be simplified for better readability, especially for those not very familiar with Bayesian games.

QUESTIONS:
No

LIMITATIONS:
Authors did discuss the limitations of this work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-12

comments: SUMMARY:
This paper characterizes the trade-off between risk and opportunity in the games where the player need to infer other participants types. Upper and lower bounds in normal-form and stochastic Bayesian games are proved.

STRENGTHS:
This paper provides a solid theoretical foundation for normal-form and stochastic Bayesian games with type uncertainty. The presentation is clear with good examples.

WEAKNESSES:
While the overall presentation is good, the problem, by its nature, can be notation-heavy. I would suggest the authors to add more recall to the previously defined quantities when involving them several pages later, and/or make a notation table.

As the authors mentioned, tightening the upper and lower bounds is a future direction. However, it is not clear how much the current results are off from the tight ones. Thus, including the comparison between the true opportunity/risk vs. theoretical bounds would be helpful for evaluation of the currents results. If possible, also including adversarial examples in the experiment section to show how current bound can come loose in some scenarios would strengthen the completeness of the work more.

QUESTIONS:
Please see weakness section.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-15