Team-Fictitious Play for Reaching Team-Nash Equilibrium in Multi-team Games
Authors: Ahmed Said Dönmez
Yüksel Arslantaş
Muhammed O. Sayin
Venue: NeurIPS 2024 poster (2024)
Keywords: Multi-agent reinforcement learning
fictitious play
multi-team games
Primary area: algorithmic_game_theory
TL;DR: A new variant of fictitious play provably converging to Team-Nash equilibrium in multi-team zero-sum games.

Abstract: Multi-team games, prevalent in robotics and resource management, involve team members striving for a joint best response against other teams. Team-Nash equilibrium (TNE) predicts the outcomes of such coordinated interactions. However, can teams of self-interested agents reach TNE? We introduce Team-Fictitious Play (Team-FP), a new variant of fictitious play where agents respond to the last actions of team members and the beliefs formed about other teams with some inertia in action updates. This design is essential in team coordination beyond the classical fictitious play dynamics. We focus on zero-sum potential team games (ZSPTGs) where teams can interact pairwise while the team members do not necessarily have identical payoffs. We show that Team-FP reaches near TNE in ZSPTGs with a quantifiable error bound. We extend Team-FP dynamics to multi-team Markov games for model-based and model-free cases. The convergence analysis tackles the challenge of non-stationarity induced by evolving opponent strategies based on the optimal coupling lemma and stochastic differential inclusion approximation methods. Our work strengthens the foundation for using TNE to predict the behavior of decentralized teams and offers a practical rule for team learning in multi-team environments. We provide extensive simulations of Team-FP dynamics and compare its performance with other widely studied dynamics such as smooth fictitious play and multiplicative weights update. We further explore how different parameters impact the speed of convergence.

Team-Fictitious Play for Reaching Team-Nash Equilibrium in Multi-team Games
Ahmed Said Donmez Bilkent University
Yuksel Arslantas
Bilkent University
said. wwwweeooimiiiooooooooooooooii lkent. edu. tr
yuksel www.aaaalaaaanaaaaaaaaaaaaaaaaaa edu. tr
Muhammed O. Sayin Bilkent University
edu.e tr sayinβee. bilkent. edu.
Abstract
Multi-team games, prevalent in robotics and resource management, involve team members striving for a joint best response against other teams. Team-Nash equi- librium (TNE) predicts the outcomes of such coordinated interactions. However, can teams of self-interester agents reach TNE? We introduce Team-Fictition Play (Team-FP), a new variant of fictitious play where agents respond to the last actions of team members and the beliefs formed about other teams with some inertia in action updates. This design essential in team coordination beyond the classical fictitious play dynamics. We focus zero-sum potential team games (ZSPTGs) where teams can interact pairwise while the team members do not necessarily have identical payoffs. We show that Team-FP reaches near TNE in ZSPTGs with quantifiable error bound. extend Team-FF dynamics to multi-team Markov games for model-basee and model-free cases. The convergence analysis tackles the challenge of non- non-stationarity induced by evolving opponent strategies based the optimal coupling lemma and stochastic differential inclusion approxima- tion methods. Our work strengthens the foundation for using TNE to predict the behavior of decentralized teams and offers practical rule for team learning in multi-team environments. We provide extensive simulations of Team-F dynamics and compare its performance with other widely studied dynamics such as smooth fictitious play and multiplicative weights update. We further explore how different parameters impact the speed of convergence.
Introduction
Multi-team games are increasingly common, e.g., in robotics and resource management (Silva and Chaimowicz 2017 Vinyals et al. 2019 Jaderberg et al., 2019). Unlike non-cooperative multi-agent settings, team members strive for a joint best response against other teams if the entire team is a single decision-maker. Team-Nash equilibrium (TNE), where team members coordinate in the best team response against other teams, can capture this predict the outcome of coordinated team interactions [Farina et al.] 2018 /Zhang et al. 202T]. Game-theoretical equilibrium often justified by its emergence from non-equilirriu adaptation of self-interested learners (e.g., see (Fudenberg and Levine 2009]). However, the question of whether the teams of self-interested agents can reach TNE mulli-team games remains largely unexplored. This paper investigates this very question.
For example, TNE generally can arise if the team members can learn to correlate their actions the best team response independent of the opponent. However, the widely studied fictitious play (FP) dynamics do not necessarily reach the best team outcome even when there are no opponents, e.g., in
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Figure I: An illustration of networked interconnections agents from different teams. Nodes in bottom and top lay- refer, resp., team members and teams. Undirected edges represent the impact of actions on the payoff func- tions. We use different colors and shapes to represent agents from the same teams, and they are connected via dashed edges.
potential games. We present a slight adjustment of FP, called Team-FP, provably reaching TNE in multi-team competition even with networked interconnections, where agents' payoffs depend only on the neighbors' actions. Similar to the FP, here, agents respond greedily to the beliefs formed about the opponent teams' joint play based on past observations. Different from the FP, Team-FP incorporates
the key features: (+) response to the last actions of team members, and (fi) inertia in action updates. These features, inspired by log-linear learning dynamics [Blume| [993]. play crucial role in driving team coordination towards TNE. Notably, Team-FP reduces to the smoothed FP [Fudenberg and Kreps] [993] (or log-linear learning) when each team has a single agent (or there is a single team).
Multi-team competition spans diverse domains. For example, robotics, resource management, online gaming, and financial markets [Kitano et al., 1997 Cardenas et al. 2009 Silva and Chaimowicz et al. 2019, (aadeberg et al.] 2019 involve multi-team competition. To model such interactions, we consider multiple teams with possibly different number of team members. These team members have networked interconnections, as depicted in Figure [ We focus on multi-team zero-sum potential team games where teams have pairwise interactions (ZSPTGs). For any opponent team strategy, team members effectively play underlying potential game, as in distributed optimization applications, c.g., see [Arslan et al. 2007] Xu et al. | [012]] Zheng et al., [014]. Notably, ZSPTGs reduce zero-Sum polymatrix games CCai et al.) 2016) if each team has single agent, and to potential games [Monderer and Shapley 1996a] if there is a single team. Additionally, widely studied
two-team zero-sum games, e.g., see [Farina al. 2018 Zhang al. 2021 Carminati al. 2022
Kalogiannis et al. 2022]. are a special case of ZSPTGs.
We show that the FP dynamics reach near TNE in ZSPTGs. This means that the empirical average of team actions converge to the near best response each team can take against the average actions of other teams. We quantify the approximation error, showing it decreases with the level of exploration the agents' responses. Similar to the FP dynamics, Team-FP also rational: teams can learn (near) optimal strategies if opponent teams play stationary strategies. These results strengthen the applicability of TNE for predicting team behavior in multi-tcam competition and provide a practical rule for teams of self-interested agents to learn coordination in multi-team settings.
key challenge in our analysis is handling the stationtt nature of learning, as opponent teams' strategies change over time. We address this by leveraging the optimal coupling lemma (e.g. see [Levin and Peres| 2017 Chapter 4]) and stochastic differential inclusion approximation methods
(e.g., see [Bnnamm et al. 2005 Perkins and Leslie, 2013) to the repeated play of games. Motivated from the recent interest multi-agent reinforcement learning, we can extend Team-FP dynamics to finite horizon multi-team Markov games for both model-based and model-free cases. We discuss this extension and analyze its convergence numerically in Appendix C
Related works. FP and its variants offer convergence guarantees in important classes of games Fuu denberg and Levine 2009], yet not in every class of games [Hart and Mas-Colel] 2003). For example, they reach equilibrium in potential games [Monderer and Shapley 1996a]. but not necessarily the most efficient one for the team. Log-linear learning can achieve effciint equilibrium for the team [Marden and Shamma 2012, Tatarenko, [017]. However, it is not clear whether such dynamics can track efficient equilibrium in dynamic environments (induced by the evolving strategies of opponent teams). Notably, Tatarenko (2018) and Donmez et al. (2024) addressed, resp., efficient learning under non-stationarity induced by the decaying exploration in agents' responses for the repeated play of potential games and non-stationarity induced by evolving stage games in Markov team problems

---

(also known as ddentical-interes Markov games). These approaches are orthogonal to our analysis to extend our results to the exact TNE convergence in repeated multi-team games or to learning in infinite horizon Markov games.
FP and its variants can reach equilibrium in two-agent zero-sum games (Hofbauer and Sandholm 2002] yet not necessarily in multi-agent zero-sum games with more than two agents. We can transform any general-sum game to a multi-agent zero-sum game by introducing a non-effective auxiliary agent (with single action). There have been several attempls to address zero-sum games beyond two-agent cases [Bergman and Fokin 1998, Cai and Daskalakis 2011 Cai et al. 2016]. For example, Ewerhart and Valkanova| [2000] addressed the convergence of continuous and discrete-ting time FP in zero-sum network games. where each agent plays multiple two-agent games with separate actions, and the overall utilities sum to zero. Notably. Cai et al. [2016] introduced zero-sum polymatrix games where agents have network separable pairwise interactions with applications in security. Recently, FP has been shown to reach Nash equilibrium in zero-sum polymatrix games (Park et al. 2023]. Following the same trend, we focus on learning in ZSPTGs extending two-team zero-sum games to multi-team games with pairwise team interactions. However, we highlight that two-team zero-sum or ZSPTGs are not necessarily multi-agent zero-sum polymatrix games (as the agent payoffs do not necessarily sum to zero) and the classical FP dynamics do not necessarily converge TNE or Nash equilibrium in such multi-team games.
Recent studies on two-team zero-sum games and adversarial team games (e.g.. see [Celli and Gatti 2018 Farina et al. 2018 Zhang et al. 2022 Carminati et al., 2022] have primarily focused on the efficient computation of team-minimax equilibrium. In particular, Celli and Gatti (2018) examines the efficiency of different communication types, highlighting the promising results of ex ante communication, referring to pre-play communication among team members. Consequently, the studies of Farina et al. [2018). Zhang et al. [2022), Carminati et al. [2022] often model teams as a single agent with imperfect recall, incorporating ante communication within the team. Notably, Farina et al. 2018 introduced the Fictitious Team Play (FTP) algorithm for extensive-form two-team zero-sum games with imperfect information, where team members communicate ex ante. In this approach, Farina et al. [2018) used fictitious play (FP) on a simplified version of the original game, embedded within the game tree. This method essentially applied FTP to the original adversarial team game. To find the best response, they used mixed-integer linear programming. Team-FP differs from such approaches by letting agents learn team up while following their self-interest based simple behavioral rules, as in the log-linear learning and FP dynamics.
The rest of the paper is organized follows. We describe ZSPTGs in 2 We present the (independent) Team-FP dynamics in $ We provide analytical and numerical results, resp., in $ and $5) We conclude the paper with some remarks in $6 Appendices include the proofs of the technical results, some further numerical experiments and the extension to multi-team Markov games.
Notation: Given finite set A, we let (AA) denote the probability simplex over A. We let f(u) Eawal If [a) for any probability distribution μ E A(A) and any functional f A - R. Furthermore. we define the smoothed best response to any functional q : A + R by
exp (q(a)/T) (1(((T))
br (a)
Va E A
+ br. argmax (g(n) + +H])), HEA(A)
for some temperature parameter > 0, where H(1)) == Earyul log(u(()))) for all μ E A(A) is the entropy regularization.
2 Game Formulation
Consider a multi-team game, characterized by the tuple (T, (4', w²) jezz), where T and I denote, resp., the teams' and agents' index sets, and A' and w' : A + R (with A := Iljer A') denote, resp., the agent i's finite action set and payoff function. Agents take their actions to maximize their payoff functions.
Definition 2.1 (Zero-sum Potential Team Game). Let I denote the index set of agents in team m and nTM := IIezz A denote the set of joint actions for team m. We say that a multi-team game zero-sum sum potential team game (ZSPTG) if for each team m E T, there exists potential function
3

---

6th :A -A- + R such that
a' a ","",a"a""0 " u'(a) = 6th (a' ")) -0" (a).
(2)
for all (a'.a) A A and i E T where a == (a JEZ=)() are the actions of other team members, m feym are the action profiles of other teams, where a' E AA f is team l's action profile. The potential functions sum to zero, i.e., we have
Σ MET
(a) = 0 A A.
(3)
Purthermore, the actions have network separable interactions across teams such that we can separate the potential functions and correspondingly payoff functions and
u²
= Σ rt VIEI"
(4)
Espom
Exer
for some om A 1 R and 2
A R.
The following example generalizes the potential game formulation for distributed optimization (e.g.. [Arslan et al., 2007 Xu al. 2012 Zheng al. 2014)) to two-team zero-sum potential games. Example 2.2. Consider two teams of agents interacting over a network. We can represent their interactions via graph G (V. F.). where the set of vertices V refers to the agents and the set of (undirected) edges refers to their interactions. Let agent i from team m receive a local payoff r A'' x ( A - R depending on the actions of the neighboring agents only. Agent adds the neighboring team members' local payoffs whereas subtracts the other neighbors' local payoffs in his/her total payoff. In other words, his/her total payoff given by
Σ TIJEZII'
Σ
Iysemprr.
(5)
}(J-)E
j((j)E
This yields that the team m has the potential function
Σ
Σ
r
(6)
jEZ
jEZm
and therefore, these potential functions sum to zero. However, the potential function is not generally
the sum of the payoffss in potential gamess.
Remark 2.3 (General-sum ZSPTGs). In ZSPTGs, the underlying game can be a general-sum game although the team-polt comm-ooenttills sum to zero, as described in (3). For example, consider two competing teams whose team members have identical payoff's corresponding to their team potentials. If the teams have different number of members, then the agents' payoff's do not sum zero or a constant while the team potentials do so.
In the following, we introduce TNE, generalizing team minimax equilibrium for two-team zero-sum games, c.g., see (Von Stengel and Koller 1997]. to multi-team games. Particularly, TNE, no team has an incentive to change their team strategy.
Definition 2.4 (Team-Nash Gap). Given the strategy profile of teams {TT E A(A")) MET, we define the team-Nash gap for team
TNG (T)
max #EA(AM)
80t (TT)
and ( strategy profile of teams { TMET is e-TNE TFGG S.
where -m fTr Jeam- Correspondingly, we say that the = EmET TNG
3 Team-FP Dynamics
We first present the Team-FP dynamics combining the log-linear learning and fictitious play for learning multi-team games played repeatedly, and then extend Team-FP to multi-team MGs in Appendix

---

Algorithm 1 (Independent) Team-FF
initialize: { and {a (bezm arbitrarily for each stage O, o,..... do play d br,("" (:, ak-1, Th Tx") ")) or of = ak-1 1 in coordinated way (or independently) update of = o + an T for all +
end for
Let oj. E A' denote the agent i's action at the kth repetition in the repeated play of the underlying ZSPTG. Correspondingly, ox" (ai) denote the team m's action profile. Observing the joint actions of team m, agents & T can form a belief about the team m's joint strategy. Let Tk E ((A")) denote the belief they formed. Consider actions pure strategy where the associated action gets played with probability 1. Then, agents & I can update their beliefs about team m's
strategy according to
Th- m ax xxk Wk = 0, 1,...
such that the belief T241 also corresponds to the (weighted) empirical average of the past action profiles {ah" ................................ axa)
Agent i E T" either takes the previous action - (i.e., at = - ) or takes the action aj ~ br, (w' (-, ))) according to the smoothed best response (as described in (I)) to the previous actions of team members ax-1 == fak-1 "iiee"eee (() and the beliefs "h " = { hy formed about other teams. It is instructive to note that the definition of potential function a=' (-), as described in yields that
br, (u'(, (' Tk )) = br, (1 ")) Vi T
We introduce Team-FP and independent Team-FP dynamics depending how agents update their actions. In the former, a single agent can get chosen randomly, as in the classical log-lineat learning. In the latter, each agent update his/her action with probability 8 E (0, 1) independent of others, as in the independent log-linear learning. The latter addresses the coordination burden in the update of actions within teams. Algorithm provides deseriptions of these dynamics for the typical agent i from team m.
Remark 3.1 (Scalability). Agents can have networked interactions such that their payoff functions depend only the actions of certain agents such as one/two-hop neighbors. For such cases, agents can form beliefs about these neighbors' strategies only, as if these neighbors play according to some stationary strategy. For example, assume that the payoff of agent & I (outside team m) depends only on the actions of team-m members from some neighborhood N i.c., i jj:ja N } Agent can form a belief about these agents' strategies according to
n im + ax km) Ak 0, 1,
(10)
#2+1
where aith = {a jev Then, the linearity of the update rule yields that the local empirical average m corresponds to the marginalization of xh such that
T im (Ha') hENNinz)) {as 1exmm AA
Σ
T (Ha' ()Hmm)
Yk.
(11)
Therefore, local observations (within neighborhoods) would still be sufficient to follow Algorithm We demonstrate the scalability of Team-FP by large-scale experiment in Appendix D Figure
We focus on the homogeneous cases where agents i &f I" have a common belief about team m strategy. Homogeneous beliefs are possible if the agents have common initial beliefs and step sizes.
We moved the extension of (Independent) Team-FP to multi-team Markov games and its numerical analysis to Appendix C
4 Convergence Results
Team-FP and Independent Team-FP dynamics reduce, resp., to the classical log-linear learning and independent lin-linear learning dynamics if there is only one team. These linear learning
5

---

dynamics are known to reach ream-optimal solution in potential games. For example, consider an n-agent potential game ((A', u² ke(()) with the potential function <(-). In both dynamics, the action profiles played form irreducible and aperiodic Markov chains. Let ğ and jú denote the unique stationary distributions, resp.. for the classical and independent versions. For the former, we have f = (Ø()) [Marden and Shamma 2012 Section 3J and ① yields that
0 S max()())) - (TT) S 7 log [A].
(12)
On the other hand, the smaller 8 0 implies closer stationary distributions in the classical and independent versions. Particularly, we have
a Malla S A(o,
(13)
as ō 0+ for any c > 0, and 0 < < S br, ( a *)) for any " and is lower bound on local actions get played in the smoothed best response Donmez et al. 2024 Lemma 5.6].
for some function A-) decaying
Te dynamics have similar convergence guarantees for multi-team games under the following assumption on the step sizes used.
Assumption 4.1. The step size og [0, satisfies the stochastic approximation conditions: ag +
as 1: + 00, k= OA- = 0o, and Ex k= of < co. Additionally, we have limg-poc 0k/Ok+ 1, and Og 0 2 aga++1
The last condition in Assumption 4.1 ensures that recent observations have comparable weight in the belief' formed. The classical choice 1/(k - 1), leading to the empirical averages of the actions played, satisfies Assumption 4.1
The following theorem shows the convergence of Algorithm I to near TNE almost surely with the approximation levels (similar to (12 and (33 inherited from the (independent) log-linear learning dynamics.
Theorem 4.2. Given a ZSPTG characterized by (T, (A', u² JEz), let every agent follow either Team-FP Independent Feam-FP, described in Algorithm 7 team-Nash gap for Ti == (TT MET satisfies
If Assumpttin 4.7 holds, then the
T log |A||
for Team-FP
sup TNG(TR) <
(14)
T log |A| + 1770. A((,e) for Independent Team-FP
10-000
almost surely, where Φ = (mx (a)).
The key challenge in our convergence analysis is the non-stationarity arising from opponent teams adapting their strategies while the team members learn to coordinate against them. We address this by constructing a reference scenario where the team members' beliefs about opponent strategies are frozen over finite-length epochs, allowing them to hypothetically "team up" under these fixed beliefs. By comparing the dynamics in the actual scenario and the reference scenario, and exploiting the averaging nature of belief formation, we can bound the approximation error between the two. The main proof concept, along with the Team-FF -P algorithm for a two-team scenario, illustrated in Figure 2] This approach is similar the used in [Donmez et al. 2024 to handle non-stationarity in Markov team problems. However, unlike [Donmez et al.) 2024]. we cannot directly show the error bound decay due the lack of contraction property in dynamics.
To overcome this limitation, we view Team-FP as smoothed fictitious play dynamics in zero-sum polymatrix games with an additive bounded eгтor term. The additive error captures the fact that team members may not perfectly achieve team coordination. We then relax the problem by considering any approximation error within the formulated bounds, rather than the actual error. To address this relaxation, we leverage stochastic differential inclusion approximation methods [Benaüm et al. 20055 Perkins and Leslie 2013). Finally, by constructing suitable Lyapunov function addressing arbitrary bounded errors in continuous- smoothed best response dynamics, we establish the convergence of the discrete-time Team-FP updates.
The following corollary to the main result shows the rationality of the (independent) Team-FP dynamics.

---

Epieode
Team 1
Team 2
Road
Road
Joint Belief
Transition Kemel
PV-)
PV-)
P(-)
Actual Sconario
Transition Kerne P(+{)
ej
Reference Sconatio
1pome
1-1
P(-)
P1-
Pt
An illustration of Team-FP dynamics for two-team games on the left-hand side. Team actions change according to a transition kernel depending on the beliefs formed about the other teams. Dashed lines represent the time shift. On the right-hand side, we depict the key proof idea that we approximate the evolution of the team actions with a reference scenario where beliefs are stationary such that team actions form homogeneous Markov chain whose unique stationary distribution corresponds to the best team response.
Figure 2:
Corollary 4.3. Given a ZSPTG characterized by (T, (A', ut )izz), let agents from team m follow either Team-FP or Independent Team-FP while other teams play according some stationary strategy π If Assumpttin 4.1 holds, then empirical average of the action profiles played by team m satisfy
log AA log A + +5-A((,))
for team-FP for Independent Team-FP
sum TNG" (T
(TT
n ") <
(15)
k-+00
almost surely.
Theorem 4.2 can be generalized to the case where the rewards are random with bounded support, rather straightforwardly. Therefore, the proof of Corollary follows from Theorem 4.2 we view the underlying game as there a single team receiving random rewards with bounded support.
Illustrative Examples
In this section, we present various simulation results demonstrating the coordination speed of Team- FP and compare to pure FP, no-regret algorithms, and a stationary opponent. We also observe the effect of parameters team-FP in games beyond ZSPTG games, where we intuitively expect it to converge. All simulations averaged over 10 independent trials to reduce the randomness. In all figures, the mean is plotted with a thick colorful line, while one standard deviation below and above the mean is shown with a shaded area of the same color. Also, for all simulations, temperature parameter is chosen to be 0.1 unless another option is mentioned. We conduct simulations for ZSPTG with two different setups: with three teams, each consisting of three agents, and another with two teams, each consisting of four agents. Unless explicitly stated otherwise, the default setting consists of two teams, with four agents in each team. The step size is chosen to be az 1/(k: + for all simulations.
the convergence speed. In addition, examine the long-run behavior of
All the simulations are executed on a computer equipped with an Intel Xeon W7-3455 CPU and 128 GB RAM. Run-time for 10 independent trials over 10th iterations vary between 1-5 hours depending on the experiment.
Achieving Implicit Coordination in Team-FP In this section, we compare the performance of Team-FF to the explicit coordination team members. We also compare Team-FP to algorithms such as Multiplicative Weights Update (MWU) and Smoothed FP (SFP), and show that these algorithms fail to achieve team coordination. We also show that Team-FP achieves near-optimum policy against. stationary opponent stated in Corollary 43
7

---

TramsPP and Gorriiation
Tear Mel and
Cfack of Compattion
Na centbille rundiaaiio
fadepeakel
12-
Campating Oppense
Trau
Fst coreitation
Strbeery (tpeent
(a) Varying group sizes of 1, 2, and
(b) Algorithm| (Independent) compared to SFP, and MWU
(c) Competitive vs Stationary
4.
Environment
Figure 3: All the above figures show the variation of TNG over time. (a) Comparison of different levels of explicit coordination for Team-FP: independent agents (group size 1), pairs of cooperating agents (group size 2). and fully coordinated teams (group size 4). (b) Performance of Team-FP and Independent Team-FP compared to MWU and SFP algorithms in a 2-team ZSPTG. (c) Convergence of Team-FP against stationary and competitive opponents in a 3-team ZSPTG.
Impact of Team Size in Team Coordination: In Team-FP self-interested team members act together without explicit communication in coordinated way to reach TNE. We can measure how this independent cooperation compares to the explicit coordination of team members. For that, we propose an example game with two teams and four agents in each team. For the first scenario, four agents act separately and use Deam-FP. In the second scenario, the agents form groups of two and act coordinated way as if they are single agent, using Team-FP. In the final scenario, all agents in team behave as if they were single agent, equivalent to the standard fictitious play (FP) dynamics in a two-person zero-sum game. This case mimics the ex-ante communication scheme from [Farina et al.) 2018). The scenarios are described by group sizes. Group sizes are one, two, and four respectively for these scenarios. The simulation results are presented in Figure 3a As expected, the explicit coordination of all members in team converges the fastest, followed by the explicit coordination of groups of two, and finally, Team-FP converges slowly as the agents do not coordinate explicitly.
Team-FP Compared to MWU and SFP: The strong side of Team-FP is, even though the agents do not communicate, the average of joint actions of teams can reach TNE, unlike other algorithms. We compare the equilibrium behavior of Peam-FP with well-known no-regret learning algorithm Multiplicative Weights Update (MWU), in which the average strategies converge to NE zero-sum polymatrix games [Cai et al.) 2016). We also compare Team-FF with the usual SFP, where each agent holds beliefs about other agents and uses the smoothed best response against them. Figure (3b] we see that both Team-FP and Independent Team-FP dynamics converge to near TNE, while the other algorithms fail to do so.
Rationality Against Stationary Opponent: In this part, we use 3-team setting where each team has 3 agents. We let team using Team-FP compete against two other stationary teams and compared with the performance of the same team in the same game when the opponents are also using Team-FF competitively. In Figure 3c] we observe that both algorithms converge, while the convergence is much faster against stationary opponents.
Te in Application In this part, we provide example to demonstrate that Team-FP has applications in various contexts.
Security Game Example: We model an airport security scenario as a two-tcam game between defender and attacker teams. In our example, a security chief on the defender team faces three independent intruders on the attacker team, as shown in Figure 4a| The chief can defend a gate at cost, while intruders decide whether to attack a gate or remain idle. Intruders gain or lose payoffs based on whether they attack undefended or defended gates, and the chief's ' payoff's mirror these outcomes. We conducted multiple trials and presented the evolution of the average Team-Nash Gap with standard deviations on the right side of Figure 4b) From a higher level, this example shows that team-minimax equilibrium can predict the outcome of games with different uncoordinated attackers. It also justifies the algorithms developed to compute ream-minime equilibrium efficiently.
8

---

Airport Delence Simulaiion
62-
0.0
Hamber
(b)
Figure 4: (a) The illustration of an airport security game: security chief guarding the six gates of an airport against three different intruders making decisions autonomously. (b) The evolution of Team Nash Gap in airport security game, showing that Team-FP dynamics reach near team-minimax equilibrium.
e D. 6m0.5
sunber Baraboe
sumber MAn6000
(a) Simulation network vs 3 Vs game.
(b) Varying T in Team-FP
(c) Varying 5 Independent Team-FP
Figure 5: The 3-team experiments tested the randomly generated network structure (a). The other figures (b), and (c), shows the variation of TNG over iterations. (a) The simulation network for multi-team ZSPTG, in which there are 3-teams with agents each team. (b) The impact of varying temperature parameter (0.1, 0.15, 0.2) in Algorithm on the closeness to TNE. (c) The effect of different values (0.2, 0.5) in (Independent) Algorithm]] on the convergence speed with fixed at 0.1
Effect of parameters in Team-FP this part, we examine how Team-FP performs for different and 8 values. Given example ZSPTG of three teams where each team has three agents in Figure 6aa we examine the evolution of TNG in the Team-FP dynamics for different values of (0.1,0.15, 0.2). We also compare the evolution of NG in the Team-FP and Independent Team- FP dynamics for 7 = 0.1, 5 = 0.2, = 0.5. All simulation results (see Figure 5) show convergence, and we observe lower final values of NG(T) for smaller Independent Team-FP requires more iterations when 8 = 0.2 for its convergence, while is much faster when 8 0.5 (see. Figure (c).. This expected updates much more frequent ò increases. However, increasing 5 too much may harm the coordination behavior of the team.
predicted (see. Figure 5b). The
Beyond ZSPTG In this part, we try several other games for Team-FP without proof convergence. We expect Team-FP to converge in 2xN and potential games other than zero-sum games as FP converges in these settings. For the first case, we try a game where one team has a single agent with only two actions with random rewards, while the other team has three agents with an underlying potential function. In this case, Team-FP converges (see. Figure (b)). In another setting, we try two teams of four agents. However, the potential functions for both teams are identical rather than summing to zero, resulting in a potential function that encompasses The team-FP algorithm converges to TNE in this case as well (see. Figure (c). However, the equilibrium is not unique in this case. Finally, we create an extension of the team-FP for Markov games (sec. Appendix C) in RL framework and try simulations this setting. In this case, there
individual potential functions.

---

Partow
Thams Niaant Game
Mhabd Free
12-
Novdd Band
08-
02-
64-
(a) Markov Games
(b) 2xN General Sum Game
(c) Potential of Potentials Game
Figure 6: All the above figures describes the variation TNG over iterations for Algorithms that are related to but outside the scope of ZSPTG. (a) The model-free and model-based Markov games of Algorithm 2 and [3] for game of 2-team each with agents, with 2 states and 10 horizon length. (b) The behavior of Team-FP dynamics in a 2xN general sum game, where a team competes against single agent with random rewards. (c) The behavior of Team-FP dynamics potential game over the underlying potential functions.
are two teams each having two agents, competing against each other in a finite horizon Markov game with a horizon length of ten. The number of states is two, and the state transition matrices are generated randomly. We observe that team-Nash Gap for MG's defined in (65) converges to near-zero.
6 Conclusion
In this paper, we introduced Team-FP, a novel fictitious play variant designed for multi-team games. We showed that Team-FP provably achieves near-TNE in ZSPTGs, with quantifiable error bound. Our convergence analysis addressed the non-stationarity challenge arising from evolving opponent team strategies, by leveraging the optimal coupling lemma and stochastic differential inclusion approximation methods. We also extended Team-FF to multi-team Markov games, encompassing both model-based and model-free scenarios, with applications in multi-team reinforcement learning. Furthermore, we conducted detailed numerical analysis of the Team-FP dynamics, focusing on the trade-off in learning to team up and competition in comparison to the classical FP and no-regret learning dynamics. We further examined the convergence of FF dynamics to TNE in multi-team games beyond ZSPTGs. These results strengthen the theoretical foundations for applying TNE to predict decentralized team behavior and provide framework for team learning in multi-team settings.
Limitations and Broader Impacts Our work quantified the almost sure convergence of the Team- FP dynamics asymptotically yet did not provide guarantees for the convergence rate. We conducted detailed numerical examples and presented the evolution of the gap in TNE to exemplify this rate qualitatively. The challenge for the non-asymptotic analysis is inherited from the discrete-time FP dynamics for which there are only rough rate analysis [Karlin] 1959 and negative examples for some edge cases [Brandt et al. 2010 Dassallakis and Pan 2014].
Our work introduces no new ethical concerns in multi-team systems and shares the assumption of stationary opponents with learning dynamics like Q-learning and fictitious play. This assumption does not disadvantage our approach. We argue that treating uncoordinated attackers as a single decision-makes is necessary, they can learn to bypass security measures. Our paper provides theoretical basis for this, ensuring more reliable Al-based solutions.
Future Research Directions This work paves the way further explore the behavior of decentral- ized teams in multi-tean interactions when the team members follow different types of dynamics for teaming up within teams (other than log- log-linear learning) and adapting to other teams' play (other than FP). Numerical examples we conducted for multi-team games beyond ZSPTGs are also promising to show the provable convergence of Team-FP dynamics in multi-team (Markov) games reducing to games with the fictitious play property (e.g., see [Monderer and Shapley 1996b)) ) if the teams coordinate in acting as a single player.
10

---

Acknowledgements
This work was supported by The Scientific and Technological Research Council of Türkiye (TUBITAK) BIDEB 2232-B International Fellowship for Early Stage Researchers under Grant Number 12IC124.
References
G. Arslan, M. F. Demirkol, and Song. Equilibrium efficiency improvement in MIMO interference systems: A decentralized stream control approach. IEEE Transactions Wireless Communica- lions, 6(8):2984-299) 2007.
M. Benaïm, J. Hofbauer, and S. Sorin. Stochastic approximations and differential inclusions. SIAM J. Control Optim, 44(1):328-33) 01888-48,222 2005.
L. M. Bergman and I. N. Fokin. On separable non-cooperative zero-sum games. Optimization, 44(1): 69-84, 1998.
L. E. Blume. The statistical mechanics of strategic interaction. Games Econom. Behav., 5(3):387-424, 1993.
F. Brandt, F. Fischer, and P. Harrenstein. On the rate of convergence of fictitious play. In Algorithmic Game Theory: Third International Symposium, SAGT 2010, Athens, Greece, October 18-20, 2010. Proceedings 3, pages 102-113. Springer, 2010.
Y. Cai and C. Daskalakis. On minmax theorems for multiplayer games. In Proceedings of the 22nd Annual ACM-SIAM Symposium on Discrete Algorithms, pages 217-234. SIAM, 2011.
Y. Cai, O. Candogan, C. Daskalakis, and C. Papadimitriou. Zero-sum polymatrix games: A general- ization of minmax. Math. Oper. Res, 41(2):648-655, 2016.
A. Cardenas, S. Amin, B. Sinopoli, A. Giani, A. Perrig. S. Sastry, et al. Challenges for securing cyber physical systems. In Workshop on Future Directions in Cyber-physical Systems Security, volume 5, page 7, 2009.
L. Carminati, Cacciamani, M. Ciccone, and N. Gatti. A marriage between adversarial team games and 2-player games: Enabling abstractions, no-regret learning, and subgame solving. In Int. Conf. Machine Learn. (ICML), pages 2638 26577. PMLR, 2022.
A. Celli and N. Gatti. Computational results for extensive-form adversarial team games. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
C. Daskalakis and Q. Pan. A counter-example to Karlin's 's strong conjecture for fictitious play. In 2014 IEEE 55th Annual Symposium on Foundations of Computer Science, pages 11-20. IEEE, 2014.
A. Donmez, O. Unlu, and Sayin. Logit-Q dynamics for efficient learning stochastic teams. arXiv preprint 2rXiv:2302.09806, 2024.
C. Ewerhart and K. Valkanova. Fictitious play networks. Games Econom. Behav., 123:182-206. 2020.
G. Farina, A. Celli, N. Gatti, and T. Sandholm. Ex coordination and collusion in zero-sum multi-player extensive-form games, Advances in Neural Inform. Process. (NeurIPS), 31, 2018.
D. Fudenberg and D. M. Kreps. Learning mixed equilibria. Games Econom. Behav, 533)3220-367, 1993.
D. Fudenberg and D. K. Levine. Learning and equilibrium. Annual Rev. Econ., 1(1):385-420) 2009.
S. Hart and A. Mas-Colell. Uncoupled dynamics not lead Nash equilibrium, The American Econ. Rev,, 83(5): 1830-1888, 2003.

---

J. Hofbauer and W. H. Sandholm. On the global convergence of stochastic fictitious play. Economet- rica, 70(6):2265-222 2294,, 2002.
J. Hu and M. P. Wellman. Nash Q-learning for general-sum stochastic games. J. Machine Learn. Res., pages 1039-1069, 2003.
M. Jaderberg, W. M. Czarnecki, Dunning, L. Marris, G. Lever, A. G. Castaneda, C. Beattie, N. C. Rabinowitz, A. Morcos, Ruderman, et al. Human-level performance in 3d multiplayer games with population- based reinforcement learning. Science, 364(6443)) 859, 2019.
Kalogiannis, I. Panageas, and B-V. Vlatakis Khaakii-Gaaragkounii.. Towards convergence to nash equilibria in two-team zero-sum games. In Internat. Conf. Learning Representations (ICLR), 2022.
S. Karlin. Mathematical methods and theory in games. Programming, and Economics, 1,2, 1959.
H. Kitano, M. Asada, Y. Kuniyoshi, I. Noda, E. Osawa, and H. Matsubara. Robocup: A challenge problem for AI. Al magazine, 18(1):73-1) 73, 1997.
D. A. Levin and Y. Peres. Markov Chains and Mixing Times. American Mathematical Society, 2nd edition, 2017.
M. L. Littman. Markov games as a framework for multi-agent reinforcement learning, Int. Conf. Machine Learn. (ICML), 1994.
R. Marden and S. Shamma. Revisiting log-linear learning: Asynchrony, completeness and payoff-based implementation. Games Econom. Behav. 75(2):788-808, 2012.
D. Monderer and S. Shapley. Potential games. Games Econom. Behav., 14(1):124-144 1996a.
D. Monderer and L. S. Shapley. Fictitious play property for games with identical interests. Journal of economic theory, 68(1):258-266 1996b.
C. Park, K. Zhang, and A. Ozdaglar. Multi-playe zero-sum Markov games with networked separable interactions. In Advances Neural Inform. Process. (NeurIPS), pages 17354-37366 2023.
S. Perkins and D. S. Leslie. Asynchronous stochastic approximation with differential inclusions. Stochastic Systems, 2(2):409-446, 2013.
L. S. Shapley. Stochastic games. Proc. Natl. Acad. Sci. USA, (9(10)10095-1100 1100, 1953.
V. N. Silva and I.. Chaimowicz. MOBA: A new arena for game AI. arXiv preprint 1705 2017.
T. Tatarenko. Game-theoretic Learning and Distributed Optimization in Memoryless Multi- Systems. Springer, 2017.
T. Tatarenko. Independent log-linear learning in potential games with continuous actions. IEEE Trans. on Control of Network Systems, 5(3):913-92) 2018.
O. Vinyals, I. Babuschkin, W. M. Czarnecki. M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, et al. Grandmaster level in StarCraft using multi-agent reinforcement learning. Nature, 575(7782):350-33 354, 2019.
B. Von Stengel and Koller. Team-maxmin equilibria. Games Econom. Behav, 21(1-2):309-311, 1997.
Y. Xu, Wang, Q. Wu, Anpalagan, and Y. Yao. Opportunistic spectrum access in cognitive radio networks: Global optimization using local interaction games. IEFE Journal of Selected Topics in Signal Processing, 6(2):180-194, 2012.
B. Zhang, L. Carminati, F. Cacciamani, G. Farina, P. Olivieri, N. Gatti, and T. Sandholm. Subgame solving adversarial team games. Advances in Neural Inform. Process. (NeurIPS), 35:26686- 26697, 2022.
12

---

K. Zhang, Z. Yang, H. Liu, T. Zhang, and T. Başar. Finite-sampa sample analysis of decentralized batch multiagent reinforcement learning with networked agents. IEEE Trans. on Automatic Control, 66 (12):5925-594)) 2021.
J. Zheng, Y. Cai, Liu, Y. Xu, B. Duan, and X. Shen. Optimal power allocation and user scheduling in multicell networks: Base station cooperation using game- theoretic approach. IEEE Transactions Wireless Communications, 13(12):6928 6942, 2014.
13

---

A Proof of Theorem 4.2
We can separate the proof into two main steps: (t) showing that team members can learn to team up approximately by constructing a reference scenario where beliet's got frozen, and (ii) addressing the bounded approximation error by leveraging the stochastic differential inclusion approximations.
A.1
Step (i) Reference Scenario and Error Analysis
We divide the horizon into T-length epochs. Then, we can write the belief update from nI' to (n + 1)f as
8 accumulated
(n (n+1y-11
(++1)1-1
Ro + 1-
t T
(n+1)T
(1-as) (1-0))
ax)
"T
Σ
Ogk
II E=++1
(1
as)
ax
(16)
k=nT
k=nf
for any n = 0, I.... denoting the epoch index. Let (())
== a x
denote the belief about team m in
epoch n. Furthermore, for notational simplicity, we define
(n+1)7-1
(n+1)f-1 Σ k=nl
3x = Oak
II
(1-ar)
and B(a)
3x.
(17)
8=k+1
Then, we can simplify (16 as
(n+1)T-1 Σ
3x
(n+1) = (1 (1-)(n)) T(n) + B(m)
ok
(18)
((n)
k=nT
the step size ((n) decays to zero monotonically at a certain rate such that Donmez et al. 2024 Lemma 5.4]
Due to 4.1
Σ (m) = 0a n=0
and
(m) < cc.
(19)
n=0
Let F(n) be a filtration generated by the o-algebra a (Au...., AnT-1), where A, denotes the joint actions taken by each team at stage f, i.c., A, (a....., a.
17T). It is instructive to note that ((x) is
F(my measurabee. We define the joint action distributions of team m at time in epoch n by
((n),
Ela Fon]-
(20)
Then, can write (18) as in the form of stochastic approximation
(n+
Bk
m (n+11) (1 3(m)) (Th) + ((n)
Σ
p(()), + wim--))
(21)
B(m)
k=RT
where with (n+1)
is Martingale difference sequence defined by
(a+1)f-1
3R
(n+1)
Σ RETT
ax (())
(22)
B(m)
Now, let's consider reference scenario for the analysis in which the beliefs (denoted by )m)) are only updated the ends of T-length epochs. In other words, we have 77" =m for all
(()
nf < (n + 1)T - and m E T. Due to the fixed beliefs about the opponent plays, Team-FP dynamies reduce to the log-linear learning in the reference scenario. Let (()), denote the joint action of team m in the reference scenario. By the nature the log-linear learning, )e form homogeneous Markov chain (MC) even though the actual action profiles 1am J do not necessarily do so. Denote the stationary distribution of the MC in the reference scenario by (o),, and p,),, for Team-FP and Independent Team-FP, respectively. The former is given by
14

---

( br, (6 (0"" (m) )) due to the log-linear learning update [Blume
1993
Marden and Shamma
2012
Then, we write the belief update (21) for team
[m+11 = (1 - ((m)) + B(m)) (brr(o") (m)))))))) + (n+1) + e
(23)
where we decompose the error (n) = (m)
+ mm (n)
(1+1
BR
Σ
(()).))
(24)
(n)
((m)
k=nT
(n)
[[n),-
(25)
The update (23) corresponds to the SFP dynamics of teams acting as single decision- with the additive error (n)
The following lemma, based on designing the optimal coupling between the actual and reference scenarios in Donmez et 2024 Lemma 5.5], plays an important role in bounding ||²])) from
above.
Lemma A.1. For some constants C, d, 2 O, we have
1 (m))..ll 11 = d'oor Vm
T.
(26)
By 24
Lemma A.l yields that we can bound the error @(m) from above by
(n+1)T-1
3k
(n)ll
Sc
Σ
fa n- dl (TT
(27)
B(m)
Due the no-recency-bie condition in 4.1 have 3K+1/Br S by the definition (17 Then, have monotonically decaying 3k, which yields
Bx
BUT
anT Ta
(28)
((n)
TB
By the assumption 4.1
we have
(n+1)f-1
lim
OuT
1 lim
II
O
(29)
12000
T0n++
T
x0000
ax+1
T
By
27
28
29
and the decaying property of ag, we obtain
lim lim (2) TI-P Vm T.
(30)
m On the other hand, E(m) is non-zero only for Independent Team-FP FP and corresponds to the difference between the stationary distributions for the classical and independent log-lineat learning. We can bound ((m) S A88, <) for some & based on (13). Hence, given any > O, there exists N such that
|,% (s, Vn 2 No.
(31)
where
for Team-FP C(E,T) = E - + A(,,)) for Independent Team-FP
(32)
Note that Cle, 7) can become arbitrarily close A(6,e) for sufficiently large T and small e, which chosen arbitrarily just for the analysis.
15

---

A.2
Step (ii) Convergence Analysis with Relaxed Errors
We focus on the convergence analysis of 23 based on the bound 31 To this end, we define the set-valued mapping
F(T) =
{ (bx, (6" (,,"" -T" + e*Th)
MET
|| C(e, T) and br, (1"" + cm A(AT) Vm)
(33)
for all π E II=== MET (AA)). Then, the update (23) and (31 yield that for sufficiently large n, the empirical averages (T()) I220 satisfy
"T 1]++1 - [(n) - (()) w(++1) E ((n) F(T()))
(34)
where the Martingale difference sequence (w(++)) } is as described in (22). We have set inclusion in (34 different from the equality in (23) since we relax the update rule by considering any error
satisfying the bound (31) rather than the actual error.
The following proposition shows that F(-) is a peculiar set-valued mapping. Particularly, given the sets X, Y from some underlying Euclidean space, say that set-valued function F(.) mapping each point Definition 2. and [Benaïm et al. 2005 Hypothesis 1.1]. if it satisfies the following conditions:
X to Fzx) C Y is Marchaud map, e.g., [Perkins and Leslie 2013
(i) F(-) is upper semi-contimuous, or equivalently, Graph( (F) = {(x, E F(x)) is closed subset of X Y.
(ii) For all X, the set F(x) is non-empty, compact, and convex subset of Y.
(iii) There exists c 0 such that supyeT(es) |Bll| s c(1 [|=l] for all X.
Proposition A.2. The set-valued functionn F() is a Marchaud map.
Next, we approximate the relaxation 34 by the differential inclusion
F(T).
(35)
Particularly, due to Proposition A.2 and 19 a linear interpolation of the iterative process {T()) 1=20 is perturbed solution to the differential inclusion (35) [Benaim et al.) [005 Proposition 1.4] and its limit set is internally chain transitive [Benaim et al. 2005 Theorem 3.6). Furthermore, we can characterize internally chain transitive sets (and therefore, the limit set of linear interpolations) through a Lyapunov function. Particularly, (T) be the set of solutions to (35) with initial point æ. We say that continuous function V -> TR a Lyapunov function of for a subset C II provided that
. V()) ((T) for all E II A. न E (T),
V(+) S (T) for all E A. (T)
Given such Lyapunov function for A, every chain internally chain transitive set L is contained in A if the set (V(r)= n E A} has empty interior [Benamm et al. 2005 Proposition 3.27]. Therfore, we propose the candidate Lyapunov function
V(=) max (0, L(T) -((,,,T))
(36)
where the auxiliary terms are defined by
L(x)= Σ MET "E
(1, +H(p)}
(37a)
E(A,E,T) = log AI XTT C(E, T) >0
(37b)
for some arbitrary to characterize the long-run behavior of (34) based on 335 Proposition A.3. The continuous function V(.) is a Lyapunov function of 335 for the set A ((T) 0}.
a
{
16

---

Proposition A.3 yields that there exists some sequence {Sa E R)220 such that Iskl + 0 as k 00 almost surely and
L(TT) < E(,, + + Vk.
(38)
By 3 we can write ((TR) as
[(Tx)=
Σ
max (WEA(AM)
(1, + [H(1)} 60 (TT))
MET
Σ
max MED(A.)
{0 Fx [)] 8² (TT))
(39)
MET
Since max, 16tH (p, π "}}
(T)
2 0 for all n, the inequalities
38
and 33 yield that
Σ MET (EE(AA))
max
(M, (mx) - E(A,e, + Ca Vm and n.
(40)
Recall that we can select >> 1, 0 and T E N arbitrarily. The definitions 332 and 37b) yield that E(A, e, 7) can be arbitrarily close to t log |A] for Team-FF and r log |A] +
2 A(6, e) for
Independent Te.m-FP. Furthermore, S& decays zero. Therefore, we can obtain 14 based (40) and Definition 2.4 This completes the proof.
B Proofs of Technical Results
The followings the proofs of Lemma A.1 and Propositions
A.2 and A.3 used in Appendix A
B.1 Proof' of Lemma A.1
We define two discrete-time processes from the beginning of epoch n. One of them the joint actions of teams in the reference scenario, defined as {wA Jæ2n == {(2k m (m) Ik2nT, and the other one is the joint actions of teams in the actual scenario, with {wk 1k2ur == ((ag) m (F()) (n)) I221I-
The transition probabilities between states depend only the beliefs the other teams. Therefore, for the reference scemario, during an epoch, this process a homogeneous MC. Let, P P(n).k be transition probabilities between the states, i.e., joint action profiles of all teams, of the original discrete-time process step k, and let P(n.) be the transition probabilities between the states for the reference scenario. Let (.)), be the distribution of Wj. and let J(n). be the distribution of wy. In this case, the expected joint actions of team at step within epoch in real and fictional scenarios can be expressed in terms of the distributions as:
(()
(w))
T (a)
Σ
((m). (w))
(41)
@:(a"==a
Step 1. In classical log- linear learning, at each step, only one agent can change their action and due to the soft-max nature this action change, any action has positive probability which bounded from below. This bound only depends on the minimum and maximum values of any agents' reward, which are defined by the game and bounded by definition, and the temperature parameter. we divide that bound by the number of agents in the team ([Zm)), )) we can obtain lower bound on the probability of changing to any joint action which can be reached within single step. Let's call that bound ç. Then, for any state w or W, there is a E 0 such that
(wlwk) TI + ((o))((WWR)) O,
(42)
(O)) > T = (()))( > 0.
(43)
Then, we can find a path with positive probability for both of these scenarios, where their state does not match until they reach n max, [. In other words, War-R Wer+n and wi: o Wy for all
==f, n...... nT + K with
TT A k=nI+h +1
Pr
We WNT
E and Pr
We WhT 2 _
(44)
k=n
17

---

Hence, satisfies the first condition for (Donmez et al.
2024 Lemma 2].
Step 2. For this part, introduce notation am E A where jm represents the jth agent from team m. For example. if all teams have identical agent numbers and agent indexes are ordered for teams, jm (m Tm + J- Also, let (()), (* ( and [(n).A = (T (m Now, consider the total variation distance between two transition probabilities. Transition probabilities between state w = {(a ( !mET and w = {("" a a=") JMET, where im can be any random
agent from team m with slight abuse of notation, can be written as function of the belief as
(T)))((())))) II (T())(((())))) mET
(45)
where p w-o is defined to be
exp
((0m (aim im im m
((n)).))) /T)
P W-
(To)..))
(46)
exp ( aa,a" im T(a)..)) /T)
arEAtm
and [(n), (T ()) Remember that due the separable structure of the network in ZSPTG
we have
6 (a im (a (T = -EE Egeane
mme im im )a)
(47a)
(ami (n)),
(47b)
Exm
where TInE is the matrix form of the potential function whose the joint actions of team m and columns are the joint actions of team f.
Now, for all Wk+1 which is reachable in one step from the state wk. i.e., at most single agent changes action each team, the relation between state transition probabilities and Pmaa() (-) can be expressed as
P(o), [Wk WaT) = II
we w22t (T()),
(48a)
MET
P(n)(wn-lWU)
II pm mET
W (T()..))
(48b)
Note that (()). a function of history of actions and the initial ((x)- Therefore, it can be computed given the past states (wk..... W(r))
We know that Pi (T (m))..) ). is bounded as it is a probability. Now, using the Lipschitz property of the soft-max function, and since oit (- (>xT)))))))) is a linear function of ((m).k we can say that pm (-) is Lipschitz continuous function with respect the input Also, using the fact that the product
(x)
of bounded Lipschitz continuous functions also Lipschitz continuous, and (48), we can say that there exists L < do such that the total variation distance between two transition probabilities is
bounded as
|| (P WxT) P. (wx)
SC > ( - #mow))
-
(49)
mET
The distance between the belief of the original scenario and the reference scenario can also be bounded thanks to the small step size. If we bound ||aß aflli < |aafli + || 2. Then using triangle inequality
N+NT-1 Σ
||(TENue - KnnwW), - =1(o/Oo) FooDN))
=
5
200
(50a)
(n+ 1yT -
<
20t
(50b)
!=n
< 2T TT
(50c)
18

---

1 2T
Let's consider late epochs where aur Then.
and set TT OnT = 1- -xe with 0 < Anr S 1.
|| (To)l)) Pw- (
(m),k)
S AnT-
(51)
TV
Hence, the second condition of [Donmez et al. 2024 Lemma 2] is met, and we can invoke the corresponding Lemma such that given the distributions of the original and reference scenarios, following inequality holds for all nT..
+ + XxT) 1+E
[|mm)) (()),lll, s 2(1
(52)
where E = = If we define constants, = 2( -1--)) p (1==(1= (1 c) d 1 -
+
and assume that the reference scenario initial distribution the stationary distribution of the MC, P(n). we can rewrite the inequality (52) as follows
((a),. - Aoo), S c. p pt + d. T.TaTT:
(53)
for all & 2 nT. Then, by (1)). and triangle inequality, we can conclude
[ (()). S $|((n), * S c. /² k-nT + + TawT,
(54)
for all k 2 nT.
B.2
Proof of Proposition A.2
The set IT is compact set by definition as it is a Cartesian product of probability simplexes. Let's consider convergent sequence (#n; Pn *х + 6n)a=1,2.... in the set {mm,,y): y F(z)), and let (T², - xt + e²) )) be the point that the sequence converges to. Given Tu. any H4, is fixed and unique value, and it is element of the compact set that is generated by mapping probability simplex with the continuous soft-max function. Then, for any π E II, nr¹ π is a fixed value within another e
compact set. Furthermore, the error term must remain within the compact (T (m",," 1=" x* e*) is also within the set ((Tn: y) y F(z)), and F II ALmET I1"ll is closed-set valued map. Therefore, the condition (1 is satisfied. Given a π E II, us E IT a fixed value corresponding to the smoothed best responses π ((mET). Hence, μ., π fixed value for
As a result,
given Note that each em E is a non-empty, bounded, closed and convex subset of Em IlA" Therefore, for any given æ E II, F(T) μ4+ x + e is a non-empty, compact, convex subset of R2METT A"Il As a result, ii is also satisfied. The function F is bounded such that
T + -1-p+k~(0))
1
sup |1a/la < sup yEF(a)
#EA |+||
sup ||||] + sup ||e|| < 2M M WEA
(55)
Hence, it satisfies the condition
m1
Since all three conditions satisfied, F is Marchaud Map.
B.3 Proof of Proposition
A.3
The smoothness the entropy regularization in I1 yields that we can invoke the envelope theorem to compute the time derivative of L(T)
d
de [())=
= Σ
TThh (NIT, =-)
(56)
MET (a)
me (1. #t),,
(57)
mET E3fm
where n. = -br (-)x)))) and (a) follows from ef, Therefore, we can write (57) as
By and (33) we have 4º j1f x²
dt L(T)= = 11=>>
d
Σ
mm (1+ p.) - mmee(pp) x²)+ + Σ ma (58)
MET Em
E
19

---

For the first two terms on the right-hand side, we have
Σ E MET E#m
( pb) smme (1 (0),,)))) m²²)
Σ MET
Th (1 m-),
(59)
((x) ) + log |A|.
(60)
where (a) is due to 33 and 44) and (b) follows from the definition the other hand, we have
337a) H(P S log |A On
ΣEΣ MET Ef
e (a
Σ
|e 6 S 17 C(E,T),
(61)
MET Exm
due to the bound on the errors. By the bounds (60 and (61), we can bound the time derivative of L(TT) as
d -(mm)<L ET) +
d dt
dt
-(m) ET),,,TT)
(62)
where the constant E(A, E) > 0 is as described in (37b). Since we have V(=) = min ((T)
TT), the strict inequality in 62 yields that V (-.) a Lyapunov function.
C Extension to Multi-team Markov Games
Markov games (MGs), introduced by Shapley [953], generalizes Markov decision processes (MDPs) to non-cooperative multi-agent environments. We can characterize a multi-team MG by the tuple (H, S, (A', (ez,,, p. Po). where H the horizon length, T and I again denote the index sets for the teams and agents, S denotes the finite set of states, and A' and r' S x A I denote, resp., the agent i's finite action set and immediate reward function, depending on current state and joint actions The state of the underlying game change according to the transition kernel p(- a). depending on the current state and joint actions, and the initial state is determined by the probability distribution Po E ASS).
Let each team m randomize their actions contingent on the current state E S and stage h E [H] = {1...., H} via a stationary strategy x" S x [H] + A(A"). Note that team members do not necessarily randomize their actions independently. Given the strategy profile of teams πI = (zz") meT.
agent i's utility function is defined by
H
U ('))= =
Σ r' (sk)
where the pair (s,,0)) an) denotes the state and action profile at stage h, the expectation is taken with respect to the randomness on these pairs (sh, an) induced by the strategy profile I and the underlying transition kerel..
For each state s, let the reward functions {r her induce a ZSPTG where team m E T has the potential function "'" (s,-) A + R satisfying (2. (3) and 44. Correspondingly. team m 's utility
function is given by
E
Σ
TT (sh an)
(64)
Let II"h := fTTm TTm S H + )(A))} denote the set of stationary strategy profiles for team m. Then, the following the counterpart of Definition ff44i multi-team MGes. Definition C.1 (Team-Nash Gap for MGs). Given the strategy profile of teams (xT E II TMeT, "I we define the team-Nash gap for team m as
TNG T
max
II, I "
(65)
REll
and (NG( =
{ Jemm- Correspondingly, say that the strategy profile of teams JMET is e-NE TNG (T) S.
EEE TNG (I), where
'The results can be generalized state-varian action rather straightforwardly
20

---

Next, we describe the stage-gang -ame framework, going back to the introduction of Markov games (Shapley [953]. and also used in multi-agent reinforcement learning algorithms such as Minimax-Q Q [Littman 1994] and Nash-Q [Hu and Wellman 2003]. Particularly, at each stage, the agents' joint actions determine the immediate rewards they receive and the next state, and therefore, the future rewards to be received. Given the strategy profile of teams = (xr"") mEET let v² S x [H] + IR denote agent i's value function such that (s) is the game value for the stage game associated with
state Similarly, let Q' : S x [H] A + IR be the Q-finction such that (* -) corresponds to agent i's payoff function for the stage game associated with state 8. The definition of the utility (63) yields that
Q'(s,h,a)= = r' (s) q
Σ p(s+ u 1), #4
(s4,
(66a)
('(s,)) - Q' h, I(s,
(66b)
The Q-function and value function implicitly depend on π.
We can extend Team-FP dynamics to MGs played repeatedly. Based on the stage game framework, agents play some stage game associated with each state E and stage h E [H] pair repeatedly whenever the underlying MG visits state at stage h. Correspondingly, agents form beliefs about the opponent teams as the opponent teams play according to some stationary strategies across these repetitions.
Let of S x [H] + A(A*) denote the beliefs formed by agents i E I about team l, and Qi : S x [H] x A-ABB denote agent i's Q-function estimate at the kth repetition. If the underlying MG visits state s at stage h at the kth repetition, then agent follows Team-FP dynamics if the payoff function is the Q-function estimate (s, h, -). Agents also recall the previous actions of their teams specific to each stage game. We denote agent i's previous action for the pair of state s and stage until and including the kith repetition by of (s, h) E A', with slight abuse of notation. Then, at stage k, agent i E I" either takes the previous action for that stage game (i.c., aj (s, h) = - h)), or takes the action a² It) br, (Q) /, - (s. h), II " (s, h))) according to the smoothed best response to the previous actions of the team members (-) = (o)=e0)) and the beliefs I m = { Jeym formed about other teams.
Let 8t,A and a denote, resp., the state and action profile stage h at the kth repetition. Then, given some reference step size {aele20. agents j & I" update their beliefs about team m's strategy according to
-ズ (s, h) = x (s, h) + (s (a h) Th (s,h)) Vk- = 0, 1...,
(67a)
A (8, h) = 1(0=ss,
(67b)
where (-) = {at (-)liez and (s, h) is the number of times state get visited at stage h until the kith repetition. Correspondingly, by (66), each agent I updates their Q-function estimates according to
Qs+1(s, h, = Qi(s, + Ae a) (oh(o, (,,,
(68)
where du (s, h a) E [0, 1] is also some step size. If the agent i knows the model of the underlying MG, then have
= - + [(ns) Σ p(5+ a) vf h + 1),
(69a)
E5
an [To=ss, (cc2))))))
(69b)
for all a E A and
( h) = (s.,),, (s,,) ((s,h) (M].
(70)
If the agent does not know the model, then we have
( h, = Th. 4 1), Ae a) = ()
(7la)
(71b)
21

---

Algorithm 2 Model-based (Independent) Team-FF for MGs
1: initialize: {mn(-) {a- (-)) (ez and Qb(+) arbitrarily for the typical agent E T
2: for each repetition 0, 1, do
3: for each stage h = 1,.... do
4: require: (-)mmmm. {ak (0) and (+)
5: observe current state
6: set 8 h)
7:
play (3) ~ br, (or independently) simultaneously with other agents
or a - (5) in a coordinated way
8:
observe 0R,k ag (8)
9: receive hi. = (8,
end for
-i 1H 11: require: trajectory {5
set
() Q2 h, (s, (s, II (s,
(x(,,,,)) +[NCH
Σ
p(s+ 8,,) o4(04, 1)
#46S
13:
update the beliefs and the Q-functions
(8,1(,,)) h) = (s. h) + ( (sf(s,)) (s(),))) vC
Qi+1(s, = (s, h [ee= (ex(MM (2h(s, - (b(s,
for all (s, E S [H]
14: end for
where rf denotes the reward received at stage h at the kth repetition and we approximate the expected continuation payoff by looking one stage ahead, as in the classical Q-learming algorithm, and e& (s, h. a) is the number of times the pair (s, a) gets realized at stage until the kth repetition. Algorithms and provide descriptions of the extensions, resp., for the model-based and model-free cases from the perspective of the typical agent E I from the typical team m E
Remark C.2. Algorithm reduces Algorithm 1 if |S| = and H
D Large-scale Numerical Examples
In this section, we give a large-scale experiment that show the scalability of Team-FP in networked game where only 2-hop neighbors of agents affect their payoff' function. We simulated three- team game with nine agents per team, resulting in large joint action space of size 2d7. After ten independent trials, we plotted the evolution of the Team-Nash Gap Figure Despite the problem's scale, the empirical averages of team actions converge to the Team-Nar -ash equilibrium at similar rate, even with sparse network interconnections, as shown the top right of Figure
22

---

Algorithm 3 Model-free (Independent) Team-FF for MGs
1: initialize: {mn(-) {a- (e) and Q+(+) arbitrarily for the typical agent E T
2: for each repetition k 0, 1, do
3: for each stage h = 1..... do
4: require: (-)mmmm. {ak (-))eemm and (+)
5: observe current state
6: set 8 h)
7:
play (3) ~ br, (3), (or independently) simultaneously with other agents
or a - 15) in a coordinated way
8:
observe 0R,k 0x (8)
9: receive Ti. = r',
end for
11: require: trajectory (8h,R, dn Th,k 18
set
()
(s, h), Ix (s,
(t, = -h, + [op +
13:
update the beliefs and the Q-functions
Ii,1(s,)) (s, h) + [( (0)(), h) (s,) vl v m
(s, Q (s. a) +
(8, h, Qi(s,h,a)
for all (s,1, E [ x A 14: end for
Team-FP
Team-FF Large Scale
3.5
3.0
Team Nosh Gop
2.5
2.0
1.
0.5
0.0
102
10a
10a
105
10
Number Rerations
Figure 7: The evolution of Team Nash Gap in the large-scale example provided in the top right, showing that Team-FP dynamics reach near ream-minimax equilibrium.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction is clear about the claims of the paper.
Guidelines:
23

---

The answer NA means that the abstract and introduction do not include the claims
made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed at the end before the conclusion.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairnsss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
For all the theorems, we provide assumptions and proof' along with proper references.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch provide intuition.
24

---

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Although there are some randomness in the generation, and we did not save the seeds, we share the code, and it should work as in the simulations for any other randomly generated games. Therefore, anyone can reproduce the results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recogmize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We share code for experiments. However, the code may be too complex to understand as most of the documentation for the code is missing.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
25

---

reproducible, they While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7
gwwdes Cttp///oeeeSeisseeeeeeeeeee www.iissssoooiiiioii. for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We specify experimental settings of each experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We include mean and standard deviation of independent trials.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma emror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
26

---

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: We give specifications the computer and rough run-time the algorithms. However, we do not provide exact run-times required time for preliminary experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:///waaurr.ccccoooo/////aaaa
Answer:
[Yes]
Justification:
We discuss how our work does not raise any ethical concerns in the conclusion
section.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no social impact of the work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
27

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not contain risks
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification:
We do not use existing assets
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, saperswwthcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: There new assets.
28

---

Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the datasct/codc/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
There are no human subjects or potential risks.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
There are no human subjects or potential risks.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "VQiWPn8ucC", "reviewer": "Reviewer_Jh6E", "summary": "In this work, the authors introduce a novel variant of virtual play, referred to as **Team-Fictitious Play (Team-FP)**, aimed at assisting self-interested agents within teams to reach **Team Nash Equilibrium (TNE)** in multi-team games. The paper focuses on zero-sum potential team games (ZSPTGs), where teams interact pairwise, but the payoffs to team members are not necessarily identical. The main contributions include the introduction of inertia in action updates and responses to the last actions of team members, which are crucial for team coordination. The authors provide theoretical convergence guarantees and validate the efficacy of the approach through extensive simulations.", "questions": "All my questions are written inside the weakness part.", "limitations": "The paper explicitly discusses its limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The introduction of Team-FP fills a gap in the multi-team game theory literature, particularly in the context of zero-sum potential team games.\n2. The paper offers rigorous theoretical analysis and practical insights, including convergence proofs and error bounds.\n3. Extensive simulations compare Team-FP with other algorithms, demonstrating its effectiveness and exploring the impact of various parameters on convergence speed.\n4. The grammar and expression are accurate and professional.", "weaknesses": "- Adding more background or appendix sections on Nash equilibrium and related solution algorithms would greatly help the reader's understanding.\n- The theoretical analysis section is quite technical; including more intuitive explanations and illustrations could be beneficial. For instance, adding a diagram in Section 3 to visually depict the workflow of Team-FP might help.\n- A more detailed analysis of the parameters used in Team-FP and their sensitivity to performance could provide deeper insights.\n- While the paper focuses on zero-sum potential team games, discussing how Team-FP could be adapted or extended to other types of multi-team games would be valuable.\n- The numerical experiments section could benefit from more qualitative analysis.\n- The discussion on practical applications is not sufficiently thorough; adding more experiments and analysis in specific application scenarios would be beneficial.\n- Section 5 could include experimental comparisons with other multi-team learning algorithms.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s thorough review and constructive comments. We can gladly include further clarifications on the points raised, which will improve our paper’s accessibility. In the following, we address the reviewer’s questions:\n\n**[How can we visualize the learning dynamic and the solution approach?]** We have included Figure 1 to visually depict the Team-FP dynamics and the fundamental idea in our technical analysis in the PDF attached to the global response.\n\n**[How can we extend Team-FP to other classes of games?]** Similar to the FP dynamics, Team-FP dynamics are **game agnostic**. In other words, agents do not know whether the other teams have aligned or misaligned objectives. Furthermore, agents do not know whether other team members have identical or different yet aligned objectives. As discussed in the conclusion section, various essential classes of games have the fictitious play property. The generalization of our approach to such multi-team games is an interesting future research direction. In the illustrative examples section, we have discussed several other types of multi-team scenarios beyond zero-sum potential team games, such as **$2\\times N$ general-sum game** and **potential-of-potentials game**. As observed in these examples, we expect the almost-sure convergence of Team-FP for the games where FP converges if teams were single agents since in the Team-FP dynamics, team members can learn to coordinate in the (evolving) best team response and can act as if they are a single decision-maker.\n\n**[How sensitive is the performance to the parameters used?]** In the illustrative examples section, we have numerically examined and discussed the parameters $\\tau$ and $\\delta$. Our results indicate that decreasing $\\tau$ (i.e., reducing exploration) leads to a lower Team-Nash Gap at the steady state. However, exploration plays an essential role in escaping suboptimal team responses. Increasing $\\delta$ (or reducing friction) can accelerate convergence. However, friction in the action updates is critical for reaching the optimal team response.\n\n**[How does Team-FP perform in practical applications?]** The reviewer raises a significant concern. In the rebuttal period, we have implemented a real-life application scenario to address this question. To this end, we generalized the widely studied **airport security scenario** to multi-team games. Attackers have aligned objectives due to their adversarial nature. Attackers are autonomous decision-makers with different objectives. Though their objectives can differ to a certain extent, they are aligned in maximizing the defender’s cost due to their adversarial nature. This makes our multi-team formulation an ideal model for such cases. We have described the game and presented the numerical results in the global response and its supplementary PDF. Our analysis shows that **defending our systems according to the worst-case as if the attackers are coordinated centrally is important since the attackers can learn such coordination based on simple behavioral rules consistent with their self-interests, as in the Team-FP dynamics.**\n\n**[How does Team-FP perform compared to other multi-team learning dynamics?]** The existing results on multi-team games are primarily computational. On the other hand, in this paper, we present simple behavioral rules (slight adjustments of the widely studied fictitious play dynamics) that can reach Team-NE to **strengthen the predictive power of Team-NE and justify the algorithms developed to compute Team-NE efficiently** for multi-team games, as exemplified in our airport security scenario. Furthermore, their simplicity makes the dynamics scalable for **large-scale problems**, as the global response discusses. To our knowledge, Team-FP is the first multi-team learning dynamic that does not rely on any (ex-ante) communication among team members. Notably, the Fictitious Team Play (FTP) algorithm, presented by [Farina et al., \"*Ex ante* coordination and collusion in zero-sum multi-player extensive-form games,\" In NeurIPS, 2018], is related, and we have compared FTP and Team-FP dynamics in the global response and its supplementary PDF.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "f5PWFGZByb", "reviewer": "Reviewer_Rftf", "summary": "This paper introduces Team-Fictitious Play (Team-FP) dynamics as a novel approach for teams of self-interested agents to converge to Team-Nash equilibrium in multi-team games. The study focuses on games where multiple teams interact strategically, aiming to maximize their collective utilities. For this purpose, this paper (1) introduces Team-FP as a method for teams to converge to Team-Nash equilibrium in multi-team games; (2) extends the convergence analysis of Team-FP dynamics to multi-team Markov games; (3) demonstrates the effectiveness of Team-FP dynamics through theoretical analysis and empirical evaluations in various multi-team game scenarios.", "questions": "The authors provide some experiments, but it would be helpful to see more extensive experiments, including a larger-scale multi-team game to demonstrate the effectiveness of the algorithm in a more complex setting.", "limitations": "The authors have adequately addressed the limitations", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The introduction of Team-FP represents a novel approach to address the challenge of teams reaching equilibrium in multi-team games.\n2. The detailed numerical analysis and simulations conducted to evaluate the behavior of Team-FP dynamics in various multi-team game settings reflect the thoroughness and quality of the method.", "weaknesses": "1. The notations are confusing and it's hard to keep up. For example, 'agent index' and 'team index' both use lowercase letters. \n2. Limited discussion on computational complexity. The paper could provide more insights into the computational complexity of implementing Team-FP dynamics in large-scale multi-team games. Discussing the scalability of the approach, potential bottlenecks, and computational efficiency considerations would be beneficial for understanding the practical feasibility of deploying Team-FP in complex settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough reading of our paper and constructive comments. We appreciate the feedback regarding the notation. We have reserved uppercase letters to denote sets; e.g., $A^1$ denotes agent 1’s action set. For the explicit exposition, we underlined the parameters related to teams, e.g., $\\underline{a}^1$ (and $\\underline{\\pi}^1$) denotes the joint team action (and team strategy) of team 1 while $a^1$ (and $\\pi^1$) denotes agent 1’s local action (and local strategy). \n\nSince the Team-FP dynamics have linear updates, they are **scalable, especially for sparse networked interconnections**, as discussed in Remark 3.1 of the paper. For example, if at most 2-hop neighbors of the agents affect their reward (as in Example 2.1 of the paper), the computational complexity grows linearly with the number of teams and the maximum number of agents in a team. Furthermore, the complexity grows exponentially with the number of 2-hop neighbors of agents, where the base of the exponent is the number of local actions. \n\nTo illustrate the scalability of the Team-FP dynamics, we have conducted experiments on a **large-scale game** as requested by the reviewer and described its details and the numerical results in the global response and its supplementary PDF. We also highlight that the global response includes the **practical application of an airport security scenario** modeled as a multi-team game.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1HcPYvZbIj", "reviewer": "Reviewer_te9F", "summary": "This paper addresses the complex problem of multi-team games by introducing a new variant of fictitious play called Team-FP. The method aims to enable teams of self-interested agents to reach Team-Nash Equilibrium (TNE) in multi-team games, with a particular focus on zero-sum potential team games (ZSPTGs). The authors present a rigorous convergence analysis and extend the Team-FP dynamics to multi-team Markov games. Extensive simulations are provided, comparing Team-FP with other algorithms to demonstrate its effectiveness and practical applicability.", "questions": "1. How can we get the Eq.4?\n\n2. Can you provide experimental evidence to support your convergence proof?", "limitations": "n/a", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper introduces Team-FP, a novel variant of fictitious play specifically designed for multi-team games. The approach incorporates inertia in action updates and agents’ responses to the last actions of team members, which enhances team coordination. This creative extension of classical fictitious play is valuable in advancing the understanding of team dynamics in multi-agent settings.\n\nThe paper is well-structured and organized, making it easy to follow the progression from the introduction of the problem to the presentation of the proposed methods.", "weaknesses": "Despite the novelty of the Team-FP approach, the modifications to classical fictitious play are not significantly groundbreaking. The methods, while creative, represent incremental improvements rather than major innovations in the field.\n\nI am not familiar with the convergence proof; thus, I cannot and do not have the time to verify the proof. The authors put a lot of effort into the theoretical part. The experiment setting and results are simple. For a theoretical paper, it is better to prove the convergence through experimental results.\n\nThus, I think the paper is below the acceptance bar of NeurIPS.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough and constructive comments. Firstly, we want to highlight that the equation (4) is inherited from polymatrix games. For example, the equation always holds for two-team games. As discussed in the introduction, the polymatrix structure (or network separable interactions) is essential for generalizing two-team zero-sum games to more than two teams. \n\n**[Novelty of the Team-FP]** We see this *slight* adjustment of the fictitious play dynamics in the direction of log-linear learning as a *strength*, justifying it as a simple behavioral rule consistent with the agents’ autonomous adaptations based on their self-interests. Notably, both dynamics have been studied extensively to model the interactions of human-like self-interested learners in the learning-in-games and behavioral-game-theory literature. Therefore, by showing the convergence of Team-FP dynamics to Team-NE, we provide **behavioral support for the predictive power of Team-NE** in multi-team environments (with uncoordinated team members), as exemplified in our **airport security scenario** in the global response. These convergence results also **justify the design of sophisticated equilibrium computation methods** to predict the outcome of such multi-team interactions. Furthermore, the simplicity of the dynamics ensures its **scalability for large-scale games** with networked interconnections, as discussed in the global response and its supplementary PDF.\n\n**[Experimental Support for Convergence]** Our extensive numerical simulations show convergence to Team-NE approximately as the Team-Nash Gap decays to near zero. The inherent exploration in softmax response induces the approximation error, and such occasional deviations from the greedy best response play an important role in escaping suboptimal team responses.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Pzb006amSs", "reviewer": "Reviewer_LmHf", "summary": "This paper introduces a new variant of fictitious play where agents respond to the last actions of team members. The authors need to improve their academic writing skills largely. The expression between paragraphs and sentences lacks logic and consistency. Moreover, the paper does not list the challenges and gaps in current research and why this research can solve the recent issue. Furthermore, from my point of view, I did not see particular novelty or practical value in this research.", "questions": "I suggest the author reorganize the paper and improve the academic writing skills.", "limitations": "I suggest the author reorganize the paper and improve the academic writing skills.", "rating": 3, "confidence": 4, "soundness": 1, "presentation": 1, "contribution": 1, "strengths": "This paper introduces a new variant of fictitious play where agents respond to the last actions of team members.", "weaknesses": "The authors need to improve their academic writing skills largely. The expression between paragraphs and sentences lacks logic and consistency. Moreover, the paper does not list the challenges and gaps in current research and why this research can solve the recent issue. Furthermore, from my point of view, I did not see particular novelty or practical value in this research.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We disagree with the reviewer’s claim that expressions between paragraphs and sentences lack logic and consistency except possibly for very few instances. We would appreciate it if the reviewer could explicitly refer to any specific examples so that we can address them.\n\nThe first two paragraphs of the introduction highlight the challenges and the gaps in the literature and how this paper can solve these issues. \n\n**What are the challenges and gaps in the current research?**\n* There are **no results** addressing whether Team-Nash equilibrium (TNE) can predict the outcome of interactions among multiple teams of autonomous decision-makers in non-cooperative environments despite the recent interest in multi-team games and the efficient computation of TNE. \n* Joint team responses can require the correlation of the responses among team members. Therefore, the widely studied fictitious play dynamics do not necessarily reach TNE even in two-team zero-sum games. There are also **no results** achieving such correlation without communication or ex-ante coordination for multi-team games.\n\n**Why can this research address this issue?**\n* We **present a slight adjustment** of the fictitious play dynamics in the direction of log-linear learning, another widely studied dynamics. This adjustment ensures that the dynamics involve simple behavioral rules consistent with the autonomous agents’ adaptation according to their self-interests, and team members can still learn to coordinate in the best team response against other teams without communication or ex-ante coordination.\n* We **prove the almost-sure convergence** to TNE in a broad class of multi-team games, including the two-team zero-sum and multi-agent zero-sum polymatrix games as special cases.\n\n**What are the practical novelty or value of this research?**\n* Our results **strengthen the predictive power of TNE** for multi-team games, **justify the recent interest** in efficient computation of TNE for multi-team games, and **provide a theoretical foundation** to design/regulate multi-team games under TNE. \n* We have exemplified the practical impact of the results on an **airport security scenario** in the global response. The example shows that designing airport security for the worst-case of the team-minimax equilibrium against autonomous attackers that do not coordinate is not too conservative since the attackers can learn to coordinate in the worst response by following simple behavioral rules, as in the Team-FP dynamics. \n* Due to the **scalability** of the Team-FP dynamics, teams of agents can also follow the Team-FP dynamics to learn the coordination in the best team response against other teams without the burden of communication and ex-ante coordination, especially in **large-scale systems with networked interconnections**. The global response includes such a large-scale numerical example.\n\nRegarding the ethics flag, we have **only used randomly generated games** in our paper without external data and provided our codes as supplementary material. There cannot be ethical issues regarding data representativeness and quality.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their valuable time and constructive comments.

Based on Reviewer Jh6E’s request, we have illustrated the Team-FP dynamics and the fundamental idea of the proof in Figure 1 of the attached PDF. Furthermore, based on the other comments, we have conducted several new experiments despite the limited time and computational resources. We have also provided the results of these experiments in the PDF. In the following, we describe these experiments in more detail.

**[A Practical Application]** We can model an **airport security scenario** as a two-team game between defender and attacker teams. Note that airport security has been studied from a game-theoretical lens extensively, and the findings have been **deployed in real life**, e.g., see [An et al., “PROTECT - A deployed game theoretic system for strategic security allocation for the United States Coast Guard,” AI Magazine, 2013]. In our example, we consider a single security chief in the defender team and three different intruders in the attacker team, as illustrated on the left-hand side of Figure 2 of the PDF. The chief can defend a gate at the expense of some cost. Intruders autonomously decide whether to attack a specific gate or remain idle. The intruders receive positive (or negative) payoffs if they attack an undefended (or defended) gate. Correspondingly, the chief gets a positive (or negative) payoff if intruders attack defended (or undefended) gates. The security chief has $2^6=64$ actions, while each intruder has $6+1=7$ actions. We have conducted $50$ independent trials and presented the evolution of the average of the Team-Nash Gap along with the standard deviations as shaded areas on the right-hand side of Figure 2. From a higher level, this example shows that **team-minimax equilibrium can predict the outcome of airport security games for the likely scenario of different uncoordinated attackers. It also justifies the algorithms developed to compute team-minimax equilibrium efficiently.** 

**[A Large-Scale Example]** We have simulated a three-team game with nine agents per team and $27$ agents. Note that this configuration has a huge joint action space of $2^{27}$ action profiles. We have conducted ten independent trials. We have plotted the evolution of the Team-Nash Gap in Figure 3 of the PDF. Despite the scale of the problem, our experiments show that the empirical averages of the team actions converge to the Team-Nash equilibrium **at a comparable rate when the networked interconnections are sparse**, as depicted in the top right of Figure 3. 

**[A Comparison with Other Multi-team Learning Dynamics]** The existing learning dynamics for multi-team games are very limited as the literature mainly focuses on efficient computation of equilibrium rather than identifying whether equilibrium can **emerge** as an outcome of self-interested adaptations. **One exception** is the fictitious team play in [Farina et al., "*Ex ante* coordination and collusion in zero-sum multi-player extensive-form games," In NeurIPS, 2018]. In our setup, their fictitious team play dynamics (developed for extensive-form games) become the classical fictitious play dynamics where the entire team acts as a single decision-maker due to the ex-ante coordination among team members. Our paper examined this scenario in Figure 2(a) as a full coordination setup. In Figure 4 of the PDF, we have replotted this scenario to highlight the comparison.  Although the fictitious team play reaches equilibrium faster due to the ex-ante coordination, such coordination brings in additional burdens beyond time. In contrast, Team-FP dynamics are scalable without such coordination burden, as shown in the large-scale example above. Figure 2-(a) in the paper also shows the trade-off between coordination burden and learning speed by including the half-coordination scenario.

Meta-review: The paper received 4 reviews, 3 of which were detailed and provided useful comments. These reviews appreciate the gap that this paper fills in the literature by introducing the team-fictitious play algorithm and proving its theoretical convergence properties in team zero-sum potential games. The theoretical results are substantiated by numerical experiments (partially in large games) some of which were provided during the rebuttal by the authors. There was an additional ethics review which raised some concerns which are standard to this type of research and which can be addressed by the inclusion of an appropriate discussion by the authors (as already done in their response to this review).

Based on the 3 quality reviews and the ethics review, and the authors-reviewers and reviewers-reviewers discussions, my recommendation is to accept this paper as a poster. The reason is that the paper presents a novel contribution to the learning in games literature and provides enough evidence to substantiate the importance of the proposed algorithm as a simple (and, thus, scalable) extension of fictitious play in team-games. Nevertheless, this remains a borderline decision since the scope and the contribution of the paper is potentially limited by the theoretical nature of the settings that it studies. 

For the preparation of the camera-ready version, the reviewing team trusts that the authors will include the ethics discussion as promised in their response and the additional illustrations and experiments presented in their attached document. Also, the reviewing team encourages the author to acknowledge the other important comments and limitations raised in the discussions with reviewers.
Decision: Accept (poster)