SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In words, this proposition guarantees that learning dynamics that lead to NE for a specific game continue to do so for nearby games.
4.3 Stabilization through a single higher-order player
The previous section's analysis allowed all players to utilize higher-order learning. In some cases, it may not be necessary for all players to utilize higher-order learning. In this section, we present sufficient conditions under which a single player using higher-order gradient play with the remainder utilizing fixed order gradient play can still lead to NE.
Assumption 4.1.
A. Let  $(w, \lambda)$  be a left eigenvalue pair of  $\mathcal{M}$ , i.e.,
$$
w ^ {\mathrm {T}} \mathcal {M} = \lambda w ^ {\mathrm {T}},
$$
with  $\mathbf{Re}[\lambda] \geq 0$  and  $w^{\mathrm{T}} = \left(w_{1}^{\mathrm{T}} \ldots w_{n}^{\mathrm{T}}\right)$  partitioned consistently with (8). Then  $w_{i} \neq 0$  for all  $i$ .
B. Let  $(v, \lambda)$  be a right eigenvalue pair of  $\mathcal{M}$ , i.e.,
$$
\mathcal {M} v = \lambda v,
$$
with  $\mathbf{Re}[\lambda] \geq 0$  and  $v = \left(v_{1}^{\mathrm{T}} \ldots v_{n}^{\mathrm{T}}\right)^{\mathrm{T}}$  partitioned consistently with (8). Then  $v_{i} \neq 0$  for all  $i$ .
Recall the definitions of  $\mathcal{A}$ ,  $\mathcal{B}_i$ , and  $\mathcal{C}_i$  from (14).
Proposition 4.3. Let  $\mathcal{M}$  be non-singular and satisfy Assumption 4.1. Then for any  $i$ , the pair  $(\mathcal{A},\mathcal{B}_i)$  is stabilizable and the pair  $(\mathcal{A},\mathcal{C}_i)$  is detectable.
As a consequence of Proposition 4.3, it is possible for a completely mixed-strategy NE to be stabilized where a single player utilizes higher-order gradient play with the remaining players utilizing fixed order gradient play.
5 Non-convergence to NE in higher-order gradient play
We now show that linear higher-order gradient play dynamics need not lead to NE. Given any such dynamics, we construct a game with a unique NE that is unstable under given dynamics.
5.1 The Jordan anti-coordination game
The Jordan anti-coordination game, introduced in (36), was used in (17) to prove that fixed order uncoupled learning dynamics do not lead to NE. The game consists of three players with
$$
u _ {1} (x _ {1}, x _ {2}) = x _ {1} ^ {\mathrm {T}} \left( \begin{array}{c c} 0 & 1 \\ 1 & 0 \end{array} \right) x _ {2}, \quad u _ {2} (x _ {2}, x _ {3}) = x _ {2} ^ {\mathrm {T}} \left( \begin{array}{c c} 0 & 1 \\ 1 & 0 \end{array} \right) x _ {3}, \quad u _ {3} (x _ {3}, x _ {1}) = x _ {3} ^ {\mathrm {T}} \left( \begin{array}{c c} 0 & 1 \\ 1 & 0 \end{array} \right) x _ {1},
$$
and a unique mixed-strategy NE at  $x_1^* = x_2^* = x_3^* = \left(\frac{1}{2}, \frac{1}{2}\right)^{\mathrm{T}}$ . We will let  $\Gamma(\mu)$  denote the Jordan anti-coordination game but with the utility function of player 1 modified to  $u_1(x_1, x_2) = x_1^{\mathrm{T}}(\mu M_{12})x_2$ , where  $\mu \in \mathbb{R}_+$ . Since scaling payoffs does not change the nature of the game,  $\Gamma(\mu)$  has the same unique NE as  $\Gamma(1)$ .
5.2 Destabilization using rescaled anti-coordination
Suppose all three players use variants of linear higher-order gradient play in  $\Gamma(\mu)$ . As before, we denote the higher-order dynamics of player  $i$  as the linear dynamical system (9). To study the local behavior of the dynamics around the unique mixed-strategy NE of  $\Gamma(\mu)$ , we define  $w_1(t)$ ,  $w_2(t)$ , and  $w_3(t)$  as in (5). Then, we analyze the local stability of the mixed-strategy NE through (10).
The following lemma will be essential in proving our main result.
Lemma 5.1. Let  $A \in \mathbb{R}^{n \times n}$ ,  $B \in \mathbb{R}^{n \times 1}$  and  $C \in \mathbb{R}^{1 \times n}$ . If  $CB = CAB = 0$ , and  $CA^m B \neq 0$  for some  $m \geq 2$ . Then for sufficiently large  $\mu > 0$ ,  $A - \mu BC$  is not a stability matrix.
The proof of Lemma 5.1 is presented in Appendix B.2 and it uses root-locus arguments (see references (31; 30)).
Proposition 5.1. If linear higher-order gradient play dynamics are locally exponentially stable at the unique  $NE$  of  $\Gamma(1)$ , then there exists  $\mu > 0$  such that the unique  $NE$  of  $\Gamma(\mu)$  is unstable under such dynamics.
The proof of Proposition 5.1 is presented in Appendix B.3. The main idea of the proof is to write the local dynamics matrix in the form  $A - \mu BC$ , and then use Lemma 5.1. We also provide a similar proof for sufficiently small  $\mu$  in Appendix C.
The results might be puzzling because, for all  $\mu > 0$ , all games  $\Gamma(\mu)$  are strategically equivalent. Convergence guarantees for learning dynamics are usually established amongst classes of games. Thus, it is generally expected that dynamics will behave similarly for all games in a particular class. In this case, we design linear learning dynamics that are affected by simple rescaling of the payoff matrices.
6 Strong stabilization of mixed-strategy NE
Results from Section 4 imply that the mixed-strategy NE in a two-player  $2 \times 2$  (identical-interest) coordination game can be stabilized. Here, we argue why dynamics that stabilize this mixed-strategy equilibrium are not reasonable. Specifically, we show that such dynamics must be inherently unstable as an open system, i.e., as dynamics that respond to an exogenous payoff stream, and this instability is problematic with respect to such payoffs.
First, we inspect which type of mixed-strategy NE requires unstable learning dynamics for stabilization. For this purpose, consider the system in (12) for  $n = k_{1} = k_{2} = 2$ :
$$
\mathcal {A} = \left( \begin{array}{c c c c} 0 & m _ {1 2} & 0 & 0 \\ m _ {2 1} & 0 & 0 & 0 \\ 0 & m _ {1 2} & - 1 & 0 \\ m _ {2 1} & 0 & 0 & - 1 \end{array} \right) \quad \mathcal {B} = \left( \begin{array}{c c} 1 & 0 \\ 0 & 1 \\ 0 & 0 \\ 0 & 0 \end{array} \right) \quad \mathcal {C} = \left( \begin{array}{c c c c} 0 & m _ {1 2} & - 1 & 0 \\ m _ {2 1} & 0 & 0 & - 1 \end{array} \right). \tag {15}
$$
Figure 3: Single-player stabilization of the Jordan game.
Around an isolated mixed-strategy NE, the matrix  $\mathcal{A}$  above should be non-singular. Accordingly, it must be that  $m_{12} \neq 0$  and  $m_{21} \neq 0$ . The ability to stabilize a system via another stable system is referred to as strong stabilization (see Appendix A.3). The next proposition gives a sufficient condition under which an isolated mixed-strategy NE is not strongly stabilizable.
Proposition 6.1. If  $m_{12}m_{21} > 0$ , then (15) is not strongly stabilizable.
The proof of Proposition 6.1 uses the "parity interlacing principle" (see reference (37)) and is presented in Appendix B.4.
The nature of the game can be inferred from the scalars  $m_{12}$  and  $m_{21}$ . For example in zero-sum games we have  $M_{12} = -M_{21}^{\mathrm{T}}$ , which gives
$$
m _ {1 2} = N ^ {\mathrm {T}} M _ {1 2} N = N ^ {\mathrm {T}} M _ {1 2} ^ {\mathrm {T}} N = - N ^ {\mathrm {T}} M _ {2 1} N = - m _ {2 1}.
$$
In coordination games, we have  $M_{12} = M_{21}$ , which gives
$$
m _ {1 2} = N ^ {\mathrm {T}} M _ {1 2} N = N ^ {\mathrm {T}} M _ {2 1} N = m _ {2 1}.
$$
Therefore, the mixed-strategy NE in a coordination game is not strongly stabilizable.
Now let us examine the implications of inherently unstable learning dynamics. A reasonable expectation of learning dynamics is that in the case of a constant payoff vector, i.e.,  $p_i(t) \equiv p^*$ , we expect
$$
\lim _ {t \to \infty} x _ {i} (t) = \beta (p ^ {*}),
$$
where  $\beta (p^{*})$  is a best response to  $p^*$  , i.e.,
$$
\beta (p^{*}) = \operatorname *{arg  max}_{x_{i}\in \Delta (k_{i})}x_{i}^{\mathrm{T}}p^{*}.
$$
For any higher-order gradient play dynamics, if  $E_{i}$  is a stability matrix, then whenever  $p_i(t) \equiv p^*$  for some constant vector  $p^*$ , one can show  $\xi_i(t) \to 0$ , which implies that  $x_{i}(t)$  is generated by standard gradient play dynamics in the limit. However, if  $p_i(t) \equiv p^*$  and the dynamics are inherently unstable, the term  $N_{i}G_{i}\xi_{i}(t)$  need not vanish. Indeed, one can construct  $p^*$  such that  $x_{i}(t)$  does not converge to the best response of  $p^*$  (see the example in Section 7.2). The inability of learning dynamics to converge to the best response of a constant payoff vector does not reflect "natural" behavior.
7 Numerical experiments
7.1 Jordan anti-coordination game: Stabilization through a single player
The payoff matrices of the Jordan anti-coordination game, introduced in Section 5.1, satisfy Assumption 4.1. Therefore, we can stabilize its mixed-strategy NE, allowing only one player to use higher-order learning while others continue to use standard gradient play. For this purpose, let  $\xi_1 \in \mathbb{R}$  and choose  $H_1 = \gamma \lambda$ ,  $G_1 = -\gamma \lambda$ ,  $F_1 = \lambda$  and  $E_1 = -\lambda$ , where  $\lambda = 50$  and  $\gamma = 5$ . Such dynamics resemble anticipatory gradient play but on the filtered low-dimensional payoff. Figure 3 illustrates convergence of the players' strategies to NE.
(a) Stabilization of the mixed-strategy NE.
(b) Inherently unstable dynamics do not converge to best response of  $p^* = \begin{pmatrix} 0 & 1 \end{pmatrix}^\mathrm{T}$ .
Figure 4: Stabilizing the mixed-strategy equilibrium of a coordination game and its consequences.
7.2 Stabilization of mixed-strategy NE in coordination games
Consider the (identical interest) coordination game:
$$
u _ {1} (x _ {1}, x _ {2}) = x _ {1} ^ {\mathrm {T}} \left( \begin{array}{c c} 1 & 0 \\ 0 & 1 \end{array} \right) x _ {2}, \quad u _ {2} (x _ {2}, x _ {1}) = x _ {2} ^ {\mathrm {T}} \left( \begin{array}{c c} 1 & 0 \\ 0 & 1 \end{array} \right) x _ {1}.
$$
The game has two pure strategy NE and one completely mixed-strategy NE at  $x^{*} = \left(\frac{1}{2},\frac{1}{2}\right)^{\mathrm{T}}$ . Consider the following set of parameters for higher-order gradient play:  $E_{1} = \lambda$ ,  $F_{1} = -2\lambda$ ,  $G_{1} = \gamma \lambda$ ,  $H_{1} = -\gamma \lambda$ ,  $E_{2} = -\lambda_{2}$ ,  $F_{2} = \lambda_{2}$ ,  $G_{2} = -\gamma_{2}\lambda_{2}$ , and  $H_{2} = \gamma_{2}\lambda_{2}$ . The numerical values are  $\lambda = 0.5$ ,  $\gamma = 20$ ,  $\lambda_{2} = 50$ , and  $\gamma_{2} = 1$ . Figure 4a illustrates convergence to the mixed-strategy NE of this coordination game. Suppose we break the feedback loop and use  $p^{*} = (0\quad 1)^{\mathrm{T}}$  as the input to the first player dynamics. The response to such input is illustrated in Figure 4b. We see that  $E_{1} > 0$  is a scalar, and so  $\xi_{1}$  grows without bound. The strategy  $x_{1}$ , which is projected to the simplex, converges to  $(1\quad 0)^{\mathrm{T}}$ , which is not a best response to the input  $p^{*}$ .
To recap, we studied the role of higher-order gradient play with linear higher-order dynamics. We showed that for any game with an isolated completely mixed-strategy NE, there exist higher-order gradient play dynamics that lead to that NE, both for the original game and for nearby games. On the other hand, we showed that for any higher-order gradient play dynamics, the dynamics do not lead to NE for a suitably rescaled anti-coordination game. We also provided an argument against dynamics that lead to the mixed-strategy NE in a coordination game, showing they are not reasonable.
Regarding the higher-order gradient play dynamics that lead to NE, the interpretation of the results herein should not be that these dynamics are either a descriptive model of learning or a prescriptive recommendation for computation. Rather, the results are a contribution towards delineating what is possible or impossible in multi-agent learning. In that sense, they may be seen as a complement to the contributions in (17). Namely, dynamics being uncoupled is not a barrier to converging to mixed-strategy NE when allowing higher-order learning.
More generally, the present results open new questions related to the discussion in (14) on what constitutes "natural" learning dynamics. In the case of anticipatory higher-order learning, there is a clear interpretation of the effect of higher-order terms. However, it is unclear how to interpret general higher-order dynamics. Furthermore, the results herein regarding inherent instability of higher-order dynamics that converge to the mixed-strategy NE of a coordination game suggests that higher-order learning can be "unnatural". Possible restrictions on dynamics, in addition to being uncoupled, could include having no asymptotic regret; maintaining qualitative behavior in the face of strategically equivalent games (cf., Section 5.2); or having an interpretable relationship between payoff streams and strategic evolutions such as "passivity", which generalizes and extends the notion of contractive games to contractive learning dynamics (e.g., (38; 39; 40)).
In terms of limitations, the current paper only addresses the payoff vector setup and does not address the setup where players only have access to instantaneous scalar payoffs. Furthermore, results in Sections 5 and 6 are limited to certain setups and require further generalization.
[43] M. Hassouneh, H.-C. Lee, and E. Abed, "Washout filters in feedback control: benefits, limitations and extensions," in Proceedings of the 2004 American Control Conference, pp. 3950-3955, 2004.
In Appendix A, we review some relevant topics from linear systems theory. In Appendix B, we provide proofs of theorems, lemmas, and propositions that were presented in the paper. In Appendix C, we provide an alternative destabilization proof for sufficiently small gain  $\mu$ .
A Background on linear systems
Here, we review some standard background material for linear dynamics systems. There are several references (e.g., (41; 42)) with more detailed exposition.
The partitioned matrix
$$
\left( \begin{array}{c c} A & B \\ \hline C & D \end{array} \right)
$$
represents a general linear dynamical system
$$
\dot {x} (t) = A x (t) + B u (t), \quad x (0) = x _ {o} \tag {16a}
$$
$$
y (t) = C x (t) + D u (t) \tag {16b}
$$
whose solution is
$$
y (t) = C e ^ {A t} x _ {o} + \int_ {0} ^ {t} C e ^ {A (t - \tau)} B u (\tau) d \tau + D u (t).
$$
The variables  $x \in \mathbb{R}^n$ ,  $u \in \mathbb{R}^m$ , and  $y \in \mathbb{R}^p$  here are used temporarily as generic placeholders (and will play a different role in the ensuing discussion). The notation  $P \sim \left( \begin{array}{cc} A & B \\ \hline C & D \end{array} \right)$  assigns the label  $P$  to the dynamics (16).
A.2 Washout filters
In the special case of
$$
P \sim \left( \begin{array}{c c} - I & I \\ \hline - I & I \end{array} \right)
$$
$$
\dot {x} (t) = - x (t) + u (t), \quad x (0) = x _ {o}
$$
$$
y (t) = - x (t) + u (t)
$$
and  $x, u, y \in \mathbb{R}^n$ , it is straightforward to show that if
$$
\lim _ {t \to \infty} u (t) = u ^ {*},
$$
$$
\lim _ {t \to \infty} y (t) = 0.
$$
Linear systems with this property are known as "washout filters". For an extended discussion, see reference (43).
A.3 Stability and stabilization
A matrix,  $M$ , is a stability matrix if, for every eigenvalue,  $\lambda$ , of  $M$ ,  $\mathbf{Re}[\lambda] < 0$ .
The linear system,  $P \sim \left( \begin{array}{cc} A & B \\ \hline C & D \end{array} \right)$  is stable if  $A$  is a stability matrix.
The linear system  $K \sim \left( \begin{array}{cc} E & F \\ \hline G & H \end{array} \right)$  stabilizes  $P$  if the combined linear dynamics
$$
\dot {x} = A x + B u
$$
$$
\dot {\xi} = E \xi + F y
$$
$$
u = G \xi + H y
$$
$$
y = C x
$$
$$
\left( \begin{array}{c} \dot {x} \\ \dot {\xi} \end{array} \right) = \underbrace {\left( \begin{array}{c c} A + B H C & B G \\ F C & E \end{array} \right)} _ {M} \left( \begin{array}{c} x \\ \xi \end{array} \right)
$$
are stable, i.e.,  $M$  is a stability matrix.
The following conditions are necessary and sufficient for the existence of such a  $K$ . For all complex  $\lambda$  with  $\mathbf{Re}[\lambda] \geq 0$ :
- The pair  $(A, B)$  is stabilizable:  $(A - \lambda I B) = n$  has full row rank.
- The pair  $(A, C)$  is detectable:  $\binom{C}{A - \lambda I} = n$  has full column rank.
The linear system  $K$  strongly stabilizes  $P$  if (i)  $K$  stabilizes  $P$  and (ii)  $E$  is a stability matrix. Necessary and sufficient conditions for the existence of a strongly stabilizing  $K$  are presented in reference (37).
A.4 Decentralized stabilization
Let  $P \sim \left( \begin{array}{cc} A & B \\ \hline C & 0 \end{array} \right)$ , with  $A$  having dimensions  $n \times n$ , have the structure
$$
B = \left( \begin{array}{c c c} B _ {1} & \ldots & B _ {k} \end{array} \right) \& C = \left( \begin{array}{c} C _ {1} \\ \vdots \\ C _ {k} \end{array} \right)
$$
for some integer  $k$ . Suppose there exist linear systems
$$
K _ {1} \sim \left( \begin{array}{c c} E _ {1} & F _ {1} \\ \hline G _ {1} & H _ {1} \end{array} \right), \ldots , K _ {k} \sim \left( \begin{array}{c c} E _ {k} & F _ {k} \\ \hline G _ {k} & H _ {k} \end{array} \right)
$$
such that the combined linear dynamics
$$
\dot {x} = A x + \left( \begin{array}{c c c} B _ {1} & ... & B _ {k} \end{array} \right) \left( \begin{array}{c} u _ {1} \\ \vdots \\ u _ {k} \end{array} \right)
$$
$$
\dot {\xi} _ {1} = E _ {1} \xi_ {1} + F _ {1} y _ {1}
$$
$$
\dot {\xi} _ {k} = E _ {k} \xi_ {k} + F _ {k} y _ {k}
$$
$$
u _ {1} = G _ {1} \xi_ {1} + H _ {1} y _ {1}
$$
$$
u _ {k} = G _ {k} \xi_ {k} + H _ {k} y _ {k}
$$
$$
y _ {1} = C _ {1} x
$$
$$
y _ {k} = C _ {k} x
$$
are stable. Then the  $(K_{1},\dots,K_{k})$  achieve decentralized stabilization of  $P$ . The previous conditions of  $(A,B)$  stabilizable and  $(A,C)$  detectable are necessary, but not sufficient, conditions for decentralized stabilization.
The following theorem from reference (29) provides necessary and sufficient conditions for decentralized stabilization. First, for any partition  $Q \cup R = \{1, 2, \dots, k\}$  define  $B|^{Q}$  as the matrix formed by extracting the block columns of  $B = (B_{1} \dots B_{k})$  with indices in  $Q$ , i.e.,
$$
B | ^ {Q} = \left( \begin{array}{c c c} B _ {q _ {1}} & \ldots & B _ {q _ {| Q |}} \end{array} \right)
$$
with  $\{q_1,\dots ,q_{|Q|}\} = Q$ . Likewise, define  $C|_{R}$  as the matrix formed from the block rows of  $C = \begin{pmatrix} C_1\\ \vdots \\ C_k \end{pmatrix}$ , i.e.,
$$
C | _ {R} = \left( \begin{array}{c} C _ {r _ {1}} \\ \vdots \\ C _ {r _ {| R |}} \end{array} \right)
$$
with  $\{r_1,\dots,r_{|R|}\} = R$
Theorem A.1 ((29), Theorem 3). There exist  $(K_{1},\dots,K_{k})$  that achieve decentralized stabilization of  $P$  if and only if
$$
\mathbf {r a n k} \left( \begin{array}{c c} A - \lambda I & B | ^ {Q} \\ C | _ {R} & 0 \end{array} \right) = n
$$
for all complex  $\lambda$  with  $\mathbf{Re}[\lambda] \geq 0$  and all partitions,  $Q \cup R = \{1,2,\dots,k\}$ .
The above rank condition must hold for all partitions  $Q \cup R = \{1, \dots, k\}$ . If  $R = \emptyset$ , one recovers the rank condition for (centralized) stabilizability. Likewise,  $Q = \emptyset$  results in the rank condition for detectability.
We will examine the conditions of Theorem A.1 on the system (13)-(14). We need to inspect the rank of
$$
\left( \begin{array}{c c} \mathcal {A} - \lambda I & \mathcal {B} | ^ {Q} \\ \mathcal {C} | _ {R} & 0 \end{array} \right)
$$
for all partitions  $Q \cup R = \{1, 2, \dots, n\}$ . Note that the partitions of either  $Q = \emptyset$  or  $R = \emptyset$  are already covered by Proposition 4.1.
First, note that
$$
\left( \begin{array}{c c} \mathcal {A} - \lambda I & \mathcal {B} | ^ {Q} \\ \mathcal {C} | _ {R} & 0 \end{array} \right) = \left( \begin{array}{c c c c c} \mathcal {M} - \lambda I & 0 & \left(\mathcal {E} _ {q _ {1}} & \ldots & \mathcal {E} _ {q _ {| Q |}}\right) \\ \mathcal {M} & - (\lambda + 1) I & 0 \\ \mathcal {M} | _ {R} & - I | _ {R} & 0 \end{array} \right).
$$
Let  $\mathcal{M}$  be an  $\ell \times \ell$  matrix. Then
$$
\ell = \sum_ {i = 1} ^ {n} \left(k _ {i} - 1\right).
$$
We need the rank of the above matrix to be  $2\ell$  for all  $\lambda$  with  $\mathbf{Re}[\lambda] \geq 0$ . Because of the presence of  $\mathcal{A} - \lambda I$ , loss of rank below  $2\ell$  is only possible at eigenvalues of  $\mathcal{A}$ . Since we are only concerned with  $\mathbf{Re}[\lambda] \geq 0$ , we focus on eigenvalues of  $\mathcal{M}$  (which excludes  $\lambda = 0$  by hypothesis, since  $\mathcal{M}$  is non-singular).
Without affecting the rank, we can multiply the bottom block row by  $-( \lambda + 1)$  and add the middle block rows corresponding  $R$  to the rescaled bottom block row to get
$$
\mathbf {r a n k} \left( \begin{array}{c c} \mathcal {A} - \lambda I & \mathcal {B} | ^ {Q} \\ \mathcal {C} | _ {R} & 0 \end{array} \right) = \mathbf {r a n k} \left( \begin{array}{c c c c c} \mathcal {M} - \lambda I & 0 & \left(\mathcal {E} _ {q _ {1}} & \ldots & \mathcal {E} _ {q _ {| Q |}}\right) \\ \mathcal {M} & - (\lambda + 1) I & & 0 \\ - \lambda \mathcal {M} | _ {R} & 0 & & 0 \end{array} \right).
$$
Switching the top and bottom block rows results in
$$
\left( \begin{array}{c c c c} - \lambda \mathcal {M} | _ {R} & 0 & 0 \\ \mathcal {M} & - (\lambda + 1) I & 0 \\ \mathcal {M} - \lambda I & 0 & \left(\mathcal {E} _ {q _ {1}} \quad \ldots \quad \mathcal {E} _ {q _ {| Q |}}\right) \end{array} \right).
$$
We can now exploit the block triangular structure. The bottom block row provides a row rank of
$$
\sum_ {q \in Q} (k _ {q} - 1),
$$
The middle block row provides a row rank of  $\ell$ . Finally, the top block row provides a row rank of
$$
\sum_ {r \in R} \left(k _ {r} - 1\right).
$$
The last assertion is because  $\mathcal{M}$  is non-singular, by hypothesis, and therefore it has linearly independent rows. Since  $Q\cup R = \{1,\ldots n\}$ , we have the desired row rank of  $2\ell$
B.2 Proof of Lemma 5.1
$$
H (s) = C (s I - A) ^ {- 1} B.
$$
Since  $H(s)$  is a rational function, we can write it as
$$
H (s) = \frac {p (s)}{q (s)},
$$
for polynomials  $p$  and  $q$  that have no common roots. The assumption that  $CA^m B \neq 0$  for some  $m$  assures that  $H(s)$  is not identically equal to zero.
Suppose that for some  $\mu$  and  $s'$  that is not an eigenvalue of  $A$ ,
$$
q \left(s ^ {\prime}\right) + \mu p \left(s ^ {\prime}\right) = 0.
$$
Then  $s'$  is an eigenvalue of  $A - \mu BC$ , since
$$
\begin{array}{l} \det  \left[ s ^ {\prime} I - (A - \mu B C) \right] = \det  \left[ s ^ {\prime} I - A \right] \det  \left[ I + \mu \left(s ^ {\prime} I - A\right) ^ {- 1} B C \right] \\ = \det  \left[ s ^ {\prime} I - A \right] (1 + \mu C (s ^ {\prime} I - A) ^ {- 1} B) \\ = \det  \left[ s ^ {\prime} I - A \right] \left(q \left(s ^ {\prime}\right) + \mu p \left(s ^ {\prime}\right)\right) \frac {1}{q \left(s ^ {\prime}\right)}. \\ \end{array}
$$
Note that the roots of  $q(s)$  are a subset of the roots of  $\operatorname{det}[sI - A]$ . For sufficiently large  $|s|$ , we can rewrite  $H(s)$  as
$$
\begin{array}{l} H (s) = \frac {1}{s} C \left(I - \frac {1}{s} A\right) ^ {- 1} B \\ = \frac {1}{s} C \left(\sum_ {k = 0} ^ {\infty} \frac {1}{s ^ {k}} A ^ {k}\right) B. \\ \end{array}
$$
By assumption,  $CB = 0$  and  $CAB = 0$ , which implies that the first two terms of the series equal zero. Accordingly,
$$
\limsup_{|s|\to \infty}|s|^{3}|H(s)| <   \infty .
$$
The main implication here is that the degree of  $q(s)$  is at least 3 more than the degree of  $p(s)$ . Root-locus arguments in references (30) and (31) (asymptote rule) imply that
$$
q (s) + \mu p (s)
$$
has roots with positive real parts for large  $\mu$ .
B.3 Proof of Proposition 5.1
The local dynamics matrix in the game  $\Gamma (\mu)$  can be written in the form  $A - \mu BC$  with
$$
A = \left( \begin{array}{c c c c c c c c c} 0 & 0 & 0 & G _ {1} & - H _ {1} & 0 & 0 & 0 & 0 \\ - (1 + H _ {3}) & 0 & 0 & 0 & 0 & G _ {3} & - H _ {3} & 0 & 0 \\ 0 & - (1 + H _ {2}) & 0 & 0 & 0 & 0 & 0 & G _ {2} & - H _ {2} \\ 0 & 0 & 0 & E _ {1} & - F _ {1} & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & - 1 & 0 & 0 & 0 & 0 \\ - F _ {3} & 0 & 0 & 0 & 0 & E _ {3} & - F _ {3} & 0 & 0 \\ - 1 & 0 & 0 & 0 & 0 & 0 & - 1 & 0 & 0 \\ 0 & - F _ {2} & 0 & 0 & 0 & 0 & 0 & E _ {2} & - F _ {2} \\ 0 & - 1 & 0 & 0 & 0 & 0 & 0 & 0 & - 1 \end{array} \right) \quad B = \left( \begin{array}{c} H _ {1} + 1 \\ 0 \\ 0 \\ F _ {1} \\ 1 \\ 0 \\ 0 \\ 0 \end{array} \right) \tag {17a}
$$
$$
C = \left( \begin{array}{l l l l l l l l l} 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right), \tag {17b}
$$