SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. Based on Lemma 3.8, we can guarantee the existence of a round  $T^{\star} \coloneqq T_{n/2-1}$  when the players choose the strategy profile  $\left( \frac{n}{2} + 1, \frac{n}{2} \right)$  for the first time. In addition, at round  $T^{\star}$ , it holds that  $R_{n/2-1}^{(T^{\star})} > 0$  and  $C_{n/2+1}^{(T^{\star})} = R_{n/2}^{(T^{\star})} = 0$ . The latter condition ensures that the strategy profile  $\left( \frac{n}{2}, \frac{n}{2} + 1 \right)$  has not been played up to time  $T^{\star}$ .
As indicated by Observation 3.3, row  $\frac{n}{2}$  has non-zero entries at columns  $\frac{n}{2}$  and  $\frac{n}{2} + 1$ . Therefore, if the cumulative utilities  $R_{n/2}^{(T^{\star})}$  at time  $T^{\star}$  is zero, this implies that neither of these columns has been chosen up to that point. By the same reasoning, column  $\frac{n}{2} + 1$  has a non-zero entry only at row  $\frac{n}{2}$ , indicating that this row has not been chosen as well.
In order to continue, we require an estimate of the duration during which the strategy profile  $\left(\frac{n}{2} + 1, \frac{n}{2}\right)$  will be played. Observation 3.3 guarantees that the utility vector of the row player is the following.
$$
A e _ {\frac {n}{2}} = (0, \dots , 0, \underbrace {2 n - 2} _ {\frac {n}{2}}, \underbrace {2 n - 3} _ {\frac {n}{2} + 1}, 0, \dots , 0) \tag {3}
$$
We can combine this information with the fact that  $R_{n/2+1}^{(T^*)} \neq 0$ ,  $R_{n/2}^{(T^*)} = 0$ , and that the respective entries in the payoff vector (3) of the row player differ by exactly one. This allows us to conclude that it will take at least  $R_{\frac{n}{2} + 1}^{(T^*)}$  iterations for the cumulative utilities to become equal, i.e.  $R_{\frac{n}{2} + 1} = R_{\frac{n}{2}}$ . Therefore, the lower bound on the number of iterations holds regardless of the tie-breaking rule.
Now, if at any time  $T$  the agents play the strategy profile  $(i^{(T)}, j^{(T)}) = \left(\frac{n}{2}, \frac{n}{2} + 1\right)$ , we can conclude that  $T \geq R_{\frac{n}{2} + 1}^{(T^{\star})}$ . Using Equation (2) of Lemma 3.8 we obtain:
$$
T \geq R _ {\frac {n}{2} - 1} ^ {(T ^ {\star})} \geq 1 6 ^ {\frac {n}{2} - 1} \left(\prod_ {\ell = 2} ^ {\frac {n}{2} - 1} (\ell - 1)\right) ^ {4} R _ {n - 1} ^ {(T _ {1})} \geq \Omega \left(4 ^ {n} \left(\left(\frac {n}{2} - 2\right)!\right) ^ {4}\right)
$$
Now that we have established the necessary technical result in Lemma 3.9, we are ready to present the proof of Theorem 3.1.
Proof of Theorem 3.1. Let  $T^{\star} \geq 1$  denote the first time at which  $(i^{(t)}, j^{(t)}) = (\frac{n}{2}, \frac{n}{2} + 1)$ . For any  $t \leq T^{\star} - 1$ , it holds that  $\hat{x}_{\frac{n}{2}}^{(t)} = \hat{y}_{\frac{n}{2} + 1}^{(t)} = 0$ . Thus, Lemma 3.6 implies that  $(\hat{x}^{(t)}, \hat{y}^{(t)})$  is not an approximate NE. On the other hand, at each round  $t \geq T^{\star}$  we are ensured that  $\hat{x}_{\frac{n}{2}}^{(t)}, \hat{y}_{\frac{n}{2} + 1}^{(t)}$  converges to 1 with rate  $1 / t$ . Applying Lemma 3.6 for  $\epsilon \coloneqq \sqrt{\epsilon}$  we get that if  $(\hat{x}^{(t)}, \hat{y}^{(t)})$  is an  $\epsilon$ -NE then  $t \geq T^{\star} + \frac{1}{n\sqrt{\epsilon}} \geq \Omega (4^n ((n / 2 - 2)!)^4) + \frac{1}{n\sqrt{\epsilon}}$ . It is evident that, even with a uniformly random initialization, the probability of selecting  $(n, 1)$  as the starting point for fictitious play is  $1 / n^2$  and so the claim of the theorem follows.
(a)  $t\in [T_i^0,T_i^1)$
Figure 3: The figure illustrates the active row and column player for each time period, with the played strategy highlighted in purple and the corresponding payoff vectors of the row and column player highlighted in green. Additionally, the time period of each played strategy is indicated for clarity.
(b)  $t\in [T_i^1,T_i^2)$
(c)  $t\in [T_i^2,T_i^3)$
(d)  $t\in [T_i^3,T_i^4)$
3.3 Strategy Switches and Proof of Lemma 3.8
This subsection is dedicated to presenting the proof of Lemma 3.8. However, before delving into the proof, we first provide additional statements that serve to shed light on the sequence of strategy profiles generated by fictitious play.
According to Proposition 3.5, if the strategy  $(i,j)$  is chosen by fictitious play, then either in row  $i$  or in column  $j$ , there is at most one element greater than  $A_{ij}$ . Therefore, in a subsequent round, fictitious play will necessarily choose this element as its strategy. Let  $i'$  be the row where the element of greater value is located. Intuitively, we can imagine that as fictitious play continues to play the same strategy  $(i,j)$ , the payoff vector  $Ae_j$  is repeatedly added to the cumulative vector of the row player. Since this vector has a greater value at coordinate  $i'$  than at coordinate  $i$ , a strategy switch will eventually occur. We state this observation in Proposition 3.10 and defer the proof to the Appendix.
Proposition 3.10 (Strategy Switch). Let  $(i^{(t)}, j^{(t)})$  be a strategy selected by fictitious play at round  $t$ , and  $(i^{(t)}, j^{(t)}) \neq \left(\frac{n}{2}, \frac{n}{2}\right)$ . Then, in a subsequent round, fictitious play will choose the strategy of greater value that is either on row  $i^{(t)}$  or column  $j^{(t)}$ .
As previously hinted, the structure of the payoff matrix dictates the sequence of strategy profiles chosen by fictitious play. The sequence of strategy switches alternates between rows and columns. Continuing with the example from the previous paragraph, let us assume that the row player made the most recent strategy switch from  $(i,j)$  to  $(i',j)$ . This implies that the element  $A_{i'j}$  is greater than  $A_{ij}$ , as otherwise a strategy switch would not have taken place, as established in Proposition 3.10. Moreover, by Proposition 3.5, we know that there must be an element of greater value in either row  $i'$  or column  $j$ . Since  $A_{i'j}$  is greater than  $A_{ij}$ , any element of greater value must be located in row  $i'$ . The same reasoning applies for the case where the column player was the last to switch strategies. We can summarize this observation by stating that if one player is the last to switch, then the other player must switch next. We formally state this in Corollary 3.11 and defer the proof to the Appendix.
Corollary 3.11 (Successive Strategy Switches). Let  $t$  be a round in which a player changes their strategy. Then exactly one of the following statements is true:
1. If the row player changes their strategy at round  $t$ , i.e.  $i^{(t)} \neq i^{(t-1)}$ , then the column player can only make the next strategy switch.
2. If the column player changes their strategy at round  $t$ , i.e.  $j^{(t)} \neq j^{(t-1)}$ , then the row player can only make the next strategy switch.
Applying the same concept, we can observe that starting from the lower-left corner, fictitious play follows a spiral trajectory. The resulting spiral is illustrated in Figure 2. We now proceed with the main result of this section, Lemma 3.8.
Proof. Since  $(i^{(1)}, j^{(1)}) = (n, 1)$  all the above claims trivially for  $T_0 = 1$ . We assume that the claim holds for  $i$  and will now establish it inductively for  $i + 1$ .
By the induction hypothesis, agents play strategies  $(n - i,i + 1)$  at round  $T_{i}^{0}\coloneqq T_{i}$ . Furthermore, row  $n - i$  admits cumulative utility of  $R_{n - i}^{(T_i^0)}$  while row  $i + 1$  admits cumulative utility of  $R_{i + 1}^{(T_i^0)} = 0$ . According to Observation 3.3, the payoff vectors of the row and column agent are highlighted in Figure 3a. By combining these facts, we can establish the following.
Proposition 3.12 (Abridged; Full Version in Proposition B.3). There exists a round  $T_{i}^{1} > T_{i}^{0}$  at which the strategy profile is  $(i + 1, i + 1)$  for the first time, column  $i + 1$  admits cumulative utility  $C_{i + 1}^{(T_i^1)} \geq (4i + 1) \cdot (R_{i + 1}^{(T_i^0)} + 1)$ , and  $C_{n - i}^{(T_i^1)} = 0$ .
By Proposition 3.12, at round  $T_{i}^{1}$ , the agents play strategies  $(i + 1, i + 1)$ . Furthermore, the cumulative utility of column  $n - i$  equals to  $C_{n - i}^{(T_i^1)} = 0$ . According to Observation 3.3, the payoff vectors of the row and column agent are highlighted in Figure 3b. Combining these facts, we get the following:
Proposition 3.13 (Abridged; Full Version in Proposition B.4). There exists a round  $T_{i}^{2} > T_{i}^{1}$  at which the strategy profile is  $(i + 1, n - i)$  for the first time, row  $i + 1$  admits cumulative utility  $R_{i + 1}^{(T_i^2)} \geq (4i + 2) \cdot C_{i + 1}^{(T_i^1)}$ , and  $R_{n - i - 1}^{(T_i^2)} = 0$ .
By Proposition 3.13, at round  $T_{i}^{2}$ , the agents play strategies  $(i + 1, n - i)$ . Furthermore, the cumulative utility of row  $n - i - 1$  equals to  $R_{n - i - 1}^{(T_{i}^{2})} = 0$ . According to Observation 3.3, the payoff vectors of the row and column agent are highlighted in Figure 3c. By combining these facts, we get the following:
Proposition 3.14 (Abridged; Full Version in Proposition B.5). There exists a round  $T_i^3 > T_i^2$  at which the strategy profile is  $(n - (i + 1), n - i)$  for the first time,  $R_{i+2}^{(T_i^3)} = 0$ .
By Proposition 3.14, at round  $T_{i}^{3}$ , the agents play strategies  $(n - i - 1, n - i)$ . The cumulative of column  $i + 2$  equals to  $R_{i + 2}^{(T_{i}^{3})} = 0$ . According to Observation 3.3, the payoff vectors of the row and column agent are highlighted in Figure 3d. By combining these facts, we can establish the following.
Proposition 3.15 (Abridged; Full Version in Proposition B.6). There exists a round  $T_i^4 > T_i^3$  at which the strategy profile is  $(n - (i + 1), (i + 1) + 1)$  for the first time, row  $n - i - 1$  admits cumulative utility  $R_{n - (i + 1)}^{(T_i^4)} \geq (4i + 4) \cdot C_{n - i}^{(T_i^3)}$ , all rows  $k \in [(i + 1) + 1, n - (i + 1) - 1]$  admit  $R_k^{(T_i^4)} = 0$  and all columns  $k \in [(i + 1) + 2, n - (i + 1)]$  admit  $C_k^{(T_i^4)} = 0$ .
Proposition 3.15 establishes that there exists a round  $T_{i+1} \coloneqq T_i^4$  at which the strategy profile  $(n - (i + 1), (i + 1) + 1)$  is played for the first time. Furthermore, Proposition 3.15 confirms that all rows  $k \in \{(i + 1) + 1, n - (i + 1) - 1\}$  admit  $R_k^{(T_{i+1})} = 0$  and all columns  $k \in \{(i + 1) + 2, n - (i + 1)\}$  admit  $C_k^{(T_{i+1})} = 0$ . We still need to verify the recursive relation Equation (2). By combining Proposition 3.12, 3.13, 3.14 and 3.15, we can deduce
$$
R _ {n - i - 1} ^ {(T _ {i + 1})} \geq (4 i + 4) (4 i + 3) (4 i + 2) (4 i + 1) R _ {n - i} ^ {(T _ {i})}.
$$
In this section, we aim to experimentally validate our findings on a  $4 \times 4$  payoff matrix. Our analysis focuses on three key aspects: the round in which a new strategy switch occurs, the Nash gap throughout the game, and the empirical strategy employed by the  $x$  player. We present the plot from the row player's perspective, which is identical to that of the column player.
In Figure 4a, we provide the time steps of all strategy switches. As it is expected from the analysis, fictitious play visits all strategies, specifically in increasing order of their utility, to reach the pure Nash equilibrium. Moreover, in Figure 4b we observe a recurring pattern in the Nash gap diagram, where the gap increases after the selection of a new strategy with a higher utility and decreases until the next strategy switch. However, this pattern stops after the pure Nash equilibrium is reached, which is the unique approximate Nash equilibrium in accordance with Lemma 3.8.
t	player	from	to
2	i	4	1
4	j	1	4
11	i	1	3
38	j	4	2
174	i	3	2
990	j	2	3
(a) Table of transitions
(b) Nash Gap
Figure 4: The figure displays the three aspects that were analyzed in the experiments. To preserve the crucial qualitative features in both diagrams, the  $x$ -axis was set to a logarithmic scale.
(c) Empirical strategy of  $x$  player
In summary, this paper has provided a thorough examination of the convergence rate of fictitious play within a specific subset of potential games. Our research has yielded a recursive rule for constructing payoff matrices, demonstrating that fictitious play, regardless of the tie-breaking rule employed, may require super exponential time to reach a Nash equilibrium even in two-player identical payoff games. This contribution to the literature differs from previous studies and sheds new light on the limitations of fictitious play in the particular class of potential games.
Limitations and Broader Impacts: Our work is theoretical in nature, and we do not identify any limitations or negative ethical or societal implications.
Ioannis Panageas would like to acknowledge a start-up grant and SF50818. This work was supported by the Swiss National Science Foundation (SNSF) under grant number 200021_205011, by Hasler Foundation Program: Hasler Responsible AI (project number 21043) and Innovation project supported by Innosuisse (contract agreement 100.960 IP-ICT).
[1] J. D. Abernethy, K. A. Lai, and A. Wibisono. Fast convergence of fictitious play for diagonal payoff matrices. In D. Marx, editor, Proceedings of the 2021 ACM-SIAM Symposium on Discrete Algorithms, SODA 2021, Virtual Conference, January 10 - 13, 2021, pages 1387-1404. SIAM, 2021. doi: 10.1137/1.9781611976465.84. URL https://doi.org/10.1137/1.9781611976465.84.
[2] L. Baudin and R. Laraki. Fictitious play and best-response dynamics in identical interest and zero-sum stochastic games. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvári, G. Niu, and S. Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 1664-1690. PMLR, 2022. URL https://proceedings.mlr.press/v162/baudin22a.html.
[3] L. Baudin and R. Laraki. Smooth fictitious play in stochastic games with perturbed payoffs and unknown transitions. Advances in Neural Information Processing Systems, 35:20243-20256, 2022.
[4] G. W. Brown. Some notes on computation of games solutions. Technical report, RAND CORP SANTA MONICA CA, 1949.
[5] G. W. Brown. Iterative solution of games by fictitious play. Act. Anal. Prod Allocation, 13(1): 374, 1951.
[6] O. Candogan, A. E. Ozdaglar, and P. A. Parrilo. Dynamics in near-potential games. Games Econ. Behav., 82:66-90, 2013. doi: 10.1016/j.geb.2013.07.001. URL https://doi.org/10.1016/j.geb.2013.07.001.
[24] L. Shapley. Some topics in two-person games. Advances in game theory, 52:1-29, 1964.
[25] B. Swenson and S. Kar. On the exponential rate of convergence of fictitious play in potential games. In 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017, Monticello, IL, USA, October 3-6, 2017, pages 275-279. IEEE, 2017. doi: 10.1109/ALLERTON.2017.8262748. URL https://doi.org/10.1109/ALLERTON.2017.8262748.
[26] Y. Yang and J. Wang. An overview of multi-agent reinforcement learning from game theoretical perspective. CoRR, abs/2011.00583, 2020. URL https://arxiv.org/abs/2011.00583.
In this section, we present a comprehensive proof of Lemma 3.6. This lemma establishes a fundamental property of our recursive construction, as defined in Definition 3.2: the only  $\epsilon^2$ -approximate Nash equilibrium is located at the maximum value element. Consequently, the fictitious play dynamics fail to reach even an approximate Nash equilibrium (within the specified degree of approximation) unless the strategy  $\left(\frac{n}{2}, \frac{n}{2} + 1\right)$  has been played for a sufficient duration. Our theoretical findings are further supported by our experiments. As shown in Figure 4b, the Nash gap does not vanish before the final strategy switch occurs. To provide a precise definition of the Nash gap, we present it below.
Definition A.1 (Nash Gap for Identical Payoff). The Nash gap at round  $t$  for a two-player identical payoff games is
$$
\left(\max  _ {i \in [ n ]} [ A y ^ {(t)} ] _ {i} - (x ^ {(t)}) ^ {\top} A y ^ {(t)}\right) + \left(\max  _ {j \in [ n ]} [ (x ^ {(t)}) ^ {\top} A ] _ {j} - (x ^ {(t)}) ^ {\top} A y ^ {(t)}\right)
$$
To establish Lemma 3.6, we employ a similar approach as in the other proofs presented in this work. Specifically, we heavily rely on the structure of our payoff matrix and utilize an induction technique to demonstrate that the majority of the probability mass must be concentrated in the maximum element. The induction argument starts from the outermost elements of the matrix, namely row  $n - 1$ , column 1, and row 1, column  $n - 1$ . By successive induction steps, we establish that until the maximum element is reached, none of the elements in those rows or columns can possess a significant probability mass.
Lemma A.2 (Proof of Lemma 3.6). Let  $\epsilon \in (0,1 / 56n^3]$  and  $(x^{\star},y^{\star})$  an  $\epsilon^2$ -NE. Then for each  $i\in [0,n / 2 - 2]$ ,
-  $x_{i+1}^{\star} \leq \epsilon$  and  $x_{n-i}^{\star} \leq \epsilon$ .
-  $y_{i+1}^{\star} \leq \epsilon$  and  $y_{n-i}^{\star} \leq \epsilon$ .
Proof. We will prove the claim by induction. First, assume that for all  $j \in [0, i - 1]$ , we have:
$x_{j + 1}^{\star}\leq \epsilon$  and  $x_{n - j}^{\star}\leq \epsilon$
-  $y_{j+1}^{\star} \leq \epsilon$  and  $y_{n-j}^{\star} \leq \epsilon$ .
Next, we proceed to establish the inequalities  $x_{n - i}^{\star},y_{i + 1}^{\star},x_{i + 1}^{\star},y_{n - i}^{\star}\leq \epsilon$ . We demonstrate these inequalities in the exact order as presented, as their proof relies on the underlying structure of the matrix. It is important to note that there are interdependencies between these inequalities, which we will address accordingly.
Case 1:  $x_{n - i}^{\star}\leq \epsilon$
Let assume that  $x_{n - i}^{\star} > \epsilon$  and we will reach a contradiction. From Observation 3.3, we notice that the utility of row  $n - i$  equals
$$
\left[ A y ^ {\star} \right] _ {n - i} = (4 i + 1) y _ {i + 1} ^ {\star} + 4 i y _ {n - i + 1} ^ {\star} \tag {4}
$$
At the same time the utility of row  $i + 1$  equals
$$
[ A y ^ {\star} ] _ {i + 1} = (4 i + 2) y _ {i + 1} ^ {\star} + (4 i + 3) y _ {n - i} ^ {\star}
$$
As a result, by taking the difference on the utilities of row  $i + 1$  and  $n - i$  we get,
$$
\begin{array}{l} [ A y ^ {\star} ] _ {i + 1} - [ A y ^ {\star} ] _ {n - i} = (4 i + 2) y _ {i + 1} ^ {\star} + (4 i + 3) y _ {n - i} ^ {\star} - (4 i + 1) y _ {i + 1} ^ {\star} - 4 i y _ {n - i + 1} ^ {\star} \\ = y _ {i + 1} ^ {\star} + (4 i + 3) y _ {n - i} ^ {\star} - 4 i y _ {n - i + 1} ^ {\star} \\ \geq y _ {i + 1} ^ {\star} + y _ {n - i} ^ {\star} - 4 i \epsilon \\ \end{array}
$$
where the last inequality follows by the fact that  $y_{n - i + 1}^{\star}\leq \epsilon$  (Inductive Hypothesis). As a result, we conclude that
$$
[ A y ^ {\star} ] _ {i + 1} - [ A y ^ {\star} ] _ {n - i} \geq y _ {i + 1} ^ {\star} + y _ {i + 1} ^ {\star} - 4 i \epsilon
$$
In case  $y_{i + 1}^{\star} + y_{n - i}^{\star} \geq (4i + 1)\epsilon$  then  $[Ay^{\star}]_{i + 1} - [Ay^{\star}]_{n - i} \geq \epsilon$ . Hence if the row player puts  $x_{n - i}^{\star}$  probability mass to row  $i + 1$  by transferring the probability mass from row  $n - i$  to row  $i + 1$  then it increases its payoff by  $x_{n - i}^{\star}([Ay^{\star}]_{i + 1} - [Ay^{\star}]_{n - i}) > \epsilon^{2}$ . The latter contradicts with the assumption that  $(x^{\star},y^{\star})$  is an  $\epsilon^2$ -NE. Thus, we conclude in the following two statements,
$$
y _ {i + 1} ^ {\star} + y _ {n - i} ^ {\star} \leq (4 i + 1) \epsilon \quad \text {a n d} \quad [ A y ^ {\star} ] _ {n - i} \leq 2 (4 i + 1) ^ {2} \epsilon
$$
where the last inequality is obtained by combining the first inequality with Equation (4). Now consider the sum of the utilities of rows  $k \in [i + 2, n - i - 1]$ . By the construction of the payoff matrix  $A$  we can easily establish the following claim.
Proposition A.3. The sum of utilities of rows  $k \in [i + 2, n - i - 1]$  satisfies the inequality,
$$
\sum_ {k = i + 2} ^ {n - i - 1} \left[ A y ^ {\star} \right] _ {k} \geq \sum_ {k = i + 2} ^ {n - i - 1} y _ {k} ^ {\star}
$$
By Proposition A.3 we are ensured that
$$
\begin{array}{l} \sum_ {k = i + 2} ^ {n - i - 1} \left[ A y ^ {\star} \right] _ {k} \geq \sum_ {k = i + 2} ^ {n - i - 1} y _ {k} ^ {\star} \\ = \quad 1 - \sum_ {k = 1} ^ {i + 1} y _ {k} ^ {\star} - \sum_ {k = n - i} ^ {n} y _ {k} ^ {\star} \\ = \quad 1 - \left(y _ {i + 1} ^ {\star} + y _ {n - i} ^ {\star}\right) - \left(\sum_ {k = 1} ^ {i} y _ {k} ^ {\star} + \sum_ {k = n - i + 1} ^ {n} y _ {k} ^ {\star}\right) \\ \geq 1 - (4 i + 1) \epsilon - n \epsilon \\ \geq 1 - (5 n + 1) \epsilon \\ \end{array}
$$
where the second inequality follows by the fact that  $y_{i + 1}^{\star} + y_{n - i}^{\star}\leq (4i + 1)\epsilon$  and the fact that  $y_{j}^{\star}\leq \epsilon$  for all  $k\in [1,i]\cup [n - i + 1,n]$  (Inductive Hypothesis).
Due to the fact that
$$
\sum_ {k = i + 2} ^ {n - i - 1} \left[ A y ^ {\star} \right] _ {k} \geq 1 - (5 n + 1) \epsilon
$$
we are ensured that there exists a row  $k^{\star} \in [i + 2, n - i - 1]$  with utility  $[Ay^{\star}]_{k^{\star}} \geq \frac{1 - (5n + 1)\epsilon}{n}$ . Now consider the difference between the utility of row  $k^{\star}$  and the row  $n - i$ .
$$
[ A y ^ {\star} ] _ {k ^ {\star}} - [ A y ^ {\star} ] _ {n - i} \geq \frac {1 - (5 n + 1) \epsilon}{n} - 2 (4 i + 1) ^ {2} \epsilon \geq \frac {1 - (5 n + 1) \epsilon}{n} - 2 (4 n + 1) ^ {2} \epsilon \geq \epsilon
$$
where the last inequality holds for  $\epsilon \leq 1 / 56n^3$ . Hence if the row player puts  $x_{n - i}^{\star}$  probability mass to row  $k^{\star}$  then it increases its payoff by  $x_{n - i}^{\star}([Ay^{\star}]_{k^{\star}} - [Ay^{\star}]_{n - i}) > \epsilon^{2}$ . The latter contradicts with the assumption that  $(x^{\star},y^{\star})$  is an  $\epsilon^2$ -NE. Thus we have reached to a final contradiction that  $x_{n - i}^{\star} > \epsilon$ .
Case 2:  $y_{i + 1}^{\star}\leq \epsilon$
Similar to the previous case, we assume that  $y_{i + 1}^{\star} > \epsilon$  and proceed to derive a contradiction. From Observation 3.3, we notice that the utility of column  $i + 1$  is given by:
$$
[ (x ^ {\star}) ^ {\top} A ] _ {i + 1} = (4 i + 2) x _ {i + 1} ^ {\star} + (4 i + 1) x _ {n - i} ^ {\star}
$$
At the same time the utility of column  $n - i$  equals
$$
[ (x ^ {\star}) ^ {\top} A ] _ {n - i} = (4 i + 3) x _ {i + 1} ^ {\star} + (4 i + 4) x _ {n - i - 1} ^ {\star}
$$
As a result, by taking the difference on the utilities of columns  $n - i$  and  $i + 1$  we get,
$$
\begin{array}{l} [ (x ^ {\star}) ^ {\top} A ] _ {n - i} - [ (x ^ {\star}) ^ {\top} A ] _ {i + 1} = (4 i + 3) x _ {i + 1} ^ {\star} + (4 i + 4) x _ {n - i - 1} ^ {\star} - (4 i + 2) x _ {i + 1} ^ {\star} - (4 i + 1) x _ {n - i} ^ {\star} \\ = x _ {i + 1} ^ {\star} + (4 i + 4) x _ {n - i - 1} ^ {\star} - (4 i + 1) x _ {n - i} ^ {\star} \\ \geq x _ {i + 1} ^ {\star} + x _ {n - i - 1} ^ {\star} - (4 i + 1) \epsilon \\ \end{array}
$$
where the last inequality follows by the fact  $x_{n - i}^{\star} \leq \epsilon$  (Inductive Step Case 1). As a result, we conclude that
$$
[ (x ^ {\star}) ^ {\top} A ] _ {n - i} - [ (x ^ {\star}) ^ {\top} A ] _ {i + 1} \geq x _ {i + 1} ^ {\star} + x _ {n - i - 1} ^ {\star} - (4 i + 1) \epsilon
$$
In case  $x_{i+1}^{\star} + x_{n-i-1}^{\star} \geq (4i+2)\epsilon$  then  $[(x^{\star})^{\top}A]_{n-i} - [(x^{\star})^{\top}A]_{i+1} \geq \epsilon$ . Hence if the column player puts  $y_{i+1}^{\star}$  probability mass to column  $n-i$  then it increases its payoff by  $y_{i+1}^{\star}([x^{\star})^{\top}A]_{n-i} - [(x^{\star})^{\top}A]_{i+1}) > \epsilon^{2}$ . The latter contradicts with the assumption that  $(x^{\star},y^{\star})$  is an  $\epsilon^{2}$ -NE. Thus we conclude in the following two statements:
$$
x _ {i + 1} ^ {\star} + x _ {n - i - 1} ^ {\star} \leq (4 i + 2) \epsilon \quad \text {a n d} \quad [ (x ^ {\star}) ^ {\top} A ] _ {i + 1} \leq 2 (4 i + 2) ^ {2} \epsilon
$$
Now consider the sum of the utilities of columns  $k \in [i + 2, n - i - 1]$ . By the construction of the payoff matrix  $A$  we can easily establish the following claim.
Proposition A.4. The sum of utilities of columns  $k \in [i + 2, n - i - 1]$  satisfies the inequality,
$$
\sum_ {k = i + 2} ^ {n - i - 1} \left[ \left(x ^ {\star}\right) ^ {\top} A \right] _ {k} \geq \sum_ {k = i + 2} ^ {n - i - 1} x _ {k} ^ {\star}
$$
By Proposition A.4 we are ensured that