SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \sum_ {k = i + 2} ^ {n - i - 1} \left[ \left(x ^ {\star}\right) ^ {\top} A \right] _ {k} \geq \sum_ {k = i + 2} ^ {n - i - 1} x _ {k} ^ {\star} \\ = \quad 1 - \sum_ {k = 1} ^ {i + 1} x _ {j} ^ {\star} - \sum_ {k = n - i} ^ {n} x _ {j} ^ {\star} \\ = \quad 1 - \left(x _ {i + 1} ^ {\star} + x _ {n - i} ^ {\star}\right) - \left(\sum_ {k = 1} ^ {i} x _ {i} ^ {\star} + \sum_ {k = n - i + 1} ^ {n} x _ {j} ^ {\star}\right) \\ \geq 1 - \left((4 i + 2) \epsilon + \epsilon\right) - \left(\sum_ {k = 1} ^ {i} x _ {i} + \sum_ {k = n - i + 1} ^ {n} x _ {j} ^ {\star}\right) \\ \geq 1 - (5 n + 3) \epsilon \\ \end{array}
$$
where the second to last inequality follows by the facts:  $x_{i+1}^{\star} + x_{n-i-1}^{\star} \leq (4i + 2)\epsilon$  and so  $x_{i+1}^{\star} \leq (4i + 2)\epsilon$ , and the Inductive Step Case 1,  $x_{n-i}^{\star} \leq \epsilon$ . Moreover, the last inequality holds by the Inductive Hypothesis:  $x_{k}^{\star} \leq \epsilon$  for all  $k \in [1,i] \cup [n - i + 1,n]$ .
$$
\sum_ {k = i + 2} ^ {n - i - 1} \left[ \left(x ^ {\star}\right) ^ {\top} A \right] _ {k} \geq 1 - (5 n + 3) \epsilon
$$
we are ensured that there exists a column  $k^{\star} \in [i + 2, n - i - 1]$  with utility  $\left[(x^{\star})^{\top}A\right]_{k^{\star}} \geq \frac{1 - (5n + 3)\epsilon}{n}$ . Now consider the difference between the utility of column  $k^{\star}$  and the column  $i + 1$ .
$$
[ (x ^ {\star}) ^ {\top} A ] _ {k ^ {\star}} - [ (x ^ {\star}) ^ {\top} A ] _ {i + 1} \geq \frac {1 - (5 n + 3) \epsilon}{n} - 2 (4 i + 2) ^ {2} \epsilon \geq \frac {1 - (5 n + 3) \epsilon}{n} - 2 (4 n + 2) ^ {2} \epsilon \geq \epsilon
$$
where the last inequality follows by the fact that  $\epsilon \leq 1 / 56n^3$ . Hence if the column player puts  $y_{i + 1}^{\star}$  probability mass to column  $k^{\star}$  then it increases its payoff by  $y_{i + 1}^{\star}([ (x^{\star})^{\top}A]_{k^{\star}} - [(x^{\star})^{\top}A]_{i + 1}) > \epsilon^{2}$ . The latter contradicts with the assumption that  $(x^{\star}, y^{\star})$  is an  $\epsilon^2$ -NE. Thus we have reached to a final contradiction that  $y_{i + 1}^{\star} > \epsilon$ .
Case 3:  $x_{i + 1}^{\star}\leq \epsilon$
Let assume that  $x_{i+1}^{\star} > \epsilon$  and we will reach a contradiction. From Observation 3.3, we notice that the utility of row  $i + 1$  equals,
Next, we examine the row  $n - (i + 1)$  in the inner submatrix. If this row is not well-defined, it implies that the inductive step  $j = i$  has reached the  $2 \times 2$  submatrix. In this case, the proof of Lemma 3.6 has already been completed. Otherwise, the utility of row  $n - (i + 1)$  equals
$$
[ A y ^ {\star} ] _ {n - (i + 1)} = (4 (i + 1) + 1) y _ {(i + 1) + 1} ^ {\star} + (4 (i + 1)) y _ {n - (i + 1) + 1} ^ {\star} \geq (4 i + 4) y _ {n - i} ^ {\star}
$$
As a result, by taking the difference on the utilities of row  $i + 1$  and  $n - (i + 1)$  we get,
$$
\begin{array}{l} [ A y ^ {\star} ] _ {n - (i + 1)} - [ A y ^ {\star} ] _ {i + 1} = (4 i + 4) y _ {n - i} ^ {\star} - (4 i + 2) y _ {i + 1} ^ {\star} - (4 i + 3) y _ {n - i} ^ {\star} \\ = y _ {n - i} ^ {\star} - (4 i + 2) y _ {i + 1} ^ {\star} \\ \geq y _ {n - i} ^ {\star} - (4 i + 2) \epsilon \\ \end{array}
$$
where the last inequality follows by the fact that  $y_{i + 1}^{\star}\leq \epsilon$  (Inductive Step Case 2). As a result, we conclude that
$$
\left[ A y ^ {\star} \right] _ {n - (i + 1)} - \left[ A y ^ {\star} \right] _ {i + 1} \geq y _ {n - 1} ^ {\star} - (4 i + 2) \epsilon
$$
In case  $y_{n - i}^{\star} \geq (4i + 3)\epsilon$  then  $[Ay^{\star}]_{n - (i + 1)} - [Ay^{\star}]_{i + 1} \geq \epsilon$ . Hence, if the row player puts  $x_{i + 1}^{\star}$  probability mass to row  $n - (i + 1)$  then it increases its payoff by  $x_{i + 1}^{\star}([Ay^{\star}]_{n - (i + 1)} - [Ay^{\star}]_{i + 1}) > \epsilon^{2}$ . The latter contradicts with the assumption that  $(x^{\star},y^{\star})$  is an  $\epsilon^2$ -NE. Thus, we conclude the following two statements:
$$
y _ {n - i} ^ {\star} \leq (4 i + 3) \epsilon \quad \text {a n d} \quad [ A y ^ {\star} ] _ {i + 1} \leq 2 (4 i + 3) ^ {2} \epsilon
$$
Now consider the sum of the utilities of rows  $k \in [i + 2, n - i - 1]$ . From Proposition A.3
$$
\begin{array}{l} \sum_ {k = i + 2} ^ {n - i - 1} \left[ A y ^ {\star} \right] _ {k} = \sum_ {k = i + 2} ^ {n - i - 1} y _ {k} ^ {\star} = 1 - \left(\sum_ {k = 1} ^ {i + 1} y _ {k} ^ {\star} + \sum_ {k = n - i} ^ {n} y _ {k} ^ {\star}\right) \geq 1 - y _ {n - i} ^ {\star} - \left(\sum_ {k = 1} ^ {i + 1} y _ {k} ^ {\star} + \sum_ {k = n - i + 1} ^ {n} y _ {k} ^ {\star}\right) \\ \geq 1 - (4 i + 3) \epsilon - n \epsilon = 1 - (5 n + 3) \epsilon \\ \end{array}
$$
Thus, we are ensured that there exists a row  $k^{\star} \in [i + 2, n - i - 1]$  with utility  $[Ay^{\star}]_{k^{\star}} \geq \frac{1 - (5n + 3)\epsilon}{n}$ . Now consider the difference between the utility of row  $k^{\star}$  and the row  $n - i$ .
$$
[ A y ^ {\star} ] _ {k ^ {\star}} - [ A y ^ {\star} ] _ {i + 1} \geq \frac {1 - (5 n + 3) \epsilon}{n} - 2 (4 i + 3) ^ {2} \epsilon \geq \frac {1 - (5 n + 3) \epsilon}{n} - 2 (4 n + 3) ^ {2} \epsilon \geq \epsilon
$$
where the last inequality follows by the fact that  $\epsilon \leq 1 / 56n^3$ . Hence if the row player puts  $x_{i + 1}^{\star}$  probability mass to row  $k^{\star}$  then it increases its payoff by  $x_{i + 1}^{\star}([Ay^{\star}]_{k^{\star}} - [Ay^{\star}]_{i + 1}) > \epsilon^{2}$ . The
latter contradicts with the assumption that  $(x^{\star},y^{\star})$  is an  $\epsilon^2$ -NE. Thus we have reached to a final contradiction that  $x_{i + 1}^{\star} > \epsilon$ .
Case 4:  $y_{n - i}^{\star}\leq \epsilon$
Let assume that  $y_{n - i}^{\star} > \epsilon$  and we will reach a contradiction. From Observation 3.3, we notice that the utility of column  $n - i$  equals,
$$
[ (x ^ {\star}) ^ {\top} A ] _ {n - i} = (4 i + 3) x _ {i + 1} ^ {\star} + (4 i + 4) x _ {n - (i + 1)} ^ {\star}
$$
Now, we consider the column  $(i + 1) + 1$  in the inner submatrix. In case this column is not well-defined, it means that the inductive step  $j = i$  has reached the  $2 \times 2$  submatrix and so the proof of the Lemma 3.6 has already been completed. Otherwise, the utility of column  $(i + 1) + 1$  equals
$$
[ (x ^ {\star}) ^ {\top} A ] _ {(i + 1) + 1} = (4 (i + 1) + 2) x _ {(i + 1) + 1} ^ {\star} + (4 (i + 1) + 1) x _ {n - (i + 1) + 1} ^ {\star} \geq (4 i + 5) x _ {n - i} ^ {\star}
$$
As a result, by taking the difference on the utilities of columns  $i + 1$  and  $n - (i + 1)$  we get,
$$
\begin{array}{l} [ (x ^ {\star}) ^ {\top} A ] _ {(i + 1) + 1} - [ (x ^ {\star}) ^ {\top} A ] _ {n - i} = (4 i + 5) x _ {n - i} ^ {\star} - (4 i + 3) x _ {i + 1} ^ {\star} - (4 i + 4) x _ {n - (i + 1)} ^ {\star} \\ = x _ {n - i} ^ {\star} - (4 i + 3) x _ {i + 1} ^ {\star} \\ \geq x _ {n - i} ^ {\star} - (4 i + 3) \epsilon \\ \end{array}
$$
where the last inequality follows by the fact that  $x_{i+1}^{\star} \leq \epsilon$  (Inductive Step Case 3). As a result, we conclude that
$$
[ (x ^ {\star}) ^ {\top} A ] _ {(i + 1) + 1} - [ (x ^ {\star}) ^ {\top} A ] _ {n - i} \geq x _ {n - i} ^ {\star} - (4 i + 3) \epsilon
$$
In case  $x_{n - i}^{\star} \geq (4i + 4)\epsilon$  then  $[(x^{\star})^{\top}A]_{(i + 1) + 1} - [(x^{\star})^{\top}A]_{n - i} \geq \epsilon$ . Hence if the column player puts  $y_{n - i}^{\star}$  probability mass to column  $(i + 1) + 1$  then it increases its payoff by  $y_{n - i}^{\star}([(x^{\star})^{\top}A]_{(i + 1) + 1} - [(x^{\star})^{\top}A]_{n - i}) > \epsilon^{2}$ . The latter contradicts with the assumption that  $(x^{\star},y^{\star})$  is an  $\epsilon^2$ -NE. Thus we conclude in the following two statements:
$$
x _ {n - i} ^ {\star} \leq (4 i + 4) \epsilon \quad \text {a n d} \quad \left[ \left(x ^ {\star}\right) ^ {T} A \right] _ {n - i} \leq 2 (4 i + 4) ^ {2} \epsilon
$$
Now consider the sum of the utilities of columns  $k \in [i + 2, n - i - 1]$ . From Proposition A.4
$$
\begin{array}{l} \sum_ {k = i + 2} ^ {n - i - 1} \left[ \left(x ^ {\star}\right) ^ {\top} A \right] _ {k} = \sum_ {k = i + 2} ^ {n - i - 1} x _ {k} ^ {\star} = 1 - \left(\sum_ {k = 1} ^ {i + 1} x _ {k} ^ {\star} + \sum_ {k = n - i} ^ {n} x _ {k} ^ {\star}\right) \\ \geq \quad 1 - x _ {n - i} ^ {\star} - \left(\sum_ {k = 1} ^ {i + 1} x _ {j} ^ {\star} + \sum_ {k = n - i + 1} ^ {n} x _ {j} ^ {\star}\right) \\ \geq 1 - (4 i + 4) \epsilon - n \epsilon = 1 - (5 n + 4) \epsilon \\ \end{array}
$$
Thus, we are ensured that there exists a column  $k^{\star} \in [i + 2, n - i - 1]$  with utility  $\left[(x^{\star})^{\top} A\right]_{k^{\star}} \geq \frac{1 - (5n + 4)\epsilon}{n}$ . Now consider the difference between the utility of column  $k^{\star}$  and the column  $n - i$ .
$$
[ (x ^ {\star}) ^ {\top} A ] _ {k ^ {\star}} - [ (x ^ {\star}) ^ {\top} A ] _ {n - i} \geq \frac {1 - (5 n + 4) \epsilon}{n} - 2 (4 i + 4) ^ {2} \epsilon \geq \frac {1 - (5 n + 4) \epsilon}{n} - 2 (4 n + 4) ^ {2} \epsilon \geq \epsilon
$$
where the last inequality follows by the fact that  $\epsilon \leq 1 / 56n^3$ . Hence if the column player puts  $y_{n - i}^{\star}$  probability mass to column  $k^{\star}$  then it increases its payoff by  $y_{n - i}^{\star}\left([(x^{\star})^{\top}A]_{k^{\star}} - [(x^{\star})^{\top}A]_{n - i}\right) > \epsilon^2$ . The latter contradicts with the assumption that  $(x^{\star}, y^{\star})$  is an  $\epsilon^2$ -NE. Thus we have reached to a final contradiction that  $y_{n - i}^{\star} > \epsilon$ .
A.1 Proof of Theorem A.3
Proposition A.5. The sum of utilities of rows  $k \in [i + 2, n - i - 1]$  admits,
Proof. By Observation 3.3 we derive the following equation.
$$
{[ A y ^ {\star} ] _ {i + 2}} = {[ A y ^ {\star} ] _ {(i + 1) + 1} = (4 i + 2) y _ {(i + 1) + 1} ^ {\star} + (4 i + 3) y _ {n - (i + 1)} ^ {\star} \geq y _ {(i + 1) + 1} ^ {\star}}
$$
The claim can be immediately derived from the inequality given above,  $\sum_{k = i + 2}^{n - i - 1}[Ay^{\star}]_k\geq \sum_{k = i + 2}^{n - i - 1}y_k^{\star}$
A.2 Proof of Theorem A.4
Proposition A.6. The sum of utilities of columns  $k \in [i + 2, n - i - 1]$  admits,
$$
{[ (x ^ {\star}) ^ {\top} A ] _ {i + 2}} = {[ (x ^ {\star}) ^ {\top} A ] _ {(i + 1) + 1} \geq (4 i + 2) x _ {(i + 1) + 1} ^ {\star} + (4 i + 3) x _ {n - (i + 1)} ^ {\star} \geq x _ {(i + 1) + 1} ^ {\star}}
$$
The claim can be immediately derived from the inequality given above,  $\sum_{k = i + 2}^{n - i - 1}[(x^{\star})^{\top}A]_k\geq \sum_{k = i + 2}^{n - i - 1}x_k^{\star}$
B Proof of Lemma 3.8
B.1 Omitted Proofs of Section 3.3
Proposition B.1 (Proof of Proposition 3.10). Let  $(i^{(t)}, j^{(t)})$  be a strategy selected by fictitious play at round  $t$ , and  $(i^{(t)}, j^{(t)}) \neq \left(\frac{n}{2}, \frac{n}{2}\right)$ . Then, in a subsequent round, fictitious play will choose the strategy of greater value that is either on row  $i^{(t)}$  or column  $j^{(t)}$ .
Proof. To establish the claim, we employ the concept of a cumulative utility vector, as defined in Definition 2.7. According to Proposition 3.5, the row  $i^{(t)}$  and column  $j^{(t)}$  combined have three distinct non-zero elements. Without loss of generality, let's assume that the greater element is in column  $j^{(t)}$ , but in a different row, denoted as  $i'$ .
Firstly, we observe that any subsequent strategy will involve only those three elements. This is because in the cumulative utility vector, which determines the strategy to be played in each round, only the coordinates corresponding to those elements are updated as long as the strategy  $(i^{(t)},j^{(t)})$  is being played.
Moreover, we can demonstrate that among these elements, the one with the greater value will be played next, and this transition is deterministic. This means that the row player will choose the strategy associated with the greater element. We note that this decision is implicitly affected by the strategy of column player.
Let's first exclude the case where the next strategy switch involves the column player. We initially assumed that the greatest element is in column  $j^{(t)}$  but on a different row. Consequently, the only non-zero element in a different column than  $j^{(t)}$  must have a smaller utility compared to the element  $(i^{(t)}, j^{(t)})$ . Therefore, there is no incentive to switch to a strategy with lower utility. This confirms that the column player will not opt for a different strategy, ensuring that the next strategy switch, if it occurs, will necessarily involve the row player.
Regarding the row player, we aim to prove that there will be a round where they will change to a different strategy, and consequently, to a different row. To analyze this, let's examine how the cumulative utility vector of the row player changes from round to round.
$$
i := i ^ {(t)} \in \operatorname * {a r g m a x} _ {i \in [ n ]} \left[ \sum_ {s = 1} ^ {t - 1} A e _ {j ^ {(s)}} \right] _ {i}
$$
Therefore, as long as the column player continues to use the same strategy, the row  $Ae_{j(t)}$  will repeatedly be added to the cumulative vector, reinforcing the coordinate of row  $i'$ . However, this cannot happen indefinitely, as the cumulative utility vector is bounded. After a certain number of rounds, the row player will eventually choose the strategy associated with row  $i'$ . This proves that claim for the case of the row player.
Similarly, if the greater element is located in the same row but on a different column, a similar argument can be made to prove that the column player will switch strategies next.
Corollary B.2 (Proof of Corollary 3.11). Let  $t$  be a round in which a player changes their strategy. Then exactly one of the following statements is true:
Proof. This corollary is a simple application of Proposition 3.10. We will only prove the first claim. Let  $t$  be the round when the row player changes their strategy, i.e.,  $i^{(t)} \neq i^{(t-1)}$ . According to Proposition 3.5, there are three non-zero elements combined in  $i^{(t)}, j^{(t)}$ . Since the row player changes their strategy, it follows from Proposition 3.10 that the other element in column  $j^{(t)}$  but not in row  $i^{(t)}$  must necessarily have a smaller value than  $(i^{(t)}, j^{(t)})$ . Therefore, the element with the greater value must necessarily be in a different column. Hence, by applying Proposition 3.10, we conclude that the column player can only make the next strategy switch. This proves the claim.
B.2 Auxiliary Propositions for Lemma 3.6
In this subsection, we provide the full version of the proposition used to establish the proof Lemma 3.6.
Proposition B.3. There exists a round  $T_{i}^{1} > T_{i}^{0}$  at which
(i) the strategy profile is  $(i + 1,i + 1)$  for the first time,
(ii) for all rounds  $t \in [T_i^0, T_i^1 - 1]$ , the strategy profile is  $(n - i, i + 1)$ ,
(iii) column  $i + 1$  admits cumulative utility  $C_{i + 1}^{(T_i^1)} \geq (4i + 1) \cdot (R_{i + 1}^{(T_i^0)} + 1)$ ,
(iv) all rows  $k \in [(i + 1) + 1, n - i - 1]$  admit  $R_k^{(T_i^1)} = 0$  and all columns  $k \in [i + 2, n - i]$  admit  $C_k^{(T_i^1)} = 0$ .
Proof. The proposition is composed of multiple parts, each of which is proven separately. To begin with, we must establish that the new strategy profile chosen by fictitious play will be  $(i + 1,i + 1)$ .
According to the inductive hypothesis in Theorem 3.1, we know that the strategy  $(n - i,i + 1)$  was played at round  $T_{i}^{0}$  for the first time, while any strategy involving a row  $\in [i + 1,n - (i + 1)]$  has not been played until that point. Additionally, the inductive hypothesis also states that the strategy played before time  $T_{i}^{0}$  was  $(n - i,n - i + 1)$ . Therefore, according to Corollary 3.11, it is guaranteed that the next strategy switch will be initiated by the row player.
In other words, as long as the column player continues to play their current strategy, strategy  $(n - i,i + 1)$  will be played in every subsequent round, which establishes Item (ii). This implies that the row player's cumulative utility will increase by  $Ae_{i + 1}$ . As per Observation 3.3, column  $i + 1$  only has non-zero elements in positions  $i + 1$  and  $n - i$ .
$$
A e _ {i + 1} = \left[ 0, \dots , \underbrace {4 i + 2} _ {i + 1}, 0, \dots , 0, \underbrace {4 i + 1} _ {n - i}, 0, \dots \right] \tag {5}
$$
Since strategy  $i + 1$  has a higher value, a strategy switch in a later time step is certain. Consequently, there will be a round  $T_{i}^{1}$  in which the strategy  $(i + 1, i + 1)$  will be played for the first time, establishing Item (i).
We must determine the point at which the strategy switch to  $(i + 1, i + 1)$  will take place. According to Equation (5), the value of strategy  $i + 1$  is exactly one greater than the value of strategy  $n - i$ . From the inductive hypothesis, we know that row  $i + 1$  has a cumulative utility of zero, whereas row  $n - i$  has a cumulative utility of  $R_{i + 1}^{(T_i^0)}$ . Therefore, it will take precisely  $R_{i + 1}^{(T_i^0)}$  steps for those strategies to have equal cumulative utility values. Consequently, the strategy switch will either occur in that round or the immediate next, as the cumulative utility of row  $i + 1$  will have surpassed that of row  $n - i$ .
In order to proceed with Item (iii), we compute the updated cumulative utility of column  $(i + 1)$ . As shown in Equation (5), column  $i + 1$  has a value of  $(4i + 1)$  at position  $n - i$ . Thus, if row  $n - i$  has been played for a minimum of  $R_{i + 1}^{(T_i^0)}$  rounds, then the cumulative utility of column  $i + 1$  is greater than  $(4i + 1) \cdot R_{i + 1}^{(T_i^0)}$ . Given that row  $n - i$  has also been played previously (inductive hypothesis), it is reasonable to conclude that:
$$
C _ {i + 1} ^ {(T _ {i} ^ {1})} \geq (4 i + 1) \cdot (R _ {i + 1} ^ {(T _ {i} ^ {0})} + 1)
$$
Lastly, according to Observation 3.4, there exists a non-zero element in row  $i + 1$  at position  $i + 1$ . Due to the fact that column  $i + 1$  has already been played (according to Item (ii)), we can conclude that the cumulative utility of row  $i + 1$  must be non-zero. Combining this with inductive hypothesis, it concludes the proof of Item (iv).
B.2.1 Proof of Theorem 3.13
Proposition B.4. There exists a round  $T_{i}^{2} > T_{i}^{1}$  at which
(i) the strategy profile is  $(i + 1, n - i)$  for the first time,
(ii) for all rounds  $t \in [T_i^1, T_i^2 - 1]$ , the strategy profile is  $(i + 1, i + 1)$ ,
(iii) row  $i + 1$  admits cumulative utility  $R_{i + 1}^{(T_i^2)} \geq (4i + 2) \cdot C_{i + 1}^{(T_i^1)}$ ,
(iv) all rows  $k \in [(i + 1) + 1, n - i - 1]$  admit  $R_k^{(T_i^2)} = 0$  and all columns  $k \in [i + 2, n - (i + 1)]$  admit  $C_k^{(T_i^2)} = 0$ .
Proof. We repeat the same reasoning as in the proof of Proposition 3.12. To begin with, we must establish that the new strategy profile chosen by fictitious play will be  $(i + 1, n - i)$ .
We can infer from both Items (i) and (ii) in Proposition 3.12 that the most recent strategy switch was made by the row player. Therefore, based on Corollary 3.11, we are certain that the next strategy switch will be initiated by the column player.
In other words, as long as the row player continues to play their current strategy, strategy  $(i + 1,i + 1)$  will be played in every subsequent round, which establishes Item (ii). This implies that the column player's cumulative utility will increase by  $e_{i + 1}^{\top}A$ . As per Observation 3.3, row  $i + 1$  only has non-zero elements at positions  $i + 1$  and  $n - i$ .
$$
e _ {i + 1} ^ {\top} A = \left[ 0, \dots , \underbrace {4 i + 2} _ {i + 1}, 0, \dots , 0, \underbrace {4 i + 3} _ {n - i}, 0, \dots \right] \tag {6}
$$
Since strategy  $n - i$  has a higher value, a strategy switch in a later time step is certain. Consequently, there will be a round  $T_{i}^{2}$  in which the strategy  $(i + 1, n - i)$  will be played for the first time, establishing Item (i).
We must determine the point at which the strategy switch to  $(i + 1, n - i)$  will take place. According to Equation (6), the value of strategy  $n - i$  is exactly one greater than the value of strategy  $i + 1$ . From the Proposition 3.12, we know that column  $n - i$  has a cumulative utility of zero, whereas column  $i + 1$  has a cumulative utility of  $C_{i + 1}^{(T_i^1)}$ . Therefore, it will take precisely  $C_{i + 1}^{(T_i^1)}$  steps for those strategies to have equal cumulative utility values. Consequently, the strategy switch will either occur in that round or the immediate next, as the cumulative utility of column  $n - i$  will have surpassed that of column  $i + 1$ .
In order to proceed with Item (iii), we compute the updated cumulative utility of row  $(i + 1)$ . As shown in Equation (6), row  $i + 1$  has a value of  $(4i + 2)$  at position  $i + 1$ . Thus, if column  $i + 1$  has been played for a minimum of  $C_{i + 1}^{(T_i^1)}$  rounds, then the cumulative utility of row  $i + 1$  satisfies  $R_{i + 1}^{(T_i^2)} \geq (4i + 2) \cdot C_{i + 1}^{(T_i^1)}$ .
Lastly, according to Observation 3.4, there exists a non-zero element in column  $n - i$  at position  $i + 1$ . Due to the fact that row  $i + 1$  has already been played (according to Item (ii)), we can conclude that the cumulative utility of column  $n - i$  must be non-zero. Combining this with Proposition 3.12, it concludes the proof of Item (iv).
B.2.2 Proof of Theorem 3.14
Proposition B.5. There exists a round  $T_{i}^{3} > T_{i}^{2}$  at which
(i) the strategy profile is  $(n - (i + 1), n - i)$  for the first time,
(ii) for all rounds  $t \in [T_i^2, T_i^3 - 1]$ , the strategy profile is  $(i + 1, n - i)$ ,
(iii) column  $n - i$  admits cumulative utility  $C_{n - i}^{(T_i^3)} \geq (4i + 3) \cdot R_{i + 1}^{(T_i^2)}$ ,
(iv) all rows  $k \in [(i + 1) + 1, n - (i + 1) - 1]$  admit  $R_k^{(T_i^3)} = 0$  and all columns  $k \in [i + 2, n - (i + 1)]$  admit  $C_k^{(T_i^3)} = 0$ .
Proof. We repeat the same reasoning as in the proof of Proposition 3.12. To begin with, we must establish that the new strategy profile chosen by fictitious play will be  $(n - i - 1, n - i)$ .
We can infer from both Items (i) and (ii) in Proposition 3.13 that the most recent strategy switch was made by the column player. Therefore, based on Corollary 3.11, we are certain that the next strategy switch will be initiated by the row player.
In other words, as long as the column player continues to play their current strategy, strategy  $(i + 1, n - i)$  will be played in every subsequent round, which establishes Item (ii). This implies that the row player's cumulative utility will increase by  $Ae_{n - i}$ . As per Observation 3.3, column  $n - i$  only has non-zero elements at positions  $i + 1$  and  $n - (i + 1)$ .
$$
A e _ {n - i} = [ 0, \dots , \underbrace {4 i + 3} _ {i + 1}, 0, \dots , 0, \underbrace {4 i + 4} _ {n - (i + 1)}, 0, \dots ] \tag {7}
$$
Since strategy  $n - (i + 1)$  has a higher value, a strategy switch in a later time step is certain. Consequently, there will be a round  $T_{i}^{3}$  in which the strategy  $(n - i - 1, n - i)$  will be played for the first time, establishing Item (i).
We must determine the point at which the strategy switch to  $(n - i - 1, n - i)$  will take place. According to Equation (7), the value of strategy  $n - (i + 1)$  is exactly one greater than the value of strategy  $i + 1$ . From the Proposition 3.13, we know that row  $n - (i + 1)$  has a cumulative utility of zero, whereas row  $i + 1$  has a cumulative utility of  $R_{i + 1}^{(T_i^2)}$ . Therefore, it will take precisely  $R_{i + 1}^{(T_i^2)}$  steps for those strategies to have equal cumulative utility values. Consequently, the strategy switch will either occur in that round or the immediate next, as the cumulative utility of row  $n - (i + 1)$  will have surpassed that of row  $i + 1$ .
In order to proceed with Item (iii), we compute the updated cumulative utility of column  $n - i$ . As shown in Equation (7), column  $n - i$  has a value of  $(4i + 3)$  at position  $i + 1$ . Thus, if row  $i + 1$  has been played for a minimum of  $R_{i + 1}^{(T_i^2)}$  rounds, then the cumulative utility of column  $n - i$  satisfies  $C_{n - i}^{(T_i^3)} \geq (4i + 3) \cdot R_{i + 1}^{(T_i^2)}$ .