# Measuring the Non-Transitivity in Chess
## 3 Theoretical Formulations and Methods
#### 3.3.1 Payoff Matrix Construction
On the other hand, one of the most common issue faced by human chess players is the phenomenon of becoming stuck at a certain Elo rating [38; 39; 40]. Having performed experiments using real data, we also investigated whether non-transitivity has implications for a human player's skill or rating progression. In particular, we investigated whether a link exists between the non-transitivity of the strategy space of real-world chess to the phenomenon of players getting stuck at certain Elo ratings. This was conducted by comparing the histogram of player ratings every year using the data from Section 3.3.1, with the corresponding Nash Clustering plot. ### Algorithm Hyperparameter Considerations

Algorithms 2 and 3, introduced in Sections 3.3.1 and 3.5, respectively, contain hyperparameters, and the choice of values for these hyperparameters merits further discussion. In Algorithm 2, the hyperparameter is the size of the payoff matrix \(m\). The time complexity of Algorithm 2 with respect to the payoff matrix dimension \(m\) is \(\mathcal{O}(m^{2})\). Therefore, having a large payoff matrix would severely slow down the payoff matrix construction. However, using a small payoff matrix would result in more data being unused. Algorithm 2 constructs a payoff matrix that is skew-symmetric and, so, the diagonal of the payoff matrix is be \(0\). As \(m\) gets smaller, the bin widths get wider and, thus, a wider range of Elo ratings are absorbed into a single bin. As a result, the non-diagonal elements of the matrix would correspond to two-way match-ups between two players with a more significant difference in Elo ratings, which is less likely to be present in the actual data, as players are typically matched with other players having similar Elo ratings. Therefore, a majority of the non-diagonal elements of the matrix will be filled using the Elo formula in Equation (3), instead of real data, which effectively turns the game into a purely transitive Elo game [5]. This leads to us not being able to observe the spinning top geometry in the strategy space occupied by real-world strategies. In our experiments, we used an \(m\) of \(1500\), as this was sufficiently large to demonstrate the existence of the spinning top geometry, while keeping the payoff matrix construction reasonably fast. In Algorithm 3, the hyperparameters are the population size \(k\) and the number of rounds \(n\). The time complexity of Algorithm 3 is more affected by the number of rounds \(n\), on the order of \(\mathcal{O}(n)\). While it grows slower than the time complexity of Algorithm 2 with respect to \(m\), it is still desirable to keep \(n\) relatively small, as fixed-memory Fictitious Play would have to be carried out for a number of population sizes. However, \(n\) should be large enough to demonstrate the convergence behaviour of the training process, if any. From our experiments, we found that 2000 rounds were sufficient for this purpose. The choice of population sizes is elaborated more in Section 4.3. ## 4 Results and Discussions

### Non-Transitivity Quantification

Following Sections 3.3.2 and 3.3.3, non-transitivity quantification was carried out on Lichess game data from 2013 to 2020, and FICS 2019 data, using the average Elo rating as a measure of transitive strength, and the Nash Cluster size and number of RPS cycles as measures of non-transitivity. The results are as shown in Figure 3.

Figures 3 and A1 illustrate that the spinning top geometry was observed in both Nash Clustering and RPS Cycles plots, for all of the considered years. Furthermore, both the Nash cluster size and number of RPS cycles peaked at a similar Elo rating range (1300-1700) for all of the years. This provides comprehensive evidence that the strategies adopted in real-world games do not vary throughout the years or from one online platform to another, and that the non-transitivity of real-world strategies is highest in the Elo rating range of 1300-1700.