Maia-2: A Unified Model for Human-AI Alignment in Chess
Authors: Zhenwei Tang
Difan Jiao
Reid McIlroy-Young
Jon Kleinberg
Siddhartha Sen
Ashton Anderson
Venue: NeurIPS 2024 poster (2024)
Keywords: Human-AI Alignment
Action Prediction
Chess
Skill-aware Attention
Primary area: human-AI_interaction

Abstract: There are an increasing number of domains in which artificial intelligence (AI) systems both surpass human ability and accurately model human behavior. This introduces the possibility of algorithmically-informed teaching in these domains through more relatable AI partners and deeper insights into human decision-making. Critical to achieving this goal, however, is coherently modeling human behavior at various skill levels. Chess is an ideal model system for conducting research into this kind of human-AI alignment, with its rich history as a pivotal testbed for AI research, mature superhuman AI systems like AlphaZero, and precise measurements of skill via chess rating systems. Previous work in modeling human decision-making in chess uses completely independent models to capture human style at different skill levels, meaning they lack coherence in their ability to adapt to the full spectrum of human improvement and are ultimately limited in their effectiveness as AI partners and teaching tools. In this work, we propose a unified modeling approach for human-AI alignment in chess that coherently captures human style across different skill levels and directly captures how people improve. Recognizing the complex, non-linear nature of human learning, we introduce a skill-aware attention mechanism to dynamically integrate players’ strengths with encoded chess positions, enabling our model to be sensitive to evolving player skill. Our experimental results demonstrate that this unified framework significantly enhances the alignment between AI and human players across a diverse range of expertise levels, paving the way for deeper insights into human decision-making and AI-guided teaching tools.

Maia-2: A Unified Model for Human-AI Alignment in Chess
Zhenwei Tang University of Toronto
Difan Jiao
University of Toronto
j josephtang9cs. toronto. edu
difanjiao@cs, toronto.
Reid Mcllroy- Mcllroy-Young
Jon Kleinberg
Siddhartha Sen
Harvard University
Comell University
Microsoft Research
reidmcy@seas harvardd
kleinberg&cornell.cccm
Sidsen@microsoff com
Ashton Anderson
University of Toronto
ashtonêcs. toronto.eeee edu
Abstract
There increasing number of domains in which artificial intelligence (AI) systems both surpass human ability and accurately model human behavior. This introduces the possibility of algorihmiiaall-inforre through more relatable AI partners and deeper insights into human decision-making. Critical to achieving this goal, however, is coherently modeling human behavior at various skill levels. Chess is an ideal model system for conducting research into this kind of human-AI alignment, with its rich history a pivotal testbed for Al research, mature superhuman AI systems like AlphaZero, and precise measurements of skill via chess rating systems. Previous work in modeling human decision-making in chess uses completely independent models to capture human style at different skill levels, meaning they lack coherence in their ability to adapt to the full spectrum of human improvement and are ultimately limited in their effectiveness as AI partners and teaching tools. this work, we propose unified modeling approach for human-Al alignment in chess that coherently captures human style across different skill levels and directly captures how people improve. Recognizing the complex, non- non-linear nature of human learning, introduce skill-aware attention mechanism to dynamically integrate players' strengths with encoded chess positions, enabling model to be sensitive to evolving player skill. Our experimental results demonstrate that this unified framework significantly enhances the alignment between AI and human players across diverse range of expertise levels, paving the way for deeper insights into human decision-making and Al Al-guided teaching tools. Our implementation is available here
teaching in these domains
Introduction
There are an increasing number of domains which artificial intelligence (AI) systems both surpass human ability and accurately model human behavior. This combination of machine mastery over a domain and computational understanding of human behavior in introduces the possibility of algoritmmically-ifoomeee teaching and learning. Al-powered aids could guide people along reliable and efficient improvement paths, synthesized from their knowledge of both human trajectories and
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

objective performance. Relatable Al partners, on the other hand, could learn to act alongside human counterparts in synergistic and complementary ways.
Researchers have begun to tackle this challenge in the model system of chess. Once held to be an ideal testbed for developing artificial intelligence, it is now the perfect domain pursue Auman-Al alignment. The AI community finally surpassed all human ability in chess approximately 20 years ago, milestone achievement and watershed cultural moment. Now, superhuman Al chess engines are ubiquitous and widely used. Despite this transformation, chess has never been more popular, becoming a mainstream activity in many countries during the last few years. There is now both unprecedented demand for chess education, as well as mature superhuman AI that could in principle help meet it.
However, existing models fall short of being effective learning tools and relatable partners. Traditional chess engines such Stockfish and AlphaZero are unimaginably strong, but they don't play in ways that humans can easily understand or learn from. Comparing one's own decisions with those of traditional engines, it is easy to see how near-perfect AI would have improved upon your play but hard to see how you could realistically do the same. Recent work has resulted in the development of Maia, suite of models that aim mimic human behavior in chess at various skill levels by learning to predict actual human moves from a wealth of online gameplay data M. While substantially more human-like these models still cannot power effective algorithmic teaching tools because of several limitations.
First and foremost, Maia models players at different skill levels completely independently; games by players at one skill level and those by an adjacent skill level are fed into separate instances of the same architecture and result in separate models. This has the downside that predictions from one model are independently made of predictions from any other. Viewed as a whole, they are volatile: the Maia models might predict that at one level players will approach a position correctly, then the next level they will make a horrible mistake, then at the next level they will do fine again, and so on. In word, they fail cohere. People don't improve along volatile paths, they steadily get better. The unrealistically incoherent predictions made by separate models don't 't suggest realistic pathways that people can take order to get better. In order to serve as algorithmic teachers or learning aids, our models of human behavior must be coherent.
Building coherent model of human skill in chess is difficult, because the breadth of skill in chess is almost incomprehensibly large. Decisions made by beginners bear only the faintest of relations to those made by masters. A difference of 200 points in chess rating systems roughly equates win rate for the higher-rated player-typically higher than the best record of any team in the entire National Basketball Association. On the online chess platform we study, there are players who are 2600 rating points apart-or 13 successive steps of 75%-vs.-25% dominance apart from each other. Capturing this breadth of skill in a single model, in a coherent, smooth fashion, is a challenge.
75%
We contribute unified modeling approach for human-Al alignment in chess that coherently captures human style across different skill levels and directly captures how people improve. Since model builds directly on original Maia, we call Maia-2. Maia-2 consists of a standard residual network tower that processes chess positions into features, and our novel contribution of a skill-aware attention module with channel-wit patching. This innovation takes the position representation outputted by the residual network tower and simple player skill encodings and learns how player skill levels interact with chess positions to produce the moves humans make. Unlike previous models, Maia-2 only requires the current board position as input (as opposed to six), which dramatically reduces training time and increases flexibility (c.g. for applying the model in non-game contexts where there may be no 6-board history). In addition to policy and value heads like previous work, we also add an additional auxiliary information head that helps the model learn a deeper understanding of human chess moves.
We evaluate Maia-2 along two key dimensions: move prediction accuracy and coherence. Testing it against the original Maia models, Stockfish, and AlphaZero, Maia-2 emerges the most accurate human move predictor by far, surpassing original Maia by almost 2 full percentage points. Analyzing move prediction accuracy by skill level, Maia-2 matches and surpasses all other models on all skill levels. Furthermore, Maia-2's gains in perplexity are similarly striking, reducing average perplexity from a previous record of 4.67 bits down to 4.07 bits. Maia-2 achieves these accuracy gains while being substantially more coherent than the original Maia models. For example, call model's 's treatment of position monotonic it assigns a monotonically increasing probability to the

---

correct move as we increase skill. While original Maia treats 1% of random sample of positions monotonically, Maia-2 treats remarkable 27% of the same positions monotonically. This is in keeping with our intuitive understanding of how chess players steadily and smoothly improve across the skill range. Finally, we conduct an investigation of the human chess concepts Maia-2 learns and varies with skill via linear probes, and find that skill-dependent concepts like overall board evaluation indeed vary with skill, but skill-independent concepts do not, which also accords with our understanding of how human players make decisions.
2 Related Work
Chess and AI. This paper draws on the long history of chess at the forefront of Al research 2
3
We engage with 3 distinct approaches to building chess Al: heuristic [6], learned 7 and textual g Heuristic search: The original approach to computer chess was heuristics- based 4 91. This method
was famously used by IBM's Deep Blue to defeat Garry Kasparov [2] and is currently used by Stockfish [6, one of the strongest chess engines in the world. Learned search: Alpha(Zero) Go 7 IO] is set of neural networks that learn to play Go with methods that generalized to other games, including chess, with AlphaZero [TO]. Chess AI with learned search is also extended multi-agent systems [T]. where diverse AI systems can outperform a single Al in challenging tasks such as chess. Chess as text: Large language models [12]
13 [4] have recently been found perform well on tasks that the models were not explicitly trained on [5] including playing chess without fine-tuning [16] 17 [8]. This has lead to chess knowledge being one of the tested features in BIG- Bench [19. popular LLM evaluation suite. Additionally, fine-tuning a language model can lead to systems that not only play chess, but can also generate comments, describe positions, and create other simple analyses of a game 220 21
Human-Al Alignment in Chess. Building chess engine that can defeat any human has been a solved problem for over 20 years. This has led to a new research agenda in extracting useful knowledge from these superhuman systems. A direct way of doing this is to probe an Al chess engine in a human representation space, Without any prior human knowledge or guidance, evidence of human chess concepts learned by AlphaZero is found and measured by linear probes [22). Going further, AlphaZero also encodes knowledge that extends beyond existing human knowledge but is ultimately lcarnable by humans (3 Anotherr direction was the creation of a 'behavioral stylometry' model that can identify chess players from the moves they play [24]. An alternative approach to creating systems that can act as guides humans is demonstrated by Maia [ 23), in which a model is trained to predict the next move a human will play, instead of optimizing for winning the game. In addition predicting human actions the models have been fine-tuned to predict given player's The prediction accuracy can be improved via a reinforcement learning-ste search [26).
actions 25
3 Methodology
We propose a unified model architecture to capture human decision-making in chess across broad spectrum of skill levels. Since this model builds upon the previous Maia move-matching models, we call it Maia-2. As shown in Figure [ Maia-2 first encodes active and opponent skill levels and the chess positions, respectively. Then the encoded skill levels and positions are fused using our skill-aware attention with channel-wise patching architecture. The fused representations are then used for move prediction (policy head), auxiliary information prediction (auxiliary head), and game outcome prediction (value head). We now discuss each of these components in detail.
3.1
Skill Level Encoder
Instead of directly incorporating player ratings as numerical inputs, we use categorical skill level embeddings for two reasons. First, player behavior and decision-making in chess are not linearly related to their rating. Categorical embeddings allow for capturing complex, non-linear relationships between player strength and their moves. They can encode nuanced differences in play style and strategy that are not directly proportional to player ratings. Second, Generalization across similar skill levels: Players within certain skill level may exhibit similar playing styles, strategies, and common mistakes. Categorical embeddings group players into these ranges, helping the model
3

---

Ruiia Backlees
Skill-errrAAline Atfuntonn
SIOOO GIOEO
Pesi
Cundii & Catt Gaat
Sull-Aaart Quary
Tuckrg
x Cane Gual
Maiil beudital
Ohes in Oud
Xax
Facd
Figure 1: Overview of the Maia-2 model architecture.
better generalize across players with similar strengths, as opposed to treating each rating numerical input.
Let E R Blx,, be the matrix of player rating embeddings, where each row corresponds to the embedding of a skill level with dimension d,: E [000) e(1000, .... e(2000, +co)] Given the skill levels a and of an active player (i.e. the player to move) and the opponent player, we look up the embedding matrix E by rows to map the skill levels to active and opponent skill embeddings:
e, = E[a] e, E[o].
Note that previous work [ 26 uses completely independent models for human-Al alignment at different skill levels e.g. decisions by 1100-rated chess players are encoded in one model and decisions by 1500-rated players are encoded in separate model. Further, these models ignore opponent skill level, meaning that predictions cannot vary as a function However, the active player's decisions may be significantly affected by the opponent's 's skill level certain types of situations, or even general. Players may adjust their strategy based on their perception of the opponent's skill, c.g. a higher-skill opponent might prompt play, while against lower-skill opponent a player may pursue more aggressive tactics. Thus, the interaction between the skill levels of both players is important component of matching human moves. Unlike existing models that ignore opponent skill level (and actually only consider games in which both players the same skill level), explicitly model not only opponent skill but also the complex interplay between the two players' skill levels, and how affects human decision-making
opponent strength.
(or less) cautious
3.2 Position Encoder
Position representation. We use well-established method [10 to represent each chess position as a multi-channel tensor Pi Pimpat E Co x8x8, which includes channels for each type of chess piece, which color is move, and states of the position that are not derivable from the position alone (castling rights and en passant), where Chaant denotes the number of channels. One important departure from previous work is that we only use the current chess position, and not the last few chess positions that occurred in the game (models have typically incorporated the six most recent positions in the game). Many games with perfect information, including chess, can be modeled as alternating Markov games [27 77.. where future states are independent of past states given the current game state. Therefore, the current chess position theoretically encapsulates all the information necessary to make future decisions. Although human decision-making in chess may sometimes subtly depend on the historical lead-up to the current position, these effects are anecdotally small. In exchange, gain large practical benefits. First, modeling Al-human move matching in Markovian way vastly improves training efficiency by reducing the computational load via significantly smaller data usage for each decision. Second, it also enhances flexibility, enabling resulting model make predictions even without historical data, which is particularly advantageous in situations where only the current position is available, like chess training puzzles or any position that didn't necessarily occur in full game.
Position encoding. To process the position representation Pimput. we encode Panpat with the well- established ResNet-based 28 backbone architecture for chess position modeling with KCom sequen- tially connected blocks D: Penexded Backbone Ko.. (Pimpat E IR C x8x8 where Penculed denotes the encoded position representation of Cpatch channels. More details about position representation and the backbone architecture can be found in Appendix Section B

---

3.3 Bridging Skill Levels and Positions
central challenge we face is learning how players at different skill levels interact with chess positions differently. How does an expert player evaluate and process a chess position to come up with a move, and how does this differ from a novice? The relationship between positions and skill levels is complicated by the non-linearity in how players of various skill levels interpret and react to chess positions. This complexity presents significant challenge in human move prediction using a unified model for diverse skill levels. To bridge skill levels and positions decision-make makers and decisions we propose skill-aware attention with channel-wise patching.
Channel-wise patching. In contrast to the wrsa-wise patching approach in Vision Transform- (ViTs) [29], we employ channel-wise patching. Each channel is flattened and linearly transformed, regarding the number of channels in Pencoded. i.c., Cpench. as the sequence length: Paunched = Patching (Penodded) E R x64, P p= Paiched W + b E R Xda, where W E R64xd
and b E R dh denote the parameters of the linear projection from the patching dimension to the hidden dimension of the skill-aware attention blocks dan- This is particularly suitable for patching encoded chess positions as inputs to Transformer-like architectures, where channels are essentially feature maps that represent different learned latent concepts. These concepts in feature maps are then interactively selected and aggregated considering skill levels via skill-aware attention.
Skill-aware Attention. Given position representations Panched and skill level representations C, and Co, our proposed skill- aware multi- head self- atlf-attention is computed as follows. For each head k. we learn weight matrices 8 dimension of each hcad. The queries Qk. keys Kg, and values V& for each head are computed as: P Pusched W.K Vi2 Pranched WV- In order to fuse player skill levels and chess positions progressively and interactively, we inject skill level embeddings into queries within the multi-head self-attention: Qi Qx + (e,, @ e,, W*, where W R2d, xdh denotes the weight matrix for feature transformation to the query space, and ( is the concatenation operator. We choose to incorporate skill levels in queries because queries directly influence how attention is distributed across patched channels. Using skill-aware queries Qi. the attention mechanism can adjust its focus to reflect the strategic considerations and positional understanding of players different skill levels. This adjustment allows Maia-2 to adaptively prioritize features of the positions that are more relevant to the skill levels involved, enhancing the model's contextual sensitivity. The skill-aware scaled dot- dot-product
& E Rd Rduxds, dd W E Rddx and W E Rd where d,, denote the
Qk P putched W K₂
attention for each head is thus defined as: h4 softmax [2 K
Vidy Vx. The outputs of all heads hi hu,,....hhh hi, are concatenated and then linearly transformed: Pan [(h & h2 ... @ ha WO), where w E Rhdd,x Xd.n denote the weight matrix for multi-head attention and o(-) denotes the activation function. We apply the vanilla ViT's ford-forward network and add & norm components upon Pau to obtain the output of each skill-aware attention block Pau- In Maia-2, we employ sequence of aware attention blocks to progressively fuse skill levels and positions. Specifically, the output Pou for the previous block is fed into the next block the input. We denote the final output after KAu blocks as P. This procedure enables the model to refine its understanding and interpretation of the positions with each successive block.
3.4 Model Training
Infusing auxiliary information. To enhance the model's understanding of the game state, we inject auxiliary information as labels, including legal moves represented by multi-hot vectors and human move information: one-hot vectors of which piece is moved, which piece is captured (if any), the move's originating square, the move's 's destination square, and whether or not the move will deliver a check. These segments are used as labels for classification, serving a dual purpose: 1) It offers a more granular understanding of human moves by providing detailed context beyond just the move indices produced by the policy head labels, enriching the model's insight of player decisions; and 2) ensures the model also learns about objective (i.e. chess-specific as opposed to behavioral) knowledge in chess, which is essential for developing comprehensive understanding of both human moves and the fundamental mechanics of the game.
5

---

Table 1: Move prediction accuracy on the Maia-2 Testset. Skilled, Advanced, and Master are grouped according to Section and Arg denotes macro-averaged results.
Stockfish
Leela
Maia
Maia Maia-2gutset
Maia-2 2
3
9
15
1500
2200
3200
1100
1500
1900
Skilled
36.22
36.00
36,86
40.46
39.79
39.97
51.48
50.79
48,51
51.51
51.72
Advanced
38.25
38.78
39.83
44.45
43.97
44.29
49.13
52.61
52.26
53.54
54.15
Master
40.71
43.26
44.61
48.69
47.11
47.75
45.85
50.76
53.20
53.16
53.87
Avg
38.39
39.35
40.43
44.53
43.62
44.00
48.82
51.39
51.32
52.74
53.25
Data balancing and filtering. Chess games between players of significantly different skill levels are relatively rare but help us understand how players of lower skill levels approach games against far stronger opponents and vice versa. While previous work has ignored these games completely, they play a central role in our approach. Since games between players of similar skill levels vastly outnumber more uneven matchups, we use data balancing strategy to effectively train our unified model for aligning players across all skill levels, in which games between players of different skill levels are over-sampled. Online chess platforms feature a variety of game types, including blitz, rapid. and classical, each representing games played at different time controls (amount of time given to each player for the whole game). We focus on Rapid games, which are medium-length games that lie between the fast-paced decisions of "Blitz" games and the slower, more strategic considerations of "Classical" games. In addition, we follow the procedures [ to filter valid positions within each game. More details about data balancing and filtering are available in Appendix B
Training objectives. With the fused skill level and position representation P input, we construct the policy head on top to predict human moves, which is optimized using cross-entropy loss with one-hot labels representing the recorded human move. We also build the auxiliary information head to infuse additional knowledge into Maia-2 as introduced in Section 3.4 This head is trained using bit-wise binary cross-entropy loss with multi-hot labels. Finally, following previous work (1 include a value head to predict the game outcome as regression task, where the labels 1, 0, -1 denote winning, drawing, and losing, respectively. The training objectives of these heads are balanced to contribute equally to Maia-2 model optimization. Hyperparameter settings used for Maia-2 training can be found in Appendix Table
4 Results
We empirically evaluate Maia-2 along two key dimensions: move prediction accuracy, how well it predict human moves varying skill levels, and move prediction coherence, how aligned its predictions are across skill levels. We train Maia-2 on Lichess games played between Jan 2013 and Nov 2023, with the exception of December 2019, since that is the month used for testing in the original Maia paper (and we also test on this month for consistency) [. After game filtering and balancing, end up with training set of 169M games (9.1B positions). We also train Maia-2ulset with identical model architecture and training configurations as Maia-2, except it only has access to the same training data that Maia had for fair comparisons. Dataset statisties are reported in Appendix Tables 7 and TO We compare Maia-2 with Stockfish (6), the strongest chess engine, Leela, an open- source counterpart AlphaZero [10. and Maia (.]. the state-of-the-art model for human-like chess play. Maia is actually a set of separate models, each trained on a different set of players at different skill levels from 1100 to 1900. We use the benchmarking Maia Testset M for performance comparisons where both players have identical skill levels. We report the results on Maia Testset by grouping players into three categories: Skilled (Rapid rating up to 1600, which slightly exceeds the initial rating of 1500), Advanced (Rapid rating between 1600 and 2000), and Master (Rapid rating over 2000). In addition, we aim to evaluate move prediction across diverse skill combinations with the Cross-skill Testsef constructed from Dec 2023 games. Finally, we construct Grounded Testset with 450,000 positions that has recorded Stockfish evaluations, which can serve as grounded facts to measure move quality. Statistics of datasets are summarized in Appendix Table g

---

Mala 1303
Malo
2000
TLL
Oeponent Nayer 2a8 5al
38.6
360
145
614
112
133
134
121
644
1200
112
10:
134
1500
2000
2000
1600
2000
Arthon Iewe
Artian
Artis Feve
Figure 2: Move prediction accuracy across diverse skill levels. Colors represent performance, with warmer tones indicating higher accuracy.
4.1 Move Prediction Accuracy
Mala-2. In Table we show the top-1 move prediction accuracy of all models across all groups of players on the Mata-I Testset. Maia-2 demonstrates strong and consistent performance across all skill levels, surpassing all baselines. Specifically, despite Maia-l models being specifically trained to mimic chess moves by players at specific skill levels, Maia-2 emerges as a unified one-for me-for-all model that is consistently effective across the entire spectrum of chess skills. The largest improvement is on Advanced players, where Maia-2 gains 1.5 percentage points over the nearest competitor (Maia 500).. When averaging across skill levels, Maia-2 outperforms all other models by almost 2 full percentage points in overall accuracy. Note that the ceiling accuracy of human move prediction is far below 100% given the randomness and diversity of human decisions-even the same player won't always make the same decision when faced with the same position. Our 2 percentage point gain is substantial considering that the difference between Maia-1 and Leela, the previous state-of-the-an model for this task and traditional chess engine not trained for this task at all, is only 6 percentage points. Furthermore, Maia-1 1 is essentially a mixture of 9 experts targeting the specific players' skill level, where each expert has 10.3M parameters. Regarding the routing function to select the best- best-performing expert as nonparameterized function, Maia-1 1 has 92M parameters in total. Maia-2, on the other hand, is one-for-all model with 23.3M parameters under our default settings. Therefore, Maia-2 achieves better human move prediction accuracy with even much fewer trainable parameters.
Baseline models. Both Maia-2 and Maia-l significantly outperform Stockfish and Leela, typically by 5-15 percentage points. Note that Stockfish and Leela aim to play optimal chess (as most humans do too), and only "predict" human moves when their approximations to optimality happen to overlap with those of human players. However, we compare to these traditional chess engines because besides Maia-1, there are still the default method of creating "human- like" AI agents. The accuracy gap between Maia-1 I architectures and traditional chess engines demonstrates the necessity of developing specialized models to mimic human chess moves.
Maia-2 Maia-2 differs from Maia-1 in two main ways: it has different architecture and it has access to more training data. To control for the difference in training data and isolate the effects of our architecture, we create Maia-2uatst which has access the exact same training data that Maia-1 was developed with. Comparing the two, we see that Maia-2 matches or outperforms all baselines and alternate models. Recall that Maia-2 and Maia-2unteer don't 't have the recent history passed as input to them, yet still achieve state-of-the-art results. It is important to note that each Maia-I model is specifically trained for its respective skill level, relying solely on games where the active and opponent skill levels match for its training data. On the contrary, the unified modeling approach with skill-aware attention of Maia-2sabser allows utilize a broader spectrum of games, featuring a variety of skill-level pairings, for training purposes. Consequently, while both Maia- I and Maia-2ubbee draw from the same source dataset, Maia-2qutset can leverage significantly larger portion of this data for its training. improving its learning and predictive capabilities. The improvement from Maia-2uubset to Maia-2 underscores the importance of extensive training with vast datasets. A broader range of games provides Maia-2 with access to more comprehensive and nuanced patterns in human chess moves. Using Maia-2aubset as a comparison, we can determine the relative contributions of model architecture and training data to Maia-2's 1.9 percentage point gap over its nearest rival (Maia 1500). This calculation suggests that 73% of the increase in performance due to the architecture improvements and 27% is due to increased training data.
7

---

Table 2: Move prediction perplexity on the Grounded Testset
Table 3: Ablation study results, where Maia- 2ubeett is compared with versions without skill- aware attention ("wlo Att) and without infusing auxiliary information ("w/o Aux").
Skilled
Advanced
Master
Avg
Maia 1100
5.05
5.56
6.01
5.54
Maia 1500
4.98
5.31
5.50
5.26
Skilled
Advanced
Master
Avg
Maia 1900
4.44
4.71
4.86
4.67
w/o Att
50.63
52.89
51.73
51.75
Maia-2
4.30
3.90
4.02
4.07
w/o Aux
50,96
52.98
52.34
52.09
Maia-2
51.51
53.54
53.16
52.74
Move prediction perplexity. While top-1 accuracy gains are important, they may overshadow larger improvements in prediction quality. To account for this, we also measure the perplexity of move predictions. which reflects the model's confidence in its predictions. A lower perplexity indicates the model is more confident and accurate in human move prediction, as it corresponds likelihood of the correct human move, As shown in Table 2, Maia-2 consistently yields substantially lower perplexity all groups of skill levels compared to Maia-l. In particular, Maia-2 significantly outperforms Maia-l I in Advanced and Master moves with relatively large margins, demonstrating the effectiveness of our unified modeling approach across diverse skill levels.
higher
Adaptive move predictions. We now evaluate Maia-2's ability to predict compare across diverse skill combinations using the Cross-skill Testset. As shown in Figure 2 Maia-2 consistently outperforms both Maia 1100 and Maia 1900 almost all combinations of active player and opponent player skill levels. In particular, although Maia 1100 and Maia 1900 demonstrate competent performance within their respective domains of expertise, their predictive accuracy decreases substantially outside of these targeted skill levels. This is because Maia-I 1 models are static and cannot respond to varied skill levels and adjust their predictions accordingly. In contrast, our proposed unified modeling approach with skill-aware attention enables Maia-2 to adapt its predictions to account for the skill levels of both the active player and the opponent player, so that varying skill level configurations correspondingly can result in better aligned human move predictions. More results comparisons with other Maia- 1 versions can be found in Figure 6 in the Appendix.
Move quality. One of our key motivations for creating a unified model of human chess behavior is to guide the development of future algorithmic learning tools. As such, understanding the mistakes that people make is of fundamental interest. Can Maia-2 predict mistakes better than Ma Figure TO in the Appendix shows the move prediction accuracy on the Grounded Testset as a function of move quality, measured by win-rate loss, which is calculated following the same procedures as prior studies [1 (25). All models generally decrease in their ability to predict worse moves, since humans are generally trying to avoid mistakes, and high-quality moves are more certain whereas lower-quality moves can be more random and thus hard to predict. Nevertheless, Maia-2 outperforms all versions of Maia-1 I across most of the move quality range, demonstrating the effectiveness of our unified modeling approach for human move prediction.
We are also interested in how certain the models are about their predictions of various move qualities. Figure (A) (top) shows the probabilities that Maia-2 (x-axis) and Maia-1 (y-axis) attribute to the moves people actually played in Grounded Testset. The concentration of points in the lower right quadrant (closer to P(Maia-2)= = and P(Maia 1900) = O) suggests that Maia-2 assigns higher probability to the true move than Maia-1 1 does, indicating superior predictive performance. Conversely, the less dense upper left quadrant indicates fewer instances where Maia 1900 outperforms Maia- 2. Remarkably, while this consistently occurs across all move qualities, the distinction is more pronounced for Blunders and Errors compared to Optimal moves. Additionally, the bottom row of Figure (A) shows the log odds ratio between P(x, y) and P x) in the top row. The abundance of blue points below the diagonal indicates that Maia-2 is almost always more confident in the correct thross-thheboart improvement in move prediction. Maia-2 offers superior and more confident prediction across diverse move qualities. We also conduct an ablation study shown in Table 3
move than Maia-l is, indicating
we point the readers to Appendix A for more information.
4.2 Move Prediction Coherence
Maia-2's s accuracy across the spectrum of human skill is certainly desirable, but perhaps an even more important dimension is prediction coherence skill varies. A central drawback of Maia-l is that it
8

---

Bander
Emeer
Oatiral
Varying
Opporemti
the auth
your aol jort ii foee
cul Kin
Sapye 22 Ladee
Ood, Ruo
Sal bat
kar
(A)
1204
1000
2000
1299
1663
21
Activ Paaye's Skil Leve
Oppeoeet Player's
Figure 3: (A). (Top) Joint probability assigned to human moves played by Maia-2 (x) and Maia 1900
(y), split by move quality. Blunders (left) reduce the expected win-rate by 2 10%, Errors (middle) by
5-10%, and Optimal (right) by S 0%. (Bottom) Log odds ratio of p(a,, y) and p(y. 2) from top. (B).
Move prediction agreement as (left) active player and (right) opponent player skill are varied. All cells are evaluated on the same set of positions but with altered skill level configurations.
Table 4: Percentage of monotonic and transitional positions.
% MoMoooonic
% Transstioonll
Skilled
Advanced
Master
Skilled
Advanced
Master
Maia-1 1661
1.42 28.51
1.14
13.34
18.14 23.39
20.48
Maia-2
27.61
26.38
22.59
21.72
models players at different skill levels independently from each other, which results in particularly volatile predictions: the same position might elicit very different predicted behavior from models of adjacent skill levels. This is problematic because we know from personal experience that this type of volatility is rare: players don't It change that much as they improve. This limits Maia's ability to perform well in downstream tasks such as serving as a teaching aid, as its understanding of one skill level bears little resemblance to its understanding of the next. In reality, players move from one skill level to another by making small, consistent adjustments. Does Maia-2 reflect this behavioral coherence?
Prediction smoothness. We measure the coherence of Maia-2's predictions by testing for smoothness features in its entire set of predictions. Call a model's 's treatment of position monotonic if the predicted probability of the correct move increases with skill monotonically. In the Grounded Testset of 100K positions, we find that Maia-1 only treats 1 1% of them monotonically. In stark contrast, however, Maia-2 treats 27% of them monotonically, clearly demonstrating that Maia-2 is much more coherent. Similarly. call model's 's treatment of position transitional if it predicts suboptimal move for some prefix of skills and then transitions to an optimal move for all subsequent skill levels. Again, Maia-2 treats substantially more positions transitionally- around 22% of them compared with 17% for Maia.
It's important to note that Maia-2 is deliberately designed to encourage coherence across skill levels without rigidly enforcing it. Our objective is not to impose coherence might obscure legitimate differences player behavior between skill levels, but to create a model architecture that naturally encourages coherence where the data supports it.
hard constraint, which
Move prediction agreement. As first test, we measure move prediction coherence we vary active player skill and opponent player skill in Maia-2. The results shown in Figure (3((B) reveal several trends. First, increasingly varying either the active opponent rating results in lower agreement, suggesting that Maia-2 smoothly varies its predictions with skill. Second, comparing the two heatmaps reveals that Maia-2 has clearly learned that varying one's own skill has much larger effects than varying the opponent's coannent's-chanee channgin one's own skill against a fixed opponent can change the decision up to 22% of the time, but changing the opponent's skill while fixing our own skill will only change the decision up to 6% of the time. This is intuitive, as players must change their decisions in order to play at a higher level, while in theory one's 's opponent shouldn't affect one's decision. Of course, humans are not optimal agents and sometimes take their opponent's 's skill level into account when deciding on move wwllfulllllly or which is reflected in our results.
Chess concept understanding. Human chess players of varying strengths differ in their ability to recognize important features and patterns on the board, e.g., stronger players are adept at discerning
9

---

middle qame evaluatioo
miodle game, piece value
active
1.0
1.0
1-haa Alasfizn Poer 56 idtere Altention
decur Allaniani Fost Sell Sell-haree Atteriie
0.9
0.07
0.02
Accuracy
0.7
0.7
0.05
2.55
0.50
Pro S6.8 Sabre Jn
Fio Seltt ziteetion
0.68
0.4
Posl Pou dwore Altention
Post, 54 -eaar Atarricn
0.62
1203
1600
2000
1200
1600
2000
Lovd
Playa Sull Lovd
Ragar pall Level
Player 2u
[ol
Skill Dependent Conceetts
(B)
Skill Independent Cancepts
Figure 4: Maia-2's chess concept recognition as a function of skill level, as measured by linear activation probes right before (blue) and after (orange) skill-aware attention. (a) Stockfish overall board evaluation for middle-game positions. (b) Stockfish evaluation of middle-game bonuses and penalties to pieces for white. (c) Does the active player own two bishops? (d) Can the active player capture the opponent's queen?
subtle nuances. We now turn focus to a critical question: does Maia-2 vary in its ability to capture human chess concepts when given different skill levels? Following the chess concepts probing strategy for AlphaZero [22] show how Maia-2's grasp of various concepts varies with skill. The left two plots in Figure 4 show concepts for which Maia-2 clearly distinguishes between skill levels, with higher-skill players paying more attention them than lower-skill players. These are general board evaluations as given by Stockfish [6], or aggregate piece values. Note that pre-skill-aware attention is always flat because by construction it cannot vary with skill, since skill-aware attention has not been applied yet. The two plots on the right depict concepts that live closer to fundamental chess rules, and as such are less dependent on player skill. For delendentt concepts, the figures reveal an increasing trend in mastery level after -wirre attention, aligning with the increase in dedicated skill levels. Meanwhile, the model's mastery level decreases after passing through the skill-award attention modules, potentially adjusting for the imperfections of human players. Conversely, the skill-aware attention blocks are not responsive to skill- dill--nneeendeett concepts.
Discussion
Human Study. In addition to human move matching, we also consider engagement, another dimension of human study. In particular, we implement a randomized experiment on Lichess: human players challenge our bots, and we randomize whether players play against Maia-1 or Maia-2. Our result is that our higher move-matching and our vastly improved coherence, across all skill levels, come at no cost human subject engagement, and in fact slightly increase engagement: players rematch Maia-2 almost percentage point more than Maia-1 (41.2% vs. 40.3%). Although engagement is not main objective, this is further promising evidence that have achieved more human-alii -alinndd model that coherently captures human style across different skill levels.
Ethical Considerations. We believe Maia-2 poses limited risk while offering large potential benefits. Our data is highly aggregated, with almost 1 billion games being used for training, and chess domain is generally low-risk. Meanwhile, helping people improve in chess could lead to increased cognitive skills, confidence boosts, and help with general life satisfaction. Our vision is for Maia-2 to power Al partners and training aids; it cannot currently replace skilled human tutors and coaches.
Limitation. Our work has limitations. First, we are excited by the applications that Maia-2 will enable, such as more relatable Al partners and Al-powered learning aids, the development of which is out of scope for the current work. Maia-2 does not yet incorporate search, although previous work has demonstrated that with proper regularization it can help improve move prediction performance [26]. Relatedly, we group the strongest players in single bucket, although modeling the very best players in the world remains difficult due to the complexity and depth of their moves.
10

---

Acknowledgements
We thank the anonymous reviewers for helpful comments. This research was supported in part by grant from the Natural Sciences and Engineering Research Council of Canada (NSERC), Microsoft Research Award, Simons Collaboration grant, and grant from the MacArthur Foundation.

---

References
[1] Reid Mcllroy- Young, Siddhartha Sen, Jon Kleinberg, and Ashton Anderson. Aligning super- human with human behavior: Chess as a model system. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1677-1687, 2020.
[2] Feng-hsiung Hsu, Ibm's deep blue chess grandmaster chips. IEEE micro, 19(2)700-11, 1999.
[3] William Clark, Jan Golinski, and Simon Schaffer. The sciences in enlightened Europe. Univer- sity of Chicago Press, 1999.
[4] Frederic Friedel. Reconstructing turing's "paper machine".
[5] Claude Shannon. A mathematical theory of communication. Bell Syst. Tech. J., 27:623-656, 1948.
[6] Tord Romstad, Marco Costalba, Joona Kiiski, and et al. Stockfish. stockfishchees..rg, 2023. Accessed: 2024-01-05.
[7] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mas- tering the game of go with deep neural networks and tree search. nature, 529(7587):484448) 2016.
[8] Xidong Feng, Yicheng Luo, Ziyan Wang, Hongrui Tang, Mengyue Yang, Kun Shao, David Mguni, Yali Du, and Jun Wang. Chessgpt: Bridging policy learning and language modeling. arXiv preprini arXiv:2306. 09200, 2023.
[9] Feng-Hsiung Hsu. Behind Deep Blue: Building the computer that defeated the world chess champion. Princeton University Press, 2002.
[10] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Mare Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv,, 2017.
[11] Tom Zahavy, Vivek Veeriah, Shaobo Hou, Kevin Waugh, Matthew Lai, Edouard Leurent, Nenad Tomasev, Lisa Schut, Demis Hassabis, and Satinder Singh. Diversifying ai: Towards creative chess with alphazero. arXiv preprint arXiv:2308. 09175, 2023.
[12] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neurul information processing systems, 30, 2017.
[13] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 33:1877-1901, 2020.
[14] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.0922
2023.
[15] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199-22 22213, 2022.
[16] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen I1m applications via multi-agent conversation framework. arXiv preprint wxXv:2308.08115, 2023.
[17] Max Hager. LLMChess.
[18] Nicholas Carlini. carlini/chess-Imm
[19] Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonse al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:22066 200.::22.000.655.... 2022.
[20] Andrew Lee, David Wu, Emily Dinan, and Mike Lewis. Improving chess commentaries by combining language models with symbolic reasoning engines. arXiv preprint 27X11:221..00199. 2022.
12

---

[21] David Noever, Matt Ciolino, and Josh Kalin. The chess transformer: Mastering play using generative language models. arXiv preprint arXiv:2008.000 04005, 2020.
[22] Thomas McGrath, Andrei Kapishnikov, Nenad Tomašev, Adam Pearce, Martin Wattenberg, Demis Hassabis, Been Kim, Ulrich Paquet, and Vladimir Kramnik. Acquisition of chess knowl- edge in alphazero. Proceedings of the National Academy of Sciences, 119(47):e220622511 19, 2022.
[23] Lisa Schut, Nenad Tomasev, Tom McGrath, Demis Hassabis, Ulrich Paquet, and Been Kim. Bridging the human-ai knowledge gap: Concept discovery and transfer in alphazero. arXiv preprint arXiv:23 10. 2023.
[24] Reid Mellroy- Meelnooy-Young, Yu Wang. Siddhartha Sen, Jon Kleinberg, and Ashton Anderson. Detecting individual decision-making style: Exploring behavioral stylometry in chess. Advances in Neural Information Processing Systems, 34:24482-24497, 2021.
[25] Reid Mcllroy- Mellroy-Young, Russell Wang, Siddhartha Sen. Jon Kleinberg, and Ashton Anderson. Learning models of individual behavior in chess. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1263-1263, 2022.
[26] Athul Paul Jacob, David J Wu, Gabriele Farina, Adam Lerer, Hengyuan Hu, Anton Bakhtin, Ja- cob Andreas, and Noam Brown. Modeling strong and human-like gameplay with kl-regularized search. In International Conference on Machine Learning, pages 9695-9728. PMLR, 2022.
[27] Michael L Littman. Markov games as framework for multi-agent reinforcement learning, In Machine learning proceedings 1994, pages 157-163. Elsevier, 1994.
[28] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[29] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint MXXi:2010.119223 2020.
13

---

A
Maln 1100
Mala 1300
Mala 1500
Mala 1700
Mala 1900
X
Stronger Players
Mate-In-1 Puzzle of raatinn 1500
X
1100
1300
1590
1700
1900
X
Mala-2 with Increasing Player's Skill Levels
Figure 5: Maia-2 and Maia-l 1 solving Mate-in- chess puzzle of rating 1500. Green arrows represent correct move predictions, while red arrows indicate incorrect predictions. The darkness the green color correlates with the model's confidence, with darker arrows denoting higher probability of making the correct move.
2000
1000
38.0
31A
20.8
114
26.1
11.
234
364
12.
363
387
322
1200
Mala
Male Best
Male-2
2000
3033
334
12B
616
114
900
Cappeel pavers
1600
Nove
1200
1200
1200
1800
1600
Figure 6: Move prediction accuracy across diverse skill levels. Colors represent performance, with warmer tones indicating higher accuracy. Missing skill combinations were too rare to be statistically reliable.
A More Experimental Results
Case study: Smoothness. We evaluate the smoothness of Maia-l 1 and Maia-2 by case study in puzzle solving: Mate-in-1 puzzle of 1500 skill level is presented to both Maia-1 and Maia-2, where smoothness be evaluated by checking whether the predictions skill level increases. Call a model's 's treatment of a position as monotonic if the predicted probability of the correct move increases with skill monotonically, predicted probabilities of the best move (in green arrows) increase monotonically from 0.22 to 0.45 the skill levels rise from 1 1100 1900, while Maia-l predictions are rather turbulent. Similarly, call model's ''s treatment of a position transitional if it predicts a suboptimal move for some prefix of skills and then transitions an optimal move for all subsequent skill levels. As shown in Figure Maia-2 can mimic weaker players to whom the puzzle is hard to solve, while stronger Maia-2 with skill level configured above or equal to 1500 can successfully solve the puzzle. However, Maia 1100 surprisingly solved the puzzle, while the stronger Maia-1 models, e.g., Maia 1700 failed to make the optimal move. Therefore, in the considered case, as opposed Maia-1, Maia-2 yields smooth predictions provided that its treatment of this position is monotonic and transitional.
monotonic and transitional as
can observe from Figure that the Maia-2
14

---

Average Blunder Rate
0.16
Average Centipawn Loss
Average Blunder Rate
-00
Average Centipawn Loss
0.14
65
0.12
60
0.10
55
0.08
1200
2000
Player's Skill Level
Figure Quality predicted moves quantified by blunder rate and centipawn Loss.
Quality of predicted moves. Given the same chess position but increasing skill levels, coherent, skill-aware human move prediction model should make progressively higher-quality moves. We measure the average centipawn loss standard move quality metric, the lower the better) and the average blunder rate (fraction of times an egregious mistake is predicted) in positions randomly sampled from December 2023 games. The centipawn loss determined by comparing the moves predicted by Maia-2 against the top move as evaluated by Stockfish at depth 20 (human grandmaster- level play), while blunders are classified as moves resulting a win-rate loss of 10% or more. As shown in Figure 7 both the centipawn loss and the blunder rate exhibit a smooth, monotonic decrease as player skill rises, demonstrating Maia-2's capability of adjusting predictions coherently to align with the increasingly skilled players.
Value head. As proxy of model evaluation given board position, we train the model value head which is potentially significant for wide variety downstream tasks. The value head is trained as regression task from to I indicating from losing winning positions, and finally normalized to continuous value between and similar to AlphaZero. The correct label for value head is the actual game results. To check the evaluation quality of model value head, we calibrate the value head results with actual game outcomes, and generate a quantile-quantile plot in Figure where the win probability is discretized uniformly into 100 bins from 0 to 1.
1.0
Empirical Win Probabilty
0.8
0.4
0.2
0.0
0.2
0.4
0.6
0.8
1.0
Predicted Probability
Figure 8: Value head Q-Q plot.
15

---

Ablation study. To understand which components of our architecture are most responsible for the performance gains, we conduct an ablation study on the Maia Testset. We train a version of Maia-2 using naive method of incorporating encoded skill levels without the proposed skill-aware attention module ("w/o Att"). this model, the skill level encodings ea and eo are directly concatenated with flattened Paocoddd- without bridging them with the skill-aware attention (skipping Section 3.3), and directly connected to prediction heads. As shown in Table 3 Maia- consistently performs better with skill-aware attention, demonstrating the necessity of modeling the complexity and non- non-linearity in player's ' skill development in sophisticated ways and the effectiveness of our proposed unified modeling approach with skill-aware attention to model such nuances. We also train a version of Maia-2 without the auxiliary information head ("w/o Aux"), ). These results show that infusing auxiliary information labels during model training also results in significant performance improvement, although not as dramatically as skill-aware attention.
2.0
gamee avaluation
2.5
place vallo
0.00
NeToO JWY appanent
0.00
Captiure powitlle on wethve
0.9
0.5
0.67
--- Post Ski huoor Atteriion
Avsl Awaro Aftertie
0.00
6.7
0.64
Pro buil Awart Attesss
PVB 36-1 Ar Aleriie
6:43
6.42
0.45
1200
1808
2002
1206
1900
2010
1200
1000
2000
2200
1000
2003
Concepts
(b)
(c)
Independent Concepts
(5)
Figure 9: Maia-2's chess concept recognition function of skill level, measured by linear activation probes right before (blue) and after (orange) skill-aware attention. (a) Stockfish overall board evaluation for end-game positions. (b) Stockfish evaluation of end-game bonuses and penalties to pieces for white. (c) Does the opponent player own two bishops? (d) Is capture possible on square d3 for the active player?
70
1900
1500
60
Move Prediction Accuracy
Maia 1100
Maia-2
50
40
30
20
0.00
0.05
Win-rate Loss
Figure 10: Move prediction accuracy as function of move quality, quantified by win-rate loss.
Accuracy as a function of move quality. Figure 10 shows the move prediction accuracy on the Grounded Testset as a function of move quality, measured by win-rate loss, which is calculated following the same procedures as prior studies I 25 All models generally decrease in their ability to predict worse moves, since humans are generally trying to avoid mistakes, and high- quality moves are more certain whereas lower-quality moves can be more random and thus hard to predict. Nevertheless, Maia-2 outperforms all versions of Maia across most the move quality range, demonstrating the effectiveness of our unified modeling approach for human move prediction. Maia-2's s overall gains are not constrained to any specific move quality type, are spread across the entire range.
16

---

Table 5: Hyperparameter Settings.
#Games per chunk Nehunk
20000
#Maximum games per skill level Nange
20
Initial learning rate
le
Weight decay
1e 1e-5
Batch size (positions)
8192
Minimum move ply
10
Maximum move ply
300
Remaining seconds threshold #Backbone blocks Koov # #Attention block Att
30
12
#Input channels Cinput #Intermediate channels Cmid #Encoded channels Cpatch
18
256
Skill level embedding dimension d,
128
Attention head dimension dh. Attention intermediate dimension dan
64
1024
#Attention heads
Table 6: Statistics of the Maia-I Testset.
#Positions
Rating Range
Total 106,740
Skilled
56,812
< 1599
Advanced
41,747
1600 1999
Master
8,181
Z 2000
B Reproductibility
Implementation details. To maintain consistent perspective from both sides of players, we implemented board flipping to train and test Maia-2; that is, positions with black to move were mirrored such that all analyses could be conducted from the white side's s viewpoint. We further refined our dataset through game and position filtering, selecting only rapid games from Lichess with available clock information and disregarding the initial 10 plys of each game as well as positions where either player had less than thirty seconds remaining. The filtration is significant for eliminating the noise introduced by rushed decisions under time constraints, which could skew the true representation of player's skill. The choice of exclusive rapid games is also informed by the distinct rating systems across different game types, which necessitates the separation of data to maintain rating consistency. We report all hyperparameters involved in training Maia-2 in Table It took approximately 13 days to train Maia-2 with 2 x A 100 (80G) GPUs under our default settings.
Data Balancing. Chess games between players of significantly diffferent skill levels are relatively rare but help us understand how players of lower skill levels approach games against far stronger opponents and vice versa. While previous work has ignored these games completely, they play central role in our approach. Since games between players of similar skill levels vastly outnumber more uneven matchups, we use a data balancing strategy to effectively train our unified model for aligning players across all skill levels, in which games between players different skill levels are
Table 7: Statistics of training datasets.
Maia-2. Maia-2ubbet
Maia-2
#Games Consumed
930.19M
5.14B
#Games Trained
21.31M
168.93M
#Positions Trained
1.18B
9.15B
17

---

over-samppln sampled. To be precise, we pre process the data in chunks of Nehunk games each. We then scan each data chunk find games satisfying various (active player skill, opponent skill) combinations. Each skill combination can include at most Nrange games. We continue scanning the data chunk until all the skill combinations have Nrange games or the data chunk fully consumed. Note that Naus, the higher the balancing factor Naux' the less likely it is that rare skill combinations will fully reach Nrange, which will lead to less balanced data overall. On the other hand, if the balancing factor is too small, fewer games will be selected from each data chunk, which is data-inefficient. We choose a fair compromise between data efficiency and balanced data so that our training data encompasses a broad spectrum of skill levels without biasing excessively towards the more frequently-occuringg equal-skill matchups.
Data Filtering. Online chess platforms feature variety of game types, including blitz, rapid, and classical, each representing games played at different time controls (amount of time given to each player for the whole game). We use data from Lichess, a well-known large open-source chess platform, and its lopen database, In Lichess, since each game type is given separate rating, ratings across different game types are not comparable (e.g. a rating of 1800 in "Rapid" is significantly weaker than a rating of 1800 in "Blitz" on Lichess). Previous work [1 mixes player ratings across these game types together for training and evaluation. Instead of mixing data across game types, we focus on Rapid games only, which are medium-lengle games that lie between the fast-paced decisions of "Blitz" games and the slower, more strategic considerations of "Classical" games. "Blitz" and "Bullet" games, characterized by their quick pace, are composed of many decisions made under time pressure, introducing randomness that may not accurately reflect player intentions and skills. In contrast, Classical games are played less frequently, leading to data scarcity. Rapid chess is an ideal compromise between quality of play and quantity of data. addition, we follow the procedures in to filter valid positions within each game.
Chess position representation. We follow the well-established prior works 110 to represent chess positions as multi-channel 8 x matrices, including:
The first 12 channels categorize the board's pieces by type and color, with one channel each for white and black Pawns, Knights, Bishops, Rooks, Queens, and Kings. A cell is marked denote the presence of piece in the corresponding location, and O otherwise.
Piece Representation:
Player's Turn: A single channel (the 13th) indicates the current player's turn, filled entirely with Is for white and Os for black, providing the model with context on whose move is being evaluated.
Castling Rights: Four channels (14th to 17th) encode the castling rights for both players, with the entire channel set to if the right is available or 0 otherwise.
En Passant Target: The final channel (18th) marks the square available for en passant capture, if any, with I and Os elsewhere.
Chess concepts probing. Given a board position, we vary the skill level injected to Maia-2 to extract the Pancoded E C the learned respresentation of the ResNet--aase based backbone served as the control group which remain constant to varying skill ratings, and the output hidden states P Cran x after skill-aware attention directly connected to model heads. We randomly pick 500,000 positions from Lichess December 2023 database and calculate Stockfish built-in or custom implemented concepts using the Forsyth-Edwand Notation of the positions.
We carefully choose Pancoded and P as internal representations for the reason that Pancoded is the comprehensive understanding of chess board for the backbone network right before skill-aware attention, and P is the final hidden states that directly influence model decision and evaluation. Following the work of Alphazero concept probing [22] for chess concepts with continuous values we train Lasso regressors with coefficient for L L'-regularization selected by 5-fold cross-validation, and for binary concepts we train logistic regressors with downsampling. During evaluation on unbalanced test set, we measure the performance of continuous-value probes with the coefficient of determination 2², and score the binary-valued probes with the Macro Fl.
18

---

Table 8: Statistics of the Cross-skill Testset.
Opponent
1300 (399
1700-175
20000
91,264
100,000
100,000
1700 1709
91,054
100,000
1000000
100,000
100,0000
1000000
1600 1699
81,386
99,938
100,000
100,000
100,000
99.933
91,610
100,000
91,291
7700-1999
07,602
100,000
100,000
100,000
90,544
$1,692
1200 1259
100,000
100,000
100,000
100,000
82,230
Table 9: Number of games in the balanced dataset used for training Maia Maia-2 Subset-
Black
<1100
1100-1199
1200
1299
1300
1299
L400
1899
1500
1600
1899
1700
7700-1799
1799
1890
1999
>=2000
<1100
583,437
1100-
583,403
583,433
1200 1299
427.160
583,402
583,643
1300 1399
176,930
408.819
583.421
583.443
1400 1499
113,683
204.425
499.185
583.433
583.445
1899
57,381
174,417
384.670
366.851
583,435
1200- 1759
32,621
47.671
100.841
183.967
359,056
$71 666
583,421
$81.478
1899
16.655
26,195
52,043
105.085
178.957
348.979
557,921
$83,115
583,422
582.715
>2000
5,927
7,953
14,825
28.929
49,237
$8.317
134,432
290,452
531,162
582,993
582,542
Table 10: Number of games in the balanced dataset used for training Maia-2. 2.
White
Black
1300 1499
1999
2-2000
4,698,087
1100 1199
4,697,937
4,697.979
1200 1299
3,165,114
4,097.922 972
4.698,0776
1300
2.561.344
4,698,015
4,898.013
1,509,456 450
1,549.218
3,008.53
4,698.033
4.698,093
1500 1599
1,259, 824
1,400 474
2.348.803
3,805.200
4.608.047
4.09%,10
1600 1699
627,583
790.364
1,415,147
2,257.66
3,514,299
4,698.655
4.698.092
1800-1899
251,902
89A978
1.66,987
2,4998322
3.6655572
1900 1999
190,385
171.133
272,258
445.251
818.918
1.452,505
1,940.257
3,436,275
4,697,958
4,891.311
>2000
238,477
186.001
277,451
423,092
651,232
4,200.48
1,718.683
2,587,575
3,967,891
4,007,533
4,697,178
19

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors their evaluation. While "[Yes] " is generally preferable to "[No]" it is perfectly acceptable to answer " "[No]" " provided proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset used"). In general, answering "[No] or "[NA]" is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often
nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly outline the main contributions and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
20

---

Justification:
We discuss the limitations of our work in Section 5.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specifiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texi system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
We do not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We include details of model design and model training in Section experi- mental setup in Section 4, as well as hyperparameter settings and implementation details in Appendix B.
Guidelines:
21

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We include the link to our code including the data processing pipeline in the abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nisscc/public/guuss http://///giiiee//edeuuubiiiiiii
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
22

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We include dataset details and hyperparameter settings in Section and Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We train Maia-2 with huge amount (9.1B) of chess positions. Therefore, it is hard to evaluate Maia-2 multiple times with different train/test splits.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the required computational resources in Appendix B.
Guidelines:
23

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification:
We conduct our work under the guidance of NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
We discuss the potential social impacts of our work in Section and Section
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [Yes]

---

Justification:
We discuss the limited risks of our work as human- model in Section 5.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We include all assets used in our paper in main texts and references.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, haperswithcode com/ datasets has curated licenses for some datascts. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification:
We release our code with documentations and comments.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/moodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
25

---

Answer:
[NA]
Justification:
Our work does not involve crowdsourcing.
Guidelines:
The answer NA means that the paper does involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: Our work does not involve crowdsourcing.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "bFt5kpd0Ld", "reviewer": "Reviewer_zUsz", "summary": "This work tackles the problem of modeling chess agents at varying skill levels. Prior work learns separate models for each skill level, so the authors introduce their method “Hermes” which uses skill-aware attention to adapt the predictions based on the skills i.e. chess ratings of both players in the game. Technically, Hermes uses a categorical embedding for different buckets of player ratings, and uses multi-head attention to project the game state to the player move, some auxiliary game information, and a value head. The active and opponent skill embeddings are projected and added to the query matrix in the attention step, to make the model skill-aware. The authors train and evaluate the model on large datasets of chess gameplay across varying ratings, and conduct in-depth behavior analysis of their models compared to the baselines. They demonstrate that Hermes slightly outperforms the baselines with higher confidence predictions. Further, it enables additional capabilities such as modeling chess players over the entire spectrum of skill levels, and monotonic improvement of predicted moves as the skill increases.", "questions": "1. How does the granularity of skill clusters affect the model's performance?\n2. What is the training and test split over the dataset?\n3. Could the model interpolate between varying skill levels? I would be curious to see what would be its performance on unseen skill levels.\n4. Does the Q-Q plot for Value head prediction only include accurate predictions?\n5. Is there a qualitative analysis of where the improvement in Hermes over the baseline comes from? Does it improve all skill levels or is it restricted to a low number of buckets?", "limitations": "1. The paper focuses primarily on modeling chess agents, so, a discussion on the broader impact of this method would be valuable. Here, the model assumes access to the ground truth skill embedding of the player. However, It would be interesting to see if it is possible to extract the skill embedding from the past moves i.e. implicit modeling of player/opponent skill.\n2. The model's accuracy is around 54%, which is still pretty low for the model to be deployed. The evaluation metrics are limited to the model's prediction accuracy. A human study or evaluation against skill-specific agents could further demonstrate Hermes' effectiveness in modeling skills at all levels.\n4. The paper has limited citations to past work on modeling diverse agents, so it is difficult to assess the novelty and the impact of the contribution. Further, one of the main contributions is the skill-aware module, so I would suggest the authors to include some baselines or discussion to explain the intuition and benefits of using this mechanism over other simpler baselines.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper tackles an important problem of skill-aware modeling of human behavior. The authors instantiate this in the setting of chess games between players of varying skill levels. The intuition of the overall method is well understood, the framework and the parameters are clearly mentioned, and the results are shown over the standard datasets and compared to recent baselines.\n2. The paper is well written. In the paper, the processing steps for the dataset, including filtering and balancing are explained clearly. The authors do not discard rare games between players at very different chess ratings. Is using these rare games helpful for the model? It would be nice to see some ablation showing the effect of this additional data on model performance.\n3. Hermes uses auxiliary information prediction to align the model better. This is very intuitive as forcing the model to predict auxiliary information from the current state should improve its understanding of the game concepts. I am curious to know if this has been explored in the past work. Else, it would be interesting to see if this could improve the performance of baseline agents\n4. The paper presents an in-depth analysis of the performance and predictions of Hermes under different conditions. It shows that Hermes is more confident in its predictions than the baseline that learns individual models at individual skill levels. Further, Hermes performs similarly across all pairs of skills showing that it captures and models the players' ratings over the entire spectrum, while the baseline is accurate only around the particular skill level they are trained on. Finally, they also show that the model implicitly differentiates in its predictions on the moves that are skill-dependent and independent.", "weaknesses": "1. The paper focuses on improving the predictive model performance for chess games, but the improvement over the baseline is very small (2%). Further, the table doesn't include any error ranges and has no statistical testing to show if the performance improvement is statistically significant.\n2. The paper does not cite some recent work on modeling agents with varying skill levels [1]. I would recommend the authors further include and cite works on modeling diverse agents and partners, specifically in multi-agent learning literature.\n3. In Hermes, the players’ skill embeddings are projected and added to the query matrix in the attention step, to make the model skill-aware. However, there are no experiments or intuition to explain the significance of this architectural choice. Over simple conditional networks (or more recent works such as FiLM [2]), it would be nice to see what is the significance of this particular attention mechanism.\n4. Some corrections regarding the manuscript: Figure 3 refers to an Errors (middle) figure which is missing. I believe there is a typing error in Figure 5, where the top row has Hermes labels which should be Mia.\n\n[1] Jacob, Athul Paul et al. “Modeling Boundedly Rational Agents with Latent Inference Budgets.” ArXiv abs/2312.04030 (2023): n. Pag.\n\n[2] Perez, Ethan et al. “FiLM: Visual Reasoning with a General Conditioning Layer.” AAAI Conference on Artificial Intelligence (2017).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Main focuses & Broader Impact (L1, L2, W1)**\n\nWe contribute an approach to human move prediction that is not only the new state of the art for accuracy, but our model achieves **coherence** in its predictions. To power algorithmic teaching tools, we believe that it is not enough to treat different skill levels independently and make predictions that don’t make coherent sense. Instead, we need coherent move predictions to algorithmically capture the trajectory of human ability as we progress from beginner mistakes to expert decisions. This way, we enable the building of systems that guide people along efficient learning paths. To accomplish this, we design a skill-aware attention mechanism for the **unified modeling** of human behavior across various skill levels, instead of modeling each skill level independently as previous methods did.\n\n**Move prediction accuracy (W1, Q5, L2)**\n\nThank you for pointing out that we didn’t sufficiently explain the significance of our performance gains. In the human move prediction problem for amateur players, the ceiling accuracy is far below 100% given the randomness and diversity of their decisions—even the same player won’t always make the same decision when faced with the same position. Our 2 percentage point gain is substantial considering that the difference between Maia and Leela, the previous state-of-the-art model for this task and a traditional chess engine not trained for this task at all, is only 6 percentage points. We will update the paper to make this clearer.\n\nIn response to Q5, Hermes demonstrates a performance improvement over Maia across virtually all combinations of the active player’s and the opponent’s skill levels (see Figure 2). \n\nIn response to W1, like other large models such as LLMs, it’s computationally infeasible to run the model repeatedly with different data splits due to the massive volume of data involved (9.1B positions). Nonetheless, besides discrete move matching accuracy, we also adopt a continuous and thus more stable metric, perplexity, in Table 2, which shows more significant improvement.\n\n**Skill-aware Attention (W3, L3)**\n\nWe agree that we didn’t sufficiently explain our architecture choice. The rationale behind our design is that each channel (feature map) of the ResNet output represents different aspects of a chess position, and the attention blocks actively select and interact with the features according to the given skill level. Evidence can be found in Figure 4: for skill-dependent concepts, the representation before attention blocks understands them uniformly across all skill levels with high accuracy, whereas after the attention blocks the representation is attenuated with higher skill levels understanding the concepts better than lower skill levels. This shows that attention blocks effectively “pretend” not to know the concepts to model the degree of imperfection in human moves, whereas skill-independent concepts are understood similarly by the representations before and after the attention block. Table 3 “w/o Att” shows that simply concatenating the skill embeddings with the flattened ResNet outputs did not work well. Thus, a more sophisticated way of conditioning is needed.\n\nThe primary methodological contribution lies in the **unified modeling** approach for **coherent** human behavior modeling that is enabled by this specifically designed model architecture. While we believe that the architecture has conceptual benefits that help make these contributions possible, and we will include the rationale behind the architecture in the revisions, we do not claim that the architecture is optimal in any sense; we view the specifics of the architecture as secondary to the advances, i.e., move prediction coherence and accuracy, provided by the unified modeling approach. We will also mention other architecture choices such as FiLM as promising avenues for future work.\n\n**Human studies (L2)**\n\nIn an ideal human experiment, we would give a position to a human at a particular rating, and compare their chosen move to our model output. Our experiments do exactly this, and thus we view them as massive human studies that measure the move-matching accuracy and coherence with the recorded behaviors of real humans.\n\nIn addition to this main experiment, we've also performed additional experiments that address other dimensions of your question. In particular, we've implemented a randomized experiment on Lichess: human players challenge our bots, and we randomize whether players play against Maia or Hermes. Our final result is that our higher move-matching and our vastly improved coherence, across all skill levels, come at no cost to human subject engagement, and in fact slightly increase engagement: Hermes even seems to be slightly more engaging: players rematch Hermes after the first game 1.5 percentage points more than Maia (40.6% vs. 39.1%). Although engagement is not our main objective (move-matching and coherence are) this is further promising evidence that we have achieved our goal of a human-aligned AI model that coherently captures human style across different skill levels.\n\n**Please see the comments for the rest of our response.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lcdryPNSmy", "reviewer": "Reviewer_vWC3", "summary": "The paper proposes a unified modeling approach named Hermes for aligning human and AI behaviors in chess. It addresses the limitations of previous models by integrating a skill-aware attention mechanism that dynamically adapts to various skill levels of players. The Hermes model aims to enhance AI-guided teaching tools and provide deeper insights into human decision-making in chess. The model is evaluated based on move prediction accuracy and coherence, showing improvements over existing models.", "questions": "See weaknesses.\n\nThe font in figures is too thin.", "limitations": "The paper discussed the limitations in appendix.", "rating": 3, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper presents an approach to human-AI alignment in chess through the introduction of a skill-aware attention mechanism that dynamically adapts to players’ skill levels. This technique addresses the non-linear nature of human learning and significantly improves the coherence of AI models across different skill levels. The paper is well-structured and clearly written. The evaluation of the Hermes model is thorough, demonstrating notable improvements in move prediction accuracy and coherence over existing models like Maia and traditional chess engines such as Stockfish and AlphaZero.", "weaknesses": "Despite its strengths, the paper has several limitations. While the skill-aware attention mechanism seems \"innovation\", the overall novelty of the paper is somewhat limited as it builds on existing models and techniques without introducing fundamentally new concepts in AI or chess modeling. Additionally, the paper does not adequately address potential biases introduced by relying heavily on data from a specific online platform, which may affect the generalizability of the model. The experiments mainly report move prediction accuracy and move prediction coherence, which are not sufficient to support the model’s practical effectiveness in gameplay scenarios. Most importantly, the paper lacks human-AI experiments that would substantiate the claims of alignment. \n\nThus, I find the method lacks novelty and the experiments are insufficient to support the claims, leading me to recommend the paper for rejection.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Contribution**\n\nTo the best of our knowledge, we are the first to emphasize coherence in human behavior modeling. It is important to reconsider the assertion that there are \"no fundamentally new concepts,\" as our work introduces the first human behavior model that is coherent across various skill levels and even improves upon the state-of-the-art move-matching accuracy.\n\nThe primary methodological contribution lies in the unified modeling approach for coherent human behavior modeling that is enabled by this specifically designed model architecture. While we believe that the architecture has conceptual benefits that help make these contributions possible, and we will include the rationale behind the architecture in the revisions, we do not claim that the architecture is optimal in any sense; we view the specifics of the architecture as secondary to the advances, i.e., move prediction coherence and accuracy, provided by the unified modeling approach. We will also mention other architecture choices as promising avenues for future work.\n\n**Generalization to other platforms**\n\nGiven the universal and objective nature of chess rules and strategies, it is unlikely that human behaviors in chess are biased from one platform to another. For example, given a recorded chess game, we strongly believe it would be extremely difficult to predict whether it was played on Lichess or some other platform. This observation mitigates the concerns of Hermes’s generalizability towards other platforms.\n\n**Human Studies**\n\nIn an ideal human experiment, we would give a position to a human at a particular rating, and compare their chosen move to our model output. Our experiments do exactly this, and thus we view them as massive human studies that measure the move-matching accuracy and coherence with the recorded behaviors of real humans.\n\nIn addition to this main experiment, we've also performed additional experiments that address other dimensions of your question. In particular, we've implemented a randomized experiment on Lichess: human players challenge our bots, and we randomize whether players play against Maia or Hermes. Our final result is that our higher move-matching and our vastly improved coherence, across all skill levels, come at no cost to human subject engagement, and in fact slightly increase engagement: Hermes even seems to be slightly more engaging: players rematch Hermes after the first game 1.5 percentage points more than Maia (40.6% vs. 39.1%). Although engagement is not our main objective (move-matching and coherence are) this is further promising evidence that we have achieved our goal of a human-aligned AI model that coherently captures human style across different skill levels.\n\n**Typos/Minors**\n\nThanks, we will modify them.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PGS6squ3qc", "reviewer": "Reviewer_nN7Z", "summary": "This work explores developing a unified model to predict human moves in Chess. To address the coherence challenges, the authors propose to use skill-aware attention with channel-wise patching to encode skill levels and board positions into a neural network model. Experimental results show their proposed model achieves comparable or better performance in prediction accuracy and coherence compared to SOTA models.", "questions": "1. About the volatile predictions of Maia model, is this problem more due to prediction errors or the volatile nature of human playing data? For example, a move that relies on several subsequent moves to be effective might be a bad choice for middle-level player, but a good choice for both low-level (where the opponent doesn’t know how to counter) and high-level players (where the active player can manage the subsequent changes). I’m not an expert in Chess, just wondering whether this is a possible cause of the incoherent problem. \n2. In Figure 10, is Maia-2 actually the proposed Hermes model? This evaluation of win-rate seems indirect. Why not directly pair Hermes-1500 (by setting the active player level to 1500, for example) with Maia-1500 and evaluate the final outcome, to determine which model is better if the goal is to evaluate move quality?", "limitations": "No significant limitation.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "This work is overall very solid. The proposed model is clearly presented, with enough details for reproduction. Evaluation is also convincing and thorough.", "weaknesses": "A minor problem is about the skill level encoder. The proposed Hermes model encodes both players’ skills, but Maia model only encodes one player’s, which adds an advantage to Hermes model and makes comparisons in human move prediction potentially unfair.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Skill level encoder (W1)**\n\nMaia implicitly encodes both players’ skill levels by only selecting the games between the same-strength players for model training. In Table 1, we have equated the active and opponent skill levels to ensure fair comparisons, and our model outperforms Maia in this setting. Note that Maia is restricted to training on games between players at the same skill level, which is a significant limitation on the distribution of data it considers.\n\nNot only does Hermes outperform Maia in Maia’s setting, but our unified model is also capable of properly modeling situations where the player skill levels are different. This allows for better flexibility and improved performance on both matched and unmatched skill-level settings (Figure 2). In addition, having both ratings can help derive insights into human behavior. For example, improving the players' skills themselves affects human decisions more than the varied opponent skill levels (Figure 3B).\n\n**Maia’s Incoherence (Q1)**\n\nWe strongly suspect that Maia’s incoherence is much more due to noise than to signal. While it’s possible that some positions may have non-monotonic relationships with the probability of choosing the “best” move, it is highly unlikely that the true relationships are as chaotic as Maia’s predictions (which often change direction 4–5 times throughout the 9-step skill range).\n\n**Figure 10 (Q2)**\n\nYes, it is Hermes, which we already modified. Thank you for pointing out this typo which we will fix. We didn’t aim to evaluate the predicted move quality of Hermes or Maia because our goal is to replicate human moves instead of maximizing quality. Figure 10 shows the human move prediction accuracy conditioned on move quality: for better or worse moves, how well can we predict? The higher the win-rate loss, the lower the move quality. Therefore, we find that human move prediction models are good at higher-quality moves, which are more certain, and they get worse at lower-quality moves, which are more random and thus hard to predict. Our unified model Hermes still outperforms the Maia under such settings.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yBmSaANrry", "reviewer": "Reviewer_n9LM", "summary": "The paper introduces a unified modeling approach called Hermes for aligning human and AI performance in chess. Hermes effectively captures human chess styles across different skill levels, providing a coherent reflection of player improvement. The approach incorporates a skill-aware attention mechanism that dynamically combines player skill levels with encoded chess positions, allowing the model to adapt to varying player skills. Experimental results demonstrate that this unified framework significantly enhances the alignment between AI and human players, facilitating a deeper understanding of human decision-making and the development of AI-guided teaching tools.", "questions": "1. How is skill defined in this paper? Is it manually designed or given by the chess setting? How does it handle this setting, and why can't we implicitly obtain different skill levels?\n2. How could this method be expanded to settings other than chess, such as those used in multi-agent systems[1]?\n3. How can this method be expanded to different chess variants or adapted for a large-scale chess model?\n\nRef:\n[1] Yuan L, Zhang Z, Li L, et al. A survey of progress on cooperative multi-agent reinforcement learning in open environment[J]. arXiv preprint arXiv:2312.01058, 2023.", "limitations": "A/N", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is easy to read and follow, with detailed descriptions and accompanying code.\n2. The method is simple yet effective, demonstrating remarkable results in various settings.", "weaknesses": "1. The method is actually straightforward, as it only employs the multi-head attention mechanism, a general technique applicable in various settings and applications.\n2. The motivation for using attention and the reasons for its effectiveness are unclear.\n3. The concept of skill used in this paper is not well-defined, which may limit its applicability to the chess environment only.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Design choices (W2)**\n\nWe agree that we didn’t sufficiently explain our architecture choice. The rationale behind our design is that each channel (feature map) of the ResNet output represents different aspects of a chess position, and the attention blocks actively select and interact with the features according to the given skill level. Evidence can be found in Figure 4: for skill-dependent concepts, the representation before attention blocks understands them uniformly across all skill levels with high accuracy, whereas after the attention blocks the representation is attenuated with higher skill levels understanding the concepts better than lower skill levels. This shows that attention blocks effectively “pretend” not to know the concepts to model the degree of imperfection in human moves, whereas skill-independent concepts are understood similarly by the representations before and after the attention block. Table 3 “w/o Att” shows that simply concatenating the skill embeddings with the flattened ResNet outputs did not work well. Thus, a more sophisticated way of conditioning is needed.\n\n**Contribution (W1)**\n\nTo the best of our knowledge, we are the first to emphasize **coherence** in human behavior modeling. In particular, our work introduces the first human behavior model that is coherent across various skill levels and even improves the move-matching accuracy.\n\nThe primary methodological contribution lies in the **unified modeling** approach for **coherent** human behavior modeling that is enabled by this specifically designed model architecture. While we believe that the architecture has conceptual benefits that help make these contributions possible, and we will include the rationale behind the architecture in the revisions, we do not claim that the architecture is optimal in any sense; we view the specifics of the architecture as secondary to the advances, i.e., move prediction coherence and accuracy, provided by the unified modeling approach. We will also mention other architecture choices as promising avenues for future work.\n\n**Definition of Skill (W3, Q1)**\n\nThe skill level is defined by the Elo rating system, which was originally proposed for two-player zero-sum games [1] and widely used in Chess. Each player is associated with an Elo rating before starting a game, and the Elo rating will be updated after the game. We use the Elo rating before the game as annotated labels for skill levels.\n\nExtracting skill embeddings from past moves is another research question, which gives individual skill embeddings instead of grouped skill embeddings. The performance can be promising if sufficient past moves are provided [2]. We see this as a promising direction for future work and will add it to the paper.\n\n**Generalization to other domains (Q2)**\n\nElo rating systems now become widely applicable in many domains such as LLMs [3]. Besides Elo, our categorical skill-level modeling can be easily adapted to any continuous rating or discrete grading system. For example, proficiency in math problem-solving.\n\nUnlike multi-agent systems, we treat this setting as effectively single-agent, where the chess engine itself will predict human moves without interacting with any other agents. Although Hermes itself is not designed as a multi-agent system, it could be extended to multi-agent systems in the broader context of studying interactions between human-like agents to achieve specific goals, in particular competitions among agents.\n\n**Adapting to Chess Variants (Q3)**\n\nIt is fairly easy to apply our method for chess variants as long as the human historical data is sufficient and the skill level is provided. In particular, the Lichess database provided human historical data for Antichess (27.7M games), Atomic (21.6M games), and Chess960 (20.1M games), the only difference would be the data and the labels for the prediction heads.\n\n[1] Wikipedia. 2024. \"Elo Rating System.\" Wikimedia Foundation. Last modified July 22, 2024.\n\n[2] McIlroy-Young, Reid, et al. \"Learning models of individual behavior in chess.\" KDD 2022.\n\n[3] Chiang, Wei-Lin, et al. \"Chatbot arena: An open platform for evaluating llms by human preference.\" arXiv 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thank you for your thoughtful reviews and constructive suggestions for our work. Our work has been recognized as addressing "an important problem of skill-aware modeling of human behavior" (Reviewer 1), introducing an "innovative skill-aware attention mechanism” (Reviewer 2), and being "very solid" with "convincing and thorough" evaluation (Reviewer 3). Reviewer 4 appreciates our "simple yet effective" methodology, yielding "remarkable results in various settings." 

Here is a brief overview of the main concerns addressed.

**Main focuses & Broader Impact (R1, R2, R4)**

We contribute an approach to human move prediction that is not only the new state of the art for accuracy, but our model achieves **coherence** in its predictions. To power algorithmic teaching tools, we believe that it is not enough to treat different skill levels independently and make predictions that don’t make coherent sense. Instead, we need coherent move predictions to algorithmically capture the trajectory of human ability as we progress from beginner mistakes to expert decisions. This way, we enable the building of systems that can guide people along efficient learning paths. To accomplish this, we design a skill-aware attention mechanism for the **unified modeling** of human behavior across various skill levels, instead of modeling each skill level independently as previous methods did.

**Design choices (R1, R3, R4)**

We agree that we didn’t sufficiently explain our architecture choice. The rationale behind our design is that each channel (feature map) of the ResNet output represents different aspects of a chess position, and the attention blocks actively select and interact with the features according to the given skill level. Evidence can be found in Figure 4: for skill-dependent concepts, the representation before attention blocks understands them uniformly across all skill levels with high accuracy, whereas after the attention blocks the representation is attenuated with higher skill levels understanding the concepts better than lower skill levels. This shows that attention blocks effectively “pretend” not to know the concepts to model the degree of imperfection in human moves, whereas skill-independent concepts are understood similarly by the representations before and after the attention block. Table 3 “w/o Att” shows that simply concatenating the skill embeddings with the flattened ResNet outputs did not work well. Thus, a more sophisticated way of conditioning is needed.

Our primary methodological contribution lies in the **unified modeling** approach for **coherent** human behavior modeling that is enabled by this specifically designed model architecture. While we believe that the architecture has conceptual benefits that help make these contributions possible, and we will include the rationale behind the architecture in the revisions, we do not claim that the architecture is optimal in any sense; we view the specifics of the architecture as secondary to the advances, i.e., move prediction coherence and accuracy, provided by the unified modeling approach. We will also mention other architecture choices as promising avenues for future work.

**Human studies (R1, R2)**

In an ideal human experiment, we would give a position to a human at a particular rating, and compare their chosen move to our model output. Our experiments do exactly this, and thus we view them as massive human studies that measure the move-matching accuracy and coherence with the recorded behaviors of real humans.

In addition to this main experiment, we've also performed additional experiments that address other dimensions of this question. In particular, we've implemented a randomized experiment on Lichess: human players challenge our bots, and we randomize whether players play against Maia or Hermes. Our final result is that our higher move-matching and our vastly improved coherence, across all skill levels, come at no cost to human subject engagement, and in fact slightly increase engagement: players rematch Hermes (the new system in this paper) after the first game 1.5 percentage points more than Maia (40.6% vs. 39.1%). Although engagement is not our main objective (move-matching and coherence are) this is further promising evidence that we have achieved our goal of a human-aligned AI model that coherently captures human style across different skill levels.

More detailed responses are provided to individual reviews with pointers to **Limitations (L), Weaknesses (W), and Questions (Q)**.

We hope these clarifications have addressed your concerns and strengthened our paper.

Meta-review: This paper proposes a chess engine that predicts human moves at a specific ELO. It improves over a previous paper in this domain by using conditioning instead of bucketing. 

On one hand, the results can be significant for the chess community and make an improvement over a previous baseline.
On the other hand, it uses standard conditioning techniques, builds upon previous results, and presents incremental gains. As such, the results are somewhat incremental and will not have an impact on the wider machine learning community. 

After discussing and weighting the pros and cons, I have decided to recommend accepting the paper. I understand that chess is a hard domain, and that progress in this domain can then be applied in other domains. I am also confident that this work will impact the small "AI in chess community" and will most likely be picked up in future work. 

I have a few suggestions for how to improve the paper. 

Firstly and most importantly, the paper is dealing with modeling multiple policies with a single network. There has been a large body of work on the topic in the multi-agent RL community that is not covered well enough. As such, this work is not likely to be noticed by that community. I suggest discussing this work. For example, https://arxiv.org/abs/2308.09175 is even in chess. 

Secondly, it will be useful to explain the readers the implication of accuracy at 50%. Is this a "good number"? what I mean here is that usually we would like to reach 100%, but this is clearly not possible in chess when we model a distribution of players. Thus, we need some way to calibrate these results and understand how much headroom there is still for improvement. 

In the discussion paragraph you are proposing to fine tune the model on historical data of a specific player. Is that something you can add to the paper? What would be the accuracy in this case?

It will also be interesting to do some error analysis. For example, you can show positions in which the current model makes the correct prediction and the previous one is wrong. There might be qualitative differences between the models, or it might be limited to blundering less.. 

Please try to address my suggestions in the final version of the paper. In particular, the literature review.
Decision: Accept (poster)