Strength Estimation and Human-Like Strength Adjustment in Games
Authors: Chun Jung Chen
Chung-Chin Shih
Ti-Rong Wu
Venue: ICLR 2025 Poster (2025)
Keywords: Bradley-Terry Model
Strength Estimation
Strength Adjustment
Human-like Playing Style
Monte-Carlo Tree Search
Go
Chess
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: This paper proposes a strength system that can estimate the strength from games and provide various playing strengths while simultaneously offer a human-like behavior in both Go and chess.

Abstract: Strength estimation and adjustment are crucial in designing human-AI interactions, particularly in games where AI surpasses human players. This paper introduces a novel strength system, including a *strength estimator* (SE) and an SE-based Monte Carlo tree search, denoted as *SE-MCTS*, which predicts strengths from games and offers different playing strengths with human styles. The strength estimator calculates strength scores and predicts ranks from games without direct human interaction. SE-MCTS utilizes the strength scores in a Monte Carlo tree search to adjust playing strength and style. We first conduct experiments in Go, a challenging board game with a wide range of ranks. Our strength estimator significantly achieves over 80% accuracy in predicting ranks by observing 15 games only, whereas the previous method reached 49% accuracy for 100 games. For strength adjustment, SE-MCTS successfully adjusts to designated ranks while achieving a 51.33% accuracy in aligning to human actions, outperforming a previous state-of-the-art, with only 42.56% accuracy. To demonstrate the generality of our strength system, we further apply SE and SE-MCTS to chess and obtain consistent results. These results show a promising approach to strength estimation and adjustment, enhancing human-AI interactions in games. Our code is available at https://rlg.iis.sinica.edu.tw/papers/strength-estimator.

Published as a conference paper at ICLR 2025
STRENGTH ESTIMATION AND HUMAN-LIKE STRENGTH ADJUSTMENT IN GAMES
Chun-Jung Chen Chung-Chin Shih', Ti-Rong Wu't
Institute of Information Science, Academia Sinica, Taiwan DDepartment of Computer Science, National Taiwan University, Taiwan
ABSTRACT
Strength estimation and adjustment are crucial in designing human-Al interactions, particularly in games where Al surpasses human players. This paper introduces a novel strength system, including a strength estimator (SE) and an SE-based Monte Carlo tree search, denoted as SE-MCTS, which predicts strengths from games and offers different playing strengths with human styles. The strength estimator calculates strength scores and predicts ranks from games without direct human interaction. SE-MCTS utilizes the strength scores in a Monte Carlo tree search to adjust playing strength and style. We first conduct experiments in Go, challenging board game with a wide range of ranks. Our strength estimator significantly achieves over 80% accuracy in predicting ranks by observing 15 games only, whereas the previous method reached 49% accuracy for 100 games. For strength adjustment, SE-MCTS successfully adjusts to designated ranks while achieving a 51.33% accuracy in aligming to human actions, outperforming a previous state- of-the-art, with only 42.56% accuracy. To demonstrate the generality of our strength system, we further apply SE and SE-MCTS to chess and obtain consistent results. These results show promising approach to strength estimation and adjustment, enhancing human-Al interactions in games. Our code is available at https:///tl..ii iis wtt.siicc......../apprsstttteeti esttmmat..
INTRODUCTION
Artificial intelligence has achieved superbuman performance in various domains in recent years, especially in games (Silver et al., 2018 Schrittwieser et al. 2020 Vinyals et al. 2019 OpenAl et al.) 2019). These achievements have raised interests within the community in exploring Al programs for human interactions, particularly estimating human players' strengths and offering corresponding levels to increase entertainment or improve skills (Demediuk et al. 2017]] Fan et al. 2019 Moon & 2020; Gusmão et al., 2013; Silva et al., 2015 Hunicke & Chapman 2004). For example, since the advent of AlphaZero, human players have attempted to train themselves by using Al programs. Subsequently, many researchers have explored several methods to adjust the playing strength of AlphaZero-like programs to provide appropriate difficulty levels for human players (Wu et al. 2019
Liu et al. 2020 Fujita 2022).
However, although these methods can provide strength adjustment, two issues have arisen. First, while these methods can offer different strengths, human players often need to play several games or manually choose AI playing strength, consuming time find suitable strength for themselves. Second, the behaviors between AI programs and human players are quite different. This occurs because most strength adjustment methods mainly focus on adjusting AI strength by calibrating the win rate to around 50% for specific strengths, without considering the human behaviors at those strengths. The problem is further exacerbated when human players attempt to use AT programs to analyze games and learn from the better actions suggested by AI. Therefore, designing AI programs that can accurately estimate a player's strength, provide corresponding playing strengths, and simultaneously offer human-like behavior is crucial for using superhuman AI in human learning.
"Theee authors contributed equally. Corresponding author: tirongwu @ iis sinica.edu.tw

---

Published as a conference paper at ICLR 2025
To address this challenge, this paper proposes novel strength system, including a strength estimator and an SE-based MCTS, denoted as an SE-MCTS, which can predict strengths from games and provide different playing strengths with human-like playing style. Specifically,
propose strength estimator, based on the Bradley-Terry model (Bradley & Terry 1952). which estimates strength score of action strength score can be further used to predict the strength of any given game, providing strength estimation without direct human interaction. Next, we present novel strength adjustment approach with human-like styles, named SE-MCTS, by incorporating the strength estimator into the Monte Carlo tree search (MCTS). During the MCTS, the search limited to exploring actions that closely correspond to a given targeted strength score. We conduct experiments in Go, challenging board game for human players with a wide range of ranks. The results show several advantages of using our approach. First, the strength estimator significantly achieves over 80% accuracy in predicting ranks within 15 games, compared to the previous method only achieves 49% accuracy even after evaluating 100 games. Second, the experiments show that SE-MCTS can not only provide designated ranks but also achieve a playing style with 51.33% accuracy in aligning to human players' actions, while previous state-of-the-art only obtained 42.56% accuracy. Finally, the strength estimator can be trained with limited rank data and still accurately predict ranks. Furthermore, to demonstrate the generality of the proposed method, and significantly outperforming the previous state-of-the approach in both strength estimation and adjustment. These results show a promising direction for enhancing human-Al interactions in games.
game state, with higher scores indicating stronger actions. The
apply SE and SE-MCTS to chess, achieving consistent results
BACKGROUND
2.1 BRADLEY-TERRY MODEL
The Bradley- Brerley-Terry model (Bradley & Terry 1952) is often used for pairwise comparisons, allowing
for the estimation of outcomes between individuals based on their relative strengths. In a group of individuals, the model calculates the probability that individual defeats individual j as P(i > j)
X. where A and >; represent the positive values of individuals i and j. respectively. A higher X; indicates a stronger individual. In practice, A; is usually defined by an exponential score function as X;- ee". where 3, represents the strength score of individual i.
The Brerley-Terry model can be further generalized to include comparison among more than two individuals (Huang et al. 2006). Consider group consisting of k individuals, indexed from to k. The probability that individual i wins out over the other individuals in this group is calculated as P(i) +x Furthermore, the model can be adapted for team comparisons, where each team comprises multiple individuals. For example, assume team a consists of individuals I and 2, team b consists of individuals 2, 3, and 4, and team consists of individuals 1, and 5, Then, the probability that team a win against team and is defined as P(team a) = Aa Ag where the strength of each team determined by the product of the strengths of its individual members. Due to its generalization and broader extension, the Bradley-Temy model has been widely used in various fields, such as games (Coulom 2007a), sports (Cattelan et al. 2013), and recommendation systems (Chen & Joachims 2016).
A,
2.2 MONTE-CARLO TREE SEARCH
Monte Carlo tree search (MCTS) (Coulom 2007b Kocsis & Szepesvári 2006 is a best-first search al- gorithm that has been successfully used by AlphaZero (Silver et al. 2018) and MuZero (Schrittwieser et al. 2020) to master both board games and Atari games. In AlphaZero, each MCTS simulation begins by traversing the tree from the root node to a leaf node using the PUCT (Rosin) 201T)
formula:
a" = arg max ((ls..) c. (c.s,))
(1)
1+ 1+N(s,
where Q(s, a) represents the estimated Qavauue for the atate-action pair (s, Q). N(s, a) is the visit counts, P(s, a) is the prior heuristic value, and c is a coefficient to control the exploration. Next, the leaf node is expanded and evaluated by a two-head network, fo(s) = (p, v), where p represents the policy distribution and denotes the win rate. The policy distribution p serves the prior heuristic
2

---

Published as a conference paper at ICLR 2025
value. The win rate is used to update the estimated Q-value of each node, from the leaf node to its ancestors up to the root node. This process is repeated iteratively, with more MCTS simulations leading to better decision decision-making. Finally, the node with the largest simulation counts is decided.
2.3 MCTS-RASED STRENGTH ADJUSTMENT
Strength adjustment (Hunicke & Chapman 2004; Paulsen & Fürnkranz 2010, Silva et al. 2013 Moon & Seo 2020) is crucial in the design of human-AT interactions, especially since AlphaZero achieved superhuman performance in many games like Go, Chess, and Shogi. As MCTS is widely used in these games, various methods have been explored to adapt for strength adjustment (Sephton iet al., 20155 Wu et al. 2019 Demediuk et al., 2017 Fan et al. 2019) Moon et al. 2022). For instance, Sephton et al. )2155) proposes adjusting the playing strength by using strength index z. After the search, MCTS decides the node based on the proportionality of their simulation counts, with the N₁
probability of selecting node i calculated as "5= Ng. where N, represents the simulation counts
for node i. A larger value indicates a tendency to select stronger actions, while a smaller 2 favors weaker actions. |Wu et al. (2019) further improves this method by introducing threshold R to filter out lower-quality actions, removing nodes where Nj- < R x Nmaz. where Nmaz represents the node with largest simulation counts. The approach is used adjust the playing strength of ELF OpenGo (Tian et al., 2019), resulting in covering range of 800 Elo ratings within the interval 2 E [-2, 2j. However, both methods only change the final decision in MCTS without modifying the search tree, leaving the search trees identical for different strengths.
2.4 STRENGTH ESTIMATION
Strength estimation is another important technique related to strength adjustment. With accurate strength estimation, the Al can first predict a player's strength and subsequently provide an appropriate level of difficulty for human learning. Several methods (Moudřík & Neruda 2016 Liu et al. 2020
Egri-Nagy & Tormanen 2020; Scheible & Schütze 2014) have been proposed to estimate player strength in games. For example, Liu al. proposes estimating a player's strength by using the strength index with MCTS, as described in the previous subsection, play against human players. Specifically, the strength index is adjusted after each game according to the game outcomes. Their experiments show that 2 generally converges after about 20 games. However, this method requires human players to play against the MCTS programs with multiple games to obtain an estimation of their playing strengths. On the other hand, Moudřík Neruda (2016) proposes an alternative approach that categorizes the Go players into three ranks - strong, median, and weak and uses neural network to classify player ranks based on a game position using supervised learning. After training, given game position, the neural network predicts ranks for each position by selecting the highest probability. Furthermore, it can aggregate predictions across multiple positions. Two methods are presented: (a) sum, which sums probabilities of all positions and makes prediction based on the highest probability; and (b) vote, which predicts the rank of each position first and selects the most frequent rank. However, this approach does not consider multiple actions during training and the experiment was limited to only three ranks. In addition, strength estimation can be formulated ranking problem (Burges al. 2005 Xia et al.) 22008,, but it differs in key aspect. Ranking problems often focus on ordering items based on single query, whereas in games, strength assessed overall skills across multiple positions or games. This challenge requires aggregating rankings across various scenarios to capture player's 's ability.
METHOD
3.1 STRENGTH ESTIMATOR
We introduce the strength estimator (SE), which is designed to predict the strength of an action a at given state s based on human game records. Each state-action pair, denoted as p = (s, a). is labeled with a rank r that corresponds to the player's 's strength. For simplicity, in this paper, ranks are ordered in descending order where rank 1, denoted 71, represents the strongest level of play, and progressively higher numbers indicate weaker playing strength. Each rank corresponds to a group of players, assume that players with the same rank have equivalent strength. Ranks could be
3

---

Published as a conference paper at ICLR 2025
determined by Elo ratings; for example, players with an Elo between 2400 and 2599 are classified as r,, players with an Elo between 2200 and 2399 as r2, and so on.
Consider game collection D that consists of numerous state-action pairs p. each associated with a distinct rank r. Suppose there are n ranks in D, represented by r1, 72. Tn- Next, given a state- action pair p sampled from D, the strength estimator, denoted as SE(p) X, is designed predict the strength À corresponding to action a in state s. Consider two state-action pairs, Pl = (s, a1) and p2 (s, 02), where aj and 02 are played by ri and r2, respectively. The strength estimator is expected to predict strength λ for Pi and X2 for p2, such that Å 2 A2. Note that X A2 occurs when the actions d1 and 02 are identical or are of equal strength. Following the Bradley-Tery model, can calculate the probability that rı wins against 72 P(r1 > r2) A1+
A To generalize
ranks, consider state-action action pairs, P1, P2, .... Pn: corresponding ranks rı, 72, ... Iu respectively.
The strength estimator predicts X; for each The probability that r1 wins against all other ranks is
then calculated as P(r1 > {r2, T3...., Tn }n))
Furthermore, we extend the method to estimate the composite strength rank by incorporating multiple state-action pairs, collectively conceptualizing them team. This approach allows to effectively measure the overall capabilities players within specific ranks by considering various actions across different scenarios. Consider state-action pairs Pt,1, Pt.2, ... Pi,m= where Pa,j represents the j-th state- action pairs associated with Ty sampled from D. The strength estimator predicts the strength Ar,1, λi,2 ... Ai.m for each state-action pair, respectively. We define the composite strength for ry by aggregating all individual strengths using the geometric mean. The 1/m
composite strength, denoted as A,, is calculated as A, =
The geometric mean is used to ensure that the strength estimator provides stable estimations and reflects the rank's 's ability across different scenarios. Namely, A, should remain consistent regardless of the number of state-action pairs considered:
II SE(R)) j=1 PjND
A = (,,11,,2.... "mmm.....A..))//m
II
Nu.
(2)
where the state-action pair Pj is randomly sampled from the game collection D. We can further calculate the probability that rank ri wins against all other ranks by using the composite strength:
P(r1 > {r2, T3........................ ,r,)) A
A.
Note that proposed method of aggregating strength using the geometric mean for teams differs from the Bradley-Terry model, which utilizes product aggregation. However, the geometric mean is specifically tailored
scenarios guarantee consistent measurement of individual performance across different games, rather than focusing on team member interactions. This approach also helps to mitigate the influence of outliers. Moreover, this modification preserves the integrity of the Bradley-Tere model principles, ensuring that the order of teams is strietly followed.
3.2 TRAINING THE STRENGTHI ESTIMATOR
This subsection introduces a methodology for training strength estimator. For simplicity, we propose train a neural network, fo() B, as a strength estimator which predicts strength score 3 instead of strength X for given state-action pair This strength score, A,, serves as the exponent for strength A e as defined by the Bradley-Terry model. Then, the composite strength, A,, from the equation can be expressed by using 3 as follows:
1
A, =
II AJ
II ==1
=
Em Ai.s "i
(3)
where 3, represents the average strength scores of m state-action pairs, each with r; sampled from D.
Next, given n ranks in the game collection, the strength estimator is optimized by maximizing the likelihood L. according the ranking order (Xia et al. 2008 Chen et al. 2009). The likelihood is
4

---

Published as a conference paper at ICLR 2025
F1
B1
Bz
Boo
B2
B
{
Beo
Strength Estimator
everoge
B
Bn Bio
Figure 1: The training process of the strength estimator.
defined as follows:
C= P(ri
P(r2 Ta.......... ,4.., x x... P(rn-1 Fn) c
n-1 II
A; P(r: f124 rn)) A A +
(4)
ش== eo
i=1
Maximizing L ensures that the strength scores of rı, 72, .. Tn are strictly in descending order, such that Ti > 12 > ra .n. Then, the loss function can be defined by bog-likelihood:
L lCg - log II
log
(5)
Σ 3
Figure T the training process of the strength estimator. Assume there are ranks, T2, ... Tuy the game collection. Initially, for each r,, we sample m state-action pairs and evaluate them by the strength estimator. The strength estimator then outputs the strength score Br,j corresponding to each state-action pair. Subsequently, average all 3s, to obtain 3 for each r:- Finally, using all strength scores, Br we sequentially minimize each softmax loss as defined by the equation 5
Since the state-action pairs are collected only from human games, the strength estimator may provide unpredictable estimations for oul-of-distribution state-action pairs, which rarely appear in human games. To address this issue, we introduce an additional rank, into our training process, as depicted by the dashed rectangle in Figure [ This rank, Tso, defined the weakest among all ranks, ensuring that r; Too for all ri. To generate the state-action pairs for Toc, we first select a state-action pair, P; = (#i; a;) from any r,. Then, we disturb the state-action pair pt to Pso by modifying the action to randomly chosen legal action, resulting in Pso = (8t, aoo). Since random action as is highly likely to result in an inferior outcome, these actions are expected to correspond to the weakest rank. Note that although a random action may occasionally result in strong action, the impact of such outliers will be minimized by the average Box the number of samples, m, increases.
3.3 STRENGTH ESTIMATOR BASED MCTS FOR STRENGTII ADJUSTMENT
present a novel method that integrates a strength estimator with MCTS to adjust strength dynami- cally, named SE-MCTS. In previous strength adjustment approaches, as described in subsection 2.3 the MCTS search tree unmodified during the search, with only changing the final action decision after the search is complete. In contrast, we propose inherently modifying the search based target strength score to ensure that the search aligns more closely with the desired strength of ranks.
Specifically, in MCTS each node is evaluated by the strength estimator to obtain strength score, B(s, a), which represents the strength score of action a at state s. We can calculate the composite strength score 3(s, a), by averaging all 3 from the nodes within the subtree of state s. This is similar the method used to calculate estimated Q-values. Given targeted rank with strength score 3r,
5

---

Published as a conference paper at ICLR 2025
calculate the absolute strength difference for each node, which is denoted as ö(s, [(s, a) 3yl Higher values of å indicate that the action is unlikely chosen by a player of the target rank, while lower ó suggest that the actions closer to the strength of the target rank.
As the values 8(s, a) are unbounded and can be any non non-negative number, we normalize all values using the minimum and maximum values observed in the current search tree. This normalization ensures that the difference values are bounded within the [0, 1] interval, similar to the approach used in MuZero (Schrittwieser al., 2020). Then, the PUCT formula in MCTS selection is modified from equation as follows:
v N ((, 1+ N(s
a-" = arg max ((ssaa)
+c. +c. (P(s,a) - Ö(s, ((6s))))e.((s,a))))
(6)
where ö(s, a) is the normalized difference values of ö(8, a) and ej is a hyperparameter used to control the confidence of prior heuristic values and difference values. Note that we choose to use 8(s, a) to climinate the prior heuristic value P(s, a). rather than incorporating additional values like the use of Q(s, a). This is because, during the MCTS search, the algorithm first prioritizes actions based on higher prior heuristic values and gradually shifts the focus to actions with higher Q-values when the simulation counts increase. By combining P(s, a) and 0(s, a). we effectively adjust the prioritization of actions, thereby aligning the search more closely with the desired strengths.
4 EXPERIMENTS
4.1 EXPERIMENT SETUP
We conducted experiments using the MiniZero framework eW al. 2025). The human games collected" from Fox Weiqi (Foxwg. 2025), which is the largest online Go platform in terms of users. These games are collected from amateur 5 kyu to 9 dar and are ranked in order from the strongest to weakest as follows: 9 dan, 8 dan, .... 2 dan, 1 dan, 1-2 kyu, and 3-5 kyu, corresponding to rı, T2, ....... r11- Namely, a total of 11 ranks are used. Note that for kyu, we classify 1 kyu as one rank and 3 to 5 kyu as another rank, and we exclude games played by players ranked lower than 5 kyu. This is because kyu players are still mastering basic Go strategies, their ranks often change rapidly. Consequently, their games do not consistently correspond to their ranks. For the training dataset, we collect a total of 495,000 games, with 45,000 games from each rank. We also prepare a separate testing dataset, including candidate and query dataset. The candidate dataset is used to estimate an average strength score of each rank, including a total of 1, 1,100 games, with 100 games per rank. The query dataset is used for the strength estimator to predict the strength, containing total of 9,900 games, with 900 games per rank.
The network architecture of the strength estimator is similar to the AlphaZero network, consisting of 20 residual blocks with 256 channels. Given a state-action pair, the network outputs a policy distribution p. value v. and strength score B. The training loss for the policy and value network follows AlphaZero, while the loss for the strength estimator is defined by equation 5 During training, we aggregate the composite strength score B, by randomly selecting m = 7 state-action pairs from r,. Other training details are provided in the appendix.
4.2 PREDICTING RANKS FROM GAMES
The strength estimator can be utilized to predict ranks in games where the rank is unknown. We first calculate 3. for each r; by evaluating all games in the candidate dataset. Next, for games from the same unknown rank, Tus in the query dataset, a composite score By is calculated by the strength estimator. Finally, ra then determined to be r,, where 13,y B.] is the smallest among all ry-
We train two strength estimator networks, SE and SE where SE is trained with 11 ranks, and SE includes an additional rank, Too. for a total of 12 ranks. In addition, for comparison, we train another
We downloaded Go games from the Fox Wex online platform using its public download links. 'In the game of Go, Kyu represents the beginner to decent amateur level, ranging from 18 kyu to I kyu, with lower numbers indicating stronger kyu players; Dan denotes advanced amateur, ranging from 1 dan to 9 dan, with higher numbers indicating stronger dan players.

---

Published as a conference paper at ICLR 2025
network based on supervised learning (SL), SLgum. and SLyazee mentioned in subsection 2.4 Both SLyum and SLyot are trained to classify 11 ranks for a given state-action pair but with different aggregation methods.
The evaluation is conducted as follows. For each r; from the query dataset, each network evaluates all state-action pairs from N randomly selected games and then predicts rank. We repeat this prediction process 500 times for each N to ensure a stable estimation.
(9) dan)
SE
(8 dat)
SP
Accuracy.
Slys
(B.
BL...
1, (4 dan)
sktl
(5 dan)
(2 dan)
kya)
Tu kys)
Games
Shyas
Gammss
avecage
(a) Accuracy of rank predictions by different
(b) Accuracy of each rank for SE
networks.
Figure 2: Accuracy of rank prediction in Go, with the shaded area representing the 95% confidence
interval.
Figure shows the accuracy of predicting the games from the query dataset. From the figure, the two strength estimator networks significantly outperform the supervised learning networks. Both SE and SE perform nearly identical performance, achieving over 80% of accuracy with only 15 games and reaching an accuracy of 97.5% after evaluating 100 games. In contrast, both Sugun and SLvote reached an accuracy of 49%, even after evaluating 100 games, with SL performs slightly better than L Furthermore, as human players do not always perform consistently and may occasionally change their ranks by one rank either above or below their actual rank, the games within might involve players whose actual ranks are r;-1 or Ti+1, leading to slight fluctuations in the dataset. Therefore, we incorporate a prediction tolerance that allows for a deviation of one rank. Specifically, if the network predicts r;-1 or Ti+1 for ri. we consider this prediction accurate. The results show that both strength estimator networks achieve nearly 80% accuracy by only evaluating a single game, and perfectly predict the rank with accuracy of over 99% after only 6 games, while the supervised learning networks still cannot predict the correct ranks after 100 games. This result indicates that the ranks predicted by the strength estimators are close the actual ranks when the prediction is incorrect.
Figure 2b depicts the accuracy of each rank prediction for SExx with several interesting observations. First, T1 (9 dan) has the highest accuracy among all ranks. Since 9 dan is the highest rank on FoxWeiqe Wexeeiq,, professional Go players and Go Als are also classified at this level, thus creating significant strength gap between Fj and r2, and leading to a clearer distinction. This phenomenon results in some players, originally ranked at Tı (9 dan), who are relatively stronger compared to shows the lowest accuracy among all ranks.
T2 (8 dan), being relegated to 72- Consequently, Second, the prediction for 77 dan) is below the average. This is because new players on Fox Weiqi can only select maximum initial rank of 3 dan and must advance gradually. Therefore, many players at this rank are actually stronger than dan. Finally, the prediction for T11 (3-5 kyu) is also below average, corroborating to common understanding that the strength of kyu players is usually inconsistent. In conclusion, these results allow our strength estimator to provide evaluations of a game's 's ranking system, further offering developers to make adjustments. Details on the accuracy of each rank prediction for other networks are provided in the appendix.
4.3 ADJUSTING STRENGTH WITH STRENGTH ESTIMATOR
In this section, we evaluate the performance of SE-MCTS, as described in subsection 3.3 by incorporating the two trained strength estimator networks into MCTS to adjust the playing strength for game playing. We first calculate the composite strength score 3;, by averaging all 3 from the state-action pairs in r; from the candidate set. Although we assume that the strength score 3, for any state-action pair from r; should be similar, in practice, we observe that may vary across different
7

---

Published as a conference paper at ICLR 2025
action numbers, shown in Figure In the beginning, particularly for the first actions, all B are estimated as the same score. This is likely because the action choices at the beginning of Go have less variety, and weaker players can easily remember and imitate the openings from stronger players. Therefore, in the SE-MCTS, we propose using 3 instead of B as the target strength score for each action, where 3 represent the composite strength score at d-th action for rs.
(9 dan)
22.
100
72 dan)
P2 720
02.0 =00
60 +30
Strength Score
rs (7 don)
dan)
024 33A
35.
20 25.7
13.0 16.2
rs (5 dan)
(4 dan)
42000:334
00
550 +00
17 dan)
09 970-444
P (2 dan)
17:
420.00
12
16.2
dan)
220
312
00:
224
rw (1-2 kyu)
ru (3-5 kyu)
970
192
149
00:100
231 200 Move
220 24 R0:
Tp 330
58 254
00:200
224
Figure 3: The composite strength score from SE for each rank across different actions in games.
Figure 4: Win rate against SFx in Go.
To evaluate the performance, we select four MCTS programs: (a) MCTS, representing vanilla MCTS without any strength adjustment mechanism, (b) SA-MCTS, which utilizes the strength index z (Wu ctt al. 2019), (c) SE-MCTS, which uses SE network, and (d) SE -MCTS, which uses SE network. Except MCTS, the remaining three programs can be adjusted to different strengths across a total of 11 ranks. We use SA-MCTS, SE-MCTS, and S -MCTS; to represent the strength of each program correspond to r;. For SE-MCTS and SE -MCTS, we calculate the target strength score from the candidate dataset for strength adjustment. However, for SA-MCTS, since the strength index z does not directly correspond to any specific rank, we adjust for each to ensure that each SA-MCTS; and SE -MCTS; achieve a comparable win rate, i... approximately 50%. Each 2 is in Appendix C
Figure 4 shows the win rate for each program playing against baseline program, where the bascline is chosen as -E.--MCTS MCTS; with = 5 (5 dan). Generally, the win rate for SA-MCTS; decreases as i increases, except from
3 to i 6 where there is a slight fluctuation. This corroborates the experiments in (Wu et al. 2019). Interestingly, although SE can accurately predict the strength, SE-MCTS; cannot adjust the strength effectively. This is due the exploration in the MCTS search, which may inevitably explore actions that are rarely seen in human games. Without training using Voo. provides inaccurate strength scores for these unseen actions. In contrast, the win rate of SEx- consistently decreases as increases, demonstrating an accurate strength adjustment using strength scores. We also include an experiment with different baselines in Appendix C.I
Moreover, we are interested in whether these programs can align with the human player's 's behavior, i... if they can choose the same actions as human players. Therefore, we sample 50 human games from the query dataset for each rank and use four MCTS programs to play on every state-action pair. The accuracy is evaluated based on whether the programs choose the same actions as human players. The results are shown in Table [ From the table, generally, the accuracy of r; decreases when the number i increases for all four programs. This is because weaker players are more unpredictable
Table 1: Accuracy of move prediction for MCTS programs to human players in Go.
rank (dan/kyu)
MCTS
SA- MCTS
SE-MCT8
SE MCCS
(9 dan)
53.05% 1 0.95%
47.00%
0.95%
53.06%
0.95%
53.73%
0.95%
7g (8
53.79%.
0.97%
45.8 45.83%
53.96%
54.30%
0.97%
1% (7 dan)
52.10% + 0.98%
46.70%
0.98%
54.28%
0.97%
53.88%
0.98%
Y4 (6 dan)
52.50%. 0.92%
45.86%
0.92%
54.25%
0.92%
53.58%
0.92%
7s (5 dan)
19.48%-2 + 0.93%
42.29%
0.92%
52.00%
0.93%
50.35%
0.93%
Nh (4 dan)
49.44% 1 0.91%
42.72%
1 0.90%
53.11%
0.90%
50.87%
: 0.91%
(3
50.75% 0.89%
68.68%
53.71% %
0.89%
51.40%
0.89%
my (2 dun)
50.17% 1 0.93%
40.94%
0.92%
53.21%
0.93%
50.99%
1 0.93%
(1 dan)
48. 10% 1: 0.89%
40.94%
0.88%
$2.60%
0.89%
49.44%
0.89%
Pan kyu)
0.91%
36.58%
0.88%
50.06%
0.91%
47.
0.91%
Yor (3-5 kyn)
46.87% + 0.89%
36.64%
0.86%
$1.36%
0.89 %
48.23%
0.89%
average
50.35% + 0.28%
42.56%
0.28%
$2.87%
0.28
51.33% 0.28%
8

---

Published as a conference paper at ICLR 2025
than stronger players. For MCTS, it achieves high accuracy, exceeding 50%, but it cannot adjust strength as shown in Figure 4 In contrast, although SA-MCTS can adjust strengths, it achieves the lowest accuracy among all four programs. This is due to SA-MCTS selecting actions proportional to the simulation counts by using strength index 2. The randomness results in diverging from human playing styles. On the other hand, both SE-MCTS and SE achieve a high accuracy, better than MCTS. This is because SE-MCTS directly modifies the search by the strength scores, guiding the search to better align with human behavior. In conclusion, among all four programs, SE -MCTS S not only adjusts strengths to specific ranks but also provides playing styles that are closely aligned with those of human players at specific ranks.
4.4 TRAINING STRENGTH ESTIMATOR WITH LIMITED DATA
In this subsection, we investigate training a strength estimator with limited data. Unlike the supervised learning methods, Sm and SLyete. which require data from each rank, our method can estimate strength score and use it to predict ranks that were not observed during training. In niche games or those favored by specific group of enthusiasts, ranking systems are often not fully established due to limited number of game records, and some specific ranks may be sparsely populated with only few players. Therefore, it is intriguing to explore whether the strength estimator can generalize to these unseen strengths.
n (9 dan)
r. dan)
-- 1 (8 dan)
D (8 dan)
6 (7 dan)
- 6. [7 dan)
dan)
dan)
Acoured
dan)
n. dan)
F (3 dan)
P. dan)
Is (2 dan)
n. [2 dan)
[1 dan)
it dan)
(1-2 kyu)
(1-2
(3-5 kyu)
(3-5
Th averago
average
Games
Games
(a) 2-rank dataset.
(b) 3-rank dataset.
Figure 5: Accuracy of rank predictions on the limited dataset of Go.
We train SE on two separate datasets: the 2-rank dataset, containing Y1 (9 dan) and T11 (3-5 kyu), and the 3-rank dataset, which includes the same ranks as the 2-rank dataset, plus an additional rank, To (4 dan). During the evaluation, we utilize the same methods, as described in 4.2, by using the strength estimator to calculate 3, for each r; in both candidate and query datasets, and then perform the predictions. Figure aad and 55 55) the accuracy of rank prediction 2-rank and 3-rank datasets, respectively. On average, the strength estimator achieves an accuracy of over 80% after evaluating 38 games for the 2-rank dataset and 21 games for the 3-rank datasets, respectively. Although these numbers are larger than 15 the number of games needed by a strength estimator trained with all 11 ranks achieve over 80% accuracy it can still effectively predict ranks that were not seen during training. This suggests that the strength estimator can generalize across spectrum of ranks with few ranks. Note that if sufficient data for more ranks are available, the accuracy still improves. Our method provides way to both generalize with limited data and enhance performance more data becomes available.
4.5 GENERALIZING TO OTHER GAMES
We further experiment in another game, chess, to demonstrate the generality of our SE and SE-MCTS approaches. Similar to Go, chess also popular game with abundant human game records. The games were collected from Liches (Lichess 2025). which uses Elo ratings its ranking system. collect games with Elo ratings ranging from 1,000 to 2,600 and categorize them into eight ranks, with each rank covering 200 Elo points and 240.000 games, for total of 1,920,000 games. For the testing dataset, the candidate dataset consists of 960 games, with 120 games per rank, while the query dataset contains 9,600 games, with 1,200 games per rank. Then, apply experiments chess,
II Lichess is one of the most popular online chess platforms, with millions of active users.

---

Published as a conference paper at ICLR 2025
similar to those in Go. Note that the training algorithms remain identical, except the imput features of the neural network changed from Go to chess.
The results are consistent with the findings in Go. First, shown in Figure 6 SEx achieves over 80% accuracy in predicting ranks with only 26 games, significantly outperforming both SL and SIyetee and reaches accuracy of 93.38% after evaluating 100 games. Second, Figure 7 shows the win rate for each -MCTS, when playing against -MCTSS. The win rate consistently decreases as increases, demonstrating SE MCTS can adjust its strength in chess. Finally, Table shows the accuracy of predicting human moves, with SExe - -MCTS achieving an average of 47.25% accuracy in aligning with human actions, outperforming both MCTS (46.56%) SA-MCTS (40.21%) 1%). and SE-MCTS (38.93%). In conclusion, this experiment demonstrates the versatiliity of our approach.
216
916
ST.
27. 25.7
240
242
16-
13.8 44.6
636-133
SE11
43:
50.
BE,
16-
56:33
384-12
256
Games
Tel
22:24
252
Figure 6: Accuracy of rank prediction in chess.
Figure 7: Win rate against SEx0 -CTS5 in chess.
Table 2: Accuracy of move prediction for MCTS programs to human chess players.
rank (Elo)
MCTS
EA-MUIS
SE-WC:S
GEa MMCSS
(2400- - 2500)
51.97%
1. _0.9%%
50.17%
0.69%
41.05%
0.68%
51.51/
1. 0.69%%
(2200- 2299)
51.58%
-0.9%%
47.49%
0.09%
40.19%
0.68%
51.14%
d: 00.9%%
(2000-
2199)
19.23%
0.69%
45.01%
0.69%
41.40%
0.68%
49.19%
0.69%
41.78%
0.68%
0.69%
rs (1600-
17700)
45.45%
0.60%
38.185
0.67%
36.76%
0.67%
46.62%
0.69%
7. (1400- 11500)
44.355-
0.69%
36.84%
0.75
37
0.67%
46.12%
0.695
(1200- - 1399)
41.54%
0.68%
31.49%
0.61%
37.65%
1 0.67%
43.04%
0.68%
(1000- 1198)
89.898
0.68%
30.72%
0.64%
38.05%
0.67%
43.08%
0.68%
average
0.24%
40.21%
38.90%
0.24%
5 DISCUSSION
This paper introduces a novel strength system, including a strength estimator for evaluating the strength from game records without requiring direct interaction with human players, and an SE-MCTS for adjusting the playing strength using strength scores provided by the strength estimator. When predicting ranks in the game of Go, our strength estimator significantly achieves over 80% accuracy by examining only 15 games, whereas the previous supervised learning method only reached 49% accuracy even after evaluating 100 games. The strength estimator can be trained with limited rank data and still accurately predict unseen rank data, providing extensive generalizability. For strength adjustment, SE-MCTS successfully adjusts to designated ranks while providing a playing style that aligns with human behavior, achieving an average accuracy of 51.33%. compared to the previous state-of-the-art method that only reached 42.56% accuracy. Furthermore. we apply our method to the game of chess and obtain consistent results to Go, demonstrating the generality of our approach.
One limitation of our work is that the strength estimator relies on human game records for training. However, this issue could potentially be addressed by using all models trained by AlphaZero, which may serve as players of different playing strengths to generate games. Besides, the strength system also provides several benefits for future directions. For example, game designers can use strength estimator to evaluate their ranking systems. The strength estimator can evaluate a game by examining the strength scores for each action, and use to identify incorrect actions for human players or for cheat detection (Alayed et al.) (0013). Furthermore, we can extend our strength estimator by incorporating aepooent-speciife strength scores to address the Bradley-Terry model's limitations in capturing intransitivity (Balduzzi et al. 2018 Bertrand et al. 2023) Omidshafiei et al., 2019 Vadori
& Savani 2024). Finally, the search tree of SE-MCTS can offer the opportunity for explainability of Al actions in human learning.
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
This paper presents a method for estimating player strength and adjusting it to specific ranks, allowing agents to play in human-like manner. potential ethical concern that our method could be exploited by human players to develop human-like agents for cheating in human competitions. However, we emphasize that all models trained in this paper are used strictly for research purposes and adhere established ethical guidelines.
REPRODUCIBILITY STATEMENT
We have provided detailed descriptions of the method, implementation, and training hyperpa- rameters in Section 3 Section 4 and Appendix [ to facilitate the reproduction of our exper- iments. The source code, along with README file containing instructions is available at https:/rlg..is..o http::/lh.iiiiioooo.oo/////p/tep
ACKNOWLEDGEMENT
This research is partially supported by the National Science and Technology Council (NSTC) of the Republic of China (Taiwan) under Grant Number NSTC 13-2221-EE0011000- NSTC 113-263--F-A49-00 and NSTC 113-2221-F-A49-1277
REFERENCES
Hashem Alayed, Fotos Frangoudes, and Clifford Neuman. Behavioral-based cheating detection in online first person shooters using machine learning techniques. In 2013 IEEE Conference on Computational Inteligence in Games (CIG), pp. 1-8, August 2013.
David Balduzzi. Karl Tuyls, Julien Perolat, and Thore Graepel. Re-evaluating evaluation. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
Quentin Bertrand, Wojciech Marian Czarnecki, Gauthier Gidel. On the Limitations of the Elo, Real-World Games are Transitive, not Additive. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, pp. 2905-2921. PMLR, April 2023.
Ralph Allan Bradley and Milton E. Terry. Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons. Biometrika, 39(3/4):324-345 1952.
Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. Learning rank using gradient descent. In Proceedings the 22nd International Conference on Machine Learning, ICML. '05, pp. 89-96, New York, NY, USA, August 2005. Association for Computing Machinery.
Manuela Cattelan, Cristiano Varin, and David Firth. Dynamic Bradley-Teny modelling of sports tournaments. Journal of the Royal Statistical Society: Series C, Volulme 62(Number 1):135-150. January 2013.
Shuo Chen and Thorsten Joachims. Predicting Matchups and Preferences in Context. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 16. pp. 775-784, New York, NY, USA. 2016. Association for Computing Machinery.
Wei Chen, Tie-yan Liu, Yanyan Lan, Zhi-ming Ma, and Hang Li. Ranking Measures and Loss Functions in Learning to Rank. In Advances in Neural Information Processing Systems, volume 22. Curran Associates, Inc., 2009.
Rémi Coulom. Computing Elo Ratings of Move Patterns in the Game of Go. ICGA Journal, 30(4): 198-208, December 2007a.
Rémi Coulom. Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. Computers and Games, Lecture Notes in Computer Science, pp. 72-83, Berlin, Heidelherg, 2007b. Springer.

---

Published as a conference paper at ICLR 2025
Simon Demediuk, Marco Tamassia, William Raffe, Fabio Zambetta, Xiaodong Li, and Florian Mueller. Monte Carlo tree search based algorithms for dynamic difficulty adjustment. In 2017 IEEE Conference Computational Intelligence and Games (CIG), pp. 53-59, August 2017.
Attila Egri-Nagy and Antti Tormanen. Derived metries for the game of Go intrinsic network strength assessment and cheat-detection. 2020 Eighth International Symposium on Computing and Networking (CANDAR). pp. 9-18, November 2020.
Tianwen Fan, Yuan Shi, Wanxiang Li, and Kokolo Ikeda. Position Control and Production of Various Strategies for Deep Learning Go Programs. In 2019 International Conference on Technologies and Applications of Artificial Intelligence (TAAI), pp. 1-6, January 2019.
Foxwq. Fox Weiqi, 2025. URL https: / www. foxwq. com/
Accessed: 2025-03-21.
Kazuhisa Fujita. AlphaDDA: Strategies for adjusting the playing strength of fully trained AlphaZero system to suitable human training partner. PeerJ Computer Science, 8:e 123, October 2022.
René Gusmão, Kennet Calixto, and Caetano Segundo. Dynamic difficulty adjustment through parameter manipulation for Space Shooter game. In 2015 14th Bruzilian Symposium on Computer Games and Digital Entertainment (SBGames), September 2015.
Tzu-Kuo Huang, Ruby Weng, and Chih-Jen Lin. Generalized Bradlcy-Teryy Models and Multi-Class Probability Estimates. Journal of Machine Learning Research, 7:85-115, January 2006.
Robin Hunicke and Vernell Chapman. AI for dynamic difficulty adjustment in games. Challenges in game artificial intelligence AAAI workshop, 2, January 2004.
Levente Koesis and Csaba Szepesvári. Bandit Based Monte-Carlo Planning. In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, volume 2006, pp. 282-293, September 2006.
Lichess. lichess.org open database, 2025. Hstandard games Accessed: 2025-03-21.
URL ltttps:/dattbase.. hiw.taabbse...chess.oooo
An-Jen Liu, Ti-Rong Wu, I-Chen Wu, Hung Guei, and Ting-Han Han Wei. Strength Adjustment and Assessment for MCTS-Based Programs [Research Frontier). IEEE Computational Intelligence Magazine, 15(3):60-73, August 2020.
Hee-Seung Moon and Jiwon Seo. Dynamic Difficulty Adjustment via Fast User Adaptation. In Ad- junct Publication of the 33rd Annual ACM Symposium on User Interface Software and Technology. pp. 13-15, October 2020.
Jac Young Moon, YouJin Choi, TacHwa Park, JunDoo Choi, Jin-Hyuk Hong, and Kyung-Joong Kim. Diversifying dynamic difficulty adjustment agent by integrating player state models into Monte-Care Carto tree search. Expert Systems with Applications, 205:1176771 November 2022.
Josef Moudřík and Roman Neruda. Determining Player Skill in the Game of Go with Deep Neural Networks. In Theory and Practice of Natural Computing, pp. 188-195, Cham, 2016. Springer International Publishing.
Shayegan Omidshafiei, Christos Papadimitriou, Georgios Piliouras, Karl Tuyls, Mark Rowland, Jean-Baptise Lespiau. Wojeiech M. Czarnecki, Marc Lanctot, Julien Perolat, and Remi Munos. a-Rank: Multi-Agem Evaluation by Evolution. Scientific Reports, 9(1):9937, July 2019.
OpenAI, Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemystaw Dębiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, Rafal Józefowicz, Scott Gray, Catherine Olsson, Jakub Pachocki, Michael Petrov, Henrique P. d O. Pinto, Jonathan Raiman, Tim Salimans, Jeremy Schlatter, Jonas Schneider, Szymon Sidor, Ilya Sutskever, Jie Tang, Filip Wolski, and Susan Zhang. Dota 2 with Large Scale Deep Reinforcement Learning, December 2019.
Philip Paulsen and Johannes Fürkkran.. A Moderately Successful Attempt to Train Chess Evaluation Functions of Different Strengths. In Proceedings of the ICML-10 Workshop on Machine Learning and Games, 2010.
12

---

Published as a conference paper at ICLR 2025
Christopher D. Rosin. Multi- bandits with episode context. Annals of Mathematics and Artificial Intelligence, 61(3):203-230, March 2011.
Christian Scheible and Hinrich Schütze. Picking the Amateur's Mind Predicting Chess Player Strength from Game Annotations. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 31 1-321, Dublin, Ireland, August 2014. Dublin City University and Association for Computational Linguistics.
Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy Lillicrap, and David Silver. Mastering Atari, Go, chess and shogi by planning with a learned model. Nature, 588 (7839):604 600)609,, December 2020.
Nick Sephton, Peter I. Cowling, and Nicholas H. Slaven. An experimental study of action selection mechanisms to create an entertaining opponent. In 2015 IEEE Conference on Computational Intelligence and Games (CIG), pp. 122-129, Tainan, Taiwan, August 2015. IEEE.
Mirna Silva, Victor Silva, and Luiz Chaimowicz. Dynamic Difficulty Adjustment through an Adaptive AL In 14th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames), pp. 173-182, November 2015.
David Silver, Thomas Hubert, Julian Schrittwicser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419): 1101)))):1111114:
December 2018.
Yuandong Tian, Jerry Ma, Qucheng Gong, Shubho Sengupta, Zhuoyuan Chen, James Pinkerton, and Larry Zitnick. ELF OpenGo: An analysis and open reimplementation of AlphaZero. In Proceedings the 36th International Conference on Machine Learning, pp. 6244 6253. PMLR, May 2019.
Nelson Vadori and Rahul Savani. Ordinal Potential-based Player Rating, In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, pp. 118-126. PMLR, April 2024.
Oriol Vinyals, Igor Babuschkin, Wojciech M. Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung. David H. Choi. Richard Powell, Timo Ewalds, Petko Georgiev, Junhyuk Oh, Dan Horgan, Manuel Kroiss, Ivo Danihelka, Aja Huang, Laurent Sifre, Trevor Cai, John P. Agapiou, Max Jaderberg, Alexander S. Vezhnevets, Rémi Leblond, Tobias Pohlen, Valentin Dalibard, David Budden, Yury Sulsky, James Molloy, Tom L. Paine, Caglar Gulcehre, Ziyu Wang, Tobias Pfaff, Yuhuai Wu, Roman Ring, Dani Yogatama, Dario Wünsch, Katrina McKinney, Oliver Smith, Tom Schaul, Timothy Lillicrap, Koray Kavukcuoglu, Demis Hassabis, Chris Apps, and David Silver. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575(7782): 350-354, November 2019.
I.-Chen Wu, Ti-Rong Wu, An-Jen Liu, Hung Guei, and Tinghan Wei. On Strength Adjustment for MCTS-Based Programs. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01): 1222-1229, July 2019.
Ti-Rong Wu, Hung Guei, Pei-Chiun Peng, Po- Wei Huang, Ting Han Wei, Chung-Chin Shih, and Yun-Jui Tsai. MiniZero: Comparative Analysis of AlphaZero and MuZero on Go, Othello, and Atari Games. IEEE Transactions on Games, 17(1):125-137, March 2025.
Fen Xia, Tie- Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. Listwise approach to learning to rank Theory and algorithm. In Proceedings of the 25th International Conference Machine Learning, pp. 1 1922-1999, January 2008
13

---

Published as a conference paper at ICLR 2025
A DETAILED TRAINING SETTINGS FOR THE STRENGTH ESTIMATOR
The feature design in the strength estimator of Go is similar to AlphaZero (Silver et al. 2018 Specifically, we use 18 channels to represent a board position, where the first 16 channels are the board configurations from the past eight moves for both black and white stones. The remaining two channels are binary indicators of the color of the next player, i.e., one channel for Black and White. For chess, the feature design also follows the same approach as AlphaZero, which includes 119 input channels. During training, for each rank, we randomly select seven state-action pairs. We also perform data augmentation to further enhance the diversity of the training data. The network is optimized using stochastic gradient descent (SGD), with the loss function specified in Equation 5 It is important to note that when training . the policy and value loss for state-action pairs of To:: are not calculated, since these heads should only consider actual human players' actions. The learning rate is initially set at 0.01 and is halved after 100,000 training steps. The entire training process encompasses 130,000 steps, consuming around 242 GPU hours for Go and 69 GPU hours for chess on NVIDIA RTX A5000 graphics card. Other hyperparameters listed in Table 3
Table 3: Hyperparameters for training strength estimators.
Parameter
Go
Chess
Number of Blocks
20
20
Input Channel
18
119
Hidden Channel
256
256
Learning Rate
0.01 to 0.005
0.01 to 0.005
Training Steps
130,000
130,000
Optimizer
SGD
SGD
Main Memory
384GB
Central Processing Unit (CPU)
Intel Xeon Silver 4216 (2.1 GHz)
Graphical Processing Unit (GPU)
NVIDIA RTX A5000
GPU Hours
242
69
B IN-DEPTH ANALYSIS FOR STRENGTH ESTIMATOR IN Go
We conduct in-depth analyses for strength estimators in Go. First, we present detailed insights into predicting ranks from games, detailed in Subsection [B]]] Second, we demonstrate the outcomes of strength prediction using fewer moves in single game, discussed in Subsection B.2 Finally, we explore predictions based solely on the first 50 actions or the last 50 actions in games, which are elaborated in Subsection B.3 and Subsection B.4 respectively.
B.1
PREDICTING RANKS FROM GAMES
Figure shows the accuracy of rank predictions for different networks. We observe that in Figure 8a and Figure {b] ST and ST.gum can only distinguish on some ranks, such as 3-5 kyu. This is because these models do not contain sufficient information to differentiate all ranks based on single state-action during training (Moudřík & Neruda 2016). Figure &c and Figure Sd even though we incorporated prediction tolerance for these two methods, they still cannot perfectly distinguish all ranks, even after 100 games. In Figure 8e and Figure 8f although our models SE and SE cannot perfectly predict all ranks without incorporating prediction tolerance, they still achieve high Saare88gaaa 8gland and Figure [8h] when we allow a prediction tolerance, we achieve 100% accuracy across all ranks. This result further indicates that our model can differentiate the strength relationship across all ranks.
performance across all ranks. In Figure
B.2 PREDICTING RANKS FROM GAME POSITIONS
We are also interested in whether we can predict the rank using only game positions. Specifically, only one game position can be chosen for each game instead of all actions when predicting the
14

---

Published as a conference paper at ICLR 2025
F. dan)
12 [9 dan)
72 can)
233
17 (7 dan)
10-
(7 dan)
Accuracy
.... (6 dan)
dan)
Accuracy
(5 dan)
(6 dan)
dan)
dan)
[2 dan)
n. dan)
(1 dan)
(1-2 kyo)
Pyi (1-2 kyu)
(3-5
03-b
average
sowrage
Games
Games
(a) L
(b) SLgun
n (9 dan)
dan)
15 18 dan)
15 dan)
(7 dan)
(7 dan)
(6 dan)
dan)
Accurasy
dan)
dan)
is (2 dan)
n. [2 dan)
i (1 dan)
n. [1 dan)
(1-2 kyn)
(1-2
(3-5 kyu)
(3-5
Games
Tha
average
Games
averago
(c) SIvetstl
(d) SL #1
08.
6. (9 dan)
r (8) dan)
(7 dan)
r. (8 dan)
dan)
dan)
dan)
Agcuragy
Accurass
dan)
17 (3 dan)
1 (2 dan)
n. dan)
[3 dan)
[1 dan)
(1-2 kys)
dan)
dan)
n. kyo)
(3-5 kyu)
kyo)
20
6
80
300
sesrage
overage
(e) SE
S
(9 dan)
[9 dan)
6. (8 dan)
(8 dan)
dan)
dan)
dan)
I1 dan)
n. dan)
I (, can)
r. [3 dan)
[3 dan)
Is dan)
(2 dan)
dan)
dan)
kyu)
kyo)
(3-5 kyu)
eu kyu)
11o
average
14
Games
average
SE
(h) SE +1
Figure Accuracy of rank prediction for different networks in Go.
ranks, shown in Figure 9 According to Figures 9a and 9b both SLvete and Ligus show similar performance as in the situation of using all actions. In our method, Figures P and 9d demonstrate that across 20 games, the accuracy decreases from 80%, when predictions are based on all actions, to approximately 60% when using just one action. This is intuitive because using the information from the entire game would help capture the player's strength. However, achieving 60% accuracy with unrelated actions among 20 games indicates that our model can still predict accurate ranks based on one action of different games.
B.3 PREDICTING RANKS FROM THE FIRST 50 ACTIONS IN THE GAME
We are interested in evaluating performance when using only the first 50 actions of game, known as the fuseki stage in Go. Figures [Oa and 10b indicate that SLyote and SLguns utilizing these initial actions, achieve similar performance to predictions made using all actions in the game. Figures 10c and T0d present the prediction results by our methods when limited the first 50 actions.
15

---

Published as a conference paper at ICLR 2025
F. dan)
13 (8 dan)
72 [9 dan)
19 (7 dan)
7. (7 dan)
... N 16 dan)
.... dan)
Accuracy
15 dan)
dan)
dan)
n (2 dan)
dan)
1 (1 dan)
n. dan)
Pr [1-2 kye)
(1-2 kyu)
(1-5 kys)
(3-5
avarage
average
Games
Games
(a) SLuote
(b) S L
n (9 dan)
dan)
15 18 dan)
1 dan)
(7 dan)
(7 dan)
(6 dan)
dan)
Agcuragy
dan)
dan)
is (2 dan)
n. dan)
n (1 dan)
n. dan)
(1-2 kyu)
(1-2
avezage kyu)
(3-5
Games
T1a
Games
averago
(c) SE
(d) SE
Figure Accuracy of rank prediction for different networks in the game positions of Go.
Clearly, the overall accuracy has declined. Notably, the accuracy for kyu players shows significant downward trends. This is mainly because the actions in the fuseki stage at these ranks are similar, thus complicating accurate predictions. This suggests that enhancing performance during the fuseki could be crucial for human players aiming to progress from kyu to dan rank.
B.4 PREDICTING RANKS FROM THE LAST 50 ACTIONS IN THE GAME
Similarly, we examine the performance when using only the last actions of the game, referred to as the yose stage in Go. Figures| 10e and 10 show that SLyote and SL employing these final actions, maintain similar performance to predictions based on all actions in the game. our method, Figures 10g and 10h display the prediction results using only the last 50 actions. As before, the overall accuracy has declined. However, the accuracy for dan players between predictions made using the entire game and just the last 50 actions does not differ significantly. This is likely because the yose stage involves complex calculations and judgments, areas where top players excel. Furthermore, in most games, especially those between the highest-skilled players, the outcome is often determined before the yose stage. This leads to less practice and proficiency in this phase among players of lower ranks. Additionally, we observe significant drop accuracy for 8 dan players when predictions are based solely yose stage. This could be because some 8 dan players have comparable yose skills to those dan players, leading to some misclassification of dan players as 9 dan.
16

---

Published as a conference paper at ICLR 2025
[9 dan)
r. dan)
(8 dan)
dan)
dan)
dan)
Accuracy
13
Accuracy
7. dan)
1 (4 dan)
n. dan)
E8 dan)
P. (8 dan)
(2 dan)
dan)
dan)
dan)
kys)
kep))
nu (3-5 kgu)
(3-5 kym)
average
avwrage
Games
Games
L with first 50 actions
(b) SLaun with first 50 actions
(9 dan)
dan)
dan)
dan)
dan)
Accuracy
1 15 dan)
Accuracy
n. dan)
(4 dan)
[4 dan)
[3 dan)
dan)
(2 dan)
dan)
0.
nu kyu)
(1-2 kyu)
nu (3-5 kyu)
Eu (3-5 kyu)
average
3/
overage
Games
Games
first 50 actions
3E with first 50 actions
dan)
dan)
dan)
ri (6 dan)
ri dan)
Accuracy
15 (5 dan)
Acouracy
n. dan)
(4 dan)
dan)
13 dan)
dan)
dan)
rs
ns kyu)
Po (1-2 leyn)
ny (3-5 kyu)
(3-5 kyu)
avezage
average
Games
Games
(e) S with last 50 actions
STaaun with last actions
1.c
dan)
dan)
13 dan)
dan)
r. (6 dan)
r. dan)
Accuracy
1 (5 dan)
Accuracy
(G dan)
dan)
dan)
dan)
dan)
ry
ry dan)
nu (1-2 kyu)
Pa (1-2 kyu)
nu (3-5 kyo)
(3-5 kyu)
0.
Games
Games
g SE last actions
(h) S8 with last 50 actions
Figure 10: Accuracy of rank prediction for different networks using only the first or the last 50 actions of the game Go.
17

---

Published as a conference paper at ICLR 2025
C DETAILED EXPERIMENTS FOR STRENGTH ADJUSTMENT
Table presents the value of 2 for SA SA-MCTS, used in subsection To ensure that each SA SA-MCTS; and aE -MCTS; achieve a comparable win rate, all methods are tested using fixed simulation count of 800. For SA -CCTS, since the strength index & does not directly correspond to any specific rank, we adjust for each r; to ensure that each SA-MCTS; and SEx -MCTS; achieve a comparable win rate. As shown in Table gradually decreases from to r11, aligning with the results in the original paper, which indicate that a greater z corresponds to a higher strength.
Table 4: z according to rank.
Rank
(9 dan)
0.6
12 (8 dan)
0.5
ra (7 dan)
0.35
ra (6 dan)
0.3
(5 dan)
0.2
r6 (4 dan)
0.15
r7 (3 dan)
0.05
rs (2 dan)
-0.1
(1 dan)
-0.2
110 (1-2 kyu)
-0.6
F11 (3-5 kyu)
-1.0
C.I ADJUSTING STRENGTH WITH DIFFERENT BASELINES
In Figure the baseline program is chosen as SE MMCTs with i 5 (5 dan). It would be interesting to examine whether the relative strength remains consistent when different baseline models are used. To further investigate this, conduct round-robin tournament by selecting five ranks (72, F4, T6. rs and F1o) and two representative methods (SA-MCTS and SE SEx -MCTS, excluding SE-MCTS due its ineffective strength adjustment. Each combination involves 250 games, requiring approximately 100 GPU hours on an NVIDIA RTX A5000. Table 5 summarizes the results, with the win rates in each cell representing the performance of the y-axis player against the x-axis player. Moreover, we compute the Elo rating of each model using this table. We initialize the rating at 1500 for each model and iteratively update the ratings match the expected win rates with the observed pairwise outcomes. The rightmost column of Table F the resulting Elo ratings. In summary, the Elo ratings confirm that higher-ranked models consistently achieve higher ratings, demonstrating the robustness of our method across different baselines.
Table 5: The round-robin tournament among ten MCTS programs: SA SA-MCTS,, SA-MCTS4, SA-MCTSe, SA-MCTS SA SA-MCTS SE -MCTS,, SE -MCTS4, SE -c SEx -MCTSs. -MCTS For simplicity, abbreviations SA-T2. SA-r4, etc. are used.
7A. 9
55.
55. 71
27mm
61 1
25
6C Px
Avg. Wie Hak
E5
M0-F 3.99
00.7 3.00
1990
14550
6459
01:30 3,00
18.76
3.0
200%
12.59 21 1.50
1400 92
30,00
56.00 4.31
43.4%
2.231%
18.70
3.0%
35.49
2.2
34
3/37
1.30%
5.2 8.3.5%
14:00 4414
13.9 4310
2.37
53219
14.9% 4.44
2.119
2129
5.32
2109 1.70
22
112%
1125
MAR
2.399
1.37%
2-345
1455 1135%
35.2
2.277
15.2
1-448
2.37%
11.0
1499
224 7
Jum 8.88%
2.999
7544
1219 533.9%
1.77%
2119 2215
Sums
1.3%%
1395
5.521
2.45PS
13/95
4.5.5%
1.53%
93
22g-75 2t
5.4
200 4.75
J835
6-6218
13N6
16.4% 14436
L-775
3645
213%
11.75
18


Reviews:
[{"review_id": "7NEz198v43", "reviewer": "Reviewer_eKyZ", "summary": "The authors introduce a strength estimation method that predicts rough player strength in go with only a few games. They use this as a method for running MCTS showing that it can be leveraged to approximate human-like play in go (and chess).", "questions": "+ Why did the authors not compare their results to any of the models of human-like play in chess or go?\n+ I'm unclear on the reasoning for the claim on line 534 \"However, this issue could potentially be addressed by using all models trained by AlphaZero, which\", how would that work? Elo is a relative measure (expected winrate vs the community), how would AlphaZero's Elo be relevant for humans without human games?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "I think this is an interesting approach to modeling players in games and I like that the authors introduce a new metric for looking at player skill, but I am not fully convinced by the results.\n\n# originality\n\nThis is a new approach to modeling humans and is breath of fresh air compared to the maximizing win rate approaches most other models use. I also like that this can be used for insight into players (as the authors note) and think this is a good new area of research.\n\n# quality\n\nThe writing is OK, but I think the presentation needs another pass. The plots are difficult to read, I'm not clear how the confidence intervals are calculated, and the colours make telling lines apart difficult (figure 4 also has colour issues), consider some tables, log axis, larger lines/dots, smoothed lines, and fewer data points.\n\n# clarity\n\nI also found the paper not easy to follow the model labels aren't intuitive and are not defined in one place so it took some time to figure out what is meant by SE±1 vs SE_\\inf±1 for example. I'm also not fully clear how the training and testing were done, which the lack of code exacerbates. \n\n # significance\n\nI like the idea and this could be a significant new direction in the area, but the numerous issues with the paper limit it.", "weaknesses": "What is the training/testing split? I feel like I missed the section on how the dataset is constructed as I could only find a short discussion in section 4.1. I'm very concerned that there is some data leakage in how the experiments were run, specifically that the authors did not partition by players between training and testing. This would mean that the models can simply learn the preferred openings of each player and use that to predict skill. The numbers they get are about the same for a player identifier in chess [1].\n\nThe lack of baseline or consideration of alternative skill estimators I also find concerning as it makes evaluating the number much more difficult. Can the authors compare their results to the raw Elo estimates given by the 100/15 games (using the opponents known Elo) or better yet compare to some simple classifier.\n\nI am also concerned by lack of code release as that is an important part of the final paper.\n\n\n[1] McIlroy-Young, Reid, et al. \"Detecting individual decision-making style: Exploring behavioral stylometry in chess.\" Advances in Neural Information Processing Systems 34 (2021): 24482-24497.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IVmdzMdcCY", "reviewer": "Reviewer_jMjA", "summary": "This paper derives a strength estimator network from human game-plays labeled with rating data and then use the SE network to inform MCTS such that the policy plays at a specific rank in a human-like fashion. The strength estimator is learned with a loss function derived from a Bradley-Terry model.", "questions": "1. L53: \"..., with higher scores indicating stronger actions\": could you clarify what stronger implies here? Is it actions that lead to expected win-rates? If so, what is the expectation over? \n2. L58: \"... correspond to a given targeted strength score\": am I right in thinking that the quality of the strength adjustment of SE-MCTS in state s relies heavily on the quality of SE(s, a)? If so, could it be the case that players at a certain rank would rarely reach state s in the first place and lead to inaccurate prediction by SE-MCTS? \n3. L173: \"... aggregating using the geometric mean\": it would help readers if the use of the geometric mean can be better motivated. Is it because of how it works out in Eq (3) for comparing a rank to multiple other ranks? It could be helpful to state this explicitly. \n4. L238: \"we sequentially perform the softmax loss...\", typo? minimise the softmax loss? \n5. an intuitive baseline, given that the authors have access to ranked game play, would be to optimise a rank-conditioned policy network $\\pi(\\cdot | s, r)$ with $r$ the set of player ranks. If I understood correctly, methods such as SA-MCTS does not require ranked game-play data whereas the proposed methods do.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "I find the application of a SE network applied to adjusting the strength of game playing agents an interesting application with practical utility. The loss function derived from the Bradley-Terry model for learning the SE network is novel and with generalisation to more than two player settings.", "weaknesses": "I have several concerns with the proposed approach in this paper. First, the Bradley-Terry model represents each candidates with a scalar score (e.g. Elo score), which has many well-documented limitations [1-4]. A salient limitation is its inability to capture intransitivity, which would become a limitation factor for the SE network as it assumes that game plays by a player at rank $r' > r$ would win against game play by a player at rank $r$ when the lower ranked play may have played an effective exploiter strategy. Perhaps the scope of the method should be limited to perfect-information games? Second, the authors have not discussed the issue of reach-probability of states in defining the loss function to the SE network. For a state-action pair (s, a), the output of SE(s, a) is most heavily influenced by the player rank that most often visit state $s$, yet this does not seem to be accounted for in the method presented. While the issue of state coverage came up, it's not clear that adding randomised exploration policies would offset the influence of different state reach probability. Finally, the notion of \"strength\" features heavily in writing, however, I find it difficult to pin down exactly what \"strength\" means as it should depend on both the player and its opponent's strategies. The distribution over strategies seem implicit and dependent on the empirical game play data. \n\n[1] https://arxiv.org/abs/2206.12301\n\n[2] https://arxiv.org/abs/1806.02643\n\n[3] https://arxiv.org/abs/1903.01373\n\n[4] https://arxiv.org/abs/2306.05366", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nVKV5R45nx", "reviewer": "Reviewer_KBHz", "summary": "This paper introduces a strength estimator to estimate the strength of GO and chess players. Additionally, a modification of the MCTS algorithm based on the strength estimator is introduced, to make it possible to adjust the playing strength of the MCTS agent to certain levels. The data for training the strength estimator is collected from FoxWeiqi, the largest online GO platform, and models are trained to estimate the gameplay levels from 3-5kyu to 9 dan, which covers most of the meaningful strength levels of human players. The experiments show the strength estimator significantly outperforms traditional supervised learning approaches, and the gameplay strength of $\\text{SE}_\\infty$-MCTS agent appears to be correlated to the target strength.", "questions": "1.\tIs it possible to train and test SE-MCTS on Chess like on GO? I believe no matter whether the results are good or bad, adding such results will always strengthen the paper.\n2.\tWhat is the exact value of $z$ for each rank?\n3.\tWhich strength estimator is used in Figure 3?\n4.\tWhat does the strength score of vanilla MCTS and SA-MCTS look like? I am interested in the strength score curves (Figure 3) of vanilla MCTS and SA-MCTS (with varied strength index/rank settings).\n5.\tWhich paper is the reference for SA-MCTS? I understand it is referred to in subsection 2.3, but there are multiple citations in 2.3, and am not sure which one proposed SA-MCTS.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "This paper presents the proposed methods clearly with sufficient details. The proposed methods are described with proper motivations. According to experiment results, the proposed strength estimator shows significant improvement compared to the traditional supervised learning algorithms. The results also show the proposed method can adjust the playing strength to some extent.", "weaknesses": "To my understanding, the strength estimator method is relevant to learning to rank [1], [2]. I believe there is some novelty, but some related background could be discussed to make the novelty more clear.\n\n[1] Burges, Chris, et al. \"Learning to rank using gradient descent.\" Proceedings of the 22nd international conference on Machine learning. 2005.\n\n[2] Li, Hang. \"A short introduction to learning to rank.\" IEICE TRANSACTIONS on Information and Systems 94.10 (2011): 1854-1862.\n\nI believe there are some ways to improve the clarity and soundness of experiment parts. For Figure 4, I would like to recommend authors expand each column to a 2D map (except for MCTS). That means comparing SA-MCTS, SE-MCTS, and $\\text{SE}_\\infty$-MCTS for all combinations of ranks, instead of comparing all ranks with only $r_5$. I understand there is a page limit but such figures can be included in the appendix.\n\nSome more concerns about the experiment are listed in my Questions and can be addressed by updating the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4BlpgjJZ60", "reviewer": "Reviewer_651t", "summary": "This paper introduces a dedicated strength estimator and an SE-based Monte Carlo tree search,  which predicts strengths from games\nand offers different playing strengths with human styles. A large amount of expert data with different playing level is used for training, and the learned strength is empoyed to adjust the prior policy in MCTS. Starting from the Bradley-Terry model, a new loss function is proposed. The experimental results demonstrate that the model can effectively predict the level of actions and guide MCTS in making decisions at various levels in both Go and Chess.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This paper is well-written with a clear and easily understandable structure.\n\n2. This paper is highly motivated and the novelty is good.\n\n3. The experiments are comprehensive, with cross-validation conducted using two applications.", "weaknesses": "1. SE-MCTS adjusts the prior probabilities by the strength level $\\hat{\\delta}$  to influence the decision-making level of MCTS, and the number of simulations has not been reduced. It seems that SA-MCTS can give actions with $k$ different strength level by setting different $z$ for just one search process, but SE-MCTS need $k$ searches for $k$ different strength level actions. Besides aligning more closely with the desired strength rank, what are the more advantages of modifying the search process compared with performing on the final action decision?\n\n2. SE-MCTS achieves different levels of actions by modifying prior probabilities. However, as mentioned in the paper, with an increase in the number of simulations, the decision-making gradually shifts towards relying on Q-values. Does this imply that with a sufficient number of simulations, SE-MCTS might become ineffective, generatinig the same action with different $\\hat{\\delta}$? Can you provide a ablation study of SE-MCTS under different numbers of MCTS simulations?\n\n3.  In Figure 4, why the win rate of SE-MCTS so low?\n\n4. Because different players have different strategies, Player A's defeat Player B does not necessarily indicate that A is stronger than B. It may simply mean that A's policy happened to exploit a weakness in B's policy. You can have pairwise matches between SE-MCTS, SA-MCTS, and MCTS with different levels, then rank them using Elo rating system, which would be more convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)