Match Prediction Using Learned History Embeddings
Authors: Maxwell Goldstein
Leon Bottou
Rob Fergus
Venue: ICLR 2022 Submitted (2022)
Keywords: skill ranking
skill rating
skill

Abstract: Contemporary ranking systems that are based on win/loss history, such as Elo  or TrueSkill  represent each player using a scalar estimate of ability (plus variance, in the latter case). While easily interpretable, this approach has a number of shortcomings: (i) latent attributes of a player cannot be represented, and (ii) it cannot seamlessly incorporate contextual information (e.g. home-field advantage). In this work, we propose a simple Transformer-based approach for pairwise competitions that recursively operates on game histories, rather than modeling players directly. By characterizing each player entirely by its history, rather than an underlying scalar skill estimate, it is able to make accurate predictions even for new players with limited history. Additionally, it is able to model both transitive and non-transitive relations and can leverage contextual information. When restricted to the same information as Elo and Glicko, our approach significantly outperforms them on predicting the outcome of real-world Chess, Baseball and Ice Hockey games. %Further gains can be achieved when game meta-data is added.


Under review as a conference paper at ICLR 2022
MATCH PREDICTION USING LEARNED HISTORY EM- BEDDINGS
Anonymous authors Paper under double-bling review
ABSTRACT
Contemporary ranking systems that are based on win/loss history, such as Elo (Elo 1978) TrueSkill (Herbrich et al., 2007) represent each player using a scalar estimate of ability (plus variance, the latter case). While easily interpretable, this approach has number of shortcomings: (i) latent attributes of a player cannot be represented, and (ii) cannot seamlessly incorporate contextual information (e.g. home-field advantage). In this work, we propose a simple Transformer- based approach for pairwise competitions that recursively operates on game histories, in addition to modeling players directly. By characterizing each player by its history, rather than an underlying scalar skill estimate, it is able to make accurate predictions even for new players with limited history. When restricted the same information as several state of the art methods, including Elo and Glicko (Glickman 1995), our approach significantly outperforms them on predicting the outcome wealdworld Chess, Baseball and Ice Hockey games.
INTRODUCTION
Systems for ranking teams or individuals in pairwise competitions have important uses in wide variety of applications that include: (online e)-sports matchups, dating apps, rating sports teams, rating chess players and training populations of agents (Balduzzi et al. 2018).
The most widely used techniques use as input the outcomes previous match-ups, this being universally available and requiring no domain insights or meta-data. For easy interpretability, most of them use a single scalar, corresponding to a latent skill attribute, to characterize a player or team. This single value is then used predict the outcome of match-ups games. However, this implies that the multitude of attributes that makeup a player can be represented by single number, which is questionable assumption.
This paper introduces a simple Transformer- based approach for predicting the outcome of competi- tions from historical data alone. Each player/team is represented by an embedding vector, computed entirely from the history of previous competitions. While scalar skill rating for each player cannot directly computed, the model can still be used for the same ranking match-up tasks as existing methods. While the model is naturally able to incorporate contextual information alongside input history, we focus on using historical data alone, restricting our contextual information nome-fieed advantage. This is the most general scenario and enables direct comparison to many widely used techniques, avoiding the confounding effect of feature engineering.
A key feature of our model is that it is not necessary to make use of player identity in order to achieve strong predictions: each player is represented entirely by their previous games and outcomes. In recursive fashion, the opponents of these games are also represented by their history. The purely relational summary of player history allows it to quickly produce accurate predictions for new players for whom little history exists, for teams at the start of a new season. Player identity can be included in the model, which enables the modeling of non-transitive relations, as we show a simple example of rock- pack-paper-scissors.

---

Under review as a conference paper at ICLR 2022
2 RELATED WORK
Skill ranking has a long history in professional sports and competitive games (Glickman 1995 Dehpanah et al., 2020). In chess, the most widely-known skill rating system is Elo (Elo 1978). This assumes that a player A has a skill rating e a and that its performance PA is distributed according to PA ~ (p o where some fixed variance. Then the probability that A beats is given by
Skill-based methods:
(0xx 0B v 20
Pr(pA > pB|0A,08)
where & is the edf of a unit Gaussian distribution with mean 0.
Replacing with a logistic distribution results a Bradley- -erry model (Bradley and Terry 1952). Both Elo and Bradley-Terry can be made Bayesian by assuming some distribution over the skills of each player, i.e. each player has a variance a A in addition to A so that each player's 's performance is modeled as Gaussian distribution with mean 0 and variance d rather than fixed a. These modifications are made in the Glicko system (Glickman 1995) and (Weng and Lin 201T) to Elo and Bradley-Terry respectively. Both models than two person competitions, which is done in TrueSkill (Herbrich et al. 2007) (Herbrich et al.) 2008 and Beayesian-Bradie Terry (Weng and Lin 2011).
be extended to allow for teams of players and more
Our approach contrasts with the methods above in that each player is represented by an embedding vector rather than a single number (plus variance). In terms of how player history is utilized, Trueskill (Herbrich et al., (207). is closest to ours but still differs significant ways: (a) it uses player-specific representations, whereas ours in agnostic to the player ID; (b) combines information using graphical model, rather than deep learning model and (c) it is a flat model, whereas our builds the representation by recursively expanding opponent histories.
Embedding approaches: (Delalleau et al. 2012) used a neural network architecture to learn latent embeddings for players, aiming to predict player enjoyment in online matchups. (Zhang al.) (010) learn factor embedding for each player and take the inner product with a context vector to get skill rating. (Balakrishnan and Chopra 2012) use similar methods for user preferences ("do you prefer A or B?"". Our approach differs in that it focuses on the more general no-context scenario. In this setting the data only consists of outcomes (win/loss or points differential) and time features, better revealing differences in modeling performance.
Non-transitivity: Recent work by (Balduzzi et al. 2018) proposed determinant-based framework for modeling transiinniiive interactions between RL agents on a leaderboard. Their approach draws on ideas from combinatorial Hodge theory, as laid out in (Jiang et al. (011). In the same spirit as instead rely the inherent nin-linyyyit
I-layer MLP, of deep neural networks to capture transitive and non non-transitive relations (if player ID is provided). (Chen and Joachims (016cc learn two embeddings for each player, a "blade" vector and "chest" vector and use the difference between the two to model the probability of winning that captures intransitivity.
(Chen and Joachims 2016a b) who
Multi- plltieplayer: Another group of approaches explicitly address multiplayer competitions (Herbrich ltt al. 2007] MMina et al. 2018)) e.g. multiplayer online battle area games (MOBA). Many approaches are customized to particular game environments and incorporate domain specific information, e.g. Heros of Newerth (Suznjevic et al.] (205)) and Ghost Recon Online (Delalleau et al., (0022). Our method cannot naively handle multi-player settings, but there plausible approaches to incorporat- ing this, as we discuss later.
Context: Many recent approaches have shown ways to incorporate contextual information alongside historical outcomes. (Chen and Joachims, 2016a) uses context, such as betting odds, to improve predictions. Tkil52 (Minka et al. (2018) refines TrueSkill (Herbrich et al.) 2007) incorporate context. While not the focus of our paper, since our approach is neural network beased it is able to easily learn useful features from context, if needed.
2

---

Under review as a conference paper at ICLR 2022
3
LEARNED HISTORY EMBEDDINGS
3.1 NETWORK GRAPH
The most intuitive explanation of our approach is from a network graph perspective. Consider a graph where each node n (A, B,g) represents game between players A and B along with node attribute, 9. We consider the simple case where differential between A and B and t the time of the game. There is a directed edge between two nodes n1 and ny defined by n (A1. Bi, 91) and n2 = (A2, B2. 92) if {Ai, B₁ n (A2, B2} # [ and 1 > l2. Thus edges in this graph connect nodes that share one player, and neighbors of a node n. are defined to go backward in time.
{r. t where is scalar indicating the point
The architecture for our approach relies on Transformer (Vaswani et al. 2017) layers that use attention to learn embeddings of players, operating on an increasingly large receptive field in this network graph. The "level 1" version of our method will operate on neighbors that are path length one away from the proposed match-up; the "level 2" version considers all nodes of path length less than two from proposed match-up, and so on. Note that our model acts only on the node features g = (r.t) so it has no knowledge of explicit player IDD in the match-up. thus it is acting only the histories of the players. This allows for greater flexibility and forces the model to learn richer representations compared to prior work.
3.2 LEVEL 1 HISTORY
We wish to predict the outcome of a proposed match occurring between players A and B, which will result in either win or loss (draws are discussed later). Our model makes this prediction using the T previous match-ups of each player ZA := .... 9و :.....h.. These histories are considered level since they correspond the nodes in the network graph that are depth one away from each player. When computing the probability that A beats B we must respect the underlying symmetry that p(A beats B)
and ZB := .........g of respectively.
p( B beats A). To do this we stack the
Z n and Ze
( 2 A
inputs as Z1
and run them through multi-layer Transformer
C such that o(A, == <(21) ((Z2). The resulting logit o(A, B) has the desired property o(A, B) + A),
and trained with eross entropy loss.
This level I model is only able to learn fairly simple weighted averages over the history, or generic comparisons of the form "team is doing better than team B in its past x games".
3.3 LEVEL 2 HISTORY
The model becomes more powerful once we consider all nodes in the graph that are at most path length two away from the current node. These nodes correspond to "histories of history", where all previous games in the proposed match-up history include the history of each opponent involved.
Consider players A and B with T most recent match histories ZA {of .f and 7 tof...., .e.f9...ff) gr.,.gf} respectively. We now also use the ith opponent's own history, looking at their T previous match-ups. These are represented by features hij. from the match-up with the jth opponent in player i's history, being an opponent in A's history.
We learn an embedding for each opponent i in player A's history by passing the features ht,J E 1..7 through a multi-layer Transformer and an MLP. This step summarizes the history of nodes at depth into an embedding vector ht. The level 1 history of player A is sumarized using the same Transformer + MLP to produce ÿ. The level 1 and history embeddings are combined to produce summary embedding A for player A. The equivalent summary for player B is computed in the same fashion using the same set of attention MIP layers. Figure shows this two level hierarchy.
Formally, for a given player, we have the level 1 history X E Txx,, each row of which corresponds to gi. being the number of features per match-up (k 2 for the case g r, r,,t) We also have level 2 history Y E xx'xk, which is tensor that holds ht. Both X and Y are processed with
i.c. A Ag A Bg B Ag B B2.
3

---

Under review as a conference paper at ICLR 2022
Transtoreer MLP
((0(1)
Opponent history
6()
9, 92 9
9+
9
Output logit
Player history
11p 2²))
Transformer MLP
61 AP H
HT
Transformer + MLP )(0)))
z
Figure 1: Overview of our two level player history embedding model. See text for details.
the same multi-layer Transformer o(.) Tx Txx d being the output embedding dimension (a typer-parameter), The result is then passed to an MLP which v².) Rx'x + II summarizes the embeddings. Thus, the level I and 2 histories are summarized into dimensional embeddings
(b((()) and h v(((())) respectively. These combined into matrix Z (of size 2e x T):
Z 9
Here the ith column corresponds to an embedding representing the ith match-up in a player's history. The first part of the embedding, 9. captures the player's own history (level 1), which is shared for each game in the history, and the second part of the embedding, h represents the history of opponent in the ith game.
Computing Z for each player A and B yields Z and nB, which we combine using multi-layer Transformer <(.) R22x2 - IR produce a scalar output. Optionally, contextual information about the encounter between and B can be added this stage by appending it to Z and 7B Using the same procedure as in Section 3.2 to enforce the underlying symmetry, output logit is defined to be:
Z
8
0=
-
(1)
c
c
3.4 MODEL ARCHITECTURE DETAILS
In level 2 version the model, o(.) and ((.) each consist of a layer Transformer (see Figure 2 Each of the layers uses the standard architecture from (Vaswani al. 2017): 4-head attention LayerNorm - layer MI.P Lorer The output embedding dimensions are 32 and 8. Overall, the model is 6 Transformer layers deep (6(-) and ((-)), plus single MI.P layer (v)())..
The level 1 version of the model, which does not look at opponent histories, consists of computing ((((() and ( xx)) concatenating them and passing resulting vector through an MLP. & and ub are defined as above.
Layyer MPP
Layae NLP
T-Layer MUP
put
Cutput
Ads
Attention
Alberion
Alennton
Figure 2: Architecture of the 3-layer Transformer Vaswani et al. 2017) used for functions Φ(.) and <(.). See text for details.
4

---

Under review as a conference paper at ICLR 2022
3.5 PRACTICAL IMPLEMENTATION
It is only necessary to train the model single time on fixed data set. The cost of training is linear in the number of matchups (as with Glicko2), being O(T²xM) where M is the number of matchups in the data set. Additionally, prediction is fast, being ( which takes time the order of milliseconds. Further practical speedups are possible by caching each matchup so that future matchups making use of simply look up the associated vector, rather than recomputing afresh. Overall, our approach still highly performant, albeit slightly slower than Glicko2..
EXPERIMENTS
4.1
DATA SETS
Our main evaluation uses four large real-world datasets:
CHESS1: data set from the first Kaggle chess competition matches (Kaggle A).
CHESs2: A data set provided by Kaggle from their from their second competition (Kaggle B).
HOCKEY: National Hockey League (NHL) data from 1979-2020 (Hockey).
BASEBALL: Major League Baseball (MLB) data from 1969-1991 (Baseball).
The CHESS1 data set consists of 65053 matches and contains 6573 unique players. Each player in CHESSl plays average of 17.8 games in the data set. The CHESS2 data set consists of 366000 games and 7651 unique players. We use a 60000 game subset of this data which has 4529 unique players, each of whom appears average of 15.7 times. The HOCKEY data set is provided by Hockey Summary Project (Hockey) and spans the years 1979-2020 of regular season and playoff games, during which 41 separate teams play. There team appears an average of 2145.5 times. The BASEBALL data set is provided by Baseball Databank (Baseball) and spans the years 1969-199 1991 of regular season and playoff games. During this period 28 unique teams play an average of 3304 times.
a total of 43984 matches in this data, of which each
4.2 DATASET CONSTRUCTION
For CHESS1 we take the first 55000 games to be training, the next 2000 to validation, and the remaining 8053 used test set. For CHESS2 we take the first 50000 games to be training, the next 2000 to be validation, and the remaining 8000 to be the test set. For HOCKEY we take the first 38000 games be training, the next 2000 to be validation, and the remainder be the test set. For the BASEBALL we take the first 38000 games to be training, the next 2000 to be validation, and the remaining be the test set.
The evaluation metric we report in Figure 3 is the Pearson correlation between the prediction and the outcome (0=loss, win, 0.5 draw), which for two variables X, Y is defined
Px,Y = Cov(X,Y)
oxOY
This allows us to easily compare with the continuous variable EMA baseline (described below), which does naturally form probability. In chess, if there draw, take the target value y be 0.5 in the log-loss used for training:
p + (1 p) log(1 y).
effectively asking the probabilities to close to 0.5.
4.3 EXPERIMENT CONFIGURATION
The parameessse tuned using the validation and described in Table 1.
5

---

Under review as a conference paper at ICLR 2022
Environment
CHESS
CHESS
BASEBALL
HOCKEY
Learning rate
.0005
.0005
.00025
.00025
Optimizer
Adam
Adam
Adam
Adam
Number epochs
10
10
16
Batch size
120
120
40
40
40
40
40
100
Embedding size d
32
32
32
32
Embedding size e
8
EMA half-life
500
500
120
180
Table 1: The hyper-parameters for our model in each of the environments
2.90
0.25
0.20
2.15
Mrk77
Death
0.13
16400
A2T
Depii
with CoNtect
Nethed
(a) CHESS
(b) CHESS
2144
0.0
0112
0.30
Comeesttnn
0.0
2.34
246
Melo3
IBT
Click
Decth
Decth
Depto
D6A
Mecocc
BST
Olicts
Depch 3
Depth
Deptn
(c) BASEBALL
(d) HOCKEY
Figure 3: Match prediction on the evaluation set of four real-world datasets: CHESS1, CHESS2, BASE- BALL, and HOCKEY. Pearson correlation with the true outcome of each game is plotted. Our learned history embeddings approach is shown in blue. Depth I corresponds to the architecture described Section 3.2, and depth 2 corresponds the architecture described Section 3.3. Exponential Moving Average (EMA), Bayesian Bradley-Terry (Weng and Lin 2011), Blade- (Chen and Joachims 2016c). Melo2K (Balduzzi al., 2018). and Glicko (Glickman 1995) baselines are shown orange. Error bars show +l for five seeds with different model initializations. Our method significantly outperforms the baselines, with depth 2 models generally being superior to depth 1. On the right side of each subplot show depth model with context added (black/white for CHESS 1/CHESs2; EMA point differential for BASEBALL and HOCKEY), These features boost the model performance over the context-free free version.

---

Under review as a conference paper at ICLR 2022
4.4 BASELINES
We compare our approach to the following baselines: (i) Glicko2 (Glickman, 2013), (ii) Bayesian Bradley-Terry (Weng and Lin. 2011), (iii) an exponential moving average (EMA) of the win loss (in chess) and the point differential (in baseball and hockey), EMA is a statistic on a sequence of values (Vi...., Vr) that keeps track of a recency weighted mean. The EMA my at time step depends on (Vihge and is defined as
mi = + (1-a)V
(1
where is hyper-parameter. The hyper-parameters used for Glicko2 and BBT are listed in the appendix. We use the open-source R package "Sequential Pairwise Online Rating Tech- niques" (Kaledkowski 2020). The haper-parameters for each baseline are tuned on the validation set. We use the author's 's publicly available implementation of the Blade-Chest model (Chen and Joachims 2016ch, but find it does not perform well on the four data sets, achieving correlation of numbers 0.010, 0.077, 0.012, 0.023 on CHESSI, CHESS2, BASEBALL, and HIOCKEY respectively.
4.5 RESULTS
The results are shown in Figure 3 Each approach is evaluated using 5 different random seeds for model initialization and show lo error bars. In all four environments the depth 2 method performs the best out of the models which only look at win/loss and point differential data, significantly outperforming Glicko2 and BBT baselines.
When additional meta-data is added to the depth model, performance is boosted for all four environments. In CHESS /CHESS2, the context consists of telling the model which player is playing white or black. In BASEBALL and HOCKEY, this context consists of adding the EMA prediction as context for each player.
For hockey and baseball data, the exponential moving average of point differential is a strong baseline, and it outperforms both Glicko and BBT. The strength of EMA in the sports setting can likely be explained by the fact that at any point in time each team has played roughly the same number of games, so fast adaptation these baselines we still handily beat them with our approach, showing the good inductive bias of treating the history of games as graph.
recent results is less informative than in chess. Despite the strength of
Each dataset we examine is temporal in nature (i.e. the match ups follow a distinct ordering in time), there is only one possible choice for the training/test split. In other words, doing cross validation with random splits would lead to cheating since having knowledge of future games can leak information into what happened in the past. particular, chess simply knowing that a player plays in the future leaks information about its past performance, since there is a bias for winners to continue playing and losers to drop out. This requires evaluating our model using a moving block bootstrap method on the sequential data. In the baseball domain, over five folds, sliding the training window up a season at time (162 games), our method outperforms Glicko2 (Glickman) 2013) by an average of 0.021 with standard deviation of 0.006 and outperforms BBT (Bradley and Terry 1952) by an average of 0.029 with a standard deviation of 0.009.
4.6 FAST START EVALUATION
In Figure 4 we compare our model to BBT for differing numbers of previous games. We look the minimum number of games played by each player and look at the correlation between the prediction and the response for different buckets of this value. For both CHESS I and CHESS2, the model is strong relative to the BBT when it has to quickly adapt to new players who have only played a few games. This contrasts with traditional methods such BBT, Glicko, and EMA that require burn-in period in order to adjust to the player.
4.7 NON-TRANSITIVITYS
While the experiments above make no use of player ID, explicitly adding learning representation per player enables the modeling of non- non-transitive matchups. We demonstrate this by modeling the game of rock--aper-se scierrscissors using the above model enhanced with a player ID feature. Trained on a toy mock-paper-seisson data set, our model correctly predicts the outcome of the various matchups. By
7

---

Under review as a conference paper at ICLR 2022
Chess
Chess
Hiarryy Frrden:inggs
267
207
0.45
0.33
0.22
0.30
0.90
022
0.23
0.0
0.5
Samees Tayed
Cames
Figure 4: A breakdown of our performance as a function of history size, i.e. number of match-ups for each player in the evaluation set (min over each pairing) on the CHESSl (L) and CHESS2 (R) datasets. We see that our approach (blue) is able to make accurate predictions for new players even with very few games (< 10), where as the performance of Bayesian Bradley Terry (BBT, red) degrades such settings.
contrast, Glicko (Glickman 2013) and the scalar based methods better than random. Melo2K (Balduzzi et al. 2018) and BladeChest (Chen and Joachims 2016c) are capable of modeling this intransitive behavior as well.
In the four real-world data sets, we find that adding player ID as a feature does not significantly improve the performance of model, suggesting that the non-transitive effect small.
DISCUSSION
We have introduced novel and conceptually simple model for learning from pairwise comparison data. This framework and architecture are able to capture complex inter-dependencies between players' histories and learn robust representations that are useful in predicting the winner of matches. The experiments show significant gains against widely-used skill rating approaches that use game history information alone.
The model also has several qualities that make it practical for real-world applications: First, the fast start ability, shown in Figure is vital for many online ranking systems, where new players must be reliably matched to appropriate opponents, otherwise they will quickly lose interest in the game. Second, at inference time our approach is very fast since the model is lightweight, using modest number of feed-forward layers. Furthermore, is based on standard deep learning architectures for which many software and hardware optimizations exist. This makes feasible the model on environment with very large numbers of players.
Code and trained models available at hhttps://anonmmuuu wtwpee/.....ec.e./////ooo
cam CBn-CC5A// README. md.
5.1 LIMITATIONS
The model, as described. does not compute a scalar skill quantity for each player, making difficult to interpret. However, it is possible to estimate an approximate measure by randomly sampling other players and averaging the computed win probability against each. This would result in sealar the range 0 (lose to everyone) to 1 (win against everyone).
5.2 FUTURE WORK
The simplicity of the framework allows for enhancements in a number of directions.
Improved architectures A natural direction is recurse deeper into the player interaction graph by using level n versions of our model, which would grow the receptive field. Extend- ing our model to 3 levels is straightforward, as the same machinery be used compute
8

---

Under review as a conference paper at ICLR 2022
embeddings of histories level deeper. These then by combined by S(-), with each column of Z being composed of three embedding vectors, rather than the current two.
Multi-way competitions: While our approach does not immediately generalize multi-way competitions, there are plausible approaches to doing so. The central issue is that in player game, (2) pairwise ordering constraints must be respected. This could be performed by graph neural-net (which is invariant to input ordering) acting the players of the game, each of whom has an embedding produced by our current model.
Context: Our results in Figure are proof-of-concept that adding context can improve our models performance. However, the information used (white/black for chess; EMA for ice hockey and baseball) is very basic. If the goal is to produce the best absolute performance for given sport, then the priority is to source and utilize as much relevant meta-data as possible. For ice hockey and baseball, numerous statistics about each game are typically available, some of which would have useful predictive value. Our model is well suited for learning from such heterogeneous data, given the well-established versatility of Transformer models.

---

Under review as a conference paper at ICLR 2022
REFERENCES
S. Balakrishnan and S. Chopra. Two of kind or the ratings game? adaptive pairwise preferences and latent factor models. Front. Comput. Sci., 6:197-208, 2012.
David Balduzzi, Karl Tuyls, Julien Pérolat, and Thore Graepel. Re-evaluating evaluation. CoRR, abs/ 0oss/1006.0266 0264,, 2018. URL hRtpht:://axxiii wwwpexiiii.orraaaeooooe abs.//bb////00 http://wwww...rr///////bbbb
02643
R. Bradley and M. Terry. Rank analysis of incomplete block designs: The method of paired comparisons. Biometrika, 39:324-345 345, 1952.
Shuo Chen and Thorsten Joachims. Predicting matchups and preferences in context. KDD '16. Association for Computing Machinery, 2016a.
Shuo Chen and Thorsten Joachims. Modeling intransitivity in matchup and comparison data. WSDM 16, page 227-236. Association for Computing Machinery, 2016b.
Shuo Chen and Thorsten Joachims. Modeling intransitivity in matchup and comparison data. Pro- ceedings of the Ninth ACM International Conference on Web Search and Data Mining, 2016c.
Arman Dehpanah, Muheeb Faizan Ghori, Jonathan Gemmell, and Bamshad Mobasher. The evaluation of rating systems in online free-for-all games. CoRR, abs/2008.06787, 2020. URL https
//arxiv.or/ 200/ 06787
Olivier Delalleau, Emile Contal, Eric Thibodeau-Laufer, Raul Chandias Ferrari, Yoshua Bengio, and Frank Zhang. Beyond skill rating: Advanced matchmaking ghost recon online. 1EEE Trans. Comput. Intell. Al Games, 41):167-1177,001 2012. URL /ttp:p::///o..or/11...11
TCTATG. 2012. 2188833
Arpad E. Elo. The Rating of Chessplayers, Past and Present. Arco Pub., New York, 1978. ISBN 0668047216 9780668047210. URL http:/ www amazon. com/
M. E. Glickman. A comprehensive guide to chess ratings. Amer Chess Journal, 3:59-102, 1995.
M. E. Glickman. The glicko-2 system. http: hett:://ww.gge. http://www..ccoo..oee/////coo/oc 2013.
Ralf Herbrich, Tom Minka, and Thore Graepel. Trueskill(tm):: bayesian skill rating system. In Advances in Neural Information Processing Systems 20, pages 569-576. MIT Press, Jan- uary 2007. URLhttpa:: http///wwww..co http:/ppp//wwwwooooooooottttto.. www.://t....oeeeeeeeeeeeeeeeeeee
Srueskilltme a bayesian sxieeaiinnssnillllee http://st......eeaeeee//eaaaaaaa
aysttm/
Ralf Herbrich, Tom Minka, and Thore Graepel. Trueskill through time: Revisiting the history of
chess. In Advances in Neural Information Processing Systems 20, pages 931-938 MIT Press, Jan- uary 2008. https: http:///ete.e////ooooorooooooo.. com htt.://owoo....ccoeoeeeeeeeeeaar http:///eeeeeercae/...aaaooo....
trueskill through time www.sieiiiiiiiie the sie--hissee history of-
X. Jiang, L.-H. Lim, Y. Yao, and Y. Ye. Statistical ranking and combinatorial hodge theory. Math. Program., Ser. B, 127:203-244. 2011.
Dawid Kaledkowski. Sequential Pairwise Online Rating Techniques, 2020. URL https: //
github. wtt.:///oggooooo/ooooooooo
Tom Minka, Ryan Cleven, and Yordan Zaykov. Trueskill 2: An improved bayesian
skill rating system. Technical Report MSR-TR-201888, Microsoft, March 2018
URL
Erteskill-2
Mirko Suznjevic, Maja Matijasevic, and Jelena Konfic. Application context based algorithm for player skill evaluation in moba games. In 2015 International Workshop on Network and Systems Support for Games (NetGames), pages 1-6, 2015. doi: 10. N0
10

---

Under review as a conference paper at ICLR 2022
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention all you need.
Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017. URL http:: httttps://procceednggg
wttp:pppp.ccppppp/20011000 33f6223547ce311dd05ccccc33aaaaa Paper. pdf
Ruby C. Weng and Chih-Jen Lin. A bayesian approximation method for online ranking, JMLR, 12: 267-300, 2011.
L. Zhang, Wu, Z. Wang, and C. Wang. A factor-based model for context-sensitive skill rating systems. In 22nd IEEE International Conference on Tools with Artificial Intelligence, pages 249-255, 2010.


Reviews:
[{"review_id": "iRPKym623QC", "reviewer": "Reviewer_eXjB", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strengths\n\n1) Novel, simple approach\n\t- Sequence modeling has not been used in the past (to my knowledge) for skill rating problems to any success. This is a strong new result even if the components (transformers) are not particularly novel.\n\t- The approach is also very simple and easy for practitioners to adopt. The relatively efficient prediction computation makes this an attractive option for use in real-world systems.\n2) Reproducible results\n\t- The methods provide sufficient detail to understand the system and the code is provided.\n3) Strong initial results\n\t- (Assuming the results hold under statistical tests, as described below) The results are promising for a method with little additional tuning to the domain task. \n\t- A strong simple approach is very promising for further refinement and development to address different application domains or problem features. Handling intransitivity is also an important feature that captures a key advantage of other recent skill rating techniques.\n\t\n\n# Weaknesses\n\n1) Lack of statistical tests of differences from results\n\t- Results include standard deviations but no statistical tests for differences. This applies to the plots in Figure 3 and the moving block bootstrap results reported in the text.\n\t- Without statistical tests readers cannot assess whether results are significant nor the effect sizes of differences.\n\t- Action: Perform statistical tests for differences among the methods and report significance of differences and estimated effect sizes.\n2) Unusual performance metric\n\t- Pearson correlation is not a standard metric for evaluating predictive systems. While this may be helpful to compare with the EMA baseline the results seem less central than the prediction accuracy. Additionally, in many practical matchmaking systems it is important that predictions be calibrated.\n\t- The paper would be much stronger with results reported in terms of predictive accuracy, and ideally also including reports of the model calibration curves (ex: If the model predicts a 20% win probability, what fraction of those matches are wins vs losses? If calibrated, 20% of the predicted matches will be wins.)\n\t- Action: Report results on test set prediction accuracy for all models aside EMA. \n\t\t- I would recommend reporting these results as the main outcomes in the paper and moving the correlation results to an appendix for readers interested in the EMA baseline.\n\t- Action: Report calibration curves for the predictive models (all but EMA)\n3) Missing baseline comparisons\n\t- Fast start evaluations only compare to BBT. These results should include comparisons to the other pairwise comparison methods for fair comparison.\n\t- Action: Report results for Melo2k, Glicko2, and Blade-Chest on the fast start task.\n\t- Action: Add statistical testing for differences among the methods.\n\n\nNote: The results in Figure 3 have a few inconsistencies that make them hard to compare and read. I would recommend the following changes:\n1) Use a single range for the y-axis, rather than different ranges for each plot.\n2) Include the results from Blade-Chest in the plot. The caption notes these are included but the text simply describes the numbers while omitting them (perhaps to avoid distorting the scale).\n\nNote 2: No results were recorded for the toy domain comparison. Add some simple numeric results or include these in an appendix. The text alone is not sufficient to back the claim made.", "summary_of_the_paper": "The paper presents an approach to predicting match outcomes between two individuals (teams or players) by comparing embeddings of previous match outcomes of both individuals. Embeddings are generated using a transformer architecture and can be applied recursively to embed the history of opponents of previous matches. Logits are produced by stacking embeddings and training with cross entropy after pushing outputs through an MLP. Experiments compare the method to established pairwise rating algorithms and the results show strong performance on chess, hockey, and baseball datasets. A toy demonstration shows the model can capture intransitive relationships in rock-paper-scissors.", "summary_of_the_review": "The paper technique is very novel and has clear practical applications. As a practitioner in this domain I would happily use this technique. The key weakness that reduces my score is the lack of rigor in evaluating the results. This makes it difficult to tell how strong the results are.\n\nI would move my score up if the authors provide statistical evidence of improvements, particularly if using prediction accuracy, rather than correlation coefficients.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "EX-sz7rkvwa", "reviewer": "Reviewer_ciZW", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nDifferent from Glicko which only maintains a scalar representing the player’s skill, this paper utilizes player’s embedding for match prediction. This deep learning model can also enable the flexibility to combine context information into the prediction process.\n\t\t\t\t\t\t \t\t\t\nWeaknesses (questions):  \t \t\t\t\t\t\n1- The appendix is cited but not attached to the submission, which contains hyperparameter settings.\n\n2- Given the constructed logit o in Eq.(1), how do you generate the probability P(A beats B)? It is easy to guess that the Bradly-terry model is employed, but I would appreciate it if the authors could explicitly define it as there are temperature parameters to be specified. \n\n3- From the methodology of constructing the embeddings in Sec. 3, I do not see how the proposed method can be ensured to deal with intransitivity. \n\nFor example, if A= Rock, B= Scissors, what value the logit o will be in Eq.(1); is that zero?\n\nIf so, the predicted P(Rock beats Scissors ) will be 0.5 instead of 1. Could you explain the details of dealing with intransitivity?\n\n4- In Sec 3.2, the cross entropy loss is not given. Though it is known to many readers, I would appreciate it if the authors could expand the details instead of relying on the readers’ preliminary knowledge on this.\n\n5- While Glicko can be used to predict the match outcome of cold start players, can this method be generalized to cold start players too?\n\n6- A related question to Question 2. \nSec 4.7 this paper claims the capability of modeling non-transitive games, but no empirical evidence is provided. \n", "summary_of_the_paper": "This paper aims at predicting the winning probability of a pairwise comparison. It adopts a deep neural network to combine historical comparison results and generate embeddings of two competitive players to make the final prediction. For each player, this paper considers two-levels of historical information preserved in a graph network. Experiments show that compared with traditional Glicko2 algorithms, the deep learning method has higher predictive accuracy. \t\t\t\t", "summary_of_the_review": "This paper is easy to follow and has practical merits. However, it lacks the formality of an academic paper, with jumbled symbols, important elements that are not explicitly defined, sections and paragraphs that are too short, etc. I would recommend the authors to make careful revision and correction.\n\t\t\t\t\t\t\n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "9kQLMghqOrD", "reviewer": "Reviewer_TCF7", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Pros**\n\n- The main advantage of the proposed algorithm is the representation of each player with an embedding vector that is learned based on its playing history. Actually, no further information is used that reveals player identity.\n- Two different versions have been proposed. The first one learns only an embedding vector for each player according to her history. On the other side, the second one also learns a separate embedding for each opponent in player's history (level 2). Then the level 1 and level 2 embedding are combined to represent the player.\n- Experiments have been conducted on four real-world datasets\n\n**Cons and Comments**\n\n- Some parts of the paper are not well written, making it hard for the reader to understand the proposed algorithmic scheme. Specifically, Section 3 should be revised carefully. For instance, in Section 3 the connection between Network Graph and the proposed algorithm is not clear. Also, $g^A$ seems to refer to the final outcome of a game but $g=(r,t)$ on the previous section refers to the difference on the score ($r$) between the two players at time $t$. \n- The setting of the experiments is not clear. Actually, the way where the dataset is split into training/validation/testing sets is possible to reveal information about the agents. It would be more reasonable to split the dataset based on the players. Another way would be the training of the algorithm on the Chess 1 dataset and its testing on the Chess 2. \n- The evaluation metric that is used is only the Pearson correlation. Why is the log-loss not used for the evaluation of the performance of the proposed algorithm?  Minor comment: formula of the cross entropy at Section 4.2 is not correct, please fix it.\n- It seems that the performance of the proposed algorithm is decreased as the history size is increased. Authors should discuss in details why does it happen. \n- Melo2k is one of the baselines that have been used,  but it is not presented at Section 4.4.", "summary_of_the_paper": "This paper introduces a Transformer based algorithm for predicting the outcome of a match or a game. The main novelty of this work is the representation of each player with an embedding vector that is learned based on the player's history (previous played games and their outcomes). Then, based on the learned embedding of the two players that participate in the game, the proposed algorithm predicts the output of the game. Experiments have been conducted on four real-world datasets.", "summary_of_the_review": "This work presents some interesting ideas but it is not ready for publication in its current form. As aforementioned, the empirical analysis is not complete and the authors should revise the manuscript carefully. For all these reasons, I recommend rejecting the paper. \n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a new method that uses transformers to predict the result of pairwise competitions given each players’ history of past game plays. The reviewers thought this had notable potential benefits for practice. However the reviewers’ also had some significant concerns with the current work in terms of the evaluations used, which were primarily  correlation instead of prediction accuracy or calibration etc. There was also some concern about other aspects of the presentation. We hope that the reviewers’ responses are useful to the authors in revising their work for future submissions as this method has the potential to be very useful for many domains.
Decision: Reject