Match Prediction Using Learned History Embeddings

Contemporary ranking systems that are based on win/loss history , such as Elo ( Elo , 1978 ) or TrueSkill ( Herbrich et al. , 2007 ) represent each player using a scalar estimate of ability ( plus variance , in the latter case ) . While easily interpretable , this approach has a number of shortcomings : ( i ) latent attributes of a player can not be represented , and ( ii ) it can not seamlessly incorporate contextual information ( e.g . home-field advantage ) . In this work , we propose a simple Transformer-based approach for pairwise competitions that recursively operates on game histories , in addition to modeling players directly . By characterizing each player by its history , rather than an underlying scalar skill estimate , it is able to make accurate predictions even for new players with limited history . When restricted to the same information as several state of the art methods , including Elo and Glicko ( Glickman , 1995 ) , our approach significantly outperforms them on predicting the outcome of real-world Chess , Baseball and Ice Hockey games . 1 INTRODUCTION . Systems for ranking teams or individuals in pairwise competitions have important uses in a wide variety of applications that include : ( online e ) -sports matchups , dating apps , rating sports teams , rating chess players and training populations of agents ( Balduzzi et al. , 2018 ) . The most widely used techniques use as input the outcomes of previous match-ups , this being universally available and requiring no domain insights or meta-data . For easy interpretability , most of them use a single scalar , corresponding to a latent skill attribute , to characterize a player or team . This single value is then used to predict the outcome of match-ups or games . However , this implies that the multitude of attributes that makeup a player can be represented by a single number , which is a questionable assumption . This paper introduces a simple Transformer-based approach for predicting the outcome of competitions from historical data alone . Each player/team is represented by an embedding vector , computed entirely from the history of previous competitions . While a scalar skill rating for each player can not directly be computed , the model can still be used for the same ranking or match-up tasks as existing methods . While the model is naturally able to incorporate contextual information alongside input history , we focus on using historical data alone , restricting our contextual information to home-field advantage . This is the most general scenario and enables direct comparison to many widely used techniques , avoiding the confounding effect of feature engineering . A key feature of our model is that it is not necessary to make use of player identity in order to achieve strong predictions : each player is represented entirely by their previous games and outcomes . In a recursive fashion , the opponents of these games are also represented by their history . The purely relational summary of player history allows it to quickly produce accurate predictions for new players for whom little history exists , or for teams at the start of a new season . Player identity can be included in the model , which enables the modeling of non-transitive relations , as we show in a simple example of rock-paper-scissors . 2 RELATED WORK . Skill-based methods : Skill ranking has a long history in professional sports and competitive games ( Glickman , 1995 ; Dehpanah et al. , 2020 ) . In chess , the most widely-known skill rating system is Elo ( Elo , 1978 ) . This assumes that a player A has a skill rating θA and that its performance pA is distributed according to pA ∼ N ( pA ; θA , σ2 ) where σ is some fixed variance . Then the probability that A beats b is given by Pr ( pA > pB |θA , θB ) = Φ ( θA − θB√ 2σ ) where Φ is the cdf of a unit Gaussian distribution with mean 0 . Replacing Φ with a logistic distribution results in a Bradley-Terry model ( Bradley and Terry , 1952 ) . Both Elo and Bradley-Terry can be made Bayesian by assuming some distribution over the skills of each player , i.e . each player has a variance σA in addition to θA so that each player ’ s performance is modeled as a Gaussian distribution with mean θA and variance σA , rather than a fixed σ . These modifications are made in the Glicko system ( Glickman , 1995 ) and ( Weng and Lin , 2011 ) to Elo and Bradley-Terry , respectively . Both models can be extended to allow for teams of players and more than two person competitions , which is done in TrueSkill ( Herbrich et al. , 2007 ) ( Herbrich et al. , 2008 ) and Bayesian-Bradley Terry ( Weng and Lin , 2011 ) . Our approach contrasts with the methods above in that each player is represented by an embedding vector rather than a single number ( plus variance ) . In terms of how player history is utilized , Trueskill ( Herbrich et al. , 2007 ) , is closest to ours but still differs significant ways : ( a ) it uses player-specific representations , whereas ours in agnostic to the player ID ; ( b ) it combines information using a graphical model , rather than a deep learning model and ( c ) it is a flat model , whereas our builds the representation by recursively expanding opponent histories . Embedding approaches : ( Delalleau et al. , 2012 ) used a neural network architecture to learn latent embeddings for players , aiming to predict player enjoyment in online matchups . ( Zhang et al. , 2010 ) learn a factor embedding for each player and take the inner product with a context vector to get a skill rating . ( Balakrishnan and Chopra , 2012 ) use similar methods for user preferences ( `` do you prefer A or B ? '' ) . Our approach differs in that it focuses on the more general no-context scenario . In this setting the data only consists of outcomes ( win/loss or points differential ) and time features , better revealing differences in modeling performance . Non-transitivity : Recent work by ( Balduzzi et al. , 2018 ) proposed a determinant-based framework for modeling non-transitive interactions between RL agents on a leaderboard . Their approach draws on ideas from combinatorial Hodge theory , as laid out in ( Jiang et al. , 2011 ) . In the same spirit as ( Chen and Joachims , 2016a ; b ) who use a 1-layer MLP , we instead rely on the inherent non-linearity of deep neural networks to capture transitive and non-transitive relations ( if player ID is provided ) . ( Chen and Joachims , 2016c ) learn two embeddings for each player , a `` blade '' vector and a `` chest '' vector and use the difference between the two to model the probability of winning that captures intransitivity . Multi-player : Another group of approaches explicitly address multiplayer competitions ( Herbrich et al. , 2007 ; Minka et al. , 2018 ) , e.g . multiplayer online battle area games ( MOBA ) . Many approaches are customized to particular game environments and incorporate domain specific information , e.g . Heros of Newerth ( Suznjevic et al. , 2015 ) and Ghost Recon Online ( Delalleau et al. , 2012 ) . Our method can not naively handle multi-player settings , but there are plausible approaches to incorporating this , as we discuss later . Context : Many recent approaches have shown ways to incorporate contextual information alongside historical outcomes . ( Chen and Joachims , 2016a ) uses context , such as betting odds , to improve predictions . TrueSkill2 ( Minka et al. , 2018 ) refines TrueSkill ( Herbrich et al. , 2007 ) to incorporate context . While not the focus of our paper , since our approach is neural network-based it is able to easily learn useful features from context , if needed . 3 LEARNED HISTORY EMBEDDINGS . 3.1 NETWORK GRAPH . The most intuitive explanation of our approach is from a network graph perspective . Consider a graph where each node n = ( A , B , g ) represents a game between players A and B along with a node attribute , g. We consider the simple case where g = { r , t } where r is a scalar indicating the point differential between A and B and t is the time of the game . There is a directed edge between two nodes n1 and n2 defined by n1 = ( A1 , B1 , g1 ) and n2 = ( A2 , B2 , g2 ) if { A1 , B1 } ∩ { A2 , B2 } 6= ∅1 and t1 > t2 . Thus edges in this graph connect nodes that share one player , and neighbors of a node n are defined to go backward in time . The architecture for our approach relies on Transformer ( Vaswani et al. , 2017 ) layers that use attention to learn embeddings of players , operating on an increasingly large receptive field in this network graph . The `` level 1 '' version of our method will operate on neighbors that are path length one away from the proposed match-up ; the `` level 2 '' version considers all nodes of path length less than two from proposed match-up , and so on . Note that our model acts only on the node features g = ( r , t ) so it has no knowledge of explicit player ID in the match-up , thus it is acting only on the histories of the players . This allows for greater flexibility and forces the model to learn richer representations compared to prior work . 3.2 LEVEL 1 HISTORY . We wish to predict the outcome of a proposed match occurring between players A and B , which will result in either a win or loss ( draws are discussed later ) . Our model makes this prediction using the T previous match-ups of each player ZA : = { gA1 , ... , gAT } and ZB : = { gB1 , ... , gBT } respectively . These histories are considered level 1 since they correspond to the nodes in the network graph that are depth one away from each player . When computing the probability that A beats B we must respect the underlying symmetry that p ( A beats B ) = 1 − p ( B beats A ) . To do this we stack the inputs as Z1 = ( [ ZA ZB ] ) and Z2 = ( [ ZB ZA ] ) and run them through a multi-layer Transformer ζ such that o ( A , B ) : = ζ ( Z1 ) − ζ ( Z2 ) . The resulting logit o ( A , B ) has the desired property o ( A , B ) + o ( B , A ) = 0 and is trained with cross entropy loss . This level 1 model is only able to learn fairly simple weighted averages over the history , or generic comparisons of the form `` team A is doing better than team B in its past x games '' . 3.3 LEVEL 2 HISTORY . The model becomes more powerful once we consider all nodes in the graph that are at most path length two away from the current node . These nodes correspond to `` histories of history '' , where all previous games in the proposed match-up history include the history of each opponent involved . Consider two players A and B with T most recent match histories ZA = { gA1 , ... , gAT } and ZB = { gB1 , ... , gBT } respectively . We now also use the ith opponent ’ s own history , looking at their T previous match-ups . These are represented by features hij , from the match-up with the jth opponent in player i ’ s history , i being an opponent in A ’ s history . We learn an embedding for each opponent i in player A ’ s history by passing the features hij , j ∈ 1 .. T through a multi-layer Transformer and an MLP . This step summarizes the history of nodes at depth 2 into an embedding vector ĥi . The level 1 history of playerA is sumarized using the same Transformer + MLP to produce ĝ . The level 1 and 2 history embeddings are combined to produce a summary embedding ZA for player A . The equivalent summary for player B is computed in the same fashion using the same set of attention + MLP layers . Figure 1 shows this two level hierarchy . Formally , for a given player , we have the level 1 history X ∈ RT×k , each row of which corresponds to gi , k being the number of features per match-up ( k = 2 for the case g = r , t ) . We also have level 2 history Y ∈ RT×T×k , which is a tensor that holds hij . Both X and Y are processed with 1i.e . A1 = A2 or A1 = B2 or B1 = A2 or B1 = B2 . the same multi-layer Transformer φ ( . ) : RT×k → RT×d , d being the output embedding dimension ( a hyper-parameter ) . The result is then passed to an MLP which ψ ( . ) : RT×d → Re summarizes the embeddings . Thus , the level 1 and 2 histories are summarized into e dimensional embeddings ĝ = ψ ( φ ( X ) ) and ĥi = ψ ( φ ( Y ) ) , respectively . These are are combined into a matrix Z ( of size 2e× T ) : Z = [ ĝ . . . ĝ ĥ1 . . . ĥT ] Here the ith column corresponds to an embedding representing the ith match-up in a player ’ s history . The first part of the embedding , ĝ , captures the player ’ s own history ( level 1 ) , which is shared for each game in the history , and the second part of the embedding , ĥi , represents the history of opponent in the ith game . Computing Z for each player A and B yields ZA and ZB , which we combine using a multi-layer Transformer ζ ( . ) : R2e×2T → R to produce a scalar output . Optionally , contextual information c about the encounter between A and B can be added at this stage by appending it to ZA and ZB . Using the same procedure as in Section 3.2 to enforce the underlying symmetry , output logit o is defined to be : o = ζ ZAZB c − ζ ZBZA c ( 1 )

Summary: This paper aims at predicting the winning probability of a pairwise comparison. It adopts a deep neural network to combine historical comparison results and generate embeddings of two competitive players to make the final prediction. For each player, this paper considers two-levels of historical information preserved in a graph network. Experiments show that compared with traditional Glicko2 algorithms, the deep learning method has higher predictive accuracy.