Improved Structural Discovery and Representation Learning of Multi-Agent Data

1 INTRODUCTION . The natural representation for many sources of unstructured data is intuitive to us as humans : for images , a 2D pixel representation ; for speech , a spectrogram or linear filter-bank features ; and for text , letters and characters . All of these possess fixed , rigid structure in space , time , or sequential ordering which are immediately amenable for further learning . For other unstructured data sources such as point clouds , semantic graphs , and multi-agent trajectories , such an initial ordered structure does not naturally exist . These data sources are set or graph-like in nature and therefore the natural representation is unordered , posing a significant challenge for many machine-learning techniques . A domain where this is particularly pronounced is in the fine-grained multi-agent player motions of team sport . Access to player tracking data changed how we understand and analyze sport ( Miller et al. , 2014 ; Franks et al. , 2015 ; Wei et al. , 2013 ; Cervone et al. , 2014 ; Power et al. , 2017 ; Sha et al. , 2016 ; 2018 ; Yue et al. , 2014 ) . More relevantly , sport has risen to an increasingly key space within the machine learning community as an application to expand our understanding of adversarial multiagent motion , interaction , and representation ( Lucey et al. , 2013 ; Le et al. , 2017 ; Felsen et al. , 2018 ; Zheng et al. , 2016 ; Zhan et al. , 2018 ; Yeh et al. , 2019 ; Kurach et al. , 2019 ) . In sport there exists strong , complex group-structure which is less prevalent in other multi-agent systems such as pedestrian tracking . Specifically , the formation of a team captures not only the global shape and structure the group , but also enables the ordering of each agent according to a “ role ” within the group structure . In this regard , sport possesses relational structure similar to that of faces and bodies , which can be represented as a graph of key-points . In those domains , representation based on a fixed key-point ordering has allowed for cutting edge work across numerous tasks with a variety of approaches and architectures ( Antonakos et al. , 2015 ; Cootes et al. , 2001 ; Akhter et al. , 2012 ; Joo et al. , 2015 ; Simon et al. , 2017 ) . Unlike for faces and bodies , the representation graph in sport is dynamic as players constantly move and switch positions . Thus dynamically discovering the appropriate representation of individual players according to their role in a formation affords us structural information while learning a useful representation for subsequent tasks . This challenge was addressed by the original role-based alignment of Lucey et al . ( 2013 ) and subsequently by Bialkowski et al . ( 2016 ) and Sha et al . ( 2017 ) . Role-based alignment allows us to take unstructured multi-agent data , and reformat it into a consistent vector format that enables subsequent machine learning ( Fig . 1 ) . Here we formulate the role-based alignment as consisting of the phases of formation discovery and role assignment . Formation discovery uses unaligned data to learn an optimal formation template ; during role assignment a bipartite mapping is applied between agents and roles in each frame to produce “ aligned data ” . A major limitation in past approaches was the speed of the template discovery process . In this work we propose an improved approach to the above alignment methods which provides faster and more optimal template discovery for role-based representation learning . Importantly , we seek to learn the same representation as Lucey et al . ( 2013 ) , Bialkowski et al . ( 2016 ) , and Sha et al . ( 2017 ) by maximizing the same objective function of Bialkowski et al . ( 2016 ) in a more effective manner . The reduced computational load enables on-the-fly discovery of the formation templates , new context-specific analysis , and rapid representation learning useful for modeling multi-agent spatiotemporal data . Our main contributions are : the formulation of this problem as a three-step approach ( formation discovery , role assignment , template-clustering ) , the use of soft-assignment in the formation discovery phase thereby eliminating the costly hard-assignment step of the Hungarian Algorithm ( Kuhn , 1955 ) , a resetting training procedure based on the formation eigenvalues to prevent spurious optima , quantification of the impact of initialization convergence and stability , a restriction of the training data to key-frames for faster training with minimal impact on the learned representation , and a multi-agent clustering framework which captures the covariances across agents during the template-clustering phase . 2 BACKGROUND . 2.1 REPRESENTING STRUCTURED MULTI-AGENT DATA . A collection of agents is by nature a set and therefore no defined ordering exists a priori . To impose an arbitrary ordering introduces significant entropy into the system through the possible permutations of agents in the imposed representation . To circumvent this , some approaches in representing multi-agent tracking data in sport have included sorting the players based on an “ anchor ” agent ( Mehrasa et al. , 2018 ) . This is limiting in that the optimal anchor is task-specific , making the representation less generalizable . An “ image-based ” representation ( Yue et al. , 2014 ; Zheng et al. , 2016 ; Miller et al. , 2014 ) eliminates the need for an ordering , however , this representation is lossy , sparse , and high-dimensional . The role-based alignment protocol for sport of Lucey et al . ( 2013 ) used a codebook of hand-crafted formation templates against which frame-level1 samples were aligned . This work was extended by Bialkowski et al . ( 2016 ) which learned the template directly from the data . Sha et al . ( 2017 ) further employed a hierarchical template learning framework , useful in both retrieval ( Sha et al. , 2018 ) and trajectory prediction ( Felsen et al. , 2018 ; Yeh et al. , 2019 ) . Le et al . ( 2017 ) similarly learned an agent-ordering directly from the data by learning separate role-assignment and motion-prediction policies in an iterative and alternating fashion . 2.2 PERMUTATION-EQUIVARIANT APPROACHES . Permutation-equivariant approaches seek to leverage network architectures which are insensitive to the ordering of the input data . Approaches using graph neural networks ( GNN ) ( Kipf & Welling , 2016 ; Gilmer et al. , 2017 ; Battaglia et al. , 2018 ; 2016 ; Hoshen , 2017 ) have become very popular and shown tremendous promise . These approaches are particularly valuable for tasks ( e.g . pedestrian tracking ) which lack the strong coherent group structure of sport and therefore can not leverage meth- 1Throughout we use the term “ frame ” to indicate a single moment in time in reference to the data being obtained via optical tracking from video . ods such as role-alignment . Within sport , Kipf et al . ( 2018 ) used a GNN to predict the trajectories of players while simultaneously learning the edge-weights of the graph . Yeh et al . ( 2019 ) demonstrated the advantages in using GNNs to forecast the future motion of players in sports , surpassing both the role ( Bialkowski et al. , 2016 ) and tree-based approaches ( Sha et al. , 2017 ) on most metrics . The success of these approaches , however , does not negate the value of role-based alignment . The learned formation structure provides valuable insight into high-level organization of the group . Furthermore , many traditional machine learning techniques and common deep architectures require an ordered-agent representation . This is again similar to what is seen in the modeling of faces and bodies : great success has been achieved using geometric deep learning ( Monti et al. , 2017 ; Kipf et al. , 2018 ) , but approaches based on a fixed representation remain popular and effective ( Taylor et al. , 2017 ; Kanazawa et al. , 2019 ; Arnab et al. , 2019 ; Walker et al. , 2017 ; Rayat Imtiaz Hossain & Little , 2018 ) . Interesting work has also been done to learn permutations for self-supervised feature learning or ranking tasks ( Adams & Zemel , 2011 ; Mena et al. , 2018 ; Cruz et al. , 2017 ) . Central to these approaches is the process of Sinkhorn normalization ( Sinkhorn & Knopp , 1967 ) , which allows for soft-assignment during the training process and therefore a flow of gradients . Exploring the application of Sinkhorn normalization to this task is beyond the scope of this current work , however , we provide additional context on this method in Section A.2 . 3 APPROACH . 3.1 PROBLEM FORMULATION . Mathematically , the goal of the role-alignment procedure is to find the transformation A : { U1 , U2 , . . . , UN } ×M 7→ [ R1 , R2 , . . . , RK ] which maps the unstructured setU ofN player trajectories to an ordered set ( i.e . vector ) of K role-trajectories R. 2 Each player trajectory is itself an ordered set of positions Un = [ xs , n ] Ss=1 for an agent n ∈ [ 1 , N ] and a frame s ∈ [ 1 , S ] . We recognizeM as the optimal permutation matrix which enables such an ordering . Thus our goal is to find the most probable set F∗ of 2D probability density functions where F∗ = arg max F P ( F |R ) ( 1 ) 2Generically N need not equal K as a player may be sent off during a game , but for simplicity it is safe to assume N = K in this work . P ( x ) = N∑ n=1 P ( x|n ) P ( n ) = 1 N N∑ n=1 Pn ( x ) . ( 2 ) Bialkowski et al . ( 2016 ) transforms this equation into one of entropy minimization where the goal is to minimize the amount of overlap ( i.e . the KL-Divergence ) between each role . The final optimization equation in terms of the total entropy H then becomes F∗ = arg min F N∑ n=1 H ( x|n ) . ( 3 ) See A.1 for additional details . The authors then use expectation maximization ( EM ) to approximate this solution and note similarity to k-means clustering . However , as they represent that data non-parametrically in terms of per-role heat maps , hard assignment must be applied at each iteration so the distributions may be updated . Instead , we note that equation 2 describes the occupancy of space by any agent in any point in time as a mixture of conditional distributions across each of the N -roles . This is further equivalent to the sum over n-generating distributions . Thus if we model these generating distributions as ddimensional Gaussian distributions , this reduces the template-discovery process to that of a Gaussian Mixture Model . 3.2 TOY PROBLEM FORMULATION . Understanding the notion of independence under the different formulations of this problem is key . This may be better understood by considering a toy problem : imagine we have three independent 1D Gaussian distributions we wish to sample S times from each . It is known that we sample from each distribution in rounds , effectively generating the samples in “ triplets ” , although the order within the triplets is random . We then seek to reassign the points back to their original distributions . Following the approach of Bialkowski et al . ( 2016 ) , the “ structure ” imposed by the triplet sampling is enforced through the hard-assignment at each iteration . Recall , however , that the original distributions were statistically independent ; the triplet structure we wish to respect is imposed by the assignment step , not the underlying distributions . Contrastingly , in our method the samples are treated as fully independent ; had all samples been taken from the first distribution , followed by the second , followed by the third , the outcome at the “ distribution-discovery ” phase would be identical to that having sampled the data in rounds . Only after the three distributions are estimated would the assignment of each point in every triplet be assigned to the distribution which maximized the overall likelihood in that triplet . Besides being more computationally effective ( see Section 4.2 ) , this allows us to find the true MLE of the distributions . Our method will always discover a more optimal estimate of Eq . 2 . This can be understood in considering how the assignment is performed during optimization . For each triplet , the likelihood of assigning each point to each distribution is computed in both approaches . In our approach , this gives the likelihood under each mixture component . In the hard-assignment approach , however , if two ( or more ) points in a triplet have their highest likelihood under the same component , the exclusionary assignment must result in a lower likelihood than assigning each point to its preferred Gaussian . Furthermore , in our approach , each sample contributes to every component of the mixture , thus the data under the mixture remains “ fixed ” during the optimization process . In contrast , as the hardassignments are made , the samples contributing to each distribution changes each iteration . This , in combination with the sub-optimal likelihood above , effectively “ breaks ” the expectation maximization step and can cause solutions to diverge or oscillate , which is inconsistent with a maximum likelihood solution which must monotonically increase . Thus our approach is computationally efficient , more intuitively captures the independence of the generating distributions versus the structure of the sampling , and ensures a likelihood function that will converge under expectation maximization .

Summary:   The paper proposes a new algorithm for template discovery and representation learning for unstructured multi-agent data with strong group roles (i.e. sports data). Their algorithm is several orders of magnitude faster because of a key assumption in the underlying structure of the position generating distributions of players conditioned on roles. They also formulate the problem in a novel way, splitting the structuring of the data into a three-step approach with each component requiring a different algorithm. This will allow the community to swap in new algorithms into the different components more simply than before. They finally provide some evidence of their algorithm working on a soccer dataset consisting of 380 games.