Exploring Social Posterior Collapse in Variational Autoencoder for Interaction Modeling

1 Introduction . Modeling the behavior of intelligent agents is an essential subject for autonomous systems . Safe operations of autonomous agents require accurate prediction of other agents ’ future motions . In particular , social interaction among agents should be modeled , so that downstream planning and decision-making modules can safely navigate the robots through interactive scenarios . Generative latent variable models are popular modeling options for their ability to generate diverse and naturalistic behavior [ 1 , 2 , 3 , 4 ] . In this work , we focus on one category of generative models , Variational Autoencoder ( VAE ) [ 5 ] , which has been widely used in multi-agent behavior modeling and trajectory prediction [ 1 , 4 , 6 , 7 , 8 , 9 ] . It is desirable as it learns a low-dimensional representation of the original high-dimensional data . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) However , VAEs do not necessarily learn a good representation of the data [ 10 ] . For instance , prior works in sequence modeling have found out that the model tends to ignore the latent variables if the decoder is overly powerful [ 11 , 12 , 13 ] ( e.g. , an autoregressive decoder ) . It leads us to wonder whether a VAE-based model can always learn a good representation for a multi-agent interacting system . It is a rather general question as researchers may look for different properties of the latent space , for instance , interpretability [ 14 , 15 ] and multi-modality [ 4 , 6 ] . In this work , we focus on a fundamental aspect of this general question : Does the latent space always properly model interaction ? Formally , given a latent variable model of an interacting system , where a latent variable governs each agent ’ s behavior , we wonder if the VAE learns to encode social context into the latent variables . It raises such concern since VAE handles two distinct tasks in training and testing . At the training stage , it learns to reconstruct a datum instead of generating one . For multi-agent interaction modeling , the sample for reconstruction is a set of trajectories of all the agents . Ideally , the model should learn to model the interaction among agents and jointly reconstruct the trajectories . However , there is no mechanism to prevent the model from separately reconstructing the trajectories . In fact , it could even be more efficient at the early stage of training when the model has not learned an informative embedding of social context . We then end up with a model that models each agent ’ s behavior separately without social context , which might suffer from over-estimated variance and large prediction error . More importantly , since the joint behavior of the agents is a consequence of their interactions , ignoring the causes may lead to poor generalization ability [ 15 , 16 ] . We find that a typical formulation of VAE for multi-agent interaction is indeed prone to ignoring historical social context ( i.e. , interactions over the observed time horizon ) . We refer to this phenomenon as social posterior collapse . The issue has never been discussed in the literature . Considering those potential defects , we think it is necessary to study such a crucial and fundamental issue . In this work , we first abstract the VAE formulation from a wide range of existing literature [ 1 , 4 , 6 , 8 ] . Then we analyze social posterior collapse under this formulation and propose several measures to alleviate the issue . Afterward , we study the issue under real-world settings with a realization of the abstract formulation we design . In particular , we propose a novel sparse graph attention message-passing ( sparse-GAMP ) layer and incorporate it into the model , which helps us detect and analyze social posterior collapse . Our experiments show that social posterior collapse occurs in real-world prediction tasks and that the proposed measures can effectively alleviate the issue . We also evaluate how social posterior collapse affects the model performance on these tasks . The results suggest that the model without social posterior collapse can attain better generalization performance if historical social context is informative for prediction . 2 Social Posterior Collapse in Variational Autoencoder Interaction Models . 2.1 A Latent Variable Model for Interaction Modeling . Given an interacting system with n agents , an interaction-aware trajectory prediction model takes all the agents ’ observed historical trajectories , denoted by { xi } ni=1 , as input and predicts the future trajectories of all the agents or a subset of enquired agents . We denote the collection of future trajectories by { yi } ni=1 . In this work , we focus on the latent variable model illustrated in Fig . 1 , which is abstracted from existing literature on VAEs for multi-agent behavior modeling [ 1 , 4 , 6 , 8 ] . We model interaction by introducing a set of variables { Ti } ni=1 , which aggregates each agent ’ s state and its observation of other agents . Formally , each Ti is modeled as a deterministic function of { xi } ni=1 , i.e. , Ti = fi ( { xi } ni=1 ) . Afterward , the agents make decisions based on the aggregated information over the predicted horizon . Latent variables { zi } ni=1 are introduced to model the inherent uncertainty in each agent ’ s behavior . It should be noticed that interaction over the predicted horizon is not modeled explicitly in this formulation . Although it can be achieved by exchanging information between agents recurrently ( e.g. , social pooling in [ 17 ] ) , it is a common practice to avoid explicit modeling of future interaction in consideration of computational cost and memory [ 3 , 8 , 18 ] . In this work , we train the model as a VAE , where an encoder q ( z|x , y ) is introduced to approximate the posterior distribution p ( z|x , y ) for efficient sampling at the training stage1 . The performance of variational inference is optimized if the KL-divergence between the posteriors , i.e . DKL [ q ( z|x , y ) ‖p ( z|x , y ) ] , is small [ 10 ] . To derive a better approximation , we can incorporate 1The vectors x , y and z collect the corresponding variables for all n agents . inductive bias based on the characteristics of the true posterior into the encoder function . We introduce the following simple proposition that guides our model design . Its proof can be found in Appx . A . Proposition 1 . For any i = 1 , 2 , ... , n and j = 1 , 2 , ... , n , 1 ) If j 6= i , zi and yj are conditionally independent given x and yi ; 2 ) If j 6= i , zi and zj are conditionally independent given x ; 3 ) zi and xj are conditionally independent given Ti . Following the proposition , we decompose the posterior distribution as ∏n i=1 p ( zi|Ti , yi ) . The decomposition suggests two insights . First , the encoder does not need to aggregate future context information when inferring the posterior distribution of zi for each agent . Second , the historical context variable Ti is all we need for the historical information of the agent i . The encoder and decoder can share the same function to encode historical information . 2.2 Social Posterior Collapse . We then design a VAE model reflecting the characteristics of the true posterior discovered in Sec . 2.1 . To simplify the problem , we further make the assumption of homogeneous agents . The model has three basic building blocks : 1 ) A function modeling the historical context , i.e. , Ti = fθ ( xi , x ) ; 2 ) A function decoding the distribution of the future trajectory yi given Ti and zi , i.e. , pφ ( yi|Ti , zi ) ; 3 ) A function approximating the posterior of zi conditioned on Ti and yi , i.e. , qψ ( zi|Ti , yi ) . They build up the encoder and decoder of the VAE model as follows : qθ , ψ ( z|x , y ) = n∏ i=1 qψ ( zi|fθ ( xi , x ) , yi ) , pθ , φ ( y|x , z ) = n∏ i=1 pφ ( yi|fθ ( xi , x ) , zi ) . ( 1 ) We consider a continuous latent space and model qψ and pφ as diagonal Gaussian distributions . The model is trained by maximizing the evidence lower bound ( ELBO ) : max θ , ψ , φ Ex , y∼D [ Ez∼qθ , ψ ( z|x , y ) [ log pθ , φ ( y|x , z ) ] − βDKL [ qθ , ψ ( z|x , y ) ‖p ( z ) ] ] . ( 2 ) However , we find a critical issue of this naive formulation during experiments , which is the social posterior collapse phenomenon mentioned before . With the help of the sparse graph attention mechanism introduced in Sec . 3 , we find that the model is prone to ignoring historical social context when reconstructing the future trajectory of one agent . Equivalently , the generative model collapses to the one with all the variables of other agents marginalized out . We think the reason behind it is similar to , but different from , the well-known phenomenon in VAE training—posterior collapse [ 10 ] . Due to the KL regularization term in ELBO , the variational posterior distribution is likely to collapse towards the prior . It is particularly likely to occur at the early stage of training when the latent space is still uninformative [ 19 ] . In our case , the posterior of zi collapses into qθ , ψ ( zi|xi , yi ) . It does minimize the KL-divergence : if both qθ , ψ ( zi|xi , yi ) and qθ , ψ ( zi|x , yi ) exactly approximate the true posteriors , then conditioning on more context information increases the expected value of KL regularization : ED [ DKL [ p ( zi|x , yi ) ‖p ( zi ) ] ] = I ( zi ; x , yi ) > I ( zi ; xi , yi ) = ED [ DKL [ p ( zi|xi , yi ) ‖p ( zi ) ] ] . However , we argue that an additional factor contributes to the social posterior collapse problem , which makes it a unique phenomenon for interaction modeling . At the training stage , the goal is to reconstruct the future trajectories y . The trajectory itself contains all the information needed for reconstruction . The history of the agent i provides complementary information such as the current state and static environmental information . There is no explicit regulation in the current framework to prevent the model from extracting information solely from xi and yi . In fact , it is a more efficient coding scheme when the model has not learned an informative representation of interaction . Techniques such as KL annealing [ 13 , 19 ] rely on various scheduling schemes of β to prevent KL vanishing , which has been shown effective in mitigating the posterior collapse problem . However , reducing β could merely privilege the model to gather more information from yi , which is consistent with the reconstruction objective . Therefore , we need to explore alternative solutions to tackle the social posterior collapse problem deliberately . We start with changing the model into a conditional generative one [ 20 , 21 ] . Additional edges { Ti → zi } ni=1 are added into the original graph , which are annotated with dash lines in Fig . 1 . We follow the practice in [ 20 ] and formulate the model as a Conditional Variational Autoencoder ( CVAE ) . It is straightforward to verify that the conclusion of Proposition 2.1 still applies . We still model the encoder and decoder as in Eqn . ( 1 ) . The CVAE framework introduces an additional module —a function approximating the conditional prior pη ( zi|Ti ) , which becomes pθ , η ( zi|x ) after incorporating fθ ( x ) . The objective then becomes : L ( θ , ψ , φ , η ) = Ex , y∼D [ Ez∼qθ , ψ ( z|x , y ) [ log pθ , φ ( y|x , z ) ] − βDKL [ qθ , ψ ( z|x , y ) ‖pθ , η ( z|x ) ] ] . Compared to Eqn . ( 2 ) , we no longer penalize the encoder for aggregating context information but only restrict the information encoded from future trajectories . Therefore , the encoder does not need to ignore context information to fulfill the information bottleneck . However , the model still lacks a mechanism to encourage context information encoding deliberately . To achieve the goal , we propose to incorporate an auxiliary prediction task into the training scheme . Concretely , we introduce another module yi = gζ ( Ti , zi ) . Composing gζ with fθ gives us another decoder y = hθ , ζ ( x , z ) . The difference is that the latent variables are always sampled from pη ( zi|Ti ) . The auxiliary task is training this trajectory decoder which shares the same context encoder and conditional prior with the CVAE . Because the auxiliary model does not have access to the ground-truth future trajectory , it needs to utilize context information for accurate prediction . Consequently , it encourages the model to encode context information into T and z . We use mean squared error ( MSE ) loss as the objective function . The overall objective minimizes a weighted sum of the two objective functions . The training scheme looks similar to the one in [ 20 ] , where they trained a Gaussian stochastic neural network ( GSNN ) together with the CVAE model . However , ours is different from theirs in some major aspects . The GSNN model shares the same decoder with the CVAE model . The primary motivation of incorporating another learning task is to optimize the generation procedure during training directly . In contrast , our auxiliary prediction model has a separate trajectory decoder . It is because we do not want the auxiliary task to interfere with the decoding procedure of the CVAE model . We find that sharing the decoder leads to less diversity in trajectory generation , which is not desirable .

Summary: This work discuss the problem of posterior collapse in VAE/CVAE problems and extend this when it is used in the concept of multi-agent  interaction modelling and trajectory prediction problem. They argues that the model designed naively based on VAE for this task is prone to ignore agent interaction when predicting the future trajectory of an agent. The framework proposes a novel sparse graph attention message-passing (sparse-GAMP) layer to address this problem.